From 0c94bcd6f901215336088ffe518bd241cfaddc61 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 7 Mar 2026 22:03:49 +0200 Subject: [PATCH 001/275] Add singing practice feature: dual-track analysis and realtime pitch feedback Introduces a "singing track" workflow alongside the existing reference-track analysis. The user can load a second audio file (File > Load Singing Track, Ctrl+Shift+R) or record from the microphone (Ctrl+Space) while a reference track is already open; both tracks are displayed in the same pane with visually distinct colours. New files --------- main/RealtimePitchTracker.h/.cpp Polls a WritableWaveFileModel on a 50 ms QTimer using a YIN pitch estimator (bqvec / bqfft). Emits pitchDetected(frame, hz) for each voiced frame. Written into a SparseTimeValueModel so the view auto-repaints. main/mingw_byte_fix.h Force-include workaround for the MinGW C++17 std::byte / rpcndr.h typedef clash. Pre-includes with WIN32_LEAN_AND_MEAN to suppress rpcndr.h, then restores NT types via + . main/Analyser.h/.cpp changes ----------------------------- - New ColorScheme enum (PrimaryColors / SecondaryColors). SecondaryColors selects orange for the pitch track and bright purple for notes, so the singing track is visually distinct from the reference track. - Analyser constructor takes an optional ColorScheme argument. - addVisualisations(): secondary analyser skips spectrogram creation and returns immediately; primary analyser's existing-layer check requires the spectrogram model to match its own file model. - addWaveform(): secondary analyser creates a plain Waveform layer and calls document->setModel() rather than createMainModelLayer() (which always uses the document main model). - addAnalyses(): layer-claiming checks use model-identity guards so the two analysers sharing pane 0 do not steal each other's pitch/note layers. main/MainWindow.h/.cpp changes ------------------------------- Dual-analyser state m_analyser2 secondary Analyser*, null until a singing track is loaded m_realtimePitchTracker RealtimePitchTracker* active only during recording m_realtimePitchLayer TimeValueLayer* for live pitch dots (orange) m_realtimePitchModelId SparseTimeValueModel backing the realtime layer m_pendingSingingModelId ModelId set by modelAdded() for deferred setup m_recordingInProgress bool, true between recordingStarted/recordingFinishedFull m_recordingAsSingingTrack bool, true when Ctrl+Space was pressed with a reference already loaded (RecordCreateAdditionalModel) New/overridden slots openSingingTrack() opens file picker, calls loadSingingTrack() loadSingingTrack(path) opens file as CreateAdditionalModel, removes the extra pane AddPaneCommand creates, waits for modelAdded() -> setupSingingTrackAnalyser() analyseNewSingingModel() deferred handler that calls setupSingingTrackAnalyser() setupSingingTrackAnalyser() creates m_analyser2 (SecondaryColors), calls newFileLoaded() in pane 0 alongside primary analyser teardownSingingTrackAnalyser() deletes m_analyser2 cleanly setupRealtimePitchLayer() creates SparseTimeValueModel + orange TimeValueLayer, starts RealtimePitchTracker polling the recording model teardownRealtimePitchLayer() stops tracker, removes and deletes the layer record() override: switches to RecordCreateAdditionalModel when a reference is loaded so the recording becomes the singing track; resets to RecordReplaceSession after recordingStarted() sets m_recordingInProgress, defers setupRealtimePitchLayer() via QTimer::singleShot(0) to avoid the pre-pane-creation timing window onRealtimePitchDetected() shows note name + cents deviation in the status bar recordingFinishedFull() called from analyseNow() on recording completion; removes the realtime pitch layer now that full pYIN analysis is available analyseNow() early branch routes to m_analyser2->analyseExistingFile() when m_recordingAsSingingTrack is set; falls back via 200 ms QTimer if m_analyser2 is not yet set up Signal/slot connections canShowRealtimePitch(bool) emitted from updateMenuStates(); enables/disables m_showSingingPitch and m_showSingingNotes toolbar actions modelAdded() detects additional WaveFileModel (singing file or recording) and defers analyseNewSingingModel() Toolbar ("Show and Play") "Reference:" and "Singing:" QLabel section headers distinguish the two groups m_showSingingPitch uses il.load("record") (microphone icon) rather than "values" m_showSingingNotes mirrors existing notes button for the secondary analyser closeSession() / newSession() teardownRealtimePitchLayer() and teardownSingingTrackAnalyser() are called at the top of closeSession() before the pane/document teardown loop, ensuring layers can be cleanly removed while m_document is still valid. Session restore analyseNewMainModel() scans document models after primary setup for any WaveFileModel that is not the main model; if found and m_analyser2 is null, calls setupSingingTrackAnalyser() via QTimer::singleShot(0) so that .ton sessions saved with a singing track are correctly restored on reload. meson.build changes ------------------- - Add RealtimePitchTracker.cpp to tony_sources. - Add MinGW/GCC win64 build path (alongside the existing MSVC win64 path): - Use msys2 system libraries; skip sv-dependency-builds/win64-msvc. - Omit HAVE_MEDIAFOUNDATION (requires shobjidl_core.h, MSVC-only). - Add explicit include dirs for opus, sord-0, serd-0 (versioned msys2 subdirs). - Fix vamp_symbol_args: use -Wl,--export-dynamic-symbol= for GCC instead of MSVC /EXPORT:. - Inject -include main/mingw_byte_fix.h via general_defines for non-MSVC. - Add tmp to .gitignore. --- .gitignore | 2 +- main/Analyser.cpp | 100 ++++- main/Analyser.h | 18 +- main/MainWindow.cpp | 691 +++++++++++++++++++++++++++++++++- main/MainWindow.h | 67 ++++ main/RealtimePitchTracker.cpp | 252 +++++++++++++ main/RealtimePitchTracker.h | 183 +++++++++ main/mingw_byte_fix.h | 101 +++++ meson.build | 124 ++++-- 9 files changed, 1484 insertions(+), 54 deletions(-) create mode 100644 main/RealtimePitchTracker.cpp create mode 100644 main/RealtimePitchTracker.h create mode 100644 main/mingw_byte_fix.h diff --git a/.gitignore b/.gitignore index bcc2de54..96440733 100644 --- a/.gitignore +++ b/.gitignore @@ -49,4 +49,4 @@ test-svcore-* *.dmg .notarization-uuid Dockerfile*.gen - +tmp diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 957bfe63..538746fb 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -43,7 +43,8 @@ using std::endl; using namespace sv; -Analyser::Analyser() : +Analyser::Analyser(ColorScheme colorScheme) : + m_colorScheme(colorScheme), m_document(0), m_paneStack(0), m_pane(0), @@ -250,6 +251,15 @@ Analyser::addVisualisations() { if (m_fileModel.isNone()) return "Internal error: Analyser::addVisualisations() called with no model present"; + // The secondary (singing/recording) analyser shares the primary pane. + // A spectrogram for the singing track would be visually confusing and + // would incorrectly steal the primary analyser's spectrogram layer. + // Simply skip the spectrogram for the secondary colour scheme. + if (m_colorScheme == SecondaryColors) { + m_layers[Spectrogram] = nullptr; + return ""; + } + // A spectrogram, off by default. Must go at the back because it's // opaque @@ -286,9 +296,17 @@ Analyser::addVisualisations() SpectrogramLayer *existing = qobject_cast (m_pane->getLayer(i)); if (existing) { - cerr << "recording existing spectrogram layer" << endl; - m_layers[Spectrogram] = existing; - return ""; + // Only claim this spectrogram if it belongs to the main model + // (i.e. it is a main-model layer derived from our file model). + // In dual-analyser mode the pane is shared, so we must not + // steal a spectrogram that was created by the other analyser. + ModelId existingModel = existing->getModel(); + if (existingModel == m_fileModel || + existingModel == m_document->getMainModel()) { + cerr << "recording existing spectrogram layer (matching main model)" << endl; + m_layers[Spectrogram] = existing; + return ""; + } } } @@ -315,19 +333,40 @@ Analyser::addWaveform() // little space at the bottom. // As with the spectrogram above, if one exists already we just - // use it + // use it -- but only if it's associated with our file model. + // In dual-track mode, the pane may already have a waveform layer + // belonging to the primary analyser; we must not steal it. for (int i = 0; i < m_pane->getLayerCount(); ++i) { WaveformLayer *existing = qobject_cast (m_pane->getLayer(i)); - if (existing) { - cerr << "recording existing waveform layer" << endl; + if (existing && existing->getModel() == m_fileModel) { + cerr << "recording existing waveform layer (matching our file model)" << endl; m_layers[Audio] = existing; return ""; } } - WaveformLayer *waveform = qobject_cast - (m_document->createMainModelLayer(LayerFactory::Waveform)); + WaveformLayer *waveform = nullptr; + + if (m_colorScheme == SecondaryColors) { + // The secondary analyser's file model is NOT the document main model, + // so createMainModelLayer would show the wrong (reference) audio. + // Instead create a layer directly and associate it with our model. + // The model must already have been registered with the document via + // addNonDerivedModel (done in MainWindow::setupSingingTrackAnalyser). + Layer *raw = m_document->createLayer(LayerFactory::Waveform); + waveform = qobject_cast(raw); + if (waveform) { + m_document->setModel(waveform, m_fileModel); + } + } else { + waveform = qobject_cast + (m_document->createMainModelLayer(LayerFactory::Waveform)); + } + + if (!waveform) { + return tr("Internal error: could not create waveform layer"); + } waveform->setMiddleLineHeight(0.9); waveform->setShowMeans(false); // too small & pale for this @@ -354,23 +393,46 @@ Analyser::addAnalyses() } // As with the spectrogram above, if these layers exist we use - // them + // them -- but only if their source model matches our file model. + // When two analysers share a pane (dual-track mode), each must + // claim only the layers it created, not those from the other + // analyser. TimeValueLayer *existingPitch = 0; FlexiNoteLayer *existingNotes = 0; for (int i = 0; i < m_pane->getLayerCount(); ++i) { if (!existingPitch) { - existingPitch = qobject_cast(m_pane->getLayer(i)); + TimeValueLayer *tvl = + qobject_cast(m_pane->getLayer(i)); + if (tvl) { + // Accept this layer only if its source model is our + // file model (or a model derived from our file model). + auto model = ModelById::get(tvl->getModel()); + if (model && (tvl->getModel() == m_fileModel || + model->getSourceModel() == m_fileModel)) { + existingPitch = tvl; + } + } } if (!existingNotes) { - existingNotes = qobject_cast(m_pane->getLayer(i)); + FlexiNoteLayer *fnl = + qobject_cast(m_pane->getLayer(i)); + if (fnl) { + auto model = ModelById::get(fnl->getModel()); + if (model && (fnl->getModel() == m_fileModel || + model->getSourceModel() == m_fileModel)) { + existingNotes = fnl; + } + } } } if (existingPitch && existingNotes) { - cerr << "recording existing pitch and notes layers" << endl; + cerr << "recording existing pitch and notes layers (matching our file model)" << endl; m_layers[PitchTrack] = existingPitch; m_layers[Notes] = existingNotes; return ""; } else { + // Remove any mismatched layers we may have found for our model + // (partial state from a previous failed analysis run). if (existingPitch) { m_document->removeLayerFromView(m_pane, existingPitch); m_layers[PitchTrack] = 0; @@ -493,11 +555,19 @@ Analyser::addAnalyses() } ColourDatabase *cdb = ColourDatabase::getInstance(); + + // Choose colors based on color scheme: + // Primary (reference/target track): Black pitch, Bright Blue notes + // Secondary (singer/recording track): Orange pitch, Bright Purple notes + QString pitchColour = (m_colorScheme == SecondaryColors) + ? tr("Orange") : tr("Black"); + QString notesColour = (m_colorScheme == SecondaryColors) + ? tr("Bright Purple") : tr("Bright Blue"); TimeValueLayer *pitchLayer = qobject_cast(m_layers[PitchTrack]); if (pitchLayer) { - pitchLayer->setBaseColour(cdb->getColourIndex(tr("Black"))); + pitchLayer->setBaseColour(cdb->getColourIndex(pitchColour)); auto params = pitchLayer->getPlayParameters(); if (params) { params->setPlayPan(1); @@ -510,7 +580,7 @@ Analyser::addAnalyses() FlexiNoteLayer *flexiNoteLayer = qobject_cast(m_layers[Notes]); if (flexiNoteLayer) { - flexiNoteLayer->setBaseColour(cdb->getColourIndex(tr("Bright Blue"))); + flexiNoteLayer->setBaseColour(cdb->getColourIndex(notesColour)); auto params = flexiNoteLayer->getPlayParameters(); if (params) { params->setPlayPan(1); diff --git a/main/Analyser.h b/main/Analyser.h index 918d239c..2f2c6489 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -42,7 +42,17 @@ class Analyser : public QObject, Q_OBJECT public: - Analyser(); + /** + * Color scheme for the pitch track and notes layers. + * Primary is used for the reference/target track (black pitch, blue notes). + * Secondary is used for the singer/recording track (orange pitch, purple notes). + */ + enum ColorScheme { + PrimaryColors, // Black pitch track, Bright Blue notes + SecondaryColors, // Orange pitch track, Bright Purple notes + }; + + Analyser(ColorScheme colorScheme = PrimaryColors); virtual ~Analyser(); // Process new main model, add derived layers; return "" on @@ -219,6 +229,10 @@ class Analyser : public QObject, */ void takePitchTrackFrom(sv::Layer *layer); + ColorScheme getColorScheme() const { + return m_colorScheme; + } + sv::Pane *getPane() { return m_pane; } @@ -238,6 +252,8 @@ protected slots: void materialiseReAnalysis(); protected: + ColorScheme m_colorScheme; + sv::Document *m_document; sv::ModelId m_fileModel; sv::PaneStack *m_paneStack; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index cd9fa4af..0f24f87d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -25,6 +25,8 @@ #include "view/Pane.h" #include "view/PaneStack.h" #include "data/model/WaveFileModel.h" +#include "data/model/WritableWaveFileModel.h" +#include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" #include "layer/FlexiNoteLayer.h" #include "view/ViewManager.h" @@ -47,7 +49,9 @@ #include "base/Profiler.h" #include "base/UnitDatabase.h" #include "layer/ColourDatabase.h" +#include "layer/LayerFactory.h" #include "base/Selection.h" +#include "data/model/Model.h" #include "rdf/RDFImporter.h" #include "data/fileio/DataFileReaderFactory.h" @@ -89,9 +93,11 @@ #include #include #include +#include #include #include +#include #include using std::vector; @@ -108,7 +114,13 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindowBase::MIDI_NONE, int(PaneStack::Option::NoPropertyStacks) | int(PaneStack::Option::NoPaneAccessories)), + m_analyser2(nullptr), + m_realtimePitchTracker(nullptr), + m_realtimePitchLayer(nullptr), m_overview(0), + m_showSingingPitch(nullptr), + m_showSingingNotes(nullptr), + m_loadSingingTrackAction(nullptr), m_mainMenusCreated(false), m_playbackMenu(0), m_recentFilesMenu(0), @@ -122,7 +134,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_keyReference(new KeyReference()), m_selectionAnchor(0), m_withSonification(withSonification), - m_withSpectrogram(withSpectrogram) + m_withSpectrogram(withSpectrogram), + m_recordingInProgress(false), + m_recordingAsSingingTrack(false) { setWindowTitle(QApplication::applicationName()); @@ -308,6 +322,12 @@ MainWindow::MainWindow(AudioMode audioMode, connect(this, SIGNAL(replacedDocument()), this, SLOT(documentReplaced())); connect(this, SIGNAL(sessionLoaded()), this, SLOT(analyseNewMainModel())); connect(this, SIGNAL(audioFileLoaded()), this, SLOT(analyseNewMainModel())); + + // Connect record target signals for real-time pitch tracking + if (m_recordTarget) { + connect(m_recordTarget, SIGNAL(recordStatusChanged(bool)), + this, SLOT(recordingStarted())); + } m_activityLog->hide(); setAudioRecordMode(RecordReplaceSession); @@ -333,6 +353,26 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindow::~MainWindow() { + // Clean up secondary state that may not have been torn down if the + // window was closed without going through closeSession() (e.g. on + // application exit via the window close button). + if (m_realtimePitchTracker) { + m_realtimePitchTracker->stop(); + delete m_realtimePitchTracker; + m_realtimePitchTracker = nullptr; + } + // m_realtimePitchLayer is a Layer* owned by the Document (registered via + // createEmptyLayer / deleteLayer). The Document is owned by MainWindowBase + // and will be destroyed after this destructor returns, so we only null + // the pointer here — the Document will release the layer. + m_realtimePitchLayer = nullptr; + + // m_analyser2 is a plain heap-allocated QObject (no Qt parent, not owned + // by the Document). We must delete it explicitly. + if (m_analyser2) { + delete m_analyser2; + m_analyser2 = nullptr; + } delete m_analyser; delete m_keyReference; Profiles::getInstance()->dump(); @@ -436,6 +476,16 @@ MainWindow::setupFileMenu() menu->addSeparator(); + m_loadSingingTrackAction = new QAction(il.load("fileopen"), tr("Load &Singing Track..."), this); + m_loadSingingTrackAction->setShortcut(tr("Ctrl+Shift+R")); + m_loadSingingTrackAction->setStatusTip(tr("Load a second audio file to analyse as the singing track, overlaid on the reference track")); + connect(m_loadSingingTrackAction, SIGNAL(triggered()), this, SLOT(openSingingTrack())); + connect(this, SIGNAL(canPlay(bool)), m_loadSingingTrackAction, SLOT(setEnabled(bool))); + m_keyReference->registerShortcut(m_loadSingingTrackAction); + menu->addAction(m_loadSingingTrackAction); + + menu->addSeparator(); + action = new QAction(tr("I&mport Pitch Track Data..."), this); action->setStatusTip(tr("Import pitch-track data from a CSV, RDF, or layer XML file")); connect(action, SIGNAL(triggered()), this, SLOT(importPitchLayer())); @@ -1074,7 +1124,7 @@ MainWindow::setupToolbars() tr("Record")); recordAction->setCheckable(true); recordAction->setShortcut(tr("Ctrl+Space")); - recordAction->setStatusTip(tr("Record a new audio file")); + recordAction->setStatusTip(tr("Record a new audio file. If a reference track is already loaded, the recording is added as the singing track alongside it.")); connect(recordAction, SIGNAL(triggered()), this, SLOT(record())); connect(m_recordTarget, SIGNAL(recordStatusChanged(bool)), recordAction, SLOT(setChecked(bool))); @@ -1216,6 +1266,16 @@ MainWindow::setupToolbars() toolbar = addToolBar(tr("Show and Play")); addToolBar(Qt::BottomToolBarArea, toolbar); + // "Reference:" label before the reference-track button group + { + QLabel *refLabel = new QLabel(tr(" Reference:")); + QFont f = refLabel->font(); + f.setPointSize(f.pointSize() - 1); + refLabel->setFont(f); + refLabel->setEnabled(false); // greyed out — purely decorative + toolbar->addWidget(refLabel); + } + m_showAudio = toolbar->addAction(il.load("waveform"), tr("Show Audio")); m_showAudio->setCheckable(true); connect(m_showAudio, SIGNAL(triggered()), this, SLOT(showAudioToggled())); @@ -1285,6 +1345,52 @@ MainWindow::setupToolbars() m_playNotes = 0; } + // Singing track pitch track — preceded by a "Singing:" section label + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); + toolbar->addWidget(spacer); + + { + QLabel *singLabel = new QLabel(tr("Singing:")); + QFont f = singLabel->font(); + f.setPointSize(f.pointSize() - 1); + singLabel->setFont(f); + singLabel->setEnabled(false); // greyed out — purely decorative + toolbar->addWidget(singLabel); + } + + // Use the "record" (microphone) icon to visually distinguish this from + // the reference pitch button which uses "values". + m_showSingingPitch = toolbar->addAction(il.load("record"), tr("Show Singing Pitch Track")); + m_showSingingPitch->setCheckable(true); + m_showSingingPitch->setToolTip(tr("Show/hide the singing track pitch (orange)")); + connect(m_showSingingPitch, &QAction::triggered, this, [this](bool checked) { + if (m_analyser2) { + m_analyser2->setVisible(Analyser::PitchTrack, checked); + } + if (m_realtimePitchLayer) { + m_realtimePitchLayer->showLayer(m_paneStack->getPane(0), checked); + } + }); + connect(this, SIGNAL(canShowRealtimePitch(bool)), m_showSingingPitch, SLOT(setEnabled(bool))); + m_showSingingPitch->setEnabled(false); + + // Singing track notes + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(10)); + toolbar->addWidget(spacer); + + m_showSingingNotes = toolbar->addAction(il.load("notes"), tr("Show Singing Notes")); + m_showSingingNotes->setCheckable(true); + m_showSingingNotes->setToolTip(tr("Show/hide the singing track notes (purple)")); + connect(m_showSingingNotes, &QAction::triggered, this, [this](bool checked) { + if (m_analyser2) { + m_analyser2->setVisible(Analyser::Notes, checked); + } + }); + connect(this, SIGNAL(canShowRealtimePitch(bool)), m_showSingingNotes, SLOT(setEnabled(bool))); + m_showSingingNotes->setEnabled(false); + // Spectrogram spacer = new QLabel; spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); @@ -1479,6 +1585,11 @@ MainWindow::updateMenuStates() emit canPlayPitch(havePitchTrack && haveMainModel && havePlayTarget); emit canPlayNotes(haveNotes && haveMainModel && havePlayTarget); + // Enable singing-track toolbar buttons whenever a singing track analyser + // or a realtime (recording) pitch layer is active. + bool haveSinging = (m_analyser2 != nullptr) || (m_realtimePitchLayer != nullptr); + emit canShowRealtimePitch(haveSinging); + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -1625,6 +1736,31 @@ MainWindow::updateLayerStatuses() m_notesLPW->setPan(m_analyser->getPan(Analyser::Notes)); m_showSpect->setChecked(m_analyser->isVisible(Analyser::Spectrogram)); + + // Singing track controls: enabled when either a second analyser + // (post-recording full analysis) or the realtime pitch layer is active + bool haveSingingTrack = (m_analyser2 != nullptr) || (m_realtimePitchLayer != nullptr); + + if (m_showSingingPitch) { + m_showSingingPitch->setEnabled(haveSingingTrack); + if (m_analyser2) { + m_showSingingPitch->setChecked(m_analyser2->isVisible(Analyser::PitchTrack)); + } else if (m_realtimePitchLayer && m_paneStack && m_paneStack->getPaneCount() > 0) { + m_showSingingPitch->setChecked( + !m_realtimePitchLayer->isLayerDormant(m_paneStack->getPane(0))); + } else { + m_showSingingPitch->setChecked(false); + } + } + + if (m_showSingingNotes) { + m_showSingingNotes->setEnabled(m_analyser2 != nullptr); + if (m_analyser2) { + m_showSingingNotes->setChecked(m_analyser2->isVisible(Analyser::Notes)); + } else { + m_showSingingNotes->setChecked(false); + } + } } void @@ -1720,6 +1856,13 @@ MainWindow::closeSession() { if (!checkSaveModified()) return; + // Tear down singing track and realtime pitch layer before panes/document + // are destroyed, so they can cleanly remove their layers from the pane. + teardownRealtimePitchLayer(); + teardownSingingTrackAnalyser(); + m_pendingSingingModelId = {}; + m_recordingAsSingingTrack = false; + m_analyser->fileClosed(); while (m_paneStack->getPaneCount() > 0) { @@ -1783,6 +1926,419 @@ MainWindow::openFile() } } +void +MainWindow::openSingingTrack() +{ + QString path = getOpenFileName(FileFinder::AudioFile); + if (path.isEmpty()) return; + + loadSingingTrack(path); +} + +void +MainWindow::loadSingingTrack(QString path) +{ + // Load a second audio file as the "singing" track. + // This opens the file as an additional model alongside the main + // (reference) model, then runs pYIN on it in the secondary colour scheme. + + if (!m_document) { + QMessageBox::warning(this, tr("No session"), + tr("No session open

Please open a reference audio file first.")); + return; + } + + emit activity(tr("Load singing track \"%1\"").arg(path)); + + // Record the pane count before opening so we can remove any extra panes + // that openPath(CreateAdditionalModel) creates via AddPaneCommand. + // We want both tracks to share pane 0, not appear in separate panes. + int paneCountBefore = m_paneStack ? m_paneStack->getPaneCount() : 0; + + FileOpenStatus status = openPath(path, CreateAdditionalModel); + + if (status == FileOpenFailed) { + QMessageBox::critical(this, tr("Failed to open singing track"), + tr("File open failed

File \"%1\" could not be opened").arg(path)); + return; + } else if (status == FileOpenWrongMode) { + QMessageBox::critical(this, tr("Failed to open singing track"), + tr("Audio required

Could not open \"%1\" as audio").arg(path)); + return; + } + + // openPath(CreateAdditionalModel) will have called AddPaneCommand which + // added a new pane for the singing track's waveform layer. We do NOT + // want that extra pane — both tracks must overlay pane 0. Remove any + // panes above the original count (except the time-ruler pane at index 1 + // which was already there). We delete from the top down so that index + // arithmetic stays valid. + if (m_paneStack) { + while (m_paneStack->getPaneCount() > paneCountBefore) { + Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); + // Delete any layers that ended up in this extra pane. + // setupSingingTrackAnalyser will create proper layers in pane 0, + // so these auto-created layers (e.g. the waveform from + // addOpenedAudioModel) are not needed and would otherwise + // become orphans in the document layer list. + if (m_document && extra) { + while (extra->getLayerCount() > 0) { + Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); + // Use deleteLayer with force=true: this removes the layer + // from the view directly (without creating an undo command) + // and then deletes it from the document. We must NOT call + // removeLayerFromView first, as that would push a + // RemoveLayerCommand onto the undo stack holding a pointer + // to a layer we are about to delete — a guaranteed crash on + // undo. + m_document->deleteLayer(orphan, true); + } + } + if (m_overview) m_overview->unregisterView(extra); + m_paneStack->deletePane(extra); + } + } + + // analyseNewSingingModel() is triggered on the next event-loop tick via + // the QTimer::singleShot(0, ...) in modelAdded(). It picks up the model + // id from m_pendingSingingModelId, which was set in modelAdded() when the + // audio file was registered with the document. +} + +void +MainWindow::analyseNewSingingModel() +{ + // Called when a new audio model has been added (by loadSingingTrack) + // and we want to run the secondary (singing-track) analysis on it. + if (m_pendingSingingModelId.isNone()) return; + + ModelId singingModelId = m_pendingSingingModelId; + m_pendingSingingModelId = {}; + + setupSingingTrackAnalyser(singingModelId); +} + +void +MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId) +{ + if (!m_document) return; + if (!m_paneStack || m_paneStack->getPaneCount() < 1) return; + + // Reuse the main pane (pane 0) so both pitch tracks overlay each other + Pane *pane = m_paneStack->getPane(0); + if (!pane) return; + + // Tear down any previous secondary analyser + teardownSingingTrackAnalyser(); + + // Create the secondary analyser with the singing-track colour scheme + m_analyser2 = new Analyser(Analyser::SecondaryColors); + + connect(m_analyser2, SIGNAL(layersChanged()), + this, SLOT(updateLayerStatuses())); + connect(m_analyser2, SIGNAL(layersChanged()), + this, SLOT(updateMenuStates())); + + QString error = m_analyser2->newFileLoaded( + m_document, singingModelId, m_paneStack, pane); + + if (error != "") { + QMessageBox::warning(this, tr("Failed to analyse singing track"), + tr("Analysis failed

%1

").arg(error)); + delete m_analyser2; + m_analyser2 = nullptr; + return; + } + + // Re-stack layers so the primary pitch track stays on top + m_analyser->getLayer(Analyser::PitchTrack); // ensure primary is on top + updateLayerStatuses(); + updateMenuStates(); + + emit activity(tr("Singing track loaded and analysis started")); +} + +void +MainWindow::teardownSingingTrackAnalyser() +{ + if (!m_analyser2) return; + + m_analyser2->fileClosed(); + delete m_analyser2; + m_analyser2 = nullptr; +} + +void +MainWindow::setupRealtimePitchLayer() +{ + // Create a SparseTimeValueModel and TimeValueLayer to display a + // live pitch estimate during microphone recording. + // + // At the time this is called (triggered by recordStatusChanged(true)), + // MainWindowBase::record() has already: + // 1. Created a WritableWaveFileModel for the recording. + // 2. Set it as the document main model. + // 3. Emitted audioFileLoaded() -> analyseNewMainModel() which added panes. + // + // So getMainModelId() returns the WritableWaveFileModel being filled. + // RealtimePitchTracker polls that model via getData() on a QTimer. + + if (!m_document) return; + if (!m_paneStack || m_paneStack->getPaneCount() < 1) return; + + Pane *pane = m_paneStack->getPane(0); + if (!pane) return; + + // Remove any stale realtime layer from a previous recording session. + teardownRealtimePitchLayer(); + + // The audio source is the WritableWaveFileModel being recorded into. + // In RecordReplaceSession mode it is the document's main model. + // In RecordCreateAdditionalModel mode (recording as singing track) the + // main model is still the reference track, so we must search for the + // WritableWaveFileModel among all document models instead. + ModelId audioSourceId; + + if (m_recordingAsSingingTrack && m_document) { + // Scan document models for any WritableWaveFileModel (the recording). + ModelId mainId = getMainModelId(); + for (ModelId mid : m_document->getModels()) { + if (mid == mainId) continue; + if (ModelById::isa(mid)) { + audioSourceId = mid; + cerr << "setupRealtimePitchLayer: found singing-track recording model " + << mid << endl; + break; + } + } + if (audioSourceId.isNone()) { + // Fall back to main model in case the search failed. + audioSourceId = mainId; + cerr << "setupRealtimePitchLayer: could not find singing-track recording " + "model, falling back to main model" << endl; + } + } else { + audioSourceId = getMainModelId(); + } + + if (audioSourceId.isNone()) { + cerr << "setupRealtimePitchLayer: no audio source model found, cannot set up realtime tracker" << endl; + return; + } + + // Determine sample rate from the audio source model. + sv_samplerate_t sr = 44100; + if (auto audioModel = ModelById::getAs(audioSourceId)) { + sr = audioModel->getSampleRate(); + } else if (auto wfm = getMainModel()) { + sr = wfm->getSampleRate(); + } + + // Create a SparseTimeValueModel to receive pitch estimates. + // Resolution 512 frames matches the YIN hop size in RealtimePitchTracker. + auto pitchModel = std::make_shared(sr, 512, false); + pitchModel->setObjectName(tr("Realtime Pitch (Live)")); + m_realtimePitchModelId = ModelById::add(pitchModel); + m_document->addNonDerivedModel(m_realtimePitchModelId); + + // Create a TimeValueLayer to display the pitch estimates. + // createEmptyLayer creates the layer with an appropriate empty model + // registered with the document; we then use document->setModel() to + // replace that empty model with our SparseTimeValueModel. + Layer *rawLayer = m_document->createEmptyLayer(LayerFactory::TimeValues); + m_realtimePitchLayer = qobject_cast(rawLayer); + + if (!m_realtimePitchLayer) { + cerr << "setupRealtimePitchLayer: failed to create TimeValueLayer" << endl; + // Release the pitch model we just added; the Document will not hold + // a reference to it since we never called setModel yet. + ModelById::release(m_realtimePitchModelId); + m_realtimePitchModelId = {}; + return; + } + + // Associate our pre-filled SparseTimeValueModel with the layer. + // The model was already registered via addNonDerivedModel above. + m_document->setModel(m_realtimePitchLayer, m_realtimePitchModelId); + m_realtimePitchLayer->setVerticalScale(TimeValueLayer::AutoAlignScale); + m_realtimePitchLayer->setPlotStyle(TimeValueLayer::PlotPoints); + + // Singing/recording track uses the "Orange" colour so it is visually + // distinct from the reference track (black) and notes (blue). + ColourDatabase *cdb = ColourDatabase::getInstance(); + m_realtimePitchLayer->setBaseColour(cdb->getColourIndex(tr("Orange"))); + + m_document->addLayerToView(pane, m_realtimePitchLayer); + + // Create and start the pitch tracker. + // It will poll audioSourceId (the WritableWaveFileModel) for new frames + // on each QTimer tick and write estimates into m_realtimePitchModelId. + m_realtimePitchTracker = new RealtimePitchTracker( + audioSourceId, m_realtimePitchModelId, this); + connect(m_realtimePitchTracker, &RealtimePitchTracker::pitchDetected, + this, &MainWindow::onRealtimePitchDetected); + m_realtimePitchTracker->start(); + + cerr << "setupRealtimePitchLayer: realtime pitch tracking started " + << "(audio source model " << audioSourceId << ", sr=" << sr << ")" << endl; +} + +void +MainWindow::teardownRealtimePitchLayer() +{ + if (m_realtimePitchTracker) { + m_realtimePitchTracker->stop(); + delete m_realtimePitchTracker; + m_realtimePitchTracker = nullptr; + } + + if (m_realtimePitchLayer) { + if (m_paneStack && m_paneStack->getPaneCount() > 0) { + Pane *pane = m_paneStack->getPane(0); + if (pane) { + m_document->removeLayerFromView(pane, m_realtimePitchLayer); + } + } + m_document->deleteLayer(m_realtimePitchLayer, false); + m_realtimePitchLayer = nullptr; + } + + // Note: we do NOT call ModelById::release(m_realtimePitchModelId) here. + // The Document owns the SparseTimeValueModel we added via addNonDerivedModel, + // and deleteLayer() above will have already released the model if no other + // layer is referencing it. Calling release() a second time would be a + // double-free. + m_realtimePitchModelId = {}; +} + +void +MainWindow::record() +{ + // If a reference track is already loaded, record the microphone input as + // the singing track rather than replacing the whole session. + // We do this by temporarily switching to RecordCreateAdditionalModel so + // that MainWindowBase::record() adds the WritableWaveFileModel as an + // additional (non-main) model. Our modelAdded() hook will then pick it + // up and route it through setupSingingTrackAnalyser(). + // + // If there is no main model yet (first-time record), fall through with the + // default RecordReplaceSession behaviour. + + bool haveReference = (getMainModel() != nullptr); + + if (haveReference) { + cerr << "MainWindow::record: reference track loaded — recording as singing track" << endl; + m_recordingAsSingingTrack = true; + setAudioRecordMode(RecordCreateAdditionalModel); + } else { + m_recordingAsSingingTrack = false; + setAudioRecordMode(RecordReplaceSession); + } + + MainWindowBase::record(); + + // Restore the default mode so that a subsequent "standalone" recording + // (after the singing track session is closed) behaves correctly. + setAudioRecordMode(RecordReplaceSession); +} + +void +MainWindow::recordingStarted() +{ + // recordStatusChanged(bool) is emitted both when recording starts + // (true) and stops (false). We only want to act when it starts. + if (!m_recordTarget) return; + if (!m_recordTarget->isRecording()) { + // Recording stopped - nothing to do here; recordingFinishedFull() + // is called from analyseNow() once pYIN completes. + return; + } + + cerr << "MainWindow::recordingStarted: scheduling realtime pitch layer setup" << endl; + m_recordingInProgress = true; + + // TIMING: recordStatusChanged(true) is emitted from within + // AudioCallbackRecordTarget::startRecording(), which is called by + // MainWindowBase::record() BEFORE setMainModel() and BEFORE + // emit audioFileLoaded() -> analyseNewMainModel() creates the panes. + // If we call setupRealtimePitchLayer() directly here, m_paneStack will + // have zero panes and the setup will silently bail out. + // + // Fix: defer via QTimer::singleShot(0) so the slot runs on the next + // event-loop iteration, by which time record() has finished completely + // (including emit audioFileLoaded() -> panes created). + QTimer::singleShot(0, this, [this]() { + if (!m_recordingInProgress) { + // Recording was stopped before we got a chance to set up — + // nothing to do. + return; + } + cerr << "MainWindow::recordingStarted (deferred): setting up realtime pitch layer" << endl; + setupRealtimePitchLayer(); + updateLayerStatuses(); + updateMenuStates(); + }); +} + +void +MainWindow::onRealtimePitchDetected(sv::sv_frame_t /*frame*/, double hz) +{ + // The pitch has already been written into the SparseTimeValueModel by + // RealtimePitchTracker; the view repaints automatically via dataChanged(). + // Here we show a human-readable pitch in the status bar so the singer + // gets immediate textual feedback during recording. + + if (hz <= 0.0) { + getStatusLabel()->setText(tr("Recording — pitch: (unvoiced)")); + return; + } + + // Convert Hz to MIDI note number and cents deviation. + // MIDI note 69 = A4 = 440 Hz. + double midiNote = 12.0 * std::log2(hz / 440.0) + 69.0; + int nearestNote = int(std::round(midiNote)); + int cents = int(std::round((midiNote - nearestNote) * 100.0)); + + // Note names (no flats — sharps only for display simplicity). + static const char *noteNames[] = { + "C", "C#", "D", "D#", "E", "F", + "F#", "G", "G#", "A", "A#", "B" + }; + int noteIndex = ((nearestNote % 12) + 12) % 12; + int octave = (nearestNote / 12) - 1; + QString noteName = QString("%1%2").arg(noteNames[noteIndex]).arg(octave); + + QString centsStr; + if (cents == 0) { + centsStr = tr("in tune"); + } else if (cents > 0) { + centsStr = tr("+%1 cents").arg(cents); + } else { + centsStr = tr("%1 cents").arg(cents); + } + + getStatusLabel()->setText( + tr("Recording — singing: %1 (%2 Hz, %3)") + .arg(noteName) + .arg(hz, 0, 'f', 1) + .arg(centsStr)); +} + +void +MainWindow::recordingFinishedFull() +{ + // Called after analyseNow() has completed for the newly recorded audio. + // At this point the full pYIN analysis of the recording is available. + // We can remove the coarse realtime pitch layer (the "live" orange dots) + // because the full pYIN pitch track now covers the same audio. + cerr << "MainWindow::recordingFinishedFull: pYIN done, removing realtime pitch layer" << endl; + m_recordingInProgress = false; + m_recordingAsSingingTrack = false; + teardownRealtimePitchLayer(); + updateLayerStatuses(); + updateMenuStates(); +} + void MainWindow::openLocation() { @@ -2997,6 +3553,25 @@ MainWindow::modelAdded(ModelId model) auto dtvm = ModelById::getAs(model); if (dtvm) { cerr << "A dense time-value model (such as an audio file) has been loaded" << endl; + // If there is already a main model and this is a new additional + // audio model (not the realtime pitch model), treat it as the + // singing track to be analysed with the secondary colour scheme. + ModelId mainId = getMainModelId(); + if (!mainId.isNone() && model != mainId && + m_realtimePitchModelId != model) { + // Guard against race: if modelAdded() fires twice quickly (e.g. + // for an audio model and its alignment model), only set the + // pending id once; analyseNewSingingModel() will clear it when + // it runs. + if (m_pendingSingingModelId.isNone()) { + m_pendingSingingModelId = model; + // Defer so the model is fully registered before we analyse + QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); + } else { + cerr << "modelAdded: m_pendingSingingModelId already set, ignoring model " + << model << endl; + } + } } } @@ -3027,6 +3602,61 @@ void MainWindow::analyseNow() { cerr << "analyseNow called" << endl; + + // When the user recorded a singing track alongside an existing reference + // track (RecordCreateAdditionalModel mode), the recording becomes an + // additional model, not the main model. In that case we must route + // analysis through m_analyser2 (which was set up by setupSingingTrackAnalyser + // via modelAdded() when the WritableWaveFileModel was registered). + // We must NOT re-analyse the primary reference track here. + if (m_recordingAsSingingTrack) { + cerr << "analyseNow: recording was singing track — routing to m_analyser2" << endl; + if (m_analyser2) { + CommandHistory::getInstance()->startCompoundOperation + (tr("Analyse Singing Track"), true); + + QString error = m_analyser2->analyseExistingFile(); + + CommandHistory::getInstance()->endCompoundOperation(); + + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to analyse singing track"), + tr("Analysis failed

%1

").arg(error), + QMessageBox::Ok); + } + } else { + // m_analyser2 may still be pending setup (modelAdded fires async). + // Defer the analysis until the secondary analyser is ready. + cerr << "analyseNow: m_analyser2 not ready yet, deferring singing-track analysis" << endl; + QTimer::singleShot(200, this, [this]() { + if (m_analyser2) { + CommandHistory::getInstance()->startCompoundOperation + (tr("Analyse Singing Track"), true); + QString error = m_analyser2->analyseExistingFile(); + CommandHistory::getInstance()->endCompoundOperation(); + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to analyse singing track"), + tr("Analysis failed

%1

").arg(error), + QMessageBox::Ok); + } + } else { + cerr << "analyseNow (deferred): m_analyser2 still null, singing-track analysis skipped" << endl; + } + }); + } + + // Clean up the realtime pitch layer; the full pYIN analysis of the + // singing recording (via m_analyser2) now covers the same audio. + if (m_realtimePitchTracker || m_realtimePitchLayer) { + recordingFinishedFull(); + } + return; + } + if (!m_analyser) return; CommandHistory::getInstance()->startCompoundOperation @@ -3043,6 +3673,12 @@ MainWindow::analyseNow() tr("Analysis failed

%1

").arg(error), QMessageBox::Ok); } + + // If this analyseNow was triggered by recording completion, clean up + // the realtime pitch layer now that the full pYIN analysis is available. + if (m_realtimePitchTracker || m_realtimePitchLayer) { + recordingFinishedFull(); + } } void @@ -3064,6 +3700,27 @@ MainWindow::analyseNewMainModel() return; } + // When recording as a singing track (RecordCreateAdditionalModel mode), + // MainWindowBase::record() creates an extra pane for the recording waveform. + // We want both tracks in pane 0, so remove any panes beyond the expected 2 + // (main pane + time-ruler strip) before doing anything else. + // This mirrors the pane-cleanup logic in loadSingingTrack(). + if (m_recordingAsSingingTrack) { + int expectedPanes = 2; // pane 0 (main) + pane 1 (time ruler strip) + while (m_paneStack->getPaneCount() > expectedPanes) { + Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); + if (!extra) break; + // Remove all layers from the extra pane (force=true avoids undo commands + // that would hold dangling pointers to the layer). + while (extra->getLayerCount() > 0) { + Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); + m_document->deleteLayer(orphan, true); + } + m_paneStack->deletePane(extra); + } + cerr << "analyseNewMainModel: pruned extra recording pane (recording-as-singing-track mode)" << endl; + } + int pc = m_paneStack->getPaneCount(); Pane *pane = 0; Pane *selectionStrip = 0; @@ -3117,7 +3774,35 @@ MainWindow::analyseNewMainModel() m_analyser->setAudible(Analyser::PitchTrack, false); m_analyser->setAudible(Analyser::Notes, false); } - + + // Session restore: if the loaded session contained a second audio model + // (i.e. a previously loaded singing track), set up the secondary analyser + // for it now. We scan all document models for a WaveFileModel that is + // not the main model and not already being tracked as a singing model. + // We only do this if we don't already have a secondary analyser (it may + // have been set up already e.g. via modelAdded() during session load). + if (!m_analyser2 && m_document) { + ModelId mainId = getMainModelId(); + ModelId foundSinging; + for (ModelId mid : m_document->getModels()) { + if (mid == mainId) continue; + if (mid == m_realtimePitchModelId) continue; + if (ModelById::isa(mid)) { + foundSinging = mid; + break; + } + } + if (!foundSinging.isNone()) { + cerr << "analyseNewMainModel: found existing singing track model " + << foundSinging << " in session, setting up secondary analyser" << endl; + // Defer so that the primary analyser's layers are fully in place + // before the secondary analyser tries to share the same pane. + QTimer::singleShot(0, this, [this, foundSinging]() { + setupSingingTrackAnalyser(foundSinging); + }); + } + } + updateLayerStatuses(); documentRestored(); } diff --git a/main/MainWindow.h b/main/MainWindow.h index f14d5ca9..2ed6fa51 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -18,11 +18,15 @@ #include "framework/MainWindowBase.h" #include "Analyser.h" +#include "RealtimePitchTracker.h" + +#include "data/model/SparseTimeValueModel.h" namespace sv { class VersionTester; class ActivityLog; class LevelPanToolButton; +class TimeValueLayer; } class MainWindow : public sv::MainWindowBase @@ -35,6 +39,12 @@ class MainWindow : public sv::MainWindowBase bool withSpectrogram = true); virtual ~MainWindow(); + /** + * Load a second audio file as the "singing" track whose pitch + * will be analysed alongside the primary reference track. + */ + void loadSingingTrack(QString path); + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -42,12 +52,22 @@ class MainWindow : public sv::MainWindowBase void canPlayWaveform(bool); void canPlayPitch(bool); void canPlayNotes(bool); + void canLoadSingingTrack(bool); + void canShowRealtimePitch(bool); public slots: virtual bool commitData(bool mayAskUser); // on session shutdown +protected slots: + // Override record() so that when a reference track is already loaded we + // can switch to RecordCreateAdditionalModel before starting the capture, + // causing the recording to be treated as the singing track. + virtual void record(); + protected slots: virtual void openFile(); + virtual void openSingingTrack(); + virtual void analyseNewSingingModel(); virtual void openLocation(); virtual void openRecentFile(); virtual void saveSession(); @@ -172,6 +192,11 @@ protected slots: virtual void analyseNewMainModel(); + // --- Real-time pitch tracking during microphone recording --- + virtual void recordingStarted(); + virtual void onRealtimePitchDetected(sv::sv_frame_t frame, double hz); + virtual void recordingFinishedFull(); + void moveOneNoteRight(); void moveOneNoteLeft(); void selectOneNoteRight(); @@ -181,9 +206,29 @@ protected slots: void rewind(); protected: + // Primary analyser: the reference/target track loaded by the user. Analyser *m_analyser; + // Secondary analyser: the singing/recording track. + // Null until a second audio file is loaded or a recording is completed. + Analyser *m_analyser2; + + // Real-time pitch tracker: active only during microphone recording. + RealtimePitchTracker *m_realtimePitchTracker; + + // The transient layer shown during recording (replaced by the full + // pYIN analysis once recording is complete). + sv::TimeValueLayer *m_realtimePitchLayer; + + // Model backing the realtime layer (owned by the document). + sv::ModelId m_realtimePitchModelId; + sv::Overview *m_overview; + + // Actions/toolbar items for the singing track + QAction *m_showSingingPitch; + QAction *m_showSingingNotes; + QAction *m_loadSingingTrackAction; sv::Fader *m_fader; sv::AudioDial *m_playSpeed; QPushButton *m_playSharpen; @@ -246,6 +291,28 @@ protected slots: virtual void setupHelpMenu(); virtual void setupToolbars(); + // Helpers for the singing / second-track workflow + virtual void setupSingingTrackAnalyser(sv::ModelId singingModelId); + virtual void teardownSingingTrackAnalyser(); + virtual void setupRealtimePitchLayer(); + virtual void teardownRealtimePitchLayer(); + + // When loadSingingTrack opens an additional audio file, modelAdded() + // stores the resulting ModelId here so analyseNewSingingModel() can + // pick it up on the next event-loop iteration. + sv::ModelId m_pendingSingingModelId; + + // True while a microphone recording is in progress (set in + // recordingStarted(), cleared in recordingFinishedFull()). + bool m_recordingInProgress; + + // True when the current/most-recent recording was captured as a singing + // track alongside an existing reference track (RecordCreateAdditionalModel + // mode). Set in record(), cleared in recordingFinishedFull() and + // closeSession(). When true, analyseNow() routes analysis through + // m_analyser2 rather than re-analysing the primary reference track. + bool m_recordingAsSingingTrack; + virtual void octaveShift(bool up); virtual void auxSnapNotes(sv::Selection s); diff --git a/main/RealtimePitchTracker.cpp b/main/RealtimePitchTracker.cpp new file mode 100644 index 00000000..f718bcd8 --- /dev/null +++ b/main/RealtimePitchTracker.cpp @@ -0,0 +1,252 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "RealtimePitchTracker.h" + +#include "data/model/SparseTimeValueModel.h" +#include "data/model/WritableWaveFileModel.h" +#include "data/model/Model.h" +#include "base/Event.h" + +#include +#include +#include + +using namespace sv; +using std::cerr; +using std::endl; +using std::vector; + +RealtimePitchTracker::RealtimePitchTracker(ModelId audioSourceId, + ModelId pitchModelId, + QObject *parent) + : QObject(parent), + m_audioSourceId(audioSourceId), + m_pitchModelId(pitchModelId), + m_minFreq(60.0), + m_maxFreq(1000.0), + m_threshold(0.15), + m_running(false), + m_nextFrameToProcess(0), + m_timer(new QTimer(this)) +{ + // Poll for new audio every ~50 ms on the GUI thread. + // At 44100 Hz this gives us roughly 2205 new samples per tick, + // enough for 4+ YIN hops (kHopSize = 512). + m_timer->setInterval(50); + connect(m_timer, &QTimer::timeout, + this, &RealtimePitchTracker::pollAndProcess); +} + +RealtimePitchTracker::~RealtimePitchTracker() +{ + stop(); +} + +void +RealtimePitchTracker::start() +{ + m_nextFrameToProcess = 0; + m_running = true; + m_timer->start(); + cerr << "RealtimePitchTracker: started" << endl; +} + +void +RealtimePitchTracker::stop() +{ + m_timer->stop(); + m_running = false; + cerr << "RealtimePitchTracker: stopped (processed up to frame " + << m_nextFrameToProcess << ")" << endl; +} + +void +RealtimePitchTracker::pollAndProcess() +{ + if (!m_running) return; + + // --- Get the audio source model --- + auto audioModel = ModelById::getAs(m_audioSourceId); + if (!audioModel) { + // The model may not be ready yet on the very first tick — that's fine. + return; + } + + sv_frame_t totalFrames = audioModel->getFrameCount(); + if (totalFrames < kWindowSize) { + // Not enough data yet + return; + } + + // --- Get the pitch output model --- + auto pitchModel = ModelById::getAs(m_pitchModelId); + if (!pitchModel) return; + + double sr = audioModel->getSampleRate(); + + // Process as many complete windows as are available, starting + // from where we left off last time. + sv_frame_t pos = m_nextFrameToProcess; + + while (pos + kWindowSize <= totalFrames) { + + // Fetch one window of audio from channel 0. + // getData returns floatvec_t (std::vector with a custom allocator), + // so we copy into a plain std::vector for the YIN functions. + floatvec_t rawFv = audioModel->getData(0, pos, kWindowSize); + + // getData may return fewer samples if the model hasn't flushed + // the very last portion yet — skip rather than analyse garbage. + if ((int)rawFv.size() < kWindowSize) break; + + std::vector raw(rawFv.begin(), rawFv.end()); + + double hz = yinPitch(raw, sr, m_minFreq, m_maxFreq, m_threshold); + + // Frame position of the centre of the analysis window + sv_frame_t centreFrame = pos + kWindowSize / 2; + + if (hz > 0.0) { + Event e(centreFrame, float(hz), tr("")); + pitchModel->add(e); + emit pitchDetected(centreFrame, hz); + } + + pos += kHopSize; + } + + m_nextFrameToProcess = pos; +} + +// --------------------------------------------------------------------------- +// YIN algorithm +// Reference: de Cheveigné & Kawahara, "YIN, a fundamental frequency +// estimator for speech and music", JASA 111(4), 2002. +// We implement Steps 1–5 (difference function, cumulative mean +// normalised difference, absolute threshold + parabolic interpolation). +// --------------------------------------------------------------------------- + +void +RealtimePitchTracker::yinDifference(const vector &buf, + vector &diff) +{ + int windowSize = (int)buf.size(); + int halfSize = windowSize / 2; + diff.assign(halfSize, 0.0); + + // d[0] is defined as 0 + diff[0] = 0.0; + + for (int tau = 1; tau < halfSize; ++tau) { + double sum = 0.0; + for (int j = 0; j < halfSize; ++j) { + double delta = double(buf[j]) - double(buf[j + tau]); + sum += delta * delta; + } + diff[tau] = sum; + } +} + +void +RealtimePitchTracker::yinCMND(vector &diff) +{ + int halfSize = (int)diff.size(); + if (halfSize == 0) return; + + diff[0] = 1.0; // by convention + + double runningSum = 0.0; + for (int tau = 1; tau < halfSize; ++tau) { + runningSum += diff[tau]; + if (runningSum > 0.0) { + diff[tau] = diff[tau] * double(tau) / runningSum; + } else { + diff[tau] = 1.0; + } + } +} + +double +RealtimePitchTracker::yinFindPitch(const vector &cmnd, + int minLag, int maxLag, + double threshold) +{ + int halfSize = (int)cmnd.size(); + if (maxLag >= halfSize - 1) maxLag = halfSize - 2; + if (minLag < 1) minLag = 1; + if (minLag >= maxLag) return -1.0; + + // Walk forward from minLag looking for the first value below threshold + // that is also a local minimum. + int bestTau = -1; + for (int tau = minLag; tau <= maxLag; ++tau) { + if (cmnd[tau] < threshold) { + // Walk to the bottom of the dip + while (tau + 1 <= maxLag && cmnd[tau + 1] < cmnd[tau]) { + ++tau; + } + bestTau = tau; + break; + } + } + + if (bestTau < 1 || bestTau >= halfSize - 1) { + return -1.0; // unvoiced + } + + // Parabolic interpolation around the minimum + double s0 = cmnd[bestTau - 1]; + double s1 = cmnd[bestTau]; + double s2 = cmnd[bestTau + 1]; + + double denom = 2.0 * (2.0 * s1 - s2 - s0); + double refined; + if (std::abs(denom) < 1e-12) { + refined = double(bestTau); + } else { + refined = double(bestTau) + (s2 - s0) / denom; + } + + return refined; +} + +double +RealtimePitchTracker::yinPitch(const vector &window, + double sr, + double minFreq, double maxFreq, + double thresh) +{ + int halfSize = (int)window.size() / 2; + + // Lag range: larger lag → lower frequency + int minLag = std::max(1, (int)std::floor(sr / maxFreq)); + int maxLag = std::min(halfSize - 2, (int)std::ceil(sr / minFreq)); + + if (minLag >= maxLag) return 0.0; + + vector diff; + yinDifference(window, diff); + yinCMND(diff); + + double lagSamples = yinFindPitch(diff, minLag, maxLag, thresh); + if (lagSamples <= 0.0) return 0.0; + + double hz = sr / lagSamples; + + // Final range check (parabolic interpolation can push slightly out) + if (hz < minFreq || hz > maxFreq) return 0.0; + + return hz; +} \ No newline at end of file diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h new file mode 100644 index 00000000..9618ff81 --- /dev/null +++ b/main/RealtimePitchTracker.h @@ -0,0 +1,183 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef REALTIME_PITCH_TRACKER_H +#define REALTIME_PITCH_TRACKER_H + +#include +#include + +#include + +#include "base/BaseTypes.h" +#include "data/model/Model.h" +#include "data/model/SparseTimeValueModel.h" + +namespace sv { +class WritableWaveFileModel; +} + +/** + * RealtimePitchTracker polls a WritableWaveFileModel (the model being + * filled during a live microphone recording) for new audio samples and + * estimates pitch in real time using a simplified YIN autocorrelation + * algorithm. + * + * Results are written into a SparseTimeValueModel so they can be + * displayed immediately as a TimeValueLayer overlaid on the main pane, + * giving the singer real-time visual feedback about their pitch. + * + * This is intentionally a low-latency, lower-accuracy alternative to + * the full pYIN analysis that will be run once recording is complete. + * + * Usage: + * 1. Create a RealtimePitchTracker, providing the ModelId of both: + * - the WritableWaveFileModel being recorded into (audio source), + * - the SparseTimeValueModel to write pitch estimates into. + * 2. Call start() when recording begins. + * 3. The tracker polls automatically via an internal QTimer. + * 4. Call stop() when recording ends. + */ +class RealtimePitchTracker : public QObject +{ + Q_OBJECT + +public: + /** + * Construct a tracker. + * + * @param audioSourceId ModelId of the WritableWaveFileModel being + * recorded into. The tracker polls this for + * new samples on each timer tick. + * @param pitchModelId ModelId of the SparseTimeValueModel to write + * pitch estimates into. + * @param parent Optional Qt parent. + */ + RealtimePitchTracker(sv::ModelId audioSourceId, + sv::ModelId pitchModelId, + QObject *parent = nullptr); + + virtual ~RealtimePitchTracker(); + + /** + * Start tracking. Resets all internal state. + * Must be called from the GUI thread. + */ + void start(); + + /** + * Stop tracking. No more pitch estimates will be written after + * this returns. + * Must be called from the GUI thread. + */ + void stop(); + + /** + * Minimum frequency (Hz) that the tracker will report. + * Pitches below this are treated as unvoiced. Default: 60 Hz. + */ + void setMinFrequency(double hz) { m_minFreq = hz; } + double getMinFrequency() const { return m_minFreq; } + + /** + * Maximum frequency (Hz) that the tracker will report. + * Pitches above this are treated as unvoiced. Default: 1000 Hz. + */ + void setMaxFrequency(double hz) { m_maxFreq = hz; } + double getMaxFrequency() const { return m_maxFreq; } + + /** + * YIN threshold. Lower values are more selective (fewer voiced + * detections), higher values yield more detections but more + * errors. Default: 0.15. + */ + void setThreshold(double t) { m_threshold = t; } + double getThreshold() const { return m_threshold; } + +signals: + /** + * Emitted each time a new pitch estimate is available. + * + * @param frame Sample frame at which the pitch was estimated + * (centre of the analysis window), relative to the + * start of the recording. + * @param hz Estimated pitch in Hz, or 0 if unvoiced. + */ + void pitchDetected(sv::sv_frame_t frame, double hz); + +private slots: + /// Called by the internal QTimer; polls the audio model and runs YIN. + void pollAndProcess(); + +private: + // --- Model IDs --- + sv::ModelId m_audioSourceId; // WritableWaveFileModel being recorded + sv::ModelId m_pitchModelId; // SparseTimeValueModel for output + + // --- Configuration --- + double m_minFreq; + double m_maxFreq; + double m_threshold; + + // --- State --- + bool m_running; + + /// How many input frames we have already processed (exclusive end + /// of the last complete hop). We use this to avoid re-processing + /// samples on the next timer tick. + sv::sv_frame_t m_nextFrameToProcess; + + // --- Processing parameters --- + // Window size and hop size in samples. + // At 44 100 Hz: window ≈ 46 ms, hop ≈ 12 ms. + static const int kWindowSize = 2048; + static const int kHopSize = 512; + + // --- Timer --- + QTimer *m_timer; + + // --- YIN helpers --- + + /** + * Run the YIN algorithm on a single window of audio and return + * the estimated fundamental frequency in Hz, or 0 if unvoiced. + */ + static double yinPitch(const std::vector &window, + double sr, + double minFreq, double maxFreq, + double thresh); + + /** + * Step 2: difference function. + * d[tau] = sum_{j=0}^{W/2-1} (x[j] - x[j+tau])^2 + */ + static void yinDifference(const std::vector &buf, + std::vector &diff); + + /** + * Step 3: cumulative mean normalised difference (in-place). + */ + static void yinCMND(std::vector &diff); + + /** + * Steps 4-5: find first dip below threshold with parabolic + * interpolation. Returns fractional lag in samples, or -1 if + * no dip found. + */ + static double yinFindPitch(const std::vector &cmnd, + int minLag, int maxLag, + double threshold); +}; + +#endif // REALTIME_PITCH_TRACKER_H \ No newline at end of file diff --git a/main/mingw_byte_fix.h b/main/mingw_byte_fix.h new file mode 100644 index 00000000..2984db3a --- /dev/null +++ b/main/mingw_byte_fix.h @@ -0,0 +1,101 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +/* + mingw_byte_fix.h — force-include header for MinGW + C++17 builds + ----------------------------------------------------------------- + Problem + ~~~~~~~ + rpcndr.h (pulled in by unless WIN32_LEAN_AND_MEAN is + set) contains the unconditional declaration: + + typedef unsigned char byte; + + C++17's introduces std::byte as an enum class. When + both end up in scope in the same translation unit, GCC 14 raises a + hard error: + + error: reference to 'byte' is ambiguous + candidates are: 'enum class std::byte' + 'typedef unsigned char byte' + + This fires on virtually every TU in this project because: + • svcore/system/System.h includes (which drags in + rpcndr.h via winscard.h), AND + • Qt6 headers and/or the C++ standard library bring in . + + Fix + ~~~ + This header is force-included before every TU (via -include on + the compiler command line — see meson.build). + + We include here with WIN32_LEAN_AND_MEAN pre-defined. + That causes windows.h to set its own include guard (__WINDOWS_H__) + while skipping the RPC/COM sub-headers (including rpcndr.h). + + When any later code does #include the include guard + fires and the header is a no-op — rpcndr.h is therefore never + processed and the conflicting 'byte' typedef never appears. + + None of the code in this project uses RPC/DCOM types directly, so + omitting those sub-headers has no practical effect. + + After including windows.h we include so that std::byte + is already fully defined before any other header has a chance to + race with it. + + This header is a no-op on non-MinGW / non-C++17 builds. +*/ + +#ifndef MINGW_BYTE_FIX_H +#define MINGW_BYTE_FIX_H + +#if defined(__GNUC__) && defined(_WIN32) && defined(__cplusplus) && __cplusplus >= 201703L + +/* Only act if windows.h hasn't been included yet — if it has, we + are too late to block rpcndr.h, but hopefully the conflict isn't + present in that translation unit. */ +#ifndef _WINDOWS_ + +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN +# define MINGW_BYTE_FIX_DEFINED_LEAN_AND_MEAN +# endif + +# include /* sets _WINDOWS_; skips rpcndr.h */ + +# ifdef MINGW_BYTE_FIX_DEFINED_LEAN_AND_MEAN +# undef WIN32_LEAN_AND_MEAN +# undef MINGW_BYTE_FIX_DEFINED_LEAN_AND_MEAN +# endif + +/* Restore NT types (NTSTATUS etc.) that WIN32_LEAN_AND_MEAN excluded. + These are used directly by svcore/data/fileio/test/UnsupportedFormat.cpp + and indirectly by other Windows API consumers in the tree. + must be included before ; both are safe to + include after a lean because they do not re-include + rpcndr.h. */ +# ifndef NTSTATUS +# include +# endif +# include + +#endif /* !_WINDOWS_ */ + +/* Ensure std::byte is now defined cleanly. */ +#include + +#endif /* MinGW C++17 */ + +#endif /* MINGW_BYTE_FIX_H */ \ No newline at end of file diff --git a/meson.build b/meson.build index 41c9392d..10c20115 100644 --- a/meson.build +++ b/meson.build @@ -83,7 +83,7 @@ boost_dep = dependency('boost') if system == 'linux' rc = [] - + bzip2_dep = dependency('bzip2', required: false) if not bzip2_dep.found() @@ -109,7 +109,7 @@ if system == 'linux' jack_dep = dependency('jack', version: '>= 0.100') libpulse_dep = dependency('libpulse', version: '>= 0.9') alsa_dep = dependency('alsa') - + portaudio_dep = dependency('portaudio-2.0', version: '>= 19', required: false) feature_dependencies = [ @@ -168,11 +168,11 @@ if system == 'linux' svcore_moc_args = [ '-DHAVE_MAD' ] - + vamp_symbol_args += [ '-Wl,--version-script=' + meson.current_source_dir() / 'vamp-plugin-sdk/vamp-plugin.map' ] - + elif system == 'darwin' svdeps_dir = meson.current_source_dir() / 'sv-dependency-builds/osx' @@ -182,7 +182,7 @@ elif system == 'darwin' else svdeps_libdir = svdeps_dir / 'lib' endif - + rc = [] general_defines += [ @@ -191,7 +191,7 @@ elif system == 'darwin' general_link_args += [ '-mmacosx-version-min=10.15' ] - + feature_defines = [ '-DHAVE_BZ2', '-DHAVE_SNDFILE', @@ -206,7 +206,7 @@ elif system == 'darwin' '-DHAVE_VDSP', '-D__MACOSX_CORE__', # for RtMidi ] - + # vDSP will be used in preference to FFTW most of the time, but # FFTW supports more fft lengths - without it we can end up with a # very slow implementation for unusual lengths. @@ -228,9 +228,9 @@ elif system == 'darwin' svcore_moc_args += [ '-DHAVE_MAD', ] - + endif - + feature_dependencies = [ vamphostsdk_dep, ] @@ -276,18 +276,21 @@ elif system == 'darwin' '-lid3tag', ] endif - + vamp_symbol_args += [ '-exported_symbols_list', meson.current_source_dir() / 'vamp-plugin-sdk/vamp-plugin.list' ] - + elif system == 'windows' if architecture == 'x86' svdeps_dir = meson.current_source_dir() / 'sv-dependency-builds/win32-mingw' - else + elif compiler == 'msvc' svdeps_dir = meson.current_source_dir() / 'sv-dependency-builds/win64-msvc' - endif # architecture + else + # MinGW/GCC x86_64: use system libraries from msys2, no svdeps needed + svdeps_dir = '' + endif # architecture / compiler windows = import('windows') rc = windows.compile_resources('icons/tony.rc') @@ -299,7 +302,20 @@ elif system == 'windows' '-D_USE_MATH_DEFINES', '-D_HAS_STD_BYTE=1', ] - + + if compiler != 'msvc' + # MinGW + C++17: rpcndr.h (via ) unconditionally typedefs + # "unsigned char byte", which collides with the C++17 std::byte enum. + # Fix by force-including a header that pre-includes with + # WIN32_LEAN_AND_MEAN (which skips rpcndr.h), setting the include + # guard so that subsequent #include directives are no-ops + # and rpcndr.h is never processed. + _fix_header = meson.current_source_dir() / 'main/mingw_byte_fix.h' + general_defines += [ + '-include', _fix_header, + ] + endif + feature_defines = [ '-DHAVE_BZ2', '-DHAVE_FFTW3', @@ -327,7 +343,8 @@ elif system == 'windows' 'sv-dependency-builds/win32-mingw/include', 'sv-dependency-builds/win32-mingw/include/opus', ] - else + elif compiler == 'msvc' + # MSVC x86_64: use pre-built dependency tree and MediaFoundation feature_defines += [ '-DHAVE_MEDIAFOUNDATION', ] @@ -336,19 +353,39 @@ elif system == 'windows' 'sv-dependency-builds/win64-msvc/include', 'sv-dependency-builds/win64-msvc/include/opus', ] - endif # architecture - - if buildtype.startswith('release') - feature_additional_libpaths = [ - '-L' + svdeps_dir / 'lib', - ] else - feature_additional_libpaths = [ - '-L' + svdeps_dir / 'lib/debug', - '-L' + svdeps_dir / 'lib', + # MinGW/GCC x86_64: system headers from msys2; no MediaFoundation + # (shobjidl_core.h not available in mingw-w64 headers). + # Several system headers live in versioned subdirectories that code + # includes without a path prefix, so we add them explicitly. + # Use run_command to resolve the actual Windows path for the mingw prefix. + _mingw_prefix = run_command('sh', '-c', 'echo $MINGW_PREFIX', check: true).stdout().strip() + if _mingw_prefix == '' + _mingw_prefix = 'C:/msys64/mingw64' + endif + feature_include_dirs = [ + _mingw_prefix + '/include/opus', # opusfile.h includes + _mingw_prefix + '/include/sord-0', # dataquay includes + _mingw_prefix + '/include/serd-0', # dataquay includes ] + endif # architecture / compiler + + if architecture == 'x86' or compiler == 'msvc' + if buildtype.startswith('release') + feature_additional_libpaths = [ + '-L' + svdeps_dir / 'lib', + ] + else + feature_additional_libpaths = [ + '-L' + svdeps_dir / 'lib/debug', + '-L' + svdeps_dir / 'lib', + ] + endif + else + # MinGW x86_64: all libs come from the msys2 system prefix + feature_additional_libpaths = [] endif - + feature_additional_libs = [ feature_additional_libpaths, '-lbz2', @@ -377,7 +414,8 @@ elif system == 'windows' '-lwinmm', '-lws2_32', ] - elif architecture == 'x86_64' + elif compiler == 'msvc' + # MSVC x86_64 with MediaFoundation feature_additional_libs += [ # '-NODEFAULTLIB:LIBCMT', '-lsord', @@ -390,23 +428,39 @@ elif system == 'windows' '-lwinmm', '-lws2_32', ] - else + elif architecture == 'x86_64' + # MinGW x86_64: sord/serd have a -0 suffix in the msys2 packages + feature_additional_libs += [ + '-lsord-0', + '-lserd-0', + '-ladvapi32', + '-lwinmm', + '-lws2_32', + ] + else error('Build for architecture ' + architecture + ' is not supported on this platform') - endif # architecture - + endif # architecture / compiler + svcore_moc_args = [ '-DHAVE_MAD', '-DQ_OS_WIN', ] - vamp_symbol_args += [ - '-EXPORT:vampGetPluginDescriptor' - ] + if compiler == 'msvc' + vamp_symbol_args += [ + '-EXPORT:vampGetPluginDescriptor' + ] + else + # MinGW/GCC: use GNU ld syntax to export the Vamp entry point + vamp_symbol_args += [ + '-Wl,--export-dynamic-symbol=vampGetPluginDescriptor' + ] + endif else error('This operating system ("' + system + '") is not supported by this build file') endif # system - + all_include_dirs = include_directories( [ 'bqvec', @@ -918,7 +972,7 @@ svgui_moc_files = qt.preprocess( 'svgui/widgets/WindowShapePreview.h', 'svgui/widgets/WindowTypeSelector.h', ]) - + svapp_files = [ 'svapp/align/Align.cpp', 'svapp/align/ExternalProgramAligner.cpp', @@ -1023,12 +1077,14 @@ tony_main_files = [ 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', + 'main/RealtimePitchTracker.cpp', ] tony_main_moc_files = qt.preprocess( moc_headers: [ 'main/MainWindow.h', 'main/Analyser.h', + 'main/RealtimePitchTracker.h', ]) qt_resource_files = qt.preprocess( From 7346208b017f6f404a5703189f66bf0e9776d87a Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 7 Mar 2026 22:26:59 +0200 Subject: [PATCH 002/275] Create build.bat Co-authored-by: Github Copilot copilot@github.com --- build.bat | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 build.bat diff --git a/build.bat b/build.bat new file mode 100644 index 00000000..db0584ab --- /dev/null +++ b/build.bat @@ -0,0 +1,99 @@ +@echo off +setlocal + +set MSYS2_MINGW=%MSYS2_MINGW% +if "%MSYS2_MINGW%"=="" set MSYS2_MINGW=C:\msys64\mingw64 + +if not exist "%MSYS2_MINGW%\bin\ninja.exe" ( + echo ERROR: ninja.exe not found under %MSYS2_MINGW%\bin + echo Set MSYS2_MINGW to your mingw64 prefix if it is not C:\msys64\mingw64 + exit /b 1 +) + +set PATH=%MSYS2_MINGW%\bin;%MSYS2_MINGW%\..\usr\bin;%PATH% + +:: ── Parse arguments ────────────────────────────────────────────────────────── +:: build.bat → build only +:: build.bat run → build then launch Tony.exe +:: build.bat launch → launch Tony.exe without building +:: build.bat clean → wipe build_mingw and reconfigure +:: build.bat help → print this help +:: ───────────────────────────────────────────────────────────────────────────── + +set BUILD_DIR=%~dp0build_mingw +set ACTION=%1 +if "%ACTION%"=="" set ACTION=build + +if /i "%ACTION%"=="help" goto :help +if /i "%ACTION%"=="clean" goto :clean +if /i "%ACTION%"=="launch" goto :launch +if /i "%ACTION%"=="run" goto :build +if /i "%ACTION%"=="build" goto :build + +echo ERROR: Unknown action "%ACTION%". Run "build.bat help" for usage. +exit /b 1 + +:: ── clean ──────────────────────────────────────────────────────────────────── +:clean +echo Removing %BUILD_DIR% ... +if exist "%BUILD_DIR%" rmdir /s /q "%BUILD_DIR%" +echo Reconfiguring with meson ... +meson setup "%BUILD_DIR%" --buildtype=debugoptimized +if errorlevel 1 exit /b %errorlevel% +echo Done. Run build.bat to compile. +exit /b 0 + +:: ── build ──────────────────────────────────────────────────────────────────── +:build +if not exist "%BUILD_DIR%\build.ninja" ( + echo Build directory not configured. Running meson setup ... + meson setup "%BUILD_DIR%" --buildtype=debugoptimized + if errorlevel 1 exit /b %errorlevel% +) + +echo Building Tony.exe ... +ninja -C "%BUILD_DIR%" Tony.exe +if errorlevel 1 ( + echo. + echo BUILD FAILED. See errors above. + exit /b %errorlevel% +) + +echo. +echo Build succeeded: %BUILD_DIR%\Tony.exe +if /i "%ACTION%"=="run" goto :launch +exit /b 0 + +:: ── launch ─────────────────────────────────────────────────────────────────── +:launch +set EXE=%BUILD_DIR%\Tony.exe +if not exist "%EXE%" ( + echo ERROR: %EXE% not found. Run "build.bat" first. + exit /b 1 +) +echo Launching %EXE% ... +start "" "%EXE%" +exit /b 0 + +:: ── help ───────────────────────────────────────────────────────────────────── +:help +echo. +echo build.bat [action] +echo. +echo Actions: +echo (none) Build Tony.exe (default) +echo run Build Tony.exe then launch it +echo launch Launch Tony.exe without rebuilding +echo clean Delete the build directory and reconfigure from scratch +echo help Show this message +echo. +echo Environment: +echo MSYS2_MINGW Path to the mingw64 prefix (default: C:\msys64\mingw64) +echo The bin\ subdirectory must contain ninja.exe, gcc.exe etc. +echo. +echo The script prepends %%MSYS2_MINGW%%\bin and the msys2 usr\bin to PATH +echo before invoking ninja, so moc.exe and the GCC runtime DLLs are found +echo automatically regardless of which shell (PowerShell, cmd, git bash) you +echo call this from. +echo. +exit /b 0 From f9e5929f1bc1193ae4d42c238f1d039d4522f346 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 7 Mar 2026 23:54:40 +0200 Subject: [PATCH 003/275] Update meson.build Enabled `HAVE_MEDIAFOUNDATION` for the MinGW x86_64 branch and added the four required MF link libraries (`-lmfplat -lmfreadwrite -lmfuuid -lpropsys`), which are all present in msys2's mingw-w64 packages. Co-authored-by: Github Copilot copilot@github.com --- meson.build | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/meson.build b/meson.build index 10c20115..4ea289bd 100644 --- a/meson.build +++ b/meson.build @@ -354,8 +354,16 @@ elif system == 'windows' 'sv-dependency-builds/win64-msvc/include/opus', ] else - # MinGW/GCC x86_64: system headers from msys2; no MediaFoundation - # (shobjidl_core.h not available in mingw-w64 headers). + # MinGW/GCC x86_64: system headers from msys2. + # MediaFoundation IS available in mingw-w64 headers (mfapi.h, mfidl.h, + # mfreadwrite.h, etc. all ship with the mingw-w64 packages in msys2). + # The only header that was missing was shobjidl_core.h — mingw-w64 ships + # the umbrella shobjidl.h instead. MediaFoundationReadStream.cpp now + # includes when built with __MINGW32__, so we can enable + # HAVE_MEDIAFOUNDATION here too and get .m4a / .aac / .mp4 support. + feature_defines += [ + '-DHAVE_MEDIAFOUNDATION', + ] # Several system headers live in versioned subdirectories that code # includes without a path prefix, so we add them explicitly. # Use run_command to resolve the actual Windows path for the mingw prefix. @@ -429,10 +437,15 @@ elif system == 'windows' '-lws2_32', ] elif architecture == 'x86_64' - # MinGW x86_64: sord/serd have a -0 suffix in the msys2 packages + # MinGW x86_64: sord/serd have a -0 suffix in the msys2 packages. + # MediaFoundation libraries are available from the mingw-w64 runtime. feature_additional_libs += [ '-lsord-0', '-lserd-0', + '-lmfplat', + '-lmfreadwrite', + '-lmfuuid', + '-lpropsys', '-ladvapi32', '-lwinmm', '-lws2_32', From 2f3833752fb9730f1f9b73a52ef230106e1ad9dc Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 8 Mar 2026 01:49:54 +0200 Subject: [PATCH 004/275] =?UTF-8?q?fix:=20Record=20with=20reference=20trac?= =?UTF-8?q?k=20loaded=20=E2=80=94=20prevent=20crash=20on=20singing-track?= =?UTF-8?q?=20recording?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When pressing Record with a reference audio file already loaded (RecordCreateAdditionalModel mode), two bugs combined to crash the app: 1. analyseNewMainModel fired during recording. MainWindowBase::record() unconditionally emits audioFileLoaded() at the end of every recording session, which is connected to analyseNewMainModel(). In singing-track mode the main model is unchanged, but analyseNewMainModel() re-ran m_analyser->newFileLoaded() on the reference track, tearing down its pitch/note layers mid-flight. Fix: early-return guard at the top of analyseNewMainModel() when m_recordingAsSingingTrack is true. 2. Pane cleanup called deleteLayer() on the live recording model. MainWindowBase::record() creates an extra pane containing a WaveformLayer backed by the WritableWaveFileModel currently being recorded. The cleanup code in record() called m_document->deleteLayer(orphan, true), which invoked releaseModel() and — since no other layer referenced the model — called ModelById::release(), destroying the WritableWaveFileModel while audio was still being written to it. The destructor fired aboutToBeDeleted() → AudioCallbackRecordTarget nulled its model pointer and stopped recording → cascade session teardown → crash. Fix: replace deleteLayer() with a view-only detach: orphan->setLayerDormant(extra, true); extra->removeLayer(orphan); This removes the layer from the pane's display list without touching the Document model registry, keeping the WritableWaveFileModel alive. setupSingingTrackAnalyser() (deferred one event-loop tick) then adds its own waveform layer in pane 0 with a proper document reference to the model. Also included from the previous session (Session 7 groundwork, which was necessary but not sufficient on its own): - Analyser::newFileLoaded() gains a deferAnalysis parameter to skip pYIN when called on a still-recording WritableWaveFileModel. - modelAdded() routes to setupSingingTrackAnalyser(model, deferAnalysis=true) instead of the full analyseNewSingingModel() during recording. - record() saves m_paneCountBeforeRecording to identify the extra pane. - m_paneCountBeforeRecording added to MainWindow.h. Co-authored-by: Github Copilot copilot@github.com --- main/Analyser.cpp | 5 +- main/Analyser.h | 9 +++- main/MainWindow.cpp | 116 ++++++++++++++++++++++++++++++++------------ main/MainWindow.h | 11 ++++- 4 files changed, 106 insertions(+), 35 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 538746fb..54f8b5cb 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -84,7 +84,8 @@ Analyser::getAnalysisSettings() QString Analyser::newFileLoaded(Document *doc, ModelId model, - PaneStack *paneStack, Pane *pane) + PaneStack *paneStack, Pane *pane, + bool deferAnalysis) { m_document = doc; m_fileModel = model; @@ -103,7 +104,7 @@ Analyser::newFileLoaded(Document *doc, ModelId model, bool autoAnalyse = settings.value("auto-analysis", true).toBool(); settings.endGroup(); - return doAllAnalyses(autoAnalyse); + return doAllAnalyses(autoAnalyse && !deferAnalysis); } QString diff --git a/main/Analyser.h b/main/Analyser.h index 2f2c6489..b368e68b 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -56,11 +56,16 @@ class Analyser : public QObject, virtual ~Analyser(); // Process new main model, add derived layers; return "" on - // success or error string on failure + // success or error string on failure. + // If deferAnalysis is true, skip running pYIN (waveform and + // visualisation layers are still created). Use this when the + // model is a WritableWaveFileModel that is still being recorded + // into; call analyseExistingFile() once recording completes. QString newFileLoaded(sv::Document *newDocument, sv::ModelId model, sv::PaneStack *paneStack, - sv::Pane *pane); + sv::Pane *pane, + bool deferAnalysis = false); // Remove any derived layers, process the main model, add derived // layers; return "" on success or error string on failure diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0f24f87d..4a5c00dd 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -136,7 +136,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_withSonification(withSonification), m_withSpectrogram(withSpectrogram), m_recordingInProgress(false), - m_recordingAsSingingTrack(false) + m_recordingAsSingingTrack(false), + m_paneCountBeforeRecording(0) { setWindowTitle(QApplication::applicationName()); @@ -2019,7 +2020,7 @@ MainWindow::analyseNewSingingModel() } void -MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId) +MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { if (!m_document) return; if (!m_paneStack || m_paneStack->getPaneCount() < 1) return; @@ -2039,8 +2040,10 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId) connect(m_analyser2, SIGNAL(layersChanged()), this, SLOT(updateMenuStates())); + // deferAnalysis=true: only set up waveform/visualisation layers now; + // pYIN will be run later by analyseNow() once recording is complete. QString error = m_analyser2->newFileLoaded( - m_document, singingModelId, m_paneStack, pane); + m_document, singingModelId, m_paneStack, pane, deferAnalysis); if (error != "") { QMessageBox::warning(this, tr("Failed to analyse singing track"), @@ -2055,7 +2058,11 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId) updateLayerStatuses(); updateMenuStates(); - emit activity(tr("Singing track loaded and analysis started")); + if (deferAnalysis) { + emit activity(tr("Recording singing track — analysis will run when recording stops")); + } else { + emit activity(tr("Singing track loaded and analysis started")); + } } void @@ -2219,7 +2226,7 @@ MainWindow::record() // We do this by temporarily switching to RecordCreateAdditionalModel so // that MainWindowBase::record() adds the WritableWaveFileModel as an // additional (non-main) model. Our modelAdded() hook will then pick it - // up and route it through setupSingingTrackAnalyser(). + // up and route it through setupSingingTrackAnalyser() with deferred pYIN. // // If there is no main model yet (first-time record), fall through with the // default RecordReplaceSession behaviour. @@ -2229,9 +2236,14 @@ MainWindow::record() if (haveReference) { cerr << "MainWindow::record: reference track loaded — recording as singing track" << endl; m_recordingAsSingingTrack = true; + // Remember pane count so we can prune the extra pane that + // MainWindowBase::record() creates via AddPaneCommand for the + // recording's waveform layer. We want both tracks in pane 0. + m_paneCountBeforeRecording = m_paneStack ? m_paneStack->getPaneCount() : 0; setAudioRecordMode(RecordCreateAdditionalModel); } else { m_recordingAsSingingTrack = false; + m_paneCountBeforeRecording = 0; setAudioRecordMode(RecordReplaceSession); } @@ -2240,6 +2252,42 @@ MainWindow::record() // Restore the default mode so that a subsequent "standalone" recording // (after the singing track session is closed) behaves correctly. setAudioRecordMode(RecordReplaceSession); + + // Remove any extra panes that AddPaneCommand created for the recording's + // waveform layer. setupSingingTrackAnalyser() will add a proper waveform + // layer for the recording into pane 0, so the auto-created extra pane is + // redundant and visually confusing. + // + // IMPORTANT: do NOT call m_document->deleteLayer() here. The extra pane + // contains a WaveformLayer whose model is the WritableWaveFileModel that + // is currently being recorded into. deleteLayer() calls releaseModel(), + // which — since this waveform layer is the only layer referencing it — + // destroys the WritableWaveFileModel while the AudioCallbackRecordTarget + // is still writing to it, causing a crash. + // + // Instead, directly detach each layer from the pane view (Pane::removeLayer) + // without going through the Document. The layers remain in the Document's + // internal layer list and will be cleaned up when the document is closed or + // the model is properly released later (via setupSingingTrackAnalyser which + // will take ownership of the model through its own waveform layer). + if (m_recordingAsSingingTrack && m_paneStack) { + while (m_paneStack->getPaneCount() > m_paneCountBeforeRecording) { + Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); + if (extra) { + // Detach layers from this view only — do not delete them or + // release their models. Pane::removeLayer just removes the + // layer from the view's display list; it does not touch the + // Document model registry. + while (extra->getLayerCount() > 0) { + Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); + orphan->setLayerDormant(extra, true); + extra->removeLayer(orphan); + } + } + if (m_overview) m_overview->unregisterView(extra); + m_paneStack->deletePane(extra); + } + } } void @@ -3561,12 +3609,27 @@ MainWindow::modelAdded(ModelId model) m_realtimePitchModelId != model) { // Guard against race: if modelAdded() fires twice quickly (e.g. // for an audio model and its alignment model), only set the - // pending id once; analyseNewSingingModel() will clear it when - // it runs. + // pending id once. if (m_pendingSingingModelId.isNone()) { m_pendingSingingModelId = model; - // Defer so the model is fully registered before we analyse - QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); + + if (m_recordingAsSingingTrack) { + // The model is a WritableWaveFileModel still being + // recorded into. Set up m_analyser2 with waveform/ + // visualisation layers but defer pYIN until recording + // finishes (analyseNow() will call analyseExistingFile()). + // Defer one event-loop tick so the model is fully + // registered with the document before we touch it. + QTimer::singleShot(0, this, [this, model]() { + m_pendingSingingModelId = {}; + setupSingingTrackAnalyser(model, /*deferAnalysis=*/true); + }); + } else { + // Normal case: a finished audio file was loaded as a + // singing track. Run full analysis immediately. + // Defer so the model is fully registered before we analyse. + QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); + } } else { cerr << "modelAdded: m_pendingSingingModelId already set, ignoring model " << model << endl; @@ -3684,6 +3747,20 @@ MainWindow::analyseNow() void MainWindow::analyseNewMainModel() { + // When recording as a singing track alongside an existing reference track + // (RecordCreateAdditionalModel mode), MainWindowBase::record() still emits + // audioFileLoaded() at the end — which triggers this slot. But in that + // mode the main model has NOT changed (it is still the reference track), + // so there is nothing for this slot to do. All secondary-track setup is + // handled by modelAdded() → setupSingingTrackAnalyser(). Proceeding here + // would wrongly call m_analyser->newFileLoaded() on the reference track a + // second time, tearing down its existing pitch/note layers and re-running + // pYIN — causing errors, crashes, and a corrupt UI state. + if (m_recordingAsSingingTrack) { + cerr << "analyseNewMainModel: recording-as-singing-track mode, skipping (main model unchanged)" << endl; + return; + } + auto model = getMainModel(); SVDEBUG << "MainWindow::analyseNewMainModel: main model is " << model << endl; @@ -3700,27 +3777,6 @@ MainWindow::analyseNewMainModel() return; } - // When recording as a singing track (RecordCreateAdditionalModel mode), - // MainWindowBase::record() creates an extra pane for the recording waveform. - // We want both tracks in pane 0, so remove any panes beyond the expected 2 - // (main pane + time-ruler strip) before doing anything else. - // This mirrors the pane-cleanup logic in loadSingingTrack(). - if (m_recordingAsSingingTrack) { - int expectedPanes = 2; // pane 0 (main) + pane 1 (time ruler strip) - while (m_paneStack->getPaneCount() > expectedPanes) { - Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - if (!extra) break; - // Remove all layers from the extra pane (force=true avoids undo commands - // that would hold dangling pointers to the layer). - while (extra->getLayerCount() > 0) { - Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); - m_document->deleteLayer(orphan, true); - } - m_paneStack->deletePane(extra); - } - cerr << "analyseNewMainModel: pruned extra recording pane (recording-as-singing-track mode)" << endl; - } - int pc = m_paneStack->getPaneCount(); Pane *pane = 0; Pane *selectionStrip = 0; diff --git a/main/MainWindow.h b/main/MainWindow.h index 2ed6fa51..3a232afe 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -292,7 +292,10 @@ protected slots: virtual void setupToolbars(); // Helpers for the singing / second-track workflow - virtual void setupSingingTrackAnalyser(sv::ModelId singingModelId); + // deferAnalysis=true skips pYIN (used when the model is a + // WritableWaveFileModel still being recorded into). + virtual void setupSingingTrackAnalyser(sv::ModelId singingModelId, + bool deferAnalysis = false); virtual void teardownSingingTrackAnalyser(); virtual void setupRealtimePitchLayer(); virtual void teardownRealtimePitchLayer(); @@ -313,6 +316,12 @@ protected slots: // m_analyser2 rather than re-analysing the primary reference track. bool m_recordingAsSingingTrack; + // Pane count saved just before MainWindowBase::record() is called in + // singing-track mode. After record() returns, any panes above this + // count are extra panes created by AddPaneCommand for the recording's + // waveform layer; we remove them so both tracks share pane 0. + int m_paneCountBeforeRecording; + virtual void octaveShift(bool up); virtual void auxSnapNotes(sv::Selection s); From 1dcdb1c1e338202be73b574bd30c6630693e5acd Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 8 Mar 2026 06:06:28 +0200 Subject: [PATCH 005/275] Add Analyser::removeAllLayers and singing track recording cleanup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes a crash when replacing a singing-track recording with a new one, and prevents orphan models and layers from remaining in the document after recording teardown. The key issue: when tearing down m_analyser2 (the secondary analyser for the previous recording), we cannot simply call fileClosed() because it leaves layers registered in the document without removing them from views. This causes dangling pointers in m_layerViewMap and prevents Document::releaseModel() from freeing the old recording model, leading to stale models appearing when setting up the new recording. The solution adds three mechanisms: 1. Analyser::removeAllLayers() — explicitly removes each layer from views and deletes it from the document, then calls fileClosed(). Use this when tearing down an analyser while the document is alive. 2. MainWindow::drainPendingExtraPanes() — handles extra panes created by record() that contain orphan waveform layers. Must be called after m_analyser2 is initialized (so its WaveformLayer holds the model reference), before deletePane() is called on the extra panes. 3. Proper sequencing in setupSingingTrackAnalyser() and teardownSingingTrackAnalyser() to ensure layers are deleted only when safe, and to track the current recording model ID so setupRealtimePitchLayer() doesn't scan and find stale models. Co-authored-by: Github Copilot copilot@github.com --- main/Analyser.cpp | 47 ++++++ main/Analyser.h | 8 + main/MainWindow.cpp | 404 ++++++++++++++++++++++++++++++++++++++------ main/MainWindow.h | 33 ++++ 4 files changed, 443 insertions(+), 49 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 54f8b5cb..70bb895e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -178,6 +178,53 @@ Analyser::fileClosed() m_reAnalysingSelection = Selection(); } +void +Analyser::removeAllLayers() +{ + cerr << "Analyser::removeAllLayers" << endl; + + // First discard any re-analysis candidate layers (these are not in + // m_layers, but they are registered with the document). + discardPitchCandidates(); + + // Remove and delete each layer this analyser owns, in reverse stacking + // order (Notes on top, then PitchTrack, Spectrogram, Audio at bottom). + // We iterate over a fixed order rather than the map itself because + // deleteLayer() can trigger layerAboutToBeDeleted() which modifies m_layers. + static const Component order[] = { Notes, Spectrogram, PitchTrack, Audio }; + + for (Component c : order) { + auto it = m_layers.find(c); + if (it == m_layers.end() || !it->second) continue; + + Layer *layer = it->second; + it->second = nullptr; // clear before deleteLayer fires the slot + + if (m_document) { + // Use deleteLayer(force=true) directly — do NOT call + // removeLayerFromView first. + // + // removeLayerFromView creates a RemoveLayerCommand in the undo + // history with m_added=false. If deleteLayer then destroys the + // layer object, that command holds a dangling pointer. When + // CommandHistory is later cleared (e.g. on closeSession) the + // RemoveLayerCommand destructor checks !m_added and calls + // m_d->deleteLayer(m_layer) on the already-deleted layer — + // use-after-free / crash, and the old model stays alive in the + // undo entry, causing its orange dots to reappear. + // + // deleteLayer(force=true) removes the layer from all views + // internally (without generating any undo command), then + // releases the model if unreferenced and deletes the layer. + // This is the correct path for a silent, non-undoable replace. + m_document->deleteLayer(layer, true); + } + } + + // fileClosed() clears the rest of the state (candidates, selection, etc.) + fileClosed(); +} + bool Analyser::getDisplayFrequencyExtents(double &min, double &max) { diff --git a/main/Analyser.h b/main/Analyser.h index b368e68b..f7a26fc6 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -73,6 +73,14 @@ class Analyser : public QObject, // Discard any layers etc associated with the current document void fileClosed(); + + // Remove all layers this analyser owns from the pane and delete them + // from the document, then call fileClosed(). Use this instead of + // fileClosed() when the analyser is being torn down while the document + // is still alive (e.g. when replacing a singing-track recording). + // Unlike fileClosed(), this actually cleans up the view and the document + // model registry so no orphan layers or models remain. + void removeAllLayers(); void setIntelligentActions(bool); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4a5c00dd..9958d59c 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -137,7 +137,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_withSpectrogram(withSpectrogram), m_recordingInProgress(false), m_recordingAsSingingTrack(false), - m_paneCountBeforeRecording(0) + m_paneCountBeforeRecording(0), + m_currentRecordingModelId() { setWindowTitle(QApplication::applicationName()); @@ -1862,6 +1863,7 @@ MainWindow::closeSession() teardownRealtimePitchLayer(); teardownSingingTrackAnalyser(); m_pendingSingingModelId = {}; + m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; m_analyser->fileClosed(); @@ -1893,6 +1895,12 @@ MainWindow::closeSession() m_paneStack->deletePane(pane); } + // m_pendingExtraPanes holds panes that were moved to m_hiddenPanes via + // hidePane() in record(). The getHiddenPaneCount() loop above already + // handled them — they are now deleted. Clear our list so the pointers + // are not used again. + m_pendingExtraPanes.clear(); + delete m_document; m_document = 0; m_viewManager->clearSelections(); @@ -2025,7 +2033,11 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal if (!m_document) return; if (!m_paneStack || m_paneStack->getPaneCount() < 1) return; - // Reuse the main pane (pane 0) so both pitch tracks overlay each other + // Reuse the main pane (pane 0) so both pitch tracks overlay each other. + // NOTE: when called from the recording flow, m_pendingExtraPanes may hold + // extra panes that were hidden but not yet deleted (see record()). We must + // check getPaneCount() AFTER accounting for those hidden panes. Pane 0 + // is always the main analysis pane created by analyseNewMainModel(). Pane *pane = m_paneStack->getPane(0); if (!pane) return; @@ -2050,9 +2062,26 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal tr("Analysis failed

%1

").arg(error)); delete m_analyser2; m_analyser2 = nullptr; + // Do NOT drain m_pendingExtraPanes here — nothing holds a reference + // to the recording model at this point, so deleteLayer(orphan, true) + // would free the live WritableWaveFileModel mid-capture → crash. + // The hidden extra pane will be cleaned up by closeSession()'s + // getHiddenPaneCount() loop, or on the next successful recording. return; } + // m_analyser2->newFileLoaded() has now created its own WaveformLayer + // referencing singingModelId. This means it is safe to delete the orphan + // WaveformLayer that MainWindowBase::record() put in the extra pane: + // Document::releaseModel() will not free singingModelId because + // m_analyser2's layer still holds a reference to it. + // + // We must do this BEFORE calling m_paneStack->deletePane(), because + // deleteLayer(force=true) iterates Document::m_layerViewMap to remove the + // layer from any views — and that map still contains the live pane pointer. + // After deletePane() the pointer would be dangling → crash. + drainPendingExtraPanes(); + // Re-stack layers so the primary pitch track stays on top m_analyser->getLayer(Analyser::PitchTrack); // ensure primary is on top updateLayerStatuses(); @@ -2065,12 +2094,127 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal } } +void +MainWindow::drainPendingExtraPanes() +{ + // This helper is called from setupSingingTrackAnalyser() after m_analyser2 + // has been initialised with its own WaveformLayer referencing the recording + // model. At that point it is safe to call deleteLayer(orphan, true) on the + // extra pane's waveform layer because: + // (a) m_analyser2's WaveformLayer holds a reference to the model, so + // Document::releaseModel() will not free it. + // (b) The extra pane widget is still alive (we haven't called deletePane + // yet), so Document::m_layerViewMap iteration in deleteLayer(true) is + // valid and won't dereference a dangling pointer. + // + // It is also called from teardownSingingTrackAnalyser() and closeSession() + // as a safety net — in those contexts m_document may be null so we just + // call deletePane() to destroy the widget without touching the document. + if (m_pendingExtraPanes.empty()) return; + + cerr << "MainWindow::drainPendingExtraPanes: draining " + << m_pendingExtraPanes.size() << " pending extra pane(s)" << endl; + + for (Pane *extra : m_pendingExtraPanes) { + if (!extra) continue; + + if (m_document) { + // The extra pane created by MainWindowBase::record() in + // RecordCreateAdditionalModel mode contains two layers: + // + // 1. m_timeRulerLayer — a SHARED layer that also lives in pane 0. + // We must NOT call deleteLayer() on it; that would remove the + // ruler from every view including pane 0. Instead, use + // removeLayerFromView(extra, layer) so only the extra pane's + // entry is removed from m_layerViewMap (this creates an undo + // command, but the layer pointer remains valid so no crash). + // + // 2. The orphan WaveformLayer from createImportedLayer() — unique + // to this pane, with the WritableWaveFileModel as its model. + // deleteLayer(force=true) is safe here because: + // (a) m_analyser2 was set up before this drain, so its + // WaveformLayer still holds a reference to the model + // → releaseModel() will not free the live recording. + // (b) The extra pane widget is still alive → m_layerViewMap + // iteration in deleteLayer(true) is valid. + // + // We distinguish them by whether the model is a WritableWaveFileModel. + int layerCount = extra->getLayerCount(); + for (int i = layerCount - 1; i >= 0; --i) { + Layer *lay = extra->getLayer(i); + if (!lay) continue; + + ModelId layerModel = lay->getModel(); + + if (ModelById::isa(layerModel)) { + // Orphan recording waveform: full delete (safe because + // m_analyser2's WaveformLayer still holds the model ref). + // deleteLayer(force=true) iterates m_layerViewMap and calls + // view->removeLayer() — the pane is still alive here so + // the pointer is valid. + cerr << "MainWindow::drainPendingExtraPanes: deleteLayer orphan " + << lay << " [" << lay->objectName().toStdString() + << "] (WritableWaveFileModel)" << endl; + m_document->deleteLayer(lay, true); + } else { + // Shared layer (e.g. TimeRuler): we cannot call deleteLayer + // because that would remove the layer from ALL views. We + // also cannot call removeLayerFromView because that pushes a + // RemoveLayerCommand onto the undo stack with a raw pointer + // to the extra pane — if undo is later triggered the pane + // is already destroyed → crash. + // + // Instead, use detachLayerFromView (no undo command): removes + // the layer from the pane's display list AND updates + // m_layerViewMap so deletePane() leaves no dangling pointer. + cerr << "MainWindow::drainPendingExtraPanes: detachLayerFromView " + << lay << " [" << lay->objectName().toStdString() + << "] (shared layer, keeping in other views)" << endl; + m_document->detachLayerFromView(extra, lay); + } + } + } + + // Now destroy the pane widget. By this point all WritableWaveFileModel + // layers have been deleted from the document and m_layerViewMap no + // longer references this pane for them. Shared layers were properly + // detached via removeLayerFromView. deletePane() can safely destroy + // the widget without leaving dangling pointers in m_layerViewMap. + if (m_paneStack) { + m_paneStack->deletePane(extra); + } + } + + m_pendingExtraPanes.clear(); +} + void MainWindow::teardownSingingTrackAnalyser() { if (!m_analyser2) return; - m_analyser2->fileClosed(); + // removeAllLayers() removes each layer from the pane and deletes it from + // the document (releasing the model if unreferenced), then calls + // fileClosed() to clear the analyser's internal state. This is the + // correct teardown when the document is still alive (e.g. when replacing + // a previous singing-track recording with a new one). + // + // NOTE: we do NOT call drainPendingExtraPanes() here. The pending extra + // panes always belong to the most-recently-started recording (the one + // about to begin, not the one being torn down). Draining them here would + // call deleteLayer(orphan) while m_analyser2 for the NEW recording doesn't + // exist yet, so nothing would hold the new model reference → crash. + // drainPendingExtraPanes() is called from setupSingingTrackAnalyser() once + // m_analyser2 is set up and its WaveformLayer holds the model reference. + // closeSession() handles any residual hidden panes via its own + // getHiddenPaneCount() loop using removeLayerFromView + deletePane. + if (m_document) { + m_analyser2->removeAllLayers(); + } else { + // Document is already gone (e.g. closeSession destroyed it); just + // clear the in-memory state without touching the document. + m_analyser2->fileClosed(); + } delete m_analyser2; m_analyser2 = nullptr; } @@ -2107,22 +2251,44 @@ MainWindow::setupRealtimePitchLayer() ModelId audioSourceId; if (m_recordingAsSingingTrack && m_document) { - // Scan document models for any WritableWaveFileModel (the recording). - ModelId mainId = getMainModelId(); - for (ModelId mid : m_document->getModels()) { - if (mid == mainId) continue; - if (ModelById::isa(mid)) { - audioSourceId = mid; - cerr << "setupRealtimePitchLayer: found singing-track recording model " - << mid << endl; - break; + // Use the model ID captured in modelAdded() when the recording + // WritableWaveFileModel was first registered. Do NOT scan all + // document models here: a previous recording's WritableWaveFileModel + // may still be registered (because its orphan waveform layer, which + // was view-detached but not deleted from m_document->m_layers, holds + // a reference that prevents releaseModel() from freeing it). A scan + // would find that stale model first and point the tracker at the + // completed old recording, replaying its entire pitch content as dots. + if (!m_currentRecordingModelId.isNone()) { + audioSourceId = m_currentRecordingModelId; + cerr << "setupRealtimePitchLayer: using captured recording model " + << audioSourceId << endl; + } else { + // Fallback: m_currentRecordingModelId not yet set (modelAdded + // deferred lambda hasn't fired). Scan as last resort but prefer + // the model with the fewest frames (most recently started). + ModelId mainId = getMainModelId(); + sv_frame_t fewestFrames = -1; + for (ModelId mid : m_document->getModels()) { + if (mid == mainId) continue; + if (ModelById::isa(mid)) { + auto wfm = ModelById::getAs(mid); + sv_frame_t frames = wfm ? wfm->getFrameCount() : 0; + if (audioSourceId.isNone() || frames < fewestFrames) { + audioSourceId = mid; + fewestFrames = frames; + } + } + } + if (audioSourceId.isNone()) { + audioSourceId = mainId; + cerr << "setupRealtimePitchLayer: could not find singing-track " + "recording model, falling back to main model" << endl; + } else { + cerr << "setupRealtimePitchLayer: fallback scan found recording " + << "model " << audioSourceId + << " (fewest frames=" << fewestFrames << ")" << endl; } - } - if (audioSourceId.isNone()) { - // Fall back to main model in case the search failed. - audioSourceId = mainId; - cerr << "setupRealtimePitchLayer: could not find singing-track recording " - "model, falling back to main model" << endl; } } else { audioSourceId = getMainModelId(); @@ -2200,13 +2366,26 @@ MainWindow::teardownRealtimePitchLayer() } if (m_realtimePitchLayer) { - if (m_paneStack && m_paneStack->getPaneCount() > 0) { - Pane *pane = m_paneStack->getPane(0); - if (pane) { - m_document->removeLayerFromView(pane, m_realtimePitchLayer); - } + // Use deleteLayer(force=true) directly — do NOT call + // removeLayerFromView first. + // + // removeLayerFromView creates a RemoveLayerCommand in the undo + // history with m_added=false. If deleteLayer then destroys the + // layer object, that command holds a dangling pointer. When + // CommandHistory is later cleared (e.g. on the next closeSession) + // the RemoveLayerCommand destructor checks !m_added and calls + // m_d->deleteLayer(m_layer) on the already-deleted layer — + // use-after-free / crash, and the old SparseTimeValueModel can + // stay alive inside the undo entry long enough that its orange + // dots reappear during the next recording. + // + // deleteLayer(force=true) removes the layer from all views + // internally (without generating any undo command), then + // releases the model if unreferenced and deletes the layer. + // This is the correct path for a silent, non-undoable teardown. + if (m_document) { + m_document->deleteLayer(m_realtimePitchLayer, true); } - m_document->deleteLayer(m_realtimePitchLayer, false); m_realtimePitchLayer = nullptr; } @@ -2221,6 +2400,17 @@ MainWindow::teardownRealtimePitchLayer() void MainWindow::record() { + // If recording is already in progress this click is a STOP request, not a + // start request. Delegate straight to the base class (which calls stop()) + // without doing any pre-flight teardown. The teardown would destroy + // m_analyser2 and remove the live recording model from m_playSource while + // audio is still being captured — causing a crash or a null m_analyser2 + // when recordCompleted() fires analyseNow() moments later. + if (m_recordTarget && m_recordTarget->isRecording()) { + MainWindowBase::record(); + return; + } + // If a reference track is already loaded, record the microphone input as // the singing track rather than replacing the whole session. // We do this by temporarily switching to RecordCreateAdditionalModel so @@ -2235,6 +2425,102 @@ MainWindow::record() if (haveReference) { cerr << "MainWindow::record: reference track loaded — recording as singing track" << endl; + + // If a previous singing-track recording (or loaded singing file) is + // still active, discard it now before we start capturing a new one. + // teardownRealtimePitchLayer() stops any live tracker still running + // (edge case: user re-records before pYIN finished on the last one). + // teardownSingingTrackAnalyser() removes the old recording's layers + // from the pane and releases its model so the document is clean. + // m_pendingSingingModelId is cleared so the modelAdded() race guard + // doesn't block the new recording's model from being registered. + if (m_realtimePitchTracker || m_realtimePitchLayer) { + cerr << "MainWindow::record: tearing down leftover realtime pitch layer" << endl; + teardownRealtimePitchLayer(); + } + + // Pre-flight orphan cleanup: delete the WaveformLayer that + // MainWindowBase::record() created via createImportedLayer() for the + // previous singing recording, and remove that model from m_playSource. + // + // This MUST be done before teardownSingingTrackAnalyser() (which calls + // removeAllLayers() and would otherwise release the singing model while + // the orphan layer still holds a reference) AND before deletePane() + // (which would destroy the extra pane and leave a dangling pointer in + // m_document->m_layerViewMap for the orphan layer — causing a crash in + // deleteLayer(true) when it tries to call removeLayer on the dead pane). + // + // At this point the extra pane is still alive (deletePane hasn't run), + // so m_layerViewMap contains a valid pane pointer, and deleteLayer(true) + // is safe. + // + // Two cases arise depending on when the user presses Record again: + // + // (A) User re-records while pYIN is still running (or before + // recordingFinishedFull() has fired): m_currentRecordingModelId + // is still set to the previous recording's WritableWaveFileModel. + // + // (B) User re-records after pYIN has completed: recordingFinishedFull() + // already cleared m_currentRecordingModelId to {}. However, + // m_analyser2 is still alive and its getMainModelId() still returns + // the previous singing model's ID (fileClosed() clears m_layers but + // NOT m_fileModel). We use that ID for the orphan scan instead. + // + // In both cases we identify orphan layers by scanning all document + // layers for any layer whose model matches the previous singing model ID, + // excluding layers that m_analyser2 owns (those are cleaned up by + // removeAllLayers() inside teardownSingingTrackAnalyser() below). + { + ModelId prevSingingModelId = m_currentRecordingModelId; + if (prevSingingModelId.isNone() && m_analyser2) { + prevSingingModelId = m_analyser2->getMainModelId(); + if (!prevSingingModelId.isNone()) { + cerr << "MainWindow::record: m_currentRecordingModelId cleared " + << "(post-pYIN re-record); using m_analyser2 model id " + << prevSingingModelId << " for orphan cleanup" << endl; + } + } + + if (m_document && !prevSingingModelId.isNone()) { + std::vector orphans; + for (Layer *layer : m_document->getLayers()) { + if (layer->getModel() != prevSingingModelId) continue; + // Skip layers owned by m_analyser2 — removeAllLayers() handles those. + bool ownedByAnalyser2 = false; + if (m_analyser2) { + for (int c = Analyser::Audio; c <= Analyser::Spectrogram; ++c) { + if (m_analyser2->getLayer(static_cast(c)) == layer) { + ownedByAnalyser2 = true; + break; + } + } + } + if (!ownedByAnalyser2) { + orphans.push_back(layer); + } + } + for (Layer *orphan : orphans) { + cerr << "MainWindow::record: deleting orphan layer " << orphan + << " referencing previous singing model " + << prevSingingModelId << endl; + m_document->deleteLayer(orphan, true); + } + // Explicitly remove from play source — belt-and-suspenders in case + // deleteLayer(true)'s layerInAView(false) path didn't fire. + if (m_playSource) { + m_playSource->removeModel(prevSingingModelId); + } + m_currentRecordingModelId = {}; + } + } + + if (m_analyser2) { + cerr << "MainWindow::record: tearing down previous singing-track analyser" << endl; + teardownSingingTrackAnalyser(); + } + m_pendingSingingModelId = {}; + m_recordingInProgress = false; + m_recordingAsSingingTrack = true; // Remember pane count so we can prune the extra pane that // MainWindowBase::record() creates via AddPaneCommand for the @@ -2258,34 +2544,48 @@ MainWindow::record() // layer for the recording into pane 0, so the auto-created extra pane is // redundant and visually confusing. // - // IMPORTANT: do NOT call m_document->deleteLayer() here. The extra pane - // contains a WaveformLayer whose model is the WritableWaveFileModel that - // is currently being recorded into. deleteLayer() calls releaseModel(), - // which — since this waveform layer is the only layer referencing it — - // destroys the WritableWaveFileModel while the AudioCallbackRecordTarget - // is still writing to it, causing a crash. + // WHY WE CANNOT DELETE THE ORPHAN LAYER HERE: + // At this point m_analyser2 has NOT yet been created — its setup is deferred + // via QTimer::singleShot(0) queued inside modelAdded(). The orphan + // WaveformLayer in the extra pane is currently the ONLY layer referencing + // the WritableWaveFileModel being recorded into. Calling + // deleteLayer(orphan, true) would invoke Document::releaseModel(), which + // would destroy the live recording model mid-capture — crash. + // + // WHY WE CANNOT CALL Pane::removeLayer() + deletePane() HERE: + // Pane::removeLayer() removes the layer from the View's internal display + // list but does NOT update Document::m_layerViewMap. After deletePane() + // destroys the widget, m_layerViewMap still contains the now-dangling pane + // pointer. On the next recording attempt, the pre-flight orphan cleanup + // calls deleteLayer(orphan, true), which iterates m_layerViewMap and calls + // (*j)->removeLayer(layer) on the stale pointer — use-after-free crash. // - // Instead, directly detach each layer from the pane view (Pane::removeLayer) - // without going through the Document. The layers remain in the Document's - // internal layer list and will be cleaned up when the document is closed or - // the model is properly released later (via setupSingingTrackAnalyser which - // will take ownership of the model through its own waveform layer). + // SOLUTION: use PaneStack::hidePane() to move the extra pane out of the + // visible list (so getPaneCount() drops back and the UI doesn't show it) + // while keeping the widget alive with valid m_layerViewMap entries. + // Store the pane in m_pendingExtraPanes. setupSingingTrackAnalyser() will + // drain that list once m_analyser2 is set up and its WaveformLayer holds a + // reference to the recording model, at which point deleteLayer(orphan, true) + // is safe (the model won't be freed because m_analyser2's layer still refs it) + // and deletePane() can safely destroy the now-clean pane widget. if (m_recordingAsSingingTrack && m_paneStack) { while (m_paneStack->getPaneCount() > m_paneCountBeforeRecording) { Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - if (extra) { - // Detach layers from this view only — do not delete them or - // release their models. Pane::removeLayer just removes the - // layer from the view's display list; it does not touch the - // Document model registry. - while (extra->getLayerCount() > 0) { - Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); - orphan->setLayerDormant(extra, true); - extra->removeLayer(orphan); - } - } + if (!extra) break; + + // Unregister from the overview before hiding so it stops rendering. if (m_overview) m_overview->unregisterView(extra); - m_paneStack->deletePane(extra); + + // hidePane() moves the pane from the visible list to m_hiddenPanes, + // calls pw->hide() on the widget, and updates getPaneCount() — so + // this while loop will terminate correctly. + m_paneStack->hidePane(extra); + + // Store for deferred cleanup in setupSingingTrackAnalyser(). + m_pendingExtraPanes.push_back(extra); + + cerr << "MainWindow::record: hiding extra pane " << extra + << " — deferred deletion queued for setupSingingTrackAnalyser" << endl; } } } @@ -2382,6 +2682,7 @@ MainWindow::recordingFinishedFull() cerr << "MainWindow::recordingFinishedFull: pYIN done, removing realtime pitch layer" << endl; m_recordingInProgress = false; m_recordingAsSingingTrack = false; + m_currentRecordingModelId = {}; teardownRealtimePitchLayer(); updateLayerStatuses(); updateMenuStates(); @@ -3618,8 +3919,13 @@ MainWindow::modelAdded(ModelId model) // recorded into. Set up m_analyser2 with waveform/ // visualisation layers but defer pYIN until recording // finishes (analyseNow() will call analyseExistingFile()). - // Defer one event-loop tick so the model is fully - // registered with the document before we touch it. + // Also store the model ID so setupRealtimePitchLayer() + // can target this exact model rather than scanning all + // document models (which would wrongly pick up a previous + // recording's WritableWaveFileModel that is still + // registered because its orphan waveform layer prevents + // releaseModel() from freeing it). + m_currentRecordingModelId = model; QTimer::singleShot(0, this, [this, model]() { m_pendingSingingModelId = {}; setupSingingTrackAnalyser(model, /*deferAnalysis=*/true); diff --git a/main/MainWindow.h b/main/MainWindow.h index 3a232afe..52f63617 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -20,6 +20,8 @@ #include "Analyser.h" #include "RealtimePitchTracker.h" +#include + #include "data/model/SparseTimeValueModel.h" namespace sv { @@ -300,6 +302,15 @@ protected slots: virtual void setupRealtimePitchLayer(); virtual void teardownRealtimePitchLayer(); + // Drain m_pendingExtraPanes: delete orphan layers and the pane widgets + // that were deferred from record()'s pane-cleanup step. Must be called + // after m_analyser2 has created its WaveformLayer (so deleteLayer won't + // free the recording model), and while the pane widgets are still alive + // (so m_layerViewMap iteration in deleteLayer(force=true) is valid). + // Also called as a safety net from teardownSingingTrackAnalyser() and + // closeSession() to avoid leaking widgets. + void drainPendingExtraPanes(); + // When loadSingingTrack opens an additional audio file, modelAdded() // stores the resulting ModelId here so analyseNewSingingModel() can // pick it up on the next event-loop iteration. @@ -322,6 +333,28 @@ protected slots: // waveform layer; we remove them so both tracks share pane 0. int m_paneCountBeforeRecording; + // The WritableWaveFileModel being recorded into in the current (or most + // recent) singing-track recording. Set in modelAdded() when + // m_recordingAsSingingTrack is true, cleared in closeSession() and + // recordingFinishedFull(). Used by setupRealtimePitchLayer() to + // identify the correct audio source model without scanning all document + // models — a scan would incorrectly pick up a previous recording's + // WritableWaveFileModel that is still registered in the document because + // its orphan waveform layer (view-detached but still in m_document's + // layer list) holds a reference that prevents releaseModel() from + // freeing it. + sv::ModelId m_currentRecordingModelId; + + // Extra panes created by MainWindowBase::record() via AddPaneCommand + // that we want to hide immediately but cannot delete yet because + // m_analyser2 hasn't been set up yet (it is deferred via QTimer::singleShot). + // Stored here so that setupSingingTrackAnalyser() can properly delete + // them — AFTER m_analyser2 has created its own WaveformLayer referencing + // the recording model, making it safe to call deleteLayer(orphan, true) + // on the extra pane's waveform layer without releasing the recording model. + // Also drained by teardownSingingTrackAnalyser() and closeSession(). + std::vector m_pendingExtraPanes; + virtual void octaveShift(bool up); virtual void auxSnapNotes(sv::Selection s); From 275a1b5e796836975949fb7adec9f19a9469abad Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 8 Mar 2026 16:24:00 +0200 Subject: [PATCH 006/275] Add singing audio playback controls Add toggle for singing audio playback in the analysis layer and a "play reference while recording" feature to help singers time their performance. Reference playback is persisted in settings and automatically managed during recording sessions. Co-authored-by: Github Copilot copilot@github.com --- main/MainWindow.cpp | 95 ++++++++++++++++++++++++++++++++++++++++++++- main/MainWindow.h | 3 ++ 2 files changed, 96 insertions(+), 2 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 9958d59c..0a3c0b13 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -120,6 +120,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_overview(0), m_showSingingPitch(nullptr), m_showSingingNotes(nullptr), + m_playSingingAudio(nullptr), + m_playRefWhileRecording(nullptr), m_loadSingingTrackAction(nullptr), m_mainMenusCreated(false), m_playbackMenu(0), @@ -1393,6 +1395,55 @@ MainWindow::setupToolbars() connect(this, SIGNAL(canShowRealtimePitch(bool)), m_showSingingNotes, SLOT(setEnabled(bool))); m_showSingingNotes->setEnabled(false); + // Singing track audio playback toggle + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(10)); + toolbar->addWidget(spacer); + + m_playSingingAudio = toolbar->addAction(il.load("speaker"), tr("Play Singing Audio")); + m_playSingingAudio->setCheckable(true); + m_playSingingAudio->setChecked(true); + m_playSingingAudio->setToolTip(tr("Enable/disable playback of the recorded singing audio")); + connect(m_playSingingAudio, SIGNAL(triggered()), this, SLOT(playSingingAudioToggled())); + connect(this, SIGNAL(canShowRealtimePitch(bool)), m_playSingingAudio, SLOT(setEnabled(bool))); + m_playSingingAudio->setEnabled(false); + + // Play reference track while recording — lets the singer hear the + // reference audio through headphones to time their performance. + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); + toolbar->addWidget(spacer); + + { + QLabel *recLabel = new QLabel(tr("While recording:")); + QFont f = recLabel->font(); + f.setPointSize(f.pointSize() - 1); + recLabel->setFont(f); + recLabel->setEnabled(false); + toolbar->addWidget(recLabel); + } + + m_playRefWhileRecording = toolbar->addAction(il.load("speaker"), + tr("Play Reference While Recording")); + m_playRefWhileRecording->setCheckable(true); + { + QSettings settings; + settings.beginGroup("MainWindow"); + m_playRefWhileRecording->setChecked( + settings.value("playrefwhilerecording", false).toBool()); + settings.endGroup(); + } + m_playRefWhileRecording->setToolTip( + tr("Play the reference track through speakers/headphones during recording " + "so you can time your singing against it")); + connect(m_playRefWhileRecording, &QAction::toggled, this, [this](bool on) { + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", on); + settings.endGroup(); + }); + connect(this, SIGNAL(canPlay(bool)), m_playRefWhileRecording, SLOT(setEnabled(bool))); + // Spectrogram spacer = new QLabel; spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); @@ -1716,6 +1767,15 @@ MainWindow::playNotesToggled() updateLayerStatuses(); } +void +MainWindow::playSingingAudioToggled() +{ + if (m_analyser2) { + m_analyser2->toggleAudible(Analyser::Audio); + } + updateLayerStatuses(); +} + void MainWindow::updateLayerStatuses() { @@ -1763,6 +1823,15 @@ MainWindow::updateLayerStatuses() m_showSingingNotes->setChecked(false); } } + + if (m_playSingingAudio) { + m_playSingingAudio->setEnabled(m_analyser2 != nullptr); + if (m_analyser2) { + m_playSingingAudio->setChecked(m_analyser2->isAudible(Analyser::Audio)); + } else { + m_playSingingAudio->setChecked(true); // default on when track arrives + } + } } void @@ -2617,12 +2686,24 @@ MainWindow::recordingStarted() // (including emit audioFileLoaded() -> panes created). QTimer::singleShot(0, this, [this]() { if (!m_recordingInProgress) { - // Recording was stopped before we got a chance to set up — - // nothing to do. return; } cerr << "MainWindow::recordingStarted (deferred): setting up realtime pitch layer" << endl; setupRealtimePitchLayer(); + + // If the "play reference while recording" toggle is on, start + // playback from frame 0 so the singer hears the reference track. + // The audio IO was already resumed by record() so m_playSource + // can be started directly without calling MainWindowBase::play() + // (which would stop recording if isRecording() is true). + if (m_recordingAsSingingTrack && + m_playRefWhileRecording && m_playRefWhileRecording->isChecked() && + m_playSource && !m_playSource->isPlaying()) { + cerr << "MainWindow::recordingStarted: starting reference playback" << endl; + m_viewManager->setPlaybackFrame(0); + m_playSource->play(0); + } + updateLayerStatuses(); updateMenuStates(); }); @@ -2684,6 +2765,16 @@ MainWindow::recordingFinishedFull() m_recordingAsSingingTrack = false; m_currentRecordingModelId = {}; teardownRealtimePitchLayer(); + + // Stop reference playback that was started for the singer's benefit. + // Suspend the audio IO so it doesn't keep consuming CPU while idle. + if (m_playSource && m_playSource->isPlaying()) { + cerr << "MainWindow::recordingFinishedFull: stopping reference playback" << endl; + m_playSource->stop(); + if (m_audioIO) m_audioIO->suspend(); + else if (m_playTarget) m_playTarget->suspend(); + } + updateLayerStatuses(); updateMenuStates(); } diff --git a/main/MainWindow.h b/main/MainWindow.h index 52f63617..9d736b8a 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -105,6 +105,7 @@ protected slots: virtual void playAudioToggled(); virtual void playPitchToggled(); virtual void playNotesToggled(); + virtual void playSingingAudioToggled(); virtual void editDisplayExtents(); @@ -230,6 +231,8 @@ protected slots: // Actions/toolbar items for the singing track QAction *m_showSingingPitch; QAction *m_showSingingNotes; + QAction *m_playSingingAudio; + QAction *m_playRefWhileRecording; QAction *m_loadSingingTrackAction; sv::Fader *m_fader; sv::AudioDial *m_playSpeed; From 70f621f27cfab73fe5a73d1105890a13dc281fde Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 8 Mar 2026 17:24:22 +0200 Subject: [PATCH 007/275] Apply recording latency compensation for singing tracks Co-authored-by: Github Copilot copilot@github.com --- main/MainWindow.cpp | 36 +++++++++++++++++++++++++++++++++++- main/MainWindow.h | 7 +++++++ 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0a3c0b13..1511a154 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -140,7 +140,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_recordingInProgress(false), m_recordingAsSingingTrack(false), m_paneCountBeforeRecording(0), - m_currentRecordingModelId() + m_currentRecordingModelId(), + m_recordingLatencyFrames(0) { setWindowTitle(QApplication::applicationName()); @@ -2591,6 +2592,7 @@ MainWindow::record() m_recordingInProgress = false; m_recordingAsSingingTrack = true; + m_recordingLatencyFrames = 0; // reset; will be computed in recordingStarted() // Remember pane count so we can prune the extra pane that // MainWindowBase::record() creates via AddPaneCommand for the // recording's waveform layer. We want both tracks in pane 0. @@ -2700,6 +2702,21 @@ MainWindow::recordingStarted() m_playRefWhileRecording && m_playRefWhileRecording->isChecked() && m_playSource && !m_playSource->isPlaying()) { cerr << "MainWindow::recordingStarted: starting reference playback" << endl; + + // Measure round-trip hardware latency so we can compensate the + // singing recording's timeline after the take. + // output latency = time from play() call until audio exits the speaker + // input latency = time from sound entering the mic until it arrives here + // The singer's response to reference frame 0 arrives in the recording + // at approximately frame (outputLatency + inputLatency), so we will + // shift the model's start frame by -(outputLatency + inputLatency). + sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); + sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; + m_recordingLatencyFrames = outputLatency + inputLatency; + cerr << "MainWindow::recordingStarted: output latency=" << outputLatency + << " input latency=" << inputLatency + << " round-trip compensation=" << m_recordingLatencyFrames << " frames" << endl; + m_viewManager->setPlaybackFrame(0); m_playSource->play(0); } @@ -4071,6 +4088,23 @@ MainWindow::analyseNow() // We must NOT re-analyse the primary reference track here. if (m_recordingAsSingingTrack) { cerr << "analyseNow: recording was singing track — routing to m_analyser2" << endl; + + // Apply round-trip latency compensation: shift the singing model's + // global start frame backward by the round-trip hardware latency so + // the singer's audio (which arrives late due to output + input latency) + // aligns with the reference during playback. This must happen before + // pYIN analysis so that all derived layers (pitch, notes) inherit the + // same timeline offset. Only applied when reference playback was + // active during the recording (m_recordingLatencyFrames > 0). + if (m_recordingLatencyFrames > 0 && !m_currentRecordingModelId.isNone()) { + auto wfm = ModelById::getAs(m_currentRecordingModelId); + if (wfm) { + cerr << "analyseNow: applying latency compensation: setStartFrame(" + << -m_recordingLatencyFrames << ")" << endl; + wfm->setStartFrame(-m_recordingLatencyFrames); + } + } + if (m_analyser2) { CommandHistory::getInstance()->startCompoundOperation (tr("Analyse Singing Track"), true); diff --git a/main/MainWindow.h b/main/MainWindow.h index 9d736b8a..dcb5ed7b 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -348,6 +348,13 @@ protected slots: // freeing it. sv::ModelId m_currentRecordingModelId; + // Round-trip hardware latency (output + input, in frames at the model + // sample rate) stored when a singing-track recording is made with the + // "play reference while recording" toggle on. Applied as a negative + // start-frame offset to the singing model so its timeline aligns with + // the reference during playback. Reset to 0 at the start of each recording. + sv::sv_frame_t m_recordingLatencyFrames; + // Extra panes created by MainWindowBase::record() via AddPaneCommand // that we want to hide immediately but cannot delete yet because // m_analyser2 hasn't been set up yet (it is deferred via QTimer::singleShot). From 4d42858be0f8528871ed2fb0f24947403952fb3c Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 8 Mar 2026 20:27:30 +0200 Subject: [PATCH 008/275] Add background music playback alongside reference track Co-authored-by: Github Copilot copilot@github.com --- main/MainWindow.cpp | 243 ++++++++++++++++++++++++++++++++++++++++++++ main/MainWindow.h | 22 ++++ 2 files changed, 265 insertions(+) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 1511a154..bcafee36 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -123,6 +123,12 @@ MainWindow::MainWindow(AudioMode audioMode, m_playSingingAudio(nullptr), m_playRefWhileRecording(nullptr), m_loadSingingTrackAction(nullptr), + m_backgroundMusicModelId(), + m_backgroundMusicLayer(nullptr), + m_loadBackgroundMusicAction(nullptr), + m_playBackgroundMusic(nullptr), + m_bgMusicLPW(nullptr), + m_loadingBackgroundMusic(false), m_mainMenusCreated(false), m_playbackMenu(0), m_recentFilesMenu(0), @@ -279,6 +285,12 @@ MainWindow::MainWindow(AudioMode audioMode, connect(m_audioLPW, SIGNAL(levelChanged(float)), this, SLOT(audioGainChanged(float))); connect(m_audioLPW, SIGNAL(panChanged(float)), this, SLOT(audioPanChanged(float))); + m_bgMusicLPW = new LevelPanToolButton(frame); + m_bgMusicLPW->setIncludeMute(false); + m_bgMusicLPW->setObjectName(tr("Background Music Level and Pan")); + connect(m_bgMusicLPW, SIGNAL(levelChanged(float)), this, SLOT(backgroundMusicGainChanged(float))); + connect(m_bgMusicLPW, SIGNAL(panChanged(float)), this, SLOT(backgroundMusicPanChanged(float))); + if (m_withSonification) { m_pitchLPW = new LevelPanToolButton(frame); @@ -489,6 +501,12 @@ MainWindow::setupFileMenu() m_keyReference->registerShortcut(m_loadSingingTrackAction); menu->addAction(m_loadSingingTrackAction); + m_loadBackgroundMusicAction = new QAction(il.load("fileopen"), tr("Load &Background Music..."), this); + m_loadBackgroundMusicAction->setStatusTip(tr("Load an audio file to play as background music alongside the reference track (not analysed)")); + connect(m_loadBackgroundMusicAction, SIGNAL(triggered()), this, SLOT(openBackgroundMusic())); + connect(this, SIGNAL(canPlay(bool)), m_loadBackgroundMusicAction, SLOT(setEnabled(bool))); + menu->addAction(m_loadBackgroundMusicAction); + menu->addSeparator(); action = new QAction(tr("I&mport Pitch Track Data..."), this); @@ -1445,6 +1463,34 @@ MainWindow::setupToolbars() }); connect(this, SIGNAL(canPlay(bool)), m_playRefWhileRecording, SLOT(setEnabled(bool))); + // Background music section: an additional audio track that plays + // alongside the reference track but is never analysed. + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); + toolbar->addWidget(spacer); + + { + QLabel *bgLabel = new QLabel(tr("Background:")); + QFont f = bgLabel->font(); + f.setPointSize(f.pointSize() - 1); + bgLabel->setFont(f); + bgLabel->setEnabled(false); + toolbar->addWidget(bgLabel); + } + + m_playBackgroundMusic = toolbar->addAction(il.load("speaker"), tr("Mix Background Music")); + m_playBackgroundMusic->setCheckable(true); + m_playBackgroundMusic->setChecked(true); + m_playBackgroundMusic->setToolTip( + tr("Enable/disable mixing the background music track during playback and recording")); + m_playBackgroundMusic->setEnabled(false); + connect(m_playBackgroundMusic, SIGNAL(triggered()), this, SLOT(backgroundMusicToggled())); + + m_bgMusicLPW->setImageSize(lpwSize); + m_bgMusicLPW->setBigImageSize(bigLpwSize); + m_bgMusicLPW->setEnabled(false); + toolbar->addWidget(m_bgMusicLPW); + // Spectrogram spacer = new QLabel; spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); @@ -1833,6 +1879,25 @@ MainWindow::updateLayerStatuses() m_playSingingAudio->setChecked(true); // default on when track arrives } } + + // Background music toggle: enabled when a background music track is loaded + if (m_playBackgroundMusic) { + bool haveBgMusic = (m_backgroundMusicLayer != nullptr); + m_playBackgroundMusic->setEnabled(haveBgMusic); + if (m_bgMusicLPW) m_bgMusicLPW->setEnabled(haveBgMusic); + if (haveBgMusic) { + auto params = m_backgroundMusicLayer->getPlayParameters(); + bool audible = params ? params->isPlayAudible() : true; + m_playBackgroundMusic->setChecked(audible); + if (m_bgMusicLPW) { + m_bgMusicLPW->setEnabled(audible); + m_bgMusicLPW->setLevel(params ? params->getPlayGain() : 1.f); + m_bgMusicLPW->setPan(params ? params->getPlayPan() : 0.f); + } + } else { + m_playBackgroundMusic->setChecked(true); // default on when track arrives + } + } } void @@ -1932,6 +1997,7 @@ MainWindow::closeSession() // are destroyed, so they can cleanly remove their layers from the pane. teardownRealtimePitchLayer(); teardownSingingTrackAnalyser(); + teardownBackgroundMusic(); m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; @@ -2097,6 +2163,174 @@ MainWindow::analyseNewSingingModel() setupSingingTrackAnalyser(singingModelId); } +void +MainWindow::openBackgroundMusic() +{ + QString path = getOpenFileName(FileFinder::AudioFile); + if (path.isEmpty()) return; + loadBackgroundMusic(path); +} + +void +MainWindow::loadBackgroundMusic(QString path) +{ + if (!m_document) { + QMessageBox::warning(this, tr("No session"), + tr("No session open

Please open a reference audio file first.")); + return; + } + + emit activity(tr("Load background music \"%1\"").arg(path)); + + // Tear down any previously loaded background music track. + teardownBackgroundMusic(); + + // Record the pane count before opening so we can remove the extra pane + // that openPath(CreateAdditionalModel) creates via AddPaneCommand. + // The background music doesn't need its own pane. + int paneCountBefore = m_paneStack ? m_paneStack->getPaneCount() : 0; + + // Set the flag so modelAdded() captures the new model ID and skips + // singing-track analysis for this model. + m_loadingBackgroundMusic = true; + FileOpenStatus status = openPath(path, CreateAdditionalModel); + m_loadingBackgroundMusic = false; + + if (status == FileOpenFailed) { + m_backgroundMusicModelId = {}; + QMessageBox::critical(this, tr("Failed to open background music"), + tr("File open failed

File \"%1\" could not be opened").arg(path)); + return; + } else if (status == FileOpenWrongMode) { + m_backgroundMusicModelId = {}; + QMessageBox::critical(this, tr("Failed to open background music"), + tr("Audio required

Could not open \"%1\" as audio").arg(path)); + return; + } + + if (m_backgroundMusicModelId.isNone()) { + cerr << "loadBackgroundMusic: modelAdded did not capture a model ID — aborting" << endl; + return; + } + + // Create the WaveformLayer for the background music BEFORE removing the + // orphan layers from the extra pane. This ensures the background music + // model has at least one layer referencing it when the orphan is deleted, + // so Document::releaseModel() does not free it prematurely. + // + // Use createLayer() (not createEmptyLayer()) — the same path that + // Analyser::addWaveform() uses for the secondary analyser — then + // immediately rebind it to the background music model with setModel(). + if (m_paneStack && m_paneStack->getPaneCount() > 0) { + Pane *pane = m_paneStack->getPane(0); + if (pane && m_document) { + Layer *rawLayer = m_document->createLayer(LayerFactory::Waveform); + m_backgroundMusicLayer = qobject_cast(rawLayer); + if (m_backgroundMusicLayer) { + m_document->setModel(m_backgroundMusicLayer, m_backgroundMusicModelId); + ColourDatabase *cdb = ColourDatabase::getInstance(); + m_backgroundMusicLayer->setBaseColour( + cdb->getColourIndex(tr("Green"))); + m_document->addLayerToView(pane, m_backgroundMusicLayer); + + // The waveform is only needed to register the model with + // the play source — we don't want it rendered on screen. + m_backgroundMusicLayer->showLayer(pane, false); + + // Set initial audibility from the toggle state. + auto params = m_backgroundMusicLayer->getPlayParameters(); + if (params) { + bool wantAudible = !m_playBackgroundMusic || + m_playBackgroundMusic->isChecked(); + params->setPlayAudible(wantAudible); + } + + cerr << "loadBackgroundMusic: waveform layer added for model " + << m_backgroundMusicModelId << endl; + } else { + cerr << "loadBackgroundMusic: failed to create WaveformLayer — aborting" << endl; + return; + } + } + } + + // Remove the extra pane that openPath(CreateAdditionalModel) created. + // Now safe to delete the orphan WaveformLayer: our new layer already + // holds a reference to the model so Document::releaseModel() will not + // free it when the orphan is deleted. + if (m_paneStack) { + while (m_paneStack->getPaneCount() > paneCountBefore) { + Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); + if (m_document && extra) { + while (extra->getLayerCount() > 0) { + Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); + m_document->deleteLayer(orphan, true); + } + } + if (m_overview) m_overview->unregisterView(extra); + m_paneStack->deletePane(extra); + } + } + + updateLayerStatuses(); + updateMenuStates(); +} + +void +MainWindow::teardownBackgroundMusic() +{ + if (m_backgroundMusicLayer) { + // Explicitly remove from play source before deleting the layer, so + // the model is removed from the mix even if layerInAView(false) is not + // triggered through the normal path. + if (m_playSource && !m_backgroundMusicModelId.isNone()) { + m_playSource->removeModel(m_backgroundMusicModelId); + } + if (m_document) { + m_document->deleteLayer(m_backgroundMusicLayer, true); + } + m_backgroundMusicLayer = nullptr; + } + m_backgroundMusicModelId = {}; +} + +void +MainWindow::backgroundMusicToggled() +{ + if (!m_backgroundMusicLayer) return; + auto params = m_backgroundMusicLayer->getPlayParameters(); + if (!params) return; + bool wantAudible = m_playBackgroundMusic && m_playBackgroundMusic->isChecked(); + params->setPlayAudible(wantAudible); + if (m_bgMusicLPW) m_bgMusicLPW->setEnabled(wantAudible); + cerr << "backgroundMusicToggled: background music " + << (wantAudible ? "unmuted" : "muted") << endl; +} + +void +MainWindow::backgroundMusicGainChanged(float gain) +{ + if (!m_backgroundMusicLayer) return; + auto params = m_backgroundMusicLayer->getPlayParameters(); + if (!params) return; + if (gain == 0.f) { + params->setPlayAudible(false); + if (m_playBackgroundMusic) m_playBackgroundMusic->setChecked(false); + } else { + params->setPlayAudible(true); + if (m_playBackgroundMusic) m_playBackgroundMusic->setChecked(true); + params->setPlayGain(gain); + } +} + +void +MainWindow::backgroundMusicPanChanged(float pan) +{ + if (!m_backgroundMusicLayer) return; + auto params = m_backgroundMusicLayer->getPlayParameters(); + if (params) params->setPlayPan(pan); +} + void MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { @@ -4010,6 +4244,14 @@ MainWindow::modelAdded(ModelId model) auto dtvm = ModelById::getAs(model); if (dtvm) { cerr << "A dense time-value model (such as an audio file) has been loaded" << endl; + + // If we're loading background music, capture the model ID and return — + // do NOT treat it as a singing track or queue any secondary analysis. + if (m_loadingBackgroundMusic) { + m_backgroundMusicModelId = model; + return; + } + // If there is already a main model and this is a new additional // audio model (not the realtime pitch model), treat it as the // singing track to be analysed with the secondary colour scheme. @@ -4274,6 +4516,7 @@ MainWindow::analyseNewMainModel() for (ModelId mid : m_document->getModels()) { if (mid == mainId) continue; if (mid == m_realtimePitchModelId) continue; + if (mid == m_backgroundMusicModelId) continue; if (ModelById::isa(mid)) { foundSinging = mid; break; diff --git a/main/MainWindow.h b/main/MainWindow.h index dcb5ed7b..6a3b7756 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -69,6 +69,7 @@ protected slots: protected slots: virtual void openFile(); virtual void openSingingTrack(); + virtual void openBackgroundMusic(); virtual void analyseNewSingingModel(); virtual void openLocation(); virtual void openRecentFile(); @@ -106,6 +107,9 @@ protected slots: virtual void playPitchToggled(); virtual void playNotesToggled(); virtual void playSingingAudioToggled(); + virtual void backgroundMusicToggled(); + virtual void backgroundMusicGainChanged(float gain); + virtual void backgroundMusicPanChanged(float pan); virtual void editDisplayExtents(); @@ -234,6 +238,19 @@ protected slots: QAction *m_playSingingAudio; QAction *m_playRefWhileRecording; QAction *m_loadSingingTrackAction; + + // Background music track: an additional audio file that plays alongside + // the reference track but is never analysed. The toggle enables/disables + // mixing during both normal playback and recording. + sv::ModelId m_backgroundMusicModelId; + sv::WaveformLayer *m_backgroundMusicLayer; + QAction *m_loadBackgroundMusicAction; + QAction *m_playBackgroundMusic; + sv::LevelPanToolButton *m_bgMusicLPW; + // True while loadBackgroundMusic() is calling openPath() so that + // modelAdded() can capture the model ID without treating it as a singing + // track or queuing a secondary analysis. + bool m_loadingBackgroundMusic; sv::Fader *m_fader; sv::AudioDial *m_playSpeed; QPushButton *m_playSharpen; @@ -305,6 +322,11 @@ protected slots: virtual void setupRealtimePitchLayer(); virtual void teardownRealtimePitchLayer(); + // Background music helpers: load/tear-down a non-analysed audio track + // that plays alongside the reference track. + void loadBackgroundMusic(QString path); + void teardownBackgroundMusic(); + // Drain m_pendingExtraPanes: delete orphan layers and the pane widgets // that were deferred from record()'s pane-cleanup step. Must be called // after m_analyser2 has created its WaveformLayer (so deleteLayer won't From 442bdd5b48f867c3efa33dbea11d3fafb9b234f5 Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 29 May 2026 19:46:03 +0300 Subject: [PATCH 009/275] feat: Implement FFT-based YIN difference function for faster real-time pitch tracking Co-Authored-By: Claude Sonnet 4.6 --- main/RealtimePitchTracker.cpp | 141 ++++++++++++++++++++++------------ main/RealtimePitchTracker.h | 32 ++++---- 2 files changed, 107 insertions(+), 66 deletions(-) diff --git a/main/RealtimePitchTracker.cpp b/main/RealtimePitchTracker.cpp index f718bcd8..f4fc99f7 100644 --- a/main/RealtimePitchTracker.cpp +++ b/main/RealtimePitchTracker.cpp @@ -19,11 +19,14 @@ #include "data/model/Model.h" #include "base/Event.h" +#include "bqfft/FFT.h" + #include #include #include using namespace sv; +using namespace breakfastquay; using std::cerr; using std::endl; using std::vector; @@ -39,12 +42,12 @@ RealtimePitchTracker::RealtimePitchTracker(ModelId audioSourceId, m_threshold(0.15), m_running(false), m_nextFrameToProcess(0), - m_timer(new QTimer(this)) + m_timer(new QTimer(this)), + m_fft(nullptr) { - // Poll for new audio every ~50 ms on the GUI thread. - // At 44100 Hz this gives us roughly 2205 new samples per tick, - // enough for 4+ YIN hops (kHopSize = 512). - m_timer->setInterval(50); + // Poll for new audio every ~20 ms. Smaller interval reduces the lag + // between the singer producing a note and the dot appearing on screen. + m_timer->setInterval(20); connect(m_timer, &QTimer::timeout, this, &RealtimePitchTracker::pollAndProcess); } @@ -52,6 +55,7 @@ RealtimePitchTracker::RealtimePitchTracker(ModelId audioSourceId, RealtimePitchTracker::~RealtimePitchTracker() { stop(); + delete m_fft; } void @@ -59,6 +63,9 @@ RealtimePitchTracker::start() { m_nextFrameToProcess = 0; m_running = true; + // Reset FFT so it is recreated fresh for the new recording's sample rate. + delete m_fft; + m_fft = nullptr; m_timer->start(); cerr << "RealtimePitchTracker: started" << endl; } @@ -96,6 +103,10 @@ RealtimePitchTracker::pollAndProcess() double sr = audioModel->getSampleRate(); + // Lag range: larger lag → lower frequency + int minLag = std::max(1, (int)std::floor(sr / m_maxFreq)); + int maxLag = std::min(kWindowSize / 2 - 2, (int)std::ceil(sr / m_minFreq)); + // Process as many complete windows as are available, starting // from where we left off last time. sv_frame_t pos = m_nextFrameToProcess; @@ -113,7 +124,18 @@ RealtimePitchTracker::pollAndProcess() std::vector raw(rawFv.begin(), rawFv.end()); - double hz = yinPitch(raw, sr, m_minFreq, m_maxFreq, m_threshold); + // --- FFT-based YIN --- + vector diff; + yinDifferenceFFT(raw, diff); + yinCMND(diff); + double lagSamples = yinFindPitch(diff, minLag, maxLag, m_threshold); + + double hz = 0.0; + if (lagSamples > 0.0) { + double candidate = sr / lagSamples; + if (candidate >= m_minFreq && candidate <= m_maxFreq) + hz = candidate; + } // Frame position of the centre of the analysis window sv_frame_t centreFrame = pos + kWindowSize / 2; @@ -131,31 +153,78 @@ RealtimePitchTracker::pollAndProcess() } // --------------------------------------------------------------------------- -// YIN algorithm +// YIN algorithm — FFT-accelerated difference function // Reference: de Cheveigné & Kawahara, "YIN, a fundamental frequency // estimator for speech and music", JASA 111(4), 2002. -// We implement Steps 1–5 (difference function, cumulative mean -// normalised difference, absolute threshold + parabolic interpolation). +// The difference function (Step 2) uses FFT-based autocorrelation +// (O(n log n)) instead of the direct sum (O(n²)). +// Steps 3–5 (CMND, absolute threshold, parabolic interpolation) are +// the same as the original formulation. // --------------------------------------------------------------------------- void -RealtimePitchTracker::yinDifference(const vector &buf, - vector &diff) +RealtimePitchTracker::yinDifferenceFFT(const vector &buf, + vector &diff) { - int windowSize = (int)buf.size(); - int halfSize = windowSize / 2; - diff.assign(halfSize, 0.0); + // buf has size kWindowSize (= 2 * halfSize). + // YIN treats the first halfSize samples as the "signal" and uses + // lags 0..halfSize-1, requiring access up to buf[halfSize + tau]. + int frameSize = (int)buf.size(); // kWindowSize + int halfSize = frameSize / 2; // yinBufferSize + int fftBins = halfSize + 1; // complex bins from real FFT of frameSize + + // Lazy-create FFT (reused across hops — same size every call). + if (!m_fft) { + m_fft = new FFT(frameSize); + } - // d[0] is defined as 0 - diff[0] = 0.0; + // --- Power terms (iterative, O(n)) --- + // powerTerms[tau] = sum_{j=tau}^{halfSize+tau-1} buf[j]^2 + vector powerTerms(halfSize); + powerTerms[0] = 0.0; + for (int j = 0; j < halfSize; ++j) + powerTerms[0] += double(buf[j]) * double(buf[j]); + for (int tau = 1; tau < halfSize; ++tau) { + powerTerms[tau] = powerTerms[tau-1] + - double(buf[tau-1]) * double(buf[tau-1]) + + double(buf[tau + halfSize]) * double(buf[tau + halfSize]); + } + + // --- Forward FFT of the full input --- + vector audioReal(fftBins), audioImag(fftBins); + m_fft->forward(buf.data(), audioReal.data(), audioImag.data()); + + // --- Kernel: reversed first half, zero-padded to frameSize --- + // Convolving x[0..frameSize-1] with this kernel gives the + // YIN-style autocorrelation via the overlap at lag+halfSize-1. + vector kernel(frameSize, 0.0f); + for (int j = 0; j < halfSize; ++j) + kernel[j] = buf[halfSize - 1 - j]; + vector kernelReal(fftBins), kernelImag(fftBins); + m_fft->forward(kernel.data(), kernelReal.data(), kernelImag.data()); + + // --- Complex multiply in frequency domain --- + vector acfReal(fftBins), acfImag(fftBins); + for (int j = 0; j < fftBins; ++j) { + acfReal[j] = audioReal[j]*kernelReal[j] - audioImag[j]*kernelImag[j]; + acfImag[j] = audioReal[j]*kernelImag[j] + audioImag[j]*kernelReal[j]; + } + + // --- Inverse FFT → time-domain autocorrelation --- + vector acfOut(frameSize); + m_fft->inverse(acfReal.data(), acfImag.data(), acfOut.data()); + + // bqfft inverse is unnormalized (unlike vamp FFT which divides by n). + const double scale = 1.0 / frameSize; + // --- Compute difference function --- + // d[tau] = powerTerms[0] + powerTerms[tau] - 2*r[tau] + // r[tau] lives at acfOut[tau + halfSize - 1] after the convolution. + diff.assign(halfSize, 0.0); + diff[0] = 0.0; for (int tau = 1; tau < halfSize; ++tau) { - double sum = 0.0; - for (int j = 0; j < halfSize; ++j) { - double delta = double(buf[j]) - double(buf[j + tau]); - sum += delta * delta; - } - diff[tau] = sum; + diff[tau] = powerTerms[0] + powerTerms[tau] + - 2.0 * double(acfOut[tau + halfSize - 1]) * scale; } } @@ -222,31 +291,3 @@ RealtimePitchTracker::yinFindPitch(const vector &cmnd, return refined; } -double -RealtimePitchTracker::yinPitch(const vector &window, - double sr, - double minFreq, double maxFreq, - double thresh) -{ - int halfSize = (int)window.size() / 2; - - // Lag range: larger lag → lower frequency - int minLag = std::max(1, (int)std::floor(sr / maxFreq)); - int maxLag = std::min(halfSize - 2, (int)std::ceil(sr / minFreq)); - - if (minLag >= maxLag) return 0.0; - - vector diff; - yinDifference(window, diff); - yinCMND(diff); - - double lagSamples = yinFindPitch(diff, minLag, maxLag, thresh); - if (lagSamples <= 0.0) return 0.0; - - double hz = sr / lagSamples; - - // Final range check (parabolic interpolation can push slightly out) - if (hz < minFreq || hz > maxFreq) return 0.0; - - return hz; -} \ No newline at end of file diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h index 9618ff81..dd251b5c 100644 --- a/main/RealtimePitchTracker.h +++ b/main/RealtimePitchTracker.h @@ -28,6 +28,10 @@ namespace sv { class WritableWaveFileModel; } +namespace breakfastquay { +class FFT; +} + /** * RealtimePitchTracker polls a WritableWaveFileModel (the model being * filled during a live microphone recording) for new audio samples and @@ -139,31 +143,27 @@ private slots: sv::sv_frame_t m_nextFrameToProcess; // --- Processing parameters --- - // Window size and hop size in samples. - // At 44 100 Hz: window ≈ 46 ms, hop ≈ 12 ms. + // Window size: 2048 samples @ 44100 Hz ≈ 46 ms (two full periods of 60 Hz). + // Hop size: 256 samples ≈ 5.8 ms — finer dot density than the old 512. static const int kWindowSize = 2048; - static const int kHopSize = 512; + static const int kHopSize = 256; // --- Timer --- QTimer *m_timer; - // --- YIN helpers --- + // --- FFT for fast YIN difference function --- + breakfastquay::FFT *m_fft; // lazy-created on first poll - /** - * Run the YIN algorithm on a single window of audio and return - * the estimated fundamental frequency in Hz, or 0 if unvoiced. - */ - static double yinPitch(const std::vector &window, - double sr, - double minFreq, double maxFreq, - double thresh); + // --- YIN helpers --- /** - * Step 2: difference function. - * d[tau] = sum_{j=0}^{W/2-1} (x[j] - x[j+tau])^2 + * FFT-based difference function (O(n log n) vs the naive O(n²)). + * Computes d[tau] = sum_{j=0}^{halfSize-1} (x[j] - x[j+tau])^2 + * using the autocorrelation identity and bqfft. + * buf must have size kWindowSize; diff is sized to kWindowSize/2. */ - static void yinDifference(const std::vector &buf, - std::vector &diff); + void yinDifferenceFFT(const std::vector &buf, + std::vector &diff); /** * Step 3: cumulative mean normalised difference (in-place). From d6ac0cc45155a39170b9050535f22614510e020c Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 29 May 2026 20:07:40 +0300 Subject: [PATCH 010/275] fix: Update pitch model scale units to Hz Co-Authored-By: Claude Sonnet 4.6 --- main/MainWindow.cpp | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index bcafee36..86c2f75a 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2612,9 +2612,12 @@ MainWindow::setupRealtimePitchLayer() } // Create a SparseTimeValueModel to receive pitch estimates. - // Resolution 512 frames matches the YIN hop size in RealtimePitchTracker. + // Resolution 256 frames matches the YIN hop size in RealtimePitchTracker. + // Unit "Hz" is required so TimeValueLayer::shouldAutoAlign() defers to + // the pane's log-frequency coordinate system (same as the pYIN pitch track). auto pitchModel = std::make_shared(sr, 512, false); pitchModel->setObjectName(tr("Realtime Pitch (Live)")); + pitchModel->setScaleUnits("Hz"); m_realtimePitchModelId = ModelById::add(pitchModel); m_document->addNonDerivedModel(m_realtimePitchModelId); From 36d0ce73e1237ddb9ea20b5247412a26813afa8c Mon Sep 17 00:00:00 2001 From: jhhr Date: Mon, 1 Jun 2026 22:02:40 +0300 Subject: [PATCH 011/275] feat: real-time pitch tracking and mute secondary analyser pitch/note layers - Add RealtimePitchTracker for live pitch display during recording - Wire realtime pitch layer setup/teardown into MainWindow recording flow - Fix secondary analyser pitch+note tracks playing audibly after pYIN completes: mute them directly via PlayParameters (not setAudible, which would corrupt the shared QSettings key used by the primary analyser) Co-Authored-By: Claude Sonnet 4.6 --- main/Analyser.cpp | 14 +++ main/MainWindow.cpp | 17 ++- main/RealtimePitchTracker.cpp | 189 ++++++++++++---------------------- main/RealtimePitchTracker.h | 137 ++++++++---------------- 4 files changed, 129 insertions(+), 228 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 70bb895e..2b5f5d0f 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -161,6 +161,20 @@ Analyser::doAllAnalyses(bool withPitchTrack) loadState(Notes); loadState(Spectrogram); + // The secondary analyser's pitch and note tracks are visual-only (there is + // no UI toggle to control their audibility, and sonifying two pitch/note + // tracks at once is confusing). Mute them directly — do NOT call + // setAudible(), which would also call saveState() and corrupt the primary + // analyser's shared settings key. + if (m_colorScheme == SecondaryColors) { + for (Component c : { PitchTrack, Notes }) { + if (m_layers[c]) { + auto params = m_layers[c]->getPlayParameters(); + if (params) params->setPlayAudible(false); + } + } + } + stackLayers(); emit layersChanged(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 86c2f75a..41789a57 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2654,7 +2654,7 @@ MainWindow::setupRealtimePitchLayer() // It will poll audioSourceId (the WritableWaveFileModel) for new frames // on each QTimer tick and write estimates into m_realtimePitchModelId. m_realtimePitchTracker = new RealtimePitchTracker( - audioSourceId, m_realtimePitchModelId, this); + audioSourceId, this); connect(m_realtimePitchTracker, &RealtimePitchTracker::pitchDetected, this, &MainWindow::onRealtimePitchDetected); m_realtimePitchTracker->start(); @@ -2964,16 +2964,13 @@ MainWindow::recordingStarted() } void -MainWindow::onRealtimePitchDetected(sv::sv_frame_t /*frame*/, double hz) +MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) { - // The pitch has already been written into the SparseTimeValueModel by - // RealtimePitchTracker; the view repaints automatically via dataChanged(). - // Here we show a human-readable pitch in the status bar so the singer - // gets immediate textual feedback during recording. - - if (hz <= 0.0) { - getStatusLabel()->setText(tr("Recording — pitch: (unvoiced)")); - return; + // Called on the GUI thread via Qt::QueuedConnection (RealtimePitchTracker + // emits from its background thread). Write the point into the model here + // so all model mutations stay on the GUI thread. + if (auto m = ModelById::getAs(m_realtimePitchModelId)) { + m->add(Event(frame, float(hz), tr(""))); } // Convert Hz to MIDI note number and cents deviation. diff --git a/main/RealtimePitchTracker.cpp b/main/RealtimePitchTracker.cpp index f4fc99f7..0254e298 100644 --- a/main/RealtimePitchTracker.cpp +++ b/main/RealtimePitchTracker.cpp @@ -14,10 +14,8 @@ #include "RealtimePitchTracker.h" -#include "data/model/SparseTimeValueModel.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/Model.h" -#include "base/Event.h" #include "bqfft/FFT.h" @@ -32,154 +30,118 @@ using std::endl; using std::vector; RealtimePitchTracker::RealtimePitchTracker(ModelId audioSourceId, - ModelId pitchModelId, QObject *parent) - : QObject(parent), + : QThread(parent), m_audioSourceId(audioSourceId), - m_pitchModelId(pitchModelId), m_minFreq(60.0), m_maxFreq(1000.0), - m_threshold(0.15), - m_running(false), - m_nextFrameToProcess(0), - m_timer(new QTimer(this)), - m_fft(nullptr) + m_threshold(0.15) { - // Poll for new audio every ~20 ms. Smaller interval reduces the lag - // between the singer producing a note and the dot appearing on screen. - m_timer->setInterval(20); - connect(m_timer, &QTimer::timeout, - this, &RealtimePitchTracker::pollAndProcess); } RealtimePitchTracker::~RealtimePitchTracker() { stop(); - delete m_fft; } void RealtimePitchTracker::start() { - m_nextFrameToProcess = 0; - m_running = true; - // Reset FFT so it is recreated fresh for the new recording's sample rate. - delete m_fft; - m_fft = nullptr; - m_timer->start(); - cerr << "RealtimePitchTracker: started" << endl; + QThread::start(); } void RealtimePitchTracker::stop() { - m_timer->stop(); - m_running = false; - cerr << "RealtimePitchTracker: stopped (processed up to frame " - << m_nextFrameToProcess << ")" << endl; + requestInterruption(); + wait(); } void -RealtimePitchTracker::pollAndProcess() +RealtimePitchTracker::run() { - if (!m_running) return; + FFT *fft = nullptr; + sv_frame_t nextFrameToProcess = 0; - // --- Get the audio source model --- - auto audioModel = ModelById::getAs(m_audioSourceId); - if (!audioModel) { - // The model may not be ready yet on the very first tick — that's fine. - return; - } + cerr << "RealtimePitchTracker: background thread started" << endl; - sv_frame_t totalFrames = audioModel->getFrameCount(); - if (totalFrames < kWindowSize) { - // Not enough data yet - return; - } + while (!isInterruptionRequested()) { - // --- Get the pitch output model --- - auto pitchModel = ModelById::getAs(m_pitchModelId); - if (!pitchModel) return; + auto audioModel = ModelById::getAs(m_audioSourceId); + if (!audioModel) { + msleep(5); + continue; + } - double sr = audioModel->getSampleRate(); + sv_frame_t totalFrames = audioModel->getFrameCount(); + if (totalFrames < kWindowSize) { + msleep(5); + continue; + } - // Lag range: larger lag → lower frequency - int minLag = std::max(1, (int)std::floor(sr / m_maxFreq)); - int maxLag = std::min(kWindowSize / 2 - 2, (int)std::ceil(sr / m_minFreq)); + double sr = audioModel->getSampleRate(); - // Process as many complete windows as are available, starting - // from where we left off last time. - sv_frame_t pos = m_nextFrameToProcess; + if (!fft) { + fft = new FFT(kWindowSize); + } - while (pos + kWindowSize <= totalFrames) { + int minLag = std::max(1, (int)std::floor(sr / m_maxFreq)); + int maxLag = std::min(kWindowSize / 2 - 2, (int)std::ceil(sr / m_minFreq)); - // Fetch one window of audio from channel 0. - // getData returns floatvec_t (std::vector with a custom allocator), - // so we copy into a plain std::vector for the YIN functions. - floatvec_t rawFv = audioModel->getData(0, pos, kWindowSize); + bool processedAny = false; - // getData may return fewer samples if the model hasn't flushed - // the very last portion yet — skip rather than analyse garbage. - if ((int)rawFv.size() < kWindowSize) break; + while (nextFrameToProcess + kWindowSize <= totalFrames) { - std::vector raw(rawFv.begin(), rawFv.end()); + floatvec_t rawFv = audioModel->getData(0, nextFrameToProcess, kWindowSize); - // --- FFT-based YIN --- - vector diff; - yinDifferenceFFT(raw, diff); - yinCMND(diff); - double lagSamples = yinFindPitch(diff, minLag, maxLag, m_threshold); + if ((int)rawFv.size() < kWindowSize) break; - double hz = 0.0; - if (lagSamples > 0.0) { - double candidate = sr / lagSamples; - if (candidate >= m_minFreq && candidate <= m_maxFreq) - hz = candidate; - } + vector raw(rawFv.begin(), rawFv.end()); - // Frame position of the centre of the analysis window - sv_frame_t centreFrame = pos + kWindowSize / 2; + vector diff; + yinDifferenceFFT(raw, diff, fft); + yinCMND(diff); + double lagSamples = yinFindPitch(diff, minLag, maxLag, m_threshold); - if (hz > 0.0) { - Event e(centreFrame, float(hz), tr("")); - pitchModel->add(e); - emit pitchDetected(centreFrame, hz); + if (lagSamples > 0.0) { + double hz = sr / lagSamples; + if (hz >= m_minFreq && hz <= m_maxFreq) { + sv_frame_t centreFrame = nextFrameToProcess + kWindowSize / 2; + emit pitchDetected(centreFrame, hz); + processedAny = true; + } + } + + nextFrameToProcess += kHopSize; } - pos += kHopSize; + // Sleep only when no new hops were available; otherwise spin + // immediately to drain any remaining frames. + if (!processedAny) { + msleep(5); + } } - m_nextFrameToProcess = pos; + delete fft; + cerr << "RealtimePitchTracker: background thread stopped" << endl; } // --------------------------------------------------------------------------- // YIN algorithm — FFT-accelerated difference function // Reference: de Cheveigné & Kawahara, "YIN, a fundamental frequency // estimator for speech and music", JASA 111(4), 2002. -// The difference function (Step 2) uses FFT-based autocorrelation -// (O(n log n)) instead of the direct sum (O(n²)). -// Steps 3–5 (CMND, absolute threshold, parabolic interpolation) are -// the same as the original formulation. // --------------------------------------------------------------------------- void RealtimePitchTracker::yinDifferenceFFT(const vector &buf, - vector &diff) + vector &diff, + FFT *fft) { - // buf has size kWindowSize (= 2 * halfSize). - // YIN treats the first halfSize samples as the "signal" and uses - // lags 0..halfSize-1, requiring access up to buf[halfSize + tau]. - int frameSize = (int)buf.size(); // kWindowSize - int halfSize = frameSize / 2; // yinBufferSize - int fftBins = halfSize + 1; // complex bins from real FFT of frameSize - - // Lazy-create FFT (reused across hops — same size every call). - if (!m_fft) { - m_fft = new FFT(frameSize); - } + int frameSize = (int)buf.size(); + int halfSize = frameSize / 2; + int fftBins = halfSize + 1; // --- Power terms (iterative, O(n)) --- - // powerTerms[tau] = sum_{j=tau}^{halfSize+tau-1} buf[j]^2 vector powerTerms(halfSize); powerTerms[0] = 0.0; for (int j = 0; j < halfSize; ++j) @@ -192,36 +154,30 @@ RealtimePitchTracker::yinDifferenceFFT(const vector &buf, // --- Forward FFT of the full input --- vector audioReal(fftBins), audioImag(fftBins); - m_fft->forward(buf.data(), audioReal.data(), audioImag.data()); + fft->forward(buf.data(), audioReal.data(), audioImag.data()); // --- Kernel: reversed first half, zero-padded to frameSize --- - // Convolving x[0..frameSize-1] with this kernel gives the - // YIN-style autocorrelation via the overlap at lag+halfSize-1. vector kernel(frameSize, 0.0f); for (int j = 0; j < halfSize; ++j) kernel[j] = buf[halfSize - 1 - j]; vector kernelReal(fftBins), kernelImag(fftBins); - m_fft->forward(kernel.data(), kernelReal.data(), kernelImag.data()); + fft->forward(kernel.data(), kernelReal.data(), kernelImag.data()); - // --- Complex multiply in frequency domain --- + // --- Complex multiply --- vector acfReal(fftBins), acfImag(fftBins); for (int j = 0; j < fftBins; ++j) { acfReal[j] = audioReal[j]*kernelReal[j] - audioImag[j]*kernelImag[j]; acfImag[j] = audioReal[j]*kernelImag[j] + audioImag[j]*kernelReal[j]; } - // --- Inverse FFT → time-domain autocorrelation --- + // --- Inverse FFT --- vector acfOut(frameSize); - m_fft->inverse(acfReal.data(), acfImag.data(), acfOut.data()); + fft->inverse(acfReal.data(), acfImag.data(), acfOut.data()); - // bqfft inverse is unnormalized (unlike vamp FFT which divides by n). + // bqfft inverse is unnormalized — divide by frameSize. const double scale = 1.0 / frameSize; - // --- Compute difference function --- - // d[tau] = powerTerms[0] + powerTerms[tau] - 2*r[tau] - // r[tau] lives at acfOut[tau + halfSize - 1] after the convolution. diff.assign(halfSize, 0.0); - diff[0] = 0.0; for (int tau = 1; tau < halfSize; ++tau) { diff[tau] = powerTerms[0] + powerTerms[tau] - 2.0 * double(acfOut[tau + halfSize - 1]) * scale; @@ -234,7 +190,7 @@ RealtimePitchTracker::yinCMND(vector &diff) int halfSize = (int)diff.size(); if (halfSize == 0) return; - diff[0] = 1.0; // by convention + diff[0] = 1.0; double runningSum = 0.0; for (int tau = 1; tau < halfSize; ++tau) { @@ -257,12 +213,9 @@ RealtimePitchTracker::yinFindPitch(const vector &cmnd, if (minLag < 1) minLag = 1; if (minLag >= maxLag) return -1.0; - // Walk forward from minLag looking for the first value below threshold - // that is also a local minimum. int bestTau = -1; for (int tau = minLag; tau <= maxLag; ++tau) { if (cmnd[tau] < threshold) { - // Walk to the bottom of the dip while (tau + 1 <= maxLag && cmnd[tau + 1] < cmnd[tau]) { ++tau; } @@ -272,22 +225,14 @@ RealtimePitchTracker::yinFindPitch(const vector &cmnd, } if (bestTau < 1 || bestTau >= halfSize - 1) { - return -1.0; // unvoiced + return -1.0; } - // Parabolic interpolation around the minimum double s0 = cmnd[bestTau - 1]; double s1 = cmnd[bestTau]; double s2 = cmnd[bestTau + 1]; double denom = 2.0 * (2.0 * s1 - s2 - s0); - double refined; - if (std::abs(denom) < 1e-12) { - refined = double(bestTau); - } else { - refined = double(bestTau) + (s2 - s0) / denom; - } - - return refined; + if (std::abs(denom) < 1e-12) return double(bestTau); + return double(bestTau) + (s2 - s0) / denom; } - diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h index dd251b5c..221d72cc 100644 --- a/main/RealtimePitchTracker.h +++ b/main/RealtimePitchTracker.h @@ -15,14 +15,12 @@ #ifndef REALTIME_PITCH_TRACKER_H #define REALTIME_PITCH_TRACKER_H -#include -#include +#include #include #include "base/BaseTypes.h" #include "data/model/Model.h" -#include "data/model/SparseTimeValueModel.h" namespace sv { class WritableWaveFileModel; @@ -33,151 +31,98 @@ class FFT; } /** - * RealtimePitchTracker polls a WritableWaveFileModel (the model being - * filled during a live microphone recording) for new audio samples and - * estimates pitch in real time using a simplified YIN autocorrelation - * algorithm. + * RealtimePitchTracker runs on a dedicated background QThread and + * continuously polls a WritableWaveFileModel for new audio samples, + * estimating pitch in real time using FFT-accelerated YIN. * - * Results are written into a SparseTimeValueModel so they can be - * displayed immediately as a TimeValueLayer overlaid on the main pane, - * giving the singer real-time visual feedback about their pitch. - * - * This is intentionally a low-latency, lower-accuracy alternative to - * the full pYIN analysis that will be run once recording is complete. + * pitch estimates are reported via pitchDetected() signals; the + * connection to the GUI thread is automatically a QueuedConnection so + * the slot (which writes to the model and updates the status bar) runs + * safely on the GUI thread without blocking audio or rendering. * * Usage: - * 1. Create a RealtimePitchTracker, providing the ModelId of both: - * - the WritableWaveFileModel being recorded into (audio source), - * - the SparseTimeValueModel to write pitch estimates into. - * 2. Call start() when recording begins. - * 3. The tracker polls automatically via an internal QTimer. - * 4. Call stop() when recording ends. + * 1. Create a RealtimePitchTracker with the ModelId of the + * WritableWaveFileModel being recorded into. + * 2. Call start() — the background thread starts immediately. + * 3. Call stop() when recording ends — blocks until the thread exits. */ -class RealtimePitchTracker : public QObject +class RealtimePitchTracker : public QThread { Q_OBJECT public: /** - * Construct a tracker. - * * @param audioSourceId ModelId of the WritableWaveFileModel being - * recorded into. The tracker polls this for - * new samples on each timer tick. - * @param pitchModelId ModelId of the SparseTimeValueModel to write - * pitch estimates into. - * @param parent Optional Qt parent. + * recorded into. Polled from the background thread. + * @param parent Optional Qt parent (must live on GUI thread). */ RealtimePitchTracker(sv::ModelId audioSourceId, - sv::ModelId pitchModelId, QObject *parent = nullptr); virtual ~RealtimePitchTracker(); /** - * Start tracking. Resets all internal state. + * Start the background polling thread. * Must be called from the GUI thread. */ void start(); /** - * Stop tracking. No more pitch estimates will be written after - * this returns. + * Request the background thread to stop and block until it exits. * Must be called from the GUI thread. */ void stop(); - /** - * Minimum frequency (Hz) that the tracker will report. - * Pitches below this are treated as unvoiced. Default: 60 Hz. - */ + /** Minimum frequency (Hz) reported. Default: 60 Hz. */ void setMinFrequency(double hz) { m_minFreq = hz; } double getMinFrequency() const { return m_minFreq; } - /** - * Maximum frequency (Hz) that the tracker will report. - * Pitches above this are treated as unvoiced. Default: 1000 Hz. - */ + /** Maximum frequency (Hz) reported. Default: 1000 Hz. */ void setMaxFrequency(double hz) { m_maxFreq = hz; } double getMaxFrequency() const { return m_maxFreq; } - /** - * YIN threshold. Lower values are more selective (fewer voiced - * detections), higher values yield more detections but more - * errors. Default: 0.15. - */ + /** YIN threshold (0–1). Default: 0.15. */ void setThreshold(double t) { m_threshold = t; } double getThreshold() const { return m_threshold; } signals: /** - * Emitted each time a new pitch estimate is available. + * Emitted from the background thread each time a new voiced pitch + * estimate is available. Via Qt::AutoConnection this arrives in the + * GUI thread's event loop (QueuedConnection cross-thread). * - * @param frame Sample frame at which the pitch was estimated - * (centre of the analysis window), relative to the - * start of the recording. - * @param hz Estimated pitch in Hz, or 0 if unvoiced. + * @param frame Centre frame of the analysis window. + * @param hz Pitch in Hz (always > 0 when emitted). */ void pitchDetected(sv::sv_frame_t frame, double hz); -private slots: - /// Called by the internal QTimer; polls the audio model and runs YIN. - void pollAndProcess(); +protected: + /** The background polling loop — do not call directly. */ + void run() override; private: - // --- Model IDs --- - sv::ModelId m_audioSourceId; // WritableWaveFileModel being recorded - sv::ModelId m_pitchModelId; // SparseTimeValueModel for output - - // --- Configuration --- - double m_minFreq; - double m_maxFreq; - double m_threshold; + sv::ModelId m_audioSourceId; - // --- State --- - bool m_running; + double m_minFreq; + double m_maxFreq; + double m_threshold; - /// How many input frames we have already processed (exclusive end - /// of the last complete hop). We use this to avoid re-processing - /// samples on the next timer tick. - sv::sv_frame_t m_nextFrameToProcess; + // Window size: 2048 samples @ 44100 Hz ≈ 46 ms. + // Hop size: 256 samples ≈ 5.8 ms. + static const int kWindowSize = 2048; + static const int kHopSize = 256; - // --- Processing parameters --- - // Window size: 2048 samples @ 44100 Hz ≈ 46 ms (two full periods of 60 Hz). - // Hop size: 256 samples ≈ 5.8 ms — finer dot density than the old 512. - static const int kWindowSize = 2048; - static const int kHopSize = 256; + // --- YIN helpers (all called only from run()) --- - // --- Timer --- - QTimer *m_timer; + static void yinDifferenceFFT(const std::vector &buf, + std::vector &diff, + breakfastquay::FFT *fft); - // --- FFT for fast YIN difference function --- - breakfastquay::FFT *m_fft; // lazy-created on first poll - - // --- YIN helpers --- - - /** - * FFT-based difference function (O(n log n) vs the naive O(n²)). - * Computes d[tau] = sum_{j=0}^{halfSize-1} (x[j] - x[j+tau])^2 - * using the autocorrelation identity and bqfft. - * buf must have size kWindowSize; diff is sized to kWindowSize/2. - */ - void yinDifferenceFFT(const std::vector &buf, - std::vector &diff); - - /** - * Step 3: cumulative mean normalised difference (in-place). - */ static void yinCMND(std::vector &diff); - /** - * Steps 4-5: find first dip below threshold with parabolic - * interpolation. Returns fractional lag in samples, or -1 if - * no dip found. - */ static double yinFindPitch(const std::vector &cmnd, int minLag, int maxLag, double threshold); }; -#endif // REALTIME_PITCH_TRACKER_H \ No newline at end of file +#endif // REALTIME_PITCH_TRACKER_H From 629b0785caa2a5daf4090796acb2cc01f7cb018c Mon Sep 17 00:00:00 2001 From: jhhr Date: Mon, 1 Jun 2026 22:29:56 +0300 Subject: [PATCH 012/275] repoint: redirect svapp/svgui/bqaudiostream to jhhr forks with tony customizations - svapp/svgui: change owner to jhhr, branch to tony-customizations - bqaudiostream: switch from hg/sourcehut to git/github/jhhr (the local checkout was already a git clone; this makes it explicit and safe) - Update lock pins to new commit SHAs on the forked branches Co-Authored-By: Claude Sonnet 4.6 --- repoint-lock.json | 8 ++++---- repoint-project.json | 18 +++++++++--------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/repoint-lock.json b/repoint-lock.json index d3404c23..3df4ff91 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,10 +7,10 @@ "pin": "65eeb62d0c966f4c15db78ba28eab8d4a8ea0274" }, "svgui": { - "pin": "d46bd4b46a13be38c56f6704911b53cb1b6e3ceb" + "pin": "8516f9010dded9bb1303291499155a0717297bf0" }, "svapp": { - "pin": "f51a1690f3f92e2421916d39a3385d49214e60c6" + "pin": "d67336efd0d3777442f450f7ff1dc680f6b0f608" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" @@ -34,7 +34,7 @@ "pin": "88520ab5b8e5" }, "bqaudiostream": { - "pin": "5d5950bc4d93" + "pin": "9b59e706f9dda8507c5e1b07ad619c060f65bd29" }, "bqthingfactory": { "pin": "2e4bd170f57f" @@ -52,4 +52,4 @@ "pin": "340ef2791c5fdd869125b952e43ed9a1e3e8a27f" } } -} +} \ No newline at end of file diff --git a/repoint-project.json b/repoint-project.json index 72812469..5d76dd04 100644 --- a/repoint-project.json +++ b/repoint-project.json @@ -17,14 +17,14 @@ "svgui": { "vcs": "git", "service": "github", - "owner": "sonic-visualiser", - "branch": "default" + "owner": "jhhr", + "branch": "tony-customizations" }, "svapp": { "vcs": "git", "service": "github", - "owner": "sonic-visualiser", - "branch": "default" + "owner": "jhhr", + "branch": "tony-customizations" }, "checker": { "vcs": "git", @@ -64,9 +64,10 @@ "owner": "breakfastquay" }, "bqaudiostream": { - "vcs": "hg", - "service": "sourcehut", - "owner": "breakfastquay" + "vcs": "git", + "service": "github", + "owner": "jhhr", + "branch": "master" }, "bqthingfactory": { "vcs": "hg", @@ -97,5 +98,4 @@ "owner": "c4dm" } } -} - +} \ No newline at end of file From 9d57ee756470866bcb2fbfa13b141d4f147358d6 Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 18 Sep 2026 21:51:47 +0300 Subject: [PATCH 013/275] fix: prune extra panes without freeing the singing model or shared ruler Review findings 1 and 2 (tmp/SINGING_PRACTICE_REVIEW.md). loadSingingTrack() force-deleted every layer in the extra pane that openPath(CreateAdditionalModel) creates. The imported WaveformLayer was the only reference to the new model, so Document::releaseModel() freed it and the deferred analysis then failed on a dead id. Set the secondary analyser up first, so its own WaveformLayer holds the model, and prune afterwards. The same loops in loadSingingTrack() and loadBackgroundMusic() also force-deleted the shared time ruler, which after a session load lives in both pane 1 and the new pane: the ruler vanished, the object was freed, and MainWindowBase::m_timeRulerLayer dangled until the next record() dereferenced it. Both flows and the record drain now go through one helper, pruneExtraPane() in main/PaneUtils.cpp, which deletes only the layers whose model is the one being pruned and detaches everything else. It is a free function so it can be tested without a MainWindow (test-plan refactor R2). Also takes the minimal part of finding 7: analyseNewMainModel()'s session-restore scan is skipped while a singing model is pending. Without it the queued second setup tears down the layers that hold the model and frees it, which would reintroduce finding 1. Adds the automated test harness the fix is verified with: - meson.build: main/ is split into libtonycore (pitch tracker, no GUI) and libtonyapp (svgui/svapp and the rest), linked into Tony.exe with link_whole and into two new test executables. - main/test/: Tier 1 (YIN maths), Tier 2 (tracker thread, latency arithmetic) and Tier 3 (Document ownership rules, pane pruning) from tmp/SINGING_PRACTICE_TEST_PLAN.md. The four Tier 3 prune tests fail without this fix and pass with it. - main/LatencyUtils.h: computeRecordingLatency() extracted so the arithmetic is testable without a window (refactor R1). - build.bat test: runs meson test --print-errorlogs. Co-Authored-By: Claude Fable 5.1 --- build.bat | 15 ++ main/LatencyUtils.h | 36 +++ main/MainWindow.cpp | 167 ++++---------- main/MainWindow.h | 19 +- main/PaneUtils.cpp | 89 ++++++++ main/PaneUtils.h | 48 ++++ main/RealtimePitchTracker.h | 2 + main/test/PyinReference.cpp | 26 +++ main/test/PyinReference.h | 27 +++ main/test/RunSuite.h | 42 ++++ main/test/TestLatencyShift.h | 115 ++++++++++ main/test/TestRealtimePitchTracker.h | 315 +++++++++++++++++++++++++++ main/test/TestRealtimeYin.h | 221 +++++++++++++++++++ main/test/TestSignals.h | 71 ++++++ main/test/TestSingingDocument.h | 307 ++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 61 ++++++ main/test/tony-core-test.cpp | 68 ++++++ meson.build | 147 ++++++++++++- 18 files changed, 1640 insertions(+), 136 deletions(-) create mode 100644 main/LatencyUtils.h create mode 100644 main/PaneUtils.cpp create mode 100644 main/PaneUtils.h create mode 100644 main/test/PyinReference.cpp create mode 100644 main/test/PyinReference.h create mode 100644 main/test/RunSuite.h create mode 100644 main/test/TestLatencyShift.h create mode 100644 main/test/TestRealtimePitchTracker.h create mode 100644 main/test/TestRealtimeYin.h create mode 100644 main/test/TestSignals.h create mode 100644 main/test/TestSingingDocument.h create mode 100644 main/test/tony-app-test.cpp create mode 100644 main/test/tony-core-test.cpp diff --git a/build.bat b/build.bat index db0584ab..afbe0884 100644 --- a/build.bat +++ b/build.bat @@ -16,6 +16,7 @@ set PATH=%MSYS2_MINGW%\bin;%MSYS2_MINGW%\..\usr\bin;%PATH% :: build.bat → build only :: build.bat run → build then launch Tony.exe :: build.bat launch → launch Tony.exe without building +:: build.bat test → build and run the automated tests :: build.bat clean → wipe build_mingw and reconfigure :: build.bat help → print this help :: ───────────────────────────────────────────────────────────────────────────── @@ -29,6 +30,7 @@ if /i "%ACTION%"=="clean" goto :clean if /i "%ACTION%"=="launch" goto :launch if /i "%ACTION%"=="run" goto :build if /i "%ACTION%"=="build" goto :build +if /i "%ACTION%"=="test" goto :test echo ERROR: Unknown action "%ACTION%". Run "build.bat help" for usage. exit /b 1 @@ -64,6 +66,18 @@ echo Build succeeded: %BUILD_DIR%\Tony.exe if /i "%ACTION%"=="run" goto :launch exit /b 0 +:: ── test ───────────────────────────────────────────────────────────────────── +:test +if not exist "%BUILD_DIR%\build.ninja" ( + echo Build directory not configured. Running meson setup ... + meson setup "%BUILD_DIR%" --buildtype=debugoptimized + if errorlevel 1 exit /b %errorlevel% +) + +echo Building and running tests ... +meson test -C "%BUILD_DIR%" --print-errorlogs +exit /b %errorlevel% + :: ── launch ─────────────────────────────────────────────────────────────────── :launch set EXE=%BUILD_DIR%\Tony.exe @@ -84,6 +98,7 @@ echo Actions: echo (none) Build Tony.exe (default) echo run Build Tony.exe then launch it echo launch Launch Tony.exe without rebuilding +echo test Build and run the automated tests (meson test) echo clean Delete the build directory and reconfigure from scratch echo help Show this message echo. diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h new file mode 100644 index 00000000..606e2e22 --- /dev/null +++ b/main/LatencyUtils.h @@ -0,0 +1,36 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LATENCY_UTILS_H +#define TONY_LATENCY_UTILS_H + +#include "base/BaseTypes.h" + +/** + * Round-trip latency, in frames, to compensate for when a singing + * take is recorded while the reference is playing: the singer's + * response to reference frame 0 arrives in the recording at about + * frame (outputLatency + inputLatency). Either figure may be + * unavailable (reported as zero or negative); the result is never + * negative. + */ +inline sv::sv_frame_t +computeRecordingLatency(sv::sv_frame_t outputLatency, + sv::sv_frame_t inputLatency) +{ + if (outputLatency < 0) outputLatency = 0; + if (inputLatency < 0) inputLatency = 0; + return outputLatency + inputLatency; +} + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 41789a57..d5db46cc 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -18,6 +18,8 @@ #include "MainWindow.h" #include "NetworkPermissionTester.h" #include "Analyser.h" +#include "LatencyUtils.h" +#include "PaneUtils.h" #include "framework/Document.h" #include "framework/VersionTester.h" @@ -2112,42 +2114,32 @@ MainWindow::loadSingingTrack(QString path) return; } + // modelAdded() fired synchronously inside openPath() and stored the new + // model's id in m_pendingSingingModelId. Set up the secondary analyser + // NOW, before pruning the extra pane: the imported WaveformLayer in that + // pane is the only layer referencing the singing model, so deleting it + // first would make Document::releaseModel() free the model before it can + // be analysed. Once m_analyser2's own WaveformLayer references the model + // the orphan can go. This also clears m_pendingSingingModelId, so the + // analyseNewSingingModel() call queued by modelAdded() becomes a no-op. + ModelId singingModelId = m_pendingSingingModelId; + analyseNewSingingModel(); + // openPath(CreateAdditionalModel) will have called AddPaneCommand which // added a new pane for the singing track's waveform layer. We do NOT // want that extra pane — both tracks must overlay pane 0. Remove any // panes above the original count (except the time-ruler pane at index 1 // which was already there). We delete from the top down so that index - // arithmetic stays valid. + // arithmetic stays valid. If the analyser setup above failed, nothing + // else references the singing model and pruning releases it, which is + // what we want. if (m_paneStack) { while (m_paneStack->getPaneCount() > paneCountBefore) { Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - // Delete any layers that ended up in this extra pane. - // setupSingingTrackAnalyser will create proper layers in pane 0, - // so these auto-created layers (e.g. the waveform from - // addOpenedAudioModel) are not needed and would otherwise - // become orphans in the document layer list. - if (m_document && extra) { - while (extra->getLayerCount() > 0) { - Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); - // Use deleteLayer with force=true: this removes the layer - // from the view directly (without creating an undo command) - // and then deletes it from the document. We must NOT call - // removeLayerFromView first, as that would push a - // RemoveLayerCommand onto the undo stack holding a pointer - // to a layer we are about to delete — a guaranteed crash on - // undo. - m_document->deleteLayer(orphan, true); - } - } - if (m_overview) m_overview->unregisterView(extra); - m_paneStack->deletePane(extra); + if (!extra) break; + pruneExtraPane(extra, singingModelId); } } - - // analyseNewSingingModel() is triggered on the next event-loop tick via - // the QTimer::singleShot(0, ...) in modelAdded(). It picks up the model - // id from m_pendingSingingModelId, which was set in modelAdded() when the - // audio file was registered with the document. } void @@ -2261,14 +2253,8 @@ MainWindow::loadBackgroundMusic(QString path) if (m_paneStack) { while (m_paneStack->getPaneCount() > paneCountBefore) { Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - if (m_document && extra) { - while (extra->getLayerCount() > 0) { - Layer *orphan = extra->getLayer(extra->getLayerCount() - 1); - m_document->deleteLayer(orphan, true); - } - } - if (m_overview) m_overview->unregisterView(extra); - m_paneStack->deletePane(extra); + if (!extra) break; + pruneExtraPane(extra, m_backgroundMusicModelId); } } @@ -2384,7 +2370,7 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal // deleteLayer(force=true) iterates Document::m_layerViewMap to remove the // layer from any views — and that map still contains the live pane pointer. // After deletePane() the pointer would be dangling → crash. - drainPendingExtraPanes(); + drainPendingExtraPanes(singingModelId); // Re-stack layers so the primary pitch track stays on top m_analyser->getLayer(Analyser::PitchTrack); // ensure primary is on top @@ -2399,94 +2385,27 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal } void -MainWindow::drainPendingExtraPanes() +MainWindow::pruneExtraPane(Pane *extra, sv::ModelId ownedModelId) { - // This helper is called from setupSingingTrackAnalyser() after m_analyser2 - // has been initialised with its own WaveformLayer referencing the recording - // model. At that point it is safe to call deleteLayer(orphan, true) on the - // extra pane's waveform layer because: - // (a) m_analyser2's WaveformLayer holds a reference to the model, so - // Document::releaseModel() will not free it. - // (b) The extra pane widget is still alive (we haven't called deletePane - // yet), so Document::m_layerViewMap iteration in deleteLayer(true) is - // valid and won't dereference a dangling pointer. - // - // It is also called from teardownSingingTrackAnalyser() and closeSession() - // as a safety net — in those contexts m_document may be null so we just - // call deletePane() to destroy the widget without touching the document. + // The rules for what may be deleted and what only detached live + // with the helper: see PaneUtils.cpp. + ::pruneExtraPane(m_document, m_paneStack, extra, ownedModelId, m_overview); +} + +void +MainWindow::drainPendingExtraPanes(sv::ModelId singingModelId) +{ + // Called from setupSingingTrackAnalyser() after m_analyser2 has been + // initialised with its own WaveformLayer referencing the recording + // model, which is what makes pruneExtraPane() safe for the panes that + // record() hid rather than deleted. if (m_pendingExtraPanes.empty()) return; cerr << "MainWindow::drainPendingExtraPanes: draining " << m_pendingExtraPanes.size() << " pending extra pane(s)" << endl; for (Pane *extra : m_pendingExtraPanes) { - if (!extra) continue; - - if (m_document) { - // The extra pane created by MainWindowBase::record() in - // RecordCreateAdditionalModel mode contains two layers: - // - // 1. m_timeRulerLayer — a SHARED layer that also lives in pane 0. - // We must NOT call deleteLayer() on it; that would remove the - // ruler from every view including pane 0. Instead, use - // removeLayerFromView(extra, layer) so only the extra pane's - // entry is removed from m_layerViewMap (this creates an undo - // command, but the layer pointer remains valid so no crash). - // - // 2. The orphan WaveformLayer from createImportedLayer() — unique - // to this pane, with the WritableWaveFileModel as its model. - // deleteLayer(force=true) is safe here because: - // (a) m_analyser2 was set up before this drain, so its - // WaveformLayer still holds a reference to the model - // → releaseModel() will not free the live recording. - // (b) The extra pane widget is still alive → m_layerViewMap - // iteration in deleteLayer(true) is valid. - // - // We distinguish them by whether the model is a WritableWaveFileModel. - int layerCount = extra->getLayerCount(); - for (int i = layerCount - 1; i >= 0; --i) { - Layer *lay = extra->getLayer(i); - if (!lay) continue; - - ModelId layerModel = lay->getModel(); - - if (ModelById::isa(layerModel)) { - // Orphan recording waveform: full delete (safe because - // m_analyser2's WaveformLayer still holds the model ref). - // deleteLayer(force=true) iterates m_layerViewMap and calls - // view->removeLayer() — the pane is still alive here so - // the pointer is valid. - cerr << "MainWindow::drainPendingExtraPanes: deleteLayer orphan " - << lay << " [" << lay->objectName().toStdString() - << "] (WritableWaveFileModel)" << endl; - m_document->deleteLayer(lay, true); - } else { - // Shared layer (e.g. TimeRuler): we cannot call deleteLayer - // because that would remove the layer from ALL views. We - // also cannot call removeLayerFromView because that pushes a - // RemoveLayerCommand onto the undo stack with a raw pointer - // to the extra pane — if undo is later triggered the pane - // is already destroyed → crash. - // - // Instead, use detachLayerFromView (no undo command): removes - // the layer from the pane's display list AND updates - // m_layerViewMap so deletePane() leaves no dangling pointer. - cerr << "MainWindow::drainPendingExtraPanes: detachLayerFromView " - << lay << " [" << lay->objectName().toStdString() - << "] (shared layer, keeping in other views)" << endl; - m_document->detachLayerFromView(extra, lay); - } - } - } - - // Now destroy the pane widget. By this point all WritableWaveFileModel - // layers have been deleted from the document and m_layerViewMap no - // longer references this pane for them. Shared layers were properly - // detached via removeLayerFromView. deletePane() can safely destroy - // the widget without leaving dangling pointers in m_layerViewMap. - if (m_paneStack) { - m_paneStack->deletePane(extra); - } + pruneExtraPane(extra, singingModelId); } m_pendingExtraPanes.clear(); @@ -2949,7 +2868,8 @@ MainWindow::recordingStarted() // shift the model's start frame by -(outputLatency + inputLatency). sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; - m_recordingLatencyFrames = outputLatency + inputLatency; + m_recordingLatencyFrames = + computeRecordingLatency(outputLatency, inputLatency); cerr << "MainWindow::recordingStarted: output latency=" << outputLatency << " input latency=" << inputLatency << " round-trip compensation=" << m_recordingLatencyFrames << " frames" << endl; @@ -4282,8 +4202,10 @@ MainWindow::modelAdded(ModelId model) }); } else { // Normal case: a finished audio file was loaded as a - // singing track. Run full analysis immediately. - // Defer so the model is fully registered before we analyse. + // singing track. loadSingingTrack() runs the analysis + // itself as soon as openPath() returns (it must happen + // before the extra pane is pruned); this deferred call + // is the fallback for any other route that adds a model. QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); } } else { @@ -4510,7 +4432,12 @@ MainWindow::analyseNewMainModel() // not the main model and not already being tracked as a singing model. // We only do this if we don't already have a secondary analyser (it may // have been set up already e.g. via modelAdded() during session load). - if (!m_analyser2 && m_document) { + // Skip the scan while a singing model is pending: openAudio() emits + // audioFileLoaded() for CreateAdditionalModel too, and loadSingingTrack() + // is about to set up that model itself. A second, queued setup would + // tear down m_analyser2's layers — the only references to the singing + // model — releasing it before the re-setup. + if (!m_analyser2 && m_document && m_pendingSingingModelId.isNone()) { ModelId mainId = getMainModelId(); ModelId foundSinging; for (ModelId mid : m_document->getModels()) { diff --git a/main/MainWindow.h b/main/MainWindow.h index 6a3b7756..43fd394b 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -327,14 +327,19 @@ protected slots: void loadBackgroundMusic(QString path); void teardownBackgroundMusic(); - // Drain m_pendingExtraPanes: delete orphan layers and the pane widgets - // that were deferred from record()'s pane-cleanup step. Must be called - // after m_analyser2 has created its WaveformLayer (so deleteLayer won't + // Remove an extra pane created by openAudio()/record() in + // CreateAdditionalModel mode: delete the orphan layer(s) showing + // ownedModelId, detach shared layers (the time ruler) without deleting + // them, then delete the pane widget. Another layer must already + // reference ownedModelId, or the model is released with the orphan. + void pruneExtraPane(sv::Pane *extra, sv::ModelId ownedModelId); + + // Drain m_pendingExtraPanes: prune the panes that were deferred from + // record()'s pane-cleanup step. Must be called after m_analyser2 has + // created its WaveformLayer for singingModelId (so deleteLayer won't // free the recording model), and while the pane widgets are still alive // (so m_layerViewMap iteration in deleteLayer(force=true) is valid). - // Also called as a safety net from teardownSingingTrackAnalyser() and - // closeSession() to avoid leaking widgets. - void drainPendingExtraPanes(); + void drainPendingExtraPanes(sv::ModelId singingModelId); // When loadSingingTrack opens an additional audio file, modelAdded() // stores the resulting ModelId here so analyseNewSingingModel() can @@ -384,7 +389,7 @@ protected slots: // them — AFTER m_analyser2 has created its own WaveformLayer referencing // the recording model, making it safe to call deleteLayer(orphan, true) // on the extra pane's waveform layer without releasing the recording model. - // Also drained by teardownSingingTrackAnalyser() and closeSession(). + // closeSession() deletes any leftovers via its hidden-pane loop. std::vector m_pendingExtraPanes; virtual void octaveShift(bool up); diff --git a/main/PaneUtils.cpp b/main/PaneUtils.cpp new file mode 100644 index 00000000..84cdc0df --- /dev/null +++ b/main/PaneUtils.cpp @@ -0,0 +1,89 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "PaneUtils.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/Overview.h" +#include "layer/Layer.h" + +#include + +using namespace sv; +using std::cerr; +using std::endl; + +void +pruneExtraPane(Document *document, PaneStack *paneStack, Pane *extra, + ModelId ownedModelId, Overview *overview) +{ + // PRECONDITIONS: some other layer (the secondary analyser's + // WaveformLayer, or the background music layer) already references + // ownedModelId, otherwise deleting the orphan below makes + // Document::releaseModel() free the model. The pane widget must + // still be alive, so that the Document::m_layerViewMap iteration in + // deleteLayer(force=true) is valid. + // + // The extra pane contains two kinds of layer: + // + // 1. The orphan WaveformLayer from createImportedLayer() — unique to + // this pane, with ownedModelId as its model. deleteLayer(force=true) + // removes it from the view without creating an undo command and + // deletes it from the document. We must NOT use removeLayerFromView + // for it, as that would push a RemoveLayerCommand onto the undo + // stack holding a pointer to a layer we are about to delete. + // + // 2. Shared layers, i.e. the time ruler, which openAudio()/record() + // add to the new pane when MainWindowBase::m_timeRulerLayer is set + // (e.g. after a session load) and which also lives in pane 1. We + // must NOT call deleteLayer() on it: that would remove the ruler + // from every view, free it, and leave m_timeRulerLayer dangling. + // removeLayerFromView is no good either: it pushes a + // RemoveLayerCommand with a raw pointer to the extra pane, which + // is about to be destroyed → crash on undo. Use + // detachLayerFromView (no undo command): it removes the layer + // from the pane's display list AND updates m_layerViewMap so + // deletePane() leaves no dangling pointer. + // + // We distinguish them by whether the layer's model is ownedModelId. + // The ruler's own model id is "none", so an empty ownedModelId must + // match nothing. + if (!extra) return; + + if (document) { + int layerCount = extra->getLayerCount(); + for (int i = layerCount - 1; i >= 0; --i) { + Layer *lay = extra->getLayer(i); + if (!lay) continue; + + if (!ownedModelId.isNone() && lay->getModel() == ownedModelId) { + cerr << "pruneExtraPane: deleteLayer orphan " + << lay << " [" << lay->objectName().toStdString() + << "]" << endl; + document->deleteLayer(lay, true); + } else { + cerr << "pruneExtraPane: detachLayerFromView " + << lay << " [" << lay->objectName().toStdString() + << "] (shared layer, keeping in other views)" << endl; + document->detachLayerFromView(extra, lay); + } + } + } + + // Now destroy the pane widget (works for hidden panes too). By this + // point m_layerViewMap no longer references this pane. + if (overview) overview->unregisterView(extra); + if (paneStack) paneStack->deletePane(extra); +} diff --git a/main/PaneUtils.h b/main/PaneUtils.h new file mode 100644 index 00000000..4a9058aa --- /dev/null +++ b/main/PaneUtils.h @@ -0,0 +1,48 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_PANE_UTILS_H +#define TONY_PANE_UTILS_H + +#include "base/ById.h" +#include "data/model/Model.h" + +namespace sv { +class Document; +class PaneStack; +class Pane; +class Overview; +} + +/** + * Remove an unwanted extra pane that openAudio()/record() created via + * AddPaneCommand in CreateAdditionalModel mode. Shared by the record, + * load-singing-track and load-background-music flows, all of which + * want the new audio overlaid on pane 0 rather than in its own pane. + * + * Layers in the pane whose model is ownedModelId (the imported + * waveform) are deleted from the document; any other layer (the + * shared time ruler) is only detached from this pane. The pane is + * then unregistered from the overview, if one is given, and deleted. + * The pane may be visible or hidden. + * + * Some other layer must already reference ownedModelId, otherwise + * Document::releaseModel() frees the model along with the orphan. + */ +void pruneExtraPane(sv::Document *document, + sv::PaneStack *paneStack, + sv::Pane *extra, + sv::ModelId ownedModelId, + sv::Overview *overview = nullptr); + +#endif diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h index 221d72cc..2d3b1884 100644 --- a/main/RealtimePitchTracker.h +++ b/main/RealtimePitchTracker.h @@ -101,6 +101,8 @@ class RealtimePitchTracker : public QThread void run() override; private: + friend class TestRealtimeYin; + sv::ModelId m_audioSourceId; double m_minFreq; diff --git a/main/test/PyinReference.cpp b/main/test/PyinReference.cpp new file mode 100644 index 00000000..e5ecf86c --- /dev/null +++ b/main/test/PyinReference.cpp @@ -0,0 +1,26 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "PyinReference.h" + +#include "../../pyin/YinUtil.h" + +std::vector +pyinFastDifference(const std::vector &in) +{ + int halfSize = int(in.size() / 2); + std::vector out(halfSize, 0.0); + YinUtil util(halfSize); + util.fastDifference(in.data(), out.data()); + return out; +} diff --git a/main/test/PyinReference.h b/main/test/PyinReference.h new file mode 100644 index 00000000..ad376789 --- /dev/null +++ b/main/test/PyinReference.h @@ -0,0 +1,27 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_PYIN_REFERENCE_H +#define TEST_PYIN_REFERENCE_H + +#include + +/** + * pYIN's own YinUtil::fastDifference, for a frame of in.size() samples; + * returns in.size()/2 values. Wrapped in its own translation unit + * because YinUtil.h pulls in the Vamp plugin SDK headers, which must + * not be mixed with the host SDK headers that svcore uses. + */ +std::vector pyinFastDifference(const std::vector &in); + +#endif diff --git a/main/test/RunSuite.h b/main/test/RunSuite.h new file mode 100644 index 00000000..06bf5d81 --- /dev/null +++ b/main/test/RunSuite.h @@ -0,0 +1,42 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_RUN_SUITE_H +#define TEST_RUN_SUITE_H + +#include +#include + +/** + * Run one suite with the command-line arguments given. If the + * environment variable TONY_TEST_LOG_DIR is set, the suite's results + * are also written to

/.txt. A single "-o file" + * argument cannot do that, as each suite would overwrite the last. + */ +inline bool +runSuite(QObject *suite, int argc, char *argv[]) +{ + QStringList args; + for (int i = 0; i < argc; ++i) { + args << QString::fromLocal8Bit(argv[i]); + } + QString logDir = qEnvironmentVariable("TONY_TEST_LOG_DIR"); + if (logDir != "") { + QString file = QDir(logDir).filePath + (QString("%1.txt").arg(suite->metaObject()->className())); + args << "-o" << (file + ",txt") << "-o" << "-,txt"; + } + return QTest::qExec(suite, args) == 0; +} + +#endif diff --git a/main/test/TestLatencyShift.h b/main/test/TestLatencyShift.h new file mode 100644 index 00000000..25ccb00c --- /dev/null +++ b/main/test/TestLatencyShift.h @@ -0,0 +1,115 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LATENCY_SHIFT_H +#define TEST_LATENCY_SHIFT_H + +// Tier 2: what is done with the latency figure. A singing take is +// aligned with the reference by giving its model a negative start +// frame; these tests pin down what that does to reads. + +#include "../LatencyUtils.h" + +#include "data/model/WritableWaveFileModel.h" + +#include +#include +#include + +#include +#include + +class TestLatencyShift : public QObject +{ + Q_OBJECT + + static const int kLength = 8192; + static const int kImpulseAt = 1000; // the latency, N + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + // A finished take: silence with a single impulse at kImpulseAt + std::shared_ptr makeTake() { + QString path = m_dir.filePath + (QString("take-%1.wav").arg(++m_fileCounter)); + auto model = std::make_shared + (path, 44100.0, 1, + sv::WritableWaveFileModel::Normalisation::None); + std::vector data(kLength, 0.f); + data[kImpulseAt] = 1.f; + const float *ptr = data.data(); + model->addSamples(&ptr, kLength); + model->writeComplete(); + return model; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void start_frame_shifts_reads() { + auto model = makeTake(); + QCOMPARE(model->getData(0, kImpulseAt, 1).size(), size_t(1)); + QCOMPARE(model->getData(0, kImpulseAt, 1)[0], 1.f); + + model->setStartFrame(-kImpulseAt); + + QCOMPARE(model->getStartFrame(), sv::sv_frame_t(-kImpulseAt)); + auto atZero = model->getData(0, 0, 1); + QCOMPARE(atZero.size(), size_t(1)); + QCOMPARE(atZero[0], 1.f); + auto atOld = model->getData(0, kImpulseAt, 1); + QCOMPARE(atOld.size(), size_t(1)); + QCOMPARE(atOld[0], 0.f); + } + + void negative_region_is_silent() { + auto model = makeTake(); + model->setStartFrame(-kImpulseAt); + + // Wholly before the start: nothing, or silence + for (float v : model->getData(0, -kImpulseAt - 100, 50)) { + QCOMPARE(v, 0.f); + } + + // Straddling the start: a short read is acceptable, but whatever + // comes back must be the opening silence of the file, not the + // impulse and not garbage + auto straddling = model->getData(0, -kImpulseAt - 10, 20); + QVERIFY(straddling.size() <= size_t(20)); + for (float v : straddling) { + QCOMPARE(v, 0.f); + } + } + + void end_frame_shifts() { + auto model = makeTake(); + sv::sv_frame_t before = model->getEndFrame(); + model->setStartFrame(-kImpulseAt); + QCOMPARE(model->getEndFrame(), before - kImpulseAt); + } + + void latency_sum() { + QCOMPARE(computeRecordingLatency(512, 256), sv::sv_frame_t(768)); + QCOMPARE(computeRecordingLatency(512, 0), sv::sv_frame_t(512)); + QCOMPARE(computeRecordingLatency(0, 256), sv::sv_frame_t(256)); + QCOMPARE(computeRecordingLatency(0, 0), sv::sv_frame_t(0)); + // An unavailable figure must never pull the take later + QCOMPARE(computeRecordingLatency(-1, 256), sv::sv_frame_t(256)); + QCOMPARE(computeRecordingLatency(-100, -100), sv::sv_frame_t(0)); + } +}; + +#endif diff --git a/main/test/TestRealtimePitchTracker.h b/main/test/TestRealtimePitchTracker.h new file mode 100644 index 00000000..7472d965 --- /dev/null +++ b/main/test/TestRealtimePitchTracker.h @@ -0,0 +1,315 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_REALTIME_PITCH_TRACKER_H +#define TEST_REALTIME_PITCH_TRACKER_H + +// Tier 2: the tracker thread, fed the way the record target feeds the +// model during a take. + +#include "../RealtimePitchTracker.h" + +#include "TestSignals.h" + +#include "data/model/WritableWaveFileModel.h" + +#include +#include +#include +#include + +#include +#include + +// Collects pitchDetected() on the test (GUI) thread through a queued +// connection, which is how MainWindow receives it. QSignalSpy would +// connect directly and be written to from the tracker thread. +class PitchCollector : public QObject +{ + Q_OBJECT + +public: + struct Event { + sv::sv_frame_t frame; + double hz; + }; + + std::vector events; + + PitchCollector(RealtimePitchTracker *tracker) { + connect(tracker, &RealtimePitchTracker::pitchDetected, + this, &PitchCollector::pitchDetected); + } + + int count() const { return int(events.size()); } + +public slots: + void pitchDetected(sv::sv_frame_t frame, double hz) { + events.push_back({ frame, hz }); + } +}; + +class TestRealtimePitchTracker : public QObject +{ + Q_OBJECT + + static constexpr double kRate = 44100.0; + static const int kWindow = 2048; + static const int kHop = 256; + static const int kBlock = 441; // 10 ms, the record target's cadence + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + struct Recording { + std::shared_ptr model; + sv::ModelId id; + int channels = 1; + sv::sv_frame_t written = 0; + + ~Recording() { + if (!id.isNone()) sv::ModelById::release(id); + } + + // Append one channel-count-wide signal in record-sized blocks, + // making each block readable straight away + void append(const std::vector> &channelData) { + int n = int(channelData[0].size()); + for (int start = 0; start < n; start += kBlock) { + int count = std::min(kBlock, n - start); + std::vector ptrs; + for (const auto &c : channelData) { + ptrs.push_back(c.data() + start); + } + model->addSamples(ptrs.data(), count); + model->updateModel(); + } + written += n; + } + + void appendMono(const std::vector &signal) { + append({ signal }); + } + }; + + std::unique_ptr makeRecording(int channels = 1) { + auto r = std::make_unique(); + QString path = m_dir.filePath + (QString("take-%1.wav").arg(++m_fileCounter)); + r->model = std::make_shared + (path, kRate, channels, + sv::WritableWaveFileModel::Normalisation::None); + r->channels = channels; + r->id = sv::ModelById::add(r->model); + return r; + } + + // Wait until the tracker has had the chance to handle every hop in + // n frames: either the expected number of events has arrived, or + // the count has stopped growing + static void settle(PitchCollector &spy, int expected = -1) { + QElapsedTimer timer; + timer.start(); + int last = -1; + int stableFor = 0; + while (timer.elapsed() < 5000) { + QTest::qWait(50); + if (expected >= 0 && spy.count() >= expected) return; + if (spy.count() == last) { + if (++stableFor >= 6) return; + } else { + stableFor = 0; + last = int(spy.count()); + } + } + } + + static int expectedHops(sv::sv_frame_t frames) { + if (frames < kWindow) return 0; + return int((frames - kWindow) / kHop) + 1; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + qRegisterMetaType("sv::sv_frame_t"); + qRegisterMetaType("sv_frame_t"); + } + + void emits_correct_pitch() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate))); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + + QVERIFY(spy.count() > 0); + for (const auto &event : spy.events) { + double hz = event.hz; + QVERIFY2(std::abs(TestSignals::centsBetween(hz, 330.0)) < 10.0, + qPrintable(QString("%1 Hz").arg(hz))); + } + } + + void frame_grid() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate))); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + + QVERIFY(spy.count() > 0); + sv::sv_frame_t previous = -1; + for (const auto &event : spy.events) { + sv::sv_frame_t frame = event.frame; + QVERIFY2(frame >= kWindow / 2 && (frame - kWindow / 2) % kHop == 0, + qPrintable(QString("frame %1 is off the hop grid") + .arg(frame))); + QVERIFY2(frame > previous, + qPrintable(QString("frame %1 after %2") + .arg(frame).arg(previous))); + previous = frame; + } + } + + void coverage() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate))); + int expected = expectedHops(rec->written); + settle(spy, expected); + tracker.stop(); + + QVERIFY2(std::abs(int(spy.count()) - expected) <= 2, + qPrintable(QString("%1 events for %2 hops") + .arg(spy.count()).arg(expected))); + } + + void unvoiced_gap() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + + const int n = int(kRate / 2); + rec->appendMono(TestSignals::sine(330.0, kRate, n)); + rec->appendMono(std::vector(n, 0.f)); + rec->appendMono(TestSignals::sine(330.0, kRate, n, 0.5, 0.0, 2 * n)); + settle(spy); + tracker.stop(); + + bool before = false, after = false; + for (const auto &event : spy.events) { + sv::sv_frame_t centre = event.frame; + sv::sv_frame_t windowStart = centre - kWindow / 2; + sv::sv_frame_t windowEnd = centre + kWindow / 2; + QVERIFY2(!(windowStart >= n && windowEnd <= 2 * n), + qPrintable(QString("event at %1 lies wholly in silence") + .arg(centre))); + if (windowEnd <= n) before = true; + if (windowStart >= 2 * n) after = true; + } + QVERIFY(before); + QVERIFY(after); + } + + void stop_is_prompt() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate) / 4)); + + QElapsedTimer timer; + timer.start(); + tracker.stop(); + QVERIFY2(timer.elapsed() < 200, + qPrintable(QString("stop() took %1 ms").arg(timer.elapsed()))); + QVERIFY(tracker.isFinished()); + } + + void no_events_after_stop() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate) / 2)); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + QCoreApplication::processEvents(); + + auto countAtStop = spy.count(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate) / 2)); + QTest::qWait(100); + QCOMPARE(spy.count(), countAtStop); + } + + void model_released_midway() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate) / 2)); + settle(spy, expectedHops(rec->written)); + + // As when the document lets go of the model. We keep our own + // reference until the tracker has stopped: the tracker holds + // one per iteration, and if that were the last, the model (a + // QObject owning a timer) would be destroyed on its thread. + sv::ModelById::release(rec->id); + rec->id = {}; + + QTest::qWait(100); + QVERIFY(tracker.isRunning()); + tracker.stop(); + QVERIFY(tracker.isFinished()); + } + + void starts_before_data() { + auto rec = makeRecording(); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + QTest::qWait(100); + QCOMPARE(int(spy.count()), 0); + + rec->appendMono(TestSignals::sine(330.0, kRate, int(kRate) / 2)); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + QVERIFY(spy.count() > 0); + } + + void stereo_signal_on_ch1() { + // A stereo interface with the microphone on its second input + auto rec = makeRecording(2); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + const int n = int(kRate) / 2; + rec->append({ std::vector(n, 0.f), + TestSignals::sine(330.0, kRate, n) }); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + + QEXPECT_FAIL("", "Review finding 11: the tracker reads channel 0 only", + Continue); + QVERIFY(spy.count() > 0); + } +}; + +#endif diff --git a/main/test/TestRealtimeYin.h b/main/test/TestRealtimeYin.h new file mode 100644 index 00000000..9b2c81e1 --- /dev/null +++ b/main/test/TestRealtimeYin.h @@ -0,0 +1,221 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_REALTIME_YIN_H +#define TEST_REALTIME_YIN_H + +// Tier 1: the pitch maths of RealtimePitchTracker, without the thread. + +#include "../RealtimePitchTracker.h" + +#include "TestSignals.h" +#include "PyinReference.h" + +#include "bqfft/FFT.h" + +#include +#include + +#include +#include + +class TestRealtimeYin : public QObject +{ + Q_OBJECT + + static const int kWindow = RealtimePitchTracker::kWindowSize; + + static std::vector difference(const std::vector &buf) { + breakfastquay::FFT fft(int(buf.size())); + std::vector diff; + RealtimePitchTracker::yinDifferenceFFT(buf, diff, &fft); + return diff; + } + + static std::vector naiveDifference(const std::vector &buf) { + int half = int(buf.size() / 2); + std::vector diff(half, 0.0); + for (int tau = 1; tau < half; ++tau) { + for (int j = 0; j < half; ++j) { + double delta = double(buf[j]) - double(buf[j + tau]); + diff[tau] += delta * delta; + } + } + return diff; + } + + // One analysis window through the same steps, lag limits and + // range check as RealtimePitchTracker::run(). Returns Hz, or a + // negative value if nothing would have been emitted. + static double detect(const std::vector &buf, double sampleRate, + double minFreq = 60.0, double maxFreq = 1000.0, + double threshold = 0.15) { + std::vector diff = difference(buf); + RealtimePitchTracker::yinCMND(diff); + int minLag = std::max(1, int(std::floor(sampleRate / maxFreq))); + int maxLag = std::min(kWindow / 2 - 2, + int(std::ceil(sampleRate / minFreq))); + double lag = RealtimePitchTracker::yinFindPitch + (diff, minLag, maxLag, threshold); + if (lag <= 0.0) return -1.0; + double hz = sampleRate / lag; + if (hz < minFreq || hz > maxFreq) return -1.0; + return hz; + } + + static void compareDifference(const std::vector &actual, + const std::vector &expected, + double relativeTolerance) { + QCOMPARE(actual.size(), expected.size()); + // Relative to the largest value: near a period the difference + // function approaches zero and a per-value ratio means nothing + double scale = 0.0; + for (double e : expected) scale = std::max(scale, std::abs(e)); + QVERIFY(scale > 0.0); + for (int tau = 1; tau < int(expected.size()); ++tau) { + double err = std::abs(actual[tau] - expected[tau]) / scale; + if (err > relativeTolerance) { + QFAIL(qPrintable(QString("at lag %1: got %2, expected %3, " + "relative error %4") + .arg(tau).arg(actual[tau]) + .arg(expected[tau]).arg(err))); + } + } + } + +private slots: + void sine_accuracy_data() { + QTest::addColumn("hz"); + QTest::addColumn("sampleRate"); + for (double sr : { 44100.0, 48000.0 }) { + for (double hz : { 82.0, 110.0, 220.0, 440.0, 880.0 }) { + QTest::newRow(qPrintable(QString("%1 Hz at %2").arg(hz).arg(sr))) + << hz << sr; + } + } + } + + void sine_accuracy() { + QFETCH(double, hz); + QFETCH(double, sampleRate); + double detected = detect(TestSignals::sine(hz, sampleRate, kWindow), + sampleRate); + QVERIFY2(detected > 0.0, "no pitch detected"); + double cents = TestSignals::centsBetween(detected, hz); + QVERIFY2(std::abs(cents) < 5.0, + qPrintable(QString("detected %1 Hz, %2 cents out") + .arg(detected).arg(cents))); + } + + void harmonic_tone() { + double detected = detect(TestSignals::sawtooth(196.0, 44100.0, kWindow), + 44100.0); + QVERIFY2(detected > 0.0, "no pitch detected"); + double cents = TestSignals::centsBetween(detected, 196.0); + // An octave error would be 1200 cents out + QVERIFY2(std::abs(cents) < 10.0, + qPrintable(QString("detected %1 Hz, %2 cents out") + .arg(detected).arg(cents))); + } + + void silence() { + std::vector zeros(kWindow, 0.f); + QVERIFY(detect(zeros, 44100.0) < 0.0); + } + + void noise() { + QVERIFY(detect(TestSignals::whiteNoise(kWindow, 12345), 44100.0) < 0.0); + } + + void out_of_range_low() { + double detected = detect(TestSignals::sine(40.0, 44100.0, kWindow), + 44100.0); + QVERIFY2(detected < 0.0, + qPrintable(QString("detected %1 Hz").arg(detected))); + } + + void out_of_range_high() { + double detected = detect(TestSignals::sine(2000.0, 44100.0, kWindow), + 44100.0); + QVERIFY2(detected < 0.0, + qPrintable(QString("detected %1 Hz").arg(detected))); + } + + void dc_offset() { + double detected = detect + (TestSignals::sine(220.0, 44100.0, kWindow, 0.5, 0.3), 44100.0); + QVERIFY2(detected > 0.0, "no pitch detected"); + QVERIFY(std::abs(TestSignals::centsBetween(detected, 220.0)) < 10.0); + } + + void low_level() { + // -50 dBFS: guards the precision of the single-precision FFT + double amplitude = std::pow(10.0, -50.0 / 20.0); + double detected = detect + (TestSignals::sine(220.0, 44100.0, kWindow, amplitude), 44100.0); + QVERIFY2(detected > 0.0, "no pitch detected"); + QVERIFY(std::abs(TestSignals::centsBetween(detected, 220.0)) < 10.0); + } + + void diff_matches_pyin_data() { + QTest::addColumn("useNoise"); + QTest::newRow("noise") << true; + QTest::newRow("sine") << false; + } + + void diff_matches_pyin() { + QFETCH(bool, useNoise); + std::vector buf = useNoise ? + TestSignals::whiteNoise(kWindow, 999) : + TestSignals::sine(330.0, 44100.0, kWindow); + std::vector in(buf.begin(), buf.end()); + compareDifference(difference(buf), pyinFastDifference(in), 1e-4); + } + + void diff_matches_naive() { + // Independent of pyin: the textbook O(n^2) sum + // d(tau) = sum_{j buf = TestSignals::whiteNoise(kWindow, 4242); + std::vector expected = naiveDifference(buf); + + // ...except for one known deviation, inherited from pYIN's + // YinUtil::fastDifference. The running power term for lag tau + // should cover x[tau] .. x[tau+half-1], but the recursion + // brings in x[tau+half] at each step rather than + // x[tau+half-1], so the window it sums skips x[half] and takes + // x[half+tau] instead. Harmless for pitch (one sample's energy + // in a thousand), but it has to be allowed for here, and it is + // kept in the implementation so as to stay in step with pYIN. + int half = kWindow / 2; + for (int tau = 1; tau < half; ++tau) { + expected[tau] += double(buf[half + tau]) * double(buf[half + tau]) + - double(buf[half]) * double(buf[half]); + } + + compareDifference(difference(buf), expected, 1e-4); + } + + void cmnd_properties() { + std::vector diff = + difference(TestSignals::sine(220.0, 44100.0, kWindow)); + RealtimePitchTracker::yinCMND(diff); + QCOMPARE(diff[0], 1.0); + for (double d : diff) QVERIFY(d >= 0.0); + + std::vector zeros(kWindow / 2, 0.0); + RealtimePitchTracker::yinCMND(zeros); + for (double d : zeros) QCOMPARE(d, 1.0); + } +}; + +#endif diff --git a/main/test/TestSignals.h b/main/test/TestSignals.h new file mode 100644 index 00000000..8a1e7023 --- /dev/null +++ b/main/test/TestSignals.h @@ -0,0 +1,71 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_SIGNALS_H +#define TEST_SIGNALS_H + +// Synthetic signals and pitch comparison shared by the Tony test suites. + +#include +#include +#include + +namespace TestSignals { + +const double kPi = 3.14159265358979323846; + +inline std::vector +sine(double hz, double sampleRate, int n, double amplitude = 0.5, + double dc = 0.0, int startIndex = 0) +{ + std::vector v(n); + for (int i = 0; i < n; ++i) { + v[i] = float(dc + amplitude * + std::sin(2.0 * kPi * hz * (startIndex + i) / sampleRate)); + } + return v; +} + +/** Naive (aliasing) sawtooth: all harmonics present, fundamental at hz. */ +inline std::vector +sawtooth(double hz, double sampleRate, int n, double amplitude = 0.5) +{ + std::vector v(n); + for (int i = 0; i < n; ++i) { + double phase = std::fmod(hz * i / sampleRate, 1.0); + v[i] = float(amplitude * (2.0 * phase - 1.0)); + } + return v; +} + +/** Deterministic: the same seed always yields the same buffer. */ +inline std::vector +whiteNoise(int n, unsigned seed, double amplitude = 0.5) +{ + std::mt19937 gen(seed); + std::uniform_real_distribution dist(float(-amplitude), + float(amplitude)); + std::vector v(n); + for (int i = 0; i < n; ++i) v[i] = dist(gen); + return v; +} + +inline double +centsBetween(double hz, double referenceHz) +{ + return 1200.0 * std::log2(hz / referenceHz); +} + +} + +#endif diff --git a/main/test/TestSingingDocument.h b/main/test/TestSingingDocument.h new file mode 100644 index 00000000..71d322bf --- /dev/null +++ b/main/test/TestSingingDocument.h @@ -0,0 +1,307 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_SINGING_DOCUMENT_H +#define TEST_SINGING_DOCUMENT_H + +// Tier 3: the Document ownership rules that the singing-track flows +// depend on, and the pane pruning helper built on them. No MainWindow. +// +// Each test starts from the state the application is in after a +// session load: a main model, pane 0, and a time ruler in pane 0 that +// the document's other panes share. + +#include "../PaneUtils.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/ViewManager.h" +#include "layer/Layer.h" +#include "layer/LayerFactory.h" +#include "widgets/CommandHistory.h" +#include "data/model/WritableWaveFileModel.h" + +#include +#include +#include +#include + +#include +#include + +class TestSingingDocument : public QObject +{ + Q_OBJECT + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + sv::ViewManager *m_viewManager = nullptr; + sv::PaneStack *m_paneStack = nullptr; + sv::Document *m_document = nullptr; + sv::Pane *m_mainPane = nullptr; + sv::Layer *m_ruler = nullptr; + + sv::ModelId makeAudioModel() { + QString path = m_dir.filePath + (QString("audio-%1.wav").arg(++m_fileCounter)); + auto model = std::make_shared + (path, 44100.0, 1, + sv::WritableWaveFileModel::Normalisation::None); + std::vector data(4410, 0.25f); + const float *ptr = data.data(); + model->addSamples(&ptr, sv::sv_frame_t(data.size())); + model->writeComplete(); + return sv::ModelById::add(model); + } + + // What openAudio(CreateAdditionalModel) and record() leave behind: + // a new pane holding an imported waveform layer that is the only + // reference to the new model and, optionally, the shared ruler. + struct Extra { + sv::ModelId model; + sv::Layer *waveform = nullptr; + sv::Pane *pane = nullptr; + }; + + Extra addExtraPane(bool withRuler) { + Extra extra; + extra.model = makeAudioModel(); + extra.waveform = m_document->createImportedLayer(extra.model); + extra.pane = m_paneStack->addPane(); + if (withRuler) { + m_document->addLayerToView(extra.pane, m_ruler); + } + if (extra.waveform) { + m_document->addLayerToView(extra.pane, extra.waveform); + } + return extra; + } + + // As Analyser::addWaveform() and loadBackgroundMusic() do + sv::Layer *addSecondReference(sv::ModelId model) { + sv::Layer *layer = m_document->createLayer(sv::LayerFactory::Waveform); + if (!layer) return nullptr; + m_document->setModel(layer, model); + m_document->addLayerToView(m_mainPane, layer); + return layer; + } + + static bool paneHasLayer(sv::Pane *pane, sv::Layer *layer) { + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i) == layer) return true; + } + return false; + } + + bool documentHasLayer(sv::Layer *layer) const { + return m_document->getLayers().count(layer) > 0; + } + + static bool modelAlive(sv::ModelId id) { + return bool(sv::ModelById::get(id)); + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + qRegisterMetaType("Layer*"); + } + + void init() { + m_viewManager = new sv::ViewManager; + m_paneStack = new sv::PaneStack(nullptr, m_viewManager); + m_document = new sv::Document; + m_document->setMainModel(makeAudioModel()); + m_mainPane = m_paneStack->addPane(); + m_ruler = m_document->createMainModelLayer(sv::LayerFactory::TimeRuler); + QVERIFY(m_ruler); + m_document->addLayerToView(m_mainPane, m_ruler); + } + + void cleanup() { + // The document force-deletes its layers from their views, so + // it has to go while the panes still exist + delete m_document; + delete m_paneStack; + delete m_viewManager; + m_document = nullptr; + m_paneStack = nullptr; + m_viewManager = nullptr; + m_mainPane = nullptr; + m_ruler = nullptr; + } + + // --- The rules ------------------------------------------------------- + + void force_delete_releases_sole_model() { + // The mechanism behind review finding 1 + Extra extra = addExtraPane(false); + QVERIFY(extra.waveform); + QVERIFY(modelAlive(extra.model)); + + m_document->deleteLayer(extra.waveform, true); + + QVERIFY(!documentHasLayer(extra.waveform)); + QVERIFY(!modelAlive(extra.model)); + } + + void second_layer_keeps_model_alive() { + // The background-music workaround + Extra extra = addExtraPane(false); + QVERIFY(extra.waveform); + QVERIFY(addSecondReference(extra.model)); + + m_document->deleteLayer(extra.waveform, true); + + QVERIFY(modelAlive(extra.model)); + } + + void detach_keeps_layer_and_model() { + // Document::detachLayerFromView is an addition in the svapp fork + Extra extra = addExtraPane(false); + QVERIFY(extra.waveform); + QSignalSpy commands(sv::CommandHistory::getInstance(), + SIGNAL(commandExecuted())); + + m_document->detachLayerFromView(extra.pane, extra.waveform); + + QVERIFY(!paneHasLayer(extra.pane, extra.waveform)); + QVERIFY(documentHasLayer(extra.waveform)); + QVERIFY(modelAlive(extra.model)); + QCOMPARE(int(commands.count()), 0); + } + + void force_delete_skips_play_source() { + // layerInAView(layer, false) is what takes a model out of the + // play source. A forced delete never emits it (review finding + // 6), so callers have to remove the model themselves. + Extra viaCommand = addExtraPane(false); + Extra viaForce = addExtraPane(false); + QVERIFY(viaCommand.waveform && viaForce.waveform); + QSignalSpy inAView(m_document, + SIGNAL(layerInAView(Layer *, bool))); + QVERIFY(inAView.isValid()); + + m_document->removeLayerFromView(viaCommand.pane, viaCommand.waveform); + QCOMPARE(int(inAView.count()), 1); + QCOMPARE(inAView.at(0).at(1).toBool(), false); + + m_document->deleteLayer(viaForce.waveform, true); + QCOMPARE(int(inAView.count()), 1); + } + + // --- The pruning helper ---------------------------------------------- + + void shared_ruler_survives_prune() { + // Review finding 2 + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + QVERIFY(addSecondReference(extra.model)); + QVERIFY(paneHasLayer(extra.pane, m_ruler)); + + pruneExtraPane(m_document, m_paneStack, extra.pane, extra.model); + + QVERIFY2(documentHasLayer(m_ruler), + "the shared time ruler was deleted from the document"); + QVERIFY(paneHasLayer(m_mainPane, m_ruler)); + } + + void prune_deletes_orphan_and_pane() { + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + sv::Layer *kept = addSecondReference(extra.model); + QVERIFY(kept); + QCOMPARE(m_paneStack->getPaneCount(), 2); + QSignalSpy commands(sv::CommandHistory::getInstance(), + SIGNAL(commandExecuted())); + + pruneExtraPane(m_document, m_paneStack, extra.pane, extra.model); + + QCOMPARE(m_paneStack->getPaneCount(), 1); + QCOMPARE(m_paneStack->getHiddenPaneCount(), 0); + QVERIFY(!documentHasLayer(extra.waveform)); + QVERIFY(documentHasLayer(kept)); + QVERIFY(modelAlive(extra.model)); + // Nothing undoable may be left pointing at the dead pane + QCOMPARE(int(commands.count()), 0); + } + + void prune_without_second_ref_releases_model() { + // What loadSingingTrack() relies on when the analyser could not + // be set up: pruning is then what disposes of the model + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + + pruneExtraPane(m_document, m_paneStack, extra.pane, extra.model); + + QVERIFY(!modelAlive(extra.model)); + QCOMPARE(m_paneStack->getPaneCount(), 1); + } + + void prune_none_id_deletes_nothing() { + // The ruler's own model id is "none". An empty owned id must + // not be taken to match it. + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + + pruneExtraPane(m_document, m_paneStack, extra.pane, sv::ModelId()); + + QVERIFY2(documentHasLayer(m_ruler), + "the shared time ruler was deleted from the document"); + QVERIFY(paneHasLayer(m_mainPane, m_ruler)); + QVERIFY(documentHasLayer(extra.waveform)); + QVERIFY(modelAlive(extra.model)); + QCOMPARE(m_paneStack->getPaneCount(), 1); + } + + void prune_hidden_pane() { + // The record flow hides the pane first and prunes it later + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + QVERIFY(addSecondReference(extra.model)); + m_paneStack->hidePane(extra.pane); + QCOMPARE(m_paneStack->getPaneCount(), 1); + QCOMPARE(m_paneStack->getHiddenPaneCount(), 1); + + pruneExtraPane(m_document, m_paneStack, extra.pane, extra.model); + + QCOMPARE(m_paneStack->getHiddenPaneCount(), 0); + QVERIFY2(documentHasLayer(m_ruler), + "the shared time ruler was deleted from the document"); + QVERIFY(paneHasLayer(m_mainPane, m_ruler)); + QVERIFY(!documentHasLayer(extra.waveform)); + QVERIFY(modelAlive(extra.model)); + } + + void layer_view_map_clean_after_prune() { + // If pruning left the dead pane in the document's layer-to-view + // map, force-deleting the ruler would call into the freed + // widget. Only a memory checker makes that failure certain; + // without one this at least exercises the path. + Extra extra = addExtraPane(true); + QVERIFY(extra.waveform); + QVERIFY(addSecondReference(extra.model)); + + pruneExtraPane(m_document, m_paneStack, extra.pane, extra.model); + + QVERIFY2(documentHasLayer(m_ruler), + "the shared time ruler was deleted from the document"); + m_document->deleteLayer(m_ruler, true); + QVERIFY(!paneHasLayer(m_mainPane, m_ruler)); + m_ruler = nullptr; + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp new file mode 100644 index 00000000..f2f1c19a --- /dev/null +++ b/main/test/tony-app-test.cpp @@ -0,0 +1,61 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TestSingingDocument.h" + +#include "RunSuite.h" + +#include "system/Init.h" + +#include +#include + +#include + +using namespace std; +using namespace sv; + +int main(int argc, char *argv[]) +{ + int good = 0, bad = 0; + + svSystemSpecificInitialisation(); + + // Widgets are created but never shown. meson runs this with + // QT_QPA_PLATFORM=offscreen; default to that when run by hand too. + if (qEnvironmentVariableIsEmpty("QT_QPA_PLATFORM")) { + qputenv("QT_QPA_PLATFORM", "offscreen"); + } + + // Names distinct from the application's, so that nothing here reads + // or writes the user's real Tony settings + QApplication app(argc, argv); + app.setOrganizationName("tony-tests"); + app.setApplicationName("test-tony-app"); + + { + TestSingingDocument t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + (void)good; + + if (bad > 0) { + SVCERR << "\n********* " << bad << " test suite(s) failed!\n" << endl; + return 1; + } else { + SVCERR << "All tests passed" << endl; + return 0; + } +} diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp new file mode 100644 index 00000000..4471d199 --- /dev/null +++ b/main/test/tony-core-test.cpp @@ -0,0 +1,68 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TestRealtimeYin.h" +#include "TestRealtimePitchTracker.h" +#include "TestLatencyShift.h" + +#include "RunSuite.h" + +#include "system/Init.h" + +#include + +#include + +using namespace std; +using namespace sv; + +int main(int argc, char *argv[]) +{ + int good = 0, bad = 0; + + svSystemSpecificInitialisation(); + + // Names distinct from the application's, so that nothing here reads + // or writes the user's real Tony settings + QCoreApplication app(argc, argv); + app.setOrganizationName("tony-tests"); + app.setApplicationName("test-tony-core"); + + { + TestRealtimeYin t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + { + TestRealtimePitchTracker t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + { + TestLatencyShift t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + (void)good; + + if (bad > 0) { + SVCERR << "\n********* " << bad << " test suite(s) failed!\n" << endl; + return 1; + } else { + SVCERR << "All tests passed" << endl; + return 0; + } +} diff --git a/meson.build b/meson.build index 4ea289bd..96873479 100644 --- a/meson.build +++ b/meson.build @@ -1085,19 +1085,34 @@ chp_plugin = shared_library( install: true, ) -tony_main_files = [ +# main.cpp is kept apart from the rest of main/ so that the rest can be +# built once, as static libraries, and linked into both the application +# and the test executables (see main/test/). +tony_entry_files = [ 'main/main.cpp', +] + +# No GUI dependencies: usable from a QCoreApplication test. +tony_core_files = [ + 'main/RealtimePitchTracker.cpp', +] + +tony_app_files = [ 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', - 'main/RealtimePitchTracker.cpp', + 'main/PaneUtils.cpp', ] -tony_main_moc_files = qt.preprocess( +tony_core_moc_files = qt.preprocess( + moc_headers: [ + 'main/RealtimePitchTracker.h', +]) + +tony_app_moc_files = qt.preprocess( moc_headers: [ 'main/MainWindow.h', 'main/Analyser.h', - 'main/RealtimePitchTracker.h', ]) qt_resource_files = qt.preprocess( @@ -1152,16 +1167,61 @@ install_data( install_dir : get_option('datadir') / 'icons/hicolor/scalable/apps', ) -executable( - tony_main_name, - qt_resource_files, +tony_core_lib = static_library( + 'tonycore', + tony_core_moc_files, + tony_core_files, + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + ], + cpp_args: [ + feature_defines, + general_defines, + ], +) + +tony_core_dep = declare_dependency( + link_with: tony_core_lib, + include_directories: [ 'main' ], +) + +tony_app_lib = static_library( + 'tonyapp', svgui_moc_files, svapp_moc_files, - tony_main_moc_files, + tony_app_moc_files, svgui_files, svapp_files, - tony_main_files, + tony_app_files, + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + ], + cpp_args: [ + feature_defines, + general_defines, + ], +) + +tony_app_dep = declare_dependency( + link_with: tony_app_lib, + include_directories: [ 'main' ], +) + +# link_whole so that the application contains exactly the objects it did +# when these sources were compiled straight into it. +executable( + tony_main_name, + qt_resource_files, + tony_entry_files, rc, + link_whole: [ + tony_app_lib, + tony_core_lib, + ], dependencies: [ svcore_dep, qt_dep, @@ -1268,6 +1328,75 @@ svcore_data_fileio_test_exe = executable( win_subsystem: 'console' ) +# Tony's own tests. Two executables, because they differ in cost and in +# what they link: test-tony-core needs no GUI and no audio device; +# test-tony-app creates widgets (offscreen) and links svgui and svapp. + +tony_core_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestRealtimeYin.h', + 'main/test/TestRealtimePitchTracker.h', + 'main/test/TestLatencyShift.h', +]) + +tony_core_test_exe = executable( + 'test-tony-core', + tony_core_test_moc_files, + 'main/test/PyinReference.cpp', + 'pyin/YinUtil.cpp', + 'vamp-plugin-sdk/src/vamp-sdk/FFT.cpp', + 'main/test/tony-core-test.cpp', + dependencies: [ + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' +) + +tony_app_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestSingingDocument.h', +]) + +tony_app_test_exe = executable( + 'test-tony-app', + tony_app_test_moc_files, + 'main/test/tony-app-test.cpp', + dependencies: [ + tony_app_dep, + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' +) + +test('tony-core', tony_core_test_exe) +test('tony-app', tony_app_test_exe, + env: [ 'QT_QPA_PLATFORM=offscreen' ]) + test('svcore-base', svcore_base_test_exe) test('svcore-system', svcore_system_test_exe) test('svcore-data-model', svcore_data_model_test_exe) From 7fc4de83dd76656a52824315e8a7bea50201f038 Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 18 Sep 2026 21:51:56 +0300 Subject: [PATCH 014/275] repoint: redirect svcore to the jhhr fork with tony customizations svcore needed a local patch to build against Qt 6.11 / GCC 16 (QString::arg no longer takes std::atomic implicitly), so it can no longer track sonic-visualiser/svcore directly. Forked to jhhr/svcore, branch tony-customizations, as svapp, svgui and bqaudiostream already are. Pins: - svcore 2b88187: backport of the three header hunks from upstream 2dee776. - svgui 271db78: same class of Qt 6.11 break, an int() cast in Pane.cpp's debug-only modifierNames(). Co-Authored-By: Claude Fable 5.1 --- repoint-lock.json | 4 ++-- repoint-project.json | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/repoint-lock.json b/repoint-lock.json index 3df4ff91..a8e2cf1e 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -4,10 +4,10 @@ "pin": "d7ceb7d1d490674c93d334e5378108c4328e9e05" }, "svcore": { - "pin": "65eeb62d0c966f4c15db78ba28eab8d4a8ea0274" + "pin": "2b881872b91be457fc3c9f5db12398be750622e3" }, "svgui": { - "pin": "8516f9010dded9bb1303291499155a0717297bf0" + "pin": "271db78f54eff467e3891fca920ab663cc69ce53" }, "svapp": { "pin": "d67336efd0d3777442f450f7ff1dc680f6b0f608" diff --git a/repoint-project.json b/repoint-project.json index 5d76dd04..77997f54 100644 --- a/repoint-project.json +++ b/repoint-project.json @@ -11,8 +11,8 @@ "svcore": { "vcs": "git", "service": "github", - "owner": "sonic-visualiser", - "branch": "default" + "owner": "jhhr", + "branch": "tony-customizations" }, "svgui": { "vcs": "git", From 4f69a67121cc32ce5648ab74aa0395334d215e51 Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 18 Sep 2026 22:12:11 +0300 Subject: [PATCH 015/275] repoint: bump svcore for fileio test output gitignore Co-Authored-By: Claude Opus 5 --- repoint-lock.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoint-lock.json b/repoint-lock.json index a8e2cf1e..d8299f5e 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -4,7 +4,7 @@ "pin": "d7ceb7d1d490674c93d334e5378108c4328e9e05" }, "svcore": { - "pin": "2b881872b91be457fc3c9f5db12398be750622e3" + "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { "pin": "271db78f54eff467e3891fca920ab663cc69ce53" From c306e116ae325c8677884073fac2d6b0d6f9054d Mon Sep 17 00:00:00 2001 From: jhhr Date: Fri, 18 Sep 2026 23:23:19 +0300 Subject: [PATCH 016/275] test: Tier 4 (pYIN analysis) and Tier 5 (record workflow on a fake device) Tier 4, TestSingingAnalysis: the Analyser running the real pYIN plugin on synthetic audio, in primary and secondary mode, and the latency shift applied before analysis. The tony-app test now depends on the pyin plugin target, which nothing else was building. Tier 5, TestRecordWorkflow: the real MainWindow recording from FakeAudioIO, a duplex device on a worker thread that reports the latencies a test asks for and delivers a programmed input. Open review findings held by these tests: - 4, 5, 13, 14: QEXPECT_FAIL - 15 (closing the session during the take's analysis can crash): QSKIP - 3 was not reproduced; two passing tests guard it Co-Authored-By: Claude Fable 5.1 --- main/test/FakeAudioIO.h | 257 +++++++++ main/test/TestRecordWorkflow.h | 930 ++++++++++++++++++++++++++++++++ main/test/TestSingingAnalysis.h | 464 ++++++++++++++++ main/test/tony-app-test.cpp | 21 + meson.build | 4 + 5 files changed, 1676 insertions(+) create mode 100644 main/test/FakeAudioIO.h create mode 100644 main/test/TestRecordWorkflow.h create mode 100644 main/test/TestSingingAnalysis.h diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h new file mode 100644 index 00000000..34dd72e1 --- /dev/null +++ b/main/test/FakeAudioIO.h @@ -0,0 +1,257 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_FAKE_AUDIO_IO_H +#define TEST_FAKE_AUDIO_IO_H + +// A duplex audio device with no hardware behind it. A worker thread +// runs the callback in real time, as a driver would: each block it +// pulls the application's output and pushes a programmed input. +// +// The device reports whatever latencies the test asks for, so the +// application computes a known compensation, and the input can be +// made to arrive late by exactly that much. + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +class FakeAudioIO : public breakfastquay::SystemAudioIO +{ +public: + struct Config { + int sampleRate = 44100; + int blockSize = 512; + int channels = 2; + + // Reported to the application, and nothing else: the delay the + // input really has is inputDelay + int recordLatency = 0; + int playbackLatency = 0; + + // Mono input, delivered once and followed by silence. The + // input clock restarts whenever the device is resumed + std::vector input; + + // Frames of silence before the input + int inputDelay = 0; + + // Start the input clock at the first audible output sample + // instead of at resume. With inputDelay equal to the reported + // round trip, this is a singer who is exactly on time + bool inputFollowsPlayback = false; + + // Add the output to the input, inputDelay frames late: + // speakers bleeding into the microphone + bool loopback = false; + + // Whether the application keeps the input it is given just + // now. It discards input until its recording file is open, + // which is some time after it resumes the device. If unset, + // all input counts as kept + std::function inputIsKept; + }; + + FakeAudioIO(breakfastquay::ApplicationRecordTarget *target, + breakfastquay::ApplicationPlaybackSource *source, + Config config) : + SystemAudioIO(target, source), + m_config(config), + m_suspended(true), + m_stop(false), + m_clockRunning(false), + m_clock(0), + m_frames(0), + m_playStartFrame(-1), + m_sinceResume(0), + m_framesBeforePlayStart(-1), + m_resumeCount(0) + { + m_source->setSystemPlaybackBlockSize(m_config.blockSize); + m_source->setSystemPlaybackSampleRate(m_config.sampleRate); + m_source->setSystemPlaybackChannelCount(m_config.channels); + m_source->setSystemPlaybackLatency(m_config.playbackLatency); + + m_target->setSystemRecordBlockSize(m_config.blockSize); + m_target->setSystemRecordSampleRate(m_config.sampleRate); + m_target->setSystemRecordChannelCount(m_config.channels); + m_target->setSystemRecordLatency(m_config.recordLatency); + + m_thread = std::thread([this]() { run(); }); + } + + ~FakeAudioIO() override { + m_stop = true; + m_thread.join(); + } + + bool isSourceOK() const override { return true; } + bool isTargetOK() const override { return true; } + + double getCurrentTime() const override { + return double(m_frames.load()) / m_config.sampleRate; + } + + void suppressRecordSide(bool) override { } + + // No callback is running, or will start, once this returns + void suspend() override { + std::lock_guard guard(m_mutex); + m_suspended = true; + } + + void resume() override { + std::lock_guard guard(m_mutex); + if (!m_suspended) return; + m_suspended = false; + m_clock = 0; + m_clockRunning = !m_config.inputFollowsPlayback; + m_playStartFrame = -1; + m_sinceResume = 0; + m_framesBeforePlayStart = -1; + ++m_resumeCount; + } + + bool isSuspended() const { + std::lock_guard guard(m_mutex); + return m_suspended; + } + + int getResumeCount() const { + std::lock_guard guard(m_mutex); + return m_resumeCount; + } + + /** Everything the application has played, mixed to mono. */ + std::vector getCapturedOutput() const { + std::lock_guard guard(m_mutex); + return m_captured; + } + + /** + * Index into the captured output of the first audible sample + * since the last resume, or -1 if there has been none. + */ + long getPlayStartFrame() const { + std::lock_guard guard(m_mutex); + return m_playStartFrame; + } + + /** + * Number of input frames the application kept between the last + * resume and the first audible output sample, or -1: how far into + * the take the reference started. + */ + long getFramesBeforePlayStart() const { + std::lock_guard guard(m_mutex); + return m_framesBeforePlayStart; + } + +private: + Config m_config; + mutable std::mutex m_mutex; + std::thread m_thread; + bool m_suspended; + std::atomic m_stop; + bool m_clockRunning; + long m_clock; + std::atomic m_frames; + long m_playStartFrame; + long m_sinceResume; + long m_framesBeforePlayStart; + int m_resumeCount; + std::vector m_captured; + + void run() { + using namespace std::chrono; + auto period = duration_cast + (duration(double(m_config.blockSize) / + m_config.sampleRate)); + auto next = steady_clock::now(); + while (!m_stop) { + next += period; + std::this_thread::sleep_until(next); + std::lock_guard guard(m_mutex); + if (m_suspended) { + // don't try to catch up on the time spent suspended + next = steady_clock::now(); + continue; + } + process(); + } + } + + float inputAt(long clock) const { + long i = clock - m_config.inputDelay; + if (i < 0 || i >= long(m_config.input.size())) return 0.f; + return m_config.input[size_t(i)]; + } + + void process() { + const int n = m_config.blockSize; + const int ch = m_config.channels; + + std::vector> out(ch, std::vector(n, 0.f)); + std::vector outPtrs; + for (auto &v : out) outPtrs.push_back(v.data()); + int got = m_source->getSourceSamples(outPtrs.data(), ch, n); + + bool kept = !m_config.inputIsKept || m_config.inputIsKept(); + + long base = long(m_captured.size()); + for (int i = 0; i < n; ++i) { + float mix = 0.f; + if (i < got) { + for (int c = 0; c < ch; ++c) mix += out[c][i]; + mix /= float(ch); + } + m_captured.push_back(mix); + if (m_playStartFrame < 0 && std::fabs(mix) > 1e-4f) { + m_playStartFrame = base + i; + m_framesBeforePlayStart = m_sinceResume + (kept ? i : 0); + } + } + + std::vector in(n, 0.f); + for (int i = 0; i < n; ++i) { + if (!m_clockRunning && m_playStartFrame >= 0 && + base + i >= m_playStartFrame) { + m_clockRunning = true; + } + if (m_clockRunning) { + in[i] = inputAt(m_clock); + ++m_clock; + } + if (m_config.loopback) { + long j = base + i - m_config.inputDelay; + if (j >= 0) in[i] += m_captured[size_t(j)]; + } + } + + std::vector inPtrs(ch, in.data()); + m_target->putSamples(inPtrs.data(), ch, n); + if (kept) m_sinceResume += n; + + m_frames += n; + } +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h new file mode 100644 index 00000000..71a510f7 --- /dev/null +++ b/main/test/TestRecordWorkflow.h @@ -0,0 +1,930 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_RECORD_WORKFLOW_H +#define TEST_RECORD_WORKFLOW_H + +// Tier 5: the real MainWindow, recording from a fake audio device +// (FakeAudioIO.h) that runs in real time. Takes are about a second +// each, so this suite is the slow one. +// +// A take that works end to end shows no dialog, and a dialog would +// block the test for ever. A watchdog timer dismisses any modal +// dialog and records it; each test then fails in cleanup(). + +#include "TestSignals.h" +#include "FakeAudioIO.h" + +#include "../MainWindow.h" +#include "../Analyser.h" + +#include "version.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/ViewManager.h" +#include "layer/Layer.h" +#include "layer/ColourDatabase.h" +#include "layer/SingleColourLayer.h" +#include "layer/TimeValueLayer.h" +#include "layer/WaveformLayer.h" +#include "audio/AudioCallbackPlaySource.h" +#include "audio/AudioCallbackRecordTarget.h" +#include "data/model/WritableWaveFileModel.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/fileio/WavFileWriter.h" +#include "base/PlayParameters.h" +#include "base/RecordDirectory.h" +#include "transform/ModelTransformerFactory.h" +#include "widgets/InteractiveFileFinder.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +/** + * MainWindow with the fake device in place of a real one, and the + * protected state of the singing workflow opened up for inspection. + */ +class TestMainWindow : public MainWindow +{ +public: + TestMainWindow(FakeAudioIO::Config config, bool installDevice = true) : + MainWindow(AUDIO_PLAYBACK_AND_RECORD, true, false), + m_fakeConfig(config), + m_installDevice(installDevice) { } + + FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } + + void doRecord() { record(); } + void doAnalyseNow() { analyseNow(); } + void doLoadBackgroundMusic(QString path) { loadBackgroundMusic(path); } + + // As answering "No" to "do you want to save?" + void discardModifications() { m_documentModified = false; } + void doCloseSession() { discardModifications(); closeSession(); } + + void setPlayReferenceWhileRecording(bool on) { + m_playRefWhileRecording->setChecked(on); + } + + Analyser *analyser() { return m_analyser; } + Analyser *analyser2() { return m_analyser2; } + sv::Document *document() { return m_document; } + sv::PaneStack *paneStack() { return m_paneStack; } + sv::Layer *timeRuler() { return m_timeRulerLayer; } + sv::AudioCallbackRecordTarget *recordTarget() { return m_recordTarget; } + sv::AudioCallbackPlaySource *playSource() { return m_playSource; } + sv::ModelId mainModelId() { return getMainModelId(); } + + RealtimePitchTracker *realtimeTracker() { return m_realtimePitchTracker; } + sv::TimeValueLayer *realtimeLayer() { return m_realtimePitchLayer; } + sv::ModelId realtimeModelId() { return m_realtimePitchModelId; } + sv::ModelId currentRecordingModelId() { return m_currentRecordingModelId; } + sv::ModelId pendingSingingModelId() { return m_pendingSingingModelId; } + sv::ModelId backgroundMusicModelId() { return m_backgroundMusicModelId; } + sv::WaveformLayer *backgroundMusicLayer() { return m_backgroundMusicLayer; } + bool recordingInProgress() { return m_recordingInProgress; } + bool recordingAsSingingTrack() { return m_recordingAsSingingTrack; } + sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } + int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } + +protected: + void createAudioIO() override { + if (m_audioIO || m_playTarget) return; + if (!m_installDevice) return; + m_fakeConfig.inputIsKept = [this]() { + return m_recordTarget->isRecording(); + }; + m_audioIO = new FakeAudioIO + (m_recordTarget, m_playSource->getApplicationPlaybackSource(), + m_fakeConfig); + m_playSource->setSystemPlaybackTarget(m_audioIO); + } + + // The base class deleteAudioIO() deletes m_audioIO, which is right + // for the fake as well + +private: + FakeAudioIO::Config m_fakeConfig; + bool m_installDevice; +}; + +class TestRecordWorkflow : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + static constexpr int hop = 256; // as set in Analyser::addAnalyses() + + // Whole numbers of samples per period: see TestSingingAnalysis.h + static constexpr double lowHz = 220.5; + static constexpr double highHz = 294.0; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + TestMainWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + + static std::vector tone(double hz, double seconds) { + return TestSignals::sawtooth(hz, rate, int(seconds * rate), 0.5f); + } + + // A low note then a high one. The step between them is the event + // whose position the latency test compares across the two tracks + static std::vector melody(double secondsPerNote) { + auto m = tone(lowHz, secondsPerNote); + auto high = tone(highHz, secondsPerNote); + m.insert(m.end(), high.begin(), high.end()); + return m; + } + + QString writeWav(const std::vector &data) { + QString path = m_dir.filePath + (QString("audio-%1.wav").arg(++m_fileCounter)); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + if (!writer.isOK() || + !writer.writeSamples(&ptr, sv::sv_frame_t(data.size())) || + !writer.close()) { + return {}; + } + return path; + } + + void makeWindow(FakeAudioIO::Config config, bool installDevice = true) { + delete m_window; + m_window = new TestMainWindow(config, installDevice); + } + + // Complete, and with the transform threads gone as well. A model + // released while a transform thread still holds it is destroyed + // on that thread, which is review finding 15; no test but the one + // about that finding should depend on it + static bool analysed(Analyser *a) { + return a && a->getLayer(Analyser::PitchTrack) && + a->getLayer(Analyser::Notes) && + a->getInitialAnalysisCompletion() >= 100 && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + } + + // Open path as the reference, replacing whatever is loaded, and + // wait for pYIN + void openReference(QString path) { + QVERIFY(!path.isEmpty()); + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + } + + void startTake() { + QVERIFY(!m_window->recordTarget()->isRecording()); + m_window->doRecord(); + QVERIFY(m_window->recordTarget()->isRecording()); + } + + // Stop, then wait for pYIN on the take + void stopTake() { + QVERIFY(m_window->recordTarget()->isRecording()); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + } + + void take(int ms) { + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(ms); + stopTake(); + } + + static sv::EventVector pitchEvents(sv::Layer *layer) { + if (!layer) return {}; + auto model = sv::ModelById::getAs + (layer->getModel()); + if (!model) return {}; + return model->getAllEvents(); + } + + static sv::EventVector pitchEvents(Analyser *a) { + return a ? pitchEvents(a->getLayer(Analyser::PitchTrack)) + : sv::EventVector(); + } + + static double medianHz(const sv::EventVector &events) { + std::vector values; + for (const auto &e : events) values.push_back(e.getValue()); + if (values.empty()) return 0.0; + std::sort(values.begin(), values.end()); + return values[values.size() / 2]; + } + + // Frame of the first event above the midpoint of the two notes, + // or -1 + static sv::sv_frame_t stepFrame(const sv::EventVector &events) { + double mid = std::sqrt(lowHz * highHz); + for (const auto &e : events) { + if (e.getValue() > mid) return e.getFrame(); + } + return -1; + } + + static int colourOf(sv::Layer *layer) { + auto scl = qobject_cast(layer); + return scl ? scl->getBaseColour() : -1; + } + + static int colourNamed(QString name) { + return sv::ColourDatabase::getInstance()->getColourIndex(name); + } + + // Amplitude of the hz component of data[from, from+n) + static double amplitudeAt(const std::vector &data, + size_t from, size_t n, double hz) { + if (from + n > data.size()) return -1.0; + double re = 0.0, im = 0.0; + for (size_t i = 0; i < n; ++i) { + double phase = 2.0 * TestSignals::kPi * hz * double(i) / rate; + re += data[from + i] * std::cos(phase); + im += data[from + i] * std::sin(phase); + } + return 2.0 * std::sqrt(re * re + im * im) / double(n); + } + + int layersOnModel(sv::ModelId id) { + int n = 0; + for (sv::Layer *layer : m_window->document()->getLayers()) { + if (layer->getModel() == id) ++n; + } + return n; + } + + bool paneHasLayer(int paneIndex, sv::Layer *layer) { + sv::Pane *pane = m_window->paneStack()->getPane(paneIndex); + if (!pane) return false; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i) == layer) return true; + } + return false; + } + + bool documentHasLayer(sv::Layer *layer) { + for (sv::Layer *l : m_window->document()->getLayers()) { + if (l == layer) return true; + } + return false; + } + + // Save a session holding only the reference, and open it again. + // Only a session load sets MainWindowBase::m_timeRulerLayer, which + // is what puts the shared ruler into the extra panes + void reopenAsSession() { + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QString session = m_dir.filePath + (QString("session-%1.ton").arg(++m_fileCounter)); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + openReference(session); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_window->timeRuler(), + "the session load did not find the time ruler"); + QVERIFY(paneHasLayer(1, m_window->timeRuler())); + } + + void verifyRulerIntact() { + QVERIFY(m_window->timeRuler()); + QVERIFY2(documentHasLayer(m_window->timeRuler()), + "the shared time ruler was deleted from the document"); + QVERIFY2(paneHasLayer(1, m_window->timeRuler()), + "the shared time ruler is no longer in the ruler pane"); + } + + // Not a slot: QtTest would run it as a test + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + } + m_dialogs.push_back(description); + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + QSettings().clear(); + + // Otherwise the MainWindow constructor asks, in a dialog + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + // As main() does; without it a .ton file is not a session + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + + // Takes go here and not among the user's recordings + sv::RecordDirectory::setRecordContainerDirectory + (m_dir.filePath("recorded")); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", false); + settings.endGroup(); + } + + void cleanup() { + if (m_window) { + if (m_window->recordTarget()->isRecording()) { + m_window->doRecord(); + } + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + sv::RecordDirectory::setRecordContainerDirectory(""); + } + + void record_creates_singing_track() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + int panes = m_window->paneStack()->getPaneCount(); + QCOMPARE(panes, 2); // analysis pane and ruler strip + QVERIFY(!m_window->analyser2()); + + take(1000); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + sv::ModelId singing = a2->getMainModelId(); + QVERIFY(singing != m_window->mainModelId()); + auto wave = sv::ModelById::getAs(singing); + QVERIFY2(wave, "the singing model is not a WritableWaveFileModel"); + QVERIFY(wave->getFrameCount() > sv::sv_frame_t(0.8 * rate)); + + QCOMPARE(colourOf(a2->getLayer(Analyser::PitchTrack)), + colourNamed("Orange")); + QCOMPARE(colourOf(a2->getLayer(Analyser::Notes)), + colourNamed("Bright Purple")); + QVERIFY(paneHasLayer(0, a2->getLayer(Analyser::PitchTrack))); + QVERIFY(paneHasLayer(0, a2->getLayer(Analyser::Notes))); + + auto events = pitchEvents(a2); + QVERIFY(events.size() > 50); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(events), highHz)) < 10.0); + + // and the reference was left alone + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + } + + void live_dots_appear() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(1000); + + QVERIFY(m_window->realtimeTracker()); + QVERIFY(m_window->realtimeLayer()); + QVERIFY(paneHasLayer(0, m_window->realtimeLayer())); + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + auto events = model->getAllEvents(); + QVERIFY2(events.size() > 20, + qPrintable(QString("only %1 live dots after a second") + .arg(events.size()))); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(events), highHz)) < 10.0); + + stopTake(); + } + + void live_dots_removed() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(800); + sv::ModelId liveModel = m_window->realtimeModelId(); + QVERIFY(!liveModel.isNone()); + stopTake(); + if (QTest::currentTestFailed()) return; + + QVERIFY(!m_window->realtimeTracker()); + QVERIFY(!m_window->realtimeLayer()); + QVERIFY(m_window->realtimeModelId().isNone()); + QVERIFY2(!sv::ModelById::get(liveModel), + "the live pitch model outlived its layer"); + QVERIFY(!m_window->recordingInProgress()); + QVERIFY(!m_window->recordingAsSingingTrack()); + } + + // The automated latency test. The device reports a round trip of K + // frames, and its input is the reference melody starting K frames + // after the first reference sample is played: a singer exactly on + // time. After the take the two pitch tracks should line up. + void latency_end_to_end() { + const int K = 3 * 4096; + FakeAudioIO::Config config; + config.playbackLatency = 2 * 4096; + config.recordLatency = 4096; + config.input = melody(0.75); + config.inputDelay = K; + config.inputFollowsPlayback = true; + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(melody(0.75))); + if (QTest::currentTestFailed()) return; + + take(2200); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, + "the reference was never played"); + + auto wave = sv::ModelById::getAs + (m_window->analyser2()->getMainModelId()); + QVERIFY(wave); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(-K)); + + sv::sv_frame_t refStep = stepFrame(pitchEvents(m_window->analyser())); + sv::sv_frame_t sungStep = stepFrame(pitchEvents(m_window->analyser2())); + QVERIFY(refStep > 0); + QVERIFY2(sungStep > 0, "the take never reached the second note"); + + sv::sv_frame_t error = sungStep - refStep; + sv::sv_frame_t gap = m_window->fake()->getFramesBeforePlayStart(); + QString detail = QString("sung step at %1, reference step at %2: " + "%3 frames (%4 ms) apart; the reference " + "started %5 frames into the take") + .arg(sungStep).arg(refStep).arg(error) + .arg(1000.0 * double(error) / rate, 0, 'f', 1).arg(gap); + + // What is left over is the time between the start of the take + // and the start of the reference, and nothing else + QVERIFY2(std::llabs(error - gap) <= 2 * hop, qPrintable(detail)); + + QEXPECT_FAIL("", "Review finding 14: the take starts before the " + "reference does, and the gap is not compensated", + Continue); + QVERIFY2(std::llabs(error) <= 2 * hop, qPrintable(detail)); + } + + void latency_zero_when_toggle_off() { + FakeAudioIO::Config config; + config.playbackLatency = 4096; + config.recordLatency = 4096; + config.input = tone(highHz, 3.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(false); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(0)); + auto wave = sv::ModelById::getAs + (m_window->analyser2()->getMainModelId()); + QVERIFY(wave); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); + QCOMPARE(m_window->fake()->getPlayStartFrame(), -1L); + } + + // The take and the live pitch model are both in the play source + // while the reference plays (review finding 3), but neither is + // heard: the play source fills its buffers seconds ahead of the + // playback position, and the take has no audio that far ahead yet. + // This test holds that in place. + // + // Pure tones here, so that the reference has nothing at the + // frequency of the input. 26400 samples is a whole number of + // periods of both. + void no_self_monitoring() { + FakeAudioIO::Config config; + config.input = TestSignals::sine(highHz, rate, int(3 * rate), 0.5); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(TestSignals::sine(lowHz, rate, + int(3 * rate), 0.5))); + if (QTest::currentTestFailed()) return; + + take(1500); + if (QTest::currentTestFailed()) return; + + auto output = m_window->fake()->getCapturedOutput(); + long start = m_window->fake()->getPlayStartFrame(); + QVERIFY2(start >= 0, "the reference was never played"); + size_t from = size_t(start) + size_t(0.5 * rate); + double reference = amplitudeAt(output, from, 26400, lowHz); + double input = amplitudeAt(output, from, 26400, highHz); + QVERIFY2(reference > 0.1, + qPrintable(QString("reference amplitude in the output is %1") + .arg(reference))); + QVERIFY2(input < 0.005, + qPrintable(QString("the output has the input's frequency " + "in it, at amplitude %1").arg(input))); + } + + // As above with the take's own waveform muted, leaving only the + // synth that follows the live pitch model + void no_synth_tone() { + FakeAudioIO::Config config; + config.input = TestSignals::sine(highHz, rate, int(3 * rate), 0.5); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(TestSignals::sine(lowHz, rate, + int(3 * rate), 0.5))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && + m_window->analyser2()->getLayer + (Analyser::Audio), 2000); + auto params = m_window->analyser2()->getLayer(Analyser::Audio) + ->getPlayParameters(); + QVERIFY(params); + params->setPlayAudible(false); + QTest::qWait(1800); + stopTake(); + if (QTest::currentTestFailed()) return; + + auto output = m_window->fake()->getCapturedOutput(); + long start = m_window->fake()->getPlayStartFrame(); + QVERIFY2(start >= 0, "the reference was never played"); + size_t from = size_t(start) + size_t(0.8 * rate); + double synth = amplitudeAt(output, from, 26400, highHz); + QVERIFY2(synth >= 0.0 && synth < 0.005, + qPrintable(QString("the output has a tone at the live " + "pitch, at amplitude %1").arg(synth))); + } + + void rerecord_cleans_up() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + + take(800); + if (QTest::currentTestFailed()) return; + Analyser *first = m_window->analyser2(); + sv::ModelId firstModel = first->getMainModelId(); + QVERIFY(sv::ModelById::get(firstModel)); + + take(800); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->analyser2()); + sv::ModelId secondModel = m_window->analyser2()->getMainModelId(); + QVERIFY(secondModel != firstModel); + QVERIFY2(!sv::ModelById::get(firstModel), + "the first take's model was not released"); + QCOMPARE(layersOnModel(firstModel), 0); + QCOMPARE(layersOnModel(secondModel), 1); + + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + QVERIFY(m_window->paneStack()->getHiddenPaneCount() <= 1); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(events.size() > 50); + + // The play source's model list has no accessor, so whether it + // holds stale ids (finding 6) is not checked here + } + + // No device at all: the base class record() gives up quietly + void record_failure_resets_flags() { + makeWindow(FakeAudioIO::Config(), false); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + + QEXPECT_FAIL("", "Review finding 4: m_recordingAsSingingTrack is " + "left true when record() fails", Abort); + QVERIFY(!m_window->recordingAsSingingTrack()); + + // so that the next file opened is analysed as usual + openReference(writeWav(tone(highHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + highHz)) < 10.0); + } + + void analyse_now_during_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + sv::Layer *refLayer = + m_window->analyser()->getLayer(Analyser::PitchTrack); + sv::ModelId refModel = refLayer->getModel(); + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(600); + m_window->doAnalyseNow(); + QTest::qWait(600); + QVERIFY(m_window->recordTarget()->isRecording()); + m_window->doRecord(); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + sv::Layer *refLayerNow = + m_window->analyser()->getLayer(Analyser::PitchTrack); + QEXPECT_FAIL("", "Review finding 5: Analyse Now during a take makes " + "the end of the take re-analyse the reference", Continue); + QVERIFY2(refLayerNow == refLayer && refLayerNow->getModel() == refModel, + "the reference pitch track was replaced"); + } + + void load_singing_track() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + sv::Layer *refLayer = + m_window->analyser()->getLayer(Analyser::PitchTrack); + sv::ModelId refModel = refLayer->getModel(); + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + + // Set up exactly once: there and then, and not again when the + // queued calls run + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + sv::ModelId singing = a2->getMainModelId(); + QVERIFY(m_window->pendingSingingModelId().isNone()); + QCoreApplication::processEvents(); + QCOMPARE(m_window->analyser2(), a2); + QVERIFY(sv::ModelById::get(singing)); + QCOMPARE(layersOnModel(singing), 1); + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QCOMPARE(m_window->analyser2(), a2); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(a2)), highHz)) < 10.0); + + // The open half of finding 7: the reference is handed to its + // analyser a second time, which today changes nothing + QCOMPARE(m_window->analyser()->getLayer(Analyser::PitchTrack), + refLayer); + QCOMPARE(refLayer->getModel(), refModel); + } + + void load_background_music() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + int rulerPaneLayers = m_window->paneStack()->getPane(1)->getLayerCount(); + + m_window->doLoadBackgroundMusic(writeWav(tone(highHz, 1.0))); + QCoreApplication::processEvents(); + + sv::ModelId music = m_window->backgroundMusicModelId(); + QVERIFY(!music.isNone()); + QVERIFY(sv::ModelById::get(music)); + QVERIFY(m_window->backgroundMusicLayer()); + QVERIFY(paneHasLayer(0, m_window->backgroundMusicLayer())); + QCOMPARE(layersOnModel(music), 1); + auto params = m_window->backgroundMusicLayer()->getPlayParameters(); + QVERIFY(params && params->isPlayAudible()); + + QVERIFY2(!m_window->analyser2(), + "the background music was analysed as a singing track"); + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + QCOMPARE(m_window->paneStack()->getPane(1)->getLayerCount(), + rulerPaneLayers); + } + + // Only after a session load is the shared ruler put into the extra + // pane, so only here does pruning that pane touch it. The take + // afterwards is what used to read the freed ruler. + void load_singing_track_after_session() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + reopenAsSession(); + if (QTest::currentTestFailed()) return; + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + verifyRulerIntact(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + take(600); + if (QTest::currentTestFailed()) return; + verifyRulerIntact(); + } + + void load_background_music_after_session() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + reopenAsSession(); + if (QTest::currentTestFailed()) return; + + m_window->doLoadBackgroundMusic(writeWav(tone(highHz, 1.0))); + verifyRulerIntact(); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->backgroundMusicLayer()); + + take(600); + if (QTest::currentTestFailed()) return; + verifyRulerIntact(); + } + + void session_round_trip() { + const int K = 8192; + FakeAudioIO::Config config; + config.playbackLatency = 4096; + config.recordLatency = 4096; + config.input = tone(highHz, 3.0); + config.inputDelay = K; + config.inputFollowsPlayback = true; + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(1200); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(K)); + + QString session = m_dir.filePath("round-trip.ton"); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + QVERIFY(!m_window->analyser2()); + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2->getMainModelId() != m_window->mainModelId()); + QCOMPARE(colourOf(a2->getLayer(Analyser::PitchTrack)), + colourNamed("Orange")); + QCOMPARE(colourOf(a2->getLayer(Analyser::Notes)), + colourNamed("Bright Purple")); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(a2)), highHz)) < 10.0); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + + auto wave = sv::ModelById::getAs + (a2->getMainModelId()); + QVERIFY(wave); + QEXPECT_FAIL("", "Review finding 13: SVFileReader does not re-apply " + "\"start\" to wave file models", Continue); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(-K)); + } + + void close_session_resets() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + m_window->doCloseSession(); + + QVERIFY(!m_window->analyser2()); + QVERIFY(!m_window->realtimeTracker()); + QVERIFY(!m_window->realtimeLayer()); + QVERIFY(m_window->realtimeModelId().isNone()); + QVERIFY(m_window->currentRecordingModelId().isNone()); + QVERIFY(m_window->pendingSingingModelId().isNone()); + QVERIFY(!m_window->recordingAsSingingTrack()); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + QCOMPARE(m_window->paneStack()->getPaneCount(), 0); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + + // and the window still works + openReference(writeWav(tone(highHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + highHz)) < 10.0); + } + + // Closing while pYIN is still running on the take. About one run + // in five under CPU load, the take's model is destroyed on the + // transform thread ("Timers cannot be stopped from another + // thread") and the process dies with an access violation soon + // after. A crash cannot be an expected failure, so this is + // skipped until the finding is fixed. + void close_session_during_analysis() { + QSKIP("Review finding 15: closing the session during the take's " + "analysis can crash"); + + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(600); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + m_window->doCloseSession(); + + QVERIFY(!m_window->analyser2()); + QCOMPARE(m_window->paneStack()->getPaneCount(), 0); + + openReference(writeWav(tone(highHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + highHz)) < 10.0); + } +}; + +#endif diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h new file mode 100644 index 00000000..e701c92b --- /dev/null +++ b/main/test/TestSingingAnalysis.h @@ -0,0 +1,464 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_SINGING_ANALYSIS_H +#define TEST_SINGING_ANALYSIS_H + +// Tier 4: the Analyser running the real pYIN plugin on synthetic +// audio, in primary and secondary mode, and the latency shift that +// the record path applies before analysis. No MainWindow. +// +// The test main points VAMP_PATH at the directory holding the test +// executable, which is where the build leaves pyin.dll. + +#include "TestSignals.h" + +#include "../Analyser.h" + +#include "framework/Document.h" +#include "framework/SVFileReader.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/ViewManager.h" +#include "layer/Layer.h" +#include "layer/ColourDatabase.h" +#include "layer/SingleColourLayer.h" +#include "layer/WaveformLayer.h" +#include "data/model/WritableWaveFileModel.h" +#include "data/model/SparseTimeValueModel.h" +#include "base/PlayParameters.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +class TestSingingAnalysis : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + static constexpr int hop = 256; // as set in Analyser::addAnalyses() + + // The latency shift used throughout: close to half a second, and + // a whole number of hops so that the shifted and unshifted + // analyses land on the same frame grid + static constexpr sv::sv_frame_t shift = 86 * hop; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + sv::ViewManager *m_viewManager = nullptr; + sv::PaneStack *m_paneStack = nullptr; + sv::Document *m_document = nullptr; + sv::Pane *m_pane = nullptr; + + sv::ModelId makeAudioModel(const std::vector &data) { + QString path = m_dir.filePath + (QString("audio-%1.wav").arg(++m_fileCounter)); + auto model = std::make_shared + (path, rate, 1, + sv::WritableWaveFileModel::Normalisation::None); + const float *ptr = data.data(); + model->addSamples(&ptr, sv::sv_frame_t(data.size())); + model->writeComplete(); + return sv::ModelById::add(model); + } + + // Both tones have a whole number of samples per period (200 and + // 150). A synthetic tone is exactly periodic, so when its period is + // not a whole number of samples the best integer lag is a multiple + // of it, and pYIN reports a subharmonic: 220, 330 and 440 Hz all + // come out as 110 Hz (lag 401), where 150 Hz (lag 294) is right. + // No voice is that periodic, so this is about the test signal and + // not about the application. + static constexpr double referenceHz = 220.5; + static constexpr double singingHz = 294.0; + + static std::vector tone(double hz, double seconds) { + return TestSignals::sawtooth(hz, rate, int(seconds * rate), 0.5f); + } + + static std::vector silenceThenTone(sv::sv_frame_t silence, + double hz, double seconds) { + std::vector data(size_t(silence), 0.f); + auto t = tone(hz, seconds); + data.insert(data.end(), t.begin(), t.end()); + return data; + } + + // Run the analyser on the model and wait for pYIN. If startFrame + // is non-zero, apply it between layer setup and analysis, which + // is what MainWindow::analyseNow() does after a take. + void analyse(Analyser &analyser, sv::ModelId model, + sv::sv_frame_t startFrame = 0) { + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QString error; + if (startFrame != 0) { + error = analyser.newFileLoaded + (m_document, model, m_paneStack, m_pane, true); + QCOMPARE(error, QString()); + auto wave = sv::ModelById::getAs(model); + QVERIFY(wave); + wave->setStartFrame(startFrame); + error = analyser.analyseExistingFile(); + } else { + error = analyser.newFileLoaded + (m_document, model, m_paneStack, m_pane, false); + } + QCOMPARE(error, QString()); + QVERIFY(analyser.getLayer(Analyser::PitchTrack)); + QVERIFY(analyser.getLayer(Analyser::Notes)); + QVERIFY2(done.count() > 0 || done.wait(30000), + "pYIN did not complete within 30 seconds"); + } + + sv::ModelId addSingingModel(const std::vector &data) { + sv::ModelId id = makeAudioModel(data); + m_document->addNonDerivedModel(id); + return id; + } + + static sv::EventVector pitchEvents(Analyser &analyser) { + sv::Layer *layer = analyser.getLayer(Analyser::PitchTrack); + if (!layer) return {}; + auto model = sv::ModelById::getAs + (layer->getModel()); + if (!model) return {}; + return model->getAllEvents(); + } + + static double medianHz(const sv::EventVector &events) { + std::vector values; + for (const auto &e : events) values.push_back(e.getValue()); + if (values.empty()) return 0.0; + std::sort(values.begin(), values.end()); + return values[values.size() / 2]; + } + + static int colourOf(sv::Layer *layer) { + auto scl = qobject_cast(layer); + return scl ? scl->getBaseColour() : -1; + } + + static bool audible(sv::Layer *layer) { + auto params = layer ? layer->getPlayParameters() : nullptr; + return params && params->isPlayAudible(); + } + + struct Callback : public sv::SVFileReaderPaneCallback { + sv::PaneStack *stack; + Callback(sv::PaneStack *s) : stack(s) { } + sv::Pane *addPane() override { return stack->addPane(); } + void setWindowSize(int, int) override { } + void addSelection(sv::sv_frame_t, sv::sv_frame_t) override { } + }; + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + // These are the test executable's own settings (see the + // organisation name in the test main). Start from the + // defaults whatever an earlier run left behind. + QSettings().clear(); + + // As the MainWindow and MainWindowBase constructors do. The + // Analyser connects to signals by their unqualified argument + // type names, which only resolve once these are registered. + qRegisterMetaType("sv_frame_t"); + qRegisterMetaType("sv_samplerate_t"); + qRegisterMetaType("ModelId"); + QSettings settings; + settings.beginGroup("Transformer"); + settings.setValue("use-flexi-note-model", true); + settings.endGroup(); + + // As the MainWindow constructor does; only the colours that + // the Analyser asks for by name + auto cdb = sv::ColourDatabase::getInstance(); + cdb->addColour(Qt::black, "Black"); + cdb->addColour(QColor(255, 150, 50), "Orange"); + cdb->addColour(QColor(180, 180, 180), "Grey"); + cdb->setUseDarkBackground + (cdb->addColour(QColor(30, 150, 255), "Bright Blue"), true); + cdb->setUseDarkBackground + (cdb->addColour(QColor(225, 74, 255), "Bright Purple"), true); + } + + void init() { + m_viewManager = new sv::ViewManager; + m_paneStack = new sv::PaneStack(nullptr, m_viewManager); + m_document = new sv::Document; + m_document->setMainModel(makeAudioModel(tone(referenceHz, 2.0))); + m_pane = m_paneStack->addPane(); + } + + void cleanup() { + delete m_document; + delete m_paneStack; + delete m_viewManager; + m_document = nullptr; + m_paneStack = nullptr; + m_viewManager = nullptr; + m_pane = nullptr; + } + + void pitch_track_found() { + Analyser analyser; + analyse(analyser, m_document->getMainModel()); + if (QTest::currentTestFailed()) return; + + auto events = pitchEvents(analyser); + QVERIFY2(events.size() > 100, + qPrintable(QString("only %1 pitch events for 2 s of tone") + .arg(events.size()))); + + double median = medianHz(events); + double cents = TestSignals::centsBetween(median, referenceHz); + QVERIFY2(std::abs(cents) < 10.0, + qPrintable(QString("median %1 Hz is %2 cents from the reference") + .arg(median).arg(cents))); + } + + void shift_aligns_onset() { + // The core latency-compensation test. The same recording, + // half a second of silence and then a tone, is analysed with + // and without the start-frame shift that analyseNow() applies. + auto data = silenceThenTone(shift, singingHz, 1.5); + + sv::sv_frame_t unshifted = 0, shifted = 0; + + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + auto events = pitchEvents(analyser); + QVERIFY(!events.empty()); + unshifted = events.begin()->getFrame(); + analyser.removeAllLayers(); + } + + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data), -shift); + if (QTest::currentTestFailed()) return; + auto events = pitchEvents(analyser); + QVERIFY(!events.empty()); + shifted = events.begin()->getFrame(); + analyser.removeAllLayers(); + } + + // Measured: 22016 and 0, the onset exactly. Two hops either + // way allows for pYIN deciding a little later that the tone + // is voiced; the difference between the runs gets one. + QVERIFY2(std::abs(unshifted - shift) <= 2 * hop, + qPrintable(QString("unshifted onset at %1, expected near %2") + .arg(unshifted).arg(shift))); + QVERIFY2(std::abs(shifted) <= 2 * hop, + qPrintable(QString("shifted onset at %1, expected near 0") + .arg(shifted))); + QVERIFY2(std::abs((unshifted - shifted) - shift) <= hop, + qPrintable(QString("onset moved by %1, expected %2") + .arg(unshifted - shifted).arg(shift))); + } + + void shift_survives_session() { + sv::ModelId singing = addSingingModel(tone(singingHz, 0.5)); + auto wave = sv::ModelById::getAs(singing); + QVERIFY(wave); + wave->setStartFrame(-shift); + QString singingPath = wave->getLocation(); + + // The model is only written if a layer in a view uses it + sv::Layer *layer = m_document->createLayer(sv::LayerFactory::Waveform); + QVERIFY(layer); + m_document->setModel(layer, singing); + m_document->addLayerToView(m_pane, layer); + + // As MainWindowBase::toXml() + QString xml; + { + QTextStream out(&xml); + out << "\n" + << "\n\n"; + m_document->toXml(out, "", ""); + out << "\n"; + m_pane->toXml(out, " "); + out << "\n\n"; + } + QVERIFY2(xml.contains(QString("start=\"%1\"").arg(-shift)), + "the shifted start frame was not written to the session"); + + sv::ViewManager viewManager; + sv::PaneStack paneStack(nullptr, &viewManager); + auto document = new sv::Document; + Callback callback(&paneStack); + { + sv::SVFileReader reader(document, callback, m_dir.path()); + reader.parseXml(xml); + QVERIFY2(reader.isOK(), qPrintable(reader.getErrorString())); + } + + std::shared_ptr reloaded; + for (sv::Layer *l : document->getLayers()) { + auto m = sv::ModelById::getAs(l->getModel()); + if (m && m->getLocation() == singingPath) reloaded = m; + } + sv::sv_frame_t start = reloaded ? reloaded->getStartFrame() : 0; + bool found = bool(reloaded); + reloaded.reset(); + delete document; + + QVERIFY2(found, "the singing model was not reloaded"); + QEXPECT_FAIL("", "Review finding 13: SVFileReader does not re-apply " + "\"start\" to wave file models", Continue); + QCOMPARE(start, -shift); + } + + void secondary_layers_muted() { + // Guards commit 36d0ce7 + Analyser primary(Analyser::PrimaryColors); + analyse(primary, m_document->getMainModel()); + if (QTest::currentTestFailed()) return; + + Analyser secondary(Analyser::SecondaryColors); + analyse(secondary, addSingingModel(tone(singingHz, 1.0))); + if (QTest::currentTestFailed()) return; + + QVERIFY(primary.getLayer(Analyser::PitchTrack) != + secondary.getLayer(Analyser::PitchTrack)); + QVERIFY(primary.getLayer(Analyser::Notes) != + secondary.getLayer(Analyser::Notes)); + + QVERIFY(!audible(secondary.getLayer(Analyser::PitchTrack))); + QVERIFY(!audible(secondary.getLayer(Analyser::Notes))); + QVERIFY(audible(primary.getLayer(Analyser::PitchTrack))); + QVERIFY(audible(primary.getLayer(Analyser::Notes))); + + // Muting them must not have gone through the settings that + // the two analysers share + QSettings settings; + settings.beginGroup("Analyser"); + QVERIFY(settings.value + (QString("audible-%1").arg(int(Analyser::PitchTrack)), + true).toBool()); + QVERIFY(settings.value + (QString("audible-%1").arg(int(Analyser::Notes)), + true).toBool()); + settings.endGroup(); + } + + void secondary_colours() { + Analyser primary(Analyser::PrimaryColors); + analyse(primary, m_document->getMainModel()); + if (QTest::currentTestFailed()) return; + + Analyser secondary(Analyser::SecondaryColors); + analyse(secondary, addSingingModel(tone(singingHz, 1.0))); + if (QTest::currentTestFailed()) return; + + auto cdb = sv::ColourDatabase::getInstance(); + QCOMPARE(colourOf(primary.getLayer(Analyser::PitchTrack)), + cdb->getColourIndex(QString("Black"))); + QCOMPARE(colourOf(primary.getLayer(Analyser::Notes)), + cdb->getColourIndex(QString("Bright Blue"))); + QCOMPARE(colourOf(secondary.getLayer(Analyser::PitchTrack)), + cdb->getColourIndex(QString("Orange"))); + QCOMPARE(colourOf(secondary.getLayer(Analyser::Notes)), + cdb->getColourIndex(QString("Bright Purple"))); + } + + void secondary_tracks_own_audio() { + // Two analysers in one pane: each pitch track must come from + // its own model, which the two different pitches show + Analyser primary(Analyser::PrimaryColors); + analyse(primary, m_document->getMainModel()); + if (QTest::currentTestFailed()) return; + + Analyser secondary(Analyser::SecondaryColors); + analyse(secondary, addSingingModel(tone(singingHz, 1.0))); + if (QTest::currentTestFailed()) return; + + double p = medianHz(pitchEvents(primary)); + double s = medianHz(pitchEvents(secondary)); + QVERIFY2(std::abs(TestSignals::centsBetween(p, referenceHz)) < 10.0, + qPrintable(QString("primary median %1 Hz").arg(p))); + QVERIFY2(std::abs(TestSignals::centsBetween(s, singingHz)) < 10.0, + qPrintable(QString("secondary median %1 Hz").arg(s))); + } + + void secondary_waveform_is_own_model() { + // The secondary analyser must show the singing audio, not + // make a second layer on the document's main model + sv::ModelId singing = addSingingModel(tone(singingHz, 1.0)); + Analyser secondary(Analyser::SecondaryColors); + QString error = secondary.newFileLoaded + (m_document, singing, m_paneStack, m_pane, true); + QCOMPARE(error, QString()); + + sv::Layer *audio = secondary.getLayer(Analyser::Audio); + QVERIFY(audio); + QVERIFY(audio->getModel() == singing); + + // deferAnalysis: no pYIN yet + QVERIFY(!secondary.getLayer(Analyser::PitchTrack)); + QVERIFY(!secondary.getLayer(Analyser::Notes)); + } + + void remove_all_layers() { + Analyser primary(Analyser::PrimaryColors); + analyse(primary, m_document->getMainModel()); + if (QTest::currentTestFailed()) return; + + sv::ModelId singing = addSingingModel(tone(singingHz, 1.0)); + Analyser secondary(Analyser::SecondaryColors); + analyse(secondary, singing); + if (QTest::currentTestFailed()) return; + + sv::Layer *primaryPitch = primary.getLayer(Analyser::PitchTrack); + int primaryLayers = 0; + for (sv::Layer *l : m_document->getLayers()) { + if (l->getModel() != singing && l->getSourceModel() != singing) { + ++primaryLayers; + } + } + + secondary.removeAllLayers(); + + for (sv::Layer *l : m_document->getLayers()) { + QVERIFY2(l->getModel() != singing, + "a layer still shows the singing model"); + QVERIFY2(l->getSourceModel() != singing, + "a layer derived from the singing model is left"); + } + QVERIFY2(!sv::ModelById::get(singing), + "the singing model was not released"); + QVERIFY(!secondary.getLayer(Analyser::Audio)); + QVERIFY(!secondary.getLayer(Analyser::PitchTrack)); + QVERIFY(!secondary.getLayer(Analyser::Notes)); + + // and the primary analyser's layers were left alone + QCOMPARE(int(m_document->getLayers().size()), primaryLayers); + QVERIFY(primary.getLayer(Analyser::PitchTrack) == primaryPitch); + QVERIFY(m_document->getLayers().count(primaryPitch) > 0); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index f2f1c19a..6ad3eaf0 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -12,12 +12,15 @@ */ #include "TestSingingDocument.h" +#include "TestSingingAnalysis.h" +#include "TestRecordWorkflow.h" #include "RunSuite.h" #include "system/Init.h" #include +#include #include #include @@ -43,12 +46,30 @@ int main(int argc, char *argv[]) app.setOrganizationName("tony-tests"); app.setApplicationName("test-tony-app"); + // Tier 4 runs the real pYIN plugin, which the build leaves next to + // this executable. Replace rather than extend VAMP_PATH, so that a + // pYIN installed elsewhere on this machine is never the one tested. + qputenv("VAMP_PATH", + QDir::toNativeSeparators(app.applicationDirPath()).toLocal8Bit()); + { TestSingingDocument t; if (runSuite(&t, argc, argv)) ++good; else ++bad; } + { + TestSingingAnalysis t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + { + TestRecordWorkflow t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 96873479..487b10db 100644 --- a/meson.build +++ b/meson.build @@ -1367,6 +1367,8 @@ tony_core_test_exe = executable( tony_app_test_moc_files = qt.preprocess( moc_headers: [ 'main/test/TestSingingDocument.h', + 'main/test/TestSingingAnalysis.h', + 'main/test/TestRecordWorkflow.h', ]) tony_app_test_exe = executable( @@ -1395,6 +1397,8 @@ tony_app_test_exe = executable( test('tony-core', tony_core_test_exe) test('tony-app', tony_app_test_exe, + depends: pyin_plugin, + timeout: 300, env: [ 'QT_QPA_PLATFORM=offscreen' ]) test('svcore-base', svcore_base_test_exe) From 6bd43c6986c56e4b8e0902b684a082c67dcf5895 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 00:04:17 +0300 Subject: [PATCH 017/275] fix: keep the latency shift across reload, measure the start gap, cancel analyses on close Findings 13, 14 and 15 of the singing practice review. 13: bump svapp so that SVFileReader restores a wave file model's start frame. The take's latency shift now survives a session round trip. 14: the take starts before the reference does, by a gap that varies from run to run. Measure it in the audio callback, using the new getFramesReceived() and setPlayStartCallback() hooks in the svapp fork, and add it to the recording latency before the take is shifted. A GUI thread estimate remains as a fallback. FakeAudioIO now delivers input before it pulls output, as PortAudio and JACK do. 15: Analyser::cancelAnalyses() cancels the running pYIN transforms and waits for their threads before the models are released, on file close, layer removal and re-analysis. Closing a session during analysis crashed about one run in three under load. Remove the QEXPECT_FAIL and QSKIP markers for the three findings. Co-Authored-By: Claude Fable 5.1 --- main/Analyser.cpp | 30 +++++++++++++- main/Analyser.h | 7 ++++ main/MainWindow.cpp | 53 ++++++++++++++++++++++-- main/MainWindow.h | 15 +++++++ main/test/FakeAudioIO.h | 71 +++++++++++++++++---------------- main/test/TestRecordWorkflow.h | 52 +++++++++++++----------- main/test/TestSingingAnalysis.h | 2 - repoint-lock.json | 2 +- 8 files changed, 166 insertions(+), 66 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 2b5f5d0f..8d85721e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -17,6 +17,7 @@ #include "transform/TransformFactory.h" #include "transform/ModelTransformer.h" +#include "transform/ModelTransformerFactory.h" #include "transform/FeatureExtractionModelTransformer.h" #include "framework/Document.h" #include "data/model/WaveFileModel.h" @@ -115,7 +116,11 @@ Analyser::analyseExistingFile() if (!m_pane) return "Internal error: Analyser::analyseExistingFile() called with no pane present"; if (m_fileModel.isNone()) return "Internal error: Analyser::analyseExistingFile() called with no model present"; - + + // The layers removed below are kept alive by the undo history, so + // an analysis still running on them would carry on unseen + cancelAnalyses(); + if (m_layers[PitchTrack]) { m_document->removeLayerFromView(m_pane, m_layers[PitchTrack]); m_layers[PitchTrack] = 0; @@ -182,10 +187,30 @@ Analyser::doAllAnalyses(bool withPitchTrack) return warning; } +void +Analyser::cancelAnalyses() +{ + std::vector derived(m_reAnalysisCandidates.begin(), + m_reAnalysisCandidates.end()); + for (Component c : { PitchTrack, Notes }) { + auto it = m_layers.find(c); + if (it != m_layers.end() && it->second) derived.push_back(it->second); + } + + auto mtf = ModelTransformerFactory::getInstance(); + for (Layer *layer : derived) { + ModelId modelId = layer->getModel(); + // cancel() returns once the transform thread has exited; it + // does nothing if the transform has already finished + if (!modelId.isNone()) mtf->cancel(modelId); + } +} + void Analyser::fileClosed() { cerr << "Analyser::fileClosed" << endl; + cancelAnalyses(); m_layers.clear(); m_reAnalysisCandidates.clear(); m_currentCandidate = -1; @@ -197,6 +222,9 @@ Analyser::removeAllLayers() { cerr << "Analyser::removeAllLayers" << endl; + // Before any model is released: see cancelAnalyses() + cancelAnalyses(); + // First discard any re-analysis candidate layers (these are not in // m_layers, but they are registered with the document). discardPitchCandidates(); diff --git a/main/Analyser.h b/main/Analyser.h index f7a26fc6..b1f6df47 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -71,6 +71,13 @@ class Analyser : public QObject, // layers; return "" on success or error string on failure QString analyseExistingFile(); + // Stop any pitch or note analysis still running on our model, and + // wait for its thread to exit. A running transform holds shared + // pointers to its input and output models, so this must happen + // before those models are released, or the last reference may be + // dropped (and the model destroyed) on the transform thread + void cancelAnalyses(); + // Discard any layers etc associated with the current document void fileClosed(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index d5db46cc..22c5aab4 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -149,10 +149,26 @@ MainWindow::MainWindow(AudioMode audioMode, m_recordingAsSingingTrack(false), m_paneCountBeforeRecording(0), m_currentRecordingModelId(), - m_recordingLatencyFrames(0) + m_recordingLatencyFrames(0), + m_recordingStartGapEstimate(0), + m_recordingStartGapMeasured(-1), + m_awaitingReferenceStart(false) { setWindowTitle(QApplication::applicationName()); + // Called from the audio callback: see recordingStarted() + if (m_playSource) { + m_playSource->setPlayStartCallback([this](int blockFrames) { + if (!m_awaitingReferenceStart.exchange(false)) return; + if (!m_recordTarget || !m_recordTarget->isRecording()) return; + // The device drivers deliver the input of a block before they + // ask for its output (PortAudioIO, JACKAudioIO), so the count + // already includes the input that goes with this first block + sv_frame_t gap = m_recordTarget->getFramesReceived() - blockFrames; + m_recordingStartGapMeasured = (gap > 0 ? gap : 0); + }); + } + #ifdef Q_OS_MAC #if (QT_VERSION >= QT_VERSION_CHECK(5, 2, 0)) setUnifiedTitleAndToolBarOnMac(true); @@ -2749,6 +2765,9 @@ MainWindow::record() m_recordingAsSingingTrack = true; m_recordingLatencyFrames = 0; // reset; will be computed in recordingStarted() + m_recordingStartGapEstimate = 0; + m_awaitingReferenceStart = false; + m_recordingStartGapMeasured = -1; // Remember pane count so we can prune the extra pane that // MainWindowBase::record() creates via AddPaneCommand for the // recording's waveform layer. We want both tracks in pane 0. @@ -2868,11 +2887,27 @@ MainWindow::recordingStarted() // shift the model's start frame by -(outputLatency + inputLatency). sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; + // + // The take is already running by now: record() started it, and + // this lambda runs an event-loop turn (and a layer setup) later. + // Whatever the device delivers before the reference starts sits + // at the front of the take, ahead of reference frame 0, so it is + // part of the shift as well. What it has delivered so far is + // only an estimate of that, because the play source takes a + // little while to start; the audio callback reports the real + // figure, and refineRecordingLatency() picks it up. + m_recordingStartGapEstimate = + m_recordTarget ? m_recordTarget->getFramesReceived() : 0; m_recordingLatencyFrames = - computeRecordingLatency(outputLatency, inputLatency); + computeRecordingLatency(outputLatency, inputLatency) + + m_recordingStartGapEstimate; cerr << "MainWindow::recordingStarted: output latency=" << outputLatency << " input latency=" << inputLatency - << " round-trip compensation=" << m_recordingLatencyFrames << " frames" << endl; + << " estimated start gap=" << m_recordingStartGapEstimate + << " total compensation=" << m_recordingLatencyFrames << " frames" << endl; + + m_recordingStartGapMeasured = -1; + m_awaitingReferenceStart = true; m_viewManager->setPlaybackFrame(0); m_playSource->play(0); @@ -2883,6 +2918,17 @@ MainWindow::recordingStarted() }); } +void +MainWindow::refineRecordingLatency() +{ + sv_frame_t measured = m_recordingStartGapMeasured; + if (measured < 0 || measured == m_recordingStartGapEstimate) return; + cerr << "MainWindow::refineRecordingLatency: start gap was " << measured + << " frames, not the estimated " << m_recordingStartGapEstimate << endl; + m_recordingLatencyFrames += measured - m_recordingStartGapEstimate; + m_recordingStartGapEstimate = measured; +} + void MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) { @@ -4260,6 +4306,7 @@ MainWindow::analyseNow() // pYIN analysis so that all derived layers (pitch, notes) inherit the // same timeline offset. Only applied when reference playback was // active during the recording (m_recordingLatencyFrames > 0). + refineRecordingLatency(); if (m_recordingLatencyFrames > 0 && !m_currentRecordingModelId.isNone()) { auto wfm = ModelById::getAs(m_currentRecordingModelId); if (wfm) { diff --git a/main/MainWindow.h b/main/MainWindow.h index 43fd394b..c7ade16d 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -21,6 +21,7 @@ #include "RealtimePitchTracker.h" #include +#include #include "data/model/SparseTimeValueModel.h" @@ -380,7 +381,21 @@ protected slots: // "play reference while recording" toggle on. Applied as a negative // start-frame offset to the singing model so its timeline aligns with // the reference during playback. Reset to 0 at the start of each recording. + // + // It also includes the start gap: the part of the take recorded before + // the reference began to play. That starts out as an estimate made just + // before play() is called, and refineRecordingLatency() replaces the + // estimate with the measured figure once the audio callback has it. sv::sv_frame_t m_recordingLatencyFrames; + sv::sv_frame_t m_recordingStartGapEstimate; + + // Written by the audio callback when the first block of the reference is + // handed to the device during a take: the number of frames of the take + // that came before that block. -1 until then. + std::atomic m_recordingStartGapMeasured; + std::atomic m_awaitingReferenceStart; + + void refineRecordingLatency(); // Extra panes created by MainWindowBase::record() via AddPaneCommand // that we want to hide immediately but cannot delete yet because diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h index 34dd72e1..d5f3a55e 100644 --- a/main/test/FakeAudioIO.h +++ b/main/test/FakeAudioIO.h @@ -16,7 +16,8 @@ // A duplex audio device with no hardware behind it. A worker thread // runs the callback in real time, as a driver would: each block it -// pulls the application's output and pushes a programmed input. +// pushes a programmed input and then pulls the application's output, +// in that order, like PortAudioIO and JACKAudioIO. // // The device reports whatever latencies the test asks for, so the // application computes a known compensation, and the input can be @@ -56,11 +57,14 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO // Start the input clock at the first audible output sample // instead of at resume. With inputDelay equal to the reported - // round trip, this is a singer who is exactly on time + // round trip, this is a singer who is exactly on time. + // inputDelay must be at least a block, because the input of + // the block in which playback starts has already gone bool inputFollowsPlayback = false; // Add the output to the input, inputDelay frames late: - // speakers bleeding into the microphone + // speakers bleeding into the microphone. Again inputDelay + // must be at least a block bool loopback = false; // Whether the application keeps the input it is given just @@ -77,8 +81,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO m_config(config), m_suspended(true), m_stop(false), - m_clockRunning(false), - m_clock(0), + m_resumeFrame(0), m_frames(0), m_playStartFrame(-1), m_sinceResume(0), @@ -122,8 +125,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO std::lock_guard guard(m_mutex); if (!m_suspended) return; m_suspended = false; - m_clock = 0; - m_clockRunning = !m_config.inputFollowsPlayback; + m_resumeFrame = long(m_captured.size()); m_playStartFrame = -1; m_sinceResume = 0; m_framesBeforePlayStart = -1; @@ -171,8 +173,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO std::thread m_thread; bool m_suspended; std::atomic m_stop; - bool m_clockRunning; - long m_clock; + long m_resumeFrame; std::atomic m_frames; long m_playStartFrame; long m_sinceResume; @@ -199,8 +200,14 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO } } - float inputAt(long clock) const { - long i = clock - m_config.inputDelay; + // The input at a position in the captured output's timeline + float inputAt(long frame) const { + long origin = m_resumeFrame; + if (m_config.inputFollowsPlayback) { + if (m_playStartFrame < 0) return 0.f; + origin = m_playStartFrame; + } + long i = frame - origin - m_config.inputDelay; if (i < 0 || i >= long(m_config.input.size())) return 0.f; return m_config.input[size_t(i)]; } @@ -208,15 +215,29 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO void process() { const int n = m_config.blockSize; const int ch = m_config.channels; + const long base = long(m_captured.size()); + + std::vector in(n, 0.f); + for (int i = 0; i < n; ++i) { + in[i] = inputAt(base + i); + if (m_config.loopback) { + long j = base + i - m_config.inputDelay; + if (j >= 0 && j < base) in[i] += m_captured[size_t(j)]; + } + } + + bool kept = !m_config.inputIsKept || m_config.inputIsKept(); + long keptBefore = m_sinceResume; + + std::vector inPtrs(ch, in.data()); + m_target->putSamples(inPtrs.data(), ch, n); + if (kept) m_sinceResume += n; std::vector> out(ch, std::vector(n, 0.f)); std::vector outPtrs; for (auto &v : out) outPtrs.push_back(v.data()); int got = m_source->getSourceSamples(outPtrs.data(), ch, n); - bool kept = !m_config.inputIsKept || m_config.inputIsKept(); - - long base = long(m_captured.size()); for (int i = 0; i < n; ++i) { float mix = 0.f; if (i < got) { @@ -226,30 +247,10 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO m_captured.push_back(mix); if (m_playStartFrame < 0 && std::fabs(mix) > 1e-4f) { m_playStartFrame = base + i; - m_framesBeforePlayStart = m_sinceResume + (kept ? i : 0); - } - } - - std::vector in(n, 0.f); - for (int i = 0; i < n; ++i) { - if (!m_clockRunning && m_playStartFrame >= 0 && - base + i >= m_playStartFrame) { - m_clockRunning = true; - } - if (m_clockRunning) { - in[i] = inputAt(m_clock); - ++m_clock; - } - if (m_config.loopback) { - long j = base + i - m_config.inputDelay; - if (j >= 0) in[i] += m_captured[size_t(j)]; + m_framesBeforePlayStart = keptBefore + (kept ? i : 0); } } - std::vector inPtrs(ch, in.data()); - m_target->putSamples(inPtrs.data(), ch, n); - if (kept) m_sinceResume += n; - m_frames += n; } }; diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 71a510f7..2c241a09 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -516,7 +516,19 @@ private slots: auto wave = sv::ModelById::getAs (m_window->analyser2()->getMainModelId()); QVERIFY(wave); - QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(-K)); + + // The shift is the round trip plus what the application took + // the start gap to be (review finding 14). The device knows + // what the gap really was. The application counts in whole + // blocks, in the audio callback, so the two agree to within a + // few samples; an estimate made on the GUI thread is out by a + // block or more + sv::sv_frame_t gap = m_window->fake()->getFramesBeforePlayStart(); + sv::sv_frame_t assumedGap = -wave->getStartFrame() - K; + QVERIFY2(std::llabs(assumedGap - gap) <= 16, + qPrintable(QString("the application took the start gap to " + "be %1 frames; it was %2") + .arg(assumedGap).arg(gap))); sv::sv_frame_t refStep = stepFrame(pitchEvents(m_window->analyser())); sv::sv_frame_t sungStep = stepFrame(pitchEvents(m_window->analyser2())); @@ -524,20 +536,14 @@ private slots: QVERIFY2(sungStep > 0, "the take never reached the second note"); sv::sv_frame_t error = sungStep - refStep; - sv::sv_frame_t gap = m_window->fake()->getFramesBeforePlayStart(); QString detail = QString("sung step at %1, reference step at %2: " "%3 frames (%4 ms) apart; the reference " - "started %5 frames into the take") + "started %5 frames into the take, and the " + "application took that to be %6") .arg(sungStep).arg(refStep).arg(error) - .arg(1000.0 * double(error) / rate, 0, 'f', 1).arg(gap); + .arg(1000.0 * double(error) / rate, 0, 'f', 1) + .arg(gap).arg(assumedGap); - // What is left over is the time between the start of the take - // and the start of the reference, and nothing else - QVERIFY2(std::llabs(error - gap) <= 2 * hop, qPrintable(detail)); - - QEXPECT_FAIL("", "Review finding 14: the take starts before the " - "reference does, and the gap is not compensated", - Continue); QVERIFY2(std::llabs(error) <= 2 * hop, qPrintable(detail)); } @@ -830,7 +836,9 @@ private slots: take(1200); if (QTest::currentTestFailed()) return; - QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(K)); + // the round trip plus the start gap, which varies + sv::sv_frame_t shift = m_window->recordingLatencyFrames(); + QVERIFY(shift >= K); QString session = m_dir.filePath("round-trip.ton"); QVERIFY(m_window->saveSessionFile(session)); @@ -858,9 +866,7 @@ private slots: auto wave = sv::ModelById::getAs (a2->getMainModelId()); QVERIFY(wave); - QEXPECT_FAIL("", "Review finding 13: SVFileReader does not re-apply " - "\"start\" to wave file models", Continue); - QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(-K)); + QCOMPARE(wave->getStartFrame(), -shift); } void close_session_resets() { @@ -893,16 +899,14 @@ private slots: highHz)) < 10.0); } - // Closing while pYIN is still running on the take. About one run - // in five under CPU load, the take's model is destroyed on the - // transform thread ("Timers cannot be stopped from another - // thread") and the process dies with an access violation soon - // after. A crash cannot be an expected failure, so this is - // skipped until the finding is fixed. + // Closing while pYIN is still running on the take (review finding + // 15). Unless the analysis is cancelled first, about one run in + // three under CPU load destroys the take's model on the transform + // thread ("Timers cannot be stopped from another thread") and the + // process dies with an access violation soon after. A regression + // shows up as a crash of the whole test program, and not reliably: + // run it under load to check. void close_session_during_analysis() { - QSKIP("Review finding 15: closing the session during the take's " - "analysis can crash"); - FakeAudioIO::Config config; config.input = tone(highHz, 3.0); makeWindow(config); diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index e701c92b..0d720016 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -327,8 +327,6 @@ private slots: delete document; QVERIFY2(found, "the singing model was not reloaded"); - QEXPECT_FAIL("", "Review finding 13: SVFileReader does not re-apply " - "\"start\" to wave file models", Continue); QCOMPARE(start, -shift); } diff --git a/repoint-lock.json b/repoint-lock.json index d8299f5e..d942da38 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -10,7 +10,7 @@ "pin": "271db78f54eff467e3891fca920ab663cc69ce53" }, "svapp": { - "pin": "d67336efd0d3777442f450f7ff1dc680f6b0f608" + "pin": "fc4c6b375b01bb12aec39df9604f841e27588bc3" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" From 6bcb697d5ef457c33b185fd6f4bf4b084c6063b9 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 00:18:36 +0300 Subject: [PATCH 018/275] fix: place live dots on the reference timeline and keep them until pYIN is done Findings 8, 9 and 10 of the singing practice review. 8: a live dot is drawn at its frame minus the recording latency, where the finished pitch track will put the same sound, and dropped if that is negative. The latency is refined from the audio callback's start gap measurement first; dots placed with the earlier estimate are removed. 9: recordingFinishedFull() stops the tracker and resets the take's state at Stop, but leaves the dots on show until the analyser reports initialAnalysisCompleted. They go at once if the analysis could not be started. 10: pitch events still queued when the take ends are ignored, so they neither draw a dot nor overwrite the status bar. Tests: live_dot_shift (Tier 2), live_dots_compensated, live_dots_stay_until_analysis and stale_pitch_event_ignored (Tier 5). The Tier 5 tests failed against the unfixed code. Co-Authored-By: Claude Fable 5.1 --- main/LatencyUtils.h | 13 ++++ main/MainWindow.cpp | 116 ++++++++++++++++++++-------- main/MainWindow.h | 7 +- main/test/TestLatencyShift.h | 14 ++++ main/test/TestRecordWorkflow.h | 136 ++++++++++++++++++++++++++++++++- 5 files changed, 252 insertions(+), 34 deletions(-) diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index 606e2e22..fba94a51 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -33,4 +33,17 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, return outputLatency + inputLatency; } +/** + * Where to draw a live pitch dot for sound found at the given frame + * of a take that is going to be shifted earlier by the given latency + * once it is finished. A negative result means the sound came before + * the reference started, and the dot should be dropped. + */ +inline sv::sv_frame_t +compensatedLiveFrame(sv::sv_frame_t frame, sv::sv_frame_t latency) +{ + if (latency < 0) latency = 0; + return frame - latency; +} + #endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 22c5aab4..ad70a76a 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2599,13 +2599,24 @@ MainWindow::setupRealtimePitchLayer() } void -MainWindow::teardownRealtimePitchLayer() +MainWindow::stopRealtimePitchTracker() { if (m_realtimePitchTracker) { m_realtimePitchTracker->stop(); delete m_realtimePitchTracker; m_realtimePitchTracker = nullptr; } +} + +void +MainWindow::teardownRealtimePitchLayer() +{ + stopRealtimePitchTracker(); + + if (m_realtimeLayerTeardownConnection) { + disconnect(m_realtimeLayerTeardownConnection); + m_realtimeLayerTeardownConnection = {}; + } if (m_realtimePitchLayer) { // Use deleteLayer(force=true) directly — do NOT call @@ -2935,8 +2946,29 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) // Called on the GUI thread via Qt::QueuedConnection (RealtimePitchTracker // emits from its background thread). Write the point into the model here // so all model mutations stay on the GUI thread. - if (auto m = ModelById::getAs(m_realtimePitchModelId)) { - m->add(Event(frame, float(hz), tr(""))); + // + // Events still queued when the take ended arrive here as well. The + // dots may still be on show then, waiting for pYIN, but the take is + // over: leave them, and the status bar, alone. + if (!m_recordingInProgress) return; + + // Draw the dot where the finished pitch track will put this sound: + // the take is going to be shifted earlier by the recording latency. + // The first dots may have been placed using the estimate of the + // start gap. They all belong to sound from before the reference + // started, which has no place on the reference's timeline. + sv_frame_t latencyBefore = m_recordingLatencyFrames; + refineRecordingLatency(); + auto m = ModelById::getAs(m_realtimePitchModelId); + if (m && m_recordingLatencyFrames != latencyBefore) { + for (const Event &e : m->getAllEvents()) m->remove(e); + } + + sv_frame_t dotFrame = compensatedLiveFrame(frame, m_recordingLatencyFrames); + if (dotFrame < 0) return; + + if (m) { + m->add(Event(dotFrame, float(hz), tr(""))); } // Convert Hz to MIDI note number and cents deviation. @@ -2971,17 +3003,33 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) } void -MainWindow::recordingFinishedFull() +MainWindow::recordingFinishedFull(Analyser *analysing) { - // Called after analyseNow() has completed for the newly recorded audio. - // At this point the full pYIN analysis of the recording is available. - // We can remove the coarse realtime pitch layer (the "live" orange dots) - // because the full pYIN pitch track now covers the same audio. - cerr << "MainWindow::recordingFinishedFull: pYIN done, removing realtime pitch layer" << endl; + // Called from analyseNow() once the pYIN analysis of the newly + // recorded audio has been started. The take is over, so the tracker + // goes now. The coarse realtime pitch layer (the "live" orange dots) + // stays on show until the analyser passed in reports that the full + // pYIN pitch track is there to replace it. With no analyser (the + // analysis could not be started) there is nothing to wait for. + cerr << "MainWindow::recordingFinishedFull: take finished" << endl; m_recordingInProgress = false; m_recordingAsSingingTrack = false; m_currentRecordingModelId = {}; - teardownRealtimePitchLayer(); + + if (analysing && m_realtimePitchLayer) { + stopRealtimePitchTracker(); + if (m_realtimeLayerTeardownConnection) { + disconnect(m_realtimeLayerTeardownConnection); + } + m_realtimeLayerTeardownConnection = + connect(analysing, &Analyser::initialAnalysisCompleted, + this, [this]() { + cerr << "MainWindow: pYIN done, removing realtime pitch layer" << endl; + teardownRealtimePitchLayer(); + }); + } else { + teardownRealtimePitchLayer(); + } // Stop reference playback that was started for the singer's benefit. // Suspend the audio IO so it doesn't keep consuming CPU while idle. @@ -4316,7 +4364,12 @@ MainWindow::analyseNow() } } - if (m_analyser2) { + // The realtime pitch layer stays until the full pYIN analysis of + // the singing recording (via m_analyser2) is there to replace it, + // or goes at once if that analysis could not be started. + bool wasLive = (m_realtimePitchTracker || m_realtimePitchLayer); + + auto analyseSingingTrack = [this]() -> bool { CommandHistory::getInstance()->startCompoundOperation (tr("Analyse Singing Track"), true); @@ -4330,35 +4383,34 @@ MainWindow::analyseNow() tr("Failed to analyse singing track"), tr("Analysis failed

%1

").arg(error), QMessageBox::Ok); + return false; } + return true; + }; + + if (m_analyser2) { + bool ok = analyseSingingTrack(); + if (wasLive) recordingFinishedFull(ok ? m_analyser2 : nullptr); } else { // m_analyser2 may still be pending setup (modelAdded fires async). // Defer the analysis until the secondary analyser is ready. cerr << "analyseNow: m_analyser2 not ready yet, deferring singing-track analysis" << endl; - QTimer::singleShot(200, this, [this]() { + if (wasLive) { + // The take is over either way; the dots wait for the + // deferred analysis + stopRealtimePitchTracker(); + m_recordingInProgress = false; + } + QTimer::singleShot(200, this, [this, wasLive, analyseSingingTrack]() { + bool ok = false; if (m_analyser2) { - CommandHistory::getInstance()->startCompoundOperation - (tr("Analyse Singing Track"), true); - QString error = m_analyser2->analyseExistingFile(); - CommandHistory::getInstance()->endCompoundOperation(); - if (error != "") { - QMessageBox::warning - (this, - tr("Failed to analyse singing track"), - tr("Analysis failed

%1

").arg(error), - QMessageBox::Ok); - } + ok = analyseSingingTrack(); } else { cerr << "analyseNow (deferred): m_analyser2 still null, singing-track analysis skipped" << endl; } + if (wasLive) recordingFinishedFull(ok ? m_analyser2 : nullptr); }); } - - // Clean up the realtime pitch layer; the full pYIN analysis of the - // singing recording (via m_analyser2) now covers the same audio. - if (m_realtimePitchTracker || m_realtimePitchLayer) { - recordingFinishedFull(); - } return; } @@ -4379,10 +4431,10 @@ MainWindow::analyseNow() QMessageBox::Ok); } - // If this analyseNow was triggered by recording completion, clean up - // the realtime pitch layer now that the full pYIN analysis is available. + // If this analyseNow was triggered by recording completion, the + // realtime pitch layer goes when the full pYIN analysis is available. if (m_realtimePitchTracker || m_realtimePitchLayer) { - recordingFinishedFull(); + recordingFinishedFull(error == "" ? m_analyser : nullptr); } } diff --git a/main/MainWindow.h b/main/MainWindow.h index c7ade16d..6d017352 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -203,7 +203,7 @@ protected slots: // --- Real-time pitch tracking during microphone recording --- virtual void recordingStarted(); virtual void onRealtimePitchDetected(sv::sv_frame_t frame, double hz); - virtual void recordingFinishedFull(); + virtual void recordingFinishedFull(Analyser *analysing = nullptr); void moveOneNoteRight(); void moveOneNoteLeft(); @@ -322,6 +322,7 @@ protected slots: virtual void teardownSingingTrackAnalyser(); virtual void setupRealtimePitchLayer(); virtual void teardownRealtimePitchLayer(); + virtual void stopRealtimePitchTracker(); // Background music helpers: load/tear-down a non-analysed audio track // that plays alongside the reference track. @@ -397,6 +398,10 @@ protected slots: void refineRecordingLatency(); + // Set while the live dots of a finished take wait for pYIN to + // produce the pitch track that replaces them. + QMetaObject::Connection m_realtimeLayerTeardownConnection; + // Extra panes created by MainWindowBase::record() via AddPaneCommand // that we want to hide immediately but cannot delete yet because // m_analyser2 hasn't been set up yet (it is deferred via QTimer::singleShot). diff --git a/main/test/TestLatencyShift.h b/main/test/TestLatencyShift.h index 25ccb00c..02d2056a 100644 --- a/main/test/TestLatencyShift.h +++ b/main/test/TestLatencyShift.h @@ -110,6 +110,20 @@ private slots: QCOMPARE(computeRecordingLatency(-1, 256), sv::sv_frame_t(256)); QCOMPARE(computeRecordingLatency(-100, -100), sv::sv_frame_t(0)); } + + // Review finding 8: a live dot goes where the finished pitch track + // will put the same sound + void live_dot_shift() { + QCOMPARE(compensatedLiveFrame(5000, 768), sv::sv_frame_t(4232)); + QCOMPARE(compensatedLiveFrame(768, 768), sv::sv_frame_t(0)); + QCOMPARE(compensatedLiveFrame(5000, 0), sv::sv_frame_t(5000)); + // Sound from before the reference started has nowhere to go: + // the caller drops anything negative + QVERIFY(compensatedLiveFrame(767, 768) < 0); + QVERIFY(compensatedLiveFrame(0, 768) < 0); + // An unavailable latency must never pull the dot later + QCOMPARE(compensatedLiveFrame(5000, -1), sv::sv_frame_t(5000)); + } }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 2c241a09..73d3a00d 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -53,6 +53,7 @@ #include #include #include +#include #include #include #include @@ -109,6 +110,12 @@ class TestMainWindow : public MainWindow sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { + onRealtimePitchDetected(frame, hz); + } + QString statusText() { return getStatusLabel()->text(); } + void setStatusText(QString text) { getStatusLabel()->setText(text); } + protected: void createAudioIO() override { if (m_audioIO || m_playTarget) return; @@ -481,8 +488,10 @@ private slots: stopTake(); if (QTest::currentTestFailed()) return; + // The dots go when pYIN finishes, and the application hears of + // that an event-loop turn after the models say so + QTRY_VERIFY(!m_window->realtimeLayer()); QVERIFY(!m_window->realtimeTracker()); - QVERIFY(!m_window->realtimeLayer()); QVERIFY(m_window->realtimeModelId().isNone()); QVERIFY2(!sv::ModelById::get(liveModel), "the live pitch model outlived its layer"); @@ -490,6 +499,131 @@ private slots: QVERIFY(!m_window->recordingAsSingingTrack()); } + // Review finding 9: the pitch track replaces the dots. Between Stop + // and the end of pYIN the dots are all the singer has to look at + void live_dots_stay_until_analysis() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(1000); + sv::ModelId liveModel = m_window->realtimeModelId(); + QVERIFY(!liveModel.isNone()); + + // Stop. pYIN has been started and cannot have finished: its + // completion arrives through the event loop, which has not run + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!analysed(m_window->analyser2())); + + QVERIFY2(m_window->realtimeLayer(), + "the live dots went at Stop, before pYIN had anything " + "to show in their place"); + QVERIFY(paneHasLayer(0, m_window->realtimeLayer())); + auto model = sv::ModelById::getAs(liveModel); + QVERIFY(model); + int dots = model->getEventCount(); + QVERIFY(dots > 20); + + // The take is over all the same: nothing is tracking it, and + // the state is that of a finished take + QVERIFY(!m_window->realtimeTracker()); + QVERIFY(!m_window->recordingInProgress()); + QVERIFY(!m_window->recordingAsSingingTrack()); + + // and they go no sooner than the pitch track is complete + QTRY_VERIFY_WITH_TIMEOUT(!m_window->realtimeLayer(), 30000); + QVERIFY(m_window->analyser2()->getInitialAnalysisCompletion() >= 100); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + QVERIFY2(!sv::ModelById::get(liveModel), + "the live pitch model outlived its layer"); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + } + + // Review finding 10: pitch events still queued when the take ends + // must not draw dots or write to the status bar + void stale_pitch_event_ignored() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(600); + sv::ModelId liveModel = m_window->realtimeModelId(); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + + auto model = sv::ModelById::getAs(liveModel); + QVERIFY(model); + int dots = model->getEventCount(); + m_window->setStatusText("after the take"); + m_window->doRealtimePitchDetected(20000, 440.0); + QCOMPARE(model->getEventCount(), dots); + QCOMPARE(m_window->statusText(), QString("after the take")); + + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QTRY_VERIFY(!m_window->realtimeLayer()); + m_window->setStatusText("after the analysis"); + m_window->doRealtimePitchDetected(20000, 440.0); + QCOMPARE(m_window->statusText(), QString("after the analysis")); + } + + // Review finding 8: during the take, the dots sit where the pitch + // track will. Same device and singer as latency_end_to_end, but + // looked at before Stop + void live_dots_compensated() { + const int K = 3 * 4096; + FakeAudioIO::Config config; + config.playbackLatency = 2 * 4096; + config.recordLatency = 4096; + config.input = melody(0.75); + config.inputDelay = K; + config.inputFollowsPlayback = true; + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(melody(0.75))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(2200); + + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + auto dots = model->getAllEvents(); + stopTake(); + if (QTest::currentTestFailed()) return; + + sv::sv_frame_t refStep = stepFrame(pitchEvents(m_window->analyser())); + sv::sv_frame_t dotStep = stepFrame(dots); + QVERIFY(refStep > 0); + QVERIFY2(dotStep > 0, "the live dots never reached the second note"); + + // The dots are coarser than pYIN: allow them a YIN window + sv::sv_frame_t error = dotStep - refStep; + QVERIFY2(std::llabs(error) <= 2048, + qPrintable(QString("live step at %1, reference step at %2: " + "%3 frames (%4 ms) apart; the round trip " + "is %5 frames and the reference started " + "%6 frames into the take") + .arg(dotStep).arg(refStep).arg(error) + .arg(1000.0 * double(error) / rate, 0, 'f', 1) + .arg(K) + .arg(m_window->fake() + ->getFramesBeforePlayStart()))); + + // Nothing is drawn ahead of the reference + QVERIFY(dots.empty() || dots.front().getFrame() >= 0); + } + // The automated latency test. The device reports a round trip of K // frames, and its input is the reference melody starting K frames // after the first reference sample is played: a singer exactly on From e40d6d728f546d504ad7ad6349381ab384f25aa4 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 00:30:54 +0300 Subject: [PATCH 019/275] fix: reset take state when record fails, ignore Analyse Now mid-take, track the input mixdown Review findings 4, 5 and 11; their QEXPECT_FAIL markers are removed. - record(): the base class gives up without a signal when recording cannot start. Clear m_recordingAsSingingTrack and m_recordingInProgress then, so the next file opened is analysed and Analyse Now is not routed to a singing track that is not there. - analyseNow(): return early while a take is being recorded. Run mid-take it ended the take's bookkeeping, so the end of the take re-analysed the reference and discarded edits to its pitch track. recordCompleted is emitted after isRecording() goes false, so the end of a take is unaffected. - RealtimePitchTracker: read the mixdown (channel -1) rather than channel 0, as the pYIN transform does, so a microphone on the second input of a stereo interface gives live dots. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 22 ++++++++++++++++++++++ main/RealtimePitchTracker.cpp | 5 ++++- main/test/TestRealtimePitchTracker.h | 7 +++++-- main/test/TestRecordWorkflow.h | 12 ++++++++---- 4 files changed, 39 insertions(+), 7 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index ad70a76a..5829c8ed 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2792,6 +2792,17 @@ MainWindow::record() MainWindowBase::record(); + // The base class gives up without a signal when the device cannot be + // opened or the recording cannot be started. No take is coming then, + // so nothing must be left waiting for one: with the flag still set, + // the next file opened would not be analysed, and Analyse Now would + // be routed to a singing track that is not there. + if (!m_recordTarget || !m_recordTarget->isRecording()) { + cerr << "MainWindow::record: recording did not start" << endl; + m_recordingAsSingingTrack = false; + m_recordingInProgress = false; + } + // Restore the default mode so that a subsequent "standalone" recording // (after the singing track session is closed) behaves correctly. setAudioRecordMode(RecordReplaceSession); @@ -4338,6 +4349,17 @@ MainWindow::analyseNow() { cerr << "analyseNow called" << endl; + // Not during a take. The analysis of a take is started from here + // when the take ends (recordCompleted, by which time isRecording() + // is false). Run in the middle of one, it would analyse the part + // recorded so far and end the take's bookkeeping early, so that the + // end of the take would re-analyse the reference instead, discarding + // any edits made to its pitch track. + if (m_recordTarget && m_recordTarget->isRecording()) { + cerr << "analyseNow: recording in progress, ignoring" << endl; + return; + } + // When the user recorded a singing track alongside an existing reference // track (RecordCreateAdditionalModel mode), the recording becomes an // additional model, not the main model. In that case we must route diff --git a/main/RealtimePitchTracker.cpp b/main/RealtimePitchTracker.cpp index 0254e298..271d6369 100644 --- a/main/RealtimePitchTracker.cpp +++ b/main/RealtimePitchTracker.cpp @@ -92,7 +92,10 @@ RealtimePitchTracker::run() while (nextFrameToProcess + kWindowSize <= totalFrames) { - floatvec_t rawFv = audioModel->getData(0, nextFrameToProcess, kWindowSize); + // Channel -1 is the mixdown of all channels, which is also + // what the pYIN transform is given: the microphone need not + // be on the first input of the interface. + floatvec_t rawFv = audioModel->getData(-1, nextFrameToProcess, kWindowSize); if ((int)rawFv.size() < kWindowSize) break; diff --git a/main/test/TestRealtimePitchTracker.h b/main/test/TestRealtimePitchTracker.h index 7472d965..9ea637d2 100644 --- a/main/test/TestRealtimePitchTracker.h +++ b/main/test/TestRealtimePitchTracker.h @@ -306,9 +306,12 @@ private slots: settle(spy, expectedHops(rec->written)); tracker.stop(); - QEXPECT_FAIL("", "Review finding 11: the tracker reads channel 0 only", - Continue); QVERIFY(spy.count() > 0); + for (const auto &event : spy.events) { + double hz = event.hz; + QVERIFY2(std::abs(TestSignals::centsBetween(hz, 330.0)) < 10.0, + qPrintable(QString("%1 Hz").arg(hz))); + } } }; diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 73d3a00d..69657f69 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -816,8 +816,6 @@ private slots: m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); - QEXPECT_FAIL("", "Review finding 4: m_recordingAsSingingTrack is " - "left true when record() fails", Abort); QVERIFY(!m_window->recordingAsSingingTrack()); // so that the next file opened is analysed as usual @@ -845,14 +843,20 @@ private slots: m_window->doAnalyseNow(); QTest::qWait(600); QVERIFY(m_window->recordTarget()->isRecording()); + + // The take carries on as if nothing had been asked + QVERIFY(m_window->recordingAsSingingTrack()); + QVERIFY(m_window->recordingInProgress()); + QVERIFY(m_window->realtimeTracker()); + QVERIFY(m_window->realtimeLayer()); + QVERIFY(pitchEvents(m_window->analyser2()).empty()); + m_window->doRecord(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); sv::Layer *refLayerNow = m_window->analyser()->getLayer(Analyser::PitchTrack); - QEXPECT_FAIL("", "Review finding 5: Analyse Now during a take makes " - "the end of the take re-analyse the reference", Continue); QVERIFY2(refLayerNow == refLayer && refLayerNow->getModel() == refModel, "the reference pitch track was replaced"); } From 3a520a864b308f277b34391f68275455470ce569 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 00:50:44 +0300 Subject: [PATCH 020/275] fix: take released models out of the play source Ids of freed models collected in the play source: the live pitch model of every take, the empty model createEmptyLayer() made for the live dot layer, and the pitch track and note models of a replaced singing track. Forced layer deletes and setModel() never reach layerInAView(false), which was the only way out. The fix is in the svapp fork (ed817c2): the document announces each model release and MainWindowBase removes the model from the play source then; removeModel() works for a model that is already gone; continuous synths are deleted with their model. Here: the live dot layer is made with createLayer(), so no throwaway model; rerecord_cleans_up checks that every id in the play source is a live model that a layer uses; force_delete_skips_play_source checks the new signal. Review finding 6. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 20 +++++++++++------- main/test/TestRecordWorkflow.h | 36 +++++++++++++++++++++++++++++---- main/test/TestSingingDocument.h | 13 +++++++++--- repoint-lock.json | 2 +- 4 files changed, 56 insertions(+), 15 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 5829c8ed..0328bb94 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2447,6 +2447,10 @@ MainWindow::teardownSingingTrackAnalyser() // m_analyser2 is set up and its WaveformLayer holds the model reference. // closeSession() handles any residual hidden panes via its own // getHiddenPaneCount() loop using removeLayerFromView + deletePane. + // + // The forced deletes do not fire layerInAView(false), which is what + // usually takes a model out of the play source. The models go from + // there when the document releases them (modelAboutToBeReleased). if (m_document) { m_analyser2->removeAllLayers(); } else { @@ -2556,11 +2560,11 @@ MainWindow::setupRealtimePitchLayer() m_realtimePitchModelId = ModelById::add(pitchModel); m_document->addNonDerivedModel(m_realtimePitchModelId); - // Create a TimeValueLayer to display the pitch estimates. - // createEmptyLayer creates the layer with an appropriate empty model - // registered with the document; we then use document->setModel() to - // replace that empty model with our SparseTimeValueModel. - Layer *rawLayer = m_document->createEmptyLayer(LayerFactory::TimeValues); + // Create a TimeValueLayer to display the pitch estimates. Not with + // createEmptyLayer(): that gives the layer an empty model of its own, + // which goes into the play source only to be released again by the + // setModel() below. + Layer *rawLayer = m_document->createLayer(LayerFactory::TimeValues); m_realtimePitchLayer = qobject_cast(rawLayer); if (!m_realtimePitchLayer) { @@ -2758,8 +2762,10 @@ MainWindow::record() << prevSingingModelId << endl; m_document->deleteLayer(orphan, true); } - // Explicitly remove from play source — belt-and-suspenders in case - // deleteLayer(true)'s layerInAView(false) path didn't fire. + // deleteLayer(true) does not fire layerInAView(false). The + // model leaves the play source when it is released, which + // is normally in the teardown below; this makes sure of it + // even if something else still holds the model. if (m_playSource) { m_playSource->removeModel(prevSingingModelId); } diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 69657f69..f9e557ec 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -779,13 +779,24 @@ private slots: if (QTest::currentTestFailed()) return; int panes = m_window->paneStack()->getPaneCount(); - take(800); + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); + sv::ModelId firstLive = m_window->realtimeLayer()->getModel(); + QTest::qWait(800); + stopTake(); if (QTest::currentTestFailed()) return; Analyser *first = m_window->analyser2(); sv::ModelId firstModel = first->getMainModelId(); QVERIFY(sv::ModelById::get(firstModel)); - take(800); + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); + sv::ModelId secondLive = m_window->realtimeLayer()->getModel(); + QVERIFY(secondLive != firstLive); + QTest::qWait(800); + stopTake(); if (QTest::currentTestFailed()) return; QVERIFY(m_window->analyser2()); @@ -803,8 +814,25 @@ private slots: auto events = pitchEvents(m_window->analyser2()); QVERIFY(events.size() > 50); - // The play source's model list has no accessor, so whether it - // holds stale ids (finding 6) is not checked here + // Nothing of the first take is left in the play source: not + // its audio and not its live pitch model (review finding 6) + QTRY_VERIFY_WITH_TIMEOUT(!m_window->realtimeLayer(), 5000); + auto playing = m_window->playSource()->getModels(); + QVERIFY(!playing.count(firstModel)); + QVERIFY(!playing.count(firstLive)); + QVERIFY(!playing.count(secondLive)); + for (sv::ModelId id : playing) { + QVERIFY2(sv::ModelById::get(id), + qPrintable(QString("the play source holds model %1, " + "which no longer exists") + .arg(id.untyped))); + QVERIFY2(layersOnModel(id) > 0, + qPrintable(QString("the play source holds model %1, " + "which no layer uses") + .arg(id.untyped))); + } + // audio, pitch track and notes, of the reference and of the take + QCOMPARE(int(playing.size()), 6); } // No device at all: the base class record() gives up quietly diff --git a/main/test/TestSingingDocument.h b/main/test/TestSingingDocument.h index 71d322bf..fe3522a3 100644 --- a/main/test/TestSingingDocument.h +++ b/main/test/TestSingingDocument.h @@ -184,22 +184,29 @@ private slots: } void force_delete_skips_play_source() { - // layerInAView(layer, false) is what takes a model out of the - // play source. A forced delete never emits it (review finding - // 6), so callers have to remove the model themselves. + // layerInAView(layer, false) is what usually takes a model out + // of the play source. A forced delete never emits it (review + // finding 6); what the main window goes by then is + // modelAboutToBeReleased. Extra viaCommand = addExtraPane(false); Extra viaForce = addExtraPane(false); QVERIFY(viaCommand.waveform && viaForce.waveform); QSignalSpy inAView(m_document, SIGNAL(layerInAView(Layer *, bool))); QVERIFY(inAView.isValid()); + QSignalSpy released(m_document, + SIGNAL(modelAboutToBeReleased(ModelId))); + QVERIFY(released.isValid()); m_document->removeLayerFromView(viaCommand.pane, viaCommand.waveform); QCOMPARE(int(inAView.count()), 1); QCOMPARE(inAView.at(0).at(1).toBool(), false); + QCOMPARE(int(released.count()), 0); m_document->deleteLayer(viaForce.waveform, true); QCOMPARE(int(inAView.count()), 1); + QCOMPARE(int(released.count()), 1); + QVERIFY(!modelAlive(viaForce.model)); } // --- The pruning helper ---------------------------------------------- diff --git a/repoint-lock.json b/repoint-lock.json index d942da38..f4e9ff74 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -10,7 +10,7 @@ "pin": "271db78f54eff467e3891fca920ab663cc69ce53" }, "svapp": { - "pin": "fc4c6b375b01bb12aec39df9604f841e27588bc3" + "pin": "ed817c2b27d06c02c53451036417be564dc0702b" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" From d8770a3f2498fe07fd240190a27edcfcb4c70490 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 01:19:46 +0300 Subject: [PATCH 021/275] fix: set the reference up once, give the live pitch model the hop size audioFileLoaded() is emitted for additional models too, so loading a singing track or background music ran analyseNewMainModel() again on an unchanged main model. That handed the reference to its analyser a second time, which kept its layers but forgot its pitch candidates (leaving their layers in the pane), and connected the pane's regionOutlined() to the window once more. Remember the main model last analysed and return early when it has not changed. The early return also skips the documentRestored() that used to mark the document unmodified right after a track had been added to it; it now stays modified. The live pitch model was made with resolution 512 against a hop of 256. kWindowSize and kHopSize are public now and the model uses kHopSize. Tests first: layersChanged() spies in load_singing_track and load_background_music, and a resolution check in live_dots_appear, all three failing before the fix. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 17 +++++++++++++++-- main/MainWindow.h | 6 ++++++ main/RealtimePitchTracker.h | 11 ++++++----- main/test/TestRecordWorkflow.h | 18 ++++++++++++++++-- 4 files changed, 43 insertions(+), 9 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0328bb94..4493d834 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2019,6 +2019,7 @@ MainWindow::closeSession() m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; + m_analysedMainModelId = {}; m_analyser->fileClosed(); @@ -2551,10 +2552,11 @@ MainWindow::setupRealtimePitchLayer() } // Create a SparseTimeValueModel to receive pitch estimates. - // Resolution 256 frames matches the YIN hop size in RealtimePitchTracker. + // Its resolution is the YIN hop size: one estimate per hop. // Unit "Hz" is required so TimeValueLayer::shouldAutoAlign() defers to // the pane's log-frequency coordinate system (same as the pYIN pitch track). - auto pitchModel = std::make_shared(sr, 512, false); + auto pitchModel = std::make_shared + (sr, RealtimePitchTracker::kHopSize, false); pitchModel->setObjectName(tr("Realtime Pitch (Live)")); pitchModel->setScaleUnits("Hz"); m_realtimePitchModelId = ModelById::add(pitchModel); @@ -4499,6 +4501,17 @@ MainWindow::analyseNewMainModel() return; } + // openAudio() emits audioFileLoaded() for CreateAdditionalModel too (a + // singing track, background music), with the main model unchanged. + // Going on would hand the reference to m_analyser a second time, which + // keeps its layers but forgets its pitch candidates, and would connect + // the pane's regionOutlined() to us once more. + if (getMainModelId() == m_analysedMainModelId) { + SVDEBUG << "MainWindow::analyseNewMainModel: main model unchanged, nothing to do" << endl; + return; + } + m_analysedMainModelId = getMainModelId(); + int pc = m_paneStack->getPaneCount(); Pane *pane = 0; Pane *selectionStrip = 0; diff --git a/main/MainWindow.h b/main/MainWindow.h index 6d017352..a1f1a7d9 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -348,6 +348,12 @@ protected slots: // pick it up on the next event-loop iteration. sv::ModelId m_pendingSingingModelId; + // The main model last handed to m_analyser by analyseNewMainModel(). + // audioFileLoaded() is emitted for additional models too (a singing + // track, background music), and the reference must not be set up again + // for those. Cleared in closeSession(). + sv::ModelId m_analysedMainModelId; + // True while a microphone recording is in progress (set in // recordingStarted(), cleared in recordingFinishedFull()). bool m_recordingInProgress; diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h index 2d3b1884..9b9cbbe1 100644 --- a/main/RealtimePitchTracker.h +++ b/main/RealtimePitchTracker.h @@ -51,6 +51,12 @@ class RealtimePitchTracker : public QThread Q_OBJECT public: + // Window size: 2048 samples @ 44100 Hz ≈ 46 ms. + // Hop size: 256 samples ≈ 5.8 ms. One estimate per hop, so this is + // also the resolution of the model the estimates go into. + static constexpr int kWindowSize = 2048; + static constexpr int kHopSize = 256; + /** * @param audioSourceId ModelId of the WritableWaveFileModel being * recorded into. Polled from the background thread. @@ -109,11 +115,6 @@ class RealtimePitchTracker : public QThread double m_maxFreq; double m_threshold; - // Window size: 2048 samples @ 44100 Hz ≈ 46 ms. - // Hop size: 256 samples ≈ 5.8 ms. - static const int kWindowSize = 2048; - static const int kHopSize = 256; - // --- YIN helpers (all called only from run()) --- static void yinDifferenceFFT(const std::vector &buf, diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index f9e557ec..1e31801b 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -83,6 +83,7 @@ class TestMainWindow : public MainWindow // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } + bool isDocumentModified() { return m_documentModified; } void doCloseSession() { discardModifications(); closeSession(); } void setPlayReferenceWhileRecording(bool on) { @@ -463,6 +464,9 @@ private slots: auto model = sv::ModelById::getAs (m_window->realtimeModelId()); QVERIFY(model); + // One dot per hop (review finding 12) + QCOMPARE(int(model->getResolution()), + int(RealtimePitchTracker::kHopSize)); auto events = model->getAllEvents(); QVERIFY2(events.size() > 20, qPrintable(QString("only %1 live dots after a second") @@ -897,6 +901,8 @@ private slots: sv::Layer *refLayer = m_window->analyser()->getLayer(Analyser::PitchTrack); sv::ModelId refModel = refLayer->getModel(); + QSignalSpy refSetUp(m_window->analyser(), SIGNAL(layersChanged())); + QVERIFY(refSetUp.isValid()); m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); @@ -917,11 +923,16 @@ private slots: QVERIFY(std::fabs(TestSignals::centsBetween (medianHz(pitchEvents(a2)), highHz)) < 10.0); - // The open half of finding 7: the reference is handed to its - // analyser a second time, which today changes nothing + // The other half of finding 7: the reference is not handed to + // its analyser a second time. That would keep its layers, but + // forget its pitch candidates and leave them in the pane + QCOMPARE(int(refSetUp.count()), 0); QCOMPARE(m_window->analyser()->getLayer(Analyser::PitchTrack), refLayer); QCOMPARE(refLayer->getModel(), refModel); + + // The second pass used to end by marking the document unmodified + QVERIFY(m_window->isDocumentModified()); } void load_background_music() { @@ -930,9 +941,12 @@ private slots: if (QTest::currentTestFailed()) return; int panes = m_window->paneStack()->getPaneCount(); int rulerPaneLayers = m_window->paneStack()->getPane(1)->getLayerCount(); + QSignalSpy refSetUp(m_window->analyser(), SIGNAL(layersChanged())); + QVERIFY(refSetUp.isValid()); m_window->doLoadBackgroundMusic(writeWav(tone(highHz, 1.0))); QCoreApplication::processEvents(); + QCOMPARE(int(refSetUp.count()), 0); sv::ModelId music = m_window->backgroundMusicModelId(); QVERIFY(!music.isNone()); From 66842a9c901397a0208d903bb370f8b8b610832f Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 01:44:26 +0300 Subject: [PATCH 022/275] fix: keep the take out of the mix while it is recorded; tests for untested gaps The take's waveform was audible during recording, and stayed out of the output only because the play source reads ahead of what has been recorded. setupSingingTrackAnalyser() now mutes it for a deferred (recording) setup and recordingFinishedFull() gives it back. While it is muted, Play Singing Audio shows and toggles the state wanted after the take; that toggle is not written to the settings the reference shares. The live pitch model needed nothing: sparse models are muted by default. take_muted_while_recording was written first and failed on the take being audible. Tests for what the review listed as untested: - reference_candidates_survive_load: pitch candidates on the reference survive loading a singing track and background music (fails with the early return in analyseNewMainModel() disabled) - reload_singing_track: a singing track loaded over another leaves nothing of the first in the play source - record_without_reference: the standalone branch of record() - stereo_other_input_noisy: noise on the unused input - load_background_music also checks the modified flag Also corrects the comment in setupRealtimePitchLayer() that still described the tracker as a QTimer poller. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 50 +++++++- main/MainWindow.h | 9 ++ main/test/TestRealtimePitchTracker.h | 21 ++++ main/test/TestRecordWorkflow.h | 182 +++++++++++++++++++++++++++ 4 files changed, 257 insertions(+), 5 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4493d834..ccc0abfc 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -147,6 +147,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_withSpectrogram(withSpectrogram), m_recordingInProgress(false), m_recordingAsSingingTrack(false), + m_singingAudioMutedForTake(false), + m_singingAudioAfterTake(true), m_paneCountBeforeRecording(0), m_currentRecordingModelId(), m_recordingLatencyFrames(0), @@ -1835,7 +1837,10 @@ MainWindow::playNotesToggled() void MainWindow::playSingingAudioToggled() { - if (m_analyser2) { + if (m_singingAudioMutedForTake) { + // Muted whatever the button says; it takes effect after the take + m_singingAudioAfterTake = !m_singingAudioAfterTake; + } else if (m_analyser2) { m_analyser2->toggleAudible(Analyser::Audio); } updateLayerStatuses(); @@ -1891,7 +1896,9 @@ MainWindow::updateLayerStatuses() if (m_playSingingAudio) { m_playSingingAudio->setEnabled(m_analyser2 != nullptr); - if (m_analyser2) { + if (m_singingAudioMutedForTake) { + m_playSingingAudio->setChecked(m_singingAudioAfterTake); + } else if (m_analyser2) { m_playSingingAudio->setChecked(m_analyser2->isAudible(Analyser::Audio)); } else { m_playSingingAudio->setChecked(true); // default on when track arrives @@ -2389,6 +2396,21 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal // After deletePane() the pointer would be dangling → crash. drainPendingExtraPanes(singingModelId); + // A take being recorded stays out of the mix until it is over. The + // play source happens to read ahead of what has been recorded, so the + // take is silent anyway with the buffer sizes of today; this does not + // depend on that. Not with setAudible(), which would write the state + // to the settings the reference shares. + if (deferAnalysis) { + m_singingAudioAfterTake = m_analyser2->isAudible(Analyser::Audio); + if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { + if (auto params = audio->getPlayParameters()) { + params->setPlayAudible(false); + m_singingAudioMutedForTake = true; + } + } + } + // Re-stack layers so the primary pitch track stays on top m_analyser->getLayer(Analyser::PitchTrack); // ensure primary is on top updateLayerStatuses(); @@ -2428,9 +2450,25 @@ MainWindow::drainPendingExtraPanes(sv::ModelId singingModelId) m_pendingExtraPanes.clear(); } +void +MainWindow::restoreSingingAudioAfterTake() +{ + if (!m_singingAudioMutedForTake) return; + m_singingAudioMutedForTake = false; + if (!m_analyser2) return; + if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { + if (auto params = audio->getPlayParameters()) { + params->setPlayAudible(m_singingAudioAfterTake); + } + } + updateLayerStatuses(); +} + void MainWindow::teardownSingingTrackAnalyser() { + m_singingAudioMutedForTake = false; + if (!m_analyser2) return; // removeAllLayers() removes each layer from the pane and deletes it from @@ -2591,9 +2629,10 @@ MainWindow::setupRealtimePitchLayer() m_document->addLayerToView(pane, m_realtimePitchLayer); - // Create and start the pitch tracker. - // It will poll audioSourceId (the WritableWaveFileModel) for new frames - // on each QTimer tick and write estimates into m_realtimePitchModelId. + // Create and start the pitch tracker. Its thread reads new frames + // from audioSourceId (the WritableWaveFileModel) and emits + // pitchDetected(); onRealtimePitchDetected() writes the estimates into + // m_realtimePitchModelId on this thread. m_realtimePitchTracker = new RealtimePitchTracker( audioSourceId, this); connect(m_realtimePitchTracker, &RealtimePitchTracker::pitchDetected, @@ -3034,6 +3073,7 @@ MainWindow::recordingFinishedFull(Analyser *analysing) m_recordingInProgress = false; m_recordingAsSingingTrack = false; m_currentRecordingModelId = {}; + restoreSingingAudioAfterTake(); if (analysing && m_realtimePitchLayer) { stopRealtimePitchTracker(); diff --git a/main/MainWindow.h b/main/MainWindow.h index a1f1a7d9..bb135233 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -348,6 +348,15 @@ protected slots: // pick it up on the next event-loop iteration. sv::ModelId m_pendingSingingModelId; + // A take is kept out of the playback mix while it is being recorded: + // with the reference playing, the singer would otherwise hear + // themselves late, and on speakers that goes back into the microphone. + // m_singingAudioAfterTake is what Play Singing Audio asks for, and + // what the take is given when the recording is over. + bool m_singingAudioMutedForTake; + bool m_singingAudioAfterTake; + void restoreSingingAudioAfterTake(); + // The main model last handed to m_analyser by analyseNewMainModel(). // audioFileLoaded() is emitted for additional models too (a singing // track, background music), and the reference must not be set up again diff --git a/main/test/TestRealtimePitchTracker.h b/main/test/TestRealtimePitchTracker.h index 9ea637d2..3fc9ebba 100644 --- a/main/test/TestRealtimePitchTracker.h +++ b/main/test/TestRealtimePitchTracker.h @@ -313,6 +313,27 @@ private slots: qPrintable(QString("%1 Hz").arg(hz))); } } + + void stereo_other_input_noisy() { + // As above, but the unused input is not silent: the mixdown + // carries its noise along with the voice + auto rec = makeRecording(2); + RealtimePitchTracker tracker(rec->id); + PitchCollector spy(&tracker); + tracker.start(); + const int n = int(kRate) / 2; + rec->append({ TestSignals::whiteNoise(n, 7, 0.02), + TestSignals::sine(330.0, kRate, n) }); + settle(spy, expectedHops(rec->written)); + tracker.stop(); + + QVERIFY(spy.count() > expectedHops(rec->written) / 2); + for (const auto &event : spy.events) { + double hz = event.hz; + QVERIFY2(std::abs(TestSignals::centsBetween(hz, 330.0)) < 20.0, + qPrintable(QString("%1 Hz").arg(hz))); + } + } }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 1e31801b..9822ca54 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -89,6 +89,7 @@ class TestMainWindow : public MainWindow void setPlayReferenceWhileRecording(bool on) { m_playRefWhileRecording->setChecked(on); } + QAction *playSingingAudioAction() { return m_playSingingAudio; } Analyser *analyser() { return m_analyser; } Analyser *analyser2() { return m_analyser2; } @@ -283,6 +284,21 @@ class TestRecordWorkflow : public QObject return 2.0 * std::sqrt(re * re + im * im) / double(n); } + // Every model the play source holds is alive and in use by a layer + // (review finding 6) + void verifyPlaySourceClean() { + for (sv::ModelId id : m_window->playSource()->getModels()) { + QVERIFY2(sv::ModelById::get(id), + qPrintable(QString("the play source holds model %1, " + "which no longer exists") + .arg(id.untyped))); + QVERIFY2(layersOnModel(id) > 0, + qPrintable(QString("the play source holds model %1, " + "which no layer uses") + .arg(id.untyped))); + } + } + int layersOnModel(sv::ModelId id) { int n = 0; for (sv::Layer *layer : m_window->document()->getLayers()) { @@ -380,6 +396,12 @@ private slots: settings.beginGroup("MainWindow"); settings.setValue("playrefwhilerecording", false); settings.endGroup(); + + // The audible flags are shared by both analysers; a test that + // failed half way must not leave the next one's tracks muted + settings.beginGroup("Analyser"); + settings.remove(""); + settings.endGroup(); } void cleanup() { @@ -775,6 +797,66 @@ private slots: "pitch, at amplitude %1").arg(synth))); } + // Review finding 3. The two tests above find the take and the live + // pitch silent in the output, but only because the play source reads + // ahead of what has been recorded. Neither is to be audible while it + // is being recorded, whatever the buffers do. + void take_muted_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + auto takeParams = [this]() { + return m_window->analyser2()->getLayer(Analyser::Audio) + ->getPlayParameters(); + }; + + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && + m_window->analyser2()->getLayer + (Analyser::Audio), 2000); + QVERIFY(m_window->realtimeLayer()); + auto liveParams = m_window->realtimeLayer()->getPlayParameters(); + QVERIFY(liveParams); + QVERIFY2(!liveParams->isPlayAudible(), + "the live pitch model is audible during the take"); + QVERIFY(takeParams()); + QVERIFY2(!takeParams()->isPlayAudible(), + "the take is audible while it is being recorded"); + + // The button goes on saying what the user asked for + QVERIFY(m_window->playSingingAudioAction()->isChecked()); + + QTest::qWait(800); + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY2(takeParams()->isPlayAudible(), + "the take was left muted after recording"); + QVERIFY(m_window->playSingingAudioAction()->isChecked()); + + // Switched off during a take, it stays muted afterwards + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && + m_window->analyser2()->getLayer + (Analyser::Audio), 2000); + m_window->playSingingAudioAction()->trigger(); + QVERIFY(!m_window->playSingingAudioAction()->isChecked()); + QVERIFY(!takeParams()->isPlayAudible()); + QTest::qWait(800); + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!takeParams()->isPlayAudible()); + QVERIFY(!m_window->playSingingAudioAction()->isChecked()); + + // The reference was not touched by any of this + QVERIFY(m_window->analyser()->isAudible(Analyser::Audio)); + } + void rerecord_cleans_up() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); @@ -839,6 +921,40 @@ private slots: QCOMPARE(int(playing.size()), 6); } + // With nothing loaded the take is not a singing track: it becomes + // the main model and the primary analyser gets it + void record_without_reference() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->recordingAsSingingTrack()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); + QTRY_VERIFY_WITH_TIMEOUT + (!pitchEvents(m_window->realtimeLayer()).empty(), 3000); + QTest::qWait(800); + + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + + QVERIFY(!m_window->analyser2()); + QVERIFY(!m_window->mainModelId().isNone()); + QCOMPARE(m_window->analyser()->getMainModelId(), + m_window->mainModelId()); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + highHz)) < 10.0); + + QTRY_VERIFY_WITH_TIMEOUT(!m_window->realtimeLayer(), 5000); + QVERIFY(!m_window->realtimeTracker()); + QVERIFY(!m_window->recordingInProgress()); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + verifyPlaySourceClean(); + } + // No device at all: the base class record() gives up quietly void record_failure_resets_flags() { makeWindow(FakeAudioIO::Config(), false); @@ -935,6 +1051,71 @@ private slots: QVERIFY(m_window->isDocumentModified()); } + // Loading over a singing track that is already there: the path + // through teardownSingingTrackAnalyser() that record() does not take + void reload_singing_track() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + + m_window->loadSingingTrack(writeWav(tone(highHz, 2.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + sv::ModelId first = m_window->analyser2()->getMainModelId(); + sv::ModelId firstPitch = m_window->analyser2() + ->getLayer(Analyser::PitchTrack)->getModel(); + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + sv::ModelId second = m_window->analyser2()->getMainModelId(); + QVERIFY(second != first); + + QVERIFY2(!sv::ModelById::get(first), + "the first singing track's model was not released"); + QVERIFY(!sv::ModelById::get(firstPitch)); + QCOMPARE(layersOnModel(second), 1); + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + + auto playing = m_window->playSource()->getModels(); + QVERIFY(!playing.count(first)); + QVERIFY(!playing.count(firstPitch)); + verifyPlaySourceClean(); + if (QTest::currentTestFailed()) return; + // audio, pitch track and notes, of the reference and of the track + QCOMPARE(int(playing.size()), 6); + + // A stale id used to keep the end of playback where the longest + // model ever loaded had ended + QVERIFY(m_window->playSource()->getPlayEndFrame() < + sv::sv_frame_t(1.2 * rate)); + } + + // Finding 7, the scenario itself: pitch candidates on the reference + // are still the analyser's after another track has been loaded + void reference_candidates_survive_load() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + Analyser *a = m_window->analyser(); + + QString error = a->reAnalyseSelection + (sv::Selection(sv::sv_frame_t(0.5 * rate), + sv::sv_frame_t(1.5 * rate)), + Analyser::FrequencyRange()); + QVERIFY2(error.isEmpty(), qPrintable(error)); + QTRY_VERIFY_WITH_TIMEOUT(a->haveHigherPitchCandidate(), 30000); + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QVERIFY2(a->haveHigherPitchCandidate(), + "the reference analyser forgot its pitch candidates"); + + m_window->doLoadBackgroundMusic(writeWav(tone(highHz, 1.0))); + QCoreApplication::processEvents(); + QVERIFY2(a->haveHigherPitchCandidate(), + "the reference analyser forgot its pitch candidates"); + } + void load_background_music() { makeWindow(FakeAudioIO::Config()); openReference(writeWav(tone(lowHz, 1.0))); @@ -947,6 +1128,7 @@ private slots: m_window->doLoadBackgroundMusic(writeWav(tone(highHz, 1.0))); QCoreApplication::processEvents(); QCOMPARE(int(refSetUp.count()), 0); + QVERIFY(m_window->isDocumentModified()); sv::ModelId music = m_window->backgroundMusicModelId(); QVERIFY(!music.isNone()); From 222a488815e7eae75ee449ac2eb74a33fc991430 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 20:50:12 +0300 Subject: [PATCH 023/275] fix: reset latency compensation for every take; make weak tests able to fail record() reset the latency and start-gap state only for a take with a reference, so a standalone take after a compensated one had its live dots shifted by the old latency. The resets now run for every take, below the early return for a Stop. Tests: - standalone_take_after_compensated_take covers the regression. - prune_none_id_deletes_nothing now holds a layer without a model, so it fails when the isNone() guard is removed. - record_failure_resets_flags asserts both flags and that Analyse Now re-analyses the reference afterwards. - take_silent_in_output_after_reseek shows the take mute at the device output, past the play source's read-ahead. - close_session_during_analysis asserts a transform is running before the close; stale_pitch_event_ignored queues a real event. - A check in live_dots_compensated that could not fail is removed. Comments in record(), MainWindow.h and PaneUtils.cpp corrected. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 20 +++- main/MainWindow.h | 4 +- main/PaneUtils.cpp | 7 +- main/test/TestRecordWorkflow.h | 201 ++++++++++++++++++++++++++++---- main/test/TestSingingDocument.h | 15 ++- 5 files changed, 213 insertions(+), 34 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index ccc0abfc..aa6629e2 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2721,6 +2721,17 @@ MainWindow::record() bool haveReference = (getMainModel() != nullptr); + // A new take starts with no latency compensation, whichever kind of take + // it is: the live dots are drawn with these, and a standalone take must + // not inherit the shift of a singing take made before it. The figures + // for a singing take are computed in recordingStarted(). This is below + // the early return above on purpose: a Stop must leave them alone, the + // shift is applied afterwards in analyseNow(). + m_recordingLatencyFrames = 0; + m_recordingStartGapEstimate = 0; + m_awaitingReferenceStart = false; + m_recordingStartGapMeasured = -1; + if (haveReference) { cerr << "MainWindow::record: reference track loaded — recording as singing track" << endl; @@ -2822,10 +2833,6 @@ MainWindow::record() m_recordingInProgress = false; m_recordingAsSingingTrack = true; - m_recordingLatencyFrames = 0; // reset; will be computed in recordingStarted() - m_recordingStartGapEstimate = 0; - m_awaitingReferenceStart = false; - m_recordingStartGapMeasured = -1; // Remember pane count so we can prune the extra pane that // MainWindowBase::record() creates via AddPaneCommand for the // recording's waveform layer. We want both tracks in pane 0. @@ -2842,8 +2849,9 @@ MainWindow::record() // The base class gives up without a signal when the device cannot be // opened or the recording cannot be started. No take is coming then, // so nothing must be left waiting for one: with the flag still set, - // the next file opened would not be analysed, and Analyse Now would - // be routed to a singing track that is not there. + // Analyse Now would be routed to a singing track that is not there, + // and the reference would not be re-analysed. (Opening a file is not + // affected: that closes the session, which clears the flag.) if (!m_recordTarget || !m_recordTarget->isRecording()) { cerr << "MainWindow::record: recording did not start" << endl; m_recordingAsSingingTrack = false; diff --git a/main/MainWindow.h b/main/MainWindow.h index bb135233..df967e57 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -396,7 +396,9 @@ protected slots: // sample rate) stored when a singing-track recording is made with the // "play reference while recording" toggle on. Applied as a negative // start-frame offset to the singing model so its timeline aligns with - // the reference during playback. Reset to 0 at the start of each recording. + // the reference during playback, and to the live dots during the take. + // Reset to 0 in record() at the start of every take, standalone ones + // included, but not by a Stop: analyseNow() applies it after that. // // It also includes the start gap: the part of the take recorded before // the reference began to play. That starts out as an estimate made just diff --git a/main/PaneUtils.cpp b/main/PaneUtils.cpp index 84cdc0df..209b48e8 100644 --- a/main/PaneUtils.cpp +++ b/main/PaneUtils.cpp @@ -58,8 +58,11 @@ pruneExtraPane(Document *document, PaneStack *paneStack, Pane *extra, // deletePane() leaves no dangling pointer. // // We distinguish them by whether the layer's model is ownedModelId. - // The ruler's own model id is "none", so an empty ownedModelId must - // match nothing. + // The ruler's model is the main model (createMainModelLayer() gives + // it that id), so it never matches the id of an additional model. + // A layer that has no model at all has the "none" id, though, so an + // empty ownedModelId must match nothing: without the isNone() guard + // such a layer would be deleted rather than detached. if (!extra) return; if (document) { diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 9822ca54..84e7f6c6 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -583,6 +583,18 @@ private slots: if (QTest::currentTestFailed()) return; QTest::qWait(600); sv::ModelId liveModel = m_window->realtimeModelId(); + + // The real case: an event queued before the take ends and + // delivered after it. Frame 20000 is well inside the take, and + // with no reference playing there is no latency to take off + // it, so it would make a dot. The tracker may have queued + // events of its own as well; the same goes for them + QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(0)); + QVERIFY2(QMetaObject::invokeMethod + (m_window, "onRealtimePitchDetected", Qt::QueuedConnection, + Q_ARG(sv::sv_frame_t, sv::sv_frame_t(20000)), + Q_ARG(double, 440.0)), + "the pitch event could not be queued"); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); @@ -590,6 +602,19 @@ private slots: QVERIFY(model); int dots = model->getEventCount(); m_window->setStatusText("after the take"); + QCoreApplication::processEvents(); + QVERIFY2(model->getEventCount() == dots, + qPrintable(QString("a pitch event queued during the take " + "and delivered after it drew a dot: %1 " + "dots, there were %2") + .arg(model->getEventCount()).arg(dots))); + QVERIFY2(m_window->statusText() == QString("after the take"), + qPrintable(QString("a pitch event queued during the take " + "and delivered after it wrote \"%1\" to " + "the status bar") + .arg(m_window->statusText()))); + + // and one that arrives later still m_window->doRealtimePitchDetected(20000, 440.0); QCOMPARE(model->getEventCount(), dots); QCOMPARE(m_window->statusText(), QString("after the take")); @@ -645,9 +670,6 @@ private slots: .arg(K) .arg(m_window->fake() ->getFramesBeforePlayStart()))); - - // Nothing is drawn ahead of the reference - QVERIFY(dots.empty() || dots.front().getFrame() >= 0); } // The automated latency test. The device reports a round trip of K @@ -729,10 +751,13 @@ private slots: } // The take and the live pitch model are both in the play source - // while the reference plays (review finding 3), but neither is - // heard: the play source fills its buffers seconds ahead of the - // playback position, and the take has no audio that far ahead yet. - // This test holds that in place. + // while the reference plays (review finding 3), and neither is to be + // heard. This test holds that in place for the whole path, from the + // input to the device output. It does not by itself show that the + // take is muted: the play source fills its buffers seconds ahead of + // the playback position, the take has no audio that far ahead yet, + // and so a take this short is silent even unmuted. For the mute see + // take_muted_while_recording and take_silent_in_output_after_reseek. // // Pure tones here, so that the reference has nothing at the // frequency of the input. 26400 samples is a whole number of @@ -763,8 +788,13 @@ private slots: "in it, at amplitude %1").arg(input))); } - // As above with the take's own waveform muted, leaving only the - // synth that follows the live pitch model + // The live pitch model is a SparseTimeValueModel, which would be + // played as a synth tone at the pitch it holds; such a model is + // inaudible unless something switches it on, and nothing does. + // This test holds that at the output, in a later window than the + // test above. The read-ahead covers for it in the same way, though: + // what shows the live model silent when it could be heard is + // take_silent_in_output_after_reseek void no_synth_tone() { FakeAudioIO::Config config; config.input = TestSignals::sine(highHz, rate, int(3 * rate), 0.5); @@ -776,13 +806,6 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; - QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && - m_window->analyser2()->getLayer - (Analyser::Audio), 2000); - auto params = m_window->analyser2()->getLayer(Analyser::Audio) - ->getPlayParameters(); - QVERIFY(params); - params->setPlayAudible(false); QTest::qWait(1800); stopTake(); if (QTest::currentTestFailed()) return; @@ -798,9 +821,11 @@ private slots: } // Review finding 3. The two tests above find the take and the live - // pitch silent in the output, but only because the play source reads - // ahead of what has been recorded. Neither is to be audible while it - // is being recorded, whatever the buffers do. + // pitch silent in the output, but for the take that much is true + // even unmuted, because the play source reads ahead of what has + // been recorded. Neither is to be audible while it is being + // recorded, whatever the buffers do: this test checks the play + // parameters, and the next one the output. void take_muted_while_recording() { FakeAudioIO::Config config; config.input = tone(highHz, 4.0); @@ -857,6 +882,55 @@ private slots: QVERIFY(m_window->analyser()->isAudible(Analyser::Audio)); } + // Review finding 3, at the device. The read-ahead that keeps the + // take out of the output in no_self_monitoring is taken away here: + // once more has been recorded than the play source buffers, playback + // is sent back to the start, so that everything the fill thread now + // reads is audio the take already has, and pitches the live model + // already has. Only their being muted keeps them out. + void take_silent_in_output_after_reseek() { + FakeAudioIO::Config config; + config.input = TestSignals::sine(highHz, rate, int(8 * rate), 0.5); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(TestSignals::sine(lowHz, rate, + int(6 * rate), 0.5))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(3600); + + auto wave = sv::ModelById::getAs + (m_window->currentRecordingModelId()); + QVERIFY2(wave, "there is no take being recorded"); + sv::sv_frame_t recorded = wave->getFrameCount(); + QVERIFY2(recorded > sv::sv_frame_t(3.2 * rate), + qPrintable(QString("only %1 frames of the take exist, not " + "enough to outrun the read-ahead") + .arg(recorded))); + + size_t reseek = m_window->fake()->getCapturedOutput().size(); + m_window->playSource()->play(0); + QTest::qWait(1500); + stopTake(); + if (QTest::currentTestFailed()) return; + + auto output = m_window->fake()->getCapturedOutput(); + size_t from = reseek + size_t(0.3 * rate); + double reference = amplitudeAt(output, from, 26400, lowHz); + double input = amplitudeAt(output, from, 26400, highHz); + QVERIFY2(reference > 0.1, + qPrintable(QString("reference amplitude in the output after " + "the reseek is %1").arg(reference))); + QVERIFY2(input >= 0.0 && input < 0.005, + qPrintable(QString("the output has the input's frequency in " + "it, at amplitude %1 (reference %2): the " + "take, or a tone at the live pitch, is " + "played while it is being recorded") + .arg(input).arg(reference))); + } + void rerecord_cleans_up() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); @@ -955,6 +1029,56 @@ private slots: verifyPlaySourceClean(); } + // The latency of a compensated take must not outlive it. A take + // with nothing loaded has no reference to line up with: its dots + // belong where they were heard, the first of them at the centre + // of the first YIN window + void standalone_take_after_compensated_take() { + FakeAudioIO::Config config; + config.playbackLatency = 4096; + config.recordLatency = 4096; + config.input = tone(highHz, 6.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_window->recordingLatencyFrames() >= 8192, + qPrintable(QString("the first take was compensated by %1 " + "frames only; the device reports 8192") + .arg(m_window->recordingLatencyFrames()))); + QTRY_VERIFY_WITH_TIMEOUT(!m_window->realtimeLayer(), 5000); + m_window->doCloseSession(); + + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->recordingAsSingingTrack()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); + QTRY_VERIFY_WITH_TIMEOUT + (!pitchEvents(m_window->realtimeLayer()).empty(), 3000); + QTest::qWait(800); + + sv::sv_frame_t latency = m_window->recordingLatencyFrames(); + auto dots = pitchEvents(m_window->realtimeLayer()); + QVERIFY(!dots.empty()); + sv::sv_frame_t firstDot = dots.front().getFrame(); + + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + + QVERIFY2(latency == 0 && + firstDot >= sv::sv_frame_t(RealtimePitchTracker::kWindowSize / 2), + qPrintable(QString("the standalone take ran with a latency " + "of %1 frames and its first dot at frame " + "%2; there is nothing to compensate for, " + "and no dot can come before frame %3") + .arg(latency).arg(firstDot) + .arg(RealtimePitchTracker::kWindowSize / 2))); + } + // No device at all: the base class record() gives up quietly void record_failure_resets_flags() { makeWindow(FakeAudioIO::Config(), false); @@ -965,13 +1089,35 @@ private slots: QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY(!m_window->recordingInProgress()); - // so that the next file opened is analysed as usual - openReference(writeWav(tone(highHz, 1.0))); - if (QTest::currentTestFailed()) return; + // The reference is still there, and Analyse Now is about it. + // With the flag left set it would be routed to a singing track + // that does not exist, and the reference left as it was. (Not + // "the next file opened": opening one closes the session, which + // clears the flag whatever record() did.) + QVERIFY(analysed(m_window->analyser())); + sv::ModelId pitchBefore = + m_window->analyser()->getLayer(Analyser::PitchTrack)->getModel(); + QSignalSpy relayered(m_window->analyser(), SIGNAL(layersChanged())); + + m_window->doAnalyseNow(); + + // The misrouted Analyse Now waits 200 ms for the singing track's + // analyser before it gives up. Wait that out before failing, so + // that it does not fire after this test has ended + if (relayered.isEmpty()) QTest::qWait(300); + + sv::Layer *pitchNow = + m_window->analyser()->getLayer(Analyser::PitchTrack); + QVERIFY2(!relayered.isEmpty() && + pitchNow && pitchNow->getModel() != pitchBefore, + "Analyse Now after a failed record did not re-analyse " + "the reference"); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); QVERIFY(std::fabs(TestSignals::centsBetween (medianHz(pitchEvents(m_window->analyser())), - highHz)) < 10.0); + lowHz)) < 10.0); } void analyse_now_during_take() { @@ -1278,8 +1424,17 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(600); + + // With the take's analyser there already, Stop starts pYIN + // at once rather than 200 ms later, so that it is running when + // the session is closed. Otherwise this test shows nothing + QVERIFY(m_window->analyser2()); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY2(sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), + "the race was not set up: no analysis was running when " + "the session was about to be closed"); m_window->doCloseSession(); QVERIFY(!m_window->analyser2()); diff --git a/main/test/TestSingingDocument.h b/main/test/TestSingingDocument.h index fe3522a3..b24605d9 100644 --- a/main/test/TestSingingDocument.h +++ b/main/test/TestSingingDocument.h @@ -258,13 +258,24 @@ private slots: } void prune_none_id_deletes_nothing() { - // The ruler's own model id is "none". An empty owned id must - // not be taken to match it. + // A layer that was never given a model has the "none" id. An + // empty owned id must not be taken to match it. (Not the ruler: + // createMainModelLayer() gives that the main model's id.) Extra extra = addExtraPane(true); QVERIFY(extra.waveform); + sv::Layer *noModel = + m_document->createLayer(sv::LayerFactory::TimeValues); + QVERIFY(noModel); + QVERIFY2(noModel->getModel().isNone(), + "a layer made without a model does not have the none id"); + m_document->addLayerToView(extra.pane, noModel); + QVERIFY2(paneHasLayer(extra.pane, noModel), + "the layer without a model was not added to the pane"); pruneExtraPane(m_document, m_paneStack, extra.pane, sv::ModelId()); + QVERIFY2(documentHasLayer(noModel), + "the layer without a model was deleted from the document"); QVERIFY2(documentHasLayer(m_ruler), "the shared time ruler was deleted from the document"); QVERIFY(paneHasLayer(m_mainPane, m_ruler)); From b6efc8602af3b252fb6f1f17b8a3a58ea4a7da90 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 20:53:10 +0300 Subject: [PATCH 024/275] test: remove no_synth_tone It was never seen failing, even with the live pitch layer made audible: the play source's read-ahead keeps the output silent in its window either way. The live model's mute is covered by take_muted_while_recording (play parameters) and take_silent_in_output_after_reseek (device output). Co-Authored-By: Claude Fable 5.1 --- main/test/TestRecordWorkflow.h | 44 +++++----------------------------- 1 file changed, 6 insertions(+), 38 deletions(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 84e7f6c6..d6044569 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -788,44 +788,12 @@ private slots: "in it, at amplitude %1").arg(input))); } - // The live pitch model is a SparseTimeValueModel, which would be - // played as a synth tone at the pitch it holds; such a model is - // inaudible unless something switches it on, and nothing does. - // This test holds that at the output, in a later window than the - // test above. The read-ahead covers for it in the same way, though: - // what shows the live model silent when it could be heard is - // take_silent_in_output_after_reseek - void no_synth_tone() { - FakeAudioIO::Config config; - config.input = TestSignals::sine(highHz, rate, int(3 * rate), 0.5); - makeWindow(config); - m_window->setPlayReferenceWhileRecording(true); - openReference(writeWav(TestSignals::sine(lowHz, rate, - int(3 * rate), 0.5))); - if (QTest::currentTestFailed()) return; - - startTake(); - if (QTest::currentTestFailed()) return; - QTest::qWait(1800); - stopTake(); - if (QTest::currentTestFailed()) return; - - auto output = m_window->fake()->getCapturedOutput(); - long start = m_window->fake()->getPlayStartFrame(); - QVERIFY2(start >= 0, "the reference was never played"); - size_t from = size_t(start) + size_t(0.8 * rate); - double synth = amplitudeAt(output, from, 26400, highHz); - QVERIFY2(synth >= 0.0 && synth < 0.005, - qPrintable(QString("the output has a tone at the live " - "pitch, at amplitude %1").arg(synth))); - } - - // Review finding 3. The two tests above find the take and the live - // pitch silent in the output, but for the take that much is true - // even unmuted, because the play source reads ahead of what has - // been recorded. Neither is to be audible while it is being - // recorded, whatever the buffers do: this test checks the play - // parameters, and the next one the output. + // Review finding 3. The test above finds the take silent in the + // output, but that much is true even unmuted, because the play + // source reads ahead of what has been recorded. Neither the take + // nor the live pitch model is to be audible during the take, + // whatever the buffers do: this test checks the play parameters, + // and the next one the output. void take_muted_while_recording() { FakeAudioIO::Config config; config.input = tone(highHz, 4.0); From dd317d3cb126ed49e850a8d32a5b927048a3c209 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 22:09:00 +0300 Subject: [PATCH 025/275] feat: alternate pitch track to follow when recording For a singer whose range is not the reference's: a copy of the reference pitch track moved up or down by whole octaves (-3 to +3, never 0, default -1), shown in brown in pane 0. New "Follow:" section in the Show and Play toolbar: a toggle, and 8vb / 8va buttons. The copy is a model of its own in AlternatePitchTrack (new). It is rebuilt shortly after the reference pitch model changes, and rebound when Analyse Now replaces the reference pitch layer. It has no source model, so no Analyser claims its layer; it is muted, and the editable tracks are raised back above it (Analyser::stackLayers() is public for that). While a singing take is recorded the copy is drawn in full (dark brown, faded otherwise) and the reference pitch track is hidden, with showLayer() rather than Analyser::setVisible() so that the Show Pitch Track setting is not written. The reference is back the moment the take stops, and the three buttons are disabled meanwhile. Saved in the .ton as ordinary document contents: the octave count is in the layer's object name, by which analyseNewMainModel() finds the layer again after a session load. Tests: alternate_pitch_layer_names, alternate_pitch_track, alternate_pitch_follows_reanalysis, alternate_pitch_followed_during_take, alternate_pitch_session_round_trip. Co-Authored-By: Claude Fable 5.1 --- main/AlternatePitchTrack.cpp | 336 +++++++++++++++++++++++++++++++++ main/AlternatePitchTrack.h | 134 +++++++++++++ main/Analyser.h | 7 +- main/MainWindow.cpp | 176 ++++++++++++++++- main/MainWindow.h | 20 ++ main/test/TestRecordWorkflow.h | 251 ++++++++++++++++++++++++ meson.build | 2 + 7 files changed, 922 insertions(+), 4 deletions(-) create mode 100644 main/AlternatePitchTrack.cpp create mode 100644 main/AlternatePitchTrack.h diff --git a/main/AlternatePitchTrack.cpp b/main/AlternatePitchTrack.cpp new file mode 100644 index 00000000..372fe327 --- /dev/null +++ b/main/AlternatePitchTrack.cpp @@ -0,0 +1,336 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AlternatePitchTrack.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "layer/TimeValueLayer.h" +#include "layer/LayerFactory.h" +#include "layer/ColourDatabase.h" +#include "data/model/SparseTimeValueModel.h" +#include "base/PlayParameters.h" + +#include +#include +#include + +using namespace sv; + +using std::cerr; +using std::endl; + +// The layer's object name is this followed by the number of octaves +static const char *layerNamePrefix = "Alternate Pitch Track "; + +// Used until the reference has a pitch model to take them from: the +// step size Analyser::addAnalyses() gives pYIN, and the usual rate +static const int defaultResolution = 256; +static const sv_samplerate_t defaultSampleRate = 44100; + +AlternatePitchTrack::AlternatePitchTrack(QObject *parent) : + QObject(parent), + m_document(nullptr), + m_pane(nullptr), + m_layer(nullptr), + m_octaves(-1), + m_followed(false) +{ + // An edit to the reference, or pYIN filling it in, is a burst of + // changes; copy once when it is over + m_rebuildTimer.setSingleShot(true); + m_rebuildTimer.setInterval(50); + connect(&m_rebuildTimer, &QTimer::timeout, + this, &AlternatePitchTrack::rebuildNow); +} + +AlternatePitchTrack::~AlternatePitchTrack() +{ +} + +double +AlternatePitchTrack::shifted(double hz, int octaves) +{ + return std::ldexp(hz, octaves); +} + +QString +AlternatePitchTrack::layerNameFor(int octaves) +{ + return QString("%1%2").arg(layerNamePrefix).arg(octaves); +} + +bool +AlternatePitchTrack::octavesFromLayerName(QString name, int &octaves) +{ + if (!name.startsWith(layerNamePrefix)) return false; + bool ok = false; + int n = name.mid(int(qstrlen(layerNamePrefix))).toInt(&ok); + if (!ok || n == 0 || n < minOctaves || n > maxOctaves) return false; + octaves = n; + return true; +} + +QString +AlternatePitchTrack::describe(int octaves) +{ + // Not tr("%n octave(s)"): that needs a translation to be loaded + // even for English + int n = std::abs(octaves); + QString distance = (n == 1 ? tr("1 octave") : tr("%1 octaves").arg(n)); + return (octaves > 0 ? tr("%1 above the reference") : + tr("%1 below the reference")).arg(distance); +} + +bool +AlternatePitchTrack::show(Document *document, Pane *pane) +{ + if (m_layer) return true; + if (!document || !pane) return false; + + auto model = std::make_shared + (defaultSampleRate, defaultResolution, false); + model->setObjectName(tr("Alternate Pitch Track")); + model->setScaleUnits("Hz"); + ModelId modelId = ModelById::add(model); + document->addNonDerivedModel(modelId); + + // Not createEmptyLayer(): see MainWindow::setupRealtimePitchLayer() + auto layer = qobject_cast + (document->createLayer(LayerFactory::TimeValues)); + if (!layer) { + cerr << "AlternatePitchTrack::show: failed to create layer" << endl; + ModelById::release(modelId); + return false; + } + + document->setModel(layer, modelId); + takeLayer(document, pane, layer); + document->addLayerToView(pane, layer); + + return true; +} + +bool +AlternatePitchTrack::adopt(Document *document, Pane *pane) +{ + if (m_layer) return true; + if (!document || !pane) return false; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + auto layer = qobject_cast(pane->getLayer(i)); + int octaves = 0; + if (!layer || !octavesFromLayerName(layer->objectName(), octaves)) { + continue; + } + if (!ModelById::isa(layer->getModel())) { + continue; + } + m_octaves = octaves; + takeLayer(document, pane, layer); + return true; + } + + return false; +} + +void +AlternatePitchTrack::takeLayer(Document *document, Pane *pane, + TimeValueLayer *layer) +{ + m_document = document; + m_pane = pane; + m_layer = layer; + + connect(m_document, &Document::layerAboutToBeDeleted, + this, &AlternatePitchTrack::layerAboutToBeDeleted, + Qt::UniqueConnection); + + configureLayer(); +} + +void +AlternatePitchTrack::configureLayer() +{ + if (!m_layer) return; + + m_layer->setObjectName(layerNameFor(m_octaves)); + m_layer->setPresentationName(tr("Alternate Pitch Track")); + m_layer->setVerticalScale(TimeValueLayer::AutoAlignScale); + m_layer->setPlotStyle(TimeValueLayer::PlotPoints); + + // There are two pitch tracks to be heard already + if (auto params = m_layer->getPlayParameters()) { + params->setPlayAudible(false); + } + + applyColour(); +} + +void +AlternatePitchTrack::applyColour() +{ + if (!m_layer) return; + + // Close to the black of the reference, but not to be taken for + // it; and the same washed out, for when nobody is following it + ColourDatabase *cdb = ColourDatabase::getInstance(); + int index = m_followed ? + cdb->addColour(QColor(74, 37, 17), tr("Dark Brown")) : + cdb->addColour(QColor(164, 146, 136), tr("Faded Brown")); + m_layer->setBaseColour(index); +} + +void +AlternatePitchTrack::hide() +{ + m_rebuildTimer.stop(); + + if (!m_source.isNone()) { + if (auto source = ModelById::get(m_source)) { + disconnect(source.get(), nullptr, this, nullptr); + } + m_source = {}; + } + + TimeValueLayer *layer = m_layer; + m_layer = nullptr; + + // deleteLayer(force) and nothing else: see the notes on tearing a + // layer down silently in MainWindow::teardownRealtimePitchLayer(). + // The model goes with the layer, which is its only user + if (layer && m_document) { + m_document->deleteLayer(layer, true); + } + + if (m_document) { + disconnect(m_document, nullptr, this, nullptr); + } + m_document = nullptr; + m_pane = nullptr; +} + +void +AlternatePitchTrack::layerAboutToBeDeleted(Layer *layer) +{ + // Someone else's doing, the document being closed for instance + if (layer && layer == m_layer) { + m_layer = nullptr; + hide(); + } +} + +void +AlternatePitchTrack::setSource(ModelId referencePitchModel) +{ + if (!ModelById::isa(referencePitchModel)) { + referencePitchModel = {}; + } + if (referencePitchModel == m_source) return; + + if (auto previous = ModelById::get(m_source)) { + disconnect(previous.get(), nullptr, this, nullptr); + } + + m_source = referencePitchModel; + + // pYIN writes to the model from its own thread: these are queued + if (auto source = ModelById::get(m_source)) { + connect(source.get(), &Model::modelChanged, + this, &AlternatePitchTrack::sourceChanged); + connect(source.get(), &Model::modelChangedWithin, + this, &AlternatePitchTrack::sourceChanged); + } + + rebuildNow(); +} + +void +AlternatePitchTrack::sourceChanged() +{ + if (!m_rebuildTimer.isActive()) m_rebuildTimer.start(); +} + +void +AlternatePitchTrack::setOctaves(int octaves) +{ + if (octaves == 0 || octaves < minOctaves || octaves > maxOctaves) return; + if (octaves == m_octaves) return; + m_octaves = octaves; + if (m_layer) { + m_layer->setObjectName(layerNameFor(m_octaves)); + rebuildNow(); + } +} + +bool +AlternatePitchTrack::canStep(bool up) const +{ + return up ? (m_octaves < maxOctaves) : (m_octaves > minOctaves); +} + +void +AlternatePitchTrack::step(bool up) +{ + if (!canStep(up)) return; + int octaves = m_octaves + (up ? 1 : -1); + if (octaves == 0) octaves += (up ? 1 : -1); + setOctaves(octaves); +} + +void +AlternatePitchTrack::setFollowed(bool followed) +{ + if (m_followed == followed) return; + m_followed = followed; + applyColour(); +} + +void +AlternatePitchTrack::rebuildNow() +{ + m_rebuildTimer.stop(); + + if (!m_layer || !m_document) return; + + auto source = ModelById::getAs(m_source); + auto model = ModelById::getAs(m_layer->getModel()); + + // Points are drawn as wide as the resolution says, so ours must be + // that of the source. It was a guess if the layer was made before + // the reference had a pitch track + if (source && (!model || + model->getSampleRate() != source->getSampleRate() || + model->getResolution() != source->getResolution())) { + model = std::make_shared + (source->getSampleRate(), source->getResolution(), false); + model->setObjectName(tr("Alternate Pitch Track")); + model->setScaleUnits("Hz"); + ModelId modelId = ModelById::add(model); + m_document->addNonDerivedModel(modelId); + // Releases the model it replaces, and copies the play + // parameters (that is, the muting) across + m_document->setModel(m_layer, modelId); + } + + if (!model) return; + + for (const Event &e : model->getAllEvents()) model->remove(e); + + if (!source) return; + + for (const Event &e : source->getAllEvents()) { + model->add(e.withValue(float(shifted(e.getValue(), m_octaves)))); + } +} diff --git a/main/AlternatePitchTrack.h b/main/AlternatePitchTrack.h new file mode 100644 index 00000000..2b99b9bc --- /dev/null +++ b/main/AlternatePitchTrack.h @@ -0,0 +1,134 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_ALTERNATE_PITCH_TRACK_H +#define TONY_ALTERNATE_PITCH_TRACK_H + +#include +#include +#include + +#include "base/ById.h" +#include "data/model/Model.h" + +namespace sv { +class Document; +class Pane; +class Layer; +class TimeValueLayer; +} + +/** + * The alternate pitch track: a copy of the reference pitch track moved + * up or down by a whole number of octaves, for a singer whose range is + * not that of the reference. It is drawn in dark brown, faded unless a + * take is being recorded ("followed"), when it stands in for the + * reference pitch track, which MainWindow hides for the duration. + * + * The copy is a model of its own, rebuilt whenever the reference pitch + * model changes. It is never edited, never played, and has no source + * model, so no Analyser claims its layer. + * + * The layer and its model belong to the document and are saved in the + * session with it. The number of octaves is kept in the layer's object + * name, which is also what adopt() knows the layer by after a session + * has been loaded. + */ +class AlternatePitchTrack : public QObject +{ + Q_OBJECT + +public: + static constexpr int minOctaves = -3; + static constexpr int maxOctaves = 3; + + AlternatePitchTrack(QObject *parent = nullptr); + virtual ~AlternatePitchTrack(); + + /** + * Create the layer in the given pane, on top of whatever is there. + * Does nothing if the layer exists already. Returns false if it + * could not be created. + */ + bool show(sv::Document *document, sv::Pane *pane); + + /** + * Take over a layer that show() made in an earlier run, and that a + * session load has put back into the pane, along with the number + * of octaves it was saved with. Returns false if there is none. + */ + bool adopt(sv::Document *document, sv::Pane *pane); + + /** + * Delete the layer and its model from the document. The number + * of octaves is remembered. + */ + void hide(); + + bool isShown() const { return m_layer != nullptr; } + + /** + * The model to copy: that of the reference pitch track, or none. + */ + void setSource(sv::ModelId referencePitchModel); + sv::ModelId getSource() const { return m_source; } + + /** + * Distance from the reference in octaves, within minOctaves to + * maxOctaves. Never zero: that is the reference itself, and + * stepping across it goes from -1 to 1. + */ + int getOctaves() const { return m_octaves; } + void setOctaves(int octaves); + bool canStep(bool up) const; + void step(bool up); + + /** + * Drawn in full rather than faded. For the duration of a take. + */ + void setFollowed(bool followed); + bool isFollowed() const { return m_followed; } + + sv::TimeValueLayer *getLayer() const { return m_layer; } + + /** + * Copy the source again now, rather than when the event loop next + * comes round. + */ + void rebuildNow(); + + static double shifted(double hz, int octaves); + static QString layerNameFor(int octaves); + static bool octavesFromLayerName(QString name, int &octaves); + static QString describe(int octaves); + +private slots: + void sourceChanged(); + void layerAboutToBeDeleted(sv::Layer *); + +private: + sv::Document *m_document; + sv::Pane *m_pane; + sv::TimeValueLayer *m_layer; + sv::ModelId m_source; + int m_octaves; + bool m_followed; + QTimer m_rebuildTimer; + + void takeLayer(sv::Document *, sv::Pane *, sv::TimeValueLayer *); + void configureLayer(); + void applyColour(); +}; + +#endif diff --git a/main/Analyser.h b/main/Analyser.h index b1f6df47..e481ebc3 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -261,6 +261,11 @@ class Analyser : public QObject, return m_layers[type]; } + // Raise the pitch track, then the notes, to the top of the pane, + // where the editing tools find them. For when something else has + // been added to the pane on top of them + void stackLayers(); + signals: void layersChanged(); void initialAnalysisCompleted(); @@ -297,8 +302,6 @@ protected slots: QString addAnalyses(); void discardPitchCandidates(); - - void stackLayers(); // Document::LayerCreationHandler method void layersCreated(sv::Document::LayerCreationAsyncHandle, diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index aa6629e2..491e1c13 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -125,6 +125,11 @@ MainWindow::MainWindow(AudioMode audioMode, m_playSingingAudio(nullptr), m_playRefWhileRecording(nullptr), m_loadSingingTrackAction(nullptr), + m_alternatePitch(nullptr), + m_showAlternatePitch(nullptr), + m_alternatePitchUpAction(nullptr), + m_alternatePitchDownAction(nullptr), + m_referencePitchHiddenForTake(false), m_backgroundMusicModelId(), m_backgroundMusicLayer(nullptr), m_loadBackgroundMusicAction(nullptr), @@ -340,6 +345,10 @@ MainWindow::MainWindow(AudioMode audioMode, connect(m_analyser, SIGNAL(layersChanged()), this, SLOT(updateMenuStates())); + m_alternatePitch = new AlternatePitchTrack(this); + connect(m_analyser, SIGNAL(layersChanged()), + this, SLOT(syncAlternatePitchTrack())); + setupMenus(); setupToolbars(); setupHelpMenu(); @@ -410,6 +419,10 @@ MainWindow::~MainWindow() delete m_analyser2; m_analyser2 = nullptr; } + // Before the base class deletes the document: it only watches the + // document's layers, it does not own them + delete m_alternatePitch; + m_alternatePitch = nullptr; delete m_analyser; delete m_keyReference; Profiles::getInstance()->dump(); @@ -1483,6 +1496,44 @@ MainWindow::setupToolbars() }); connect(this, SIGNAL(canPlay(bool)), m_playRefWhileRecording, SLOT(setEnabled(bool))); + // The alternate pitch track: the reference pitch an octave or more + // up or down, for the singer to follow in place of the reference. + spacer = new QLabel; + spacer->setFixedWidth(m_viewManager->scalePixelSize(30)); + toolbar->addWidget(spacer); + + { + QLabel *followLabel = new QLabel(tr("Follow:")); + QFont f = followLabel->font(); + f.setPointSize(f.pointSize() - 1); + followLabel->setFont(f); + followLabel->setEnabled(false); + toolbar->addWidget(followLabel); + } + + m_showAlternatePitch = toolbar->addAction(il.load("values"), + tr("Alternate Pitch Track")); + m_showAlternatePitch->setCheckable(true); + connect(m_showAlternatePitch, SIGNAL(triggered()), + this, SLOT(alternatePitchToggled())); + + // No icons for these; the toolbar shows the text + m_alternatePitchDownAction = toolbar->addAction(tr("8vb")); + m_alternatePitchDownAction->setToolTip + (tr("Move the alternate pitch track down an octave")); + m_alternatePitchDownAction->setStatusTip + (tr("Move the alternate pitch track down an octave")); + connect(m_alternatePitchDownAction, SIGNAL(triggered()), + this, SLOT(alternatePitchDown())); + + m_alternatePitchUpAction = toolbar->addAction(tr("8va")); + m_alternatePitchUpAction->setToolTip + (tr("Move the alternate pitch track up an octave")); + m_alternatePitchUpAction->setStatusTip + (tr("Move the alternate pitch track up an octave")); + connect(m_alternatePitchUpAction, SIGNAL(triggered()), + this, SLOT(alternatePitchUp())); + // Background music section: an additional audio track that plays // alongside the reference track but is never analysed. spacer = new QLabel; @@ -1905,6 +1956,25 @@ MainWindow::updateLayerStatuses() } } + // Alternate pitch track: there to be had once there is a reference. + // No moving it during a take, when the singer is following it + if (m_showAlternatePitch) { + bool haveReference = (m_document && getMainModel() && + m_paneStack && m_paneStack->getPaneCount() > 0); + bool shown = m_alternatePitch->isShown(); + bool inTake = (m_recordTarget && m_recordTarget->isRecording()); + m_showAlternatePitch->setEnabled(haveReference && !inTake); + m_showAlternatePitch->setChecked(shown); + QString tip = tr("Show a copy of the reference pitch track %1 (brown), and follow that when recording") + .arg(AlternatePitchTrack::describe(m_alternatePitch->getOctaves())); + m_showAlternatePitch->setToolTip(tip); + m_showAlternatePitch->setStatusTip(tip); + m_alternatePitchUpAction->setEnabled + (shown && !inTake && m_alternatePitch->canStep(true)); + m_alternatePitchDownAction->setEnabled + (shown && !inTake && m_alternatePitch->canStep(false)); + } + // Background music toggle: enabled when a background music track is loaded if (m_playBackgroundMusic) { bool haveBgMusic = (m_backgroundMusicLayer != nullptr); @@ -2023,6 +2093,8 @@ MainWindow::closeSession() teardownRealtimePitchLayer(); teardownSingingTrackAnalyser(); teardownBackgroundMusic(); + m_alternatePitch->hide(); + m_referencePitchHiddenForTake = false; m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; @@ -2341,6 +2413,92 @@ MainWindow::backgroundMusicPanChanged(float pan) if (params) params->setPlayPan(pan); } +void +MainWindow::alternatePitchToggled() +{ + if (!m_document || !m_paneStack || m_paneStack->getPaneCount() < 1) { + updateLayerStatuses(); + return; + } + + if (m_alternatePitch->isShown()) { + m_alternatePitch->hide(); + // The layer went without a command, as it must (see + // teardownRealtimePitchLayer()), but the session has changed + documentModified(); + } else if (m_alternatePitch->show(m_document, m_paneStack->getPane(0))) { + // The new layer is on top, and is the one the editing tools + // would work on: put the tracks that can be edited back there, + // in the order they were + m_analyser->stackLayers(); + if (m_analyser2) m_analyser2->stackLayers(); + syncAlternatePitchTrack(); + } + + updateLayerStatuses(); +} + +void +MainWindow::alternatePitchUp() +{ + stepAlternatePitch(true); +} + +void +MainWindow::alternatePitchDown() +{ + stepAlternatePitch(false); +} + +void +MainWindow::stepAlternatePitch(bool up) +{ + if (!m_alternatePitch->isShown() || !m_alternatePitch->canStep(up)) return; + m_alternatePitch->step(up); + documentModified(); + updateLayerStatuses(); + getStatusLabel()->setText + (tr("Alternate pitch track: %1") + .arg(AlternatePitchTrack::describe(m_alternatePitch->getOctaves()))); +} + +void +MainWindow::syncAlternatePitchTrack() +{ + // The reference pitch layer, and its model with it, is replaced + // whenever the reference is analysed again + if (!m_alternatePitch->isShown()) return; + Layer *reference = m_analyser->getLayer(Analyser::PitchTrack); + m_alternatePitch->setSource(reference ? reference->getModel() : ModelId()); + updateAlternatePitchForTake(); +} + +void +MainWindow::updateAlternatePitchForTake() +{ + // During a singing take the alternate track is the one to sing to: + // it is shown in full, and the reference pitch track makes way + bool following = (m_alternatePitch->isShown() && + m_recordingAsSingingTrack && + m_recordTarget && m_recordTarget->isRecording()); + + m_alternatePitch->setFollowed(following); + + Layer *reference = m_analyser->getLayer(Analyser::PitchTrack); + Pane *pane = m_analyser->getPane(); + + if (following) { + if (!m_referencePitchHiddenForTake && reference && pane && + !reference->isLayerDormant(pane)) { + reference->showLayer(pane, false); + m_referencePitchHiddenForTake = true; + } + } else if (m_referencePitchHiddenForTake) { + m_referencePitchHiddenForTake = false; + if (reference && pane) reference->showLayer(pane, true); + } +} + void MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { @@ -2858,6 +3016,9 @@ MainWindow::record() m_recordingInProgress = false; } + updateAlternatePitchForTake(); + updateLayerStatuses(); + // Restore the default mode so that a subsequent "standalone" recording // (after the singing track session is closed) behaves correctly. setAudioRecordMode(RecordReplaceSession); @@ -2920,8 +3081,11 @@ MainWindow::recordingStarted() // (true) and stops (false). We only want to act when it starts. if (!m_recordTarget) return; if (!m_recordTarget->isRecording()) { - // Recording stopped - nothing to do here; recordingFinishedFull() - // is called from analyseNow() once pYIN completes. + // Recording stopped - recordingFinishedFull() is called from + // analyseNow() once pYIN completes. The reference pitch track + // comes back now, though: the singer has stopped following. + updateAlternatePitchForTake(); + updateLayerStatuses(); return; } @@ -4605,6 +4769,14 @@ MainWindow::analyseNewMainModel() } } + // A session saved with the alternate pitch track has its layer in + // the pane already + if (pane && m_alternatePitch->adopt(m_document, pane)) { + cerr << "analyseNewMainModel: found the alternate pitch track of the session, " + << m_alternatePitch->getOctaves() << " octave(s)" << endl; + syncAlternatePitchTrack(); + } + if (!m_withSpectrogram) { m_analyser->setVisible(Analyser::Spectrogram, false); } diff --git a/main/MainWindow.h b/main/MainWindow.h index df967e57..624deec6 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -19,6 +19,7 @@ #include "framework/MainWindowBase.h" #include "Analyser.h" #include "RealtimePitchTracker.h" +#include "AlternatePitchTrack.h" #include #include @@ -112,6 +113,11 @@ protected slots: virtual void backgroundMusicGainChanged(float gain); virtual void backgroundMusicPanChanged(float pan); + virtual void alternatePitchToggled(); + virtual void alternatePitchUp(); + virtual void alternatePitchDown(); + virtual void syncAlternatePitchTrack(); + virtual void editDisplayExtents(); virtual void analyseNow(); @@ -240,6 +246,20 @@ protected slots: QAction *m_playRefWhileRecording; QAction *m_loadSingingTrackAction; + // The alternate pitch track: the reference pitch track moved by whole + // octaves, for the singer to follow in place of the reference. While + // a singing take is being recorded it is shown in full and the + // reference pitch track is hidden; m_referencePitchHiddenForTake says + // that we hid it, and must show it again afterwards. Not hidden with + // Analyser::setVisible(), which would write the state to the settings. + AlternatePitchTrack *m_alternatePitch; + QAction *m_showAlternatePitch; + QAction *m_alternatePitchUpAction; + QAction *m_alternatePitchDownAction; + bool m_referencePitchHiddenForTake; + void stepAlternatePitch(bool up); + void updateAlternatePitchForTake(); + // Background music track: an additional audio file that plays alongside // the reference track but is never analysed. The toggle enables/disables // mixing during both normal playback and recording. diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d6044569..1bd91ae7 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -112,6 +112,15 @@ class TestMainWindow : public MainWindow sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } + AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } + void doToggleAlternatePitch() { alternatePitchToggled(); } + void doStepAlternatePitch(bool up) { + if (up) alternatePitchUp(); else alternatePitchDown(); + } + QAction *alternatePitchAction() { return m_showAlternatePitch; } + QAction *alternatePitchUpAction() { return m_alternatePitchUpAction; } + QAction *alternatePitchDownAction() { return m_alternatePitchDownAction; } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { onRealtimePitchDetected(frame, hz); } @@ -348,6 +357,32 @@ class TestRecordWorkflow : public QObject "the shared time ruler is no longer in the ruler pane"); } + int alternateLayersInDocument() { + int n = 0, octaves = 0; + for (sv::Layer *layer : m_window->document()->getLayers()) { + if (AlternatePitchTrack::octavesFromLayerName + (layer->objectName(), octaves)) ++n; + } + return n; + } + + // True once the alternate track is the reference's, note for note + bool alternateMatchesReference() { + auto alt = pitchEvents(m_window->alternatePitch()->getLayer()); + auto ref = pitchEvents(m_window->analyser()); + int octaves = m_window->alternatePitch()->getOctaves(); + if (ref.empty() || alt.size() != ref.size()) return false; + for (size_t i = 0; i < ref.size(); ++i) { + if (alt[i].getFrame() != ref[i].getFrame()) return false; + double want = AlternatePitchTrack::shifted + (ref[i].getValue(), octaves); + if (std::fabs(alt[i].getValue() - want) > 1e-3 * want) { + return false; + } + } + return true; + } + // Not a slot: QtTest would run it as a test void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -1375,6 +1410,222 @@ private slots: highHz)) < 10.0); } + // The alternate pitch track: the reference pitch track moved by + // whole octaves, as a layer of its own + + void alternate_pitch_layer_names() { + int octaves = 99; + QVERIFY(AlternatePitchTrack::octavesFromLayerName + (AlternatePitchTrack::layerNameFor(-2), octaves)); + QCOMPARE(octaves, -2); + QVERIFY(AlternatePitchTrack::octavesFromLayerName + (AlternatePitchTrack::layerNameFor(3), octaves)); + QCOMPARE(octaves, 3); + // zero is the reference itself, and the rest are not ours + for (QString name : { AlternatePitchTrack::layerNameFor(0), + AlternatePitchTrack::layerNameFor(4), + QString("Alternate Pitch Track"), + QString("Alternate Pitch Track x"), + QString("Pitch Track -1") }) { + QVERIFY2(!AlternatePitchTrack::octavesFromLayerName(name, octaves), + qPrintable(name)); + } + QCOMPARE(octaves, 3); + QCOMPARE(AlternatePitchTrack::shifted(220.0, -1), 110.0); + QCOMPARE(AlternatePitchTrack::shifted(220.0, 2), 880.0); + } + + void alternate_pitch_track() { + makeWindow(FakeAudioIO::Config()); + QVERIFY(!m_window->alternatePitchAction()->isEnabled()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + AlternatePitchTrack *alt = m_window->alternatePitch(); + QVERIFY(m_window->alternatePitchAction()->isEnabled()); + QVERIFY(!m_window->alternatePitchAction()->isChecked()); + QVERIFY(!m_window->alternatePitchUpAction()->isEnabled()); + QVERIFY(!alt->isShown()); + m_window->discardModifications(); + + m_window->doToggleAlternatePitch(); + QVERIFY(alt->isShown()); + QVERIFY(m_window->alternatePitchAction()->isChecked()); + QVERIFY(m_window->alternatePitchUpAction()->isEnabled()); + QVERIFY(m_window->isDocumentModified()); + QCOMPARE(alt->getOctaves(), -1); + QVERIFY(paneHasLayer(0, alt->getLayer())); + QCOMPARE(colourOf(alt->getLayer()), colourNamed("Faded Brown")); + QVERIFY(alternateMatchesReference()); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(alt->getLayer())), + lowHz / 2)) < 10.0); + + // never heard, and never the layer that gets edited + QVERIFY(!alt->getLayer()->getPlayParameters()->isPlayAudible()); + QVERIFY(m_window->paneStack()->getPane(0)->getSelectedLayer() + != alt->getLayer()); + + // up from one below is one above: none is the reference itself + m_window->discardModifications(); + m_window->doStepAlternatePitch(true); + QCOMPARE(alt->getOctaves(), 1); + QVERIFY(m_window->isDocumentModified()); + QVERIFY(alternateMatchesReference()); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(alt->getLayer())), + lowHz * 2)) < 10.0); + + m_window->doStepAlternatePitch(true); + m_window->doStepAlternatePitch(true); + m_window->doStepAlternatePitch(true); + QCOMPARE(alt->getOctaves(), int(AlternatePitchTrack::maxOctaves)); + QVERIFY(!m_window->alternatePitchUpAction()->isEnabled()); + QVERIFY(m_window->alternatePitchDownAction()->isEnabled()); + + // the reference was left alone + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + + sv::Layer *layer = alt->getLayer(); + m_window->doToggleAlternatePitch(); + QVERIFY(!alt->isShown()); + QVERIFY(!documentHasLayer(layer)); + QCOMPARE(alternateLayersInDocument(), 0); + verifyPlaySourceClean(); + + // and it comes back where it was + m_window->doToggleAlternatePitch(); + QCOMPARE(alt->getOctaves(), int(AlternatePitchTrack::maxOctaves)); + QVERIFY(alternateMatchesReference()); + } + + // Analyse Now gives the reference a new pitch layer and model + void alternate_pitch_follows_reanalysis() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + AlternatePitchTrack *alt = m_window->alternatePitch(); + m_window->doToggleAlternatePitch(); + sv::ModelId before = alt->getSource(); + QVERIFY(!before.isNone()); + + m_window->doAnalyseNow(); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + + QVERIFY(alt->getSource() != before); + QCOMPARE(alt->getSource(), + m_window->analyser()->getLayer(Analyser::PitchTrack) + ->getModel()); + QTRY_VERIFY_WITH_TIMEOUT(alternateMatchesReference(), 5000); + + // and an edit to the reference reaches it + auto ref = pitchEvents(m_window->analyser()); + m_window->analyser()->shiftOctave + (sv::Selection(ref.front().getFrame(), + ref.back().getFrame() + 1), true); + QTRY_VERIFY_WITH_TIMEOUT + (std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(alt->getLayer())), lowHz)) < 10.0, + 5000); + QVERIFY(alternateMatchesReference()); + } + + void alternate_pitch_followed_during_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + AlternatePitchTrack *alt = m_window->alternatePitch(); + m_window->doToggleAlternatePitch(); + sv::Pane *pane = m_window->paneStack()->getPane(0); + sv::Layer *reference = + m_window->analyser()->getLayer(Analyser::PitchTrack); + QVERIFY(!reference->isLayerDormant(pane)); + + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(colourOf(alt->getLayer()), colourNamed("Dark Brown")); + QVERIFY(!alt->getLayer()->isLayerDormant(pane)); + QVERIFY(reference->isLayerDormant(pane)); + QVERIFY(!m_window->alternatePitchAction()->isEnabled()); + QVERIFY(!m_window->alternatePitchDownAction()->isEnabled()); + QTest::qWait(600); + QVERIFY(reference->isLayerDormant(pane)); + + stopTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(colourOf(alt->getLayer()), colourNamed("Faded Brown")); + QVERIFY(!reference->isLayerDormant(pane)); + QVERIFY(m_window->alternatePitchAction()->isEnabled()); + QVERIFY(alternateMatchesReference()); + + // The setting that Show Pitch Track keeps was not touched + QVERIFY(m_window->analyser()->isVisible(Analyser::PitchTrack)); + QSettings settings; + settings.beginGroup("Analyser"); + QVERIFY(settings.value + (QString("visible-%1").arg(int(Analyser::PitchTrack)), + true).toBool()); + settings.endGroup(); + + // A take with no alternate track hides nothing + m_window->doToggleAlternatePitch(); + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!reference->isLayerDormant(pane)); + stopTake(); + } + + void alternate_pitch_session_round_trip() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + AlternatePitchTrack *alt = m_window->alternatePitch(); + m_window->doToggleAlternatePitch(); + m_window->doStepAlternatePitch(false); + QCOMPARE(alt->getOctaves(), -2); + + QString session = m_dir.filePath("alternate.ton"); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + QVERIFY(!alt->isShown()); + + // A session without one must not be given the last one's + openReference(writeWav(tone(highHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(!alt->isShown()); + QCOMPARE(alternateLayersInDocument(), 0); + + openReference(session); + if (QTest::currentTestFailed()) return; + QVERIFY(alt->isShown()); + QCOMPARE(alt->getOctaves(), -2); + QVERIFY(m_window->alternatePitchAction()->isChecked()); + QCOMPARE(alternateLayersInDocument(), 1); + QVERIFY(paneHasLayer(0, alt->getLayer())); + QCOMPARE(colourOf(alt->getLayer()), colourNamed("Faded Brown")); + QVERIFY(!alt->getLayer()->getPlayParameters()->isPlayAudible()); + QTRY_VERIFY_WITH_TIMEOUT(alternateMatchesReference(), 5000); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(alt->getLayer())), + lowHz / 4)) < 10.0); + + // The reference is still the black one, and still the one edited + QCOMPARE(colourOf(m_window->analyser()->getLayer(Analyser::PitchTrack)), + colourNamed("Black")); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + QVERIFY(m_window->paneStack()->getPane(0)->getSelectedLayer() + != alt->getLayer()); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform diff --git a/meson.build b/meson.build index 487b10db..64d0c19e 100644 --- a/meson.build +++ b/meson.build @@ -1098,6 +1098,7 @@ tony_core_files = [ ] tony_app_files = [ + 'main/AlternatePitchTrack.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', @@ -1113,6 +1114,7 @@ tony_app_moc_files = qt.preprocess( moc_headers: [ 'main/MainWindow.h', 'main/Analyser.h', + 'main/AlternatePitchTrack.h', ]) qt_resource_files = qt.preprocess( From 6183cac891b4c89ce7197992e1031e3be746f894 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 22:30:27 +0300 Subject: [PATCH 026/275] fix: connect Analyser::layerAboutToBeDeleted so that it is called The string-based connect named the slot's argument "Layer *", while moc records "sv::Layer *" for a class outside namespace sv, so the connect failed with only a warning and the slot never ran. Connect by member pointer instead, uniquely, as newFileLoaded() is called for every take. The slot now also nulls any m_layers entry for the deleted layer, and resets the current candidate index when a candidate goes. A layer deleted behind the analyser's back used to leave a stale pointer that crashed on teardown; the new test segfaulted on the old code. Co-Authored-By: Claude Fable 5.1 --- main/Analyser.cpp | 25 ++++++++++++++++----- main/test/TestRecordWorkflow.h | 40 ++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 5 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 8d85721e..e12f8537 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -97,8 +97,13 @@ Analyser::newFileLoaded(Document *doc, ModelId model, return "Internal error: Analyser::newFileLoaded() called with no model, or a non-WaveFileModel"; } - connect(doc, SIGNAL(layerAboutToBeDeleted(Layer *)), - this, SLOT(layerAboutToBeDeleted(Layer *))); + // By member pointer: this class is outside namespace sv, so moc + // records the slot as taking sv::Layer *, which a SLOT() string + // saying Layer * never matches. Unique, because this is called + // again with the same document for every take + connect(doc, &Document::layerAboutToBeDeleted, + this, &Analyser::layerAboutToBeDeleted, + Qt::UniqueConnection); QSettings settings; settings.beginGroup("Analyser"); @@ -1039,8 +1044,8 @@ Analyser::discardPitchCandidates() void Analyser::layerAboutToBeDeleted(Layer *doomed) { - cerr << "Analyser::layerAboutToBeDeleted(" << doomed << ")" << endl; - + // Called for every layer the document deletes, ours or not + vector notDoomed; foreach (Layer *layer, m_reAnalysisCandidates) { @@ -1049,7 +1054,17 @@ Analyser::layerAboutToBeDeleted(Layer *doomed) } } - m_reAnalysisCandidates = notDoomed; + if (notDoomed.size() != m_reAnalysisCandidates.size()) { + m_reAnalysisCandidates = notDoomed; + // The index no longer means the candidate it did + m_currentCandidate = -1; + } + + // A layer of ours deleted by someone else, e.g. by a command + // dropped from the undo history + for (auto &entry : m_layers) { + if (entry.second == doomed) entry.second = nullptr; + } } void diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 1bd91ae7..844a4deb 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1265,6 +1265,46 @@ private slots: "the reference analyser forgot its pitch candidates"); } + // Bug 1: the analyser was never told of a layer deleted by someone + // else, and went on listing deleted pitch candidates + void candidates_deleted_from_outside() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + Analyser *a = m_window->analyser(); + + QString error = a->reAnalyseSelection + (sv::Selection(sv::sv_frame_t(0.5 * rate), + sv::sv_frame_t(1.5 * rate)), + Analyser::FrequencyRange()); + QVERIFY2(error.isEmpty(), qPrintable(error)); + QTRY_VERIFY_WITH_TIMEOUT(a->haveHigherPitchCandidate(), 30000); + + std::vector candidates; + sv::Pane *pane = a->getPane(); + for (int i = 0; i < pane->getLayerCount(); ++i) { + sv::Layer *layer = pane->getLayer(i); + if (layer->getLayerPresentationName() == "candidate") { + candidates.push_back(layer); + } + } + QVERIFY(!candidates.empty()); + + for (sv::Layer *layer : candidates) { + m_window->document()->deleteLayer(layer, true); + } + QVERIFY2(!a->haveHigherPitchCandidate() && + !a->haveLowerPitchCandidate(), + "the analyser still lists deleted pitch candidates"); + + // The same for one of its own layers + sv::Layer *notes = a->getLayer(Analyser::Notes); + QVERIFY(notes); + m_window->document()->deleteLayer(notes, true); + QVERIFY2(!a->getLayer(Analyser::Notes), + "the analyser still points at its deleted note layer"); + } + void load_background_music() { makeWindow(FakeAudioIO::Config()); openReference(writeWav(tone(lowHz, 1.0))); From 3f6ab7977b30fb677cdb9f8e6f38e89a6ca45f0b Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 19 Sep 2026 23:49:50 +0300 Subject: [PATCH 027/275] feat: Coverage and TakeAudio, the file-level parts of partial recordings Coverage is a sorted list of frame ranges that hold recorded material. TakeAudio::splice() writes a recording into a take's combined file at a position, with crossfaded edges; TakeAudio::erase() silences ranges. Nothing in the application uses them yet. Co-Authored-By: Claude Opus 5 (1M context) --- main/Coverage.cpp | 103 ++++++++ main/Coverage.h | 71 ++++++ main/TakeAudio.cpp | 268 +++++++++++++++++++++ main/TakeAudio.h | 83 +++++++ main/test/TestCoverage.h | 174 ++++++++++++++ main/test/TestTakeAudio.h | 453 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 14 ++ meson.build | 4 + 8 files changed, 1170 insertions(+) create mode 100644 main/Coverage.cpp create mode 100644 main/Coverage.h create mode 100644 main/TakeAudio.cpp create mode 100644 main/TakeAudio.h create mode 100644 main/test/TestCoverage.h create mode 100644 main/test/TestTakeAudio.h diff --git a/main/Coverage.cpp b/main/Coverage.cpp new file mode 100644 index 00000000..33a11c0f --- /dev/null +++ b/main/Coverage.cpp @@ -0,0 +1,103 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "Coverage.h" + +#include + +using namespace sv; + +void +Coverage::add(sv_frame_t start, sv_frame_t end) +{ + if (end <= start) return; + + Ranges result; + bool placed = false; + + for (const Range &r : m_ranges) { + if (r.end < start) { + result.push_back(r); + } else if (r.start > end) { + if (!placed) { + result.push_back(Range(start, end)); + placed = true; + } + result.push_back(r); + } else { + // overlaps or touches: becomes part of the new one + start = std::min(start, r.start); + end = std::max(end, r.end); + } + } + + if (!placed) result.push_back(Range(start, end)); + + m_ranges = result; +} + +void +Coverage::remove(sv_frame_t start, sv_frame_t end) +{ + if (end <= start) return; + + Ranges result; + + for (const Range &r : m_ranges) { + if (r.end <= start || r.start >= end) { + result.push_back(r); + continue; + } + if (r.start < start) result.push_back(Range(r.start, start)); + if (r.end > end) result.push_back(Range(end, r.end)); + } + + m_ranges = result; +} + +bool +Coverage::contains(sv_frame_t frame) const +{ + Range r; + return getRangeAt(frame, r); +} + +bool +Coverage::getRangeAt(sv_frame_t frame, Range &range) const +{ + for (const Range &r : m_ranges) { + if (frame >= r.start && frame < r.end) { + range = r; + return true; + } + } + return false; +} + +bool +Coverage::overlaps(sv_frame_t start, sv_frame_t end) const +{ + if (end <= start) return false; + for (const Range &r : m_ranges) { + if (r.start < end && r.end > start) return true; + } + return false; +} + +sv_frame_t +Coverage::getEndFrame() const +{ + if (m_ranges.empty()) return 0; + return m_ranges.back().end; +} diff --git a/main/Coverage.h b/main/Coverage.h new file mode 100644 index 00000000..3c790f12 --- /dev/null +++ b/main/Coverage.h @@ -0,0 +1,71 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_COVERAGE_H +#define TONY_COVERAGE_H + +#include "base/BaseTypes.h" + +#include + +/** + * The parts of a singing take that hold recorded material, as frame + * ranges [start, end) on the reference's timeline. The rest of the + * take's audio file is silence. Ranges are kept sorted and apart: + * two that overlap or touch are one range. + */ +class Coverage +{ +public: + struct Range { + sv::sv_frame_t start; + sv::sv_frame_t end; + + Range() : start(0), end(0) { } + Range(sv::sv_frame_t s, sv::sv_frame_t e) : start(s), end(e) { } + + sv::sv_frame_t length() const { return end - start; } + bool operator==(const Range &r) const { + return start == r.start && end == r.end; + } + }; + + typedef std::vector Ranges; + + /// Empty ranges (end <= start) are ignored by add() and remove() + void add(sv::sv_frame_t start, sv::sv_frame_t end); + void remove(sv::sv_frame_t start, sv::sv_frame_t end); + void clear() { m_ranges.clear(); } + + bool isEmpty() const { return m_ranges.empty(); } + const Ranges &getRanges() const { return m_ranges; } + + bool contains(sv::sv_frame_t frame) const; + + /// The range that contains frame, if any + bool getRangeAt(sv::sv_frame_t frame, Range &range) const; + + bool overlaps(sv::sv_frame_t start, sv::sv_frame_t end) const; + + /// End of the last range, 0 if there is none + sv::sv_frame_t getEndFrame() const; + + bool operator==(const Coverage &c) const { return m_ranges == c.m_ranges; } + bool operator!=(const Coverage &c) const { return !(*this == c); } + +private: + Ranges m_ranges; +}; + +#endif diff --git a/main/TakeAudio.cpp b/main/TakeAudio.cpp new file mode 100644 index 00000000..8c885b42 --- /dev/null +++ b/main/TakeAudio.cpp @@ -0,0 +1,268 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeAudio.h" + +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" + +#include +#include +#include + +#include +#include +#include + +using namespace sv; + +namespace { + +const sv_frame_t blockFrames = 16384; + +QString tr(const char *text) +{ + return QCoreApplication::translate("TakeAudio", text); +} + +// A stretch [start, end) of the take to be filled from a file, or +// with silence if there is no source +struct Patch { + sv_frame_t start; + sv_frame_t end; + WavFileReader *source; + sv_frame_t sourceOffset; // source frame that lands on "start" +}; + +std::unique_ptr openWav(QString path, QString &error) +{ + if (!QFileInfo(path).isFile()) { + error = tr("Audio file \"%1\" does not exist").arg(path); + return {}; + } + std::unique_ptr reader(new WavFileReader(FileSource(path))); + if (!reader->isOK()) { + error = tr("Failed to read audio file \"%1\": %2") + .arg(path).arg(reader->getError()); + return {}; + } + return reader; +} + +// count frames from "start" on, interleaved in "channels" channels, +// silent where the file has nothing (start may be past its end) +floatvec_t readFrames(WavFileReader *reader, sv_frame_t start, + sv_frame_t count, int channels) +{ + floatvec_t result(count * channels, 0.f); + if (!reader || start >= reader->getFrameCount()) return result; + + int have = reader->getChannelCount(); + floatvec_t data = reader->getInterleavedFrames(start, count); + sv_frame_t got = sv_frame_t(data.size()) / have; + + for (sv_frame_t i = 0; i < got; ++i) { + for (int c = 0; c < channels; ++c) { + if (have == channels) { + result[i * channels + c] = data[i * have + c]; + } else if (channels == 1) { + float sum = 0.f; + for (int k = 0; k < have; ++k) sum += data[i * have + k]; + result[i] = sum / float(have); + } else { + result[i * channels + c] = data[i * have + (c % have)]; + } + } + } + + return result; +} + +// How much of the patch, rather than of what it replaces, is heard at +// a frame: 1 in the middle, less within "fade" frames of either end +float weightAt(sv_frame_t frame, const Patch &patch, sv_frame_t fade) +{ + sv_frame_t fromEdge = std::min(frame - patch.start + 1, patch.end - frame); + if (fromEdge > fade) return 1.f; + return float(fromEdge) / float(fade + 1); +} + +// Patches must be in order and must not overlap +QString write(WavFileReader *old, sv_samplerate_t rate, int channels, + sv_frame_t total, const std::vector &patches, + sv_frame_t fade, QString outPath) +{ + if (fade < 0) fade = TakeAudio::defaultFadeFrames(rate); + + QString error; + + { + // To a temporary file that is moved into place on close + WavFileWriter writer(outPath, rate, channels, + WavFileWriter::WriteToTemporary); + if (!writer.isOK()) { + error = writer.getError(); + } + + for (sv_frame_t b = 0; b < total && error == ""; b += blockFrames) { + + sv_frame_t count = std::min(blockFrames, total - b); + floatvec_t block = readFrames(old, b, count, channels); + + for (const Patch &patch : patches) { + + sv_frame_t from = std::max(b, patch.start); + sv_frame_t to = std::min(b + count, patch.end); + if (from >= to) continue; + + // Never more than half the patch, or the ends would meet + sv_frame_t patchFade = std::min(fade, (patch.end - patch.start) / 2); + + floatvec_t replacement = readFrames + (patch.source, patch.sourceOffset + (from - patch.start), + to - from, channels); + + for (sv_frame_t i = from; i < to; ++i) { + float w = weightAt(i, patch, patchFade); + for (int c = 0; c < channels; ++c) { + float &sample = block[(i - b) * channels + c]; + sample = sample * (1.f - w) + + replacement[(i - from) * channels + c] * w; + } + } + } + + if (!writer.putInterleavedFrames(block)) { + error = writer.getError(); + if (error == "") error = tr("Failed to write audio data"); + } + } + + if (error == "" && !writer.close()) { + error = writer.getError(); + if (error == "") error = tr("Failed to finish writing audio file"); + } + } + + // The writer puts its file in place even when it is abandoned + if (error != "") { + QFile::remove(outPath); + return tr("Failed to write \"%1\": %2").arg(outPath).arg(error); + } + + return ""; +} + +QString checkOutPath(QString outPath, QString oldPath) +{ + if (outPath == "") return tr("No file name given to write to"); + if (QFileInfo(outPath) == QFileInfo(oldPath) || + QFileInfo::exists(outPath)) { + return tr("File \"%1\" exists already, and is not to be overwritten") + .arg(outPath); + } + return ""; +} + +} // namespace + +sv_frame_t +TakeAudio::defaultFadeFrames(sv_samplerate_t rate) +{ + // 5 ms: too short to hear as a fade, long enough not to click + return sv_frame_t(rate * 0.005); +} + +QString +TakeAudio::splice(QString oldPath, QString recordingPath, + sv_frame_t recordingOffset, sv_frame_t position, + sv_frame_t length, QString outPath, + Coverage::Range *placed, sv_frame_t fadeFrames) +{ + QString error = checkOutPath(outPath, oldPath); + if (error != "") return error; + + auto recording = openWav(recordingPath, error); + if (!recording) return error; + + std::unique_ptr old; + if (oldPath != "") { + old = openWav(oldPath, error); + if (!old) return error; + if (old->getSampleRate() != recording->getSampleRate()) { + return tr("The recording's sample rate (%1) is not that of the take so far (%2)") + .arg(recording->getSampleRate()).arg(old->getSampleRate()); + } + } + + if (recordingOffset < 0) recordingOffset = 0; + + sv_frame_t available = recording->getFrameCount() - recordingOffset; + if (length < 0 || length > available) length = available; + + if (position < 0) { + recordingOffset -= position; + length += position; + position = 0; + } + + if (length <= 0) { + return tr("The recording is too short to use"); + } + + Patch patch { position, position + length, + recording.get(), recordingOffset }; + + sv_frame_t total = patch.end; + if (old) total = std::max(total, old->getFrameCount()); + + int channels = old ? old->getChannelCount() : recording->getChannelCount(); + + error = write(old.get(), recording->getSampleRate(), channels, total, + { patch }, fadeFrames, outPath); + + if (error == "" && placed) { + *placed = Coverage::Range(patch.start, patch.end); + } + + return error; +} + +QString +TakeAudio::erase(QString oldPath, const Coverage::Ranges &ranges, + QString outPath, sv_frame_t fadeFrames) +{ + QString error = checkOutPath(outPath, oldPath); + if (error != "") return error; + + auto old = openWav(oldPath, error); + if (!old) return error; + + sv_frame_t total = old->getFrameCount(); + + // In order, apart, and within the file + Coverage tidy; + for (const Coverage::Range &r : ranges) { + tidy.add(std::max(r.start, sv_frame_t(0)), std::min(r.end, total)); + } + + std::vector patches; + for (const Coverage::Range &r : tidy.getRanges()) { + patches.push_back({ r.start, r.end, nullptr, 0 }); + } + + return write(old.get(), old->getSampleRate(), old->getChannelCount(), + total, patches, fadeFrames, outPath); +} diff --git a/main/TakeAudio.h b/main/TakeAudio.h new file mode 100644 index 00000000..a78f99f1 --- /dev/null +++ b/main/TakeAudio.h @@ -0,0 +1,83 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_AUDIO_H +#define TONY_TAKE_AUDIO_H + +#include "Coverage.h" + +#include "base/BaseTypes.h" + +#include + +/** + * The audio of a singing take is one WAV file that starts at frame 0 + * of the reference's timeline, silent wherever nothing was recorded. + * These functions make the next such file from the last: nothing is + * ever changed in place, so the file before is there to go back to. + * + * Both work through the files a block at a time, and both return an + * empty string on success and a message for the user otherwise. If + * they fail, there is no file at outPath. + */ +namespace TakeAudio +{ + /** + * The length of the fades that join new material to old, for + * audio at the given rate. + */ + sv::sv_frame_t defaultFadeFrames(sv::sv_samplerate_t rate); + + /** + * Write to outPath the take in oldPath with a recording put into + * it at the frame "position". + * + * oldPath may be empty: there is no take yet, and the result is + * silence up to the position. The recording is used from its + * frame recordingOffset on (the latency: what was recorded before + * the singer could have heard frame "position" of the reference), + * for "length" frames, or to its end if length is negative. Frames + * that would land before frame 0 of the take are left out. + * + * The result is as long as it needs to be to hold both. It has + * the channels of the old take if there was one (a recording with + * a different number is mixed down, or copied across, to suit) and + * of the recording otherwise. The ends of the new material are + * crossfaded with what was there over fadeFrames frames; negative + * means defaultFadeFrames(). + * + * If placed is not null, it receives the range of the take that + * the recording now fills. + */ + QString splice(QString oldPath, + QString recordingPath, + sv::sv_frame_t recordingOffset, + sv::sv_frame_t position, + sv::sv_frame_t length, + QString outPath, + Coverage::Range *placed = nullptr, + sv::sv_frame_t fadeFrames = -1); + + /** + * Write to outPath the take in oldPath with the given ranges made + * silent, fading out into each and in again after it. The result + * is as long as the original. + */ + QString erase(QString oldPath, + const Coverage::Ranges &ranges, + QString outPath, + sv::sv_frame_t fadeFrames = -1); +} + +#endif diff --git a/main/test/TestCoverage.h b/main/test/TestCoverage.h new file mode 100644 index 00000000..ccef2522 --- /dev/null +++ b/main/test/TestCoverage.h @@ -0,0 +1,174 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_COVERAGE_H +#define TEST_COVERAGE_H + +// Tier 2: the list of recorded ranges of a singing take + +#include "../Coverage.h" + +#include +#include + +class TestCoverage : public QObject +{ + Q_OBJECT + + typedef Coverage::Range Range; + typedef Coverage::Ranges Ranges; + +private slots: + void empty() { + Coverage c; + QVERIFY(c.isEmpty()); + QVERIFY(!c.contains(0)); + QVERIFY(!c.overlaps(0, 100)); + QCOMPARE(c.getEndFrame(), sv::sv_frame_t(0)); + c.remove(0, 100); + QVERIFY(c.isEmpty()); + } + + void add_keeps_order() { + Coverage c; + c.add(500, 600); + c.add(100, 200); + c.add(300, 400); + QCOMPARE(c.getRanges(), + (Ranges { Range(100, 200), Range(300, 400), Range(500, 600) })); + QCOMPARE(c.getEndFrame(), sv::sv_frame_t(600)); + } + + void add_ignores_empty_ranges() { + Coverage c; + c.add(100, 100); + c.add(200, 100); + QVERIFY(c.isEmpty()); + } + + void add_joins_overlapping() { + Coverage c; + c.add(100, 200); + c.add(300, 400); + c.add(500, 600); + c.add(150, 350); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 400), Range(500, 600) })); + c.add(0, 1000); + QCOMPARE(c.getRanges(), (Ranges { Range(0, 1000) })); + } + + // A recording that starts where the last one stopped is one + // stretch of singing, not two + void add_joins_touching() { + Coverage c; + c.add(100, 200); + c.add(200, 300); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 300) })); + c.add(50, 100); + QCOMPARE(c.getRanges(), (Ranges { Range(50, 300) })); + c.add(301, 400); + QCOMPARE(c.getRanges(), (Ranges { Range(50, 300), Range(301, 400) })); + } + + void add_inside_changes_nothing() { + Coverage c; + c.add(100, 400); + c.add(200, 300); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 400) })); + } + + void remove_whole() { + Coverage c; + c.add(100, 200); + c.add(300, 400); + c.remove(300, 400); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 200) })); + c.remove(0, 1000); + QVERIFY(c.isEmpty()); + } + + void remove_trims() { + Coverage c; + c.add(100, 400); + c.remove(50, 150); + QCOMPARE(c.getRanges(), (Ranges { Range(150, 400) })); + c.remove(350, 500); + QCOMPARE(c.getRanges(), (Ranges { Range(150, 350) })); + } + + void remove_splits() { + Coverage c; + c.add(100, 400); + c.remove(200, 300); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 200), Range(300, 400) })); + } + + void remove_across_several() { + Coverage c; + c.add(100, 200); + c.add(300, 400); + c.add(500, 600); + c.remove(150, 550); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 150), Range(550, 600) })); + } + + void remove_outside_changes_nothing() { + Coverage c; + c.add(100, 200); + c.remove(200, 300); + c.remove(0, 100); + c.remove(150, 150); + QCOMPARE(c.getRanges(), (Ranges { Range(100, 200) })); + } + + // The end is not part of the range + void contains_and_range_at() { + Coverage c; + c.add(100, 200); + c.add(300, 400); + QVERIFY(!c.contains(99)); + QVERIFY(c.contains(100)); + QVERIFY(c.contains(199)); + QVERIFY(!c.contains(200)); + QVERIFY(!c.contains(250)); + + Range r(7, 8); + QVERIFY(c.getRangeAt(350, r)); + QCOMPARE(r, Range(300, 400)); + QVERIFY(!c.getRangeAt(250, r)); + QCOMPARE(r, Range(300, 400)); + } + + void overlaps() { + Coverage c; + c.add(100, 200); + QVERIFY(!c.overlaps(0, 100)); + QVERIFY(c.overlaps(0, 101)); + QVERIFY(c.overlaps(150, 160)); + QVERIFY(c.overlaps(199, 300)); + QVERIFY(!c.overlaps(200, 300)); + QVERIFY(!c.overlaps(150, 150)); + } + + void equality() { + Coverage a, b; + a.add(100, 200); + a.add(200, 300); + b.add(100, 300); + QVERIFY(a == b); + b.remove(150, 160); + QVERIFY(a != b); + } +}; + +#endif diff --git a/main/test/TestTakeAudio.h b/main/test/TestTakeAudio.h new file mode 100644 index 00000000..f2ec65a5 --- /dev/null +++ b/main/test/TestTakeAudio.h @@ -0,0 +1,453 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TAKE_AUDIO_H +#define TEST_TAKE_AUDIO_H + +// Tier 2: putting a recording into the audio file of a singing take, +// and taking parts of one out again. The signals are constants and +// ramps so that every sample of the result says where it came from. + +#include "../TakeAudio.h" + +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" + +#include +#include +#include +#include + +#include +#include + +class TestTakeAudio : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + typedef std::vector Signal; // interleaved + + static constexpr double kRate = 44100.0; + static constexpr float kTolerance = 1e-3f; // the files may be 16-bit + static constexpr frame_t kFade = 100; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + QString newPath(QString stem = "audio") { + return m_dir.filePath(QString("%1-%2.wav").arg(stem).arg(++m_fileCounter)); + } + + QString writeWav(const Signal &interleaved, int channels = 1, + double rate = kRate) { + QString path = newPath("source"); + sv::WavFileWriter writer(path, rate, channels, + sv::WavFileWriter::WriteToTarget); + sv::floatvec_t data(interleaved.begin(), interleaved.end()); + if (!writer.isOK() || !writer.putInterleavedFrames(data) || + !writer.close()) { + return ""; + } + return path; + } + + static Signal constant(frame_t frames, float value) { + return Signal(frames, value); + } + + // Rises from 0.1 to 0.6, so that no frame of it is silent and each + // differs from the next by more than the tolerance only slowly: + // value says which frame, to within a few + static Signal ramp(frame_t frames) { + Signal s(frames); + for (frame_t i = 0; i < frames; ++i) s[i] = rampValue(i, frames); + return s; + } + static float rampValue(frame_t i, frame_t frames) { + return 0.1f + 0.5f * float(i) / float(frames); + } + + struct Audio { + bool ok = false; + int channels = 0; + frame_t frames = 0; + Signal samples; + float at(frame_t i, int c = 0) const { return samples[i * channels + c]; } + }; + + Audio read(QString path) { + Audio a; + sv::WavFileReader reader { sv::FileSource(path) }; + if (!reader.isOK()) return a; + a.ok = true; + a.channels = reader.getChannelCount(); + a.frames = reader.getFrameCount(); + auto data = reader.getInterleavedFrames(0, a.frames); + a.samples.assign(data.begin(), data.end()); + return a; + } + + static bool nearly(float a, float b) { return std::fabs(a - b) <= kTolerance; } + + // Every frame of [from, to) in channel c is value + static bool allNearly(const Audio &a, frame_t from, frame_t to, float value, + int c = 0) { + for (frame_t i = from; i < to; ++i) { + if (!nearly(a.at(i, c), value)) { + qWarning("frame %lld is %f, expected %f", + (long long)i, a.at(i, c), value); + return false; + } + } + return true; + } + + // [from, to) goes from one value to the other without ever + // leaving the span between them or turning back + static bool monotonicBetween(const Audio &a, frame_t from, frame_t to, + float first, float last) { + float lo = std::min(first, last) - kTolerance; + float hi = std::max(first, last) + kTolerance; + float direction = (last > first ? 1.f : -1.f); + for (frame_t i = from; i < to; ++i) { + float v = a.at(i); + if (v < lo || v > hi) { + qWarning("frame %lld is %f, outside %f..%f", + (long long)i, v, lo, hi); + return false; + } + if (i > from && (v - a.at(i - 1)) * direction < -kTolerance) { + qWarning("frame %lld turns back", (long long)i); + return false; + } + } + return true; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void fade_length() { + QCOMPARE(TakeAudio::defaultFadeFrames(44100.0), frame_t(220)); + QCOMPARE(TakeAudio::defaultFadeFrames(48000.0), frame_t(240)); + } + + // The first recording of a take, started part of the way in + void splice_into_nothing() { + QString recording = writeWav(constant(4000, 0.5f)); + QString out = newPath(); + Coverage::Range placed; + QString error = TakeAudio::splice("", recording, 0, 1000, -1, out, + &placed, kFade); + QCOMPARE(error, QString()); + QCOMPARE(placed, Coverage::Range(1000, 5000)); + + Audio a = read(out); + QVERIFY(a.ok); + QCOMPARE(a.channels, 1); + QCOMPARE(a.frames, frame_t(5000)); + QVERIFY(allNearly(a, 0, 1000, 0.f)); + QVERIFY(monotonicBetween(a, 1000, 1000 + kFade, 0.f, 0.5f)); + QVERIFY(allNearly(a, 1000 + kFade, 5000 - kFade, 0.5f)); + QVERIFY(monotonicBetween(a, 5000 - kFade, 5000, 0.5f, 0.f)); + // and it does fade: the first frame is nearly silent + QVERIFY(a.at(1000) < 0.05f); + QVERIFY(a.at(4999) < 0.05f); + } + + // What was recorded before the singer could have heard the + // reference is left out: frame "offset" of the recording is what + // lands on the position + void splice_removes_latency() { + const frame_t n = 8000, offset = 1500, position = 3000; + QString recording = writeWav(ramp(n)); + QString out = newPath(); + Coverage::Range placed; + QCOMPARE(TakeAudio::splice("", recording, offset, position, -1, out, + &placed, kFade), QString()); + QCOMPARE(placed, Coverage::Range(position, position + n - offset)); + + Audio a = read(out); + QVERIFY(a.ok); + QCOMPARE(a.frames, position + n - offset); + for (frame_t k : { frame_t(kFade), frame_t(2000), n - offset - kFade - 1 }) { + QVERIFY2(nearly(a.at(position + k), rampValue(offset + k, n)), + qPrintable(QString::number(k))); + } + } + + void splice_length_limits_what_is_used() { + QString recording = writeWav(constant(4000, 0.5f)); + QString out = newPath(); + Coverage::Range placed; + QCOMPARE(TakeAudio::splice("", recording, 500, 0, 1000, out, + &placed, kFade), QString()); + QCOMPARE(placed, Coverage::Range(0, 1000)); + QCOMPARE(read(out).frames, frame_t(1000)); + + // more than there is: what there is + out = newPath(); + QCOMPARE(TakeAudio::splice("", recording, 500, 0, 99999, out, + &placed, kFade), QString()); + QCOMPARE(placed, Coverage::Range(0, 3500)); + } + + // Singing one part again + void splice_replaces_the_middle() { + QString old = writeWav(constant(10000, 0.25f)); + QString recording = writeWav(constant(2000, -0.5f)); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 3000, -1, out, + nullptr, kFade), QString()); + + Audio a = read(out); + QVERIFY(a.ok); + QCOMPARE(a.frames, frame_t(10000)); + QVERIFY(allNearly(a, 0, 3000, 0.25f)); + // crossfaded: straight from the one to the other, with no dip + // to silence on the way and no jump + QVERIFY(monotonicBetween(a, 3000, 3000 + kFade, 0.25f, -0.5f)); + QVERIFY(allNearly(a, 3000 + kFade, 5000 - kFade, -0.5f)); + QVERIFY(monotonicBetween(a, 5000 - kFade, 5000, -0.5f, 0.25f)); + QVERIFY(allNearly(a, 5000, 10000, 0.25f)); + for (frame_t i = 2999; i < 5001; ++i) { + QVERIFY2(std::fabs(a.at(i + 1) - a.at(i)) < 0.8f / float(kFade), + qPrintable(QString("jump at %1").arg(i))); + } + + // and the take it was made from is as it was + Audio before = read(old); + QCOMPARE(before.frames, frame_t(10000)); + QVERIFY(allNearly(before, 0, 10000, 0.25f)); + } + + void splice_past_the_end_lengthens() { + QString old = writeWav(constant(3000, 0.25f)); + QString recording = writeWav(constant(2000, 0.5f)); + + // running off the end + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 2000, -1, out, + nullptr, kFade), QString()); + Audio a = read(out); + QCOMPARE(a.frames, frame_t(4000)); + QVERIFY(allNearly(a, 0, 2000, 0.25f)); + QVERIFY(allNearly(a, 2000 + kFade, 4000 - kFade, 0.5f)); + + // starting beyond it, after an intermission + out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 6000, -1, out, + nullptr, kFade), QString()); + a = read(out); + QCOMPARE(a.frames, frame_t(8000)); + QVERIFY(allNearly(a, 0, 3000, 0.25f)); + QVERIFY(allNearly(a, 3000, 6000, 0.f)); + QVERIFY(allNearly(a, 6000 + kFade, 8000 - kFade, 0.5f)); + } + + void splice_before_zero_is_clipped() { + const frame_t n = 4000; + QString recording = writeWav(ramp(n)); + QString out = newPath(); + Coverage::Range placed; + QCOMPARE(TakeAudio::splice("", recording, 0, -1000, -1, out, + &placed, kFade), QString()); + QCOMPARE(placed, Coverage::Range(0, 3000)); + Audio a = read(out); + QCOMPARE(a.frames, frame_t(3000)); + QVERIFY(nearly(a.at(1500), rampValue(2500, n))); + } + + void splice_with_no_fade() { + QString old = writeWav(constant(3000, 0.25f)); + QString recording = writeWav(constant(1000, 0.5f)); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 1000, -1, out, + nullptr, 0), QString()); + Audio a = read(out); + QVERIFY(allNearly(a, 0, 1000, 0.25f)); + QVERIFY(allNearly(a, 1000, 2000, 0.5f)); + QVERIFY(allNearly(a, 2000, 3000, 0.25f)); + } + + // Shorter than two fades: they are shortened, and do not run into + // each other or outside the recording + void splice_shorter_than_the_fades() { + QString old = writeWav(constant(3000, 0.25f)); + QString recording = writeWav(constant(50, 0.5f)); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 1000, -1, out, + nullptr, kFade), QString()); + Audio a = read(out); + QVERIFY(allNearly(a, 0, 1000, 0.25f)); + QVERIFY(allNearly(a, 1050, 3000, 0.25f)); + QVERIFY(monotonicBetween(a, 1000, 1025, 0.25f, 0.5f)); + QVERIFY(monotonicBetween(a, 1025, 1050, 0.5f, 0.25f)); + QVERIFY(a.at(1025) > 0.45f); + } + + // The take keeps its channels whatever the device gives later + void splice_mono_into_stereo() { + Signal stereo; + for (int i = 0; i < 3000; ++i) { + stereo.push_back(0.2f); + stereo.push_back(-0.2f); + } + QString old = writeWav(stereo, 2); + QString recording = writeWav(constant(1000, 0.5f)); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 1000, -1, out, + nullptr, 0), QString()); + Audio a = read(out); + QCOMPARE(a.channels, 2); + QCOMPARE(a.frames, frame_t(3000)); + QVERIFY(allNearly(a, 0, 1000, 0.2f, 0)); + QVERIFY(allNearly(a, 0, 1000, -0.2f, 1)); + QVERIFY(allNearly(a, 1000, 2000, 0.5f, 0)); + QVERIFY(allNearly(a, 1000, 2000, 0.5f, 1)); + QVERIFY(allNearly(a, 2000, 3000, -0.2f, 1)); + } + + void splice_stereo_into_mono() { + Signal stereo; + for (int i = 0; i < 1000; ++i) { + stereo.push_back(0.6f); + stereo.push_back(0.2f); + } + QString old = writeWav(constant(3000, 0.1f)); + QString recording = writeWav(stereo, 2); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, 1000, -1, out, + nullptr, 0), QString()); + Audio a = read(out); + QCOMPARE(a.channels, 1); + QVERIFY(allNearly(a, 1000, 2000, 0.4f)); + QVERIFY(allNearly(a, 2000, 3000, 0.1f)); + } + + // Longer than the block the work is done in, with the recording + // across a block boundary + void splice_long_files() { + const frame_t oldLength = 100000, n = 40000, position = 30000; + QString old = writeWav(constant(oldLength, 0.25f)); + QString recording = writeWav(ramp(n)); + QString out = newPath(); + QCOMPARE(TakeAudio::splice(old, recording, 0, position, -1, out, + nullptr, kFade), QString()); + Audio a = read(out); + QCOMPARE(a.frames, oldLength); + QVERIFY(allNearly(a, 0, position, 0.25f)); + for (frame_t k = kFade; k < n - kFade; k += 97) { + QVERIFY2(nearly(a.at(position + k), rampValue(k, n)), + qPrintable(QString::number(k))); + } + QVERIFY(allNearly(a, position + n, oldLength, 0.25f)); + } + + void splice_errors() { + QString old = writeWav(constant(3000, 0.25f)); + QString recording = writeWav(constant(1000, 0.5f)); + QString other = writeWav(constant(1000, 0.5f), 1, 48000.0); + + QString out = newPath(); + QVERIFY(TakeAudio::splice(old, m_dir.filePath("absent.wav"), + 0, 0, -1, out) != ""); + QVERIFY(!QFile::exists(out)); + + QVERIFY(TakeAudio::splice(m_dir.filePath("absent.wav"), recording, + 0, 0, -1, out) != ""); + QVERIFY(!QFile::exists(out)); + + QVERIFY(TakeAudio::splice(old, other, 0, 0, -1, out) != ""); + QVERIFY(!QFile::exists(out)); + + // nothing left of it after the latency, or before frame 0 + QVERIFY(TakeAudio::splice(old, recording, 1000, 0, -1, out) != ""); + QVERIFY(TakeAudio::splice(old, recording, 0, -1000, -1, out) != ""); + QVERIFY(TakeAudio::splice(old, recording, 0, 0, 0, out) != ""); + QVERIFY(!QFile::exists(out)); + + // never over a file that is there, the take's own least of all + QVERIFY(TakeAudio::splice(old, recording, 0, 0, -1, old) != ""); + QVERIFY(TakeAudio::splice(old, recording, 0, 0, -1, recording) != ""); + QVERIFY(TakeAudio::splice(old, recording, 0, 0, -1, "") != ""); + QVERIFY(allNearly(read(old), 0, 3000, 0.25f)); + QVERIFY(allNearly(read(recording), 0, 1000, 0.5f)); + } + + void erase_the_middle() { + QString old = writeWav(constant(10000, 0.5f)); + QString out = newPath(); + QCOMPARE(TakeAudio::erase(old, { Coverage::Range(3000, 5000) }, out, + kFade), QString()); + Audio a = read(out); + QVERIFY(a.ok); + QCOMPARE(a.frames, frame_t(10000)); + QVERIFY(allNearly(a, 0, 3000, 0.5f)); + QVERIFY(monotonicBetween(a, 3000, 3000 + kFade, 0.5f, 0.f)); + QVERIFY(allNearly(a, 3000 + kFade, 5000 - kFade, 0.f)); + QVERIFY(monotonicBetween(a, 5000 - kFade, 5000, 0.f, 0.5f)); + QVERIFY(allNearly(a, 5000, 10000, 0.5f)); + + QVERIFY(allNearly(read(old), 0, 10000, 0.5f)); + } + + // Out of order, overlapping, and off both ends of the file + void erase_several() { + QString old = writeWav(constant(10000, 0.5f)); + QString out = newPath(); + Coverage::Ranges ranges { + Coverage::Range(8000, 20000), + Coverage::Range(2000, 3000), + Coverage::Range(2500, 4000), + Coverage::Range(-500, 1000), + Coverage::Range(6000, 6000), + }; + QCOMPARE(TakeAudio::erase(old, ranges, out, 0), QString()); + Audio a = read(out); + QCOMPARE(a.frames, frame_t(10000)); + QVERIFY(allNearly(a, 0, 1000, 0.f)); + QVERIFY(allNearly(a, 1000, 2000, 0.5f)); + QVERIFY(allNearly(a, 2000, 4000, 0.f)); + QVERIFY(allNearly(a, 4000, 8000, 0.5f)); + QVERIFY(allNearly(a, 8000, 10000, 0.f)); + } + + void erase_nothing_is_a_copy() { + QString old = writeWav(ramp(5000)); + QString out = newPath(); + QCOMPARE(TakeAudio::erase(old, {}, out), QString()); + Audio a = read(out); + QCOMPARE(a.frames, frame_t(5000)); + for (frame_t i = 0; i < 5000; i += 50) { + QVERIFY(nearly(a.at(i), rampValue(i, 5000))); + } + } + + void erase_errors() { + QString old = writeWav(constant(3000, 0.5f)); + QString out = newPath(); + Coverage::Ranges ranges { Coverage::Range(1000, 2000) }; + QVERIFY(TakeAudio::erase(m_dir.filePath("absent.wav"), ranges, out) != ""); + QVERIFY(TakeAudio::erase("", ranges, out) != ""); + QVERIFY(!QFile::exists(out)); + QVERIFY(TakeAudio::erase(old, ranges, old) != ""); + QVERIFY(allNearly(read(old), 0, 3000, 0.5f)); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 4471d199..26959983 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -14,6 +14,8 @@ #include "TestRealtimeYin.h" #include "TestRealtimePitchTracker.h" #include "TestLatencyShift.h" +#include "TestCoverage.h" +#include "TestTakeAudio.h" #include "RunSuite.h" @@ -56,6 +58,18 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestCoverage t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + + { + TestTakeAudio t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 64d0c19e..2b2af92b 100644 --- a/meson.build +++ b/meson.build @@ -1094,7 +1094,9 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ + 'main/Coverage.cpp', 'main/RealtimePitchTracker.cpp', + 'main/TakeAudio.cpp', ] tony_app_files = [ @@ -1339,6 +1341,8 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestRealtimeYin.h', 'main/test/TestRealtimePitchTracker.h', 'main/test/TestLatencyShift.h', + 'main/test/TestCoverage.h', + 'main/test/TestTakeAudio.h', ]) tony_core_test_exe = executable( From 7e975f4518c715e756cd29767345a891363553e3 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 00:44:09 +0300 Subject: [PATCH 028/275] feat: SingingTakes, the state of a singing take The audio file a take's singing lives in, which ranges of it hold recorded material, and the files it had before, which are kept for undo until the session closes. It writes the next file of a take through TakeAudio::splice(), under a name in the record directory that nothing else has, and it answers whether recording from a frame would record over something and whether to ask about that. No layers or models are in it, so its logic is in tony_core and tested without a window. Nothing uses it yet. Co-Authored-By: Claude Opus 5 --- main/SingingTakes.cpp | 130 ++++++++++++++++ main/SingingTakes.h | 110 ++++++++++++++ main/test/TestSingingTakes.h | 278 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 3 + 5 files changed, 528 insertions(+) create mode 100644 main/SingingTakes.cpp create mode 100644 main/SingingTakes.h create mode 100644 main/test/TestSingingTakes.h diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp new file mode 100644 index 00000000..715b36f6 --- /dev/null +++ b/main/SingingTakes.cpp @@ -0,0 +1,130 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "SingingTakes.h" + +#include "TakeAudio.h" + +#include +#include +#include + +using namespace sv; + +SingingTakes::SingingTakes(QObject *parent) : + QObject(parent) +{ +} + +SingingTakes::~SingingTakes() +{ +} + +void +SingingTakes::clear() +{ + m_audioPath = ""; + m_coverage.clear(); + m_superseded.clear(); +} + +void +SingingTakes::setWholeFileTake(QString path, sv_frame_t frames) +{ + if (m_audioPath != "" && m_audioPath != path) { + m_superseded.push_back(m_audioPath); + } + m_audioPath = path; + m_coverage.clear(); + m_coverage.add(0, frames); +} + +QString +SingingTakes::spliceRecording(QString recordingPath, + sv_frame_t recordingOffset, + sv_frame_t position, + sv_frame_t length, + QString directory, + Coverage::Range *placed) +{ + QString outPath = nextAudioPath(directory); + if (outPath == "") { + return tr("Could not find a name to write the singing track under, " + "in \"%1\"").arg(directory); + } + + Coverage::Range range; + QString error = TakeAudio::splice(m_audioPath, recordingPath, + recordingOffset, position, length, + outPath, &range); + if (error != "") return error; + + if (m_audioPath != "") m_superseded.push_back(m_audioPath); + m_audioPath = outPath; + m_coverage.add(range.start, range.end); + + if (placed) *placed = range; + return ""; +} + +bool +SingingTakes::coversPosition(sv_frame_t position) const +{ + return m_coverage.contains(position); +} + +bool +SingingTakes::shouldConfirmRecordingAt(sv_frame_t position) const +{ + return coversPosition(position) && isOverwriteConfirmationWanted(); +} + +bool +SingingTakes::isOverwriteConfirmationWanted() +{ + QSettings settings; + settings.beginGroup("MainWindow"); + bool wanted = settings.value("confirmrecordover", true).toBool(); + settings.endGroup(); + return wanted; +} + +void +SingingTakes::setOverwriteConfirmationWanted(bool wanted) +{ + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("confirmrecordover", wanted); + settings.endGroup(); +} + +QString +SingingTakes::nextAudioPath(QString directory) +{ + QDir dir(directory); + + // The time of day is enough to tell one from the next within a + // session; the counter is there because it need only be enough. + // ":" is not allowed in a file name on Windows + QString stamp = QDateTime::currentDateTime().toString("yyyyMMdd-HHmmss-zzz"); + + for (int i = 0; i < 1000; ++i) { + QString name = (i == 0 ? + QString("take-%1.wav").arg(stamp) : + QString("take-%1-%2.wav").arg(stamp).arg(i)); + if (!dir.exists(name)) return dir.filePath(name); + } + + return ""; +} diff --git a/main/SingingTakes.h b/main/SingingTakes.h new file mode 100644 index 00000000..891d14a6 --- /dev/null +++ b/main/SingingTakes.h @@ -0,0 +1,110 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_SINGING_TAKES_H +#define TONY_SINGING_TAKES_H + +#include "Coverage.h" + +#include "base/BaseTypes.h" + +#include +#include +#include + +/** + * The state of the singing takes of a session: the audio file each + * take's singing lives in, and the ranges of that file that hold + * recorded material. Until takes proper arrive there is at most one. + * + * It knows nothing of layers, models or windows: MainWindow asks it + * where a recording is to go and what the take is afterwards, and puts + * the answer on the screen itself. The audio files are written by + * TakeAudio. + */ +class SingingTakes : public QObject +{ + Q_OBJECT + +public: + SingingTakes(QObject *parent = nullptr); + virtual ~SingingTakes(); + + /// There is a take, with an audio file to play and analyse + bool haveTake() const { return m_audioPath != ""; } + + QString getAudioPath() const { return m_audioPath; } + const Coverage &getCoverage() const { return m_coverage; } + + /// Nothing recorded and nothing to go back to: a new session + void clear(); + + /** + * The take is the whole of this file: a singing track the user + * loaded, or one restored from a session saved before coverage was + * stored with it. + */ + void setWholeFileTake(QString path, sv::sv_frame_t frames); + + /** + * Write the next audio file of the take, with the recording in + * recordingPath spliced into it at "position" on the reference's + * timeline. The recording is used from its frame recordingOffset + * on (the latency: what was recorded before the singer could have + * heard the reference at "position"), for "length" frames, or to + * its end if length is negative. The file is written into + * directory, under a name that nothing else is using. + * + * On success returns "" and the take's audio is the new file, its + * coverage takes in what was recorded, and the file before is + * remembered as superseded. Otherwise the take is as it was and + * the return is a message for the user. + */ + QString spliceRecording(QString recordingPath, + sv::sv_frame_t recordingOffset, + sv::sv_frame_t position, + sv::sv_frame_t length, + QString directory, + Coverage::Range *placed = nullptr); + + /** + * The audio files of takes that later files have replaced during + * this run. They are kept until the session closes, for undo. + */ + const QStringList &getSupersededPaths() const { return m_superseded; } + + /// Recording from this frame on would record over material that is there + bool coversPosition(sv::sv_frame_t position) const; + + /// ... and the user has not asked to stop being asked about it + bool shouldConfirmRecordingAt(sv::sv_frame_t position) const; + + /// The state of "Don't ask again" in the overwrite question + static bool isOverwriteConfirmationWanted(); + static void setOverwriteConfirmationWanted(bool wanted); + + /** + * A path in directory for the next audio file of a take, under a + * name that no file there has. Empty if every name tried was + * taken. + */ + static QString nextAudioPath(QString directory); + +private: + QString m_audioPath; + Coverage m_coverage; + QStringList m_superseded; +}; + +#endif diff --git a/main/test/TestSingingTakes.h b/main/test/TestSingingTakes.h new file mode 100644 index 00000000..75db641c --- /dev/null +++ b/main/test/TestSingingTakes.h @@ -0,0 +1,278 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_SINGING_TAKES_H +#define TEST_SINGING_TAKES_H + +// Tier 2: the state of a singing take — which audio file it is in, what +// of it holds recorded material, and what recording into it does to +// both. No window and no audio device: the files are written here and +// read back. + +#include "../SingingTakes.h" + +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +class TestSingingTakes : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + + static constexpr double kRate = 44100.0; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + // A recording of the given length, at a level that says which one it is + QString writeRecording(frame_t frames, float level) { + QString path = m_dir.filePath + (QString("recorded-%1.wav").arg(++m_fileCounter)); + sv::WavFileWriter writer(path, kRate, 1, + sv::WavFileWriter::WriteToTarget); + sv::floatvec_t data(frames, level); + if (!writer.isOK() || !writer.putInterleavedFrames(data) || + !writer.close()) { + return ""; + } + return path; + } + + QString takeDirectory() { + QDir dir(m_dir.path()); + dir.mkpath("takes"); + return dir.filePath("takes"); + } + + static frame_t framesIn(QString path) { + sv::WavFileReader reader { sv::FileSource(path) }; + return reader.isOK() ? reader.getFrameCount() : -1; + } + + static float sampleAt(QString path, frame_t frame) { + sv::WavFileReader reader { sv::FileSource(path) }; + if (!reader.isOK()) return 0.f; + auto data = reader.getInterleavedFrames(frame, 1); + return data.empty() ? 0.f : data[0]; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void init() { + SingingTakes::setOverwriteConfirmationWanted(true); + } + + void nothing_to_start_with() { + SingingTakes takes; + QVERIFY(!takes.haveTake()); + QCOMPARE(takes.getAudioPath(), QString()); + QVERIFY(takes.getCoverage().isEmpty()); + QVERIFY(takes.getSupersededPaths().isEmpty()); + // Nothing is recorded, so nothing can be recorded over + QVERIFY(!takes.coversPosition(0)); + QVERIFY(!takes.shouldConfirmRecordingAt(0)); + } + + void whole_file_take() { + SingingTakes takes; + takes.setWholeFileTake("/somewhere/sung.wav", 1000); + QVERIFY(takes.haveTake()); + QCOMPARE(takes.getAudioPath(), QString("/somewhere/sung.wav")); + QCOMPARE(int(takes.getCoverage().getRanges().size()), 1); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 1000)); + QVERIFY(takes.coversPosition(999)); + QVERIFY(!takes.coversPosition(1000)); + + // Another file loaded in its place: the one before is kept, since + // it may be wanted again by undo + takes.setWholeFileTake("/somewhere/else.wav", 10); + QCOMPARE(takes.getAudioPath(), QString("/somewhere/else.wav")); + QCOMPARE(int(takes.getCoverage().getRanges().size()), 1); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 10)); + QCOMPARE(takes.getSupersededPaths(), + QStringList { "/somewhere/sung.wav" }); + + takes.clear(); + QVERIFY(!takes.haveTake()); + QVERIFY(takes.getCoverage().isEmpty()); + QVERIFY(takes.getSupersededPaths().isEmpty()); + } + + // Every audio file of a take gets a name of its own, in the directory + // asked for, and never one that is taken: TakeAudio refuses to write + // over a file, so a name that collides would lose a recording + void every_file_has_its_own_name() { + QString dir = takeDirectory(); + QStringList made; + + for (int i = 0; i < 4; ++i) { + QString path = SingingTakes::nextAudioPath(dir); + QVERIFY(!path.isEmpty()); + QCOMPARE(QFileInfo(path).absolutePath(), + QFileInfo(dir).absoluteFilePath()); + QVERIFY2(!QFileInfo::exists(path), qPrintable(path)); + QVERIFY2(!made.contains(path), qPrintable(path)); + QFile file(path); + QVERIFY(file.open(QIODevice::WriteOnly)); + file.close(); + made << path; + } + } + + // The first recording of a take, made from part of the way in + void splice_first_recording() { + SingingTakes takes; + QString recording = writeRecording(4000, 0.5f); + QVERIFY(!recording.isEmpty()); + + Coverage::Range placed; + QString error = takes.spliceRecording(recording, 1000, 2000, -1, + takeDirectory(), &placed); + QCOMPARE(error, QString()); + + // The first 1000 frames of the recording are the latency: what was + // recorded before the singer could have heard frame 2000 + QCOMPARE(placed, Coverage::Range(2000, 5000)); + QVERIFY(takes.haveTake()); + QCOMPARE(int(takes.getCoverage().getRanges().size()), 1); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(2000, 5000)); + QVERIFY(takes.getSupersededPaths().isEmpty()); + + QString path = takes.getAudioPath(); + QCOMPARE(framesIn(path), frame_t(5000)); + QCOMPARE(sampleAt(path, 500), 0.f); + QVERIFY(std::fabs(sampleAt(path, 3500) - 0.5f) < 1e-3f); + } + + // A second recording, into the silence after the first: the file grows, + // the gap stays as it was and the coverage has two ranges + void splice_into_a_gap() { + SingingTakes takes; + QString first = writeRecording(1000, 0.5f); + QString second = writeRecording(1000, 0.25f); + QVERIFY(takes.spliceRecording(first, 0, 0, -1, + takeDirectory()).isEmpty()); + QString firstPath = takes.getAudioPath(); + + Coverage::Range placed; + QVERIFY(takes.spliceRecording(second, 0, 5000, -1, takeDirectory(), + &placed).isEmpty()); + QCOMPARE(placed, Coverage::Range(5000, 6000)); + + QCOMPARE(int(takes.getCoverage().getRanges().size()), 2); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 1000)); + QCOMPARE(takes.getCoverage().getRanges()[1], Coverage::Range(5000, 6000)); + + QVERIFY(takes.getAudioPath() != firstPath); + QCOMPARE(takes.getSupersededPaths(), QStringList { firstPath }); + QVERIFY2(QFileInfo::exists(firstPath), + "the file the take had before was not kept"); + + QString path = takes.getAudioPath(); + QCOMPARE(framesIn(path), frame_t(6000)); + QVERIFY(std::fabs(sampleAt(path, 500) - 0.5f) < 1e-3f); + QCOMPARE(sampleAt(path, 3000), 0.f); + QVERIFY(std::fabs(sampleAt(path, 5500) - 0.25f) < 1e-3f); + } + + // Recording over material that is there: the ranges join, and the new + // material is what is heard where it landed + void splice_over_existing() { + SingingTakes takes; + QString first = writeRecording(4000, 0.5f); + QString second = writeRecording(1000, 0.25f); + QVERIFY(takes.spliceRecording(first, 0, 0, -1, + takeDirectory()).isEmpty()); + QVERIFY(takes.spliceRecording(second, 0, 2000, -1, + takeDirectory()).isEmpty()); + + QCOMPARE(int(takes.getCoverage().getRanges().size()), 1); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 4000)); + + QString path = takes.getAudioPath(); + QCOMPARE(framesIn(path), frame_t(4000)); + QVERIFY(std::fabs(sampleAt(path, 1000) - 0.5f) < 1e-3f); // kept + QVERIFY(std::fabs(sampleAt(path, 2500) - 0.25f) < 1e-3f); // replaced + QVERIFY(std::fabs(sampleAt(path, 3500) - 0.5f) < 1e-3f); // kept + } + + void splice_failure_leaves_the_take_alone() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QString path = takes.getAudioPath(); + Coverage before = takes.getCoverage(); + + // Nothing of the recording is left once the latency is taken off + QString error = takes.spliceRecording(recording, 2000, 0, -1, + takeDirectory()); + QVERIFY(!error.isEmpty()); + QCOMPARE(takes.getAudioPath(), path); + QVERIFY(takes.getCoverage() == before); + QVERIFY(takes.getSupersededPaths().isEmpty()); + + // and a recording that is not there at all + error = takes.spliceRecording(m_dir.filePath("not-a-file.wav"), 0, 0, + -1, takeDirectory()); + QVERIFY(!error.isEmpty()); + QCOMPARE(takes.getAudioPath(), path); + QVERIFY(takes.getCoverage() == before); + } + + // The question before recording over something: asked inside the + // covered ranges only, and not at all once the user has said so + void overwrite_question() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 1000, -1, + takeDirectory()).isEmpty()); + // coverage is [1000, 2000) + + QVERIFY(SingingTakes::isOverwriteConfirmationWanted()); + QVERIFY(takes.shouldConfirmRecordingAt(1000)); + QVERIFY(takes.shouldConfirmRecordingAt(1999)); + + // In a gap, before or after: recording there takes nothing away, + // even if it runs on into what is there + QVERIFY(!takes.shouldConfirmRecordingAt(0)); + QVERIFY(!takes.shouldConfirmRecordingAt(999)); + QVERIFY(!takes.shouldConfirmRecordingAt(2000)); + + SingingTakes::setOverwriteConfirmationWanted(false); + QVERIFY(!SingingTakes::isOverwriteConfirmationWanted()); + QVERIFY(takes.coversPosition(1500)); + QVERIFY(!takes.shouldConfirmRecordingAt(1500)); + + SingingTakes::setOverwriteConfirmationWanted(true); + QVERIFY(takes.shouldConfirmRecordingAt(1500)); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 26959983..b6a49afa 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -16,6 +16,7 @@ #include "TestLatencyShift.h" #include "TestCoverage.h" #include "TestTakeAudio.h" +#include "TestSingingTakes.h" #include "RunSuite.h" @@ -70,6 +71,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestSingingTakes t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 2b2af92b..e19fb49d 100644 --- a/meson.build +++ b/meson.build @@ -1096,6 +1096,7 @@ tony_entry_files = [ tony_core_files = [ 'main/Coverage.cpp', 'main/RealtimePitchTracker.cpp', + 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', ] @@ -1110,6 +1111,7 @@ tony_app_files = [ tony_core_moc_files = qt.preprocess( moc_headers: [ 'main/RealtimePitchTracker.h', + 'main/SingingTakes.h', ]) tony_app_moc_files = qt.preprocess( @@ -1343,6 +1345,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', 'main/test/TestTakeAudio.h', + 'main/test/TestSingingTakes.h', ]) tony_core_test_exe = executable( From 14bf9794e82d0c3e2840968bd50c179834a1b173 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 00:44:23 +0300 Subject: [PATCH 029/275] feat: record from the playback position, splice into the take on Stop Record no longer throws the singing track away and no longer starts at frame 0. It starts where the playhead is, asking first (with "Don't ask again") if that is inside singing that is already there; the reference plays from there and the live dots are drawn from there. The pitch and notes of the take stay on show while it is recorded into, and only its audio is kept out of the mix. The recording itself is no longer the singing track: it is raw material, held in the document by a hidden, muted layer of its own, so the extra pane record() gets from AddPaneCommand can be pruned at once as in loadSingingTrack(). On Stop the recording's model is released -- which closes the file, after stopRecording() has flushed and closed the writer -- and the recording is spliced into the take's audio at the position, read from the frame the latency points at. The take's audio therefore always starts at frame 0 and setStartFrame is out of this flow; the compensation is in the audio. The singing track is then rebuilt from the new file by the loadSingingTrack() path, analysed in full: the stand-in until the model swap and ranged analysis of phase 4. Tests: recording at a position, the latency taken off by the splice, recording over part of a take and in a gap, and the overwrite question. Four existing tests assert the old behaviour and are changed with it. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 667 ++++++++++++++++++++------------- main/MainWindow.h | 88 ++++- main/test/TestRecordWorkflow.h | 480 ++++++++++++++++++++++-- 3 files changed, 916 insertions(+), 319 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 491e1c13..4bfdf55c 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -75,6 +75,7 @@ #include #include +#include #include #include #include @@ -130,6 +131,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_alternatePitchUpAction(nullptr), m_alternatePitchDownAction(nullptr), m_referencePitchHiddenForTake(false), + m_takes(nullptr), + m_takePosition(0), m_backgroundMusicModelId(), m_backgroundMusicLayer(nullptr), m_loadBackgroundMusicAction(nullptr), @@ -156,6 +159,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_singingAudioAfterTake(true), m_paneCountBeforeRecording(0), m_currentRecordingModelId(), + m_recordingLayer(nullptr), + m_rebuildingTakeAudio(false), m_recordingLatencyFrames(0), m_recordingStartGapEstimate(0), m_recordingStartGapMeasured(-1), @@ -349,6 +354,8 @@ MainWindow::MainWindow(AudioMode audioMode, connect(m_analyser, SIGNAL(layersChanged()), this, SLOT(syncAlternatePitchTrack())); + m_takes = new SingingTakes(this); + setupMenus(); setupToolbars(); setupHelpMenu(); @@ -2091,6 +2098,7 @@ MainWindow::closeSession() // Tear down singing track and realtime pitch layer before panes/document // are destroyed, so they can cleanly remove their layers from the pane. teardownRealtimePitchLayer(); + teardownRecordingLayer(); teardownSingingTrackAnalyser(); teardownBackgroundMusic(); m_alternatePitch->hide(); @@ -2098,8 +2106,15 @@ MainWindow::closeSession() m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; + m_singingAudioMutedForTake = false; m_analysedMainModelId = {}; + // The takes of the session go with it. Their audio files are left + // where they are; deleting the ones nothing refers to any more is + // phase 6 of the takes work. + m_takes->clear(); + m_takePosition = 0; + m_analyser->fileClosed(); while (m_paneStack->getPaneCount() > 0) { @@ -2554,18 +2569,13 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal // After deletePane() the pointer would be dangling → crash. drainPendingExtraPanes(singingModelId); - // A take being recorded stays out of the mix until it is over. The - // play source happens to read ahead of what has been recorded, so the - // take is silent anyway with the buffer sizes of today; this does not - // depend on that. Not with setAudible(), which would write the state - // to the settings the reference shares. - if (deferAnalysis) { - m_singingAudioAfterTake = m_analyser2->isAudible(Analyser::Audio); - if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { - if (auto params = audio->getPlayParameters()) { - params->setPlayAudible(false); - m_singingAudioMutedForTake = true; - } + // The take's audio is the file behind this model. All of a file the + // user loaded, or one a session restored, holds recorded singing; a + // file we have just spliced ourselves has the coverage the splice + // worked out, which must not be thrown away here. + if (!m_rebuildingTakeAudio) { + if (auto wfm = ModelById::getAs(singingModelId)) { + m_takes->setWholeFileTake(wfm->getLocation(), wfm->getFrameCount()); } } @@ -2608,12 +2618,36 @@ MainWindow::drainPendingExtraPanes(sv::ModelId singingModelId) m_pendingExtraPanes.clear(); } +void +MainWindow::muteSingingAudioForTake() +{ + // The singing that is there is not heard while it is being recorded + // into: the singer would hear themselves along with the reference, + // and on speakers that goes back into the microphone. Not with + // setAudible(), which would write the state to the settings the + // reference shares; the button goes on saying what the user asked + // for, and restoreSingingAudioAfterTake() applies it afterwards. + m_singingAudioAfterTake = + (m_analyser2 ? m_analyser2->isAudible(Analyser::Audio) : true); + + if (!m_analyser2) return; + if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { + if (auto params = audio->getPlayParameters()) { + params->setPlayAudible(false); + m_singingAudioMutedForTake = true; + } + } +} + void MainWindow::restoreSingingAudioAfterTake() { if (!m_singingAudioMutedForTake) return; m_singingAudioMutedForTake = false; if (!m_analyser2) return; + // By now m_analyser2 is usually the one made for the take's new audio + // file, not the one that was muted: what the user asked for is what + // matters, not which layer it is applied to if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { if (auto params = audio->getPlayParameters()) { params->setPlayAudible(m_singingAudioAfterTake); @@ -2625,8 +2659,11 @@ MainWindow::restoreSingingAudioAfterTake() void MainWindow::teardownSingingTrackAnalyser() { - m_singingAudioMutedForTake = false; - + // m_singingAudioMutedForTake is deliberately not cleared here: the + // singing track is torn down and built again in the middle of + // finishing a take, and what the user asked Play Singing Audio for + // has to survive that. restoreSingingAudioAfterTake() clears it when + // the take is over, and closeSession() when the session goes. if (!m_analyser2) return; // removeAllLayers() removes each layer from the pane and deletes it from @@ -2853,6 +2890,81 @@ MainWindow::teardownRealtimePitchLayer() m_realtimePitchModelId = {}; } +void +MainWindow::setupRecordingLayer() +{ + // A layer of our own on the recording, so that the document holds the + // WritableWaveFileModel while the device writes to it. The singing + // analyser cannot do that job any more: it is showing the take's own + // audio, pitch and notes, which stay as they are for the duration. + // + // The layer is never shown. The recording starts at frame 0 of its + // own file, which is not where its sound belongs on the reference's + // timeline, so drawing it would put the waveform in the wrong place; + // the live dots are what the singer watches. It is muted for the + // same reason the live pitch model is (review finding 3). + // + // With this layer in place the extra pane that AddPaneCommand made is + // no longer the only thing holding the model, so record() can prune + // it at once, as loadSingingTrack() does. + if (m_recordingLayer) teardownRecordingLayer(); + + if (!m_document || m_currentRecordingModelId.isNone()) return; + if (!m_paneStack || m_paneStack->getPaneCount() < 1) return; + Pane *pane = m_paneStack->getPane(0); + if (!pane) return; + + Layer *rawLayer = m_document->createLayer(LayerFactory::Waveform); + m_recordingLayer = qobject_cast(rawLayer); + if (!m_recordingLayer) { + cerr << "MainWindow::setupRecordingLayer: failed to create the layer " + << "for the recording" << endl; + return; + } + + m_document->setModel(m_recordingLayer, m_currentRecordingModelId); + m_document->addLayerToView(pane, m_recordingLayer); + m_recordingLayer->showLayer(pane, false); + if (auto params = m_recordingLayer->getPlayParameters()) { + params->setPlayAudible(false); + } + + // The new layer is on top, where the editing tools look for the layer + // to act on: put the tracks that can be edited back there, as + // alternatePitchToggled() does + m_analyser->stackLayers(); + if (m_analyser2) m_analyser2->stackLayers(); +} + +void +MainWindow::teardownRecordingLayer() +{ + // The recording has been spliced into the take (or the take came to + // nothing): the model is not needed any more, and releasing it closes + // the file handles the recording still has open. The file itself + // stays on disk; Tony never deletes a recording. + ModelId recordingModelId = m_currentRecordingModelId; + + if (m_recordingLayer) { + if (m_document) m_document->deleteLayer(m_recordingLayer, true); + m_recordingLayer = nullptr; + } + + // The fallback in record(): with no layer of ours, the pane that + // AddPaneCommand made was kept, hidden, because its waveform layer + // was all that held the model + drainPendingExtraPanes(recordingModelId); + + // deleteLayer(force) does not fire layerInAView(false). The model + // leaves the play source when it is released, which is what has just + // happened; this makes sure of it even if something else holds it + if (m_playSource && !recordingModelId.isNone()) { + m_playSource->removeModel(recordingModelId); + } + + m_currentRecordingModelId = {}; +} + void MainWindow::record() { @@ -2867,15 +2979,15 @@ MainWindow::record() return; } - // If a reference track is already loaded, record the microphone input as - // the singing track rather than replacing the whole session. - // We do this by temporarily switching to RecordCreateAdditionalModel so + // If a reference track is already loaded, record the microphone input + // into the singing track rather than replacing the whole session. We + // do that by switching to RecordCreateAdditionalModel for the call, so // that MainWindowBase::record() adds the WritableWaveFileModel as an - // additional (non-main) model. Our modelAdded() hook will then pick it - // up and route it through setupSingingTrackAnalyser() with deferred pYIN. + // additional (non-main) model; modelAdded() takes note of it, and + // finishSingingTake() splices it into the take's audio at the end. // - // If there is no main model yet (first-time record), fall through with the - // default RecordReplaceSession behaviour. + // If there is no main model yet (first-time record), fall through with + // the default RecordReplaceSession behaviour. bool haveReference = (getMainModel() != nullptr); @@ -2884,120 +2996,66 @@ MainWindow::record() // not inherit the shift of a singing take made before it. The figures // for a singing take are computed in recordingStarted(). This is below // the early return above on purpose: a Stop must leave them alone, the - // shift is applied afterwards in analyseNow(). + // splice on Stop needs them. m_recordingLatencyFrames = 0; m_recordingStartGapEstimate = 0; m_awaitingReferenceStart = false; m_recordingStartGapMeasured = -1; if (haveReference) { - cerr << "MainWindow::record: reference track loaded — recording as singing track" << endl; - - // If a previous singing-track recording (or loaded singing file) is - // still active, discard it now before we start capturing a new one. - // teardownRealtimePitchLayer() stops any live tracker still running - // (edge case: user re-records before pYIN finished on the last one). - // teardownSingingTrackAnalyser() removes the old recording's layers - // from the pane and releases its model so the document is clean. - // m_pendingSingingModelId is cleared so the modelAdded() race guard - // doesn't block the new recording's model from being registered. + + // The recording goes into the take at the playback position. It + // has to be read before the base class call, which centres the view + // on frame 0, and while we are not recording yet: once we are, + // ViewManager reports the duration of the take instead. + sv_frame_t position = m_viewManager ? m_viewManager->getPlaybackFrame() : 0; + if (position < 0) position = 0; + + // Recording from inside singing that is already there replaces it + // from that point on. The user is asked first, unless they have + // said not to be: undo can bring it back. + if (m_takes->shouldConfirmRecordingAt(position) && + !confirmRecordingOverTake()) { + cerr << "MainWindow::record: recording over the existing singing " + << "was declined" << endl; + return; + } + + m_takePosition = position; + + cerr << "MainWindow::record: recording into the singing track from " + << "frame " << position << endl; + + // Dots and a tracker of a take whose analysis never finished if (m_realtimePitchTracker || m_realtimePitchLayer) { cerr << "MainWindow::record: tearing down leftover realtime pitch layer" << endl; teardownRealtimePitchLayer(); } - // Pre-flight orphan cleanup: delete the WaveformLayer that - // MainWindowBase::record() created via createImportedLayer() for the - // previous singing recording, and remove that model from m_playSource. - // - // This MUST be done before teardownSingingTrackAnalyser() (which calls - // removeAllLayers() and would otherwise release the singing model while - // the orphan layer still holds a reference) AND before deletePane() - // (which would destroy the extra pane and leave a dangling pointer in - // m_document->m_layerViewMap for the orphan layer — causing a crash in - // deleteLayer(true) when it tries to call removeLayer on the dead pane). - // - // At this point the extra pane is still alive (deletePane hasn't run), - // so m_layerViewMap contains a valid pane pointer, and deleteLayer(true) - // is safe. - // - // Two cases arise depending on when the user presses Record again: - // - // (A) User re-records while pYIN is still running (or before - // recordingFinishedFull() has fired): m_currentRecordingModelId - // is still set to the previous recording's WritableWaveFileModel. - // - // (B) User re-records after pYIN has completed: recordingFinishedFull() - // already cleared m_currentRecordingModelId to {}. However, - // m_analyser2 is still alive and its getMainModelId() still returns - // the previous singing model's ID (fileClosed() clears m_layers but - // NOT m_fileModel). We use that ID for the orphan scan instead. - // - // In both cases we identify orphan layers by scanning all document - // layers for any layer whose model matches the previous singing model ID, - // excluding layers that m_analyser2 owns (those are cleaned up by - // removeAllLayers() inside teardownSingingTrackAnalyser() below). - { - ModelId prevSingingModelId = m_currentRecordingModelId; - if (prevSingingModelId.isNone() && m_analyser2) { - prevSingingModelId = m_analyser2->getMainModelId(); - if (!prevSingingModelId.isNone()) { - cerr << "MainWindow::record: m_currentRecordingModelId cleared " - << "(post-pYIN re-record); using m_analyser2 model id " - << prevSingingModelId << " for orphan cleanup" << endl; - } - } - - if (m_document && !prevSingingModelId.isNone()) { - std::vector orphans; - for (Layer *layer : m_document->getLayers()) { - if (layer->getModel() != prevSingingModelId) continue; - // Skip layers owned by m_analyser2 — removeAllLayers() handles those. - bool ownedByAnalyser2 = false; - if (m_analyser2) { - for (int c = Analyser::Audio; c <= Analyser::Spectrogram; ++c) { - if (m_analyser2->getLayer(static_cast(c)) == layer) { - ownedByAnalyser2 = true; - break; - } - } - } - if (!ownedByAnalyser2) { - orphans.push_back(layer); - } - } - for (Layer *orphan : orphans) { - cerr << "MainWindow::record: deleting orphan layer " << orphan - << " referencing previous singing model " - << prevSingingModelId << endl; - m_document->deleteLayer(orphan, true); - } - // deleteLayer(true) does not fire layerInAView(false). The - // model leaves the play source when it is released, which - // is normally in the teardown below; this makes sure of it - // even if something else still holds the model. - if (m_playSource) { - m_playSource->removeModel(prevSingingModelId); - } - m_currentRecordingModelId = {}; - } + // Likewise a recording that was never spliced into the take + if (m_recordingLayer || !m_currentRecordingModelId.isNone()) { + cerr << "MainWindow::record: releasing a recording left over from " + << "a take that did not finish" << endl; + teardownRecordingLayer(); } - if (m_analyser2) { - cerr << "MainWindow::record: tearing down previous singing-track analyser" << endl; - teardownSingingTrackAnalyser(); - } + // The singing track itself stays as it is: its pitch and notes are + // what the singer is adding to, and they stay on show for the take. + // Only its audio is kept out of the mix (spec 5.1). + muteSingingAudioForTake(); + m_pendingSingingModelId = {}; m_recordingInProgress = false; m_recordingAsSingingTrack = true; - // Remember pane count so we can prune the extra pane that + // Remember the pane count so we can prune the extra pane that // MainWindowBase::record() creates via AddPaneCommand for the - // recording's waveform layer. We want both tracks in pane 0. + // recording's waveform layer. We want everything in pane 0. m_paneCountBeforeRecording = m_paneStack ? m_paneStack->getPaneCount() : 0; setAudioRecordMode(RecordCreateAdditionalModel); } else { m_recordingAsSingingTrack = false; + m_takePosition = 0; m_paneCountBeforeRecording = 0; setAudioRecordMode(RecordReplaceSession); } @@ -3014,6 +3072,15 @@ MainWindow::record() cerr << "MainWindow::record: recording did not start" << endl; m_recordingAsSingingTrack = false; m_recordingInProgress = false; + restoreSingingAudioAfterTake(); + } + + // The base class centres the view on frame 0. The take is being + // recorded where playback was, and that is where the singer is + // watching (spec section 4). + if (m_recordingAsSingingTrack && m_viewManager) { + m_viewManager->setPlaybackFrame(m_takePosition); + m_viewManager->setGlobalCentreFrame(m_takePosition); } updateAlternatePitchForTake(); @@ -3023,53 +3090,41 @@ MainWindow::record() // (after the singing track session is closed) behaves correctly. setAudioRecordMode(RecordReplaceSession); - // Remove any extra panes that AddPaneCommand created for the recording's - // waveform layer. setupSingingTrackAnalyser() will add a proper waveform - // layer for the recording into pane 0, so the auto-created extra pane is - // redundant and visually confusing. - // - // WHY WE CANNOT DELETE THE ORPHAN LAYER HERE: - // At this point m_analyser2 has NOT yet been created — its setup is deferred - // via QTimer::singleShot(0) queued inside modelAdded(). The orphan - // WaveformLayer in the extra pane is currently the ONLY layer referencing - // the WritableWaveFileModel being recorded into. Calling - // deleteLayer(orphan, true) would invoke Document::releaseModel(), which - // would destroy the live recording model mid-capture — crash. - // - // WHY WE CANNOT CALL Pane::removeLayer() + deletePane() HERE: - // Pane::removeLayer() removes the layer from the View's internal display - // list but does NOT update Document::m_layerViewMap. After deletePane() - // destroys the widget, m_layerViewMap still contains the now-dangling pane - // pointer. On the next recording attempt, the pre-flight orphan cleanup - // calls deleteLayer(orphan, true), which iterates m_layerViewMap and calls - // (*j)->removeLayer(layer) on the stale pointer — use-after-free crash. - // - // SOLUTION: use PaneStack::hidePane() to move the extra pane out of the - // visible list (so getPaneCount() drops back and the UI doesn't show it) - // while keeping the widget alive with valid m_layerViewMap entries. - // Store the pane in m_pendingExtraPanes. setupSingingTrackAnalyser() will - // drain that list once m_analyser2 is set up and its WaveformLayer holds a - // reference to the recording model, at which point deleteLayer(orphan, true) - // is safe (the model won't be freed because m_analyser2's layer still refs it) - // and deletePane() can safely destroy the now-clean pane widget. - if (m_recordingAsSingingTrack && m_paneStack) { - while (m_paneStack->getPaneCount() > m_paneCountBeforeRecording) { - Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - if (!extra) break; - - // Unregister from the overview before hiding so it stops rendering. - if (m_overview) m_overview->unregisterView(extra); - - // hidePane() moves the pane from the visible list to m_hiddenPanes, - // calls pw->hide() on the widget, and updates getPaneCount() — so - // this while loop will terminate correctly. - m_paneStack->hidePane(extra); + if (m_recordingAsSingingTrack) { - // Store for deferred cleanup in setupSingingTrackAnalyser(). - m_pendingExtraPanes.push_back(extra); + // Give the recording a layer of our own to hold it in the document, + // and then remove the extra pane that AddPaneCommand made for it. + // The order matters: that pane's imported waveform layer is the + // only thing referencing the live WritableWaveFileModel until our + // layer is there, and deleting it first would have + // Document::releaseModel() destroy the model mid-capture. + setupRecordingLayer(); + + if (m_paneStack) { + while (m_paneStack->getPaneCount() > m_paneCountBeforeRecording) { + Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); + if (!extra) break; + + if (m_recordingLayer) { + pruneExtraPane(extra, m_currentRecordingModelId); + continue; + } - cerr << "MainWindow::record: hiding extra pane " << extra - << " — deferred deletion queued for setupSingingTrackAnalyser" << endl; + // Nothing of ours holds the model, so the pane's own layer + // has to keep it alive until the take is over. Hiding the + // pane takes it out of the visible list while leaving the + // widget alive with valid Document::m_layerViewMap entries: + // a pane must not be deleted while a layer of its own is + // still in that map (see "Extra-pane Pruning" in the dev + // doc). teardownRecordingLayer() prunes it at the end. + if (m_overview) m_overview->unregisterView(extra); + m_paneStack->hidePane(extra); + m_pendingExtraPanes.push_back(extra); + + cerr << "MainWindow::record: no layer for the recording; " + << "keeping its pane " << extra << " hidden for the take" + << endl; + } } } } @@ -3110,10 +3165,11 @@ MainWindow::recordingStarted() setupRealtimePitchLayer(); // If the "play reference while recording" toggle is on, start - // playback from frame 0 so the singer hears the reference track. - // The audio IO was already resumed by record() so m_playSource - // can be started directly without calling MainWindowBase::play() - // (which would stop recording if isRecording() is true). + // playback from where the take is being recorded, so the singer + // hears the reference from there. The audio IO was already + // resumed by record() so m_playSource can be started directly + // without calling MainWindowBase::play() (which would stop + // recording if isRecording() is true). if (m_recordingAsSingingTrack && m_playRefWhileRecording && m_playRefWhileRecording->isChecked() && m_playSource && !m_playSource->isPlaying()) { @@ -3123,9 +3179,10 @@ MainWindow::recordingStarted() // singing recording's timeline after the take. // output latency = time from play() call until audio exits the speaker // input latency = time from sound entering the mic until it arrives here - // The singer's response to reference frame 0 arrives in the recording - // at approximately frame (outputLatency + inputLatency), so we will - // shift the model's start frame by -(outputLatency + inputLatency). + // The singer's response to the reference at m_takePosition arrives + // in the recording at approximately frame + // (outputLatency + inputLatency), so that is the frame the splice + // reads the recording from. sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; // @@ -3150,8 +3207,8 @@ MainWindow::recordingStarted() m_recordingStartGapMeasured = -1; m_awaitingReferenceStart = true; - m_viewManager->setPlaybackFrame(0); - m_playSource->play(0); + m_viewManager->setPlaybackFrame(m_takePosition); + m_playSource->play(m_takePosition); } updateLayerStatuses(); @@ -3182,11 +3239,12 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) // over: leave them, and the status bar, alone. if (!m_recordingInProgress) return; - // Draw the dot where the finished pitch track will put this sound: - // the take is going to be shifted earlier by the recording latency. - // The first dots may have been placed using the estimate of the - // start gap. They all belong to sound from before the reference - // started, which has no place on the reference's timeline. + // Draw the dot where the finished pitch track will put this sound: the + // take is spliced into the singing track from m_takePosition on, with + // the recording latency taken off its front. The first dots may have + // been placed using the estimate of the start gap. They all belong to + // sound from before the reference started, which has no place on the + // reference's timeline. sv_frame_t latencyBefore = m_recordingLatencyFrames; refineRecordingLatency(); auto m = ModelById::getAs(m_realtimePitchModelId); @@ -3194,8 +3252,9 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) for (const Event &e : m->getAllEvents()) m->remove(e); } - sv_frame_t dotFrame = compensatedLiveFrame(frame, m_recordingLatencyFrames); - if (dotFrame < 0) return; + sv_frame_t intoTake = compensatedLiveFrame(frame, m_recordingLatencyFrames); + if (intoTake < 0) return; + sv_frame_t dotFrame = m_takePosition + intoTake; if (m) { m->add(Event(dotFrame, float(hz), tr(""))); @@ -3275,6 +3334,144 @@ MainWindow::recordingFinishedFull(Analyser *analysing) updateMenuStates(); } +void +MainWindow::finishSingingTake() +{ + // The take has stopped, and what the device recorded is raw material: + // it goes into the take's audio file at the position the take was + // started from, with the latency taken off its front, and the singing + // track is then rebuilt from the file that comes out of that. + // + // (Rebuilt means analysed in full, as a singing track loaded from a + // file is. Phase 4 of the takes work puts the new audio under the + // pitch and notes layers that are there and analyses only the range + // that changed, which is what makes this quick on a long song.) + + refineRecordingLatency(); + + sv_frame_t latency = m_recordingLatencyFrames; + sv_frame_t position = m_takePosition; + + QString recordingPath; + sv_frame_t recorded = 0; + if (auto wfm = ModelById::getAs + (m_currentRecordingModelId)) { + recordingPath = wfm->getLocation(); + recorded = wfm->getFrameCount(); + } + + // The take is over: the tracker goes first, so that the recording's + // model is never released under it, and then the model itself. + // AudioCallbackRecordTarget::stopRecording() has flushed the buffers + // and called writeComplete(), and releasing the model closes what is + // left, so the file is whole before the splice reads it. + stopRealtimePitchTracker(); + m_recordingInProgress = false; + m_recordingAsSingingTrack = false; + teardownRecordingLayer(); + + // The playhead goes back to where the take started: Play then hears + // what was just sung, and Record again records the same part. (While + // recording, ViewManager keeps the playback frame at the duration of + // the take, so it is somewhere else by now.) + if (m_viewManager) m_viewManager->setPlaybackFrame(position); + + // A take stopped the moment it was started, or one no longer than the + // latency, has nothing in it to add. Nothing has gone wrong; there is + // simply nothing to do + if (recordingPath != "" && recorded <= latency) { + cerr << "MainWindow::finishSingingTake: nothing to use: " << recorded + << " frames recorded, the first " << latency + << " of which are the latency" << endl; + recordingFinishedFull(nullptr); + return; + } + + QString error; + Coverage::Range placed; + + if (recordingPath == "") { + error = tr("The recording is no longer there to be used"); + } else { + QString directory = RecordDirectory::getRecordDirectory(); + if (directory == "") { + error = tr("Could not find a directory to write the singing " + "track into"); + } else { + // Everything before the latency is sound from before the singer + // could have heard the reference at the take's position + error = m_takes->spliceRecording(recordingPath, latency, position, + -1, directory, &placed); + } + } + + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to add the recording to the singing track"), + tr("The recording could not be added to the singing track" + "

%1

").arg(error), + QMessageBox::Ok); + // The singing track is as it was, and the recording is still in + // the record directory; there is nothing for the dots to wait for + recordingFinishedFull(nullptr); + return; + } + + cerr << "MainWindow::finishSingingTake: " << recorded << " frames " + << "recorded, used from frame " << latency << ", placed at [" + << placed.start << "," << placed.end << ") of " + << m_takes->getAudioPath() << endl; + + bool rebuilt = rebuildSingingTrackFromTake(); + + // The dots stay until the analysis that replaces them is done + recordingFinishedFull(rebuilt ? m_analyser2 : nullptr); +} + +bool +MainWindow::rebuildSingingTrackFromTake() +{ + if (!m_takes->haveTake()) return false; + + Analyser *before = m_analyser2; + + // The same path as File -> Load Singing Track, but the coverage of + // this file is the one the splice worked out, not "all of it" + m_rebuildingTakeAudio = true; + loadSingingTrack(m_takes->getAudioPath()); + m_rebuildingTakeAudio = false; + + return (m_analyser2 != nullptr && m_analyser2 != before); +} + +bool +MainWindow::confirmRecordingOverTake() +{ + QMessageBox box(this); + box.setIcon(QMessageBox::Question); + box.setWindowTitle(tr("Record over the existing singing?")); + box.setText(tr("Record over the existing singing from here?")); + box.setInformativeText + (tr("There is singing recorded from this point on. Recording from " + "here replaces it.")); + box.setStandardButtons(QMessageBox::Yes | QMessageBox::No); + box.setDefaultButton(QMessageBox::Yes); + + QCheckBox *dontAsk = new QCheckBox(tr("Don't ask again"), &box); + box.setCheckBox(dontAsk); + + bool yes = (box.exec() == QMessageBox::Yes); + + // "Don't ask again" means "always go ahead", so it is only taken as + // an answer when this one was yes + if (yes && dontAsk->isChecked()) { + SingingTakes::setOverwriteConfirmationWanted(false); + } + + return yes; +} + void MainWindow::openLocation() { @@ -4497,6 +4694,18 @@ MainWindow::modelAdded(ModelId model) return; } + // A recording being made into the singing track is raw material, + // not the singing track itself: record() gives it a layer of its + // own and finishSingingTake() splices it in at the end. The + // singing analyser is left with the take's own audio, whose pitch + // and notes stay on show for the duration. + if (m_recordingAsSingingTrack && model != getMainModelId()) { + m_currentRecordingModelId = model; + cerr << "modelAdded: the recording of the take is model " + << model << endl; + return; + } + // If there is already a main model and this is a new additional // audio model (not the realtime pitch model), treat it as the // singing track to be analysed with the secondary colour scheme. @@ -4509,30 +4718,13 @@ MainWindow::modelAdded(ModelId model) if (m_pendingSingingModelId.isNone()) { m_pendingSingingModelId = model; - if (m_recordingAsSingingTrack) { - // The model is a WritableWaveFileModel still being - // recorded into. Set up m_analyser2 with waveform/ - // visualisation layers but defer pYIN until recording - // finishes (analyseNow() will call analyseExistingFile()). - // Also store the model ID so setupRealtimePitchLayer() - // can target this exact model rather than scanning all - // document models (which would wrongly pick up a previous - // recording's WritableWaveFileModel that is still - // registered because its orphan waveform layer prevents - // releaseModel() from freeing it). - m_currentRecordingModelId = model; - QTimer::singleShot(0, this, [this, model]() { - m_pendingSingingModelId = {}; - setupSingingTrackAnalyser(model, /*deferAnalysis=*/true); - }); - } else { - // Normal case: a finished audio file was loaded as a - // singing track. loadSingingTrack() runs the analysis - // itself as soon as openPath() returns (it must happen - // before the extra pane is pruned); this deferred call - // is the fallback for any other route that adds a model. - QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); - } + // An audio file has been loaded as the singing track: the + // take's own audio, or one the user chose. + // loadSingingTrack() runs the analysis itself as soon as + // openPath() returns (it has to happen before the extra + // pane is pruned); this deferred call is the fallback for + // any other route that adds a model. + QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); } else { cerr << "modelAdded: m_pendingSingingModelId already set, ignoring model " << model << endl; @@ -4580,79 +4772,14 @@ MainWindow::analyseNow() return; } - // When the user recorded a singing track alongside an existing reference - // track (RecordCreateAdditionalModel mode), the recording becomes an - // additional model, not the main model. In that case we must route - // analysis through m_analyser2 (which was set up by setupSingingTrackAnalyser - // via modelAdded() when the WritableWaveFileModel was registered). - // We must NOT re-analyse the primary reference track here. + // A take that has just stopped is not analysed where it is: the + // recording is raw material, to be spliced into the take's audio at + // the position it was started from. finishSingingTake() does that, + // and rebuilds the singing track from the file that comes out. if (m_recordingAsSingingTrack) { - cerr << "analyseNow: recording was singing track — routing to m_analyser2" << endl; - - // Apply round-trip latency compensation: shift the singing model's - // global start frame backward by the round-trip hardware latency so - // the singer's audio (which arrives late due to output + input latency) - // aligns with the reference during playback. This must happen before - // pYIN analysis so that all derived layers (pitch, notes) inherit the - // same timeline offset. Only applied when reference playback was - // active during the recording (m_recordingLatencyFrames > 0). - refineRecordingLatency(); - if (m_recordingLatencyFrames > 0 && !m_currentRecordingModelId.isNone()) { - auto wfm = ModelById::getAs(m_currentRecordingModelId); - if (wfm) { - cerr << "analyseNow: applying latency compensation: setStartFrame(" - << -m_recordingLatencyFrames << ")" << endl; - wfm->setStartFrame(-m_recordingLatencyFrames); - } - } - - // The realtime pitch layer stays until the full pYIN analysis of - // the singing recording (via m_analyser2) is there to replace it, - // or goes at once if that analysis could not be started. - bool wasLive = (m_realtimePitchTracker || m_realtimePitchLayer); - - auto analyseSingingTrack = [this]() -> bool { - CommandHistory::getInstance()->startCompoundOperation - (tr("Analyse Singing Track"), true); - - QString error = m_analyser2->analyseExistingFile(); - - CommandHistory::getInstance()->endCompoundOperation(); - - if (error != "") { - QMessageBox::warning - (this, - tr("Failed to analyse singing track"), - tr("Analysis failed

%1

").arg(error), - QMessageBox::Ok); - return false; - } - return true; - }; - - if (m_analyser2) { - bool ok = analyseSingingTrack(); - if (wasLive) recordingFinishedFull(ok ? m_analyser2 : nullptr); - } else { - // m_analyser2 may still be pending setup (modelAdded fires async). - // Defer the analysis until the secondary analyser is ready. - cerr << "analyseNow: m_analyser2 not ready yet, deferring singing-track analysis" << endl; - if (wasLive) { - // The take is over either way; the dots wait for the - // deferred analysis - stopRealtimePitchTracker(); - m_recordingInProgress = false; - } - QTimer::singleShot(200, this, [this, wasLive, analyseSingingTrack]() { - bool ok = false; - if (m_analyser2) { - ok = analyseSingingTrack(); - } else { - cerr << "analyseNow (deferred): m_analyser2 still null, singing-track analysis skipped" << endl; - } - if (wasLive) recordingFinishedFull(ok ? m_analyser2 : nullptr); - }); - } + cerr << "analyseNow: the take that has just stopped goes into the " + << "singing track" << endl; + finishSingingTake(); return; } diff --git a/main/MainWindow.h b/main/MainWindow.h index 624deec6..28d317c5 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -20,6 +20,7 @@ #include "Analyser.h" #include "RealtimePitchTracker.h" #include "AlternatePitchTrack.h" +#include "SingingTakes.h" #include #include @@ -210,6 +211,7 @@ protected slots: virtual void recordingStarted(); virtual void onRealtimePitchDetected(sv::sv_frame_t frame, double hz); virtual void recordingFinishedFull(Analyser *analysing = nullptr); + virtual void finishSingingTake(); void moveOneNoteRight(); void moveOneNoteLeft(); @@ -260,6 +262,33 @@ protected slots: void stepAlternatePitch(bool up); void updateAlternatePitchForTake(); + // The singing takes of the session: the audio file of each take and + // the ranges of it that hold recorded singing. MainWindow only + // wires it: it decides where a recording goes and writes the files. + SingingTakes *m_takes; + + // Where on the reference's timeline the take being recorded, or the + // one most recently recorded, starts: the playback position when + // Record was pressed. The live dots are drawn from here, and this + // is where the recording is spliced into the take's audio. + sv::sv_frame_t m_takePosition; + + // Ask before recording over singing that is already there, unless + // the user has said not to. Overridden by the tests, which cannot + // answer a dialog. Returns true to go ahead with the recording. + virtual bool confirmRecordingOverTake(); + + // Rebuild the singing track from the take's audio file, the way + // Load Singing Track does: the layers of the file before go, the new + // file is opened and analysed in full. Returns true if a new + // analyser was set up. (Phase 4 of the takes work replaces this + // with a model swap that keeps the pitch and notes layers.) + bool rebuildSingingTrackFromTake(); + + // Keep the take's existing audio out of the mix while it is being + // recorded into, and put it back afterwards + void muteSingingAudioForTake(); + // Background music track: an additional audio file that plays alongside // the reference track but is never analysed. The toggle enables/disables // mixing during both normal playback and recording. @@ -334,9 +363,10 @@ protected slots: virtual void setupHelpMenu(); virtual void setupToolbars(); - // Helpers for the singing / second-track workflow - // deferAnalysis=true skips pYIN (used when the model is a - // WritableWaveFileModel still being recorded into). + // Helpers for the singing / second-track workflow. + // deferAnalysis=true skips pYIN; no caller needs that since a take is + // spliced into a finished audio file before it is analysed, but the + // model swap of phase 4 will. virtual void setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis = false); virtual void teardownSingingTrackAnalyser(); @@ -344,6 +374,15 @@ protected slots: virtual void teardownRealtimePitchLayer(); virtual void stopRealtimePitchTracker(); + // The raw recording of a take needs a layer of its own to hold it in + // the document: the singing analyser is busy with the take's audio, + // which stays on show while the recording is made. The layer is + // never shown and never heard; it goes when the recording has been + // spliced into the take, which releases the model and with it the + // file handles of the recording. + void setupRecordingLayer(); + void teardownRecordingLayer(); + // Background music helpers: load/tear-down a non-analysed audio track // that plays alongside the reference track. void loadBackgroundMusic(QString path); @@ -368,11 +407,13 @@ protected slots: // pick it up on the next event-loop iteration. sv::ModelId m_pendingSingingModelId; - // A take is kept out of the playback mix while it is being recorded: - // with the reference playing, the singer would otherwise hear - // themselves late, and on speakers that goes back into the microphone. + // The singing track is kept out of the playback mix while a take is + // being recorded into it: with the reference playing, the singer + // would otherwise hear their earlier singing along with it, and on + // speakers that goes back into the microphone. The recording itself + // is silent for the same reason (setupRecordingLayer()). // m_singingAudioAfterTake is what Play Singing Audio asks for, and - // what the take is given when the recording is over. + // what the rebuilt singing track is given when the take is over. bool m_singingAudioMutedForTake; bool m_singingAudioAfterTake; void restoreSingingAudioAfterTake(); @@ -402,23 +443,32 @@ protected slots: // The WritableWaveFileModel being recorded into in the current (or most // recent) singing-track recording. Set in modelAdded() when - // m_recordingAsSingingTrack is true, cleared in closeSession() and - // recordingFinishedFull(). Used by setupRealtimePitchLayer() to - // identify the correct audio source model without scanning all document - // models — a scan would incorrectly pick up a previous recording's - // WritableWaveFileModel that is still registered in the document because - // its orphan waveform layer (view-detached but still in m_document's - // layer list) holds a reference that prevents releaseModel() from - // freeing it. + // m_recordingAsSingingTrack is true, cleared by teardownRecordingLayer() + // when the recording has been spliced into the take, and in + // closeSession(). Used by setupRealtimePitchLayer() to identify the + // correct audio source model without scanning all document models — a + // scan would incorrectly pick up a previous recording's + // WritableWaveFileModel that is still registered in the document. sv::ModelId m_currentRecordingModelId; + // The layer that holds the raw recording in the document while it is + // being recorded into; see setupRecordingLayer(). + sv::WaveformLayer *m_recordingLayer; + + // True while the singing track is being rebuilt from an audio file + // that we have just spliced ourselves, so that the take's coverage — + // which the splice worked out — is not replaced by "the whole file", + // as it is for a file the user loads or a session restores. + bool m_rebuildingTakeAudio; + // Round-trip hardware latency (output + input, in frames at the model // sample rate) stored when a singing-track recording is made with the - // "play reference while recording" toggle on. Applied as a negative - // start-frame offset to the singing model so its timeline aligns with - // the reference during playback, and to the live dots during the take. + // "play reference while recording" toggle on. The recording is read + // from this frame on when it is spliced into the take's audio, so that + // what the singer sang in answer to the reference at m_takePosition + // lands there; and the live dots are placed with it during the take. // Reset to 0 in record() at the start of every take, standalone ones - // included, but not by a Stop: analyseNow() applies it after that. + // included, but not by a Stop: the splice needs it after that. // // It also includes the start gap: the part of the take recorded before // the reference began to play. That starts out as an estimate made just diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 844a4deb..dbe0ff94 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -27,6 +27,7 @@ #include "../MainWindow.h" #include "../Analyser.h" +#include "../SingingTakes.h" #include "version.h" @@ -43,6 +44,8 @@ #include "audio/AudioCallbackRecordTarget.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" #include "base/PlayParameters.h" #include "base/RecordDirectory.h" @@ -104,6 +107,20 @@ class TestMainWindow : public MainWindow sv::TimeValueLayer *realtimeLayer() { return m_realtimePitchLayer; } sv::ModelId realtimeModelId() { return m_realtimePitchModelId; } sv::ModelId currentRecordingModelId() { return m_currentRecordingModelId; } + sv::WaveformLayer *recordingLayer() { return m_recordingLayer; } + SingingTakes *takes() { return m_takes; } + sv::sv_frame_t takePosition() { return m_takePosition; } + + void seekTo(sv::sv_frame_t frame) { + m_viewManager->setPlaybackFrame(frame); + } + + // The question about recording over singing that is there is answered + // from here: the suite cannot answer a dialog + void setRecordOverAnswer(bool yes) { m_recordOverAnswer = yes; } + int recordOverQuestions() const { return m_recordOverQuestions; } + void clearRecordOverQuestions() { m_recordOverQuestions = 0; } + sv::ModelId pendingSingingModelId() { return m_pendingSingingModelId; } sv::ModelId backgroundMusicModelId() { return m_backgroundMusicModelId; } sv::WaveformLayer *backgroundMusicLayer() { return m_backgroundMusicLayer; } @@ -140,12 +157,19 @@ class TestMainWindow : public MainWindow m_playSource->setSystemPlaybackTarget(m_audioIO); } + bool confirmRecordingOverTake() override { + ++m_recordOverQuestions; + return m_recordOverAnswer; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well private: FakeAudioIO::Config m_fakeConfig; bool m_installDevice; + bool m_recordOverAnswer = true; + int m_recordOverQuestions = 0; }; class TestRecordWorkflow : public QObject @@ -253,6 +277,40 @@ class TestRecordWorkflow : public QObject : sv::EventVector(); } + static sv::EventVector eventsBetween(const sv::EventVector &events, + sv::sv_frame_t from, + sv::sv_frame_t to) { + sv::EventVector result; + for (const auto &e : events) { + if (e.getFrame() >= from && e.getFrame() < to) { + result.push_back(e); + } + } + return result; + } + + // The audio of the take: one file, starting at frame 0 of the + // reference's timeline, silent where nothing has been recorded + std::shared_ptr takeAudio() { + Analyser *a2 = m_window->analyser2(); + if (!a2) return nullptr; + return sv::ModelById::getAs(a2->getMainModelId()); + } + + // What is in the take's audio file itself over [from, to), rather + // than what a model makes of it + double takeAudioRms(sv::sv_frame_t from, sv::sv_frame_t to) { + QString path = m_window->takes()->getAudioPath(); + if (path.isEmpty() || to <= from) return -1.0; + sv::WavFileReader reader { sv::FileSource(path) }; + if (!reader.isOK()) return -1.0; + auto data = reader.getInterleavedFrames(from, to - from); + if (data.empty()) return -1.0; + double sum = 0.0; + for (float v : data) sum += double(v) * double(v); + return std::sqrt(sum / double(data.size())); + } + static double medianHz(const sv::EventVector &events) { std::vector values; for (const auto &e : events) values.push_back(e.getValue()); @@ -437,6 +495,10 @@ private slots: settings.beginGroup("Analyser"); settings.remove(""); settings.endGroup(); + + // Asking before recording over existing singing is the default, + // whatever a test that switched it off did + SingingTakes::setOverwriteConfirmationWanted(true); } void cleanup() { @@ -482,10 +544,28 @@ private slots: QVERIFY(a2); sv::ModelId singing = a2->getMainModelId(); QVERIFY(singing != m_window->mainModelId()); - auto wave = sv::ModelById::getAs(singing); - QVERIFY2(wave, "the singing model is not a WritableWaveFileModel"); + + // The singing track is the take's own audio file, which the + // recording was spliced into: a plain wave file starting at frame + // 0, not the recording itself + auto wave = sv::ModelById::getAs(singing); + QVERIFY2(wave, "the singing model is not a wave file model"); + QVERIFY2(!sv::ModelById::isa(singing), + "the singing model is the recording, not the take's audio"); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); QVERIFY(wave->getFrameCount() > sv::sv_frame_t(0.8 * rate)); + // and the recording has been let go of + QVERIFY(!m_window->recordingLayer()); + QVERIFY(m_window->currentRecordingModelId().isNone()); + + // The take covers what was recorded, from frame 0 here + QVERIFY(m_window->takes()->haveTake()); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, sv::sv_frame_t(0)); + QCOMPARE(ranges[0].end, wave->getFrameCount()); + QCOMPARE(colourOf(a2->getLayer(Analyser::PitchTrack)), colourNamed("Orange")); QCOMPARE(colourOf(a2->getLayer(Analyser::Notes)), @@ -730,18 +810,21 @@ private slots: QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, "the reference was never played"); - auto wave = sv::ModelById::getAs - (m_window->analyser2()->getMainModelId()); + auto wave = takeAudio(); QVERIFY(wave); + // The compensation is in the audio: the recording was read from + // the frame the latency points at, so the take's own file starts + // at frame 0 of the reference's timeline + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); - // The shift is the round trip plus what the application took - // the start gap to be (review finding 14). The device knows + // The compensation is the round trip plus what the application + // took the start gap to be (review finding 14). The device knows // what the gap really was. The application counts in whole // blocks, in the audio callback, so the two agree to within a // few samples; an estimate made on the GUI thread is out by a // block or more sv::sv_frame_t gap = m_window->fake()->getFramesBeforePlayStart(); - sv::sv_frame_t assumedGap = -wave->getStartFrame() - K; + sv::sv_frame_t assumedGap = m_window->recordingLatencyFrames() - K; QVERIFY2(std::llabs(assumedGap - gap) <= 16, qPrintable(QString("the application took the start gap to " "be %1 frames; it was %2") @@ -778,11 +861,298 @@ private slots: if (QTest::currentTestFailed()) return; QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(0)); - auto wave = sv::ModelById::getAs - (m_window->analyser2()->getMainModelId()); + auto wave = takeAudio(); QVERIFY(wave); QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); QCOMPARE(m_window->fake()->getPlayStartFrame(), -1L); + // Nothing to compensate for, so the take is where it was recorded + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, sv::sv_frame_t(0)); + } + + // Record starts the take at the playback position: what is sung lands + // there on the reference's timeline, and the take's audio file is + // silence up to it + void take_at_playback_position() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + m_window->seekTo(P); + take(1000); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePosition(), P); + QCOMPARE(m_window->recordOverQuestions(), 0); + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); + QVERIFY2(wave->getFrameCount() > P + sv::sv_frame_t(0.7 * rate) && + wave->getFrameCount() < P + sv::sv_frame_t(1.6 * rate), + qPrintable(QString("the take's audio is %1 frames long; a " + "second recorded at frame %2 should make " + "it about %3") + .arg(wave->getFrameCount()).arg(P) + .arg(P + sv::sv_frame_t(rate)))); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + QCOMPARE(ranges[0].end, wave->getFrameCount()); + + // The file holds the singing where it was sung and nothing before + const sv::sv_frame_t margin = sv::sv_frame_t(0.1 * rate); + QVERIFY2(takeAudioRms(0, P - margin) < 0.001, + "the take's audio is not silent before the position it was " + "recorded at"); + QVERIFY(takeAudioRms(P + margin, ranges[0].end - margin) > 0.05); + + // and so does its pitch track + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + QVERIFY2(std::llabs(events.front().getFrame() - P) < margin, + qPrintable(QString("the take's pitch track starts at frame " + "%1; it was recorded from frame %2") + .arg(events.front().getFrame()).arg(P))); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(events), highHz)) < 10.0); + + // and the reference is where it was + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + } + + // The latency is taken off the front of the recording as it is + // spliced in, rather than carried by the model's start frame. The + // device reports a round trip of K frames and delivers a singer who + // is exactly that late, singing a low note and then a high one; the + // step between them must land 0.75 s after the take's position. + void take_latency_removed_by_splice() { + const int K = 3 * 4096; + FakeAudioIO::Config config; + config.playbackLatency = 2 * 4096; + config.recordLatency = 4096; + config.input = melody(0.75); + config.inputDelay = K; + config.inputFollowsPlayback = true; + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(1.5 * rate); + m_window->seekTo(P); + take(2200); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, + "the reference was never played"); + QVERIFY(m_window->recordingLatencyFrames() >= K); + + auto wave = takeAudio(); + QVERIFY(wave); + QVERIFY2(wave->getStartFrame() == sv::sv_frame_t(0), + "the take's audio is shifted for the latency instead of " + "being spliced with it taken off"); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + + auto events = pitchEvents(m_window->analyser2()); + sv::sv_frame_t step = stepFrame(events); + sv::sv_frame_t want = P + sv::sv_frame_t(0.75 * rate); + QVERIFY2(step > 0, "the take never reached the second note"); + sv::sv_frame_t error = step - want; + QVERIFY2(std::llabs(error) <= 4 * hop, + qPrintable(QString("the sung step is at frame %1, %2 frames " + "(%3 ms) from where it was sung (%4); the " + "round trip is %5 frames and the " + "application compensated %6") + .arg(step).arg(error) + .arg(1000.0 * double(error) / rate, 0, 'f', 1) + .arg(want).arg(K) + .arg(m_window->recordingLatencyFrames()))); + + QVERIFY(!events.empty()); + QVERIFY2(events.front().getFrame() >= P - 4 * hop, + qPrintable(QString("the take's pitch track starts at frame " + "%1, before the position %2 it was " + "recorded from") + .arg(events.front().getFrame()).arg(P))); + } + + // Recording into the middle of a take replaces what is there from + // that point on and leaves the rest. The singer sings a low note and + // then a high one; recording the melody again from inside the high + // note leaves the high note only where the second recording did not + // reach. + void take_over_existing_replaces_it() { + FakeAudioIO::Config config; + config.input = melody(0.75); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(1600); + if (QTest::currentTestFailed()) return; + sv::sv_frame_t step = stepFrame(pitchEvents(m_window->analyser2())); + QVERIFY2(step > 0 && std::llabs(step - sv::sv_frame_t(0.75 * rate)) < + sv::sv_frame_t(0.1 * rate), + qPrintable(QString("the first recording's step to the high " + "note is at frame %1, not about %2") + .arg(step).arg(sv::sv_frame_t(0.75 * rate)))); + QString before = m_window->takes()->getAudioPath(); + sv::sv_frame_t firstEnd = + m_window->takes()->getCoverage().getEndFrame(); + + const sv::sv_frame_t P = sv::sv_frame_t(1.1 * rate); + m_window->seekTo(P); + m_window->clearRecordOverQuestions(); + take(700); + if (QTest::currentTestFailed()) return; + + // The playhead was inside the singing, so the user was asked + QCOMPARE(m_window->recordOverQuestions(), 1); + + // One range still, and it reaches further than the first take did + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, sv::sv_frame_t(0)); + QVERIFY(ranges[0].end >= P + sv::sv_frame_t(0.6 * rate)); + QVERIFY(ranges[0].end > firstEnd); + + // A new file, with the one before kept for undo + QVERIFY(m_window->takes()->getAudioPath() != before); + QCOMPARE(m_window->takes()->getSupersededPaths(), + QStringList { before }); + QVERIFY2(QFileInfo::exists(before), + "the audio file the take had before was not kept"); + + auto events = pitchEvents(m_window->analyser2()); + double mid = std::sqrt(lowHz * highHz); + + auto opening = eventsBetween(events, sv::sv_frame_t(0.1 * rate), + sv::sv_frame_t(0.6 * rate)); + QVERIFY(!opening.empty()); + QVERIFY2(medianHz(opening) < mid, + "the low note the first recording opened with is gone"); + + auto kept = eventsBetween(events, sv::sv_frame_t(0.85 * rate), + sv::sv_frame_t(1.05 * rate)); + QVERIFY(!kept.empty()); + QVERIFY2(medianHz(kept) > mid, + "the high note is gone from before the second recording, " + "which should not have touched it"); + + auto replaced = eventsBetween(events, P + sv::sv_frame_t(0.1 * rate), + P + sv::sv_frame_t(0.6 * rate)); + QVERIFY(!replaced.empty()); + QVERIFY2(medianHz(replaced) < mid, + "the high note is still there where the second recording " + "sang a low one over it"); + } + + // A recording in a gap leaves the gap a gap: the file grows to hold + // it, with silence in between + void take_in_a_gap_grows_the_file() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + sv::sv_frame_t firstEnd = + m_window->takes()->getCoverage().getEndFrame(); + QVERIFY(firstEnd > sv::sv_frame_t(0.6 * rate)); + + const sv::sv_frame_t P = sv::sv_frame_t(2.5 * rate); + m_window->seekTo(P); + m_window->clearRecordOverQuestions(); + take(800); + if (QTest::currentTestFailed()) return; + + // Recording in a gap takes nothing away, so nothing is asked + QCOMPARE(m_window->recordOverQuestions(), 0); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + QCOMPARE(ranges[0], Coverage::Range(0, firstEnd)); + QCOMPARE(ranges[1].start, P); + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(wave->getFrameCount(), ranges[1].end); + + const sv::sv_frame_t margin = sv::sv_frame_t(0.1 * rate); + QVERIFY(takeAudioRms(margin, firstEnd - margin) > 0.05); + QVERIFY2(takeAudioRms(firstEnd + margin, P - margin) < 0.001, + "the gap between the two recordings is not silent"); + QVERIFY(takeAudioRms(P + margin, ranges[1].end - margin) > 0.05); + + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!eventsBetween(events, 0, firstEnd).empty()); + QVERIFY(!eventsBetween(events, P, ranges[1].end).empty()); + QVERIFY2(eventsBetween(events, firstEnd + margin, P - margin).empty(), + "the pitch track has something in the gap between the two " + "recordings"); + } + + // The question asked before recording over singing that is there, and + // what the answer does. The dialog itself is not shown here: + // TestMainWindow answers it (the real one has "Don't ask again"). + void record_over_existing_question() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + // There was nothing to record over + QCOMPARE(m_window->recordOverQuestions(), 0); + sv::sv_frame_t end = m_window->takes()->getCoverage().getEndFrame(); + + // In a gap after it: nothing asked + m_window->seekTo(end + sv::sv_frame_t(1.0 * rate)); + take(400); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->recordOverQuestions(), 0); + + // Inside it, answered no: no recording, and nothing changed + QString audio = m_window->takes()->getAudioPath(); + m_window->setRecordOverAnswer(false); + m_window->seekTo(sv::sv_frame_t(0.2 * rate)); + m_window->doRecord(); + QCOMPARE(m_window->recordOverQuestions(), 1); + QVERIFY2(!m_window->recordTarget()->isRecording(), + "the recording started although the question was answered " + "with no"); + QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY(!m_window->realtimeLayer()); + QCOMPARE(m_window->takes()->getAudioPath(), audio); + + // Answered yes: it records + m_window->setRecordOverAnswer(true); + take(400); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->recordOverQuestions(), 2); + QVERIFY(m_window->takes()->getAudioPath() != audio); + + // "Don't ask again", and it is not asked + SingingTakes::setOverwriteConfirmationWanted(false); + m_window->seekTo(sv::sv_frame_t(0.2 * rate)); + take(400); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->recordOverQuestions(), 2); } // The take and the live pitch model are both in the play source @@ -825,16 +1195,17 @@ private slots: // Review finding 3. The test above finds the take silent in the // output, but that much is true even unmuted, because the play - // source reads ahead of what has been recorded. Neither the take - // nor the live pitch model is to be audible during the take, - // whatever the buffers do: this test checks the play parameters, - // and the next one the output. + // source reads ahead of what has been recorded. Nothing of the take + // is to be audible while it is being recorded, whatever the buffers + // do: neither the recording, nor the live pitch model, nor the + // singing that is already there. This test checks the play + // parameters, and the next one the output. void take_muted_while_recording() { FakeAudioIO::Config config; config.input = tone(highHz, 4.0); makeWindow(config); m_window->setPlayReferenceWhileRecording(true); - openReference(writeWav(tone(lowHz, 1.0))); + openReference(writeWav(tone(lowHz, 2.0))); if (QTest::currentTestFailed()) return; auto takeParams = [this]() { @@ -842,19 +1213,22 @@ private slots: ->getPlayParameters(); }; + // The first recording: there is no singing track yet, only the + // recording itself and the dots startTake(); if (QTest::currentTestFailed()) return; - QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && - m_window->analyser2()->getLayer - (Analyser::Audio), 2000); - QVERIFY(m_window->realtimeLayer()); + QVERIFY2(m_window->recordingLayer(), + "the recording has no layer to hold it in the document"); + auto recordingParams = + m_window->recordingLayer()->getPlayParameters(); + QVERIFY(recordingParams); + QVERIFY2(!recordingParams->isPlayAudible(), + "the recording is audible while it is being made"); + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); auto liveParams = m_window->realtimeLayer()->getPlayParameters(); QVERIFY(liveParams); QVERIFY2(!liveParams->isPlayAudible(), "the live pitch model is audible during the take"); - QVERIFY(takeParams()); - QVERIFY2(!takeParams()->isPlayAudible(), - "the take is audible while it is being recorded"); // The button goes on saying what the user asked for QVERIFY(m_window->playSingingAudioAction()->isChecked()); @@ -862,16 +1236,28 @@ private slots: QTest::qWait(800); stopTake(); if (QTest::currentTestFailed()) return; + QVERIFY2(!m_window->recordingLayer(), + "the recording was still held after it had been spliced in"); QVERIFY2(takeParams()->isPlayAudible(), "the take was left muted after recording"); QVERIFY(m_window->playSingingAudioAction()->isChecked()); - // Switched off during a take, it stays muted afterwards + // Recording into the take that is now there: its pitch and notes + // stay on show, and its audio is kept out of the mix + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + startTake(); if (QTest::currentTestFailed()) return; - QTRY_VERIFY_WITH_TIMEOUT(m_window->analyser2() && - m_window->analyser2()->getLayer - (Analyser::Audio), 2000); + QVERIFY2(m_window->analyser2(), + "the singing track was torn down for the take"); + QVERIFY2(pitchEvents(m_window->analyser2()).size() == events.size(), + "the singing pitch track did not stay on show for the take"); + QVERIFY2(!takeParams()->isPlayAudible(), + "the singing that is there is audible while it is being " + "recorded into"); + + // Switched off during a take, it stays muted afterwards m_window->playSingingAudioAction()->trigger(); QVERIFY(!m_window->playSingingAudioAction()->isChecked()); QVERIFY(!takeParams()->isPlayAudible()); @@ -1198,6 +1584,14 @@ private slots: // The second pass used to end by marking the document unmodified QVERIFY(m_window->isDocumentModified()); + + // A track loaded whole is a take whose singing is all of it + QVERIFY(m_window->takes()->haveTake()); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], + Coverage::Range(0, sv::ModelById::getAs + (singing)->getFrameCount())); } // Loading over a singing track that is already there: the path @@ -1391,6 +1785,12 @@ private slots: sv::sv_frame_t shift = m_window->recordingLatencyFrames(); QVERIFY(shift >= K); + // The compensation is in the take's audio, so what has to survive + // the round trip is the pitch track, not a start frame + auto before = pitchEvents(m_window->analyser2()); + QVERIFY(!before.empty()); + sv::sv_frame_t firstEventBefore = before.front().getFrame(); + QString session = m_dir.filePath("round-trip.ton"); QVERIFY(m_window->saveSessionFile(session)); m_window->doCloseSession(); @@ -1417,7 +1817,25 @@ private slots: auto wave = sv::ModelById::getAs (a2->getMainModelId()); QVERIFY(wave); - QCOMPARE(wave->getStartFrame(), -shift); + QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); + + auto after = pitchEvents(a2); + QVERIFY(!after.empty()); + QVERIFY2(std::llabs(after.front().getFrame() - firstEventBefore) <= + 2 * hop, + qPrintable(QString("the reloaded take's pitch track starts " + "at frame %1; before saving it started " + "at %2") + .arg(after.front().getFrame()) + .arg(firstEventBefore))); + + // The take is the audio file the session pointed at. Until the + // coverage is saved too (phase 5 of the takes work), a restored + // take counts as covering all of its file + QVERIFY(m_window->takes()->haveTake()); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], Coverage::Range(0, wave->getFrameCount())); } void close_session_resets() { @@ -1436,8 +1854,11 @@ private slots: QVERIFY(!m_window->realtimeLayer()); QVERIFY(m_window->realtimeModelId().isNone()); QVERIFY(m_window->currentRecordingModelId().isNone()); + QVERIFY(!m_window->recordingLayer()); QVERIFY(m_window->pendingSingingModelId().isNone()); QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY2(!m_window->takes()->haveTake(), + "the take outlived the session it was recorded in"); QCOMPARE(m_window->pendingExtraPaneCount(), 0); QCOMPARE(m_window->paneStack()->getPaneCount(), 0); QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); @@ -1684,10 +2105,9 @@ private slots: if (QTest::currentTestFailed()) return; QTest::qWait(600); - // With the take's analyser there already, Stop starts pYIN - // at once rather than 200 ms later, so that it is running when - // the session is closed. Otherwise this test shows nothing - QVERIFY(m_window->analyser2()); + // Stop splices the recording into the take and starts the analysis + // of the result there and then, so it is running when the session + // is closed. Otherwise this test shows nothing m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(sv::ModelTransformerFactory::getInstance() From 2a9b8921344d8fa5bb37c53329bc9e462368505b Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 00:52:39 +0300 Subject: [PATCH 030/275] test: record again while the last take is still being analysed The splice on Stop tears the take's analyser down to rebuild it, so pressing Record again before the analysis has finished now cancels a running pYIN in the middle of the Stop path -- the area this fork has crashed in before. A guard, not a new behaviour: it needs CPU load to say anything, and a crash is the failure. Also an indentation fix in SingingTakes::spliceRecording(). Co-Authored-By: Claude Opus 5 --- main/SingingTakes.cpp | 4 +-- main/test/TestRecordWorkflow.h | 46 ++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 715b36f6..72a0d99a 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -66,8 +66,8 @@ SingingTakes::spliceRecording(QString recordingPath, Coverage::Range range; QString error = TakeAudio::splice(m_audioPath, recordingPath, - recordingOffset, position, length, - outPath, &range); + recordingOffset, position, length, + outPath, &range); if (error != "") return error; if (m_audioPath != "") m_superseded.push_back(m_audioPath); diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index dbe0ff94..c1352d9b 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1320,6 +1320,52 @@ private slots: .arg(input).arg(reference))); } + // Recording again before the analysis of the take just made has + // finished. The second take's splice tears that analyser down in the + // middle of its pYIN, which is the area this fork has crashed in + // before; cancelAnalyses() is what keeps it safe. A regression guard, + // not a new behaviour: run it under load, a crash is the failure. + void rerecord_during_analysis() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(700); + + // Stop splices the recording in and starts the analysis of the + // result there and then + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY2(sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), + "the race was not set up: no analysis was running when the " + "second take started"); + QString first = m_window->takes()->getAudioPath(); + QVERIFY(!first.isEmpty()); + + // In a gap, so nothing is asked + m_window->seekTo(sv::sv_frame_t(1.5 * rate)); + take(500); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->recordOverQuestions(), 0); + + QVERIFY(m_window->analyser2()); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + QVERIFY(m_window->takes()->getAudioPath() != first); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + QCOMPARE(ranges[1].start, sv::sv_frame_t(1.5 * rate)); + + QVERIFY(!m_window->recordingLayer()); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + verifyPlaySourceClean(); + } + void rerecord_cleans_up() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); From 9f2f6f4ec7198ac953aac1e82f85deffec2c0183 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 01:03:24 +0300 Subject: [PATCH 031/275] fix: the cursor runs from the take's position while recording svgui's ViewManager reported the recorded duration as the playback frame, so for a take from further into the song the cursor and the view ran from frame 0 while the dots appeared elsewhere. svgui 80b3ffd adds a record start frame; record() sets it. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 5 +++++ main/test/TestRecordWorkflow.h | 26 ++++++++++++++++++++++++++ repoint-lock.json | 2 +- 3 files changed, 32 insertions(+), 1 deletion(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4bfdf55c..df52bf9d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3060,6 +3060,11 @@ MainWindow::record() setAudioRecordMode(RecordReplaceSession); } + // While recording, the playback cursor is the start of the recording + // plus what has been recorded: the views follow the take from where it + // is being made, not from frame 0 + if (m_viewManager) m_viewManager->setRecordStartFrame(m_takePosition); + MainWindowBase::record(); // The base class gives up without a signal when the device cannot be diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index c1352d9b..8bcb0bd5 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -114,6 +114,7 @@ class TestMainWindow : public MainWindow void seekTo(sv::sv_frame_t frame) { m_viewManager->setPlaybackFrame(frame); } + sv::sv_frame_t playbackFrame() { return m_viewManager->getPlaybackFrame(); } // The question about recording over singing that is there is answered // from here: the suite cannot answer a dialog @@ -874,6 +875,31 @@ private slots: // Record starts the take at the playback position: what is sung lands // there on the reference's timeline, and the take's audio file is // silence up to it + // The cursor, which the view follows, runs from where the take is being + // recorded and not from frame 0 + void take_cursor_runs_from_position() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(500); + sv::sv_frame_t during = m_window->playbackFrame(); + stopTake(); + if (QTest::currentTestFailed()) return; + + QVERIFY2(during > P && during < P + sv::sv_frame_t(2.0 * rate), + qPrintable(QString("half a second into a take recorded from " + "frame %1 the cursor was at frame %2") + .arg(P).arg(during))); + QCOMPARE(m_window->playbackFrame(), P); + } + void take_at_playback_position() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); diff --git a/repoint-lock.json b/repoint-lock.json index f4e9ff74..d28729b4 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "271db78f54eff467e3891fca920ab663cc69ce53" + "pin": "80b3ffd4e092267a2e0ad80dd525e6e2afcaa007" }, "svapp": { "pin": "ed817c2b27d06c02c53451036417be564dc0702b" From 3fc796830fb391cee410d71592fcab83098a0b46 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 01:36:38 +0300 Subject: [PATCH 032/275] feat: TakeTiming, the arithmetic of a pre-roll and a punch-out Where the reference is played from (S), which part of the recording the take keeps (the latency plus the lead-in, and at most as far as the end of the selection), when the take has enough to stop itself, where a live dot belongs, and what the lead-in's countdown says. All of it pure, so the application only feeds it and acts on the answers; nothing uses it yet. Unit tests in TestTakeTiming. Co-Authored-By: Claude Opus 5 --- main/TakeTiming.cpp | 134 ++++++++++++++++++++++++ main/TakeTiming.h | 127 ++++++++++++++++++++++ main/test/TestTakeTiming.h | 197 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 ++ meson.build | 2 + 5 files changed, 467 insertions(+) create mode 100644 main/TakeTiming.cpp create mode 100644 main/TakeTiming.h create mode 100644 main/test/TestTakeTiming.h diff --git a/main/TakeTiming.cpp b/main/TakeTiming.cpp new file mode 100644 index 00000000..76281c5c --- /dev/null +++ b/main/TakeTiming.cpp @@ -0,0 +1,134 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeTiming.h" + +#include "LatencyUtils.h" + +#include + +#include +#include + +using namespace sv; + +namespace { + +QString tr(const char *text) +{ + return QCoreApplication::translate("TakeTiming", text); +} + +} // namespace + +sv_frame_t +TakeTiming::preRollBefore(sv_frame_t position, sv_frame_t wanted) +{ + if (wanted <= 0 || position <= 0) return 0; + return std::min(wanted, position); +} + +sv_frame_t +TakeTiming::playbackStart() const +{ + sv_frame_t start = position - preRoll; + return start > 0 ? start : 0; +} + +bool +TakeTiming::havePunchOut() const +{ + return end > position; +} + +sv_frame_t +TakeTiming::spliceOffset() const +{ + // Everything before this was recorded before the singer could have + // heard the reference at P: the round trip, and the lead-in they + // were listening to before it + sv_frame_t offset = latency + preRoll; + return offset > 0 ? offset : 0; +} + +sv_frame_t +TakeTiming::spliceLength() const +{ + if (!havePunchOut()) return -1; + return end - position; +} + +sv_frame_t +TakeTiming::autoStopFrames() const +{ + if (!havePunchOut()) return 0; + sv_frame_t margin = sv_frame_t(rate * autoStopMarginSeconds()); + return spliceOffset() + (end - position) + margin; +} + +bool +TakeTiming::shouldStopAt(sv_frame_t framesReceived) const +{ + if (!havePunchOut()) return false; + return framesReceived >= autoStopFrames(); +} + +sv_frame_t +TakeTiming::liveFrameIntoTake(sv_frame_t recordedFrame) const +{ + // The same shift the splice applies to the audio, so that a dot sits + // where the finished pitch track will put the sound it stands for + return compensatedLiveFrame(recordedFrame, spliceOffset()); +} + +bool +TakeTiming::isInLeadIn(sv_frame_t framesReceived) const +{ + // Without a pre-roll there is no lead-in to wait through: what + // little comes before the latency is not worth counting down + return preRoll > 0 && framesReceived < spliceOffset(); +} + +int +TakeTiming::countdownSeconds(sv_frame_t framesReceived) const +{ + if (rate <= 0 || !isInLeadIn(framesReceived)) return 0; + double seconds = double(spliceOffset() - framesReceived) / rate; + return int(std::ceil(seconds)); +} + +QString +TakeTiming::countdownText(sv_frame_t framesReceived) const +{ + int seconds = countdownSeconds(framesReceived); + if (seconds <= 0) return ""; + return tr("Recording in %1…").arg(seconds); +} + +bool +TakeTiming::chooseRange(const Coverage::Ranges &ranges, sv_frame_t playhead, + Coverage::Range &chosen) +{ + if (ranges.empty()) return false; + + for (const Coverage::Range &range : ranges) { + if (playhead >= range.start && playhead < range.end) { + chosen = range; + return true; + } + } + + chosen = ranges[0]; + return true; +} diff --git a/main/TakeTiming.h b/main/TakeTiming.h new file mode 100644 index 00000000..01bda619 --- /dev/null +++ b/main/TakeTiming.h @@ -0,0 +1,127 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_TIMING_H +#define TONY_TAKE_TIMING_H + +#include "Coverage.h" + +#include "base/BaseTypes.h" + +#include + +/** + * The frame arithmetic of one singing take: where the reference is + * played from, which part of the recording is kept, when the take + * stops by itself, and where a live dot belongs. + * + * P (position) is where the take's new material goes on the + * reference's timeline; E (end) is where it stops if the singer is + * recording into a selection; R (preRoll) is the lead-in that is + * heard but not recorded over; L (latency) is the round trip, as + * LatencyUtils.h describes it. The device records from the press of + * Record whatever else happens: pre-roll and punch-out only change + * which part of the recording is used and when it stops. + * + * Nothing here touches a model, a window or the settings, so all of + * it is tested without either (TestTakeTiming): MainWindow fills the + * fields in and does as the answers say. + */ +struct TakeTiming +{ + /// Of the reference and of the recording alike; they must match + sv::sv_samplerate_t rate; + + /// P: where the material recorded from now on is to land + sv::sv_frame_t position; + + /// E: where the take stops by itself, or -1 for "when Stop is pressed" + sv::sv_frame_t end; + + /// R: the lead-in played before P, never taking S below frame 0 + sv::sv_frame_t preRoll; + + /// L: what was recorded before the singer could hear the reference at S + sv::sv_frame_t latency; + + TakeTiming() : + rate(0), position(0), end(-1), preRoll(0), latency(0) { } + + /** + * The lead-in there is room for before position: the pre-roll + * asked for, shortened near the start of the song and nothing at + * frame 0, since a take cannot start before the song does. + */ + static sv::sv_frame_t preRollBefore(sv::sv_frame_t position, + sv::sv_frame_t wanted); + + /** + * How much singing past E is recorded before the take stops + * itself. The latency can still be refined while the take runs, + * and the timer that watches for the end only polls now and then, + * so the margin is what makes sure everything up to E is there. + * What is recorded past E is never used. + */ + static double autoStopMarginSeconds() { return 0.25; } + + /// S: where playback starts, so the reference reaches P after R frames + sv::sv_frame_t playbackStart() const; + + /// The take has an end to stop itself at + bool havePunchOut() const; + + /// The frame of the recording that the take's new material starts at + sv::sv_frame_t spliceOffset() const; + + /// How much of the recording to use, or -1 for all there is of it + sv::sv_frame_t spliceLength() const; + + /** + * How many frames the record target must have received for the + * take to hold everything up to E, margin included. 0 when there + * is no punch-out to stop at. + */ + sv::sv_frame_t autoStopFrames() const; + + /// The take has everything it was asked for and can stop now + bool shouldStopAt(sv::sv_frame_t framesReceived) const; + + /** + * Where sound found at this frame of the recording belongs, + * counted from P. Negative means it was sung during the lead-in, + * before the take's own material begins, and has no place on the + * reference's timeline. + */ + sv::sv_frame_t liveFrameIntoTake(sv::sv_frame_t recordedFrame) const; + + /// The lead-in is still running: nothing recorded so far counts + bool isInLeadIn(sv::sv_frame_t framesReceived) const; + + /// Seconds of the lead-in left, rounded up; 0 once it is over + int countdownSeconds(sv::sv_frame_t framesReceived) const; + + /// What to show while the lead-in runs; empty once it is over + QString countdownText(sv::sv_frame_t framesReceived) const; + + /** + * The range to record into out of those selected: the one that + * holds the playhead, else the first. The ranges must be in + * order. False, leaving chosen alone, if there are none. + */ + static bool chooseRange(const Coverage::Ranges &ranges, + sv::sv_frame_t playhead, + Coverage::Range &chosen); +}; + +#endif diff --git a/main/test/TestTakeTiming.h b/main/test/TestTakeTiming.h new file mode 100644 index 00000000..25e7befb --- /dev/null +++ b/main/test/TestTakeTiming.h @@ -0,0 +1,197 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TAKE_TIMING_H +#define TEST_TAKE_TIMING_H + +// Tier 2: the frame arithmetic of a take with a pre-roll and a +// punch-out. No window, no device and no files: what MainWindow does +// with the answers is Tier 5's business (TestRecordWorkflow). + +#include "../TakeTiming.h" + +#include +#include + +class TestTakeTiming : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + + static constexpr double kRate = 44100.0; + + // A take at P with a pre-roll of R and a round trip of L, running + // until Stop + static TakeTiming take(frame_t position, frame_t preRoll = 0, + frame_t latency = 0) { + TakeTiming t; + t.rate = kRate; + t.position = position; + t.preRoll = preRoll; + t.latency = latency; + return t; + } + +private slots: + // The pre-roll is as long as it was asked for, except near the + // start of the song, where there is less room for it + void preroll_fits_before_the_position() { + const frame_t wanted = frame_t(3 * kRate); + QCOMPARE(TakeTiming::preRollBefore(frame_t(10 * kRate), wanted), wanted); + QCOMPARE(TakeTiming::preRollBefore(wanted, wanted), wanted); + + // Half a second in, the lead-in is half a second + QCOMPARE(TakeTiming::preRollBefore(frame_t(0.5 * kRate), wanted), + frame_t(0.5 * kRate)); + // and at the start of the song there is none + QCOMPARE(TakeTiming::preRollBefore(0, wanted), frame_t(0)); + // nor is there with the pre-roll switched off + QCOMPARE(TakeTiming::preRollBefore(frame_t(10 * kRate), 0), frame_t(0)); + } + + // Playback starts R frames before the take's position, so that the + // reference reaches the position as the recording starts to count + void playback_starts_before_the_position() { + QCOMPARE(take(1000).playbackStart(), frame_t(1000)); + QCOMPARE(take(5000, 3000).playbackStart(), frame_t(2000)); + QCOMPARE(take(3000, 3000).playbackStart(), frame_t(0)); + // Nothing before frame 0, whatever it is asked for + QCOMPARE(take(1000, 4000).playbackStart(), frame_t(0)); + } + + // The recording is used from the latency plus the lead-in on: both + // were recorded before the singer could hear the reference at P + void splice_offset_is_the_latency_and_the_lead_in() { + QCOMPARE(take(5000).spliceOffset(), frame_t(0)); + QCOMPARE(take(5000, 0, 1500).spliceOffset(), frame_t(1500)); + QCOMPARE(take(5000, 3000).spliceOffset(), frame_t(3000)); + QCOMPARE(take(5000, 3000, 1500).spliceOffset(), frame_t(4500)); + // A latency that was never measured is no latency + QCOMPARE(take(5000, 3000, -100).spliceOffset(), frame_t(2900)); + } + + // Without a punch-out the whole of the recording is used; with one, + // only as much of it as reaches the end of the selection + void splice_length_stops_at_the_end() { + QCOMPARE(take(5000).spliceLength(), frame_t(-1)); + + TakeTiming t = take(5000, 3000, 1500); + t.end = 9000; + QVERIFY(t.havePunchOut()); + QCOMPARE(t.spliceLength(), frame_t(4000)); + // ... which the lead-in and the latency do not change + QCOMPARE(t.spliceOffset(), frame_t(4500)); + + // A selection that ends where it starts, or before it, is no + // punch-out at all + t.end = 5000; + QVERIFY(!t.havePunchOut()); + QCOMPARE(t.spliceLength(), frame_t(-1)); + } + + // The take stops itself once the singing for the end of the + // selection must have arrived: the offset, the length, and a margin + void auto_stop_waits_for_the_end_and_a_margin() { + TakeTiming t = take(5000, 3000, 1500); + QVERIFY(!t.havePunchOut()); + QCOMPARE(t.autoStopFrames(), frame_t(0)); + QVERIFY(!t.shouldStopAt(frame_t(100 * kRate))); + + t.end = 5000 + frame_t(2 * kRate); + frame_t margin = frame_t(kRate * TakeTiming::autoStopMarginSeconds()); + frame_t want = 4500 + frame_t(2 * kRate) + margin; + QCOMPARE(t.autoStopFrames(), want); + QVERIFY(!t.shouldStopAt(want - 1)); + QVERIFY(t.shouldStopAt(want)); + QVERIFY(t.shouldStopAt(want + 10000)); + + // A longer lead-in is a later stop: the lead-in is not part of + // what the take keeps + TakeTiming longer = t; + longer.preRoll = 3000 + frame_t(kRate); + QCOMPARE(longer.autoStopFrames(), want + frame_t(kRate)); + } + + // The live dots go where the finished pitch track will put them, + // and sound from the lead-in gets no dot at all + void live_dots_start_at_the_position() { + TakeTiming t = take(5000, 3000, 1500); + QCOMPARE(t.liveFrameIntoTake(4500), frame_t(0)); + QCOMPARE(t.liveFrameIntoTake(9500), frame_t(5000)); + QVERIFY(t.liveFrameIntoTake(4499) < 0); + QVERIFY(t.liveFrameIntoTake(0) < 0); + + // With no lead-in and no latency a dot is where it was sung + QCOMPARE(take(5000).liveFrameIntoTake(1234), frame_t(1234)); + } + + // The status bar counts the lead-in down in whole seconds, and says + // nothing once the take's own material has begun + void countdown_runs_out_with_the_lead_in() { + TakeTiming t = take(frame_t(10 * kRate), frame_t(3 * kRate)); + QVERIFY(t.isInLeadIn(0)); + QCOMPARE(t.countdownSeconds(0), 3); + QCOMPARE(t.countdownText(0), QString("Recording in 3…")); + QCOMPARE(t.countdownSeconds(frame_t(1.5 * kRate)), 2); + QCOMPARE(t.countdownText(frame_t(1.5 * kRate)), + QString("Recording in 2…")); + QCOMPARE(t.countdownSeconds(frame_t(3 * kRate) - 1), 1); + + // Once the lead-in is over there is nothing to say + QVERIFY(!t.isInLeadIn(frame_t(3 * kRate))); + QCOMPARE(t.countdownSeconds(frame_t(3 * kRate)), 0); + QCOMPARE(t.countdownText(frame_t(3 * kRate)), QString()); + QCOMPARE(t.countdownText(frame_t(10 * kRate)), QString()); + + // The latency is part of the wait: the singer's answer to the + // reference at P arrives that much later + t.latency = frame_t(0.5 * kRate); + QCOMPARE(t.countdownSeconds(frame_t(3 * kRate)), 1); + + // With the pre-roll off there is no lead-in and no countdown, + // however long the round trip is + TakeTiming plain = take(frame_t(10 * kRate), 0, frame_t(0.5 * kRate)); + QVERIFY(!plain.isInLeadIn(0)); + QCOMPARE(plain.countdownText(0), QString()); + } + + // Recording into a selection: the one the playhead is in, else the + // first of them + void selection_at_the_playhead_is_the_one_recorded_into() { + Coverage::Ranges ranges; + Coverage::Range chosen; + + QVERIFY(!TakeTiming::chooseRange(ranges, 100, chosen)); + + ranges.push_back(Coverage::Range(1000, 2000)); + ranges.push_back(Coverage::Range(5000, 7000)); + + QVERIFY(TakeTiming::chooseRange(ranges, 6000, chosen)); + QCOMPARE(chosen, Coverage::Range(5000, 7000)); + QVERIFY(TakeTiming::chooseRange(ranges, 5000, chosen)); + QCOMPARE(chosen, Coverage::Range(5000, 7000)); + + // The end of a range is not in it + QVERIFY(TakeTiming::chooseRange(ranges, 2000, chosen)); + QCOMPARE(chosen, Coverage::Range(1000, 2000)); + + // The playhead outside all of them: the first + QVERIFY(TakeTiming::chooseRange(ranges, 0, chosen)); + QCOMPARE(chosen, Coverage::Range(1000, 2000)); + QVERIFY(TakeTiming::chooseRange(ranges, 100000, chosen)); + QCOMPARE(chosen, Coverage::Range(1000, 2000)); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index b6a49afa..d153ec22 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -17,6 +17,7 @@ #include "TestCoverage.h" #include "TestTakeAudio.h" #include "TestSingingTakes.h" +#include "TestTakeTiming.h" #include "RunSuite.h" @@ -77,6 +78,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTakeTiming t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index e19fb49d..d167d567 100644 --- a/meson.build +++ b/meson.build @@ -1098,6 +1098,7 @@ tony_core_files = [ 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', + 'main/TakeTiming.cpp', ] tony_app_files = [ @@ -1346,6 +1347,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestCoverage.h', 'main/test/TestTakeAudio.h', 'main/test/TestSingingTakes.h', + 'main/test/TestTakeTiming.h', ]) tony_core_test_exe = executable( From d5694a1974f4ca079d08a9f7478f350923fc9dc9 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 01:36:49 +0300 Subject: [PATCH 033/275] feat: pre-roll and recording into a selection Two toggles in the "While recording:" toolbar section, both kept in QSettings. Pre-roll plays the reference from a few seconds before the take's position (MainWindow/prerollseconds, 3 s, no UI): the cursor runs through the lead-in with it, the status bar counts it down, no live dot is drawn before the position, and the splice skips the lead-in along with the latency. Record into Selection makes the selection the take: its start is the position whatever the playhead says, the playhead only picks which selection, no overwrite question is asked, a timer stops the take once the singing for the selection's end has arrived, and the splice keeps no more than the selection. Stop can still come first. Tests: four in TestRecordWorkflow, each seen to fail with the arithmetic or the question broken for a moment. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 324 +++++++++++++++++++++++++++++---- main/MainWindow.h | 55 +++++- main/test/TestRecordWorkflow.h | 297 ++++++++++++++++++++++++++++++ 3 files changed, 640 insertions(+), 36 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index df52bf9d..42795ad2 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -125,6 +125,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_showSingingNotes(nullptr), m_playSingingAudio(nullptr), m_playRefWhileRecording(nullptr), + m_preRoll(nullptr), + m_recordIntoSelection(nullptr), m_loadSingingTrackAction(nullptr), m_alternatePitch(nullptr), m_showAlternatePitch(nullptr), @@ -133,6 +135,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_referencePitchHiddenForTake(false), m_takes(nullptr), m_takePosition(0), + m_takePreRoll(0), + m_takeEnd(-1), + m_takeTimer(nullptr), m_backgroundMusicModelId(), m_backgroundMusicLayer(nullptr), m_loadBackgroundMusicAction(nullptr), @@ -356,6 +361,12 @@ MainWindow::MainWindow(AudioMode audioMode, m_takes = new SingingTakes(this); + // Often enough to stop a take that records into a selection well + // within the margin that follows the selection's end + m_takeTimer = new QTimer(this); + m_takeTimer->setInterval(100); + connect(m_takeTimer, SIGNAL(timeout()), this, SLOT(pollTakeProgress())); + setupMenus(); setupToolbars(); setupHelpMenu(); @@ -406,6 +417,9 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindow::~MainWindow() { + // Nothing must poll a take while the window is coming down + stopTakePolling(); + // Clean up secondary state that may not have been torn down if the // window was closed without going through closeSession() (e.g. on // application exit via the window close button). @@ -1503,6 +1517,55 @@ MainWindow::setupToolbars() }); connect(this, SIGNAL(canPlay(bool)), m_playRefWhileRecording, SLOT(setEnabled(bool))); + // Pre-roll: a lead-in before the take's position, heard but not + // recorded over, so the singer comes in in time. Its length is the + // QSettings value MainWindow/prerollseconds, with no UI to change it. + m_preRoll = toolbar->addAction(il.load("rewind"), tr("Pre-roll")); + m_preRoll->setCheckable(true); + { + QSettings settings; + settings.beginGroup("MainWindow"); + m_preRoll->setChecked(settings.value("preroll", false).toBool()); + settings.endGroup(); + } + m_preRoll->setToolTip( + tr("Play the reference from a few seconds before the recording " + "position, counting down, so that you can come in in time. " + "Nothing before the position is recorded over.")); + connect(m_preRoll, &QAction::toggled, this, [this](bool on) { + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("preroll", on); + settings.endGroup(); + }); + connect(this, SIGNAL(canPlay(bool)), m_preRoll, SLOT(setEnabled(bool))); + + // Punch in and out: record the selected range and nothing else. + // The selection says where the take starts and where it stops, so + // the singer need not reach for Stop. + m_recordIntoSelection = toolbar->addAction(il.load("playselection"), + tr("Record into Selection")); + m_recordIntoSelection->setCheckable(true); + { + QSettings settings; + settings.beginGroup("MainWindow"); + m_recordIntoSelection->setChecked( + settings.value("recordintoselection", false).toBool()); + settings.endGroup(); + } + m_recordIntoSelection->setToolTip( + tr("Record into the selected range only: recording starts at the " + "start of the selection, whatever the playhead says, and stops " + "by itself at its end")); + connect(m_recordIntoSelection, &QAction::toggled, this, [this](bool on) { + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("recordintoselection", on); + settings.endGroup(); + }); + connect(this, SIGNAL(canPlay(bool)), m_recordIntoSelection, + SLOT(setEnabled(bool))); + // The alternate pitch track: the reference pitch an octave or more // up or down, for the singer to follow in place of the reference. spacer = new QLabel; @@ -2095,6 +2158,9 @@ MainWindow::closeSession() { if (!checkSaveModified()) return; + // Nothing of a take that is still running outlives its session + stopTakePolling(); + // Tear down singing track and realtime pitch layer before panes/document // are destroyed, so they can cleanly remove their layers from the pane. teardownRealtimePitchLayer(); @@ -2114,6 +2180,8 @@ MainWindow::closeSession() // phase 6 of the takes work. m_takes->clear(); m_takePosition = 0; + m_takePreRoll = 0; + m_takeEnd = -1; m_analyser->fileClosed(); @@ -2975,6 +3043,9 @@ MainWindow::record() // audio is still being captured — causing a crash or a null m_analyser2 // when recordCompleted() fires analyseNow() moments later. if (m_recordTarget && m_recordTarget->isRecording()) { + // Nothing left for the timer to watch, whether this Stop came + // from the button or from the timer itself + stopTakePolling(); MainWindowBase::record(); return; } @@ -3011,10 +3082,35 @@ MainWindow::record() sv_frame_t position = m_viewManager ? m_viewManager->getPlaybackFrame() : 0; if (position < 0) position = 0; + // Record into Selection: the selection is what is recorded, so it + // says where the take starts and where it stops, and the playhead + // only picks which selection that is. With none selected this is + // an ordinary recording from the playhead. + sv_frame_t end = -1; + if (m_recordIntoSelection && m_recordIntoSelection->isChecked() && + m_viewManager) { + Coverage::Ranges selected; + for (const Selection &s : m_viewManager->getSelections()) { + if (!s.isEmpty()) { + selected.push_back(Coverage::Range(s.getStartFrame(), + s.getEndFrame())); + } + } + Coverage::Range chosen; + if (TakeTiming::chooseRange(selected, position, chosen)) { + position = chosen.start; + end = chosen.end; + cerr << "MainWindow::record: recording into the selection [" + << position << "," << end << ")" << endl; + } + } + // Recording from inside singing that is already there replaces it // from that point on. The user is asked first, unless they have - // said not to be: undo can bring it back. - if (m_takes->shouldConfirmRecordingAt(position) && + // said not to be: undo can bring it back. Not asked when + // recording into a selection: making the selection was the answer, + // and the end of the recording is known there (spec 5.1). + if (end < 0 && m_takes->shouldConfirmRecordingAt(position) && !confirmRecordingOverTake()) { cerr << "MainWindow::record: recording over the existing singing " << "was declined" << endl; @@ -3022,9 +3118,13 @@ MainWindow::record() } m_takePosition = position; + m_takeEnd = end; + m_takePreRoll = TakeTiming::preRollBefore(position, + wantedPreRollFrames()); cerr << "MainWindow::record: recording into the singing track from " - << "frame " << position << endl; + << "frame " << position << ", with a lead-in of " + << m_takePreRoll << " frames" << endl; // Dots and a tracker of a take whose analysis never finished if (m_realtimePitchTracker || m_realtimePitchLayer) { @@ -3056,14 +3156,19 @@ MainWindow::record() } else { m_recordingAsSingingTrack = false; m_takePosition = 0; + m_takePreRoll = 0; + m_takeEnd = -1; m_paneCountBeforeRecording = 0; setAudioRecordMode(RecordReplaceSession); } - // While recording, the playback cursor is the start of the recording - // plus what has been recorded: the views follow the take from where it - // is being made, not from frame 0 - if (m_viewManager) m_viewManager->setRecordStartFrame(m_takePosition); + // While recording, the playback cursor is where the take's playback + // started plus what has been recorded: the views follow the take from + // where it is being made, not from frame 0. With a pre-roll that is + // the start of the lead-in, so the cursor runs through the lead-in in + // step with the reference. + sv_frame_t playbackStart = currentTakeTiming().playbackStart(); + if (m_viewManager) m_viewManager->setRecordStartFrame(playbackStart); MainWindowBase::record(); @@ -3081,11 +3186,12 @@ MainWindow::record() } // The base class centres the view on frame 0. The take is being - // recorded where playback was, and that is where the singer is - // watching (spec section 4). + // recorded from where playback was — the start of the lead-in when + // there is one — and that is where the singer is watching (spec + // section 4). if (m_recordingAsSingingTrack && m_viewManager) { - m_viewManager->setPlaybackFrame(m_takePosition); - m_viewManager->setGlobalCentreFrame(m_takePosition); + m_viewManager->setPlaybackFrame(playbackStart); + m_viewManager->setGlobalCentreFrame(playbackStart); } updateAlternatePitchForTake(); @@ -3131,9 +3237,100 @@ MainWindow::record() << endl; } } + + // A take with an end of its own to reach is watched until it + // gets there + startTakePolling(); } } +void +MainWindow::startTakePolling() +{ + // Only a take that is to stop by itself needs watching: nothing else + // about a take is decided while it runs + if (!m_takeTimer) return; + if (!m_recordTarget || !m_recordTarget->isRecording()) return; + if (!currentTakeTiming().havePunchOut()) return; + + m_takeTimer->start(); +} + +void +MainWindow::stopTakePolling() +{ + if (m_takeTimer) m_takeTimer->stop(); +} + +void +MainWindow::pollTakeProgress() +{ + // The take may have ended, or the session gone, between one poll and + // the next: there is nothing to watch then + if (!m_recordTarget || !m_recordTarget->isRecording() || + !m_recordingAsSingingTrack) { + stopTakePolling(); + return; + } + + // The latency may have been refined since the last poll, which moves + // the end of the take with it: currentTakeTiming() has the figure as + // it stands now + TakeTiming timing = currentTakeTiming(); + sv_frame_t received = m_recordTarget->getFramesReceived(); + + // Everything up to the end of the selection has been sung and + // recorded; what comes after it would not be used anyway + if (timing.shouldStopAt(received)) { + cerr << "MainWindow::pollTakeProgress: the singing up to frame " + << timing.end << " has arrived (" << received + << " frames recorded): stopping the take" << endl; + // The same path as the Stop button, which stops the timer + record(); + } +} + +bool +MainWindow::showTakeCountdown() const +{ + // While the lead-in of a pre-roll runs, the status bar counts it down + // instead of saying where playback is or how much has been recorded: + // what is coming in does not count yet. + // + // Everything that writes the status bar during a take has to come + // through here, because they all write often — the recorded duration + // every 10 ms, the playback position every 20 ms, the visible range + // whenever the view scrolls after the cursor — and anything written + // between two of those would be gone before it could be read. (The + // live dots write it too, and need no help: none is drawn during the + // lead-in.) + if (!m_recordingAsSingingTrack || m_takePreRoll <= 0 || !m_recordTarget) { + return false; + } + + QString countdown = currentTakeTiming().countdownText + (m_recordTarget->getFramesReceived()); + if (countdown == "") return false; + + m_myStatusMessage = countdown; + getStatusLabel()->setText(countdown); + return true; +} + +void +MainWindow::recordDurationChanged(sv_frame_t frame, sv_samplerate_t rate) +{ + if (showTakeCountdown()) return; + MainWindowBase::recordDurationChanged(frame, rate); +} + +void +MainWindow::playbackFrameChanged(sv_frame_t frame) +{ + if (showTakeCountdown()) return; + MainWindowBase::playbackFrameChanged(frame); +} + void MainWindow::recordingStarted() { @@ -3184,10 +3381,15 @@ MainWindow::recordingStarted() // singing recording's timeline after the take. // output latency = time from play() call until audio exits the speaker // input latency = time from sound entering the mic until it arrives here - // The singer's response to the reference at m_takePosition arrives - // in the recording at approximately frame + // The singer's response to the reference where playback starts + // arrives in the recording at approximately frame // (outputLatency + inputLatency), so that is the frame the splice // reads the recording from. + // With a pre-roll, playback starts at the beginning of the + // lead-in rather than at the take's position, and the splice + // skips the lead-in as well (TakeTiming::spliceOffset()). + sv_frame_t playbackStart = currentTakeTiming().playbackStart(); + sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; // @@ -3212,8 +3414,8 @@ MainWindow::recordingStarted() m_recordingStartGapMeasured = -1; m_awaitingReferenceStart = true; - m_viewManager->setPlaybackFrame(m_takePosition); - m_playSource->play(m_takePosition); + m_viewManager->setPlaybackFrame(playbackStart); + m_playSource->play(playbackStart); } updateLayerStatuses(); @@ -3228,10 +3430,53 @@ MainWindow::refineRecordingLatency() if (measured < 0 || measured == m_recordingStartGapEstimate) return; cerr << "MainWindow::refineRecordingLatency: start gap was " << measured << " frames, not the estimated " << m_recordingStartGapEstimate << endl; - m_recordingLatencyFrames += measured - m_recordingStartGapEstimate; + m_recordingLatencyFrames = currentRecordingLatency(); m_recordingStartGapEstimate = measured; } +sv_frame_t +MainWindow::currentRecordingLatency() const +{ + // Reading the measurement without taking it: pollTakeProgress() wants + // the latest figure every time it looks, but making it the stored one + // is onRealtimePitchDetected()'s business — it throws the dots placed + // with the estimate away when the figure changes. + sv_frame_t measured = m_recordingStartGapMeasured; + if (measured < 0) return m_recordingLatencyFrames; + return m_recordingLatencyFrames + (measured - m_recordingStartGapEstimate); +} + +TakeTiming +MainWindow::currentTakeTiming() const +{ + TakeTiming timing; + auto model = getMainModel(); + timing.rate = model ? model->getSampleRate() : 0; + timing.position = m_takePosition; + timing.end = m_takeEnd; + timing.preRoll = m_takePreRoll; + timing.latency = currentRecordingLatency(); + return timing; +} + +sv_frame_t +MainWindow::wantedPreRollFrames() const +{ + if (!m_preRoll || !m_preRoll->isChecked()) return 0; + + QSettings settings; + settings.beginGroup("MainWindow"); + double seconds = settings.value("prerollseconds", 3.0).toDouble(); + settings.endGroup(); + if (seconds <= 0.0) return 0; + + auto model = getMainModel(); + sv_samplerate_t rate = model ? model->getSampleRate() : 0; + if (rate <= 0) return 0; + + return sv_frame_t(seconds * rate); +} + void MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) { @@ -3246,10 +3491,10 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) // Draw the dot where the finished pitch track will put this sound: the // take is spliced into the singing track from m_takePosition on, with - // the recording latency taken off its front. The first dots may have - // been placed using the estimate of the start gap. They all belong to - // sound from before the reference started, which has no place on the - // reference's timeline. + // the recording latency and the lead-in taken off its front. The first + // dots may have been placed using the estimate of the start gap. They + // all belong to sound from before the reference started, which has no + // place on the reference's timeline. sv_frame_t latencyBefore = m_recordingLatencyFrames; refineRecordingLatency(); auto m = ModelById::getAs(m_realtimePitchModelId); @@ -3257,7 +3502,9 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) for (const Event &e : m->getAllEvents()) m->remove(e); } - sv_frame_t intoTake = compensatedLiveFrame(frame, m_recordingLatencyFrames); + // A negative answer is sound sung during the lead-in of a pre-roll, or + // before the reference started at all: no dot for it + sv_frame_t intoTake = currentTakeTiming().liveFrameIntoTake(frame); if (intoTake < 0) return; sv_frame_t dotFrame = m_takePosition + intoTake; @@ -3306,6 +3553,7 @@ MainWindow::recordingFinishedFull(Analyser *analysing) // pYIN pitch track is there to replace it. With no analyser (the // analysis could not be started) there is nothing to wait for. cerr << "MainWindow::recordingFinishedFull: take finished" << endl; + stopTakePolling(); m_recordingInProgress = false; m_recordingAsSingingTrack = false; m_currentRecordingModelId = {}; @@ -3344,17 +3592,20 @@ MainWindow::finishSingingTake() { // The take has stopped, and what the device recorded is raw material: // it goes into the take's audio file at the position the take was - // started from, with the latency taken off its front, and the singing - // track is then rebuilt from the file that comes out of that. + // started from, with the latency and the lead-in taken off its front, + // and the singing track is then rebuilt from the file that comes out. // // (Rebuilt means analysed in full, as a singing track loaded from a // file is. Phase 4 of the takes work puts the new audio under the // pitch and notes layers that are there and analyses only the range // that changed, which is what makes this quick on a long song.) + stopTakePolling(); refineRecordingLatency(); - sv_frame_t latency = m_recordingLatencyFrames; + TakeTiming timing = currentTakeTiming(); + sv_frame_t offset = timing.spliceOffset(); + sv_frame_t length = timing.spliceLength(); sv_frame_t position = m_takePosition; QString recordingPath; @@ -3382,12 +3633,12 @@ MainWindow::finishSingingTake() if (m_viewManager) m_viewManager->setPlaybackFrame(position); // A take stopped the moment it was started, or one no longer than the - // latency, has nothing in it to add. Nothing has gone wrong; there is - // simply nothing to do - if (recordingPath != "" && recorded <= latency) { + // latency and the lead-in together, has nothing in it to add. Nothing + // has gone wrong; there is simply nothing to do + if (recordingPath != "" && recorded <= offset) { cerr << "MainWindow::finishSingingTake: nothing to use: " << recorded - << " frames recorded, the first " << latency - << " of which are the latency" << endl; + << " frames recorded, the first " << offset + << " of which are the latency and the lead-in" << endl; recordingFinishedFull(nullptr); return; } @@ -3403,10 +3654,12 @@ MainWindow::finishSingingTake() error = tr("Could not find a directory to write the singing " "track into"); } else { - // Everything before the latency is sound from before the singer - // could have heard the reference at the take's position - error = m_takes->spliceRecording(recordingPath, latency, position, - -1, directory, &placed); + // Everything before the offset is sound from before the singer + // could have heard the reference at the take's position: the + // round trip, and the lead-in of a pre-roll before it. The + // length is what a punch-out allows, or all there is + error = m_takes->spliceRecording(recordingPath, offset, position, + length, directory, &placed); } } @@ -3424,7 +3677,7 @@ MainWindow::finishSingingTake() } cerr << "MainWindow::finishSingingTake: " << recorded << " frames " - << "recorded, used from frame " << latency << ", placed at [" + << "recorded, used from frame " << offset << ", placed at [" << placed.start << "," << placed.end << ") of " << m_takes->getAudioPath() << endl; @@ -4598,6 +4851,9 @@ MainWindow::updateVisibleRangeDisplay(Pane *p) const return; } + // The countdown of a pre-roll's lead-in has the status bar to itself + if (showTakeCountdown()) return; + bool haveSelection = false; sv_frame_t startFrame = 0, endFrame = 0; diff --git a/main/MainWindow.h b/main/MainWindow.h index 28d317c5..9b681ca0 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -21,12 +21,15 @@ #include "RealtimePitchTracker.h" #include "AlternatePitchTrack.h" #include "SingingTakes.h" +#include "TakeTiming.h" #include #include #include "data/model/SparseTimeValueModel.h" +class QTimer; + namespace sv { class VersionTester; class ActivityLog; @@ -150,6 +153,11 @@ protected slots: virtual void monitoringLevelsChanged(float, float); + // The status bar's "Recording: " and "Playing: ...", which + // the lead-in of a pre-roll replaces with its countdown + virtual void recordDurationChanged(sv::sv_frame_t, sv::sv_samplerate_t); + virtual void playbackFrameChanged(sv::sv_frame_t); + virtual void audioGainChanged(float); virtual void pitchGainChanged(float); virtual void notesGainChanged(float); @@ -213,6 +221,10 @@ protected slots: virtual void recordingFinishedFull(Analyser *analysing = nullptr); virtual void finishSingingTake(); + // Watches a take that is to stop at an end of its own; see + // startTakePolling() + virtual void pollTakeProgress(); + void moveOneNoteRight(); void moveOneNoteLeft(); void selectOneNoteRight(); @@ -246,6 +258,8 @@ protected slots: QAction *m_showSingingNotes; QAction *m_playSingingAudio; QAction *m_playRefWhileRecording; + QAction *m_preRoll; + QAction *m_recordIntoSelection; QAction *m_loadSingingTrackAction; // The alternate pitch track: the reference pitch track moved by whole @@ -269,10 +283,41 @@ protected slots: // Where on the reference's timeline the take being recorded, or the // one most recently recorded, starts: the playback position when - // Record was pressed. The live dots are drawn from here, and this - // is where the recording is spliced into the take's audio. + // Record was pressed, or the start of the selection recorded into. + // The live dots are drawn from here, and this is where the recording + // is spliced into the take's audio. sv::sv_frame_t m_takePosition; + // The lead-in played before m_takePosition (R, "Pre-roll"), and the + // frame the take stops itself at (E, "Record into Selection"), or -1 + // when it runs until Stop is pressed. Both are worked out in + // record() and used until the take has been spliced in; TakeTiming + // does the arithmetic. + sv::sv_frame_t m_takePreRoll; + sv::sv_frame_t m_takeEnd; + + // Polls the record target while a take that has an end to reach + // runs, and stops the take once the singing for that end has + // arrived. Not running for a take that goes on until Stop. + QTimer *m_takeTimer; + + void startTakePolling(); + void stopTakePolling(); + + // The take being recorded, or the one just recorded, as TakeTiming + // sees it: everything the splice, the dots and the automatic stop + // are worked out from. + TakeTiming currentTakeTiming() const; + + // The pre-roll asked for, in frames of the reference: the QSettings + // value MainWindow/prerollseconds (3 s), or 0 with the toggle off + sv::sv_frame_t wantedPreRollFrames() const; + + // Put the countdown of a pre-roll's lead-in in the status bar, and + // say so, if that is what belongs there just now. Everything that + // writes the status bar while a take runs asks this first. + bool showTakeCountdown() const; + // Ask before recording over singing that is already there, unless // the user has said not to. Overridden by the tests, which cannot // answer a dialog. Returns true to go ahead with the recording. @@ -485,6 +530,12 @@ protected slots: void refineRecordingLatency(); + // The best figure for the latency as things stand, measurement + // included if it has arrived: refineRecordingLatency() is what makes + // it the stored one, and only it, because the live dots placed with + // the estimate are thrown away when the figure changes. + sv::sv_frame_t currentRecordingLatency() const; + // Set while the live dots of a finished take wait for pYIN to // produce the pitch track that replaces them. QMetaObject::Connection m_realtimeLayerTeardownConnection; diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 8bcb0bd5..f6845b24 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -92,6 +92,10 @@ class TestMainWindow : public MainWindow void setPlayReferenceWhileRecording(bool on) { m_playRefWhileRecording->setChecked(on); } + void setPreRoll(bool on) { m_preRoll->setChecked(on); } + void setRecordIntoSelection(bool on) { + m_recordIntoSelection->setChecked(on); + } QAction *playSingingAudioAction() { return m_playSingingAudio; } Analyser *analyser() { return m_analyser; } @@ -110,12 +114,20 @@ class TestMainWindow : public MainWindow sv::WaveformLayer *recordingLayer() { return m_recordingLayer; } SingingTakes *takes() { return m_takes; } sv::sv_frame_t takePosition() { return m_takePosition; } + sv::sv_frame_t takePreRoll() { return m_takePreRoll; } + sv::sv_frame_t takeEnd() { return m_takeEnd; } + bool takeTimerRunning() { return m_takeTimer && m_takeTimer->isActive(); } void seekTo(sv::sv_frame_t frame) { m_viewManager->setPlaybackFrame(frame); } sv::sv_frame_t playbackFrame() { return m_viewManager->getPlaybackFrame(); } + void selectRange(sv::sv_frame_t start, sv::sv_frame_t end) { + m_viewManager->addSelection(sv::Selection(start, end)); + } + void clearSelections() { m_viewManager->clearSelections(); } + // The question about recording over singing that is there is answered // from here: the suite cannot answer a dialog void setRecordOverAnswer(bool yes) { m_recordOverAnswer = yes; } @@ -442,6 +454,26 @@ class TestRecordWorkflow : public QObject return true; } + // The pre-roll's length has no UI: it is read from the settings when + // Record is pressed. These are the test suite's own settings + // (tony-app-test), not the user's + static void setPreRollSeconds(double seconds) { + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("prerollseconds", seconds); + settings.endGroup(); + } + + // A recording of two notes, the first as long as "first" seconds and + // the second as long as "then": what is sung during a lead-in, or + // after the end of a selection, is the one the take must not hold + static std::vector twoNotes(double first, double then) { + auto signal = tone(lowHz, first); + auto second = tone(highHz, then); + signal.insert(signal.end(), second.begin(), second.end()); + return signal; + } + // Not a slot: QtTest would run it as a test void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -489,6 +521,12 @@ private slots: QSettings settings; settings.beginGroup("MainWindow"); settings.setValue("playrefwhilerecording", false); + // The toolbar toggles of the recording behaviour, and the length + // of the pre-roll: a test that switched one on must not leave it + // on for the next + settings.setValue("preroll", false); + settings.setValue("recordintoselection", false); + settings.remove("prerollseconds"); settings.endGroup(); // The audible flags are shared by both analysers; a test that @@ -1181,6 +1219,265 @@ private slots: QCOMPARE(m_window->recordOverQuestions(), 2); } + // Pre-roll: the reference is played from before the take's position + // and the singer comes in with it, but nothing sung during that + // lead-in goes into the take. The singer here sings a low note + // through the lead-in and a high one after it: the take is to hold + // the high note only, and to be the lead-in shorter than what the + // device recorded. + void preroll_lead_is_not_recorded() { + const double leadIn = 0.5; + FakeAudioIO::Config config; + config.input = twoNotes(0.6, 1.0); + makeWindow(config); + // Nothing to hear, so the round trip is zero and the lead-in is + // the whole of what the splice skips (spec 5.1) + m_window->setPlayReferenceWhileRecording(false); + setPreRollSeconds(leadIn); + m_window->setPreRoll(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + const sv::sv_frame_t R = sv::sv_frame_t(leadIn * rate); + m_window->seekTo(P); + take(1300); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->takePosition(), P); + QCOMPARE(m_window->takePreRoll(), R); + QCOMPARE(m_window->recordOverQuestions(), 0); + + // The take starts where it was told to, whatever came before + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + + // and holds the lead-in less than the 1.3 s that was recorded + sv::sv_frame_t length = ranges[0].length(); + QVERIFY2(length > sv::sv_frame_t(0.45 * rate) && + length < sv::sv_frame_t(0.95 * rate), + qPrintable(QString("1.3 s was recorded with a lead-in of " + "%1 frames, and %2 frames of it went " + "into the take") + .arg(R).arg(length))); + + const sv::sv_frame_t margin = sv::sv_frame_t(0.1 * rate); + QVERIFY2(takeAudioRms(0, P - margin) < 0.001, + "the take's audio is not silent before the position it was " + "recorded at"); + + // The note sung during the lead-in is not in the take: what is at + // its position is the note that came after the lead-in + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + auto sung = eventsBetween(events, P + margin, ranges[0].end - margin); + QVERIFY(!sung.empty()); + QVERIFY2(std::fabs(TestSignals::centsBetween + (medianHz(sung), highHz)) < 10.0, + qPrintable(QString("the take's median pitch at its position " + "is %1 Hz; the lead-in was sung at %2 and " + "the take itself at %3") + .arg(medianHz(sung)).arg(lowHz).arg(highHz))); + } + + // What the lead-in looks and sounds like while it runs: the reference + // is played from S, the cursor runs with it from there, the status bar + // counts down, and no dot is drawn until the take's position + void preroll_counts_down_before_the_position() { + const double leadIn = 1.2; + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + setPreRollSeconds(leadIn); + m_window->setPreRoll(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + const sv::sv_frame_t R = sv::sv_frame_t(leadIn * rate); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePreRoll(), R); + QTest::qWait(400); + + QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, + "the reference was never played"); + + sv::sv_frame_t during = m_window->playbackFrame(); + QVERIFY2(during >= P - R && during < P, + qPrintable(QString("0.4 s into a take at frame %1 with a " + "lead-in of %2 frames, the cursor is at " + "frame %3; it should be running through " + "the lead-in") + .arg(P).arg(R).arg(during))); + + QVERIFY2(m_window->statusText().startsWith("Recording in "), + qPrintable(QString("the status bar says \"%1\" during the " + "lead-in, rather than counting down") + .arg(m_window->statusText()))); + + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + QVERIFY2(model->getEventCount() == 0, + qPrintable(QString("%1 live dots were drawn during the " + "lead-in, before the take's position") + .arg(model->getEventCount()))); + + // Past the lead-in the dots come, none of them before the take's + // position, and there is nothing left to count down + QTest::qWait(1400); + QVERIFY(model->getEventCount() > 10); + for (const auto &e : model->getAllEvents()) { + QVERIFY2(e.getFrame() >= P, + qPrintable(QString("a live dot at frame %1, before the " + "take's position %2") + .arg(e.getFrame()).arg(P))); + } + QVERIFY(!m_window->statusText().startsWith("Recording in ")); + + stopTake(); + if (QTest::currentTestFailed()) return; + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + } + + // Record into Selection: the selection says where the take starts and + // where it ends, the take stops by itself there, and nothing sung + // afterwards is used. Then a second one over the first, which is not + // asked about, into the selection the playhead is in, and with a + // pre-roll as well. + void punch_out_stops_at_the_end_of_the_selection() { + FakeAudioIO::Config config; + // Low note for a little longer than the selection, then high: the + // high note is what must not reach the take + config.input = twoNotes(0.85, 0.8); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(false); + m_window->setRecordIntoSelection(true); + openReference(writeWav(tone(highHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(1.0 * rate); + const sv::sv_frame_t E = sv::sv_frame_t(1.8 * rate); + m_window->selectRange(P, E); + m_window->seekTo(sv::sv_frame_t(0.3 * rate)); // outside the selection + startTake(); + if (QTest::currentTestFailed()) return; + + // The selection is recorded into whatever the playhead says + QCOMPARE(m_window->takePosition(), P); + QCOMPARE(m_window->takeEnd(), E); + QVERIFY(m_window->takeTimerRunning()); + + // Nobody presses Stop: the take ends itself once the singing for + // the end of the selection has arrived + QTRY_VERIFY_WITH_TIMEOUT(!m_window->recordTarget()->isRecording(), 5000); + QVERIFY2(!m_window->takeTimerRunning(), + "the timer that watches the take is still running after it"); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + // What it kept is exactly the selection, margin and all discarded + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], Coverage::Range(P, E)); + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(wave->getFrameCount(), E); + + // and what it kept is the note sung inside the selection, not the + // one that came after its end + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + QVERIFY2(std::fabs(TestSignals::centsBetween + (medianHz(events), lowHz)) < 10.0, + qPrintable(QString("the take's median pitch is %1 Hz; the " + "selection was sung at %2 and what came " + "after its end at %3") + .arg(medianHz(events)).arg(lowHz).arg(highHz))); + + // A second punch, over the singing that is now there: the + // selection is the consent, so nothing is asked (spec 5.1). It is + // the selection the playhead is in that is recorded into, and the + // lead-in of the pre-roll is not taken out of it. + const sv::sv_frame_t P2 = sv::sv_frame_t(1.2 * rate); + const sv::sv_frame_t E2 = sv::sv_frame_t(1.9 * rate); + setPreRollSeconds(0.5); + m_window->setPreRoll(true); + m_window->clearSelections(); + m_window->selectRange(sv::sv_frame_t(0.2 * rate), + sv::sv_frame_t(0.5 * rate)); + m_window->selectRange(P2, E2); + m_window->seekTo(sv::sv_frame_t(1.5 * rate)); // in the second one + m_window->clearRecordOverQuestions(); + startTake(); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->recordOverQuestions(), 0); + QCOMPARE(m_window->takePosition(), P2); + QCOMPARE(m_window->takeEnd(), E2); + QCOMPARE(m_window->takePreRoll(), sv::sv_frame_t(0.5 * rate)); + + QTRY_VERIFY_WITH_TIMEOUT(!m_window->recordTarget()->isRecording(), 5000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + // The two ranges join, and the second one reached its end: had the + // lead-in been counted as part of what was recorded, the take + // would have stopped short of it + ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], Coverage::Range(P, E2)); + } + + // Stop can still be pressed before the end of the selection; the take + // is then as short as what was sung, and nothing is left watching it + void punch_early_stop_is_shorter() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(false); + m_window->setRecordIntoSelection(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(1.0 * rate); + const sv::sv_frame_t E = sv::sv_frame_t(3.0 * rate); + m_window->selectRange(P, E); + m_window->seekTo(P + sv::sv_frame_t(0.5 * rate)); // inside it + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePosition(), P); + QCOMPARE(m_window->takeEnd(), E); + + QTest::qWait(700); + QVERIFY2(m_window->recordTarget()->isRecording(), + "the take stopped by itself well before the end of the " + "selection"); + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->takeTimerRunning()); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + QVERIFY2(ranges[0].end > P + sv::sv_frame_t(0.4 * rate) && + ranges[0].end < E, + qPrintable(QString("0.7 s was recorded into the selection " + "[%1,%2) before Stop, and the take covers " + "[%3,%4)") + .arg(P).arg(E) + .arg(ranges[0].start).arg(ranges[0].end))); + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(wave->getFrameCount(), ranges[0].end); + } + // The take and the live pitch model are both in the play source // while the reference plays (review finding 3), and neither is to be // heard. This test holds that in place for the whole path, from the From f952dbd222aed95b3746ab2452c717472f43bd42 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 02:12:41 +0300 Subject: [PATCH 034/275] feat: swap another audio file under a take's pitch and notes Rewriting a take's audio must not cost a whole-song analysis. The pitch and notes layers now survive the change of file: the analyser gives them up without deleting them, their models are given the new audio as their source model, and the analyser made for that audio claims them by the same scan a session restore uses. No analysis anywhere, and nothing is wired into Stop yet (phase 4c does that). Two things that scan needed: it lived inside addAnalyses(), which deferAnalysis skips, so it is claimExistingAnalyses() now and runs in the deferred case too; and it left a claimed pair unconnected, which a restored session suffered from as well. Co-Authored-By: Claude Opus 5 --- main/Analyser.cpp | 185 ++++++++++++++++--------- main/Analyser.h | 21 ++- main/MainWindow.cpp | 198 +++++++++++++++++++++++---- main/MainWindow.h | 22 ++- main/test/TestRecordWorkflow.h | 240 +++++++++++++++++++++++++++++++++ 5 files changed, 575 insertions(+), 91 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index e12f8537..41c430f9 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -164,6 +164,11 @@ Analyser::doAllAnalyses(bool withPitchTrack) if (withPitchTrack) { error = addAnalyses(); if (error != "") return error; + } else { + // No analysis to run, but pitch and notes layers may be there for + // us all the same: a session restore with auto-analysis switched + // off, or another audio file swapped under the layers of a take + (void)claimExistingAnalyses(false); } loadState(Audio); @@ -272,6 +277,35 @@ Analyser::removeAllLayers() fileClosed(); } +void +Analyser::releaseLayers() +{ + cerr << "Analyser::releaseLayers" << endl; + + // Before any model is released: see cancelAnalyses(). This also has + // to happen before the pitch and notes models are given another + // source model, which is what the caller does next + cancelAnalyses(); + + // The candidates are of the audio that is going + discardPitchCandidates(); + + // Only the waveform goes. Deleting it releases the audio model + // behind it, as nothing else holds a layer on it; the file stays on + // disk. deleteLayer(force=true) and not removeLayerFromView, for the + // reasons set out in removeAllLayers() + if (Layer *audio = m_layers[Audio]) { + m_layers[Audio] = nullptr; // before deleteLayer fires the slot + if (m_document) m_document->deleteLayer(audio, true); + } + + // The pitch and notes layers are simply forgotten: they stay in the + // pane with their events, and fileClosed() clears the rest of the + // state. From here this analyser owns nothing, so deleting it -- or + // deleting a layer it used to own -- disturbs nobody + fileClosed(); +} + bool Analyser::getDisplayFrequencyExtents(double &min, double &max) { @@ -487,56 +521,9 @@ Analyser::addAnalyses() return "Internal error: Analyser::addAnalyses() called with no model present"; } - // As with the spectrogram above, if these layers exist we use - // them -- but only if their source model matches our file model. - // When two analysers share a pane (dual-track mode), each must - // claim only the layers it created, not those from the other - // analyser. - TimeValueLayer *existingPitch = 0; - FlexiNoteLayer *existingNotes = 0; - for (int i = 0; i < m_pane->getLayerCount(); ++i) { - if (!existingPitch) { - TimeValueLayer *tvl = - qobject_cast(m_pane->getLayer(i)); - if (tvl) { - // Accept this layer only if its source model is our - // file model (or a model derived from our file model). - auto model = ModelById::get(tvl->getModel()); - if (model && (tvl->getModel() == m_fileModel || - model->getSourceModel() == m_fileModel)) { - existingPitch = tvl; - } - } - } - if (!existingNotes) { - FlexiNoteLayer *fnl = - qobject_cast(m_pane->getLayer(i)); - if (fnl) { - auto model = ModelById::get(fnl->getModel()); - if (model && (fnl->getModel() == m_fileModel || - model->getSourceModel() == m_fileModel)) { - existingNotes = fnl; - } - } - } - } - if (existingPitch && existingNotes) { - cerr << "recording existing pitch and notes layers (matching our file model)" << endl; - m_layers[PitchTrack] = existingPitch; - m_layers[Notes] = existingNotes; - return ""; - } else { - // Remove any mismatched layers we may have found for our model - // (partial state from a previous failed analysis run). - if (existingPitch) { - m_document->removeLayerFromView(m_pane, existingPitch); - m_layers[PitchTrack] = 0; - } - if (existingNotes) { - m_document->removeLayerFromView(m_pane, existingNotes); - m_layers[Notes] = 0; - } - } + // As with the spectrogram above, if these layers exist we use them + // rather than making another pair + if (claimExistingAnalyses(true)) return ""; TransformFactory *tf = TransformFactory::getInstance(); @@ -668,10 +655,8 @@ Analyser::addAnalyses() params->setPlayPan(1); params->setPlayGain(0.5); } - connect(pitchLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId))); } - + FlexiNoteLayer *flexiNoteLayer = qobject_cast(m_layers[Notes]); if (flexiNoteLayer) { @@ -681,17 +666,97 @@ Analyser::addAnalyses() params->setPlayPan(1); params->setPlayGain(0.5); } - connect(flexiNoteLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId))); - connect(flexiNoteLayer, SIGNAL(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), - this, SLOT(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float))); - connect(flexiNoteLayer, SIGNAL(materialiseReAnalysis()), - this, SLOT(materialiseReAnalysis())); } - + + connectAnalysisLayers(); + return ""; } +bool +Analyser::claimExistingAnalyses(bool removeMismatched) +{ + // Accept a layer only if its source model is our file model (or the + // layer is on our file model itself). When two analysers share a + // pane (dual-track mode), each must claim only its own layers, not + // those of the other analyser. + TimeValueLayer *existingPitch = 0; + FlexiNoteLayer *existingNotes = 0; + for (int i = 0; i < m_pane->getLayerCount(); ++i) { + if (!existingPitch) { + TimeValueLayer *tvl = + qobject_cast(m_pane->getLayer(i)); + if (tvl) { + auto model = ModelById::get(tvl->getModel()); + if (model && (tvl->getModel() == m_fileModel || + model->getSourceModel() == m_fileModel)) { + existingPitch = tvl; + } + } + } + if (!existingNotes) { + FlexiNoteLayer *fnl = + qobject_cast(m_pane->getLayer(i)); + if (fnl) { + auto model = ModelById::get(fnl->getModel()); + if (model && (fnl->getModel() == m_fileModel || + model->getSourceModel() == m_fileModel)) { + existingNotes = fnl; + } + } + } + } + + if (existingPitch && existingNotes) { + cerr << "recording existing pitch and notes layers (matching our file model)" << endl; + m_layers[PitchTrack] = existingPitch; + m_layers[Notes] = existingNotes; + connectAnalysisLayers(); + return true; + } + + if (removeMismatched) { + // Half a pair is partial state from a previous failed analysis + // run, and the caller is about to make the pair itself + if (existingPitch) { + m_document->removeLayerFromView(m_pane, existingPitch); + m_layers[PitchTrack] = 0; + } + if (existingNotes) { + m_document->removeLayerFromView(m_pane, existingNotes); + m_layers[Notes] = 0; + } + } + + return false; +} + +void +Analyser::connectAnalysisLayers() +{ + // Claimed layers need these as much as ones we made ourselves: the + // analyser they belonged to before is gone, and with it its + // connections. Unique, because an analyser handed the same layers + // twice would otherwise hear each signal twice + if (auto pitchLayer = qobject_cast(m_layers[PitchTrack])) { + connect(pitchLayer, SIGNAL(modelCompletionChanged(ModelId)), + this, SLOT(layerCompletionChanged(ModelId)), + Qt::UniqueConnection); + } + + if (auto noteLayer = qobject_cast(m_layers[Notes])) { + connect(noteLayer, SIGNAL(modelCompletionChanged(ModelId)), + this, SLOT(layerCompletionChanged(ModelId)), + Qt::UniqueConnection); + connect(noteLayer, SIGNAL(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), + this, SLOT(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), + Qt::UniqueConnection); + connect(noteLayer, SIGNAL(materialiseReAnalysis()), + this, SLOT(materialiseReAnalysis()), + Qt::UniqueConnection); + } +} + void Analyser::reAnalyseRegion(sv_frame_t frame0, sv_frame_t frame1, float freq0, float freq1) { diff --git a/main/Analyser.h b/main/Analyser.h index e481ebc3..22d8bee5 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -88,7 +88,15 @@ class Analyser : public QObject, // Unlike fileClosed(), this actually cleans up the view and the document // model registry so no orphan layers or models remain. void removeAllLayers(); - + + // Give up our layers without deleting the pitch and notes ones: they + // stay in the pane, with their events, for the analyser of another + // audio file to claim (see MainWindow::swapSingingAudio()). Only the + // waveform layer goes, which releases the audio model it shows, and + // then fileClosed() as above. This is for the singing analyser; the + // primary's spectrogram would be left in the pane as well. + void releaseLayers(); + void setIntelligentActions(bool); bool getDisplayFrequencyExtents(double &min, double &max); @@ -301,6 +309,17 @@ protected slots: QString addWaveform(); QString addAnalyses(); + // Claim the pitch and notes layers that are in the pane already and + // whose models come from our file model: the layers of a session just + // restored, or the ones another audio file has been swapped under. + // True only if both are there. An odd one out is removed from the + // pane when removeMismatched is set (the caller is about to make the + // pair itself) and left alone otherwise. + bool claimExistingAnalyses(bool removeMismatched); + + // Listen to the pitch and notes layers we have just claimed or made + void connectAnalysisLayers(); + void discardPitchCandidates(); // Document::LayerCreationHandler method diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 42795ad2..8d97d415 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2276,12 +2276,10 @@ MainWindow::loadSingingTrack(QString path) emit activity(tr("Load singing track \"%1\"").arg(path)); - // Record the pane count before opening so we can remove any extra panes - // that openPath(CreateAdditionalModel) creates via AddPaneCommand. - // We want both tracks to share pane 0, not appear in separate panes. - int paneCountBefore = m_paneStack ? m_paneStack->getPaneCount() : 0; - - FileOpenStatus status = openPath(path, CreateAdditionalModel); + ModelId singingModelId; + std::vector extraPanes; + FileOpenStatus status = + openSingingAudioFile(path, singingModelId, extraPanes); if (status == FileOpenFailed) { QMessageBox::critical(this, tr("Failed to open singing track"), @@ -2293,32 +2291,178 @@ MainWindow::loadSingingTrack(QString path) return; } - // modelAdded() fired synchronously inside openPath() and stored the new - // model's id in m_pendingSingingModelId. Set up the secondary analyser - // NOW, before pruning the extra pane: the imported WaveformLayer in that - // pane is the only layer referencing the singing model, so deleting it - // first would make Document::releaseModel() free the model before it can - // be analysed. Once m_analyser2's own WaveformLayer references the model - // the orphan can go. This also clears m_pendingSingingModelId, so the + // Set up the secondary analyser NOW, before pruning the extra pane: + // the imported WaveformLayer in that pane is the only layer + // referencing the singing model, so deleting it first would make + // Document::releaseModel() free the model before it can be analysed. + // Once m_analyser2's own WaveformLayer references the model the orphan + // can go. This also clears m_pendingSingingModelId, so the // analyseNewSingingModel() call queued by modelAdded() becomes a no-op. - ModelId singingModelId = m_pendingSingingModelId; analyseNewSingingModel(); - // openPath(CreateAdditionalModel) will have called AddPaneCommand which - // added a new pane for the singing track's waveform layer. We do NOT - // want that extra pane — both tracks must overlay pane 0. Remove any - // panes above the original count (except the time-ruler pane at index 1 - // which was already there). We delete from the top down so that index - // arithmetic stays valid. If the analyser setup above failed, nothing - // else references the singing model and pruning releases it, which is - // what we want. + // If the analyser setup above failed, nothing else references the + // singing model and pruning releases it, which is what we want. + for (Pane *extra : extraPanes) { + pruneExtraPane(extra, singingModelId); + } +} + +MainWindow::FileOpenStatus +MainWindow::openSingingAudioFile(QString path, sv::ModelId &modelId, + std::vector &extraPanes) +{ + // Record the pane count before opening so we can collect any extra + // panes that openPath(CreateAdditionalModel) creates via + // AddPaneCommand. We want both tracks to share pane 0, not appear in + // separate panes. + int paneCountBefore = m_paneStack ? m_paneStack->getPaneCount() : 0; + + FileOpenStatus status = openPath(path, CreateAdditionalModel); + + // The extra pane holds the singing track's own imported waveform + // layer, which we do not want either. It is not pruned here: that + // layer is the only reference to the new model until the caller has + // one of its own, and pruning releases a model nothing references. + // Collected from the top down, which is the order they are removed in if (m_paneStack) { - while (m_paneStack->getPaneCount() > paneCountBefore) { - Pane *extra = m_paneStack->getPane(m_paneStack->getPaneCount() - 1); - if (!extra) break; - pruneExtraPane(extra, singingModelId); + for (int i = m_paneStack->getPaneCount() - 1; + i >= paneCountBefore; --i) { + if (Pane *extra = m_paneStack->getPane(i)) { + extraPanes.push_back(extra); + } } } + + // modelAdded() fired synchronously inside openPath() and stored the + // new model's id in m_pendingSingingModelId. It is left there for + // analyseNewSingingModel() to take; a caller that sets its analyser up + // some other way has to clear it itself + modelId = m_pendingSingingModelId; + + return status; +} + +QString +MainWindow::swapSingingAudio(QString path) +{ + // The audio of a take is written again from scratch whenever a + // recording is spliced into it or a part of it is erased. Analysing + // the whole of it again would take as long as the song, so the pitch + // and notes layers stay as they are and the new file goes underneath + // them. + // + // What makes that possible: an Analyser claims a pitch or notes layer + // whose model has the analyser's own audio model as its source model. + // That is how a restored session's layers find their analyser; here we + // make it true of the new audio by hand and let the same scan do the + // rest. + + if (!m_document) return tr("There is no session to swap the audio of"); + + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane) return tr("There is no pane to swap the audio in"); + + if (!m_analyser2) { + return tr("There is no singing track to swap the audio of"); + } + + Layer *pitch = m_analyser2->getLayer(Analyser::PitchTrack); + Layer *notes = m_analyser2->getLayer(Analyser::Notes); + if (!pitch || !notes) { + return tr("The singing track has no pitch and notes layers to keep"); + } + + // What the swap must leave as it was. The analyser of the new audio + // starts from the settings the two analysers share, which know + // nothing of what a take has done to these layers + const Analyser::Component components[] = { + Analyser::Audio, Analyser::PitchTrack, Analyser::Notes + }; + const int componentCount = sizeof(components) / sizeof(components[0]); + bool visible[componentCount], audible[componentCount]; + for (int i = 0; i < componentCount; ++i) { + visible[i] = m_analyser2->isVisible(components[i]); + audible[i] = m_analyser2->isAudible(components[i]); + } + Layer *selected = pane->getSelectedLayer(); + + // 1. The new audio, opened beside the reference as Load Singing Track + // does. First, so that a file that cannot be read disturbs nothing + ModelId newAudio; + std::vector extraPanes; + FileOpenStatus status = openSingingAudioFile(path, newAudio, extraPanes); + + if (status != FileOpenSucceeded || newAudio.isNone()) { + for (Pane *extra : extraPanes) pruneExtraPane(extra, newAudio); + return tr("The file \"%1\" could not be opened as audio").arg(path); + } + + // Not the analysis we want: it would be of the whole file + m_pendingSingingModelId = {}; + + // 2. The old audio's waveform layer goes, and with it the old audio; + // the pitch and notes layers stay in the pane with their events. The + // analyser has nothing left to lose by being deleted + m_analyser2->releaseLayers(); + delete m_analyser2; + m_analyser2 = nullptr; + + // 3. Those layers' models come from the new audio now. Nothing reads + // their source model between the release above and here + for (ModelId id : { pitch->getModel(), notes->getModel() }) { + if (auto model = ModelById::get(id)) { + model->setSourceModel(newAudio); + } + } + + // 4. An analyser for the new audio, which claims the two layers. No + // analysis (deferAnalysis): the layers hold the analysis of all of the + // take but the range that has just changed, and the take's coverage is + // not "the whole of this file" either + bool wasRebuilding = m_rebuildingTakeAudio; + m_rebuildingTakeAudio = true; + setupSingingTrackAnalyser(newAudio, true); + m_rebuildingTakeAudio = wasRebuilding; + + // 5. The orphan waveform in the extra pane can go, now that the + // analyser's own waveform layer holds the new audio. If the setup + // failed it is released with the orphan, and the layers are left + // showing a take whose audio has gone + for (Pane *extra : extraPanes) pruneExtraPane(extra, newAudio); + + if (!m_analyser2) { + return tr("The singing track could not be set up on \"%1\"").arg(path); + } + + // 6. What the swap was not to change. On the layers themselves: + // setVisible() and setAudible() write to the shared settings, and + // neither the muting of a take nor the pane's own stacking is the + // user's wish about the reference + for (int i = 0; i < componentCount; ++i) { + Layer *layer = m_analyser2->getLayer(components[i]); + if (!layer) continue; + layer->setLayerDormant(pane, !visible[i]); + if (auto params = layer->getPlayParameters()) { + params->setPlayAudible(audible[i]); + } + } + pane->layerParametersChanged(); + + // The selected layer is the top layer of the pane as well, which is + // where the editing tools look for the layer they act on + if (selected && selected != pane->getSelectedLayer()) { + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i) == selected) { + m_paneStack->setCurrentLayer(pane, selected); + break; + } + } + } + + updateLayerStatuses(); + updateMenuStates(); + + return ""; } void @@ -2653,7 +2797,7 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal updateMenuStates(); if (deferAnalysis) { - emit activity(tr("Recording singing track — analysis will run when recording stops")); + emit activity(tr("Singing track set up, with no analysis of its own")); } else { emit activity(tr("Singing track loaded and analysis started")); } diff --git a/main/MainWindow.h b/main/MainWindow.h index 9b681ca0..b1766ddb 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -409,9 +409,9 @@ protected slots: virtual void setupToolbars(); // Helpers for the singing / second-track workflow. - // deferAnalysis=true skips pYIN; no caller needs that since a take is - // spliced into a finished audio file before it is analysed, but the - // model swap of phase 4 will. + // deferAnalysis=true skips pYIN: swapSingingAudio() uses it, as the + // pitch and notes layers it hands the new analyser are analysed + // already. The scan for existing layers still runs. virtual void setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis = false); virtual void teardownSingingTrackAnalyser(); @@ -433,6 +433,22 @@ protected slots: void loadBackgroundMusic(QString path); void teardownBackgroundMusic(); + // Put another audio file under the take's pitch and notes layers, + // keeping those layers and everything in them. The new audio is not + // analysed: what the layers hold is the analysis of all of the take + // but the range that has just changed. Returns "" on success, or a + // message for the user. (Phase 4c: Stop splices the recording into + // the take's audio, swaps to the file that comes out and analyses + // only the range the splice wrote.) + QString swapSingingAudio(QString path); + + // Open path as an additional audio model beside the reference, the + // way Load Singing Track does. The new model's id comes back in + // modelId and the extra panes openPath() made in extraPanes; those + // are the caller's to prune, once a layer of its own holds the model + FileOpenStatus openSingingAudioFile(QString path, sv::ModelId &modelId, + std::vector &extraPanes); + // Remove an extra pane created by openAudio()/record() in // CreateAdditionalModel mode: delete the orphan layer(s) showing // ownedModelId, detach shared layers (the time ruler) without deleting diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index f6845b24..9179d877 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -81,9 +81,15 @@ class TestMainWindow : public MainWindow FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } void doRecord() { record(); } + void doPlay() { play(); } // and again to stop void doAnalyseNow() { analyseNow(); } void doLoadBackgroundMusic(QString path) { loadBackgroundMusic(path); } + // Another audio file under the take's pitch and notes layers + QString doSwapSingingAudio(QString path) { + return swapSingingAudio(path); + } + // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } bool isDocumentModified() { return m_documentModified; } @@ -196,6 +202,10 @@ class TestRecordWorkflow : public QObject static constexpr double lowHz = 220.5; static constexpr double highHz = 294.0; + // A third tone for the audio a swap puts in, no harmonic of either of + // the other two, so that what is in the output says which file it is + static constexpr double swapHz = 490.0; + QTemporaryDir m_dir; int m_fileCounter = 0; TestMainWindow *m_window = nullptr; @@ -2002,6 +2012,236 @@ private slots: sv::sv_frame_t(1.2 * rate)); } + // Another audio file under the take's pitch and notes layers: the + // layers, their models and everything in them are the same objects + // afterwards, and no analysis is run + void swap_keeps_layers_and_events() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + Analyser *before = m_window->analyser2(); + sv::Layer *pitch = before->getLayer(Analyser::PitchTrack); + sv::Layer *notes = before->getLayer(Analyser::Notes); + sv::ModelId oldAudio = before->getMainModelId(); + sv::ModelId pitchModel = pitch->getModel(); + sv::ModelId notesModel = notes->getModel(); + auto events = pitchEvents(pitch); + QVERIFY(!events.empty()); + auto coverage = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(coverage.size()), 1); + + // Twice as long as the file it replaces, so that a coverage reset + // to "the whole of this file" would show + QString error = m_window->doSwapSingingAudio(writeWav(tone(lowHz, 2.0))); + QVERIFY2(error.isEmpty(), qPrintable(error)); + + Analyser *after = m_window->analyser2(); + QVERIFY(after); + QVERIFY(after != before); + QCOMPARE(after->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(after->getLayer(Analyser::Notes), notes); + QCOMPARE(pitch->getModel(), pitchModel); + QCOMPARE(notes->getModel(), notesModel); + auto kept = pitchEvents(pitch); + QCOMPARE(kept.size(), events.size()); + for (size_t i = 0; i < events.size(); ++i) { + QCOMPARE(kept[i].getFrame(), events[i].getFrame()); + QCOMPARE(kept[i].getValue(), events[i].getValue()); + } + + // The new audio is the analyser's, and the old one is released + sv::ModelId newAudio = after->getMainModelId(); + QVERIFY(newAudio != oldAudio); + auto wfm = sv::ModelById::getAs(newAudio); + QVERIFY(wfm); + QCOMPARE(wfm->getFrameCount(), sv::sv_frame_t(2.0 * rate)); + QVERIFY2(!sv::ModelById::get(oldAudio), + "the audio that was swapped out was not released"); + QCOMPARE(layersOnModel(newAudio), 1); + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + auto playing = m_window->playSource()->getModels(); + QVERIFY(!playing.count(oldAudio)); + QVERIFY(playing.count(newAudio)); + verifyPlaySourceClean(); + if (QTest::currentTestFailed()) return; + + // The two layers' models come from the new audio now: that is what + // let the new analyser claim them + QCOMPARE(sv::ModelById::get(pitchModel)->getSourceModel(), newAudio); + QCOMPARE(sv::ModelById::get(notesModel)->getSourceModel(), newAudio); + + // Nothing was analysed, then or when the queued calls ran + QVERIFY(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers()); + QCoreApplication::processEvents(); + QCOMPARE(m_window->analyser2(), after); + QVERIFY(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers()); + QCOMPARE(pitchEvents(pitch).size(), events.size()); + + // Colours, the toggles and the take's coverage as they were + QCOMPARE(colourOf(pitch), colourNamed("Orange")); + QCOMPARE(colourOf(notes), colourNamed("Bright Purple")); + QVERIFY(after->isVisible(Analyser::Audio)); + QVERIFY(after->isVisible(Analyser::PitchTrack)); + QVERIFY(after->isVisible(Analyser::Notes)); + QVERIFY(after->isAudible(Analyser::Audio)); + QVERIFY(m_window->playSingingAudioAction()->isChecked()); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], coverage[0]); + } + + // The swapped-in audio is what is heard afterwards, and the audio it + // replaced is not + void swap_plays_the_new_audio() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(TestSignals::sine(lowHz, rate, + int(2 * rate), 0.5))); + if (QTest::currentTestFailed()) return; + + m_window->loadSingingTrack + (writeWav(TestSignals::sine(highHz, rate, int(2 * rate), 0.5))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + QString error = m_window->doSwapSingingAudio + (writeWav(TestSignals::sine(swapHz, rate, int(2 * rate), 0.5))); + QVERIFY2(error.isEmpty(), qPrintable(error)); + + m_window->seekTo(0); + m_window->doPlay(); + QTest::qWait(1500); + m_window->doPlay(); + + auto output = m_window->fake()->getCapturedOutput(); + long start = m_window->fake()->getPlayStartFrame(); + QVERIFY2(start >= 0, "nothing was played"); + size_t from = size_t(start) + size_t(0.3 * rate); + double swapped = amplitudeAt(output, from, 22050, swapHz); + double replaced = amplitudeAt(output, from, 22050, highHz); + QVERIFY2(swapped > 0.05, + qPrintable(QString("the audio swapped in is not in the " + "output: amplitude %1").arg(swapped))); + QVERIFY2(replaced >= 0.0 && replaced < 0.005, + qPrintable(QString("the audio swapped out is still in the " + "output, at amplitude %1 (the new audio " + "is at %2)").arg(replaced).arg(swapped))); + } + + // Closing the session after a swap: the layers the released analyser + // used to own are deleted by the analyser that claimed them + void swap_then_close_session() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + sv::ModelId pitchModel = m_window->analyser2() + ->getLayer(Analyser::PitchTrack)->getModel(); + + QString error = m_window->doSwapSingingAudio(writeWav(tone(lowHz, 1.0))); + QVERIFY2(error.isEmpty(), qPrintable(error)); + sv::ModelId audio = m_window->analyser2()->getMainModelId(); + + m_window->doCloseSession(); + + QVERIFY(!m_window->analyser2()); + QVERIFY2(!sv::ModelById::get(audio), + "the swapped-in audio outlived the session"); + QVERIFY2(!sv::ModelById::get(pitchModel), + "the swapped-over pitch track outlived the session"); + QCOMPARE(m_window->paneStack()->getPaneCount(), 0); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + + // and the window still works + openReference(writeWav(tone(highHz, 1.0))); + } + + // Recording again after a swap. The layers the first analyser + // released are deleted while the one that claimed them is alive: what + // the next take does with them is not the released analyser's business + void swap_then_record_again() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + int panes = m_window->paneStack()->getPaneCount(); + + take(600); + if (QTest::currentTestFailed()) return; + QString first = m_window->takes()->getAudioPath(); + QVERIFY(!first.isEmpty()); + + // Standing in for the file a splice will write in phase 4c + QString error = m_window->doSwapSingingAudio(writeWav(tone(lowHz, 2.0))); + QVERIFY2(error.isEmpty(), qPrintable(error)); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + + m_window->seekTo(0); + take(600); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->analyser2()); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + QVERIFY(m_window->takes()->getAudioPath() != first); + QVERIFY(!m_window->recordingLayer()); + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + QCOMPARE(m_window->pendingExtraPaneCount(), 0); + verifyPlaySourceClean(); + } + + // The swap while pYIN is still running on the audio it replaces: the + // transforms are cancelled before any model changes hands. A + // regression guard, a crash is the failure + void swap_during_analysis() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + m_window->loadSingingTrack(writeWav(tone(highHz, 4.0))); + QVERIFY2(sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), + "the race was not set up: no analysis was running when the " + "swap started"); + Analyser *before = m_window->analyser2(); + QVERIFY(before); + sv::Layer *pitch = before->getLayer(Analyser::PitchTrack); + sv::Layer *notes = before->getLayer(Analyser::Notes); + QVERIFY(pitch && notes); + sv::ModelId oldAudio = before->getMainModelId(); + + QString error = m_window->doSwapSingingAudio(writeWav(tone(lowHz, 1.0))); + QVERIFY2(error.isEmpty(), qPrintable(error)); + + // The transform threads are gone by the time the swap returns, so + // nothing writes into the pitch track any more -- whatever pYIN + // had got to is what stays. (The factory only strikes a + // transformer off its list when the event loop next runs) + auto events = pitchEvents(pitch); + QTRY_VERIFY_WITH_TIMEOUT(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 10000); + QTest::qWait(300); + QCOMPARE(pitchEvents(pitch).size(), events.size()); + + Analyser *after = m_window->analyser2(); + QVERIFY(after); + QVERIFY(after != before); + QCOMPARE(after->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(after->getLayer(Analyser::Notes), notes); + QVERIFY2(!sv::ModelById::get(oldAudio), + "the audio that was swapped out was not released"); + QCoreApplication::processEvents(); + verifyPlaySourceClean(); + } + // Finding 7, the scenario itself: pitch candidates on the reference // are still the analyser's after another track has been loaded void reference_candidates_survive_load() { From 2335e0dbf1cf7f1bcb24cb745641f8074da08da3 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 02:54:08 +0300 Subject: [PATCH 035/275] feat: analyse one range and merge it into a take's pitch and notes Re-recording part of a take must cost an analysis of that part, not of the whole song. Analyser::analyseRange(start, end, clipStart, clipEnd) runs the same two pYIN transforms as a whole-file analysis, with the same settings-dependent parameters (which are now built in one place, buildAnalysisTransforms(), for both), over the range widened by half a second on each side and aligned to the 256-frame grid. The results go into temporary layers that are in the document but in no view, so nothing shows them, nothing selects them and the scan for existing analyses cannot take them for the analyser's own. When both are complete their events replace what was in the widened range -- notes that start in it deleted, one running into it from the left truncated -- the temporaries are deleted and initialAnalysisCompleted() is emitted. Nothing goes on the undo stack: an analysis result never did, and phase 6 makes the recording undoable as a whole. Nothing is wired into Stop yet (phase 4c does that). The time-stamp question of the spec's risk list, settled by a test that compares a ranged result with the same range of a whole-file one: the smoothed pitch track needs no correction, because it is a fixed-sample-rate output and the host rounds each feature onto the step-size grid of the whole file, but the notes output is variable-sample-rate and pYIN times a note by its frame number within the run, so the notes come back shifted to near zero and the start of the range has to be added back. That is also why the range is aligned to the grid: off it, the notes would land between the points a whole-file run puts them on. cancelAnalyses(), releaseLayers(), removeAllLayers(), fileClosed() and the destructor all abandon a running ranged analysis and take its temporaries with them -- a cancelled transform still sets its outputs' completion to 100, so the state is cleared before anything is cancelled. Two assertions in the 4a swap tests compared the new analyser's address with the deleted one's; with another suite churning the heap first, the new one is built at the same address. They now check the model the rebuilt analyser is on instead, which is the behaviour they were after. Co-Authored-By: Claude Opus 5 --- main/Analyser.cpp | 340 +++++++++++++++++++-- main/Analyser.h | 58 ++++ main/test/TestRecordWorkflow.h | 11 +- main/test/TestSingingAnalysis.h | 506 ++++++++++++++++++++++++++++++++ 4 files changed, 887 insertions(+), 28 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 41c430f9..df2a457e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -38,6 +38,8 @@ #include #include +#include + using std::vector; using std::cerr; using std::endl; @@ -51,7 +53,9 @@ Analyser::Analyser(ColorScheme colorScheme) : m_pane(0), m_currentCandidate(-1), m_candidatesVisible(false), - m_currentAsyncHandle(0) + m_currentAsyncHandle(0), + m_rangedStart(0), + m_rangedEnd(0) { QSettings settings; settings.beginGroup("LayerDefaults"); @@ -71,6 +75,10 @@ Analyser::Analyser(ColorScheme colorScheme) : Analyser::~Analyser() { + // A ranged analysis still running would go on writing into models + // the document is about to release, and its temporary layers would + // be left in the document with nobody left who knows what they are + discardRangedAnalysis(); } std::map @@ -214,6 +222,10 @@ Analyser::cancelAnalyses() // does nothing if the transform has already finished if (!modelId.isNone()) mtf->cancel(modelId); } + + // A ranged analysis is cancelled the same way, and there is then + // nothing worth merging from it, so its temporary layers go too + discardRangedAnalysis(); } void @@ -514,38 +526,20 @@ Analyser::addWaveform() } QString -Analyser::addAnalyses() +Analyser::buildAnalysisTransforms(Transforms &transforms) { auto waveFileModel = ModelById::getAs(m_fileModel); if (!waveFileModel) { - return "Internal error: Analyser::addAnalyses() called with no model present"; + return "Internal error: Analyser::buildAnalysisTransforms() called with no model present"; } - - // As with the spectrogram above, if these layers exist we use them - // rather than making another pair - if (claimExistingAnalyses(true)) return ""; TransformFactory *tf = TransformFactory::getInstance(); - + QString plugname = "pYIN"; QString base = "vamp:pyin:pyin:"; QString f0out = "smoothedpitchtrack"; QString noteout = "notes"; - Transforms transforms; - -/*!!! we could have more than one pitch track... - QString cx = "vamp:cepstral-pitchtracker:cepstral-pitchtracker:f0"; - if (tf->haveTransform(cx)) { - Transform tx = tf->getDefaultTransformFor(cx); - TimeValueLayer *lx = qobject_cast - (m_document->createDerivedLayer(tx, m_fileModel)); - lx->setVerticalScale(TimeValueLayer::AutoAlignScale); - lx->setBaseColour(ColourDatabase::getInstance()->getColourIndex(tr("Bright Red"))); - m_document->addLayerToView(m_pane, lx); - } -*/ - QString notFound = tr("Transform \"%1\" not found. Unable to analyse audio file.

Is the %2 Vamp plugin correctly installed?"); if (!tf->haveTransform(base + f0out)) { return notFound.arg(base + f0out).arg(plugname); @@ -558,7 +552,7 @@ Analyser::addAnalyses() settings.beginGroup("Analyser"); bool precise = false, lowamp = true, onset = true, prune = true; - + std::map flags { { "precision-analysis", precise }, { "lowamp-analysis", lowamp }, @@ -567,13 +561,13 @@ Analyser::addAnalyses() }; auto keyMap = getAnalysisSettings(); - + for (auto p: flags) { auto ki = keyMap.find(p.first); if (ki != keyMap.end()) { p.second = settings.value(ki->first, ki->second).toBool(); } else { - throw std::logic_error("Internal error: One or more analysis settings keys not found in map: check addAnalyses and getAnalysisSettings"); + throw std::logic_error("Internal error: One or more analysis settings keys not found in map: check buildAnalysisTransforms and getAnalysisSettings"); } } @@ -619,9 +613,40 @@ Analyser::addAnalyses() transforms.push_back(t); t.setOutput(noteout); - + transforms.push_back(t); + return ""; +} + +QString +Analyser::addAnalyses() +{ + auto waveFileModel = ModelById::getAs(m_fileModel); + if (!waveFileModel) { + return "Internal error: Analyser::addAnalyses() called with no model present"; + } + + // As with the spectrogram above, if these layers exist we use them + // rather than making another pair + if (claimExistingAnalyses(true)) return ""; + + Transforms transforms; + QString error = buildAnalysisTransforms(transforms); + if (error != "") return error; + +/*!!! we could have more than one pitch track... + QString cx = "vamp:cepstral-pitchtracker:cepstral-pitchtracker:f0"; + if (tf->haveTransform(cx)) { + Transform tx = tf->getDefaultTransformFor(cx); + TimeValueLayer *lx = qobject_cast + (m_document->createDerivedLayer(tx, m_fileModel)); + lx->setVerticalScale(TimeValueLayer::AutoAlignScale); + lx->setBaseColour(ColourDatabase::getInstance()->getColourIndex(tr("Bright Red"))); + m_document->addLayerToView(m_pane, lx); + } +*/ + std::vector layers = m_document->createDerivedLayers(transforms, m_fileModel); @@ -883,6 +908,258 @@ Analyser::reAnalyseSelection(Selection sel, FrequencyRange range) return ""; } +QString +Analyser::analyseRange(sv_frame_t start, sv_frame_t end, + sv_frame_t clipStart, sv_frame_t clipEnd) +{ + auto waveFileModel = ModelById::getAs(m_fileModel); + if (!waveFileModel) { + return "Internal error: Analyser::analyseRange() called with no model present"; + } + if (!m_document || !m_pane) { + return "Internal error: Analyser::analyseRange() called with no document or pane present"; + } + if (!m_layers[PitchTrack] || !m_layers[Notes]) { + return "Internal error: Analyser::analyseRange() called with no pitch track and notes to merge into"; + } + + // One at a time. A second call means the audio under us has been + // replaced again, so the first run's result is of no use to anyone + discardRangedAnalysis(); + + sv_samplerate_t rate = waveFileModel->getSampleRate(); + + if (clipStart < 0) clipStart = 0; + if (clipEnd < 0 || clipEnd > waveFileModel->getEndFrame()) { + clipEnd = waveFileModel->getEndFrame(); + } + if (start < clipStart) start = clipStart; + if (end > clipEnd) end = clipEnd; + if (end <= start) return ""; + + // Half a second of context on each side, so that a note at an edge + // is found whole -- but never outside the material the caller says + // is there to analyse (the take's coverage) + sv_frame_t margin = sv_frame_t(rate / 2); + sv_frame_t from = std::max(clipStart, start - margin); + sv_frame_t to = std::min(clipEnd, end + margin); + + // Aligned to the 256-frame grid, as reAnalyseSelection() does. The + // step size is the grid a whole-file run's results sit on, and the + // notes of a ranged run are placed relative to its first block (see + // mergeRangedAnalysis()), so only a start on the grid puts them + // where a whole-file run would have put them + const sv_frame_t grid = 256; + from = (from / grid) * grid; + to = ((to + grid - 1) / grid) * grid; + if (to <= from) return ""; + + Transforms transforms; + QString error = buildAnalysisTransforms(transforms); + if (error != "") return error; + + RealTime startTime = RealTime::frame2RealTime(from, rate); + RealTime duration = RealTime::frame2RealTime(to - from, rate); + + for (Transform &t : transforms) { + t.setStartTime(startTime); + t.setDuration(duration); + } + + cerr << "Analyser::analyseRange: " << start << " to " << end + << ", widened and aligned to " << from << " to " << to + << " (clip " << clipStart << " to " << clipEnd << ")" << endl; + + // The temporary layers are registered with the document, so that + // deleting them releases their models, but they go into no view + std::vector layers = + m_document->createDerivedLayers(transforms, m_fileModel); + + for (Layer *layer : layers) { + m_rangedLayers.push_back(layer); + ModelId id = layer->getModel(); + // By model and not by layer type: all we want is the events + if (ModelById::getAs(id)) { + m_rangedNotesModel = id; + } else if (ModelById::getAs(id)) { + m_rangedPitchModel = id; + } + } + + if (m_rangedPitchModel.isNone() || m_rangedNotesModel.isNone()) { + discardRangedAnalysis(); + return tr("Transform \"pYIN\" did not run correctly (no pitch track and notes for the range)"); + } + + m_rangedStart = from; + m_rangedEnd = to; + + for (ModelId id : { m_rangedPitchModel, m_rangedNotesModel }) { + auto model = ModelById::get(id); + if (!model) continue; + // Emitted on the transform's own thread, so delivered here as a + // queued call: the merge happens on this thread like any other + connect(model.get(), SIGNAL(completionChanged(ModelId)), + this, SLOT(rangedAnalysisCompletionChanged(ModelId))); + } + + // createDerivedLayers() returns only once the transform has set both + // outputs' completion to 0, so no signal can have been missed above. + // A very short range could have finished by now all the same + rangedAnalysisCompletionChanged({}); + + return ""; +} + +void +Analyser::rangedAnalysisCompletionChanged(ModelId) +{ + if (m_rangedLayers.empty()) return; + + auto newPitch = ModelById::getAs(m_rangedPitchModel); + auto newNotes = ModelById::getAs(m_rangedNotesModel); + + if (!newPitch || !newNotes) { + cerr << "Analyser::rangedAnalysisCompletionChanged: a temporary model " + << "has gone, merging nothing" << endl; + discardRangedAnalysis(); + return; + } + + // A transform sets its outputs' completion to 100 whether it ran to + // the end or was abandoned, but an abandoned one is cancelled and + // discarded from here (see discardRangedAnalysis()), so completion + // at 100 in both means a result + if (!newPitch->isReady() || !newNotes->isReady()) return; + + mergeRangedAnalysis(); +} + +void +Analyser::mergeRangedAnalysis() +{ + auto newPitch = ModelById::getAs(m_rangedPitchModel); + auto newNotes = ModelById::getAs(m_rangedNotesModel); + + auto pitch = m_layers[PitchTrack] ? + ModelById::getAs(m_layers[PitchTrack]->getModel()) : + nullptr; + auto notes = m_layers[Notes] ? + ModelById::getAs(m_layers[Notes]->getModel()) : nullptr; + + if (!newPitch || !newNotes || !pitch || !notes) { + cerr << "Analyser::mergeRangedAnalysis: a model has gone, " + << "merging nothing" << endl; + discardRangedAnalysis(); + return; + } + + EventVector newPitchEvents = newPitch->getAllEvents(); + EventVector newNoteEvents; + + // The time-stamp question of spec section 11, settled by + // TestSingingAnalysis::ranged_matches_whole_file: the smoothed pitch + // track needs no correction, because it is a fixed-sample-rate + // output and the host rounds each feature to the nearest multiple of + // the step size of the whole file. The notes output is + // variable-sample-rate, and pYIN times a note by its frame number + // *within this run* (PYinVamp::addNoteFeatures()), so for a run that + // did not start at the beginning of the file the notes come back + // shifted to near zero + for (const Event &e : newNotes->getAllEvents()) { + newNoteEvents.push_back(e.withFrame(e.getFrame() + m_rangedStart)); + } + + // What the merge replaces is the widened range, extended to cover + // whatever the run actually produced: pYIN stamps a block a quarter + // of a block in (two hops here), so its events start a little after + // the range and end a little after it too. Clearing exactly the + // span they occupy leaves neither a stale event where a new one + // goes nor two events on one frame + sv_frame_t pitchFrom = m_rangedStart, pitchTo = m_rangedEnd; + if (!newPitchEvents.empty()) { + pitchFrom = std::min(pitchFrom, newPitchEvents.front().getFrame()); + pitchTo = std::max(pitchTo, newPitchEvents.back().getFrame() + 1); + } + + sv_frame_t noteFrom = m_rangedStart, noteTo = m_rangedEnd; + for (const Event &e : newNoteEvents) { + noteFrom = std::min(noteFrom, e.getFrame()); + noteTo = std::max(noteTo, e.getFrame() + 1); + } + + cerr << "Analyser::mergeRangedAnalysis: " << newPitchEvents.size() + << " pitch event(s) and " << newNoteEvents.size() << " note(s) for " + << m_rangedStart << " to " << m_rangedEnd << "; replacing pitch in " + << pitchFrom << " to " << pitchTo << ", notes in " << noteFrom + << " to " << noteTo << endl; + + for (const Event &e : + pitch->getEventsStartingWithin(pitchFrom, pitchTo - pitchFrom)) { + pitch->remove(e); + } + for (const Event &e : newPitchEvents) { + pitch->add(e); + } + + // A note that starts in the range goes; one that runs into the range + // from the left is cut back to the edge of it, and the new notes + // supply everything from there on. A note that reached out of the + // far end of the range loses that tail: the run has analysed the + // material there and says what is in it + for (const Event &e : notes->getAllEvents()) { + sv_frame_t f = e.getFrame(); + if (f >= noteFrom && f < noteTo) { + notes->remove(e); + } else if (f < noteFrom && f + e.getDuration() > noteFrom) { + notes->remove(e); + notes->add(e.withDuration(noteFrom - f)); + } + } + for (const Event &e : newNoteEvents) { + notes->add(e); + } + + // The events are in the models, not in a command: an analysis result + // never went onto the undo stack. Phase 6 makes the recording that + // asked for this analysis undoable as a whole + discardRangedAnalysis(); + + emit initialAnalysisCompleted(); +} + +void +Analyser::discardRangedAnalysis() +{ + if (m_rangedLayers.empty()) { + m_rangedPitchModel = {}; + m_rangedNotesModel = {}; + return; + } + + // Cleared before anything is cancelled or deleted: a completion + // signal that arrives from a transform we are abandoning then finds + // nothing to merge, and deleteLayer() below reaches + // layerAboutToBeDeleted() with the layers already forgotten + std::vector doomed; + doomed.swap(m_rangedLayers); + ModelId pitchId = m_rangedPitchModel, notesId = m_rangedNotesModel; + m_rangedPitchModel = {}; + m_rangedNotesModel = {}; + + // Before the models are released: see cancelAnalyses() + auto mtf = ModelTransformerFactory::getInstance(); + for (ModelId id : { pitchId, notesId }) { + if (!id.isNone()) mtf->cancel(id); + } + + for (Layer *layer : doomed) { + // As in removeAllLayers(): force, and no removeLayerFromView, + // so that nothing of this is left on the undo stack + if (m_document) m_document->deleteLayer(layer, true); + } +} + bool Analyser::arePitchCandidatesShown() const { @@ -1125,6 +1402,17 @@ Analyser::layerAboutToBeDeleted(Layer *doomed) m_currentCandidate = -1; } + // A temporary layer of a ranged analysis, deleted by someone else -- + // we clear m_rangedLayers before deleting them ourselves + auto r = std::find(m_rangedLayers.begin(), m_rangedLayers.end(), doomed); + if (r != m_rangedLayers.end()) { + m_rangedLayers.erase(r); + if (m_rangedLayers.empty()) { + m_rangedPitchModel = {}; + m_rangedNotesModel = {}; + } + } + // A layer of ours deleted by someone else, e.g. by a command // dropped from the undo history for (auto &entry : m_layers) { diff --git a/main/Analyser.h b/main/Analyser.h index 22d8bee5..3396f3a7 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -176,6 +176,39 @@ class Analyser : public QObject, */ QString reAnalyseSelection(sv::Selection sel, FrequencyRange range); + /** + * Analyse the frames from start to end of our audio file with the + * same transforms and parameters as a whole-file analysis, and + * merge the result into the pitch track and notes we already have + * (which must both be there: this analyser has either made them or + * claimed them). The range is widened by half a second on each + * side, so that a note at an edge is found whole, but never outside + * clipStart..clipEnd -- the caller passes the extent of the + * material that is there to be analysed (the take's coverage; the + * Analyser knows nothing of Coverage itself). A clipEnd below zero + * means the end of the file. + * + * The analysis runs into temporary layers that are in no view; + * when both are complete their events replace what was in the + * widened range and initialAnalysisCompleted() is emitted. The + * merge is not undoable: an analysis result never was. + * + * Returns "" if a run was started (or there was nothing to do), or + * a user-readable error string. A second call while one is running + * abandons the first: its material is presumed to have changed. + */ + QString analyseRange(sv::sv_frame_t start, sv::sv_frame_t end, + sv::sv_frame_t clipStart = 0, + sv::sv_frame_t clipEnd = -1); + + /** + * Return true between the start of a ranged analysis and the merge + * (or the abandonment) of its result. + */ + bool isAnalysingRange() const { + return !m_rangedLayers.empty(); + } + /** * Return true if the analysed pitch candidates are currently * visible (they are hidden from the call to reAnalyseSelection @@ -283,6 +316,7 @@ protected slots: void layerCompletionChanged(sv::ModelId); void reAnalyseRegion(sv::sv_frame_t, sv::sv_frame_t, float, float); void materialiseReAnalysis(); + void rangedAnalysisCompletionChanged(sv::ModelId); protected: ColorScheme m_colorScheme; @@ -303,12 +337,36 @@ protected slots: sv::Document::LayerCreationAsyncHandle m_currentAsyncHandle; QMutex m_asyncMutex; + // A ranged analysis in progress (analyseRange()). The layers are + // registered with the document but added to no view, so nothing + // shows them, nothing selects them and claimExistingAnalyses(), + // which looks in the pane, never takes them for ours + std::vector m_rangedLayers; + sv::ModelId m_rangedPitchModel; + sv::ModelId m_rangedNotesModel; + sv::sv_frame_t m_rangedStart; // widened, grid-aligned: what is analysed + sv::sv_frame_t m_rangedEnd; // and what the merge replaces + QString doAllAnalyses(bool withPitchTrack); QString addVisualisations(); QString addWaveform(); QString addAnalyses(); + // The two pYIN transforms of a full analysis (smoothed pitch track + // and notes) with the parameters the settings ask for. Shared with + // analyseRange(), which must produce within its range just what a + // whole-file analysis produces. "" on success, else an error string + QString buildAnalysisTransforms(sv::Transforms &transforms); + + // Merge a finished ranged analysis into the pitch and notes models + // and delete the temporaries + void mergeRangedAnalysis(); + + // Stop a ranged analysis if one is running and delete its + // temporary layers and models, merging nothing + void discardRangedAnalysis(); + // Claim the pitch and notes layers that are in the pane already and // whose models come from our file model: the layers of a session just // restored, or the ones another audio file has been swapped under. diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 9179d877..5e5d5f45 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -2041,7 +2041,10 @@ private slots: Analyser *after = m_window->analyser2(); QVERIFY(after); - QVERIFY(after != before); + // Not "after != before": the old analyser has been deleted, and a + // new one may legitimately be built at the same address (which it + // is, once another suite has churned the heap first). That the + // analyser was rebuilt shows in the model it is on, below QCOMPARE(after->getLayer(Analyser::PitchTrack), pitch); QCOMPARE(after->getLayer(Analyser::Notes), notes); QCOMPARE(pitch->getModel(), pitchModel); @@ -2233,9 +2236,13 @@ private slots: Analyser *after = m_window->analyser2(); QVERIFY(after); - QVERIFY(after != before); + // Not "after != before": the old analyser has been deleted, and a + // new one may legitimately be built at the same address (which it + // is, once another suite has churned the heap first). That the + // analyser was rebuilt shows in the model it is on, below QCOMPARE(after->getLayer(Analyser::PitchTrack), pitch); QCOMPARE(after->getLayer(Analyser::Notes), notes); + QVERIFY(after->getMainModelId() != oldAudio); QVERIFY2(!sv::ModelById::get(oldAudio), "the audio that was swapped out was not released"); QCoreApplication::processEvents(); diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 0d720016..38f6d0c8 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -36,6 +36,7 @@ #include "layer/WaveformLayer.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" +#include "data/model/NoteModel.h" #include "base/PlayParameters.h" #include @@ -46,6 +47,7 @@ #include #include +#include #include #include @@ -103,6 +105,50 @@ class TestSingingAnalysis : public QObject return data; } + static void appendSilence(std::vector &data, double seconds) { + data.insert(data.end(), size_t(seconds * rate), 0.f); + } + + // Four notes with 0.6 s of silence between them, 5 s in all: the + // gaps are wide enough that a range with half a second of margin on + // each side can sit over the third note and still leave the second + // and fourth untouched. Each pitch has a whole number of samples per + // period, as above (150, 200, 180 and 140 samples) + static std::vector fourNotes() { + std::vector data; + for (double hz : { 294.0, 220.5, 245.0, 315.0 }) { + appendSilence(data, 0.6); + auto t = tone(hz, 0.5); + data.insert(data.end(), t.begin(), t.end()); + } + appendSilence(data, 0.6); + return data; + } + + // The third note of fourNotes() is at 2.8 to 3.3 s; this range holds + // it whole and is nowhere near frame 0. Widened by half a second it + // runs from 2.25 to 3.85 s, so that both of its edges are in silence + // and the notes on either side are left out of it altogether + static constexpr double thirdNoteFrom = 2.75; + static constexpr double thirdNoteTo = 3.35; + + // The range analyseRange() really covers: half a second either side + // of the range asked for, clipped to the coverage, then out to the + // 256-frame grid + static void widenRange(sv::sv_frame_t start, sv::sv_frame_t end, + sv::sv_frame_t clipStart, sv::sv_frame_t clipEnd, + sv::sv_frame_t &from, sv::sv_frame_t &to) { + sv::sv_frame_t margin = sv::sv_frame_t(rate / 2); + from = std::max(clipStart, start - margin); + to = std::min(clipEnd, end + margin); + from = (from / hop) * hop; + to = ((to + hop - 1) / hop) * hop; + } + + static sv::sv_frame_t frameAt(double seconds) { + return sv::sv_frame_t(seconds * rate); + } + // Run the analyser on the model and wait for pYIN. If startFrame // is non-zero, apply it between layer setup and analysis, which // is what MainWindow::analyseNow() does after a take. @@ -144,6 +190,91 @@ class TestSingingAnalysis : public QObject return model->getAllEvents(); } + static sv::EventVector noteEvents(Analyser &analyser) { + sv::Layer *layer = analyser.getLayer(Analyser::Notes); + if (!layer) return {}; + auto model = sv::ModelById::getAs(layer->getModel()); + if (!model) return {}; + return model->getAllEvents(); + } + + // The singing track as 4c will set it up for the first recording of + // a take: empty pitch and notes layers on the singing model, which a + // deferred analyser claims without analysing anything (4a) + void addEmptyAnalyses(sv::ModelId singing) { + for (auto type : { sv::LayerFactory::TimeValues, + sv::LayerFactory::FlexiNotes }) { + sv::Layer *layer = m_document->createEmptyLayer(type); + QVERIFY(layer); + auto model = sv::ModelById::get(layer->getModel()); + QVERIFY(model); + model->setSourceModel(singing); + m_document->addLayerToView(m_pane, layer); + } + } + + // Wait for a ranged analysis to be merged + void waitForRange(Analyser &analyser, QSignalSpy &done) { + QVERIFY2(done.count() > 0 || done.wait(30000), + "the ranged analysis did not complete within 30 seconds"); + QVERIFY2(!analyser.isAnalysingRange(), + "the analyser still says a range is being analysed"); + } + + // Nothing of a ranged analysis may be left behind in the document + void verifyNothingLeftOver(size_t layersBefore, size_t modelsBefore) { + QCOMPARE(m_document->getLayers().size(), layersBefore); + QCOMPARE(m_document->getModels().size(), modelsBefore); + } + + // Pitch events whose frame is within [from, to), by frame + static std::map pitchIn(const sv::EventVector &events, + sv::sv_frame_t from, + sv::sv_frame_t to) { + std::map m; + for (const auto &e : events) { + if (e.getFrame() >= from && e.getFrame() < to) { + m[e.getFrame()] = e.getValue(); + } + } + return m; + } + + static sv::EventVector outsidePitch(const sv::EventVector &events, + sv::sv_frame_t from, + sv::sv_frame_t to) { + sv::EventVector out; + for (const auto &e : events) { + if (e.getFrame() < from || e.getFrame() >= to) out.push_back(e); + } + return out; + } + + // Notes that start within [from, to) + static sv::EventVector notesIn(const sv::EventVector &events, + sv::sv_frame_t from, sv::sv_frame_t to) { + sv::EventVector out; + for (const auto &e : events) { + if (e.getFrame() >= from && e.getFrame() < to) out.push_back(e); + } + return out; + } + + // Notes that lie wholly outside [from, to), so that the merge has no + // business with them at all + static sv::EventVector outsideNotes(const sv::EventVector &events, + sv::sv_frame_t from, + sv::sv_frame_t to) { + sv::EventVector out; + for (const auto &e : events) { + if (e.getFrame() + e.getDuration() <= from || + e.getFrame() >= to) { + out.push_back(e); + } + } + return out; + } + static double medianHz(const sv::EventVector &events) { std::vector values; for (const auto &e : events) values.push_back(e.getValue()); @@ -457,6 +588,381 @@ private slots: QVERIFY(primary.getLayer(Analyser::PitchTrack) == primaryPitch); QVERIFY(m_document->getLayers().count(primaryPitch) > 0); } + + void ranged_matches_whole_file() { + // The time-stamp question of spec section 11: within its range a + // ranged analysis must give what a whole-file analysis gives. + // Also the "nothing in the models yet" case: the ranged result + // is all there is afterwards + auto data = fourNotes(); + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t start = frameAt(thirdNoteFrom); + sv::sv_frame_t end = frameAt(thirdNoteTo); + sv::sv_frame_t from, to; + widenRange(start, end, 0, fileEnd, from, to); + + sv::EventVector wholePitch, wholeNotes; + { + Analyser whole(Analyser::SecondaryColors); + analyse(whole, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + wholePitch = pitchEvents(whole); + wholeNotes = noteEvents(whole); + whole.removeAllLayers(); + } + QCOMPARE(int(wholeNotes.size()), 4); + + sv::ModelId singing = addSingingModel(data); + addEmptyAnalyses(singing); + if (QTest::currentTestFailed()) return; + + Analyser analyser(Analyser::SecondaryColors); + QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, + m_pane, true), QString()); + QVERIFY(analyser.getLayer(Analyser::PitchTrack)); + QVERIFY(analyser.getLayer(Analyser::Notes)); + QVERIFY(pitchEvents(analyser).empty()); + QVERIFY(noteEvents(analyser).empty()); + + size_t layers = m_document->getLayers().size(); + size_t models = m_document->getModels().size(); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + QCOMPARE(done.count(), 1); + verifyNothingLeftOver(layers, models); + + // Nothing may have landed outside the range that was analysed: + // this is what fails if the time stamps of an output are relative + // to the start of the run rather than to the start of the file + for (const auto &e : pitchEvents(analyser)) { + QVERIFY2(e.getFrame() >= from && e.getFrame() < to + 2 * hop, + qPrintable(QString("pitch event at %1, outside the " + "analysed %2 to %3") + .arg(e.getFrame()).arg(from).arg(to))); + } + for (const auto &e : noteEvents(analyser)) { + QVERIFY2(e.getFrame() >= from && e.getFrame() < to + 2 * hop, + qPrintable(QString("note at %1, outside the analysed " + "%2 to %3") + .arg(e.getFrame()).arg(from).arg(to))); + } + + // Both runs place their events on the grid of the step size: the + // pitch track because its output is at a fixed sample rate of one + // per step, the notes because the range analysed starts on that + // grid (take that away and the notes land between the grid points) + for (const auto &e : wholePitch) QCOMPARE(e.getFrame() % hop, 0); + for (const auto &e : wholeNotes) QCOMPARE(e.getFrame() % hop, 0); + for (const auto &e : pitchEvents(analyser)) { + QCOMPARE(e.getFrame() % hop, 0); + } + for (const auto &e : noteEvents(analyser)) { + QCOMPARE(e.getFrame() % hop, 0); + } + + // The pitch track within the range asked for, frame by frame + auto a = pitchIn(wholePitch, start, end); + auto b = pitchIn(pitchEvents(analyser), start, end); + QVERIFY2(b.size() > 60, + qPrintable(QString("only %1 ranged pitch events in the range") + .arg(b.size()))); + + int odd = 0; + double worst = 0.0; + for (const auto &p : a) { + auto i = b.find(p.first); + if (i == b.end()) { ++odd; continue; } + double cents = std::abs(TestSignals::centsBetween(i->second, + p.second)); + if (cents > worst) worst = cents; + } + for (const auto &p : b) { + if (a.find(p.first) == a.end()) ++odd; + } + // Measured: 0 frames in one run and not the other, 0 cents apart. + // A few frames of slack at the voiced edges, where pYIN's HMM has + // less to go on in the shorter run + QVERIFY2(odd <= 6, + qPrintable(QString("%1 of %2 frames are in one run and not " + "the other").arg(odd).arg(a.size()))); + QVERIFY2(worst < 20.0, + qPrintable(QString("worst pitch difference %1 cents") + .arg(worst))); + + // And the notes that start within it + sv::EventVector an = notesIn(wholeNotes, start, end); + sv::EventVector bn = notesIn(noteEvents(analyser), start, end); + QCOMPARE(int(an.size()), 1); + QCOMPARE(bn.size(), an.size()); + for (size_t i = 0; i < an.size(); ++i) { + QVERIFY2(std::abs(bn[i].getFrame() - an[i].getFrame()) <= 2 * hop, + qPrintable(QString("note starts at %1, whole-file run " + "had %2") + .arg(bn[i].getFrame()).arg(an[i].getFrame()))); + QVERIFY2(std::abs(bn[i].getDuration() - an[i].getDuration()) + <= 2 * hop, + qPrintable(QString("note lasts %1, whole-file run had %2") + .arg(bn[i].getDuration()) + .arg(an[i].getDuration()))); + QVERIFY2(std::abs(TestSignals::centsBetween(bn[i].getValue(), + an[i].getValue())) + < 20.0, + qPrintable(QString("note at %1 Hz, whole-file run had %2") + .arg(bn[i].getValue()).arg(an[i].getValue()))); + } + } + + void ranged_leaves_the_rest_alone() { + // A whole-file analysis, then the same audio analysed again over + // one range: outside the widened range not one event may move + auto data = fourNotes(); + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t start = frameAt(thirdNoteFrom); + sv::sv_frame_t end = frameAt(thirdNoteTo); + sv::sv_frame_t from, to; + widenRange(start, end, 0, fileEnd, from, to); + + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + + sv::EventVector wasPitch = pitchEvents(analyser); + sv::EventVector wasNotes = noteEvents(analyser); + QCOMPARE(int(wasNotes.size()), 4); + + size_t layers = m_document->getLayers().size(); + size_t models = m_document->getModels().size(); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + verifyNothingLeftOver(layers, models); + + // pYIN stamps a block a quarter of a block in (two hops here), so + // its events reach a couple of hops past the end of the range + sv::EventVector wasOut = outsidePitch(wasPitch, from, to + 2 * hop); + sv::EventVector isOut = outsidePitch(pitchEvents(analyser), + from, to + 2 * hop); + QVERIFY2(wasOut.size() > 100, + "the range covers too much of the file to test this"); + QCOMPARE(isOut.size(), wasOut.size()); + for (size_t i = 0; i < wasOut.size(); ++i) { + QCOMPARE(isOut[i].getFrame(), wasOut[i].getFrame()); + QCOMPARE(isOut[i].getValue(), wasOut[i].getValue()); + } + + // The notes of the file that lie wholly outside: the first two + // (0.6 to 1.1 s and 1.7 to 2.2 s) and the last (3.9 to 4.4 s) + sv::EventVector wasOutN = outsideNotes(wasNotes, from, to + 2 * hop); + sv::EventVector isOutN = outsideNotes(noteEvents(analyser), + from, to + 2 * hop); + QCOMPARE(int(wasOutN.size()), 3); + QCOMPARE(isOutN.size(), wasOutN.size()); + for (size_t i = 0; i < wasOutN.size(); ++i) { + QCOMPARE(isOutN[i].getFrame(), wasOutN[i].getFrame()); + QCOMPARE(isOutN[i].getDuration(), wasOutN[i].getDuration()); + QCOMPARE(isOutN[i].getValue(), wasOutN[i].getValue()); + } + + // and the third note is still one note, near enough where it was + sv::EventVector was3 = notesIn(wasNotes, from, to + 2 * hop); + sv::EventVector is3 = notesIn(noteEvents(analyser), from, to + 2 * hop); + QCOMPARE(int(was3.size()), 1); + QCOMPARE(is3.size(), was3.size()); + QVERIFY(std::abs(is3[0].getFrame() - was3[0].getFrame()) <= 2 * hop); + } + + void ranged_truncates_note_across_the_edge() { + // A note that runs into the widened range from the left is cut + // back to its edge; the analysis supplies what is inside + std::vector data; + appendSilence(data, 0.3); + auto lng = tone(singingHz, 2.0); + data.insert(data.end(), lng.begin(), lng.end()); + appendSilence(data, 0.3); + auto second = tone(referenceHz, 0.5); + data.insert(data.end(), second.begin(), second.end()); + appendSilence(data, 0.3); + + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t start = frameAt(1.5), end = frameAt(2.0); + sv::sv_frame_t from, to; + widenRange(start, end, 0, fileEnd, from, to); + + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + + // One long note from 0.3 to 2.3 s, and a short one after it + sv::EventVector wasNotes = noteEvents(analyser); + QCOMPARE(int(wasNotes.size()), 2); + sv::Event crossing = wasNotes[0]; + QVERIFY2(crossing.getFrame() < from && + crossing.getFrame() + crossing.getDuration() > from, + qPrintable(QString("the long note (%1 for %2) does not cross " + "the edge of the widened range at %3") + .arg(crossing.getFrame()) + .arg(crossing.getDuration()).arg(from))); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector now = noteEvents(analyser); + int atCrossing = 0, inRange = 0; + for (const auto &e : now) { + if (e.getFrame() == crossing.getFrame()) { + ++atCrossing; + QCOMPARE(e.getFrame() + e.getDuration(), from); + QCOMPARE(e.getValue(), crossing.getValue()); + } + if (e.getFrame() >= from && e.getFrame() < to + 2 * hop) ++inRange; + } + QCOMPARE(atCrossing, 1); + QVERIFY2(inRange >= 1, "the analysed range came back with no note"); + + // The note after the range is untouched + QCOMPARE(now.back().getFrame(), wasNotes[1].getFrame()); + QCOMPARE(now.back().getDuration(), wasNotes[1].getDuration()); + } + + void ranged_cancelled() { + // Closing, re-recording or switching take while a ranged + // analysis runs goes through cancelAnalyses() + auto data = fourNotes(); + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + + sv::EventVector wasPitch = pitchEvents(analyser); + sv::EventVector wasNotes = noteEvents(analyser); + size_t layers = m_document->getLayers().size(); + size_t models = m_document->getModels().size(); + + int inPane = m_pane->getLayerCount(); + sv::Layer *current = m_pane->getSelectedLayer(); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(0, fileEnd, 0, fileEnd), QString()); + QVERIFY2(analyser.isAnalysingRange(), + "the whole file was analysed before we could cancel it"); + QVERIFY(m_document->getLayers().size() > layers); + + // The temporary layers are in the document but in no view, so + // nothing shows them, nothing selects them, and the scan for + // existing analyses (which looks in the pane) cannot take them + // for the analyser's own + QCOMPARE(m_pane->getLayerCount(), inPane); + QVERIFY(m_pane->getSelectedLayer() == current); + + analyser.cancelAnalyses(); + + QVERIFY(!analyser.isAnalysingRange()); + verifyNothingLeftOver(layers, models); + + // and no late merge from the transform we abandoned + QTest::qWait(500); + QCOMPARE(done.count(), 0); + verifyNothingLeftOver(layers, models); + QCOMPARE(pitchEvents(analyser).size(), wasPitch.size()); + QCOMPARE(noteEvents(analyser).size(), wasNotes.size()); + for (size_t i = 0; i < wasPitch.size(); ++i) { + QCOMPARE(pitchEvents(analyser)[i].getFrame(), + wasPitch[i].getFrame()); + } + } + + void ranged_released_while_running() { + // releaseLayers() (the model swap of 4a) and the destructor must + // take the temporaries with them too + auto data = fourNotes(); + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + + size_t layers = 0, models = 0; + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + layers = m_document->getLayers().size(); + models = m_document->getModels().size(); + + QCOMPARE(analyser.analyseRange(0, fileEnd, 0, fileEnd), QString()); + QVERIFY(analyser.isAnalysingRange()); + analyser.releaseLayers(); + QVERIFY(!analyser.isAnalysingRange()); + + // releaseLayers() deletes the waveform layer and its model + QCOMPARE(m_document->getLayers().size(), layers - 1); + } + + // A second run, abandoned by the destructor this time + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + layers = m_document->getLayers().size(); + models = m_document->getModels().size(); + + QCOMPARE(analyser.analyseRange(0, fileEnd, 0, fileEnd), QString()); + QVERIFY(analyser.isAnalysingRange()); + } + verifyNothingLeftOver(layers, models); + } + + void ranged_restarted_while_running() { + // A second call abandons the first: its material is presumed to + // have changed under us + auto data = fourNotes(); + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t firstStart = frameAt(0.5), firstEnd = frameAt(1.2); + sv::sv_frame_t start = frameAt(thirdNoteFrom); + sv::sv_frame_t end = frameAt(thirdNoteTo); + sv::sv_frame_t abandoned0, abandoned1, from, to; + widenRange(firstStart, firstEnd, 0, fileEnd, abandoned0, abandoned1); + widenRange(start, end, 0, fileEnd, from, to); + QVERIFY(abandoned1 < from); // the two ranges do not meet + + sv::ModelId singing = addSingingModel(data); + addEmptyAnalyses(singing); + if (QTest::currentTestFailed()) return; + + Analyser analyser(Analyser::SecondaryColors); + QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, + m_pane, true), QString()); + size_t layers = m_document->getLayers().size(); + size_t models = m_document->getModels().size(); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(firstStart, firstEnd, 0, fileEnd), + QString()); + QVERIFY(analyser.isAnalysingRange()); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + QVERIFY(analyser.isAnalysingRange()); + + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + QCOMPARE(done.count(), 1); + verifyNothingLeftOver(layers, models); + + // Only the second range was merged + auto events = pitchEvents(analyser); + QVERIFY(events.size() > 60); + for (const auto &e : events) { + QVERIFY2(e.getFrame() >= from && e.getFrame() < to + 2 * hop, + qPrintable(QString("pitch event at %1, from the range " + "that was abandoned").arg(e.getFrame()))); + } + QCOMPARE(int(notesIn(noteEvents(analyser), abandoned0, abandoned1).size()), + 0); + QVERIFY(notesIn(noteEvents(analyser), from, to + 2 * hop).size() >= 1); + } }; #endif From d531e3a3f73ce337c40706ff18e08e1ab0d41bd4 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 03:29:16 +0300 Subject: [PATCH 036/275] feat: merge only the middle of a ranged run analyseRange() widens the range it is asked for by half a second on each side so that pYIN has context, but the merge then replaced everything in that widened run. Two things went wrong in audio that had not changed: the run cannot stamp its own first hops, so the pitch events there were deleted and not replaced, and a note that crossed the run's left edge was cut in two. So the merge now writes only over a window in the middle of the run -- a quarter of a second each side of the range asked for, or out to an edge that the caller's coverage limit clipped, where there is no context worth keeping. Notes go by their onset, and a note that runs into the window from before it is only cut back if one of the new notes starts inside it. Found on the way: pYIN in fixed-lag mode stamps one frame of every run twice, 100 hops before the end of the run, so the merge drops the second copy. Co-Authored-By: Claude Opus 5 --- main/Analyser.cpp | 131 +++++++--- main/Analyser.h | 17 +- main/test/TestSingingAnalysis.h | 439 +++++++++++++++++++++++++++----- 3 files changed, 485 insertions(+), 102 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index df2a457e..b6639c6b 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -55,7 +55,10 @@ Analyser::Analyser(ColorScheme colorScheme) : m_candidatesVisible(false), m_currentAsyncHandle(0), m_rangedStart(0), - m_rangedEnd(0) + m_rangedEnd(0), + m_rangedMergeStart(0), + m_rangedMergeEnd(0), + m_rangedClippedEnd(false) { QSettings settings; settings.beginGroup("LayerDefaults"); @@ -941,6 +944,8 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, // is found whole -- but never outside the material the caller says // is there to analyse (the take's coverage) sv_frame_t margin = sv_frame_t(rate / 2); + bool clippedStart = (start - margin < clipStart); + bool clippedEnd = (end + margin > clipEnd); sv_frame_t from = std::max(clipStart, start - margin); sv_frame_t to = std::min(clipEnd, end + margin); @@ -994,6 +999,16 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, m_rangedStart = from; m_rangedEnd = to; + // Only the middle of the run is merged (see mergeRangedAnalysis()): + // half the margin on each side of the range asked for, so that the + // run keeps a quarter of a second of its own context on each side + // that nothing is taken from. Where the run stopped at the edge of + // the caller's coverage there is no context to keep -- nothing but + // silence beyond -- and the window reaches that edge + m_rangedMergeStart = clippedStart ? from : std::max(from, start - margin/2); + m_rangedMergeEnd = clippedEnd ? to : std::min(to, end + margin/2); + m_rangedClippedEnd = clippedEnd; + for (ModelId id : { m_rangedPitchModel, m_rangedNotesModel }) { auto model = ModelById::get(id); if (!model) continue; @@ -1070,54 +1085,102 @@ Analyser::mergeRangedAnalysis() newNoteEvents.push_back(e.withFrame(e.getFrame() + m_rangedStart)); } - // What the merge replaces is the widened range, extended to cover - // whatever the run actually produced: pYIN stamps a block a quarter - // of a block in (two hops here), so its events start a little after - // the range and end a little after it too. Clearing exactly the - // span they occupy leaves neither a stale event where a new one - // goes nor two events on one frame - sv_frame_t pitchFrom = m_rangedStart, pitchTo = m_rangedEnd; - if (!newPitchEvents.empty()) { - pitchFrom = std::min(pitchFrom, newPitchEvents.front().getFrame()); - pitchTo = std::max(pitchTo, newPitchEvents.back().getFrame() + 1); - } - - sv_frame_t noteFrom = m_rangedStart, noteTo = m_rangedEnd; - for (const Event &e : newNoteEvents) { - noteFrom = std::min(noteFrom, e.getFrame()); - noteTo = std::max(noteTo, e.getFrame() + 1); + // What the merge replaces is not the whole run but the window W in + // the middle of it (analyseRange() works it out). The run's own + // ends are where pYIN has least context, and it cannot stamp its + // first two hops at all, so what it says there is worse than what + // the models hold already -- which, for audio that has not changed, + // is the answer of a whole-file run + sv_frame_t wFrom = m_rangedMergeStart; + sv_frame_t pitchTo = m_rangedMergeEnd, noteTo = m_rangedMergeEnd; + + // The exception is an end of the run that stopped at the edge of the + // caller's coverage: nothing is kept beyond it, so the window takes + // in whatever the run stamped past it. pYIN stamps a block a + // quarter of a block in (two hops here), so the last events of a run + // lie a little past its end + if (m_rangedClippedEnd) { + if (!newPitchEvents.empty()) { + pitchTo = std::max(pitchTo, newPitchEvents.back().getFrame() + 1); + } + for (const Event &e : newNoteEvents) { + noteTo = std::max(noteTo, e.getFrame() + 1); + } } cerr << "Analyser::mergeRangedAnalysis: " << newPitchEvents.size() - << " pitch event(s) and " << newNoteEvents.size() << " note(s) for " - << m_rangedStart << " to " << m_rangedEnd << "; replacing pitch in " - << pitchFrom << " to " << pitchTo << ", notes in " << noteFrom - << " to " << noteTo << endl; + << " pitch event(s) and " << newNoteEvents.size() << " note(s) from " + << m_rangedStart << " to " << m_rangedEnd << "; merging pitch in " + << wFrom << " to " << pitchTo << ", notes in " << wFrom << " to " + << noteTo << endl; for (const Event &e : - pitch->getEventsStartingWithin(pitchFrom, pitchTo - pitchFrom)) { + pitch->getEventsStartingWithin(wFrom, pitchTo - wFrom)) { pitch->remove(e); } + // pYIN in fixed-lag mode (the default, and what we run) stamps one + // frame of every run twice: the last frame that process() emits is + // emitted again as the first of getRemainingFeatures(), 100 hops + // before the end of the run. A whole-file run has the same double + // frame near the end of the file, where it does no harm, but a ranged + // run puts it in the middle of the merge window, so drop it here + sv_frame_t lastAdded = -1; for (const Event &e : newPitchEvents) { - pitch->add(e); + if (e.getFrame() >= wFrom && e.getFrame() < pitchTo && + e.getFrame() != lastAdded) { + pitch->add(e); + lastAdded = e.getFrame(); + } + } + + // Notes go by their onset: the new notes that begin in the window + // replace the old ones that begin in it + EventVector adding; + for (const Event &e : newNoteEvents) { + if (e.getFrame() >= wFrom && e.getFrame() < noteTo) { + adding.push_back(e); + } } - // A note that starts in the range goes; one that runs into the range - // from the left is cut back to the edge of it, and the new notes - // supply everything from there on. A note that reached out of the - // far end of the range loses that tail: the run has analysed the - // material there and says what is in it - for (const Event &e : notes->getAllEvents()) { + EventVector oldNotes = notes->getAllEvents(); + + // The first of the new notes, and the first old note that begins at + // or after the window: the two notes the window's edges can run into + sv_frame_t firstAdded = adding.empty() ? -1 : adding.front().getFrame(); + sv_frame_t nextOldOnset = -1; + for (const Event &e : oldNotes) { + if (e.getFrame() >= noteTo) { + nextOldOnset = e.getFrame(); + break; + } + } + + for (const Event &e : oldNotes) { sv_frame_t f = e.getFrame(); - if (f >= noteFrom && f < noteTo) { + if (f >= wFrom && f < noteTo) { notes->remove(e); - } else if (f < noteFrom && f + e.getDuration() > noteFrom) { + } else if (f < wFrom && firstAdded >= 0 && + f + e.getDuration() > firstAdded) { + // A note that runs into the window from before it is left as + // it is unless one of the new notes starts inside it, when it + // is cut back to that onset. Where the audio has not + // changed the run finds that note going on from before the + // window, has nothing to add inside it, and the one note + // stays one note notes->remove(e); - notes->add(e.withDuration(noteFrom - f)); + notes->add(e.withDuration(firstAdded - f)); } } - for (const Event &e : newNoteEvents) { - notes->add(e); + for (const Event &e : adding) { + // The far edge the same way round: an old note that begins at or + // after the window keeps its onset, and a new note that would run + // over it is cut back + if (nextOldOnset > e.getFrame() && + e.getFrame() + e.getDuration() > nextOldOnset) { + notes->add(e.withDuration(nextOldOnset - e.getFrame())); + } else { + notes->add(e); + } } // The events are in the models, not in a command: an analysis result diff --git a/main/Analyser.h b/main/Analyser.h index 3396f3a7..dd799bc8 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -190,8 +190,12 @@ class Analyser : public QObject, * * The analysis runs into temporary layers that are in no view; * when both are complete their events replace what was in the - * widened range and initialAnalysisCompleted() is emitted. The - * merge is not undoable: an analysis result never was. + * middle of the run -- a quarter of a second each side of the range + * asked for, or out to an end of the run that was clipped -- and + * initialAnalysisCompleted() is emitted. The rest of the run is + * context only: it is where pYIN knows least, so what is there + * already is left alone. The merge is not undoable: an analysis + * result never was. * * Returns "" if a run was started (or there was nothing to do), or * a user-readable error string. A second call while one is running @@ -345,7 +349,14 @@ protected slots: sv::ModelId m_rangedPitchModel; sv::ModelId m_rangedNotesModel; sv::sv_frame_t m_rangedStart; // widened, grid-aligned: what is analysed - sv::sv_frame_t m_rangedEnd; // and what the merge replaces + sv::sv_frame_t m_rangedEnd; + sv::sv_frame_t m_rangedMergeStart; // the window inside that which the + sv::sv_frame_t m_rangedMergeEnd; // merge replaces (W) + // True where the run's far edge is the edge of the caller's coverage: + // there is no context to keep there, so the merge takes in what the + // run stamped past it. The near edge needs no such flag -- a run + // cannot stamp anything before its own first two hops anyway + bool m_rangedClippedEnd; QString doAllAnalyses(bool withPitchTrack); diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 38f6d0c8..4a436fc5 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -49,6 +49,7 @@ #include #include #include +#include #include class TestSingingAnalysis : public QObject @@ -190,10 +191,14 @@ class TestSingingAnalysis : public QObject return model->getAllEvents(); } - static sv::EventVector noteEvents(Analyser &analyser) { + static std::shared_ptr notesModel(Analyser &analyser) { sv::Layer *layer = analyser.getLayer(Analyser::Notes); - if (!layer) return {}; - auto model = sv::ModelById::getAs(layer->getModel()); + if (!layer) return nullptr; + return sv::ModelById::getAs(layer->getModel()); + } + + static sv::EventVector noteEvents(Analyser &analyser) { + auto model = notesModel(analyser); if (!model) return {}; return model->getAllEvents(); } @@ -275,6 +280,98 @@ class TestSingingAnalysis : public QObject return out; } + // The window analyseRange() merges: half the margin on each side of + // the range asked for, inside the run, out to an edge of the run that + // the coverage limit clipped + static void mergeWindow(sv::sv_frame_t start, sv::sv_frame_t end, + sv::sv_frame_t clipStart, sv::sv_frame_t clipEnd, + sv::sv_frame_t &wFrom, sv::sv_frame_t &wTo) { + sv::sv_frame_t margin = sv::sv_frame_t(rate / 2); + sv::sv_frame_t from, to; + widenRange(start, end, clipStart, clipEnd, from, to); + wFrom = (start - margin < clipStart) ? + from : std::max(from, start - margin / 2); + wTo = (end + margin > clipEnd) ? to : std::min(to, end + margin / 2); + } + + static std::set doubledFrames(const sv::EventVector &ev) { + std::set out; + for (size_t i = 1; i < ev.size(); ++i) { + if (ev[i].getFrame() == ev[i-1].getFrame()) { + out.insert(ev[i].getFrame()); + } + } + return out; + } + + static std::map byFrame(const sv::EventVector &ev) { + std::map m; + for (const auto &e : ev) m[e.getFrame()] = e.getValue(); + return m; + } + + // For a failure message: every note as onset+duration@Hz + static QString describeNotes(const sv::EventVector &events) { + QStringList out; + for (const auto &e : events) { + out << QString("%1+%2@%3").arg(e.getFrame()) + .arg(e.getDuration()).arg(double(e.getValue()), 0, 'f', 1); + } + return "[" + out.join(" ") + "]"; + } + + // The pitch track of the whole file before and after a ranged run + // over audio that has not changed: the same frames, no frame twice, + // the same values outside the merge window and near enough inside it + void comparePitchAcrossFile(const sv::EventVector &was, + const sv::EventVector &is, + sv::sv_frame_t wFrom, sv::sv_frame_t wTo) { + // The merge may not leave two events on one frame. pYIN itself + // does, once per run: in fixed-lag mode it stamps the frame 100 + // hops before the end of a run twice, so a frame that was already + // doubled in the whole-file track may still be + QStringList twice; + for (sv::sv_frame_t f : doubledFrames(is)) { + if (!doubledFrames(was).count(f)) { + twice << QString::number(f); + } + } + QVERIFY2(twice.isEmpty(), + qPrintable(QString("two pitch events on one frame after the " + "merge (window %1 to %2): %3") + .arg(wFrom).arg(wTo).arg(twice.join(" ")))); + + auto a = byFrame(was), b = byFrame(is); + QStringList odd; + for (const auto &p : a) if (!b.count(p.first)) odd << QString("-%1").arg(p.first); + for (const auto &p : b) if (!a.count(p.first)) odd << QString("+%1").arg(p.first); + QVERIFY2(odd.isEmpty(), + qPrintable(QString("pitch frames lost (-) or gained (+) by " + "the ranged run, merge window %1 to %2: %3") + .arg(wFrom).arg(wTo).arg(odd.join(" ")))); + double worst = 0.0; + for (const auto &p : a) { + float now = b[p.first]; + if (p.first >= wFrom && p.first < wTo) { + worst = std::max(worst, std::abs + (TestSignals::centsBetween(now, p.second))); + } else { + QVERIFY2(now == p.second, + qPrintable(QString("pitch at %1 changed from %2 to " + "%3, outside the merge window " + "%4 to %5").arg(p.first) + .arg(p.second).arg(now) + .arg(wFrom).arg(wTo))); + } + } + // Measured: 0 cents. The window keeps a quarter of a second of + // the run away from its edges, and that is context enough for + // pYIN to say the same as it said about the whole file + QVERIFY2(worst < 20.0, + qPrintable(QString("worst pitch difference inside the merge " + "window %1 cents").arg(worst))); + } + static double medianHz(const sv::EventVector &events) { std::vector values; for (const auto &e : events) values.push_back(e.getValue()); @@ -715,15 +812,17 @@ private slots: } } - void ranged_leaves_the_rest_alone() { - // A whole-file analysis, then the same audio analysed again over - // one range: outside the widened range not one event may move + // A whole-file analysis, then the same audio analysed again over one + // range: the file must come out of it exactly as a whole-file + // analysis left it. No hole where the merge window begins, no frame + // twice, no note split or shortened, nothing moved. Only the middle + // of the run is merged for the sake of this + void verifyRangedRunChangesNothing(sv::sv_frame_t start, + sv::sv_frame_t end) { auto data = fourNotes(); sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); - sv::sv_frame_t start = frameAt(thirdNoteFrom); - sv::sv_frame_t end = frameAt(thirdNoteTo); - sv::sv_frame_t from, to; - widenRange(start, end, 0, fileEnd, from, to); + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, end, 0, fileEnd, wFrom, wTo); Analyser analyser(Analyser::SecondaryColors); analyse(analyser, addSingingModel(data)); @@ -732,6 +831,7 @@ private slots: sv::EventVector wasPitch = pitchEvents(analyser); sv::EventVector wasNotes = noteEvents(analyser); QCOMPARE(int(wasNotes.size()), 4); + QVERIFY2(wasPitch.size() > 200, "too few pitch events to test this"); size_t layers = m_document->getLayers().size(); size_t models = m_document->getModels().size(); @@ -742,43 +842,60 @@ private slots: if (QTest::currentTestFailed()) return; verifyNothingLeftOver(layers, models); - // pYIN stamps a block a quarter of a block in (two hops here), so - // its events reach a couple of hops past the end of the range - sv::EventVector wasOut = outsidePitch(wasPitch, from, to + 2 * hop); - sv::EventVector isOut = outsidePitch(pitchEvents(analyser), - from, to + 2 * hop); - QVERIFY2(wasOut.size() > 100, - "the range covers too much of the file to test this"); - QCOMPARE(isOut.size(), wasOut.size()); - for (size_t i = 0; i < wasOut.size(); ++i) { - QCOMPARE(isOut[i].getFrame(), wasOut[i].getFrame()); - QCOMPARE(isOut[i].getValue(), wasOut[i].getValue()); - } - - // The notes of the file that lie wholly outside: the first two - // (0.6 to 1.1 s and 1.7 to 2.2 s) and the last (3.9 to 4.4 s) - sv::EventVector wasOutN = outsideNotes(wasNotes, from, to + 2 * hop); - sv::EventVector isOutN = outsideNotes(noteEvents(analyser), - from, to + 2 * hop); - QCOMPARE(int(wasOutN.size()), 3); - QCOMPARE(isOutN.size(), wasOutN.size()); - for (size_t i = 0; i < wasOutN.size(); ++i) { - QCOMPARE(isOutN[i].getFrame(), wasOutN[i].getFrame()); - QCOMPARE(isOutN[i].getDuration(), wasOutN[i].getDuration()); - QCOMPARE(isOutN[i].getValue(), wasOutN[i].getValue()); - } - - // and the third note is still one note, near enough where it was - sv::EventVector was3 = notesIn(wasNotes, from, to + 2 * hop); - sv::EventVector is3 = notesIn(noteEvents(analyser), from, to + 2 * hop); - QCOMPARE(int(was3.size()), 1); - QCOMPARE(is3.size(), was3.size()); - QVERIFY(std::abs(is3[0].getFrame() - was3[0].getFrame()) <= 2 * hop); - } - - void ranged_truncates_note_across_the_edge() { - // A note that runs into the widened range from the left is cut - // back to its edge; the analysis supplies what is inside + comparePitchAcrossFile(wasPitch, pitchEvents(analyser), wFrom, wTo); + if (QTest::currentTestFailed()) return; + + // Still four notes, each where it was. A note whose onset is in + // the window is replaced by the run's own, which may sit a hop or + // two away; one whose onset is outside it is not touched at all + sv::EventVector isNotes = noteEvents(analyser); + QVERIFY2(isNotes.size() == wasNotes.size(), + qPrintable(QString("the notes %1 became %2 (merge window " + "%3 to %4)").arg(describeNotes(wasNotes)) + .arg(describeNotes(isNotes)) + .arg(wFrom).arg(wTo))); + for (size_t i = 0; i < wasNotes.size(); ++i) { + sv::sv_frame_t f = wasNotes[i].getFrame(); + sv::sv_frame_t slack = (f >= wFrom && f < wTo) ? 2 * hop : 0; + QVERIFY2(std::abs(isNotes[i].getFrame() - f) <= slack, + qPrintable(QString("note %1 moved from %2 to %3") + .arg(i).arg(f).arg(isNotes[i].getFrame()))); + QVERIFY2(std::abs(isNotes[i].getDuration() - + wasNotes[i].getDuration()) <= slack, + qPrintable(QString("note %1 lasted %2, now %3").arg(i) + .arg(wasNotes[i].getDuration()) + .arg(isNotes[i].getDuration()))); + QVERIFY(std::abs(TestSignals::centsBetween + (isNotes[i].getValue(), + wasNotes[i].getValue())) < 20.0); + } + } + + void ranged_leaves_the_rest_alone() { + // The third note of fourNotes() analysed again, with both edges of + // the merge window landing in the silence around it + verifyRangedRunChangesNothing(frameAt(thirdNoteFrom), + frameAt(thirdNoteTo)); + } + + void ranged_range_shorter_than_the_margin() { + // One hop asked for, in the middle of the third note: the merge + // window is half a second wide all the same, and its right edge + // now falls inside that note (which the run therefore replaces, + // whole, tail and all -- the note's end is inside the run) + sv::sv_frame_t start = frameAt(3.0); + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, start + hop, 0, frameAt(5.0), wFrom, wTo); + QVERIFY(wFrom >= frameAt(2.75) && wTo > frameAt(3.25) && + wTo < frameAt(3.3)); // the note runs 2.8 to 3.3 s + verifyRangedRunChangesNothing(start, start + hop); + } + + void ranged_keeps_a_note_across_the_window_edge() { + // A note that runs into the merge window from before it: the run + // finds it going on from before the window too, so it has no note + // to add inside the window, and the one note stays one note -- + // where it used to be cut in two std::vector data; appendSilence(data, 0.3); auto lng = tone(singingHz, 2.0); @@ -790,8 +907,8 @@ private slots: sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); sv::sv_frame_t start = frameAt(1.5), end = frameAt(2.0); - sv::sv_frame_t from, to; - widenRange(start, end, 0, fileEnd, from, to); + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, end, 0, fileEnd, wFrom, wTo); Analyser analyser(Analyser::SecondaryColors); analyse(analyser, addSingingModel(data)); @@ -799,36 +916,228 @@ private slots: // One long note from 0.3 to 2.3 s, and a short one after it sv::EventVector wasNotes = noteEvents(analyser); + sv::EventVector wasPitch = pitchEvents(analyser); QCOMPARE(int(wasNotes.size()), 2); sv::Event crossing = wasNotes[0]; - QVERIFY2(crossing.getFrame() < from && - crossing.getFrame() + crossing.getDuration() > from, + QVERIFY2(crossing.getFrame() < wFrom && + crossing.getFrame() + crossing.getDuration() > wTo, qPrintable(QString("the long note (%1 for %2) does not cross " - "the edge of the widened range at %3") + "the merge window %3 to %4") .arg(crossing.getFrame()) - .arg(crossing.getDuration()).arg(from))); + .arg(crossing.getDuration()) + .arg(wFrom).arg(wTo))); QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); waitForRange(analyser, done); if (QTest::currentTestFailed()) return; + // The run begins in the middle of the note, where it cannot stamp + // its first two hops: the pitch track of the whole file must come + // through it without a hole all the same + comparePitchAcrossFile(wasPitch, pitchEvents(analyser), wFrom, wTo); + if (QTest::currentTestFailed()) return; + sv::EventVector now = noteEvents(analyser); - int atCrossing = 0, inRange = 0; - for (const auto &e : now) { - if (e.getFrame() == crossing.getFrame()) { - ++atCrossing; - QCOMPARE(e.getFrame() + e.getDuration(), from); - QCOMPARE(e.getValue(), crossing.getValue()); + QVERIFY2(now.size() == wasNotes.size() && + now[0].getFrame() == crossing.getFrame() && + now[0].getDuration() == crossing.getDuration(), + qPrintable(QString("the notes %1 became %2 (merge window " + "%3 to %4)").arg(describeNotes(wasNotes)) + .arg(describeNotes(now)).arg(wFrom).arg(wTo))); + + // and the note after the range is untouched + QCOMPARE(now.back().getFrame(), wasNotes[1].getFrame()); + QCOMPARE(now.back().getDuration(), wasNotes[1].getDuration()); + } + + void ranged_cuts_a_new_note_at_the_next_old_note() { + // A new note may not run over an old note that begins at or after + // the window: the old onset stands and the new note is cut back to + // it. The old notes here are put in by hand, as notes that the + // user has edited or that belong to material the swap replaced + std::vector data; + appendSilence(data, 1.4); + auto lng = tone(singingHz, 1.6); // 1.4 to 3.0 s + data.insert(data.end(), lng.begin(), lng.end()); + appendSilence(data, 0.3); + + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t start = frameAt(1.5), end = frameAt(2.0); + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, end, 0, fileEnd, wFrom, wTo); + QVERIFY(wFrom < frameAt(1.4) && wTo > frameAt(2.2)); + + sv::ModelId singing = addSingingModel(data); + addEmptyAnalyses(singing); + if (QTest::currentTestFailed()) return; + + Analyser analyser(Analyser::SecondaryColors); + QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, + m_pane, true), QString()); + auto notes = notesModel(analyser); + QVERIFY(notes); + + // Before the window, inside it, and one beginning after it that + // the run's note will want to run over + sv::Event before(frameAt(0.2), 200.f, frameAt(0.3), "before"); + sv::Event inside(frameAt(1.6), 200.f, frameAt(0.3), "inside"); + sv::Event after(wTo + frameAt(0.05), 200.f, frameAt(0.6), "after"); + for (const auto &e : { before, inside, after }) notes->add(e); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector now = noteEvents(analyser); + QVERIFY2(now.size() == 3, + qPrintable("notes after the merge: " + describeNotes(now))); + + // The note before the window is untouched: no new note starts + // inside it + QCOMPARE(now[0].getFrame(), before.getFrame()); + QCOMPARE(now[0].getDuration(), before.getDuration()); + + // The one inside it is gone, replaced by the run's note, which + // starts near the start of the tone and is cut back to the onset + // of the note after the window + QVERIFY2(std::abs(now[1].getFrame() - frameAt(1.4)) <= 4 * hop, + qPrintable(QString("the run's note starts at %1, the tone at " + "%2").arg(now[1].getFrame()) + .arg(frameAt(1.4)))); + QCOMPARE(now[1].getFrame() + now[1].getDuration(), after.getFrame()); + QVERIFY(std::abs(TestSignals::centsBetween(now[1].getValue(), + singingHz)) < 50.0); + + // and the note after the window keeps its onset and its length + QCOMPARE(now[2].getFrame(), after.getFrame()); + QCOMPARE(now[2].getDuration(), after.getDuration()); + } + + void ranged_at_the_edge_of_coverage() { + // Where the caller's coverage limit clips an edge of the run there + // is nothing beyond it but silence and no context worth keeping, + // so the window reaches that edge and takes in everything the run + // stamped there. Three cases: both edges clipped (the first + // recording of a take, with nothing in the models yet), the left + // edge, the right edge + std::vector data; + appendSilence(data, 0.15); + auto first = tone(singingHz, 0.5); // 0.15 to 0.65 s + data.insert(data.end(), first.begin(), first.end()); + appendSilence(data, 0.4); + auto second = tone(referenceHz, 0.5); // 1.05 to 1.55 s + data.insert(data.end(), second.begin(), second.end()); + appendSilence(data, 0.45); // silence beyond the coverage + + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t coverEnd = frameAt(1.55); + + sv::EventVector wholePitch, wholeNotes; + { + Analyser whole(Analyser::SecondaryColors); + analyse(whole, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + wholePitch = pitchEvents(whole); + wholeNotes = noteEvents(whole); + whole.removeAllLayers(); + } + QCOMPARE(int(wholeNotes.size()), 2); + + // Both edges: empty models, the coverage is the range recorded, + // and the run is the whole of it. Nothing may be dropped at + // either end, so the result is the whole-file result + { + sv::ModelId singing = addSingingModel(data); + addEmptyAnalyses(singing); + if (QTest::currentTestFailed()) return; + Analyser analyser(Analyser::SecondaryColors); + QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, + m_pane, true), QString()); + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(0, coverEnd, 0, coverEnd), + QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + auto a = byFrame(wholePitch), b = byFrame(pitchEvents(analyser)); + QStringList odd; + for (const auto &p : a) if (!b.count(p.first)) odd << QString("-%1").arg(p.first); + for (const auto &p : b) if (!a.count(p.first)) odd << QString("+%1").arg(p.first); + QVERIFY2(odd.isEmpty(), + qPrintable("pitch frames lost (-) or gained (+) by the " + "run over the whole coverage: " + + odd.join(" "))); + sv::EventVector notes = noteEvents(analyser); + QVERIFY2(notes.size() == wholeNotes.size(), + qPrintable("notes " + describeNotes(wholeNotes) + + " became " + describeNotes(notes))); + for (size_t i = 0; i < notes.size(); ++i) { + QVERIFY(std::abs(notes[i].getFrame() - + wholeNotes[i].getFrame()) <= 2 * hop); } - if (e.getFrame() >= from && e.getFrame() < to + 2 * hop) ++inRange; + // or the next analyser would claim these layers as its own + analyser.removeAllLayers(); } - QCOMPARE(atCrossing, 1); - QVERIFY2(inRange >= 1, "the analysed range came back with no note"); - // The note after the range is untouched - QCOMPARE(now.back().getFrame(), wasNotes[1].getFrame()); - QCOMPARE(now.back().getDuration(), wasNotes[1].getDuration()); + // The left edge only: a range at the very start of the coverage + // still gets its first note, and nothing before it is lost + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + sv::EventVector wasPitch = pitchEvents(analyser); + sv::sv_frame_t start = frameAt(0.1), end = frameAt(0.7); + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, end, 0, fileEnd, wFrom, wTo); + QCOMPARE(wFrom, sv::sv_frame_t(0)); // clipped to nothing + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector notes = noteEvents(analyser); + QVERIFY2(notes.size() == 2, + qPrintable("notes after the merge: " + + describeNotes(notes))); + QVERIFY(std::abs(notes[0].getFrame() - + wholeNotes[0].getFrame()) <= 2 * hop); + comparePitchAcrossFile(wasPitch, pitchEvents(analyser), wFrom, wTo); + if (QTest::currentTestFailed()) return; + analyser.removeAllLayers(); + } + + // The right edge only: the run stops at the end of the coverage, + // and what it stamped past that end (pYIN stamps a block two hops + // in) is kept, so the last of the pitch track is not lost + { + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + sv::EventVector wasPitch = pitchEvents(analyser); + sv::sv_frame_t start = frameAt(1.2), end = coverEnd; + sv::sv_frame_t wFrom, wTo; + mergeWindow(start, end, 0, coverEnd, wFrom, wTo); + QVERIFY(wTo >= coverEnd); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, coverEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector notes = noteEvents(analyser); + QVERIFY2(notes.size() == 2, + qPrintable("notes after the merge: " + + describeNotes(notes))); + QVERIFY(std::abs(notes[1].getFrame() - + wholeNotes[1].getFrame()) <= 2 * hop); + // The window reaches past the end of the run here, so the + // comparison allows for the run's own last events + comparePitchAcrossFile(wasPitch, pitchEvents(analyser), + wFrom, wTo + 4 * hop); + } } void ranged_cancelled() { From 72692a2ddddbbb2f84ae7a3be7d7bb015002d3fa Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 03:33:10 +0300 Subject: [PATCH 037/275] test: drop two range helpers the new comparisons replaced Co-Authored-By: Claude Opus 5 --- main/test/TestSingingAnalysis.h | 25 ------------------------- 1 file changed, 25 deletions(-) diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 4a436fc5..b0061bdd 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -245,16 +245,6 @@ class TestSingingAnalysis : public QObject return m; } - static sv::EventVector outsidePitch(const sv::EventVector &events, - sv::sv_frame_t from, - sv::sv_frame_t to) { - sv::EventVector out; - for (const auto &e : events) { - if (e.getFrame() < from || e.getFrame() >= to) out.push_back(e); - } - return out; - } - // Notes that start within [from, to) static sv::EventVector notesIn(const sv::EventVector &events, sv::sv_frame_t from, sv::sv_frame_t to) { @@ -265,21 +255,6 @@ class TestSingingAnalysis : public QObject return out; } - // Notes that lie wholly outside [from, to), so that the merge has no - // business with them at all - static sv::EventVector outsideNotes(const sv::EventVector &events, - sv::sv_frame_t from, - sv::sv_frame_t to) { - sv::EventVector out; - for (const auto &e : events) { - if (e.getFrame() + e.getDuration() <= from || - e.getFrame() >= to) { - out.push_back(e); - } - } - return out; - } - // The window analyseRange() merges: half the margin on each side of // the range asked for, inside the run, out to an edge of the run that // the coverage limit clipped From 2dcdf1a4a065436e4df85ae1ad75535c93377102 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 04:18:42 +0300 Subject: [PATCH 038/275] feat: empty pitch and notes for a take, and a note the run cut off Analyser::addEmptyAnalyses() makes the pair a whole-file analysis would have made, empty, for the first recording of a take: same layer and model types, resolution, units, colours, names and source model, so that the analyser's own scan, a swap and a session restore cannot tell them from analysed ones. The colours and play parameters of both paths are now configureAnalysisLayers(), and the muting of the singing track's pitch and notes silenceSecondaryAnalysisLayers(). The merge takes one more rule: a new note that the end of the run cut off (it ends within a few hops of that end) takes the end of the old note that ran past the run, where there is one. The audio out there has not changed, so that note knows where the note really ends. Left open by 4b2; spec 6.3 step 2 says so now. analyseRange() no longer clips the range to the audio model's frame count while the file is still being decoded: the count is still growing then, and the clip became "nothing at all" if a take was analysed the moment its file was opened. The transform waits for the model itself. Co-Authored-By: Claude Opus 5 --- main/Analyser.cpp | 231 ++++++++++++++++++++++++++++---- main/Analyser.h | 21 +++ main/test/TestSingingAnalysis.h | 109 +++++++++------ 3 files changed, 294 insertions(+), 67 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index b6639c6b..ae53b4a0 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -34,6 +34,8 @@ #include "layer/SpectrogramLayer.h" #include "layer/Colour3DPlotLayer.h" #include "layer/ShowLayerCommand.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/model/NoteModel.h" #include #include @@ -46,6 +48,14 @@ using std::endl; using namespace sv; +// The two pYIN outputs of a full analysis, and the step size they are +// run at: the grid every result of ours sits on +static const QString pyinPlugin = "pYIN"; +static const QString pyinBase = "vamp:pyin:pyin:"; +static const QString pyinPitchOutput = "smoothedpitchtrack"; +static const QString pyinNotesOutput = "notes"; +static const int analysisStepSize = 256; + Analyser::Analyser(ColorScheme colorScheme) : m_colorScheme(colorScheme), m_document(0), @@ -187,19 +197,7 @@ Analyser::doAllAnalyses(bool withPitchTrack) loadState(Notes); loadState(Spectrogram); - // The secondary analyser's pitch and note tracks are visual-only (there is - // no UI toggle to control their audibility, and sonifying two pitch/note - // tracks at once is confusing). Mute them directly — do NOT call - // setAudible(), which would also call saveState() and corrupt the primary - // analyser's shared settings key. - if (m_colorScheme == SecondaryColors) { - for (Component c : { PitchTrack, Notes }) { - if (m_layers[c]) { - auto params = m_layers[c]->getPlayParameters(); - if (params) params->setPlayAudible(false); - } - } - } + silenceSecondaryAnalysisLayers(); stackLayers(); @@ -538,10 +536,10 @@ Analyser::buildAnalysisTransforms(Transforms &transforms) TransformFactory *tf = TransformFactory::getInstance(); - QString plugname = "pYIN"; - QString base = "vamp:pyin:pyin:"; - QString f0out = "smoothedpitchtrack"; - QString noteout = "notes"; + QString plugname = pyinPlugin; + QString base = pyinBase; + QString f0out = pyinPitchOutput; + QString noteout = pyinNotesOutput; QString notFound = tr("Transform \"%1\" not found. Unable to analyse audio file.

Is the %2 Vamp plugin correctly installed?"); if (!tf->haveTransform(base + f0out)) { @@ -578,7 +576,7 @@ Analyser::buildAnalysisTransforms(Transforms &transforms) Transform t = tf->getDefaultTransformFor (base + f0out, waveFileModel->getSampleRate()); - t.setStepSize(256); + t.setStepSize(analysisStepSize); t.setBlockSize(2048); if (precise) { @@ -663,7 +661,132 @@ Analyser::addAnalyses() m_document->addLayerToView(m_pane, layers[i]); } - + + configureAnalysisLayers(); + connectAnalysisLayers(); + + return ""; +} + +QString +Analyser::addEmptyAnalyses() +{ + // The first recording of a take has no analysis to keep and none to + // run: what it sings is analysed over its own range and merged into + // the pitch track and notes of the take (spec 6.2, last paragraph), + // which therefore have to exist, empty, first. They are made here + // rather than in MainWindow so that they are the same layers on the + // same kind of model as a whole-file analysis leaves behind -- type, + // resolution, units, colours, names, play parameters and the source + // model that lets an analyser claim them again after a swap or a + // session restore. + auto waveFileModel = ModelById::getAs(m_fileModel); + if (!waveFileModel) { + return "Internal error: Analyser::addEmptyAnalyses() called with no model present"; + } + if (!m_document || !m_pane) { + return "Internal error: Analyser::addEmptyAnalyses() called with no document or pane present"; + } + + // Nothing to do for a take that has them: the usual case + if (m_layers[PitchTrack] && m_layers[Notes]) return ""; + + // Half a pair is partial state from a failed analysis, and no use + // to the merge either + for (Component c : { PitchTrack, Notes }) { + if (m_layers[c]) { + m_document->removeLayerFromView(m_pane, m_layers[c]); + m_layers[c] = nullptr; + } + } + + sv_samplerate_t rate = waveFileModel->getSampleRate(); + + // The names a transform's output models are given + // (ModelTransformerFactory::transformMultiple): they are what the + // layer's presentation name and the session file show + TransformFactory *tf = TransformFactory::getInstance(); + QString sourceName = waveFileModel->objectName(); + + struct Wanted { + Component component; + LayerFactory::LayerType layerType; + QString output; + }; + + const Wanted wanted[] = { + { PitchTrack, LayerFactory::TimeValues, pyinPitchOutput }, + { Notes, LayerFactory::FlexiNotes, pyinNotesOutput } + }; + + for (const Wanted &w : wanted) { + + // The layer first: a model registered with the document is not + // ours to release again (see the ownership rules in the dev doc), + // so nothing is registered until there is a layer to hold it. + // Not createEmptyLayer(): that makes a model of its own, on the + // main model's sample rate and with a resolution of 1 + Layer *layer = m_document->createLayer(w.layerType); + if (!layer) { + return "Internal error: Analyser::addEmptyAnalyses() could not create a layer"; + } + + // One value per hop of the pitch track, notes with a duration + // at the same resolution, both in Hz, as pYIN's outputs are + // described. Notified on add, which is the state a transform's + // model is switched to when it completes; these are complete + // from the start + std::shared_ptr model; + if (w.component == PitchTrack) { + auto pitch = std::make_shared + (rate, analysisStepSize, true); + pitch->setScaleUnits("Hz"); + model = pitch; + } else { + auto notes = std::make_shared + (rate, analysisStepSize, true, NoteModel::FLEXI_NOTE); + notes->setScaleUnits("Hz"); + model = notes; + } + + QString transformName = + tf->getTransformFriendlyName(pyinBase + w.output); + if (sourceName != "" && transformName != "") { + model->setObjectName(tr("%1: %2").arg(sourceName) + .arg(transformName)); + } else if (transformName != "") { + model->setObjectName(transformName); + } + + // The link a whole-file analysis makes by deriving the model + // from the audio: it is what claimExistingAnalyses() looks for + model->setSourceModel(m_fileModel); + + ModelId modelId = ModelById::add(model); + m_document->addNonDerivedModel(modelId); + + m_document->setModel(layer, modelId); + m_document->addLayerToView(m_pane, layer); + m_layers[w.component] = layer; + } + + configureAnalysisLayers(); + connectAnalysisLayers(); + + // As doAllAnalyses() does for the layers it has just made + loadState(PitchTrack); + loadState(Notes); + silenceSecondaryAnalysisLayers(); + stackLayers(); + + emit layersChanged(); + + return ""; +} + +void +Analyser::configureAnalysisLayers() +{ ColourDatabase *cdb = ColourDatabase::getInstance(); // Choose colors based on color scheme: @@ -673,8 +796,8 @@ Analyser::addAnalyses() ? tr("Orange") : tr("Black"); QString notesColour = (m_colorScheme == SecondaryColors) ? tr("Bright Purple") : tr("Bright Blue"); - - TimeValueLayer *pitchLayer = + + TimeValueLayer *pitchLayer = qobject_cast(m_layers[PitchTrack]); if (pitchLayer) { pitchLayer->setBaseColour(cdb->getColourIndex(pitchColour)); @@ -685,7 +808,7 @@ Analyser::addAnalyses() } } - FlexiNoteLayer *flexiNoteLayer = + FlexiNoteLayer *flexiNoteLayer = qobject_cast(m_layers[Notes]); if (flexiNoteLayer) { flexiNoteLayer->setBaseColour(cdb->getColourIndex(notesColour)); @@ -695,10 +818,24 @@ Analyser::addAnalyses() params->setPlayGain(0.5); } } +} - connectAnalysisLayers(); +void +Analyser::silenceSecondaryAnalysisLayers() +{ + // The secondary analyser's pitch and note tracks are visual-only (there is + // no UI toggle to control their audibility, and sonifying two pitch/note + // tracks at once is confusing). Mute them directly — do NOT call + // setAudible(), which would also call saveState() and corrupt the primary + // analyser's shared settings key. + if (m_colorScheme != SecondaryColors) return; - return ""; + for (Component c : { PitchTrack, Notes }) { + if (m_layers[c]) { + auto params = m_layers[c]->getPlayParameters(); + if (params) params->setPlayAudible(false); + } + } } bool @@ -933,9 +1070,19 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, sv_samplerate_t rate = waveFileModel->getSampleRate(); if (clipStart < 0) clipStart = 0; - if (clipEnd < 0 || clipEnd > waveFileModel->getEndFrame()) { - clipEnd = waveFileModel->getEndFrame(); + + if (waveFileModel->isReady()) { + sv_frame_t fileEnd = waveFileModel->getEndFrame(); + if (clipEnd < 0 || clipEnd > fileEnd) clipEnd = fileEnd; + } else if (clipEnd < 0) { + // A file that is still being decoded reports a frame count that + // is still growing, so it cannot say where its end is; the + // caller has not said either. Whatever is asked for, the run + // stops at the end of the file, and the transform waits for the + // model before it reads any of it + clipEnd = end; } + if (start < clipStart) start = clipStart; if (end > clipEnd) end = clipEnd; if (end <= start) return ""; @@ -954,7 +1101,7 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, // notes of a ranged run are placed relative to its first block (see // mergeRangedAnalysis()), so only a start on the grid puts them // where a whole-file run would have put them - const sv_frame_t grid = 256; + const sv_frame_t grid = analysisStepSize; from = (from / grid) * grid; to = ((to + grid - 1) / grid) * grid; if (to <= from) return ""; @@ -1155,6 +1302,23 @@ Analyser::mergeRangedAnalysis() } } + // The end of an old note that carries on past the end of the run. + // The run had to stop singing that note where it stopped listening, + // and the audio out there has not changed, so the old note is the + // one that knows where it really ends. Not where the run's end is + // the edge of the coverage: there is nothing beyond that but + // silence, and no old note to believe + sv_frame_t endBeyondRun = -1; + if (!m_rangedClippedEnd) { + for (const Event &e : oldNotes) { + if (e.getFrame() < m_rangedEnd && + e.getFrame() + e.getDuration() > m_rangedEnd) { + endBeyondRun = e.getFrame() + e.getDuration(); + break; + } + } + } + for (const Event &e : oldNotes) { sv_frame_t f = e.getFrame(); if (f >= wFrom && f < noteTo) { @@ -1171,7 +1335,18 @@ Analyser::mergeRangedAnalysis() notes->add(e.withDuration(firstAdded - f)); } } - for (const Event &e : adding) { + for (Event e : adding) { + // A note that the end of the run cut off goes on to where the + // old note it belongs to ended. A few hops of slack: the run's + // last note ends within a block or so of where it stopped + if (endBeyondRun > 0) { + sv_frame_t end = e.getFrame() + e.getDuration(); + sv_frame_t slack = 4 * analysisStepSize; + if (end > m_rangedEnd - slack && end < m_rangedEnd + slack && + endBeyondRun > end) { + e = e.withDuration(endBeyondRun - e.getFrame()); + } + } // The far edge the same way round: an old note that begins at or // after the window keeps its onset, and a new note that would run // over it is cut back diff --git a/main/Analyser.h b/main/Analyser.h index dd799bc8..d5519ed5 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -205,6 +205,19 @@ class Analyser : public QObject, sv::sv_frame_t clipStart = 0, sv::sv_frame_t clipEnd = -1); + /** + * Make an empty pitch track and empty notes for our audio, where a + * whole-file analysis would have made them full: the first + * recording of a take has nothing to keep, but analyseRange() needs + * models to merge its result into (spec 6.2, last paragraph). The + * layers, their models and everything set on them are as an + * analysed pair's are, so that this analyser's own scan for + * existing layers, a swap and a session restore cannot tell the two + * apart. Does nothing if both layers are there already; an odd one + * out is replaced. "" on success, else an error string. + */ + QString addEmptyAnalyses(); + /** * Return true between the start of a ranged analysis and the merge * (or the abandonment) of its result. @@ -364,6 +377,14 @@ protected slots: QString addWaveform(); QString addAnalyses(); + // The colours and play parameters of the pitch and notes layers, + // whichever way they were made + void configureAnalysisLayers(); + + // The singing track's pitch and notes are to be seen and not heard: + // there is a pitch track and a set of notes being sonified already + void silenceSecondaryAnalysisLayers(); + // The two pYIN transforms of a full analysis (smoothed pitch track // and notes) with the parameters the settings ask for. Shared with // analyseRange(), which must produce within its range just what a diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index b0061bdd..67b4a48d 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -203,19 +203,18 @@ class TestSingingAnalysis : public QObject return model->getAllEvents(); } - // The singing track as 4c will set it up for the first recording of - // a take: empty pitch and notes layers on the singing model, which a - // deferred analyser claims without analysing anything (4a) - void addEmptyAnalyses(sv::ModelId singing) { - for (auto type : { sv::LayerFactory::TimeValues, - sv::LayerFactory::FlexiNotes }) { - sv::Layer *layer = m_document->createEmptyLayer(type); - QVERIFY(layer); - auto model = sv::ModelById::get(layer->getModel()); - QVERIFY(model); - model->setSourceModel(singing); - m_document->addLayerToView(m_pane, layer); - } + // The singing track as MainWindow sets it up for the first recording + // of a take: the analyser takes the audio without analysing any of it + // and makes empty pitch and notes layers for the analysis of the + // recorded range to be merged into + void setUpEmpty(Analyser &analyser, sv::ModelId singing) { + QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, + m_pane, true), QString()); + QCOMPARE(analyser.addEmptyAnalyses(), QString()); + QVERIFY(analyser.getLayer(Analyser::PitchTrack)); + QVERIFY(analyser.getLayer(Analyser::Notes)); + QVERIFY(pitchEvents(analyser).empty()); + QVERIFY(noteEvents(analyser).empty()); } // Wait for a ranged analysis to be merged @@ -684,17 +683,9 @@ private slots: } QCOMPARE(int(wholeNotes.size()), 4); - sv::ModelId singing = addSingingModel(data); - addEmptyAnalyses(singing); - if (QTest::currentTestFailed()) return; - Analyser analyser(Analyser::SecondaryColors); - QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, - m_pane, true), QString()); - QVERIFY(analyser.getLayer(Analyser::PitchTrack)); - QVERIFY(analyser.getLayer(Analyser::Notes)); - QVERIFY(pitchEvents(analyser).empty()); - QVERIFY(noteEvents(analyser).empty()); + setUpEmpty(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; size_t layers = m_document->getLayers().size(); size_t models = m_document->getModels().size(); @@ -943,13 +934,9 @@ private slots: mergeWindow(start, end, 0, fileEnd, wFrom, wTo); QVERIFY(wFrom < frameAt(1.4) && wTo > frameAt(2.2)); - sv::ModelId singing = addSingingModel(data); - addEmptyAnalyses(singing); - if (QTest::currentTestFailed()) return; - Analyser analyser(Analyser::SecondaryColors); - QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, - m_pane, true), QString()); + setUpEmpty(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; auto notes = notesModel(analyser); QVERIFY(notes); @@ -990,6 +977,56 @@ private slots: QCOMPARE(now[2].getDuration(), after.getDuration()); } + void ranged_keeps_the_end_of_a_note_past_the_run() { + // A note that begins inside the merge window and is still going + // when the run ends: the run had to cut it off where it stopped + // listening, but the audio out there has not changed, so the note + // ends where the old note that covered the run's end ended + std::vector data; + appendSilence(data, 1.4); + auto lng = tone(singingHz, 2.4); // 1.4 to 3.8 s + data.insert(data.end(), lng.begin(), lng.end()); + appendSilence(data, 0.3); + + sv::sv_frame_t fileEnd = sv::sv_frame_t(data.size()); + sv::sv_frame_t start = frameAt(1.5), end = frameAt(2.0); + sv::sv_frame_t from, to, wFrom, wTo; + widenRange(start, end, 0, fileEnd, from, to); + mergeWindow(start, end, 0, fileEnd, wFrom, wTo); + // The note begins inside the window and outlasts the run + QVERIFY(wFrom < frameAt(1.4) && wTo > frameAt(1.4)); + QVERIFY(to < frameAt(3.5)); + + Analyser analyser(Analyser::SecondaryColors); + analyse(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + + sv::EventVector wasNotes = noteEvents(analyser); + QCOMPARE(int(wasNotes.size()), 1); + sv::sv_frame_t wasEnd = + wasNotes[0].getFrame() + wasNotes[0].getDuration(); + QVERIFY2(wasEnd > to, + qPrintable(QString("the long note (%1 for %2) does not " + "outlast the run, which ends at %3") + .arg(wasNotes[0].getFrame()) + .arg(wasNotes[0].getDuration()).arg(to))); + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(start, end, 0, fileEnd), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector now = noteEvents(analyser); + QVERIFY2(now.size() == 1, + qPrintable("notes after the merge: " + describeNotes(now))); + QVERIFY(std::abs(now[0].getFrame() - wasNotes[0].getFrame()) <= + 4 * hop); + QVERIFY2(now[0].getFrame() + now[0].getDuration() == wasEnd, + qPrintable(QString("the note %1 was cut off at the end of " + "the run (%2); it used to end at %3") + .arg(describeNotes(now)).arg(to).arg(wasEnd))); + } + void ranged_at_the_edge_of_coverage() { // Where the caller's coverage limit clips an edge of the run there // is nothing beyond it but silence and no context worth keeping, @@ -1024,12 +1061,9 @@ private slots: // and the run is the whole of it. Nothing may be dropped at // either end, so the result is the whole-file result { - sv::ModelId singing = addSingingModel(data); - addEmptyAnalyses(singing); - if (QTest::currentTestFailed()) return; Analyser analyser(Analyser::SecondaryColors); - QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, - m_pane, true), QString()); + setUpEmpty(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); QCOMPARE(analyser.analyseRange(0, coverEnd, 0, coverEnd), QString()); @@ -1213,13 +1247,10 @@ private slots: widenRange(start, end, 0, fileEnd, from, to); QVERIFY(abandoned1 < from); // the two ranges do not meet - sv::ModelId singing = addSingingModel(data); - addEmptyAnalyses(singing); + Analyser analyser(Analyser::SecondaryColors); + setUpEmpty(analyser, addSingingModel(data)); if (QTest::currentTestFailed()) return; - Analyser analyser(Analyser::SecondaryColors); - QCOMPARE(analyser.newFileLoaded(m_document, singing, m_paneStack, - m_pane, true), QString()); size_t layers = m_document->getLayers().size(); size_t models = m_document->getModels().size(); From 67fc29631e7df55c5262970c5bfd256f7dc1d1c1 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 04:18:57 +0300 Subject: [PATCH 039/275] feat: stop swaps the take's audio and analyses only what was recorded rebuildSingingTrackFromTake(placed) replaces the phase-2 stand-in (tear the singing track down and run pYIN over the whole of the take's audio). It swaps the new audio under the take's pitch and notes layers (4a) and analyses the range the splice wrote (4b), with the context limited to the coverage range that range sits in, so the margin never reaches into silence that was never recorded. The live dots go when that analysis reports, as before; if it merged before the call returned there is nothing to wait for and they go at once. The first recording of a take has no layers to keep: its audio is opened with no analysis of its own (loadTakeAudio()) and given empty pitch and notes (Analyser::addEmptyAnalyses()). Recording again while an analysis runs loses that analysis -- the swap releases the models it is merged into -- so the range it was asked for is remembered in m_takeAnalysisRange and the next analysis covers both. Analyse Now analyses all of the take's coverage as well (spec 7), as one run from the first range to the last. A take that has been recorded into is saved with pitch and notes layers that belong to no analyser, the model they were derived from having gone with the swap. adoptTakeLayers() gives them the take's audio as their source model just before the analyser that is to claim them is made: on a session restore (analyseRestoredSingingModel(), the queued call from modelAdded()) and in loadTakeAudio(). Without it a restored take was analysed from scratch, beside its own restored layers. If the swap or the analysis fails after the splice, the take's audio file and coverage hold the recording but the screen does not: that is now said in a dialog rather than left to disagree quietly. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 246 +++++++++++++++++++++++++++--- main/MainWindow.h | 51 +++++-- main/test/TestRecordWorkflow.h | 269 +++++++++++++++++++++++++++++++++ 3 files changed, 538 insertions(+), 28 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 8d97d415..578273ea 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -98,6 +98,7 @@ #include #include +#include #include #include #include @@ -2182,6 +2183,7 @@ MainWindow::closeSession() m_takePosition = 0; m_takePreRoll = 0; m_takeEnd = -1; + m_takeAnalysisRange = Coverage::Range(); m_analyser->fileClosed(); @@ -2478,6 +2480,22 @@ MainWindow::analyseNewSingingModel() setupSingingTrackAnalyser(singingModelId); } +void +MainWindow::analyseRestoredSingingModel() +{ + // The queued call from modelAdded(), which is how the singing track + // of a session being restored arrives: loadSingingTrack() and the + // take's own paths have set their analyser up synchronously by now + // and left nothing pending. A take that has been recorded into has + // pitch and notes layers in the pane that belong to no analyser, and + // the one about to be made is to claim them rather than analyse all + // of the file again + if (m_pendingSingingModelId.isNone()) return; + + adoptTakeLayers(m_pendingSingingModelId); + analyseNewSingingModel(); +} + void MainWindow::openBackgroundMusic() { @@ -3737,12 +3755,10 @@ MainWindow::finishSingingTake() // The take has stopped, and what the device recorded is raw material: // it goes into the take's audio file at the position the take was // started from, with the latency and the lead-in taken off its front, - // and the singing track is then rebuilt from the file that comes out. - // - // (Rebuilt means analysed in full, as a singing track loaded from a - // file is. Phase 4 of the takes work puts the new audio under the - // pitch and notes layers that are there and analyses only the range - // that changed, which is what makes this quick on a long song.) + // and the singing track is then shown and analysed from the file that + // comes out -- the new audio under the pitch and notes layers that + // are there, with only the range that changed analysed, which is what + // makes this quick on a long song. stopTakePolling(); refineRecordingLatency(); @@ -3825,26 +3841,210 @@ MainWindow::finishSingingTake() << placed.start << "," << placed.end << ") of " << m_takes->getAudioPath() << endl; - bool rebuilt = rebuildSingingTrackFromTake(); + bool analysing = rebuildSingingTrackFromTake(placed); // The dots stay until the analysis that replaces them is done - recordingFinishedFull(rebuilt ? m_analyser2 : nullptr); + recordingFinishedFull(analysing ? m_analyser2 : nullptr); } bool -MainWindow::rebuildSingingTrackFromTake() +MainWindow::rebuildSingingTrackFromTake(const Coverage::Range &placed) { if (!m_takes->haveTake()) return false; - Analyser *before = m_analyser2; + // What the analysis that follows has to cover: the range the splice + // has just written, and any range whose analysis is still running. + // That one's result is lost -- the swap below releases the models it + // is being written into, and its analyser with them -- so it is + // analysed again rather than left half done + Coverage::Range analyse = placed; + if (m_analyser2 && m_analyser2->isAnalysingRange() && + m_takeAnalysisRange.length() > 0) { + analyse.start = std::min(analyse.start, m_takeAnalysisRange.start); + analyse.end = std::max(analyse.end, m_takeAnalysisRange.end); + } + m_takeAnalysisRange = Coverage::Range(); + + QString path = m_takes->getAudioPath(); + + // A take with pitch and notes on show keeps them: the new audio goes + // underneath, and only the range that changed is analysed. The + // first recording of a take has neither, so its audio is opened as a + // singing track with no analysis of its own and given empty ones + bool haveLayers = m_analyser2 && + m_analyser2->getLayer(Analyser::PitchTrack) && + m_analyser2->getLayer(Analyser::Notes); - // The same path as File -> Load Singing Track, but the coverage of - // this file is the one the splice worked out, not "all of it" + QString error = (haveLayers ? swapSingingAudio(path) + : loadTakeAudio(path)); + + if (error == "" && m_analyser2) { + error = m_analyser2->addEmptyAnalyses(); + } + + if (error != "") { + // The recording is in the take's audio file and the take's + // coverage says so, but the screen does not: say so rather than + // leave the two quietly disagreeing + QMessageBox::warning + (this, + tr("Failed to show the recording"), + tr("The recording was added to the singing track, but it " + "could not be shown

%1

What is on screen is the " + "singing track as it was. The recording is in the take's " + "audio file, \"%2\".

").arg(error).arg(path), + QMessageBox::Ok); + return false; + } + + return startTakeAnalysis(analyse.start, analyse.end); +} + +QString +MainWindow::loadTakeAudio(QString path) +{ + // The take's audio, opened and shown with no analysis of its own: + // what is analysed is the range a recording has just gone into, and + // the pitch and notes of the rest of the take are either in the pane + // already or about to be made empty. swapSingingAudio() is this for + // a take that has its layers; the two are the same but for the + // handing over + ModelId audio; + std::vector extraPanes; + FileOpenStatus status = openSingingAudioFile(path, audio, extraPanes); + + if (status != FileOpenSucceeded || audio.isNone()) { + for (Pane *extra : extraPanes) pruneExtraPane(extra, audio); + return tr("The file \"%1\" could not be opened as audio").arg(path); + } + + // Not the analysis we want: it would be of the whole file + m_pendingSingingModelId = {}; + + // Layers of the take that no analyser owns are handed to the one + // about to be made, as after a session restore + adoptTakeLayers(audio); + + // m_rebuildingTakeAudio: the coverage of this file is the one the + // splice worked out, not "the whole of it" + bool wasRebuilding = m_rebuildingTakeAudio; m_rebuildingTakeAudio = true; - loadSingingTrack(m_takes->getAudioPath()); - m_rebuildingTakeAudio = false; + setupSingingTrackAnalyser(audio, true); + m_rebuildingTakeAudio = wasRebuilding; + + // The orphan waveform in the extra pane can go now that the + // analyser's own waveform layer holds the audio + for (Pane *extra : extraPanes) pruneExtraPane(extra, audio); + + if (!m_analyser2) { + return tr("The singing track could not be set up on \"%1\"").arg(path); + } - return (m_analyser2 != nullptr && m_analyser2 != before); + return ""; +} + +bool +MainWindow::adoptTakeLayers(ModelId audio) +{ + // An analyser claims a pitch or notes layer whose model has the + // analyser's own audio model as its source model + // (Analyser::claimExistingAnalyses()). The layers of a take that + // has had audio swapped under it have no such link: the model they + // were derived from is long gone, and a session file keeps them as + // ordinary layers. So the link is made here, for the layers in the + // pane that no analyser owns, just before the analyser that is to + // claim them is made. + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane || audio.isNone()) return false; + + TimeValueLayer *pitch = nullptr; + FlexiNoteLayer *notes = nullptr; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + + Layer *layer = pane->getLayer(i); + + // Ours, and not a take's: the live dots of a take being + // recorded, and the reference's pitch track moved by octaves + if (layer == m_realtimePitchLayer) continue; + if (m_alternatePitch && layer == m_alternatePitch->getLayer()) continue; + + auto model = ModelById::get(layer->getModel()); + if (!model) continue; + + // A layer whose source model is still there has an analyser of + // its own: the reference's pitch, notes and candidates, or a + // take whose audio has not been swapped since it was analysed + if (ModelById::get(model->getSourceModel())) continue; + + if (!pitch) pitch = qobject_cast(layer); + if (!notes) notes = qobject_cast(layer); + } + + // Half a pair is no use: the analyser claims both or neither + if (!pitch || !notes) return false; + + cerr << "MainWindow::adoptTakeLayers: the take's pitch and notes layers " + << "come from model " << audio << " now" << endl; + + for (ModelId id : { pitch->getModel(), notes->getModel() }) { + if (auto model = ModelById::get(id)) { + model->setSourceModel(audio); + } + } + + return true; +} + +bool +MainWindow::startTakeAnalysis(sv_frame_t start, sv_frame_t end) +{ + if (!m_analyser2 || end <= start) return false; + + // How far the analysis may reach for the context it needs: as far as + // the material the take has, and no further. The margin may run + // into silence that has been recorded over, but silence that was + // never recorded tells pYIN nothing and is not ours to analyse + const Coverage &coverage = m_takes->getCoverage(); + sv_frame_t clipStart = start, clipEnd = end; + Coverage::Range at; + if (coverage.getRangeAt(start, at)) clipStart = at.start; + if (coverage.getRangeAt(end - 1, at)) clipEnd = at.end; + + QString error = m_analyser2->analyseRange(start, end, clipStart, clipEnd); + + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to analyse the recording"), + tr("The singing could not be analysed

%1

").arg(error), + QMessageBox::Ok); + return false; + } + + // A range short enough to have been analysed and merged before the + // call returned leaves nothing to wait for + if (!m_analyser2->isAnalysingRange()) return false; + + m_takeAnalysisRange = Coverage::Range(start, end); + return true; +} + +bool +MainWindow::analyseTakeCoverage() +{ + // Analyse Now takes in the singing of the take as well (spec 7): all + // of its coverage is analysed again and merged into its pitch and + // notes. One run from the first range to the last, not one run per + // range: only one ranged analysis can be running at a time, and pYIN + // over the silence between two ranges costs less than a queue of + // runs would. + if (!m_analyser2 || !m_takes->haveTake()) return false; + + const Coverage::Ranges &ranges = m_takes->getCoverage().getRanges(); + if (ranges.empty()) return false; + + return startTakeAnalysis(ranges.front().start, ranges.back().end); } bool @@ -5127,9 +5327,11 @@ MainWindow::modelAdded(ModelId model) // take's own audio, or one the user chose. // loadSingingTrack() runs the analysis itself as soon as // openPath() returns (it has to happen before the extra - // pane is pruned); this deferred call is the fallback for - // any other route that adds a model. - QTimer::singleShot(0, this, SLOT(analyseNewSingingModel())); + // pane is pruned); this deferred call is what sets up the + // singing track of a session being restored, and the + // fallback for any other route that adds a model. + QTimer::singleShot + (0, this, SLOT(analyseRestoredSingingModel())); } else { cerr << "modelAdded: m_pendingSingingModelId already set, ignoring model " << model << endl; @@ -5197,6 +5399,9 @@ MainWindow::analyseNow() CommandHistory::getInstance()->endCompoundOperation(); + // The singing of a take is analysed again too, over its coverage + analyseTakeCoverage(); + if (error != "") { QMessageBox::warning (this, @@ -5347,6 +5552,11 @@ MainWindow::analyseNewMainModel() // Defer so that the primary analyser's layers are fully in place // before the secondary analyser tries to share the same pane. QTimer::singleShot(0, this, [this, foundSinging]() { + // A take recorded into is saved with pitch and notes + // layers that belong to no analyser (the model they were + // derived from went with the swap that put this audio + // under them): they are this analyser's to claim + adoptTakeLayers(foundSinging); setupSingingTrackAnalyser(foundSinging); }); } diff --git a/main/MainWindow.h b/main/MainWindow.h index b1766ddb..5f6fa18f 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -77,6 +77,9 @@ protected slots: virtual void openSingingTrack(); virtual void openBackgroundMusic(); virtual void analyseNewSingingModel(); + // The same, for the singing track of a session being restored: the + // take's layers are handed to the new analyser first + virtual void analyseRestoredSingingModel(); virtual void openLocation(); virtual void openRecentFile(); virtual void saveSession(); @@ -323,12 +326,33 @@ protected slots: // answer a dialog. Returns true to go ahead with the recording. virtual bool confirmRecordingOverTake(); - // Rebuild the singing track from the take's audio file, the way - // Load Singing Track does: the layers of the file before go, the new - // file is opened and analysed in full. Returns true if a new - // analyser was set up. (Phase 4 of the takes work replaces this - // with a model swap that keeps the pitch and notes layers.) - bool rebuildSingingTrackFromTake(); + // Show and analyse the take's audio file after a recording has been + // spliced into it over the range "placed": the new file goes under + // the take's pitch and notes layers (swapSingingAudio(), or + // loadTakeAudio() and empty layers for the first recording of a + // take) and only "placed" is analysed, over the take's coverage. + // Returns true if an analysis is running that will say when it is + // done; a failure is reported to the user from here. + bool rebuildSingingTrackFromTake(const Coverage::Range &placed); + + // Open the take's audio as the singing track with no analysis of its + // own, for a take that has no pitch and notes layers to keep. + // "" on success, else a message for the user + QString loadTakeAudio(QString path); + + // Hand the take's pitch and notes layers, if they are in pane 0 and + // belong to no analyser, to the audio model about to be analysed: + // that link is what lets an Analyser claim them. True if both were + // found and linked + bool adoptTakeLayers(sv::ModelId audio); + + // Analyse [start, end) of the take's audio and merge the result into + // its pitch and notes, with the context limited to the coverage + // range the material sits in. True if a run was started + bool startTakeAnalysis(sv::sv_frame_t start, sv::sv_frame_t end); + + // Analyse all of the take's coverage again (Analyse Now, spec 7) + bool analyseTakeCoverage(); // Keep the take's existing audio out of the mix while it is being // recorded into, and put it back afterwards @@ -436,10 +460,9 @@ protected slots: // Put another audio file under the take's pitch and notes layers, // keeping those layers and everything in them. The new audio is not // analysed: what the layers hold is the analysis of all of the take - // but the range that has just changed. Returns "" on success, or a - // message for the user. (Phase 4c: Stop splices the recording into - // the take's audio, swaps to the file that comes out and analyses - // only the range the splice wrote.) + // but the range that has just changed, which + // rebuildSingingTrackFromTake() analyses on its own. Returns "" on + // success, or a message for the user. QString swapSingingAudio(QString path); // Open path as an additional audio model beside the reference, the @@ -522,6 +545,14 @@ protected slots: // as it is for a file the user loads or a session restores. bool m_rebuildingTakeAudio; + // The range of the take that the ranged analysis now running was + // asked for; empty when none is running. It is remembered here and + // not in the Analyser because the next recording replaces the + // analyser along with the audio, and the analysis it was running + // goes with it: the next one has to cover this range as well, or + // what the singer sang would be left unanalysed. + Coverage::Range m_takeAnalysisRange; + // Round-trip hardware latency (output + input, in frames at the model // sample rate) stored when a singing-track recording is made with the // "play reference while recording" toggle on. The recording is read diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 5e5d5f45..11ba5e9a 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -39,11 +39,13 @@ #include "layer/ColourDatabase.h" #include "layer/SingleColourLayer.h" #include "layer/TimeValueLayer.h" +#include "layer/FlexiNoteLayer.h" #include "layer/WaveformLayer.h" #include "audio/AudioCallbackPlaySource.h" #include "audio/AudioCallbackRecordTarget.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" +#include "data/model/NoteModel.h" #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" @@ -90,6 +92,14 @@ class TestMainWindow : public MainWindow return swapSingingAudio(path); } + // True between the start of the analysis of a recorded range and the + // merge of its result into the take's pitch and notes + bool analysingRange() { + return m_analyser2 && m_analyser2->isAnalysingRange(); + } + sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } + sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } + // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } bool isDocumentModified() { return m_documentModified; } @@ -252,6 +262,7 @@ class TestRecordWorkflow : public QObject return a && a->getLayer(Analyser::PitchTrack) && a->getLayer(Analyser::Notes) && a->getInitialAnalysisCompletion() >= 100 && + !a->isAnalysingRange() && !sv::ModelTransformerFactory::getInstance() ->haveRunningTransformers(); } @@ -300,6 +311,13 @@ class TestRecordWorkflow : public QObject : sv::EventVector(); } + static sv::EventVector noteEvents(sv::Layer *layer) { + if (!layer) return {}; + auto model = sv::ModelById::getAs(layer->getModel()); + if (!model) return {}; + return model->getAllEvents(); + } + static sv::EventVector eventsBetween(const sv::EventVector &events, sv::sv_frame_t from, sv::sv_frame_t to) { @@ -397,6 +415,18 @@ class TestRecordWorkflow : public QObject return n; } + // One for the reference and one for the take: a second pair for the + // take means its layers were analysed again instead of being claimed + int noteLayersInPane0() { + int n = 0; + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return 0; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (qobject_cast(pane->getLayer(i))) ++n; + } + return n; + } + bool paneHasLayer(int paneIndex, sv::Layer *layer) { sv::Pane *pane = m_window->paneStack()->getPane(paneIndex); if (!pane) return false; @@ -1179,6 +1209,184 @@ private slots: "recordings"); } + // Stop no longer analyses the whole of the take's audio: the new + // audio goes under the pitch and notes layers that are there and only + // the range the recording went into is analysed and merged into them. + // So a second take elsewhere leaves the first recording's events + // exactly as they were, in the very same layers and models. + void take_analyses_only_the_new_range() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + take(900); + if (QTest::currentTestFailed()) return; + + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + sv::Layer *pitch = a2->getLayer(Analyser::PitchTrack); + sv::Layer *notes = a2->getLayer(Analyser::Notes); + QVERIFY(pitch && notes); + sv::ModelId pitchModel = pitch->getModel(); + sv::ModelId notesModel = notes->getModel(); + sv::ModelId audio = a2->getMainModelId(); + auto before = pitchEvents(pitch); + auto notesBefore = noteEvents(notes); + QVERIFY(before.size() > 50); + QVERIFY(!notesBefore.empty()); + + // Two seconds on: further than the half second of context the + // analysis of the second recording takes for itself + const sv::sv_frame_t P = sv::sv_frame_t(2.5 * rate); + m_window->seekTo(P); + take(700); + if (QTest::currentTestFailed()) return; + + // The analysis of the take was not thrown away and run again: the + // layers, and the models under them, are the same objects + Analyser *after = m_window->analyser2(); + QVERIFY(after); + QCOMPARE(after->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(after->getLayer(Analyser::Notes), notes); + QCOMPARE(pitch->getModel(), pitchModel); + QCOMPARE(notes->getModel(), notesModel); + QVERIFY2(after->getMainModelId() != audio, + "the take's audio is not the file the splice wrote"); + QVERIFY2(!sv::ModelById::get(audio), + "the audio the take had before was not released"); + + // and every event before the second recording is the very same + // event, frame and value: nothing there was analysed again + auto keptPitch = eventsBetween(pitchEvents(pitch), 0, P); + auto wasPitch = eventsBetween(before, 0, P); + QCOMPARE(keptPitch.size(), wasPitch.size()); + for (size_t i = 0; i < wasPitch.size(); ++i) { + QCOMPARE(keptPitch[i].getFrame(), wasPitch[i].getFrame()); + QCOMPARE(keptPitch[i].getValue(), wasPitch[i].getValue()); + } + + auto keptNotes = eventsBetween(noteEvents(notes), 0, P); + auto wasNotes = eventsBetween(notesBefore, 0, P); + QCOMPARE(keptNotes.size(), wasNotes.size()); + for (size_t i = 0; i < wasNotes.size(); ++i) { + QCOMPARE(keptNotes[i].getFrame(), wasNotes[i].getFrame()); + QCOMPARE(keptNotes[i].getDuration(), wasNotes[i].getDuration()); + QCOMPARE(keptNotes[i].getValue(), wasNotes[i].getValue()); + } + + // The second recording was analysed, and in the right place + QVERIFY(!eventsBetween(pitchEvents(pitch), P, + P + sv::sv_frame_t(0.6 * rate)).empty()); + QCOMPARE(m_window->analysedRangeStart(), P); + verifyPlaySourceClean(); + } + + // Recording again while the analysis of the range just recorded is + // still running. That analysis is lost -- the swap releases the models + // it was to be merged into -- so the analysis that follows has to + // cover both ranges, or the first recording would have no pitch track + // at all (the first recording of a take starts from empty models). + void take_analysis_covers_the_range_it_lost() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(900); + + // Stop splices the recording in and starts the analysis of the + // range it went into there and then + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY2(m_window->analysingRange(), + "the race was not set up: no range was being analysed when " + "the second take started"); + QCOMPARE(m_window->analysedRangeStart(), sv::sv_frame_t(0)); + sv::sv_frame_t firstEnd = m_window->analysedRangeEnd(); + QVERIFY(firstEnd > sv::sv_frame_t(0.7 * rate)); + + // A second take in a gap, recorded without letting the event + // loop run: the result of a ranged analysis is merged from a + // queued call, so the first one cannot have finished by the time + // this one stops, however quick the machine is. (The device + // records from a thread of its own, and the record target's ring + // buffer holds ten seconds.) + const sv::sv_frame_t P = sv::sv_frame_t(3.0 * rate); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QThread::msleep(250); + QVERIFY2(m_window->analysingRange(), + "the first range's analysis finished before the second take " + "stopped: something ran the event loop"); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + + // The analysis now running covers both recordings + QVERIFY(m_window->analysingRange()); + QCOMPARE(m_window->analysedRangeStart(), sv::sv_frame_t(0)); + QVERIFY(m_window->analysedRangeEnd() > P); + + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + auto events = pitchEvents(m_window->analyser2()); + QVERIFY2(!eventsBetween(events, 0, firstEnd).empty(), + "the first recording was left without a pitch track when " + "the second one interrupted its analysis"); + QVERIFY(!eventsBetween(events, P, + P + sv::sv_frame_t(0.2 * rate)).empty()); + verifyPlaySourceClean(); + } + + // The models the analysis of a recorded range is to be merged into, + // torn down while it is still running: by another singing track, and + // by the session going. A regression guard for the area this fork has + // crashed in before -- a crash is the failure. + void range_analysis_torn_down_while_running() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(900); + m_window->doRecord(); + QVERIFY2(m_window->analysingRange(), + "the race was not set up: nothing was being analysed after " + "Stop"); + + // Another singing track over it + m_window->loadSingingTrack(writeWav(tone(lowHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QVERIFY(!m_window->analysingRange()); + verifyPlaySourceClean(); + if (QTest::currentTestFailed()) return; + + // and the same again, with the session closed under it + m_window->seekTo(0); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(700); + m_window->doRecord(); + QVERIFY(m_window->analysingRange()); + + m_window->doCloseSession(); + QVERIFY(!m_window->analyser2()); + QVERIFY(!m_window->takes()->haveTake()); + QCOMPARE(m_window->paneStack()->getPaneCount(), 0); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + QTest::qWait(300); // anything still queued arrives here + + // and the window still works + openReference(writeWav(tone(highHz, 1.0))); + } + // The question asked before recording over singing that is there, and // what the answer does. The dialog itself is not shown here: // TestMainWindow answers it (the real one has "Don't ask again"). @@ -1923,6 +2131,60 @@ private slots: "the reference pitch track was replaced"); } + // Analyse Now takes in the singing of the take as well (spec 7): all + // of its coverage is analysed again and merged into the pitch track + // and notes it has, which are not thrown away and made afresh + void analyse_now_reanalyses_the_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + m_window->seekTo(P); + take(600); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + + Analyser *a2 = m_window->analyser2(); + sv::Layer *pitch = a2->getLayer(Analyser::PitchTrack); + QVERIFY(pitch); + auto model = sv::ModelById::getAs + (pitch->getModel()); + QVERIFY(model); + + // Emptied by hand, so that what comes back can only have come + // from the analysis Analyse Now asks for + for (const auto &e : model->getAllEvents()) model->remove(e); + QVERIFY(pitchEvents(pitch).empty()); + + m_window->doAnalyseNow(); + + // One run over the span of the coverage, not one per range + QVERIFY(m_window->analysingRange()); + QCOMPARE(m_window->analysedRangeStart(), ranges[0].start); + QCOMPARE(m_window->analysedRangeEnd(), ranges[1].end); + + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(noteLayersInPane0(), 2); + + auto events = pitchEvents(pitch); + QVERIFY2(!eventsBetween(events, ranges[0].start, ranges[0].end).empty(), + "the first range of the coverage was not analysed again"); + QVERIFY2(!eventsBetween(events, ranges[1].start, ranges[1].end).empty(), + "the second range of the coverage was not analysed again"); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(events), highHz)) < 10.0); + } + void load_singing_track() { makeWindow(FakeAudioIO::Config()); openReference(writeWav(tone(lowHz, 1.0))); @@ -2406,6 +2668,7 @@ private slots: auto before = pitchEvents(m_window->analyser2()); QVERIFY(!before.empty()); sv::sv_frame_t firstEventBefore = before.front().getFrame(); + QCOMPARE(noteLayersInPane0(), 2); QString session = m_dir.filePath("round-trip.ton"); QVERIFY(m_window->saveSessionFile(session)); @@ -2435,7 +2698,13 @@ private slots: QVERIFY(wave); QCOMPARE(wave->getStartFrame(), sv::sv_frame_t(0)); + // The take's layers were restored and claimed, not analysed + // again: a re-analysis would have left the restored pair in the + // pane and put a second one beside it + QCOMPARE(noteLayersInPane0(), 2); + auto after = pitchEvents(a2); + QCOMPARE(after.size(), before.size()); QVERIFY(!after.empty()); QVERIFY2(std::llabs(after.front().getFrame() - firstEventBefore) <= 2 * hop, From c8a5987bffbf81b5c61dd660d5eb5b4da4cb94c4 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 05:06:23 +0300 Subject: [PATCH 040/275] feat: a coverage strip in pane 0, stored in the session The ranges of the take that hold recorded singing are drawn as the regions of a RegionLayer in pane 0, and that layer and its model are where the coverage is kept: they are ordinary document contents, so the session file keeps them, and a session that has them is where the take's coverage comes from when it is opened again. Only a session without one still counts as covering all of its audio file. The layer is display only. Its model cannot be played and is kept out of the play source, its vertical scale is the one nothing else in the pane can align to, and no tool Tony sets for pane 0 can reach it. A stock RegionLayer cannot draw a strip at the bottom of the pane: what this looks like is a thin orange line across the middle of it. Co-Authored-By: Claude Opus 5 --- main/Coverage.cpp | 23 ++++ main/Coverage.h | 18 +++ main/CoverageStrip.cpp | 211 +++++++++++++++++++++++++++++++++ main/CoverageStrip.h | 106 +++++++++++++++++ main/MainWindow.cpp | 72 ++++++++++- main/MainWindow.h | 9 ++ main/SingingTakes.cpp | 13 +- main/SingingTakes.h | 6 + main/test/TestCoverage.h | 42 +++++++ main/test/TestRecordWorkflow.h | 198 ++++++++++++++++++++++++++++++- meson.build | 2 + 11 files changed, 690 insertions(+), 10 deletions(-) create mode 100644 main/CoverageStrip.cpp create mode 100644 main/CoverageStrip.h diff --git a/main/Coverage.cpp b/main/Coverage.cpp index 33a11c0f..ac9aaa0d 100644 --- a/main/Coverage.cpp +++ b/main/Coverage.cpp @@ -101,3 +101,26 @@ Coverage::getEndFrame() const if (m_ranges.empty()) return 0; return m_ranges.back().end; } + +EventVector +Coverage::toEvents() const +{ + EventVector events; + for (const Range &r : m_ranges) { + events.push_back(Event(r.start, 0.f, r.length(), regionLabel())); + } + return events; +} + +Coverage +Coverage::fromEvents(const EventVector &events) +{ + // add() sorts and joins, so regions in any order, and two that + // touch because the user recorded twice over the same place, come + // out as the ranges this coverage would have had all along + Coverage coverage; + for (const Event &e : events) { + coverage.add(e.getFrame(), e.getFrame() + e.getDuration()); + } + return coverage; +} diff --git a/main/Coverage.h b/main/Coverage.h index 3c790f12..2a597876 100644 --- a/main/Coverage.h +++ b/main/Coverage.h @@ -16,6 +16,7 @@ #define TONY_COVERAGE_H #include "base/BaseTypes.h" +#include "base/Event.h" #include @@ -64,6 +65,23 @@ class Coverage bool operator==(const Coverage &c) const { return m_ranges == c.m_ranges; } bool operator!=(const Coverage &c) const { return !(*this == c); } + /** + * Coverage has no file format of its own: it is stored as the + * regions of the coverage strip's RegionModel, one region per + * range. These two are that conversion, and they are what a + * session load reads the coverage back with. + * + * A region's value is 0 (the strip shows where material is, not + * how much of anything) and its label is a single space, because a + * stock RegionLayer prints the value of any region that has no + * label: see CoverageStrip. + */ + sv::EventVector toEvents() const; + static Coverage fromEvents(const sv::EventVector &events); + + /// The label every coverage region carries, and why: see toEvents() + static QString regionLabel() { return " "; } + private: Ranges m_ranges; }; diff --git a/main/CoverageStrip.cpp b/main/CoverageStrip.cpp new file mode 100644 index 00000000..29be9c69 --- /dev/null +++ b/main/CoverageStrip.cpp @@ -0,0 +1,211 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "CoverageStrip.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "layer/RegionLayer.h" +#include "layer/LayerFactory.h" +#include "layer/ColourDatabase.h" +#include "data/model/RegionModel.h" +#include "base/PlayParameters.h" + +#include + +using namespace sv; + +using std::cerr; +using std::endl; + +// Coverage is in frames of the reference's timeline, so the regions are +// exact: a resolution of more than one frame would round the end of the +// last range up +static const int coverageResolution = 1; + +// Used only if the session has no main model to take the rate from, +// which cannot happen while there is a take to show coverage of +static const sv_samplerate_t defaultSampleRate = 44100; + +CoverageStrip::CoverageStrip(QObject *parent) : + QObject(parent), + m_document(nullptr), + m_pane(nullptr), + m_layer(nullptr) +{ +} + +CoverageStrip::~CoverageStrip() +{ +} + +QString +CoverageStrip::layerName() +{ + // Not translated: it is stored in the session file and adopt() looks + // it up. Phase 7a of the takes work numbers it after its take + return "Take 1 Coverage"; +} + +bool +CoverageStrip::show(Document *document, Pane *pane) +{ + if (m_layer) return true; + if (!document || !pane) return false; + + sv_samplerate_t rate = defaultSampleRate; + if (auto main = ModelById::get(document->getMainModel())) { + rate = main->getSampleRate(); + } + + auto model = std::make_shared(rate, coverageResolution); + model->setObjectName(tr("Singing Coverage")); + ModelId modelId = ModelById::add(model); + document->addNonDerivedModel(modelId); + + // Not createEmptyLayer(): see MainWindow::setupRealtimePitchLayer() + auto layer = qobject_cast + (document->createLayer(LayerFactory::Regions)); + if (!layer) { + cerr << "CoverageStrip::show: failed to create layer" << endl; + ModelById::release(modelId); + return false; + } + + document->setModel(layer, modelId); + takeLayer(document, pane, layer); + document->addLayerToView(pane, layer); + + return true; +} + +bool +CoverageStrip::adopt(Document *document, Pane *pane) +{ + if (m_layer) return true; + if (!document || !pane) return false; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + auto layer = qobject_cast(pane->getLayer(i)); + if (!layer || layer->objectName() != layerName()) continue; + if (!ModelById::isa(layer->getModel())) continue; + takeLayer(document, pane, layer); + return true; + } + + return false; +} + +void +CoverageStrip::takeLayer(Document *document, Pane *pane, RegionLayer *layer) +{ + m_document = document; + m_pane = pane; + m_layer = layer; + + connect(m_document, &Document::layerAboutToBeDeleted, + this, &CoverageStrip::layerAboutToBeDeleted, + Qt::UniqueConnection); + + configureLayer(); +} + +void +CoverageStrip::configureLayer() +{ + if (!m_layer) return; + + m_layer->setObjectName(layerName()); + m_layer->setPresentationName(tr("Singing Coverage")); + + // EqualSpaced is the one vertical scale a RegionLayer does not draw + // and that nothing else in the pane may align itself to, so the + // pane's own log-frequency scale is left exactly as it was. It puts + // the bar half way up the pane; the thin strip along the bottom that + // the design asks for needs a change in svgui + m_layer->setVerticalScale(RegionLayer::EqualSpaced); + + // The other style, PlotSegmentation, fills the whole height of the + // pane with an opaque block per region + m_layer->setPlotStyle(RegionLayer::PlotLines); + + m_layer->setBaseColour + (ColourDatabase::getInstance()->getColourIndex(tr("Orange"))); + + // A RegionModel cannot play, so there are none; if that ever + // changes, the strip is still not something to hear + if (auto params = m_layer->getPlayParameters()) { + params->setPlayAudible(false); + } +} + +void +CoverageStrip::hide() +{ + RegionLayer *layer = m_layer; + m_layer = nullptr; + + // deleteLayer(force) and nothing else: see the notes on tearing a + // layer down silently in MainWindow::teardownRealtimePitchLayer(). + // The model goes with the layer, which is its only user + if (layer && m_document) { + m_document->deleteLayer(layer, true); + } + + if (m_document) { + disconnect(m_document, nullptr, this, nullptr); + } + m_document = nullptr; + m_pane = nullptr; +} + +void +CoverageStrip::layerAboutToBeDeleted(Layer *layer) +{ + // Someone else's doing, the document being closed for instance + if (layer && layer == m_layer) { + m_layer = nullptr; + hide(); + } +} + +void +CoverageStrip::setCoverage(const Coverage &coverage) +{ + if (!m_layer) return; + + auto model = ModelById::getAs(m_layer->getModel()); + if (!model) return; + + EventVector existing = model->getAllEvents(); + if (Coverage::fromEvents(existing) == coverage) return; + + for (const Event &e : existing) model->remove(e); + for (const Event &e : coverage.toEvents()) model->add(e); +} + +ModelId +CoverageStrip::getModelId() const +{ + return m_layer ? m_layer->getModel() : ModelId(); +} + +Coverage +CoverageStrip::getCoverage() const +{ + if (!m_layer) return {}; + auto model = ModelById::getAs(m_layer->getModel()); + if (!model) return {}; + return Coverage::fromEvents(model->getAllEvents()); +} diff --git a/main/CoverageStrip.h b/main/CoverageStrip.h new file mode 100644 index 00000000..a16cdb6d --- /dev/null +++ b/main/CoverageStrip.h @@ -0,0 +1,106 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_COVERAGE_STRIP_H +#define TONY_COVERAGE_STRIP_H + +#include "Coverage.h" + +#include "base/ById.h" +#include "data/model/Model.h" + +#include +#include + +namespace sv { +class Document; +class Pane; +class Layer; +class RegionLayer; +} + +/** + * The coverage strip of a singing take: a RegionLayer in pane 0 with one + * region per range of the take's coverage. It shows the user where the + * take holds recorded material, and it is also where that coverage is + * stored: the layer and its model are ordinary document contents, so a + * session keeps them, and a session that has them is where the coverage + * of its take comes from when it is loaded again. + * + * The strip is display only. It is never the pane's selected layer (the + * caller re-stacks the editable tracks after it is created), its model + * cannot be played (a RegionModel has no play parameters), and its + * vertical scale is the "equal spaced" one, which neither draws a scale + * of its own nor lets anything align to it, so the pane's own scale is + * untouched. + * + * It looks like the alternate pitch track's class and is used the same + * way: MainWindow only wires it. + */ +class CoverageStrip : public QObject +{ + Q_OBJECT + +public: + CoverageStrip(QObject *parent = nullptr); + virtual ~CoverageStrip(); + + /** + * Create the layer in the given pane, on top of whatever is there. + * Does nothing if the layer exists already. Returns false if it + * could not be created. + */ + bool show(sv::Document *document, sv::Pane *pane); + + /** + * Take over a layer that show() made in an earlier run, and that a + * session load has put back into the pane. Returns false if there + * is none; the coverage it holds is then read with getCoverage(). + */ + bool adopt(sv::Document *document, sv::Pane *pane); + + /// Delete the layer and its model from the document + void hide(); + + bool isShown() const { return m_layer != nullptr; } + + /// The ranges the strip shows, which are the take's coverage + void setCoverage(const Coverage &coverage); + Coverage getCoverage() const; + + sv::RegionLayer *getLayer() const { return m_layer; } + + /// The model the regions are in, which is the stored coverage + sv::ModelId getModelId() const; + + /** + * The layer's object name, which is how adopt() knows it again. + * One function because phase 7a of the takes work names the layer + * after its take. + */ + static QString layerName(); + +private slots: + void layerAboutToBeDeleted(sv::Layer *); + +private: + sv::Document *m_document; + sv::Pane *m_pane; + sv::RegionLayer *m_layer; + + void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *); + void configureLayer(); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 578273ea..917f9937 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -135,6 +135,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_alternatePitchDownAction(nullptr), m_referencePitchHiddenForTake(false), m_takes(nullptr), + m_coverageStrip(nullptr), m_takePosition(0), m_takePreRoll(0), m_takeEnd(-1), @@ -361,6 +362,7 @@ MainWindow::MainWindow(AudioMode audioMode, this, SLOT(syncAlternatePitchTrack())); m_takes = new SingingTakes(this); + m_coverageStrip = new CoverageStrip(this); // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end @@ -445,6 +447,8 @@ MainWindow::~MainWindow() // document's layers, it does not own them delete m_alternatePitch; m_alternatePitch = nullptr; + delete m_coverageStrip; + m_coverageStrip = nullptr; delete m_analyser; delete m_keyReference; Profiles::getInstance()->dump(); @@ -2169,6 +2173,7 @@ MainWindow::closeSession() teardownSingingTrackAnalyser(); teardownBackgroundMusic(); m_alternatePitch->hide(); + m_coverageStrip->hide(); m_referencePitchHiddenForTake = false; m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; @@ -2744,6 +2749,43 @@ MainWindow::updateAlternatePitchForTake() } } +void +MainWindow::syncCoverageStrip() +{ + // The strip is the store of the take's coverage as well as the + // picture of it, so it follows every change to the take: a recording + // spliced in, a file loaded, the session closed + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!m_document || !pane) return; + + if (!m_takes->haveTake() || m_takes->getCoverage().isEmpty()) { + m_coverageStrip->hide(); + return; + } + + bool wasShown = m_coverageStrip->isShown(); + if (!m_coverageStrip->show(m_document, pane)) return; + + // The play source takes in the model of every layer that is in a + // view, whether the model can be played or not, and the models it + // holds are what say where playback ends. The strip is a picture, + // not sound: one left over from a longer singing file would hold + // playback open past the end of what there is to hear + if (m_playSource && !m_coverageStrip->getModelId().isNone()) { + m_playSource->removeModel(m_coverageStrip->getModelId()); + } + + m_coverageStrip->setCoverage(m_takes->getCoverage()); + + if (!wasShown) { + // The new layer is on top, where the editing tools look for the + // layer to act on: put the tracks that can be edited back there, + // as setupRecordingLayer() does + m_analyser->stackLayers(); + if (m_analyser2) m_analyser2->stackLayers(); + } +} + void MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { @@ -2800,13 +2842,29 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal drainPendingExtraPanes(singingModelId); // The take's audio is the file behind this model. All of a file the - // user loaded, or one a session restored, holds recorded singing; a - // file we have just spliced ourselves has the coverage the splice - // worked out, which must not be thrown away here. + // user loaded holds recorded singing; a file we have just spliced + // ourselves has the coverage the splice worked out, which must not be + // thrown away here. A session that was saved with a coverage strip + // says for itself which parts of its take hold singing: that layer is + // in the pane already, waiting to be taken over. if (!m_rebuildingTakeAudio) { if (auto wfm = ModelById::getAs(singingModelId)) { - m_takes->setWholeFileTake(wfm->getLocation(), wfm->getFrameCount()); + Coverage stored; + if (!m_coverageStrip->isShown() && + m_coverageStrip->adopt(m_document, pane)) { + stored = m_coverageStrip->getCoverage(); + } + if (stored.isEmpty()) { + m_takes->setWholeFileTake(wfm->getLocation(), + wfm->getFrameCount()); + } else { + cerr << "MainWindow::setupSingingTrackAnalyser: the session's " + << "coverage strip has " << stored.getRanges().size() + << " range(s) of recorded singing in it" << endl; + m_takes->setTake(wfm->getLocation(), stored); + } } + syncCoverageStrip(); } // Re-stack layers so the primary pitch track stays on top @@ -3843,6 +3901,12 @@ MainWindow::finishSingingTake() bool analysing = rebuildSingingTrackFromTake(placed); + // The coverage has changed whether or not the new audio could be + // shown, and the strip says what it is now. After the rebuild, not + // before: a swap puts the pane's selected layer back as it found it, + // and the strip is never to be that + syncCoverageStrip(); + // The dots stay until the analysis that replaces them is done recordingFinishedFull(analysing ? m_analyser2 : nullptr); } diff --git a/main/MainWindow.h b/main/MainWindow.h index 5f6fa18f..c5b22b0b 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -20,6 +20,7 @@ #include "Analyser.h" #include "RealtimePitchTracker.h" #include "AlternatePitchTrack.h" +#include "CoverageStrip.h" #include "SingingTakes.h" #include "TakeTiming.h" @@ -284,6 +285,14 @@ protected slots: // wires it: it decides where a recording goes and writes the files. SingingTakes *m_takes; + // The coverage of the take, drawn in pane 0 and stored in the + // session with the layer that draws it. Display only. + CoverageStrip *m_coverageStrip; + + // Put the strip in step with the take's coverage: make it if there + // is a take and none yet, take it away when the take goes + void syncCoverageStrip(); + // Where on the reference's timeline the take being recorded, or the // one most recently recorded, starts: the playback position when // Record was pressed, or the start of the selection recorded into. diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 72a0d99a..28ce5e87 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -40,14 +40,21 @@ SingingTakes::clear() } void -SingingTakes::setWholeFileTake(QString path, sv_frame_t frames) +SingingTakes::setTake(QString path, const Coverage &coverage) { if (m_audioPath != "" && m_audioPath != path) { m_superseded.push_back(m_audioPath); } m_audioPath = path; - m_coverage.clear(); - m_coverage.add(0, frames); + m_coverage = coverage; +} + +void +SingingTakes::setWholeFileTake(QString path, sv_frame_t frames) +{ + Coverage whole; + whole.add(0, frames); + setTake(path, whole); } QString diff --git a/main/SingingTakes.h b/main/SingingTakes.h index 891d14a6..5fe0381c 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -50,6 +50,12 @@ class SingingTakes : public QObject /// Nothing recorded and nothing to go back to: a new session void clear(); + /** + * The take is this file, with the coverage a session kept for it in + * its coverage strip. + */ + void setTake(QString path, const Coverage &coverage); + /** * The take is the whole of this file: a singing track the user * loaded, or one restored from a session saved before coverage was diff --git a/main/test/TestCoverage.h b/main/test/TestCoverage.h index ccef2522..518e6db6 100644 --- a/main/test/TestCoverage.h +++ b/main/test/TestCoverage.h @@ -169,6 +169,48 @@ private slots: b.remove(150, 160); QVERIFY(a != b); } + + // Coverage is stored as the regions of the coverage strip's model, + // so it has to go there and come back unchanged + + void events_round_trip() { + Coverage c; + c.add(100, 200); + c.add(4000, 9000); + + sv::EventVector events = c.toEvents(); + QCOMPARE(int(events.size()), 2); + QCOMPARE(events[0].getFrame(), sv::sv_frame_t(100)); + QCOMPARE(events[0].getDuration(), sv::sv_frame_t(100)); + QCOMPARE(events[1].getFrame(), sv::sv_frame_t(4000)); + QCOMPARE(events[1].getDuration(), sv::sv_frame_t(5000)); + + // A stock RegionLayer prints the value of a region that has no + // label of its own, so every region has one + for (const sv::Event &e : events) { + QCOMPARE(e.getValue(), 0.f); + QCOMPARE(e.getLabel(), Coverage::regionLabel()); + } + + QCOMPARE(Coverage::fromEvents(events).getRanges(), c.getRanges()); + } + + void events_of_nothing() { + QVERIFY(Coverage().toEvents().empty()); + QVERIFY(Coverage::fromEvents(sv::EventVector()).isEmpty()); + } + + void events_sorted_and_joined() { + // Whatever order a model hands its events back in, and whether + // or not two of them meet, what comes out is coverage + sv::EventVector events; + events.push_back(sv::Event(500, 0.f, 100, Coverage::regionLabel())); + events.push_back(sv::Event(100, 0.f, 100, Coverage::regionLabel())); + events.push_back(sv::Event(200, 0.f, 100, Coverage::regionLabel())); + + QCOMPARE(Coverage::fromEvents(events).getRanges(), + (Ranges { Range(100, 300), Range(500, 600) })); + } }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 11ba5e9a..0142f38b 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -27,6 +27,7 @@ #include "../MainWindow.h" #include "../Analyser.h" +#include "../CoverageStrip.h" #include "../SingingTakes.h" #include "version.h" @@ -40,12 +41,14 @@ #include "layer/SingleColourLayer.h" #include "layer/TimeValueLayer.h" #include "layer/FlexiNoteLayer.h" +#include "layer/RegionLayer.h" #include "layer/WaveformLayer.h" #include "audio/AudioCallbackPlaySource.h" #include "audio/AudioCallbackRecordTarget.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" +#include "data/model/RegionModel.h" #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" @@ -158,6 +161,8 @@ class TestMainWindow : public MainWindow sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } + CoverageStrip *coverageStrip() { return m_coverageStrip; } + AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } void doToggleAlternatePitch() { alternatePitchToggled(); } void doStepAlternatePitch(bool up) { @@ -468,6 +473,66 @@ class TestRecordWorkflow : public QObject "the shared time ruler is no longer in the ruler pane"); } + // The coverage strip: the layer in pane 0 whose regions are the + // ranges of the take that hold recorded singing + + sv::RegionLayer *stripLayer() { + return m_window->coverageStrip()->getLayer(); + } + + int stripLayersInPane0() { + int n = 0; + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return 0; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i)->objectName() == + CoverageStrip::layerName()) ++n; + } + return n; + } + + // What the strip shows, from the regions of its model rather than + // from the object that keeps it + sv::EventVector stripEvents() { + sv::RegionLayer *layer = stripLayer(); + if (!layer) return {}; + auto model = sv::ModelById::getAs(layer->getModel()); + if (!model) return {}; + return model->getAllEvents(); + } + + // The topmost note layer of pane 0: what Pane::getTopFlexiNoteLayer() + // finds, and what NoteEditMode -- the only editing mode Tony sets for + // that pane -- acts on + sv::Layer *topNoteLayerInPane0() { + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return nullptr; + for (int i = pane->getLayerCount() - 1; i >= 0; --i) { + if (qobject_cast(pane->getLayer(i))) { + return pane->getLayer(i); + } + } + return nullptr; + } + + // One strip, its regions the take's coverage, and out of reach of the + // editing tools: not the pane's selected layer, and the layer the + // note tool acts on is still the take's notes + void verifyStripMatchesTake() { + QCOMPARE(stripLayersInPane0(), 1); + QVERIFY(stripLayer()); + QCOMPARE(stripEvents(), + m_window->takes()->getCoverage().toEvents()); + + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane && pane->getLayerCount() > 0); + QVERIFY2(pane->getSelectedLayer() != stripLayer(), + "the coverage strip is the pane's selected layer"); + QVERIFY(m_window->analyser2()); + QCOMPARE(topNoteLayerInPane0(), + m_window->analyser2()->getLayer(Analyser::Notes)); + } + int alternateLayersInDocument() { int n = 0, octaves = 0; for (sv::Layer *layer : m_window->document()->getLayers()) { @@ -2714,9 +2779,9 @@ private slots: .arg(after.front().getFrame()) .arg(firstEventBefore))); - // The take is the audio file the session pointed at. Until the - // coverage is saved too (phase 5 of the takes work), a restored - // take counts as covering all of its file + // The take is the audio file the session pointed at, and its + // coverage comes from the coverage strip the session kept: one + // recording from frame 0, so all of the file QVERIFY(m_window->takes()->haveTake()); auto ranges = m_window->takes()->getCoverage().getRanges(); QCOMPARE(int(ranges.size()), 1); @@ -2756,6 +2821,133 @@ private slots: highHz)) < 10.0); } + // The coverage strip: one region per range of the take that holds + // recorded singing, drawn in pane 0 and stored in the session + + void coverage_strip_follows_takes() { + FakeAudioIO::Config config; + config.input = tone(highHz, 5.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + // Nothing recorded, nothing to show + QVERIFY(!m_window->coverageStrip()->isShown()); + QCOMPARE(stripLayersInPane0(), 0); + + take(700); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->coverageStrip()->isShown()); + QCOMPARE(int(stripEvents().size()), 1); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // A second recording in a gap: a second bar, and the first + // exactly where it was + sv::sv_frame_t firstEnd = + m_window->takes()->getCoverage().getEndFrame(); + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + m_window->seekTo(P); + take(700); + if (QTest::currentTestFailed()) return; + QCOMPARE(int(stripEvents().size()), 2); + QCOMPARE(stripEvents()[0].getFrame(), sv::sv_frame_t(0)); + QCOMPARE(stripEvents()[0].getDuration(), firstEnd); + QCOMPARE(stripEvents()[1].getFrame(), P); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // A third that runs from inside the first range into the second: + // the two become one bar + m_window->seekTo(sv::sv_frame_t(0.5 * rate)); + take(2000); + if (QTest::currentTestFailed()) return; + QCOMPARE(int(stripEvents().size()), 1); + QCOMPARE(stripEvents()[0].getFrame(), sv::sv_frame_t(0)); + QVERIFY2(stripEvents()[0].getDuration() > P, + "the recording that joined the two ranges did not reach " + "the second of them"); + verifyStripMatchesTake(); + } + + // Coverage has no file format of its own: the strip's regions are + // where it is stored, and where it comes from when a session is + // opened again. Before this a restored take covered all of its file + void coverage_strip_survives_a_session() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + Coverage::Ranges before = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(before.size()), 2); + QVERIFY(before[0].end < before[1].start); + + QString session = m_dir.filePath("coverage.ton"); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + QVERIFY2(!m_window->coverageStrip()->isShown(), + "the coverage strip outlived the session it was made in"); + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + // The gap survived: a take restored before this phase covered all + // of its file, which would have been one range and no gap + QVERIFY(m_window->takes()->haveTake()); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), before); + + // ... and the layer showing it is the one the session restored, + // not a second one made beside it + QVERIFY(m_window->coverageStrip()->isShown()); + verifyStripMatchesTake(); + } + + // Nothing of a take's strip is left over for the next singing track + void coverage_strip_replaced_by_load_singing_track() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + QCOMPARE(int(stripEvents().size()), 1); + QVERIFY(stripEvents()[0].getFrame() > 0); + + // A track loaded whole is a take whose singing is all of it: one + // bar, from frame 0, and no second strip beside the first + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(int(stripEvents().size()), 1); + QCOMPARE(stripEvents()[0].getFrame(), sv::sv_frame_t(0)); + QCOMPARE(stripEvents()[0].getDuration(), wave->getFrameCount()); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // And the session closes without leaving the layer behind + m_window->doCloseSession(); + QVERIFY(!m_window->coverageStrip()->isShown()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QCOMPARE(stripLayersInPane0(), 0); + } + // The alternate pitch track: the reference pitch track moved by // whole octaves, as a layer of its own diff --git a/meson.build b/meson.build index d167d567..5ed872f2 100644 --- a/meson.build +++ b/meson.build @@ -1103,6 +1103,7 @@ tony_core_files = [ tony_app_files = [ 'main/AlternatePitchTrack.cpp', + 'main/CoverageStrip.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', @@ -1120,6 +1121,7 @@ tony_app_moc_files = qt.preprocess( 'main/MainWindow.h', 'main/Analyser.h', 'main/AlternatePitchTrack.h', + 'main/CoverageStrip.h', ]) qt_resource_files = qt.preprocess( From 1c8eeeaf07b9924ce8c4896cff3eb5be0afeda70 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 05:07:34 +0300 Subject: [PATCH 041/275] docs: say what really keeps the coverage strip out of the tools' way Analyser::stackLayers() does nothing in Tony: it goes through PaneStack::setCurrentLayer(), which needs a PropertyStack, and Tony's pane stack is built with NoPropertyStacks. The comment beside the call said it put the editable tracks back on top, which it does not. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 917f9937..2aedd955 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2778,9 +2778,15 @@ MainWindow::syncCoverageStrip() m_coverageStrip->setCoverage(m_takes->getCoverage()); if (!wasShown) { - // The new layer is on top, where the editing tools look for the - // layer to act on: put the tracks that can be edited back there, - // as setupRecordingLayer() does + // The new layer is on top, where a tool would look for the layer + // to act on, so the tracks that can be edited go back there, as + // setupRecordingLayer() does. That call does nothing in Tony, + // though: it goes through PaneStack::setCurrentLayer(), which + // needs a PropertyStack, and this pane stack has none. What + // really keeps the strip out of reach is that no tool Tony sets + // for pane 0 acts on a layer of this kind -- NoteEditMode takes + // the pane's top FlexiNoteLayer, and SelectMode, which snaps to + // the top layer, is set only for the ruler pane m_analyser->stackLayers(); if (m_analyser2) m_analyser2->stackLayers(); } From 257967b5bf0134760c573dff06c4d6541c6ea2ea Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 05:28:33 +0300 Subject: [PATCH 042/275] feat: the coverage strip is a band along the bottom, and not undoable svgui's RegionLayer has a strip plot style for it now, in place of two thin lines half way up the pane, and the layer is added through svapp's new Document::attachLayerToView(), so that undo cannot take the strip away and a session is not modified just by being opened. The test paints the layer and looks at the pixels. Co-Authored-By: Claude Fable 5.1 --- main/CoverageStrip.cpp | 20 +++++++++---------- main/test/TestRecordWorkflow.h | 36 ++++++++++++++++++++++++++++++++++ repoint-lock.json | 4 ++-- 3 files changed, 48 insertions(+), 12 deletions(-) diff --git a/main/CoverageStrip.cpp b/main/CoverageStrip.cpp index 29be9c69..2b56590c 100644 --- a/main/CoverageStrip.cpp +++ b/main/CoverageStrip.cpp @@ -85,7 +85,11 @@ CoverageStrip::show(Document *document, Pane *pane) document->setModel(layer, modelId); takeLayer(document, pane, layer); - document->addLayerToView(pane, layer); + + // Not addLayerToView(): the strip is part of the take, not something + // the user added, so undo must not take it away and making it does + // not count as a change to the session + document->attachLayerToView(pane, layer); return true; } @@ -129,16 +133,12 @@ CoverageStrip::configureLayer() m_layer->setObjectName(layerName()); m_layer->setPresentationName(tr("Singing Coverage")); - // EqualSpaced is the one vertical scale a RegionLayer does not draw - // and that nothing else in the pane may align itself to, so the - // pane's own log-frequency scale is left exactly as it was. It puts - // the bar half way up the pane; the thin strip along the bottom that - // the design asks for needs a change in svgui + // A band along the bottom of the pane, filled where there is + // singing: a plot style the svgui fork has for this. It has no + // vertical scale, no labels and takes no edits. EqualSpaced as well, + // so that nothing in the pane can align its scale to this layer m_layer->setVerticalScale(RegionLayer::EqualSpaced); - - // The other style, PlotSegmentation, fills the whole height of the - // pane with an opaque block per region - m_layer->setPlotStyle(RegionLayer::PlotLines); + m_layer->setPlotStyle(RegionLayer::PlotStrip); m_layer->setBaseColour (ColourDatabase::getInstance()->getColourIndex(tr("Orange"))); diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 0142f38b..e63de879 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -2857,6 +2857,42 @@ private slots: verifyStripMatchesTake(); if (QTest::currentTestFailed()) return; + // What it looks like: a filled band along the bottom of the pane + // where there is singing, and nothing in the gap (the svgui + // fork's PlotStrip style). The layer is painted on its own: what + // else is in the pane is not the point here + { + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane); + pane->resize(400, 200); + pane->setZoomLevel(sv::ZoomLevel + (sv::ZoomLevel::FramesPerPixel, 512)); + pane->setCentreFrame(sv::sv_frame_t(2.0 * rate)); + + QImage image(pane->width(), pane->height(), QImage::Format_RGB32); + image.fill(Qt::black); + { + QPainter painter(&image); + stripLayer()->paint(pane, painter, image.rect()); + } + + QRgb strip = sv::ColourDatabase::getInstance()->getColour + (stripLayer()->getBaseColour()).rgb(); + QRgb black = QColor(Qt::black).rgb(); + int y = image.height() - 3; + int inFirst = pane->getXForFrame(firstEnd / 2); + int inGap = pane->getXForFrame((firstEnd + P) / 2); + int inSecond = pane->getXForFrame(P + sv::sv_frame_t(0.3 * rate)); + QVERIFY(inFirst >= 0 && inSecond < image.width()); + QCOMPARE(image.pixel(inFirst, y), strip); + QCOMPARE(image.pixel(inSecond, y), strip); + QCOMPARE(image.pixel(inGap, y), black); + // a band, not a block, and nothing else of a region + for (int yy = 0; yy < image.height() - 12; ++yy) { + QCOMPARE(image.pixel(inFirst, yy), black); + } + } + // A third that runs from inside the first range into the second: // the two become one bar m_window->seekTo(sv::sv_frame_t(0.5 * rate)); diff --git a/repoint-lock.json b/repoint-lock.json index d28729b4..f77c2839 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,10 +7,10 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "80b3ffd4e092267a2e0ad80dd525e6e2afcaa007" + "pin": "008441c137fd233d5d9ca4ea296acb7c8dde8dbb" }, "svapp": { - "pin": "ed817c2b27d06c02c53451036417be564dc0702b" + "pin": "2e54ae75f50a737b119defa47f8b3c1c4faf0244" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" From 06b380d975e3aa13d3860746460eb3735e8c8212 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 15:49:51 +0300 Subject: [PATCH 043/275] feat: erasing a range from a take's audio, coverage and events SingingTakes::eraseRanges() clips the ranges asked for to what the take actually holds, writes the next audio file with TakeAudio::erase(), and moves the path, the coverage and the superseded list together or not at all. TakeEvents is the same change for the pitch and the notes: pure functions that say which events a model loses and which it gains, so that phase 6 can make an erase undoable without holding on to any model. Co-Authored-By: Claude Opus 5 --- main/SingingTakes.cpp | 39 +++++++ main/SingingTakes.h | 19 +++ main/TakeEvents.cpp | 103 +++++++++++++++++ main/TakeEvents.h | 62 ++++++++++ main/test/TestSingingTakes.h | 141 +++++++++++++++++++++++ main/test/TestTakeEvents.h | 217 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 ++ meson.build | 2 + 8 files changed, 590 insertions(+) create mode 100644 main/TakeEvents.cpp create mode 100644 main/TakeEvents.h create mode 100644 main/test/TestTakeEvents.h diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 28ce5e87..9dc18bca 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -20,6 +20,8 @@ #include #include +#include + using namespace sv; SingingTakes::SingingTakes(QObject *parent) : @@ -85,6 +87,43 @@ SingingTakes::spliceRecording(QString recordingPath, return ""; } +QString +SingingTakes::eraseRanges(const Coverage::Ranges &ranges, QString directory, + Coverage::Ranges *erased) +{ + if (erased) erased->clear(); + + // Only what holds singing can be erased. Silence that was never + // recorded is not ours to rewrite, and a selection that runs past + // the end of the singing must not make the file any longer + Coverage wanted; + for (const Coverage::Range &r : ranges) { + for (const Coverage::Range &covered : m_coverage.getRanges()) { + wanted.add(std::max(r.start, covered.start), + std::min(r.end, covered.end)); + } + } + if (wanted.isEmpty()) return ""; + + QString outPath = nextAudioPath(directory); + if (outPath == "") { + return tr("Could not find a name to write the singing track under, " + "in \"%1\"").arg(directory); + } + + QString error = TakeAudio::erase(m_audioPath, wanted.getRanges(), outPath); + if (error != "") return error; + + m_superseded.push_back(m_audioPath); + m_audioPath = outPath; + for (const Coverage::Range &r : wanted.getRanges()) { + m_coverage.remove(r.start, r.end); + } + + if (erased) *erased = wanted.getRanges(); + return ""; +} + bool SingingTakes::coversPosition(sv_frame_t position) const { diff --git a/main/SingingTakes.h b/main/SingingTakes.h index 5fe0381c..0099de53 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -84,6 +84,25 @@ class SingingTakes : public QObject QString directory, Coverage::Range *placed = nullptr); + /** + * Write the next audio file of the take with the given ranges made + * silent. The ranges are clipped to the take's coverage first: + * silence that was never recorded holds nothing to erase. The file + * is written into directory, under a name that nothing else is + * using; it is as long as the file it came from. + * + * On success returns "" and the take's audio is the new file, with + * the erased ranges gone from its coverage and the file before + * remembered as superseded. "erased", if it is not null, receives + * the ranges that were taken out: empty means that nothing of what + * was asked for held recorded singing, and nothing was done at all. + * On failure the take is exactly as it was and the return is a + * message for the user. + */ + QString eraseRanges(const Coverage::Ranges &ranges, + QString directory, + Coverage::Ranges *erased = nullptr); + /** * The audio files of takes that later files have replaced during * this run. They are kept until the session closes, for undo. diff --git a/main/TakeEvents.cpp b/main/TakeEvents.cpp new file mode 100644 index 00000000..3fc09099 --- /dev/null +++ b/main/TakeEvents.cpp @@ -0,0 +1,103 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeEvents.h" + +using namespace sv; + +namespace { + +// In order and apart, so that one pass over them is enough +Coverage::Ranges tidied(const Coverage::Ranges &ranges) +{ + Coverage coverage; + for (const Coverage::Range &r : ranges) coverage.add(r.start, r.end); + return coverage.getRanges(); +} + +bool covers(const Coverage::Ranges &ranges, sv_frame_t frame) +{ + for (const Coverage::Range &r : ranges) { + if (frame >= r.start && frame < r.end) return true; + } + return false; +} + +} // namespace + +TakeEvents::Change +TakeEvents::erasePitch(const EventVector &events, + const Coverage::Ranges &erased) +{ + Coverage::Ranges ranges = tidied(erased); + + Change change; + for (const Event &e : events) { + if (covers(ranges, e.getFrame())) change.removed.push_back(e); + } + return change; +} + +TakeEvents::Change +TakeEvents::eraseNotes(const EventVector &events, + const Coverage::Ranges &erased) +{ + Coverage::Ranges ranges = tidied(erased); + + Change change; + + for (const Event &e : events) { + + sv_frame_t start = e.getFrame(); + sv_frame_t end = start + e.getDuration(); + + // A note with no length is either in an erased range or not + if (end <= start) { + if (covers(ranges, start)) change.removed.push_back(e); + continue; + } + + // What is left of the note: the erased ranges taken out of it. + // Two stretches can be left, when a range was erased from the + // middle of the note + Coverage::Ranges left; + sv_frame_t from = start; + + for (const Coverage::Range &r : ranges) { + if (r.end <= from) continue; + if (r.start >= end) break; + if (r.start > from) { + left.push_back(Coverage::Range(from, r.start)); + } + from = r.end; + if (from >= end) break; + } + if (from < end) left.push_back(Coverage::Range(from, end)); + + // Nothing of this one was erased + if (left.size() == 1 && + left[0].start == start && left[0].end == end) { + continue; + } + + change.removed.push_back(e); + + for (const Coverage::Range &r : left) { + change.added.push_back + (e.withFrame(r.start).withDuration(r.length())); + } + } + + return change; +} diff --git a/main/TakeEvents.h b/main/TakeEvents.h new file mode 100644 index 00000000..882b3f7b --- /dev/null +++ b/main/TakeEvents.h @@ -0,0 +1,62 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_EVENTS_H +#define TONY_TAKE_EVENTS_H + +#include "Coverage.h" + +#include "base/Event.h" + +/** + * The pitch events and the notes of a take, edited to follow a change + * to its audio. Erasing part of a take takes the singing out of its + * audio file, and what was analysed there has to go with it. + * + * These are pure functions over event lists: they touch no model, so + * they can be tested without a window, and what they return is the + * change itself, which is what an undo command is made of. + */ +namespace TakeEvents +{ + /** + * What a model has to lose and then gain for the change to be made. + */ + struct Change { + sv::EventVector removed; + sv::EventVector added; + + bool isEmpty() const { return removed.empty() && added.empty(); } + }; + + /** + * The pitch events in the erased ranges go; nothing else changes. + */ + Change erasePitch(const sv::EventVector &events, + const Coverage::Ranges &erased); + + /** + * The notes lose what the erased ranges take from them. A note + * wholly inside a range goes. A note that a range starts inside is + * cut back to the start of the range. A note whose own start was + * erased begins again at the end of the range, shorter by what was + * taken: the singing its onset was is not there any more. A note + * with an erased range in the middle of it becomes two notes, as the + * audio did. + */ + Change eraseNotes(const sv::EventVector &events, + const Coverage::Ranges &erased); +} + +#endif diff --git a/main/test/TestSingingTakes.h b/main/test/TestSingingTakes.h index 75db641c..3a6957c3 100644 --- a/main/test/TestSingingTakes.h +++ b/main/test/TestSingingTakes.h @@ -246,6 +246,147 @@ private slots: QVERIFY(takes.getCoverage() == before); } + // Erasing from the middle of a recording: the file is as long as it + // was, silent where the range was, and the coverage is split in two + void erase_the_middle() { + SingingTakes takes; + QString recording = writeRecording(4000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QString before = takes.getAudioPath(); + + Coverage::Ranges erased; + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(1000, 2000) }, + takeDirectory(), &erased), QString()); + + QCOMPARE(int(erased.size()), 1); + QCOMPARE(erased[0], Coverage::Range(1000, 2000)); + + QCOMPARE(int(takes.getCoverage().getRanges().size()), 2); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 1000)); + QCOMPARE(takes.getCoverage().getRanges()[1], Coverage::Range(2000, 4000)); + + QString path = takes.getAudioPath(); + QVERIFY(path != before); + QCOMPARE(takes.getSupersededPaths(), QStringList { before }); + QVERIFY2(QFileInfo::exists(before), + "the file the take had before was not kept"); + + // As long as it was, and silent only where the erase went. The + // samples looked at are clear of the 5 ms fades at the edges + QCOMPARE(framesIn(path), frame_t(4000)); + QVERIFY(std::fabs(sampleAt(path, 500) - 0.5f) < 1e-3f); + QCOMPARE(sampleAt(path, 1500), 0.f); + QVERIFY(std::fabs(sampleAt(path, 3000) - 0.5f) < 1e-3f); + } + + // Several ranges at once, out of order, and one of them trimming the + // end of the recording + void erase_several_ranges() { + SingingTakes takes; + QString recording = writeRecording(4000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + + Coverage::Ranges erased; + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(3000, 4000), + Coverage::Range(1000, 2000) }, + takeDirectory(), &erased), QString()); + + QCOMPARE(int(erased.size()), 2); + QCOMPARE(erased[0], Coverage::Range(1000, 2000)); + QCOMPARE(erased[1], Coverage::Range(3000, 4000)); + + QCOMPARE(int(takes.getCoverage().getRanges().size()), 2); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 1000)); + QCOMPARE(takes.getCoverage().getRanges()[1], Coverage::Range(2000, 3000)); + + QString path = takes.getAudioPath(); + QCOMPARE(framesIn(path), frame_t(4000)); + QCOMPARE(sampleAt(path, 1500), 0.f); + QVERIFY(std::fabs(sampleAt(path, 2500) - 0.5f) < 1e-3f); + QCOMPARE(sampleAt(path, 3500), 0.f); + } + + // What is asked for is clipped to the coverage: silence that was + // never recorded holds nothing to erase, and a selection running + // past the singing must not make the file any longer + void erase_is_clipped_to_the_coverage() { + SingingTakes takes; + QString recording = writeRecording(2000, 0.5f); + // coverage is [2000, 4000) + QVERIFY(takes.spliceRecording(recording, 0, 2000, -1, + takeDirectory()).isEmpty()); + + Coverage::Ranges erased; + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(0, 3000) }, + takeDirectory(), &erased), QString()); + + QCOMPARE(int(erased.size()), 1); + QCOMPARE(erased[0], Coverage::Range(2000, 3000)); + QCOMPARE(int(takes.getCoverage().getRanges().size()), 1); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(3000, 4000)); + QCOMPARE(framesIn(takes.getAudioPath()), frame_t(4000)); + } + + // A selection with no recorded singing in it: nothing is written and + // nothing changes, and it is not an error + void erase_where_nothing_was_recorded() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 1000, -1, + takeDirectory()).isEmpty()); + QString path = takes.getAudioPath(); + Coverage before = takes.getCoverage(); + + Coverage::Ranges erased { Coverage::Range(1, 2) }; + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(0, 1000) }, + takeDirectory(), &erased), QString()); + QVERIFY(erased.empty()); + QCOMPARE(takes.getAudioPath(), path); + QVERIFY(takes.getCoverage() == before); + QVERIFY(takes.getSupersededPaths().isEmpty()); + + // and no ranges at all + QCOMPARE(takes.eraseRanges(Coverage::Ranges {}, takeDirectory(), + &erased), QString()); + QVERIFY(erased.empty()); + QCOMPARE(takes.getAudioPath(), path); + } + + // Erasing all there is: the take stays, with an audio file that is + // silent throughout and nothing covered + void erase_the_whole_take() { + SingingTakes takes; + QString recording = writeRecording(2000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(0, 2000) }, + takeDirectory()), QString()); + + QVERIFY(takes.haveTake()); + QVERIFY(takes.getCoverage().isEmpty()); + QCOMPARE(framesIn(takes.getAudioPath()), frame_t(2000)); + QCOMPARE(sampleAt(takes.getAudioPath(), 1000), 0.f); + } + + void erase_failure_leaves_the_take_alone() { + SingingTakes takes; + takes.setWholeFileTake(m_dir.filePath("not-a-file.wav"), 1000); + Coverage before = takes.getCoverage(); + + Coverage::Ranges erased { Coverage::Range(1, 2) }; + QString error = takes.eraseRanges + (Coverage::Ranges { Coverage::Range(0, 500) }, takeDirectory(), + &erased); + QVERIFY(!error.isEmpty()); + QVERIFY(erased.empty()); + QCOMPARE(takes.getAudioPath(), m_dir.filePath("not-a-file.wav")); + QVERIFY(takes.getCoverage() == before); + QVERIFY(takes.getSupersededPaths().isEmpty()); + } + // The question before recording over something: asked inside the // covered ranges only, and not at all once the user has said so void overwrite_question() { diff --git a/main/test/TestTakeEvents.h b/main/test/TestTakeEvents.h new file mode 100644 index 00000000..97b988da --- /dev/null +++ b/main/test/TestTakeEvents.h @@ -0,0 +1,217 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TAKE_EVENTS_H +#define TEST_TAKE_EVENTS_H + +// Tier 2: what erasing part of a take does to the pitch events and the +// notes that were analysed there. Event lists in, changes out; no +// models and no window. + +#include "../TakeEvents.h" + +#include +#include + +class TestTakeEvents : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + typedef Coverage::Range Range; + typedef Coverage::Ranges Ranges; + + // A pitch event: a frequency at a frame, no duration + static sv::Event pitchAt(frame_t frame, float hz = 220.f) { + return sv::Event(frame, hz, QString()); + } + + static sv::Event noteAt(frame_t frame, frame_t duration, + float hz = 220.f) { + return sv::Event(frame, hz, duration, "sung"); + } + + static sv::EventVector pitchTrack(frame_t from, frame_t to, + frame_t step) { + sv::EventVector events; + for (frame_t f = from; f < to; f += step) events.push_back(pitchAt(f)); + return events; + } + +private slots: + // The pitch events in the erased ranges go, and nothing is added in + // their place: there is nothing there to hear any more + void pitch_in_the_erased_ranges_goes() { + sv::EventVector events = pitchTrack(0, 1000, 100); + TakeEvents::Change change = + TakeEvents::erasePitch(events, Ranges { Range(300, 600) }); + + QVERIFY(change.added.empty()); + QCOMPARE(int(change.removed.size()), 3); + QCOMPARE(change.removed[0].getFrame(), frame_t(300)); + QCOMPARE(change.removed[1].getFrame(), frame_t(400)); + QCOMPARE(change.removed[2].getFrame(), frame_t(500)); + } + + // A range is [start, end): an event at the end of it is outside + void pitch_at_the_edges_of_a_range() { + sv::EventVector events { pitchAt(299), pitchAt(300), pitchAt(599), + pitchAt(600) }; + TakeEvents::Change change = + TakeEvents::erasePitch(events, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 2); + QCOMPARE(change.removed[0].getFrame(), frame_t(300)); + QCOMPARE(change.removed[1].getFrame(), frame_t(599)); + } + + void pitch_in_several_ranges() { + sv::EventVector events = pitchTrack(0, 1000, 100); + // Out of order, and two of them overlapping + TakeEvents::Change change = TakeEvents::erasePitch + (events, Ranges { Range(800, 900), Range(100, 250), + Range(200, 300) }); + + QCOMPARE(int(change.removed.size()), 3); + QCOMPARE(change.removed[0].getFrame(), frame_t(100)); + QCOMPARE(change.removed[1].getFrame(), frame_t(200)); + QCOMPARE(change.removed[2].getFrame(), frame_t(800)); + } + + void nothing_erased_changes_nothing() { + sv::EventVector events = pitchTrack(0, 1000, 100); + QVERIFY(TakeEvents::erasePitch(events, Ranges {}).isEmpty()); + QVERIFY(TakeEvents::eraseNotes + (sv::EventVector { noteAt(0, 1000) }, Ranges {}).isEmpty()); + + // A range that does not reach the events + QVERIFY(TakeEvents::erasePitch + (events, Ranges { Range(2000, 3000) }).isEmpty()); + QVERIFY(TakeEvents::eraseNotes + (sv::EventVector { noteAt(0, 1000) }, + Ranges { Range(2000, 3000) }).isEmpty()); + } + + // A note the erased range takes in altogether goes + void note_wholly_inside_goes() { + sv::EventVector notes { noteAt(400, 100) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(change.removed[0].getFrame(), frame_t(400)); + QVERIFY(change.added.empty()); + + // and one that fills it exactly + change = TakeEvents::eraseNotes(sv::EventVector { noteAt(300, 300) }, + Ranges { Range(300, 600) }); + QCOMPARE(int(change.removed.size()), 1); + QVERIFY(change.added.empty()); + } + + // A note that runs into the range from before it keeps its onset and + // is cut back to where the erased audio begins + void note_cut_back_at_the_start_of_a_range() { + sv::EventVector notes { noteAt(100, 300, 330.f) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(int(change.added.size()), 1); + QCOMPARE(change.added[0].getFrame(), frame_t(100)); + QCOMPARE(change.added[0].getDuration(), frame_t(200)); + // and it is the same note otherwise + QCOMPARE(change.added[0].getValue(), 330.f); + QCOMPARE(change.added[0].getLabel(), QString("sung")); + } + + // A note whose own start was erased has lost the singing its onset + // was: it begins again where the audio does, and is shorter by what + // was taken from it + void note_moved_on_from_the_end_of_a_range() { + sv::EventVector notes { noteAt(400, 500) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(int(change.added.size()), 1); + QCOMPARE(change.added[0].getFrame(), frame_t(600)); + QCOMPARE(change.added[0].getDuration(), frame_t(300)); + } + + // A range erased from the middle of a note leaves two notes, as it + // leaves two stretches of audio + void note_split_by_a_range_in_the_middle() { + sv::EventVector notes { noteAt(100, 800) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(int(change.added.size()), 2); + QCOMPARE(change.added[0].getFrame(), frame_t(100)); + QCOMPARE(change.added[0].getDuration(), frame_t(200)); + QCOMPARE(change.added[1].getFrame(), frame_t(600)); + QCOMPARE(change.added[1].getDuration(), frame_t(300)); + } + + // Several ranges through one note, given out of order + void note_through_several_ranges() { + sv::EventVector notes { noteAt(0, 1000) }; + TakeEvents::Change change = TakeEvents::eraseNotes + (notes, Ranges { Range(600, 700), Range(250, 400), + Range(200, 300) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(int(change.added.size()), 3); + QCOMPARE(change.added[0].getFrame(), frame_t(0)); + QCOMPARE(change.added[0].getDuration(), frame_t(200)); + QCOMPARE(change.added[1].getFrame(), frame_t(400)); + QCOMPARE(change.added[1].getDuration(), frame_t(200)); + QCOMPARE(change.added[2].getFrame(), frame_t(700)); + QCOMPARE(change.added[2].getDuration(), frame_t(300)); + } + + // Only the notes the ranges reach are touched + void notes_outside_are_left_alone() { + sv::EventVector notes { noteAt(0, 100), noteAt(400, 100), + noteAt(900, 100) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(change.removed[0].getFrame(), frame_t(400)); + QVERIFY(change.added.empty()); + } + + // A note that ends where the range starts, and one that starts where + // it ends, are outside it + void notes_at_the_edges_of_a_range() { + sv::EventVector notes { noteAt(200, 100), noteAt(600, 100) }; + QVERIFY(TakeEvents::eraseNotes + (notes, Ranges { Range(300, 600) }).isEmpty()); + } + + // A note of no length: nothing pYIN makes, but the rule is the same + // as for a pitch event + void a_note_of_no_length() { + sv::EventVector notes { noteAt(400, 0), noteAt(700, 0) }; + TakeEvents::Change change = + TakeEvents::eraseNotes(notes, Ranges { Range(300, 600) }); + + QCOMPARE(int(change.removed.size()), 1); + QCOMPARE(change.removed[0].getFrame(), frame_t(400)); + QVERIFY(change.added.empty()); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index d153ec22..0adb6fdb 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -16,6 +16,7 @@ #include "TestLatencyShift.h" #include "TestCoverage.h" #include "TestTakeAudio.h" +#include "TestTakeEvents.h" #include "TestSingingTakes.h" #include "TestTakeTiming.h" @@ -72,6 +73,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTakeEvents t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestSingingTakes t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 5ed872f2..cc1b8ede 100644 --- a/meson.build +++ b/meson.build @@ -1098,6 +1098,7 @@ tony_core_files = [ 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', + 'main/TakeEvents.cpp', 'main/TakeTiming.cpp', ] @@ -1348,6 +1349,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', 'main/test/TestTakeAudio.h', + 'main/test/TestTakeEvents.h', 'main/test/TestSingingTakes.h', 'main/test/TestTakeTiming.h', ]) From aec5bffe147b2e0f54a85e8336c57d37e89647d2 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 15:49:59 +0300 Subject: [PATCH 044/275] feat: Erase Singing in Selection, Select Recording at Playhead The two Edit menu actions of spec 5.2. Erase takes the singing in the selection out of the take's audio file, out of its coverage and strip, and out of the pitch and notes that were analysed there; the new audio goes under the layers by the swap of phase 4a, and nothing is analysed again. A note whose onset the erase took begins again at the end of the erased range. Erase is refused while a recorded range is being analysed, since the swap would throw that analysis away, and comes back when it finishes. Select Recording at Playhead selects the coverage range the playhead is in, so that erasing a whole recording is two commands. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 201 ++++++++++++++++++++++++ main/MainWindow.h | 19 +++ main/test/TestRecordWorkflow.h | 274 +++++++++++++++++++++++++++++++++ 3 files changed, 494 insertions(+) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 2aedd955..d9453d72 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -20,6 +20,7 @@ #include "Analyser.h" #include "LatencyUtils.h" #include "PaneUtils.h" +#include "TakeEvents.h" #include "framework/Document.h" #include "framework/VersionTester.h" @@ -136,6 +137,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_referencePitchHiddenForTake(false), m_takes(nullptr), m_coverageStrip(nullptr), + m_eraseSingingAction(nullptr), + m_selectRecordingAction(nullptr), m_takePosition(0), m_takePreRoll(0), m_takeEnd(-1), @@ -812,6 +815,36 @@ MainWindow::setupEditMenu() connect(this, SIGNAL(canSnapNotes(bool)), action, SLOT(setEnabled(bool))); menu->addAction(action); m_rightButtonMenu->addAction(action); + + menu->addSeparator(); + m_rightButtonMenu->addSeparator(); + + m_keyReference->setCategory(tr("Singing Track")); + + // No shortcuts for these two: every key worth having in this menu is + // taken, and an erase is not something to reach by accident + m_selectRecordingAction = + new QAction(tr("Select Recording at Playhead"), this); + m_selectRecordingAction->setStatusTip + (tr("Select the range of recorded singing that the playback position is in")); + connect(m_selectRecordingAction, SIGNAL(triggered()), + this, SLOT(selectRecordingAtPlayhead())); + connect(this, SIGNAL(canSelectRecording(bool)), + m_selectRecordingAction, SLOT(setEnabled(bool))); + m_selectRecordingAction->setEnabled(false); + menu->addAction(m_selectRecordingAction); + m_rightButtonMenu->addAction(m_selectRecordingAction); + + m_eraseSingingAction = new QAction(tr("Erase Singing in Selection"), this); + m_eraseSingingAction->setStatusTip + (tr("Remove the recorded singing within the selected region, leaving silence")); + connect(m_eraseSingingAction, SIGNAL(triggered()), + this, SLOT(eraseSingingInSelection())); + connect(this, SIGNAL(canEraseSinging(bool)), + m_eraseSingingAction, SLOT(setEnabled(bool))); + m_eraseSingingAction->setEnabled(false); + menu->addAction(m_eraseSingingAction); + m_rightButtonMenu->addAction(m_eraseSingingAction); } void @@ -1836,6 +1869,20 @@ MainWindow::updateMenuStates() bool haveSinging = (m_analyser2 != nullptr) || (m_realtimePitchLayer != nullptr); emit canShowRealtimePitch(haveSinging); + // Editing the singing of a take: there has to be a take with + // something recorded in it, and no take being recorded just now. + // Erasing needs a selection to erase as well, and waits for the + // analysis of a recorded range: erasing swaps the take's audio, + // which would throw that analysis's result away (see + // eraseSingingInSelection()) + bool inTake = (m_recordTarget && m_recordTarget->isRecording()); + bool haveCoverage = m_takes && m_takes->haveTake() && + !m_takes->getCoverage().isEmpty(); + bool analysingRange = (m_analyser2 && m_analyser2->isAnalysingRange()); + emit canSelectRecording(haveCoverage && !inTake); + emit canEraseSinging(haveCoverage && !inTake && haveSelection && + !analysingRange); + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -2817,6 +2864,11 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal connect(m_analyser2, SIGNAL(layersChanged()), this, SLOT(updateMenuStates())); + // Erase Singing is switched off while a recorded range is being + // analysed, so the menus have to hear when one is done with + connect(m_analyser2, SIGNAL(initialAnalysisCompleted()), + this, SLOT(updateMenuStates())); + // deferAnalysis=true: only set up waveform/visualisation layers now; // pYIN will be run later by analyseNow() once recording is complete. QString error = m_analyser2->newFileLoaded( @@ -4097,6 +4149,9 @@ MainWindow::startTakeAnalysis(sv_frame_t start, sv_frame_t end) if (!m_analyser2->isAnalysingRange()) return false; m_takeAnalysisRange = Coverage::Range(start, end); + + // Erasing is not to be had while this runs + updateMenuStates(); return true; } @@ -4117,6 +4172,152 @@ MainWindow::analyseTakeCoverage() return startTakeAnalysis(ranges.front().start, ranges.back().end); } +void +MainWindow::eraseSingingInSelection() +{ + // The singing in the selected ranges goes out of the take: silence + // in place of it in a new audio file, the ranges out of the coverage, + // and the pitch and notes that were analysed there deleted. Nothing + // is analysed: there is nothing left there to analyse. + // + // The action is disabled unless all of this holds, but a shortcut or + // a script can still reach it + if (!m_viewManager || !m_takes->haveTake()) return; + if (m_recordTarget && m_recordTarget->isRecording()) return; + + // The analysis of a recorded range is merged into the very models + // the swap below hands over, and the swap would cancel it and lose + // its result. It lasts a fraction of the recording it follows, so + // waiting for it is better than rescuing it + if (m_analyser2 && m_analyser2->isAnalysingRange()) return; + + Coverage::Ranges selected; + for (const Selection &s : m_viewManager->getSelections()) { + selected.push_back(Coverage::Range(s.getStartFrame(), + s.getEndFrame())); + } + if (selected.empty()) return; + + QString directory = RecordDirectory::getRecordDirectory(); + QString error; + + if (directory == "") { + error = tr("Could not find a directory to write the singing track " + "into"); + } + + Coverage::Ranges erased; + if (error == "") { + error = m_takes->eraseRanges(selected, directory, &erased); + } + + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to erase the singing"), + tr("The singing in the selection could not be erased" + "

%1

").arg(error), + QMessageBox::Ok); + return; + } + + if (erased.empty()) { + // Nothing of the selection held recorded singing + emit activity(tr("No recorded singing in the selection to erase")); + return; + } + + QString path = m_takes->getAudioPath(); + + cerr << "MainWindow::eraseSingingInSelection: erased " + << erased.size() << " range(s) into " << path << endl; + + // The new audio goes under the take's pitch and notes layers, which + // are edited below rather than analysed again + bool haveLayers = m_analyser2 && + m_analyser2->getLayer(Analyser::PitchTrack) && + m_analyser2->getLayer(Analyser::Notes); + + QString showError = (haveLayers ? swapSingingAudio(path) + : loadTakeAudio(path)); + + // The coverage has changed whether or not the new audio could be + // shown, and the strip says what it is now. After the swap, which + // puts the pane's selected layer back as it found it, and the strip + // is never to be that + syncCoverageStrip(); + + eraseTakeEvents(erased); + + if (showError != "") { + // As after a splice that could not be shown: the take's audio + // and its coverage have changed, and the screen has not + QMessageBox::warning + (this, + tr("Failed to show the erased singing"), + tr("The singing was erased, but the result could not be " + "shown

%1

What is on screen is the singing track " + "as it was. The erased audio is in the take's audio file, " + "\"%2\".

").arg(showError).arg(path), + QMessageBox::Ok); + } + + updateLayerStatuses(); + updateMenuStates(); + emit activity(tr("Erased the singing in the selection")); +} + +void +MainWindow::eraseTakeEvents(const Coverage::Ranges &erased) +{ + if (!m_analyser2) return; + + Layer *pitchLayer = m_analyser2->getLayer(Analyser::PitchTrack); + Layer *notesLayer = m_analyser2->getLayer(Analyser::Notes); + + auto pitch = pitchLayer ? + ModelById::getAs(pitchLayer->getModel()) : + nullptr; + auto notes = notesLayer ? + ModelById::getAs(notesLayer->getModel()) : nullptr; + + // Straight on the models, the way an analysis result is merged: no + // command and no undo entry. Phase 6 makes the erase undoable as a + // whole, from the changes TakeEvents works out here + if (pitch) { + TakeEvents::Change change = + TakeEvents::erasePitch(pitch->getAllEvents(), erased); + for (const Event &e : change.removed) pitch->remove(e); + for (const Event &e : change.added) pitch->add(e); + } + + if (notes) { + TakeEvents::Change change = + TakeEvents::eraseNotes(notes->getAllEvents(), erased); + for (const Event &e : change.removed) notes->remove(e); + for (const Event &e : change.added) notes->add(e); + } +} + +void +MainWindow::selectRecordingAtPlayhead() +{ + // The selection becomes the coverage range the playhead is in, so + // that erasing a whole recording is two commands. In a gap there is + // nothing to select, and the selection is left as it was + if (!m_viewManager || !m_takes->haveTake()) return; + if (m_recordTarget && m_recordTarget->isRecording()) return; + + Coverage::Range range; + if (!m_takes->getCoverage().getRangeAt + (m_viewManager->getPlaybackFrame(), range)) { + emit activity(tr("No recorded singing at the playback position")); + return; + } + + m_viewManager->setSelection(Selection(range.start, range.end)); +} + bool MainWindow::confirmRecordingOverTake() { diff --git a/main/MainWindow.h b/main/MainWindow.h index c5b22b0b..c99bfcc7 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -63,6 +63,8 @@ class MainWindow : public sv::MainWindowBase void canPlayNotes(bool); void canLoadSingingTrack(bool); void canShowRealtimePitch(bool); + void canEraseSinging(bool); + void canSelectRecording(bool); public slots: virtual bool commitData(bool mayAskUser); // on session shutdown @@ -102,6 +104,12 @@ protected slots: virtual void switchPitchUp(); virtual void switchPitchDown(); + // Editing the singing of a take: what is in the selection is erased + // from its audio, and the coverage range at the playhead can be + // selected to erase a whole recording (spec 5.2) + virtual void eraseSingingInSelection(); + virtual void selectRecordingAtPlayhead(); + virtual void snapNotesToPitches(); virtual void splitNote(); virtual void mergeNotes(); @@ -293,6 +301,17 @@ protected slots: // is a take and none yet, take it away when the take goes void syncCoverageStrip(); + // Erase Singing in Selection, and selecting the recording the + // playhead is in + QAction *m_eraseSingingAction; + QAction *m_selectRecordingAction; + + // Take the pitch events and the notes in these ranges out of the + // take's layers, the erased audio having taken the singing they + // describe with it. Straight on the models, as an analysis result + // is: phase 6 makes the erase as a whole undoable + void eraseTakeEvents(const Coverage::Ranges &erased); + // Where on the reference's timeline the take being recorded, or the // one most recently recorded, starts: the playback position when // Record was pressed, or the start of the selection recorded into. diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index e63de879..2fb338d0 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -95,6 +95,13 @@ class TestMainWindow : public MainWindow return swapSingingAudio(path); } + // Editing the singing of a take, as the two Edit menu actions do + void doEraseSingingInSelection() { eraseSingingInSelection(); } + void doSelectRecordingAtPlayhead() { selectRecordingAtPlayhead(); } + QAction *eraseSingingAction() { return m_eraseSingingAction; } + QAction *selectRecordingAction() { return m_selectRecordingAction; } + void doUpdateMenuStates() { updateMenuStates(); } + // True between the start of the analysis of a recorded range and the // merge of its result into the take's pitch and notes bool analysingRange() { @@ -146,6 +153,9 @@ class TestMainWindow : public MainWindow m_viewManager->addSelection(sv::Selection(start, end)); } void clearSelections() { m_viewManager->clearSelections(); } + sv::MultiSelection::SelectionList selections() { + return m_viewManager->getSelections(); + } // The question about recording over singing that is there is answered // from here: the suite cannot answer a dialog @@ -2984,6 +2994,270 @@ private slots: QCOMPARE(stripLayersInPane0(), 0); } + // Erase Singing in Selection and Select Recording at Playhead + // (spec 5.2). An erase takes the singing out of the take's audio, + // out of its coverage and strip, and out of the pitch and the notes + // that were analysed there. Nothing is analysed again: the layers + // on screen are the very ones that were there before + + void erase_a_whole_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + Coverage::Range recorded = ranges[0]; + QString audioBefore = m_window->takes()->getAudioPath(); + sv::Layer *pitch = m_window->analyser2()->getLayer(Analyser::PitchTrack); + sv::Layer *notes = m_window->analyser2()->getLayer(Analyser::Notes); + QVERIFY(pitch && notes); + QVERIFY(!pitchEvents(pitch).empty()); + QVERIFY(!noteEvents(notes).empty()); + QVERIFY(takeAudioRms(recorded.start, recorded.end) > 0.01); + QVERIFY(takeAudio()); + sv::sv_frame_t frames = takeAudio()->getFrameCount(); + + // The range to erase is what the other new action selects + m_window->seekTo(recorded.start + recorded.length() / 2); + m_window->doSelectRecordingAtPlayhead(); + QCOMPARE(int(m_window->selections().size()), 1); + QCOMPARE(m_window->selections().begin()->getStartFrame(), + recorded.start); + QCOMPARE(m_window->selections().begin()->getEndFrame(), recorded.end); + + m_window->doEraseSingingInSelection(); + + // Nothing is covered any more and the strip has gone with it. + // The take stays, in a file of its own, as long as the one it + // replaces and silent where the singing was + QVERIFY(m_window->takes()->haveTake()); + QVERIFY(m_window->takes()->getCoverage().isEmpty()); + QVERIFY(m_window->takes()->getAudioPath() != audioBefore); + QCOMPARE(m_window->takes()->getSupersededPaths().last(), audioBefore); + QVERIFY(!m_window->coverageStrip()->isShown()); + QCOMPARE(stripLayersInPane0(), 0); + QVERIFY(takeAudioRms(recorded.start + 1000, recorded.end - 1000) + < 1e-6); + + // The same two layers, with nothing left in them where the + // singing was + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY(eventsBetween(pitchEvents(pitch), + recorded.start, recorded.end).empty()); + QVERIFY(eventsBetween(noteEvents(notes), + recorded.start, recorded.end).empty()); + + // and the erase started no analysis of the take. (A transformer + // may well be running: making a selection sets Tony's own + // re-analysis of the *reference* going, which has nothing to do + // with the take. Wait for it, and see that the take's pitch and + // notes are still empty when everything has settled) + QVERIFY(!m_window->analysingRange()); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY(eventsBetween(pitchEvents(pitch), + recorded.start, recorded.end).empty()); + QVERIFY(eventsBetween(noteEvents(notes), + recorded.start, recorded.end).empty()); + + // The audio under them is the new file, as long as the one it + // replaced (a model of a file just opened takes a moment to know + // how long it is) + QTRY_VERIFY(takeAudio() && takeAudio()->getFrameCount() == frames); + verifyPlaySourceClean(); + } + + // One end of a recording erased: what is left is the rest of it, and + // a note whose onset went with the audio begins where the audio does + void erase_trims_one_end() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(1000); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + Coverage::Range recorded = ranges[0]; + sv::sv_frame_t cut = recorded.start + sv::sv_frame_t(0.4 * rate); + QVERIFY(cut < recorded.end); + sv::Layer *pitch = m_window->analyser2()->getLayer(Analyser::PitchTrack); + sv::Layer *notes = m_window->analyser2()->getLayer(Analyser::Notes); + QVERIFY(!eventsBetween(pitchEvents(pitch), recorded.start, + cut).empty()); + + m_window->selectRange(recorded.start, cut); + m_window->doEraseSingingInSelection(); + + auto left = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(left.size()), 1); + QCOMPARE(left[0], Coverage::Range(cut, recorded.end)); + QCOMPARE(int(stripEvents().size()), 1); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // Silent where the erase went, and as it was after that + QVERIFY(takeAudioRms(recorded.start + 1000, cut - 1000) < 1e-6); + QVERIFY(takeAudioRms(cut + 1000, recorded.end - 1000) > 0.01); + + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY(eventsBetween(pitchEvents(pitch), recorded.start, cut).empty()); + QVERIFY(!eventsBetween(pitchEvents(pitch), cut, recorded.end).empty()); + + auto notesLeft = noteEvents(notes); + QVERIFY(!notesLeft.empty()); + for (const auto &e : notesLeft) { + QVERIFY2(e.getFrame() >= cut, + "a note was left starting inside the erased range"); + } + QVERIFY(!m_window->analysingRange()); + } + + // A range erased from the middle: the coverage, the strip and the + // note that ran through it are each in two parts afterwards + void erase_splits_a_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(1200); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + Coverage::Range recorded = ranges[0]; + sv::sv_frame_t from = recorded.start + sv::sv_frame_t(0.4 * rate); + sv::sv_frame_t to = recorded.start + sv::sv_frame_t(0.8 * rate); + QVERIFY(to < recorded.end); + sv::Layer *pitch = m_window->analyser2()->getLayer(Analyser::PitchTrack); + sv::Layer *notes = m_window->analyser2()->getLayer(Analyser::Notes); + + m_window->selectRange(from, to); + m_window->doEraseSingingInSelection(); + + auto left = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(left.size()), 2); + QCOMPARE(left[0], Coverage::Range(recorded.start, from)); + QCOMPARE(left[1], Coverage::Range(to, recorded.end)); + QCOMPARE(int(stripEvents().size()), 2); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + QVERIFY(takeAudioRms(recorded.start + 1000, from - 1000) > 0.01); + QVERIFY(takeAudioRms(from + 1000, to - 1000) < 1e-6); + QVERIFY(takeAudioRms(to + 1000, recorded.end - 1000) > 0.01); + + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY(!eventsBetween(pitchEvents(pitch), recorded.start, + from).empty()); + QVERIFY(eventsBetween(pitchEvents(pitch), from, to).empty()); + QVERIFY(!eventsBetween(pitchEvents(pitch), to, recorded.end).empty()); + + // Nothing of a note is left over the erased audio + auto notesLeft = noteEvents(notes); + QVERIFY(!notesLeft.empty()); + for (const auto &e : notesLeft) { + QVERIFY2(e.getFrame() + e.getDuration() <= from || + e.getFrame() >= to, + "a note still runs through the erased range"); + } + QVERIFY(!m_window->analysingRange()); + } + + // The coverage range the playhead is in becomes the selection; in a + // gap there is nothing to select, and what is selected is left alone + void select_recording_at_playhead() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + m_window->clearSelections(); + + m_window->seekTo(ranges[0].start + ranges[0].length() / 2); + m_window->doSelectRecordingAtPlayhead(); + QCOMPARE(int(m_window->selections().size()), 1); + QCOMPARE(m_window->selections().begin()->getStartFrame(), + ranges[0].start); + QCOMPARE(m_window->selections().begin()->getEndFrame(), ranges[0].end); + + // In the gap between the two recordings + m_window->seekTo((ranges[0].end + ranges[1].start) / 2); + m_window->doSelectRecordingAtPlayhead(); + QCOMPARE(int(m_window->selections().size()), 1); + QCOMPARE(m_window->selections().begin()->getStartFrame(), + ranges[0].start); + + m_window->seekTo(ranges[1].start + ranges[1].length() / 2); + m_window->doSelectRecordingAtPlayhead(); + QCOMPARE(int(m_window->selections().size()), 1); + QCOMPARE(m_window->selections().begin()->getStartFrame(), + ranges[1].start); + QCOMPARE(m_window->selections().begin()->getEndFrame(), ranges[1].end); + } + + // Neither action is to be had without a take with singing in it, nor + // while one is being recorded, and erasing needs a selection as well + void erase_actions_enabled_when_they_apply() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->selectRange(0, sv::sv_frame_t(0.5 * rate)); + QVERIFY(!m_window->eraseSingingAction()->isEnabled()); + QVERIFY(!m_window->selectRecordingAction()->isEnabled()); + + m_window->clearSelections(); + take(700); + if (QTest::currentTestFailed()) return; + + // A take, but nothing selected to erase from it + m_window->clearSelections(); + QVERIFY(m_window->selectRecordingAction()->isEnabled()); + QVERIFY(!m_window->eraseSingingAction()->isEnabled()); + + m_window->selectRange(0, sv::sv_frame_t(0.5 * rate)); + QVERIFY(m_window->selectRecordingAction()->isEnabled()); + QVERIFY(m_window->eraseSingingAction()->isEnabled()); + + // and neither of them while the next take is being recorded + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->eraseSingingAction()->isEnabled()); + QVERIFY(!m_window->selectRecordingAction()->isEnabled()); + QTest::qWait(300); + stopTake(); + } + // The alternate pitch track: the reference pitch track moved by // whole octaves, as a layer of its own From beccef071b1d3ff5927efe4ed00dd33f6bc3b69f Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 16:47:39 +0300 Subject: [PATCH 045/275] feat: undo and redo of a take's recordings and erases A "Record Singing" or "Erase Singing" command holds the take's audio path, its coverage and the pitch and note events before and after, and finds the analyser, layers and models again when it runs: nothing of it is a pointer into a take that has since been rebuilt. The command of a recording is added at the splice and amended when the ranged analysis is merged into it, so one Undo takes both back; an undo pressed while the analysis is still running abandons it, and the redo analyses the range again, that result never having existed. A take's audio file is no longer opened through openPath(): a command pushed while an undo or a redo is running clears the redo stack and deletes the very command that is running. Everything Tony puts in a pane for itself now uses Document::attachLayerToView(), so the take is the top of the undo stack after a take -- except for the "Import Recorded Audio" compound that MainWindowBase::record() still makes for a pane Tony deletes again, which is why the history is cleared before a take's command is added. On closing a session, the audio files this run wrote for a take are deleted if neither the take nor a saved session refers to them any more. A file the user loaded is never deleted, and neither is the file the take is in, saved session or not. Co-Authored-By: Claude Opus 5 --- main/AlternatePitchTrack.cpp | 6 +- main/Analyser.cpp | 43 +++- main/Analyser.h | 42 +++- main/MainWindow.cpp | 362 +++++++++++++++++++++++++++++---- main/MainWindow.h | 57 +++++- main/SingingTakes.cpp | 47 +++++ main/SingingTakes.h | 42 ++++ main/TakeCommands.cpp | 80 ++++++++ main/TakeCommands.h | 114 +++++++++++ main/test/TestRecordWorkflow.h | 343 +++++++++++++++++++++++++++++++ main/test/TestSingingTakes.h | 98 +++++++++ meson.build | 1 + 12 files changed, 1183 insertions(+), 52 deletions(-) create mode 100644 main/TakeCommands.cpp create mode 100644 main/TakeCommands.h diff --git a/main/AlternatePitchTrack.cpp b/main/AlternatePitchTrack.cpp index 372fe327..292b594c 100644 --- a/main/AlternatePitchTrack.cpp +++ b/main/AlternatePitchTrack.cpp @@ -117,7 +117,11 @@ AlternatePitchTrack::show(Document *document, Pane *pane) document->setModel(layer, modelId); takeLayer(document, pane, layer); - document->addLayerToView(pane, layer); + + // Not addLayerToView(): this layer is Tony's own furniture, and hide() + // takes it away with deleteLayer(force), which leaves an AddLayerCommand + // holding a deleted layer. Undo is for what the user did + document->attachLayerToView(pane, layer); return true; } diff --git a/main/Analyser.cpp b/main/Analyser.cpp index ae53b4a0..e7cbf079 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -460,7 +460,13 @@ Analyser::addVisualisations() // This magical scale factor happens to get us a similar display // to Tony v1.0 spectrogram->setGain(0.25f); - m_document->addLayerToView(m_pane, spectrogram); + // attachLayerToView() and not addLayerToView() throughout this class: + // an analyser's layers are Tony's own furniture, made and taken away + // by the analyser itself (removeAllLayers() and releaseLayers() use + // deleteLayer(force), which leaves an AddLayerCommand holding a + // deleted layer). The undo history is for what the user did -- and + // after a take its top entry must be the take + m_document->attachLayerToView(m_pane, spectrogram); spectrogram->setLayerDormant(m_pane, true); m_layers[Spectrogram] = spectrogram; @@ -520,7 +526,7 @@ Analyser::addWaveform() params->setPlayGain(1); } - m_document->addLayerToView(m_pane, waveform); + m_document->attachLayerToView(m_pane, waveform); m_layers[Audio] = waveform; return ""; @@ -659,7 +665,7 @@ Analyser::addAnalyses() if (f) m_layers[Notes] = f; if (t) m_layers[PitchTrack] = t; - m_document->addLayerToView(m_pane, layers[i]); + m_document->attachLayerToView(m_pane, layers[i]); } configureAnalysisLayers(); @@ -766,7 +772,7 @@ Analyser::addEmptyAnalyses() m_document->addNonDerivedModel(modelId); m_document->setModel(layer, modelId); - m_document->addLayerToView(m_pane, layer); + m_document->attachLayerToView(m_pane, layer); m_layers[w.component] = layer; } @@ -1067,6 +1073,10 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, // replaced again, so the first run's result is of no use to anyone discardRangedAnalysis(); + // ... and neither is what the merge before it changed + m_rangedPitchChange = TakeEvents::Change(); + m_rangedNotesChange = TakeEvents::Change(); + sv_samplerate_t rate = waveFileModel->getSampleRate(); if (clipStart < 0) clipStart = 0; @@ -1261,9 +1271,16 @@ Analyser::mergeRangedAnalysis() << wFrom << " to " << pitchTo << ", notes in " << wFrom << " to " << noteTo << endl; + // Every remove and add is noted as it is made: the two changes + // together are what an undo of the recording this analysis belongs to + // has to reverse (getRangedPitchChange()) + m_rangedPitchChange = TakeEvents::Change(); + m_rangedNotesChange = TakeEvents::Change(); + for (const Event &e : pitch->getEventsStartingWithin(wFrom, pitchTo - wFrom)) { pitch->remove(e); + m_rangedPitchChange.removed.push_back(e); } // pYIN in fixed-lag mode (the default, and what we run) stamps one // frame of every run twice: the last frame that process() emits is @@ -1276,6 +1293,7 @@ Analyser::mergeRangedAnalysis() if (e.getFrame() >= wFrom && e.getFrame() < pitchTo && e.getFrame() != lastAdded) { pitch->add(e); + m_rangedPitchChange.added.push_back(e); lastAdded = e.getFrame(); } } @@ -1323,6 +1341,7 @@ Analyser::mergeRangedAnalysis() sv_frame_t f = e.getFrame(); if (f >= wFrom && f < noteTo) { notes->remove(e); + m_rangedNotesChange.removed.push_back(e); } else if (f < wFrom && firstAdded >= 0 && f + e.getDuration() > firstAdded) { // A note that runs into the window from before it is left as @@ -1333,6 +1352,8 @@ Analyser::mergeRangedAnalysis() // stays one note notes->remove(e); notes->add(e.withDuration(firstAdded - f)); + m_rangedNotesChange.removed.push_back(e); + m_rangedNotesChange.added.push_back(e.withDuration(firstAdded - f)); } } for (Event e : adding) { @@ -1352,17 +1373,19 @@ Analyser::mergeRangedAnalysis() // over it is cut back if (nextOldOnset > e.getFrame() && e.getFrame() + e.getDuration() > nextOldOnset) { - notes->add(e.withDuration(nextOldOnset - e.getFrame())); - } else { - notes->add(e); + e = e.withDuration(nextOldOnset - e.getFrame()); } + notes->add(e); + m_rangedNotesChange.added.push_back(e); } - // The events are in the models, not in a command: an analysis result - // never went onto the undo stack. Phase 6 makes the recording that - // asked for this analysis undoable as a whole + // The events went straight into the models, as a transform's do, with + // no command of their own. What the merge changed is remembered + // instead, for the command of the recording that asked for it: that + // one command undoes the splice and the analysis of it together discardRangedAnalysis(); + emit rangedAnalysisMerged(); emit initialAnalysisCompleted(); } diff --git a/main/Analyser.h b/main/Analyser.h index d5519ed5..2f90469c 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -23,6 +23,8 @@ #include #include +#include "TakeEvents.h" + #include "framework/Document.h" #include "base/Selection.h" #include "base/Clipboard.h" @@ -192,10 +194,11 @@ class Analyser : public QObject, * when both are complete their events replace what was in the * middle of the run -- a quarter of a second each side of the range * asked for, or out to an end of the run that was clipped -- and - * initialAnalysisCompleted() is emitted. The rest of the run is - * context only: it is where pYIN knows least, so what is there - * already is left alone. The merge is not undoable: an analysis - * result never was. + * rangedAnalysisMerged() and initialAnalysisCompleted() are emitted. + * The rest of the run is context only: it is where pYIN knows least, + * so what is there already is left alone. The merge makes no command + * of its own; what it changed is kept, for the command of the + * recording that asked for it (getRangedPitchChange()). * * Returns "" if a run was started (or there was nothing to do), or * a user-readable error string. A second call while one is running @@ -226,6 +229,27 @@ class Analyser : public QObject, return !m_rangedLayers.empty(); } + /** + * Abandon a ranged analysis if one is running, merging nothing. For + * an undo of the recording that asked for it: the result must not + * land on a take that has been put back as it was. + */ + void cancelRangedAnalysis() { discardRangedAnalysis(); } + + /** + * What the last ranged merge took out of and put into the pitch + * track and the notes. Reversing these two changes undoes the + * merge, which is how the recording that asked for it is made + * undoable (the merge itself still puts nothing on the undo stack). + * Valid from rangedAnalysisMerged() until the next analyseRange(). + */ + const TakeEvents::Change &getRangedPitchChange() const { + return m_rangedPitchChange; + } + const TakeEvents::Change &getRangedNotesChange() const { + return m_rangedNotesChange; + } + /** * Return true if the analysed pitch candidates are currently * visible (they are hidden from the call to reAnalyseSelection @@ -328,6 +352,11 @@ class Analyser : public QObject, void layersChanged(); void initialAnalysisCompleted(); + // A ranged analysis has just been merged into the pitch and notes, + // and getRangedPitchChange() / getRangedNotesChange() say what it + // changed. Emitted before initialAnalysisCompleted() + void rangedAnalysisMerged(); + protected slots: void layerAboutToBeDeleted(sv::Layer *); void layerCompletionChanged(sv::ModelId); @@ -371,6 +400,11 @@ protected slots: // cannot stamp anything before its own first two hops anyway bool m_rangedClippedEnd; + // What the last merge did, for the undo command of the recording + // that asked for the analysis (see getRangedPitchChange()) + TakeEvents::Change m_rangedPitchChange; + TakeEvents::Change m_rangedNotesChange; + QString doAllAnalyses(bool withPitchTrack); QString addVisualisations(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index d9453d72..7dad5d8b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -28,6 +28,7 @@ #include "view/Pane.h" #include "view/PaneStack.h" #include "data/model/WaveFileModel.h" +#include "data/model/ReadOnlyWaveFileModel.h" #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" @@ -139,6 +140,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_coverageStrip(nullptr), m_eraseSingingAction(nullptr), m_selectRecordingAction(nullptr), + m_openTakeCommand(nullptr), m_takePosition(0), m_takePreRoll(0), m_takeEnd(-1), @@ -426,6 +428,14 @@ MainWindow::~MainWindow() // Nothing must poll a take while the window is coming down stopTakePolling(); + // The command history is a singleton and outlives the window, and a + // take's command holds the window it belongs to. closeSession() has + // usually cleared it already; this is for the paths that do not go + // through it. Before anything is torn down, while a command that has + // a layer to delete can still find the document + m_openTakeCommand = nullptr; + CommandHistory::getInstance()->clear(); + // Clean up secondary state that may not have been torn down if the // window was closed without going through closeSession() (e.g. on // application exit via the window close button). @@ -2228,9 +2238,21 @@ MainWindow::closeSession() m_singingAudioMutedForTake = false; m_analysedMainModelId = {}; - // The takes of the session go with it. Their audio files are left - // where they are; deleting the ones nothing refers to any more is - // phase 6 of the takes work. + // Nothing is left waiting for a merge, and the history that holds the + // commands is cleared at the end of this function + m_openTakeCommand = nullptr; + + // The takes of the session go with it, and so do the audio files this + // run wrote for them that nothing refers to any more (spec 5.4). Not + // the file the take is in, even in a session that was never saved: it + // is the only copy of the singing apart from the raw recordings. Not + // a file the user brought either -- only what Tony itself wrote + QStringList gone = m_takes->removeUnusedFiles(); + if (!gone.isEmpty()) { + cerr << "MainWindow::closeSession: deleted " << gone.size() + << " superseded take audio file(s)" << endl; + } + m_takes->clear(); m_takePosition = 0; m_takePreRoll = 0; @@ -2396,6 +2418,48 @@ MainWindow::openSingingAudioFile(QString path, sv::ModelId &modelId, return status; } +MainWindow::FileOpenStatus +MainWindow::openTakeAudioFile(QString path, sv::ModelId &modelId) +{ + // The audio file of a take, opened as a model of the document and + // nothing else: no pane, no layer, no entry in Recent Files and -- the + // point of it -- no command. + // + // openPath() cannot be used for these files. It makes a pane through + // an AddPaneCommand, and CommandHistory::addCommand() clears the redo + // stack and deletes what is on it: a command pushed while an undo or a + // redo is running destroys the very command that is running. Undo of + // a take swaps its audio, so this is not avoidable any other way. + modelId = {}; + + FileSource source(path); + if (!source.isAvailable()) return FileOpenFailed; + source.waitForData(); + + // The rate the rest of the session is at, as openAudio() works it out + sv_samplerate_t rate = 0; + if (Preferences::getInstance()->getFixedSampleRate() != 0) { + rate = Preferences::getInstance()->getFixedSampleRate(); + } else if (Preferences::getInstance()->getResampleOnLoad() && + getMainModel()) { + rate = getMainModel()->getSampleRate(); + } + + auto model = std::make_shared(source, rate); + if (!model->isOK()) return FileOpenFailed; + + ModelId id = ModelById::add(model); + m_document->addNonDerivedModel(id); + + // modelAdded() has just queued a call to set up a singing track on it, + // as it does for any audio model that is not the main one. The caller + // makes the analyser itself, so that call must find nothing to do + m_pendingSingingModelId = {}; + + modelId = id; + return FileOpenSucceeded; +} + QString MainWindow::swapSingingAudio(QString path) { @@ -2440,20 +2504,15 @@ MainWindow::swapSingingAudio(QString path) } Layer *selected = pane->getSelectedLayer(); - // 1. The new audio, opened beside the reference as Load Singing Track - // does. First, so that a file that cannot be read disturbs nothing + // 1. The new audio, as a model of the document and nothing more. + // First, so that a file that cannot be read disturbs nothing ModelId newAudio; - std::vector extraPanes; - FileOpenStatus status = openSingingAudioFile(path, newAudio, extraPanes); + FileOpenStatus status = openTakeAudioFile(path, newAudio); if (status != FileOpenSucceeded || newAudio.isNone()) { - for (Pane *extra : extraPanes) pruneExtraPane(extra, newAudio); return tr("The file \"%1\" could not be opened as audio").arg(path); } - // Not the analysis we want: it would be of the whole file - m_pendingSingingModelId = {}; - // 2. The old audio's waveform layer goes, and with it the old audio; // the pitch and notes layers stay in the pane with their events. The // analyser has nothing left to lose by being deleted @@ -2478,12 +2537,10 @@ MainWindow::swapSingingAudio(QString path) setupSingingTrackAnalyser(newAudio, true); m_rebuildingTakeAudio = wasRebuilding; - // 5. The orphan waveform in the extra pane can go, now that the - // analyser's own waveform layer holds the new audio. If the setup - // failed it is released with the orphan, and the layers are left - // showing a take whose audio has gone - for (Pane *extra : extraPanes) pruneExtraPane(extra, newAudio); - + // 5. With no analyser there is no waveform layer holding the new + // audio, and the layers are left showing a take whose audio has gone. + // The model stays registered with the document, which releases it when + // it goes; see loadTakeAudio() if (!m_analyser2) { return tr("The singing track could not be set up on \"%1\"").arg(path); } @@ -2616,7 +2673,7 @@ MainWindow::loadBackgroundMusic(QString path) ColourDatabase *cdb = ColourDatabase::getInstance(); m_backgroundMusicLayer->setBaseColour( cdb->getColourIndex(tr("Green"))); - m_document->addLayerToView(pane, m_backgroundMusicLayer); + m_document->attachLayerToView(pane, m_backgroundMusicLayer); // The waveform is only needed to register the model with // the play source — we don't want it rendered on screen. @@ -2730,6 +2787,9 @@ MainWindow::alternatePitchToggled() m_analyser->stackLayers(); if (m_analyser2) m_analyser2->stackLayers(); syncAlternatePitchTrack(); + // As above: the layer arrives without a command, so the change to + // the session has to be noted here + documentModified(); } updateLayerStatuses(); @@ -2869,6 +2929,11 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal connect(m_analyser2, SIGNAL(initialAnalysisCompleted()), this, SLOT(updateMenuStates())); + // The result of a ranged analysis completes the undo command of the + // recording that asked for it (spec 5.4) + connect(m_analyser2, &Analyser::rangedAnalysisMerged, + this, &MainWindow::takeAnalysisMerged); + // deferAnalysis=true: only set up waveform/visualisation layers now; // pYIN will be run later by analyseNow() once recording is complete. QString error = m_analyser2->newFileLoaded( @@ -3168,7 +3233,7 @@ MainWindow::setupRealtimePitchLayer() ColourDatabase *cdb = ColourDatabase::getInstance(); m_realtimePitchLayer->setBaseColour(cdb->getColourIndex(tr("Orange"))); - m_document->addLayerToView(pane, m_realtimePitchLayer); + m_document->attachLayerToView(pane, m_realtimePitchLayer); // Create and start the pitch tracker. Its thread reads new frames // from audioSourceId (the WritableWaveFileModel) and emits @@ -3269,7 +3334,7 @@ MainWindow::setupRecordingLayer() } m_document->setModel(m_recordingLayer, m_currentRecordingModelId); - m_document->addLayerToView(pane, m_recordingLayer); + m_document->attachLayerToView(pane, m_recordingLayer); m_recordingLayer->showLayer(pane, false); if (auto params = m_recordingLayer->getPlayParameters()) { params->setPlayAudible(false); @@ -3922,6 +3987,10 @@ MainWindow::finishSingingTake() QString error; Coverage::Range placed; + // What an undo of this recording has to put back + QString pathBefore = m_takes->getAudioPath(); + Coverage coverageBefore = m_takes->getCoverage(); + if (recordingPath == "") { error = tr("The recording is no longer there to be used"); } else { @@ -3957,8 +4026,33 @@ MainWindow::finishSingingTake() << placed.start << "," << placed.end << ") of " << m_takes->getAudioPath() << endl; + // The command of this recording, made before the audio is shown and + // added to the history after: a range short enough is analysed and + // merged before the rebuild returns, and the merge has to find the + // command it belongs to. Any command still waiting for an analysis is + // closed first -- the rebuild below folds that range into its own run, + // so the merge it was waiting for is never coming + closeOpenTakeCommand(false); + SingingTakeCommand *command = + new SingingTakeCommand(this, tr("Record Singing"), + pathBefore, coverageBefore, + m_takes->getAudioPath(), m_takes->getCoverage()); + m_openTakeCommand = command; + bool analysing = rebuildSingingTrackFromTake(placed); + if (analysing) { + // The events of this run are not in the command yet, and an undo + // pressed before they are abandons the run: a redo has to make it + // again, over the range the run really covers + command->setPendingAnalysis(m_takeAnalysisRange); + } else if (m_openTakeCommand == command) { + // Nothing was analysed, or it failed to start: nothing to wait for + m_openTakeCommand = nullptr; + } + + addTakeCommand(command); + // The coverage has changed whether or not the new audio could be // shown, and the strip says what it is now. After the rebuild, not // before: a swap puts the pane's selected layer back as it found it, @@ -4032,17 +4126,12 @@ MainWindow::loadTakeAudio(QString path) // a take that has its layers; the two are the same but for the // handing over ModelId audio; - std::vector extraPanes; - FileOpenStatus status = openSingingAudioFile(path, audio, extraPanes); + FileOpenStatus status = openTakeAudioFile(path, audio); if (status != FileOpenSucceeded || audio.isNone()) { - for (Pane *extra : extraPanes) pruneExtraPane(extra, audio); return tr("The file \"%1\" could not be opened as audio").arg(path); } - // Not the analysis we want: it would be of the whole file - m_pendingSingingModelId = {}; - // Layers of the take that no analyser owns are handed to the one // about to be made, as after a session restore adoptTakeLayers(audio); @@ -4054,11 +4143,11 @@ MainWindow::loadTakeAudio(QString path) setupSingingTrackAnalyser(audio, true); m_rebuildingTakeAudio = wasRebuilding; - // The orphan waveform in the extra pane can go now that the - // analyser's own waveform layer holds the audio - for (Pane *extra : extraPanes) pruneExtraPane(extra, audio); - if (!m_analyser2) { + // The audio model is left registered with the document, which + // releases it when it goes: Document::releaseModel() is the + // document's own business, and holding a model nothing shows + // costs only memory return tr("The singing track could not be set up on \"%1\"").arg(path); } @@ -4169,6 +4258,13 @@ MainWindow::analyseTakeCoverage() const Coverage::Ranges &ranges = m_takes->getCoverage().getRanges(); if (ranges.empty()) return false; + // Analyse Now is not undoable (spec 7), and its merge must not be + // taken for the one a recording's command is waiting for. The run + // that command was waiting for is about to be abandoned by this one, + // so the events it would have added never existed; the command keeps + // the range, and analyses it again if it is ever redone + closeOpenTakeCommand(false); + return startTakeAnalysis(ranges.front().start, ranges.back().end); } @@ -4206,6 +4302,10 @@ MainWindow::eraseSingingInSelection() "into"); } + // What an undo of this erase has to put back + QString pathBefore = m_takes->getAudioPath(); + Coverage coverageBefore = m_takes->getCoverage(); + Coverage::Ranges erased; if (error == "") { error = m_takes->eraseRanges(selected, directory, &erased); @@ -4247,7 +4347,18 @@ MainWindow::eraseSingingInSelection() // is never to be that syncCoverageStrip(); - eraseTakeEvents(erased); + TakeEvents::Change pitchChange, notesChange; + eraseTakeEvents(erased, &pitchChange, ¬esChange); + + // Undoable as a whole: the audio file, the coverage and the events + // that went with the singing. Nothing was analysed, so the command is + // complete the moment it is made + SingingTakeCommand *command = + new SingingTakeCommand(this, tr("Erase Singing"), + pathBefore, coverageBefore, + m_takes->getAudioPath(), m_takes->getCoverage()); + command->setEventChanges(pitchChange, notesChange); + addTakeCommand(command); if (showError != "") { // As after a splice that could not be shown: the take's audio @@ -4268,7 +4379,9 @@ MainWindow::eraseSingingInSelection() } void -MainWindow::eraseTakeEvents(const Coverage::Ranges &erased) +MainWindow::eraseTakeEvents(const Coverage::Ranges &erased, + TakeEvents::Change *pitchChange, + TakeEvents::Change *notesChange) { if (!m_analyser2) return; @@ -4282,13 +4395,14 @@ MainWindow::eraseTakeEvents(const Coverage::Ranges &erased) ModelById::getAs(notesLayer->getModel()) : nullptr; // Straight on the models, the way an analysis result is merged: no - // command and no undo entry. Phase 6 makes the erase undoable as a - // whole, from the changes TakeEvents works out here + // command of their own. What was changed goes back to the caller, + // which puts the erase as a whole on the undo stack if (pitch) { TakeEvents::Change change = TakeEvents::erasePitch(pitch->getAllEvents(), erased); for (const Event &e : change.removed) pitch->remove(e); for (const Event &e : change.added) pitch->add(e); + if (pitchChange) *pitchChange = change; } if (notes) { @@ -4296,7 +4410,185 @@ MainWindow::eraseTakeEvents(const Coverage::Ranges &erased) TakeEvents::eraseNotes(notes->getAllEvents(), erased); for (const Event &e : change.removed) notes->remove(e); for (const Event &e : change.added) notes->add(e); + if (notesChange) *notesChange = change; + } +} + +bool +MainWindow::saveSessionFile(QString path) +{ + bool saved = MainWindowBase::saveSessionFile(path); + + // The .ton that has just been written names the take's audio file. + // Recording into the take again writes another file and supersedes + // that one, but the saved session still needs it, so it is not ours to + // delete when the session closes + if (saved && m_takes) m_takes->protectPath(m_takes->getAudioPath()); + + return saved; +} + +void +MainWindow::addTakeCommand(SingingTakeCommand *command) +{ + if (!command) return; + + // Undo after a take must undo the take, not take away some layer of + // Tony's own. Everything this application adds to a pane is kept off + // the undo stack (Document::attachLayerToView()), but two places in + // the svapp fork still push commands that Tony cannot reach: + // MainWindowBase::record() in RecordCreateAdditionalModel mode + // ("Import Recorded Audio") and MainWindowBase::openAudio() in + // CreateAdditionalModel mode ("Import \"...\""), which every swap of + // the take's audio goes through. Each makes a pane and a layer that + // Tony then deletes, so their unexecute() would work on freed memory. + // Until the fork stops making them, the history is cleared here: + // there is then one entry, and Undo is the take. Nothing of an + // earlier operation is lost from disk -- every audio file a take has + // had stays there until the session closes. + // + // Any command still waiting for a merge goes with it, so nothing is + // left pointing at a command that has been deleted (the command being + // added is not on the stack yet, so the clear cannot reach it) + if (m_openTakeCommand != command) m_openTakeCommand = nullptr; + CommandHistory::getInstance()->clear(); + + // Already done: the take's audio has been written and is on screen. + // CommandHistory marks the document modified, which is right for both + // of these operations + CommandHistory::getInstance()->addCommand(command, false); +} + +void +MainWindow::closeOpenTakeCommand(bool cancelAnalysis) +{ + if (cancelAnalysis && m_analyser2 && m_analyser2->isAnalysingRange()) { + cerr << "MainWindow::closeOpenTakeCommand: abandoning the analysis of " + << "[" << m_takeAnalysisRange.start << "," + << m_takeAnalysisRange.end << ")" << endl; + m_analyser2->cancelRangedAnalysis(); + m_takeAnalysisRange = Coverage::Range(); + + // The live dots of the take were waiting for that analysis to + // replace them, and it is not coming + teardownRealtimePitchLayer(); + + // Erasing is to be had again now that nothing is running + updateMenuStates(); } + + m_openTakeCommand = nullptr; +} + +void +MainWindow::takeAnalysisMerged() +{ + if (!m_openTakeCommand || !m_analyser2) return; + + // The recording is on the undo stack already, with the splice in it + // and nothing of the analysis: the events the merge has just changed + // complete it, so that one Undo takes both back + m_openTakeCommand->setEventChanges(m_analyser2->getRangedPitchChange(), + m_analyser2->getRangedNotesChange()); + m_openTakeCommand = nullptr; +} + +void +MainWindow::applyTakeEventChanges(const TakeState &state) +{ + if (!m_analyser2) return; + + Layer *pitchLayer = m_analyser2->getLayer(Analyser::PitchTrack); + Layer *notesLayer = m_analyser2->getLayer(Analyser::Notes); + + auto pitch = pitchLayer ? + ModelById::getAs(pitchLayer->getModel()) : + nullptr; + auto notes = notesLayer ? + ModelById::getAs(notesLayer->getModel()) : nullptr; + + // The removals first, in both directions: where the audio did not + // change, an analysis can put back the very event it took out, and + // that event has to be there once at the end and not twice + if (pitch) { + for (const Event &e : state.pitchRemove) pitch->remove(e); + for (const Event &e : state.pitchAdd) pitch->add(e); + } + if (notes) { + for (const Event &e : state.notesRemove) notes->remove(e); + for (const Event &e : state.notesAdd) notes->add(e); + } +} + +bool +MainWindow::applyTakeState(SingingTakeCommand *command, const TakeState &state) +{ + // An undo or a redo of a recording or an erase. Nothing of the + // command is a layer or a model: what the take is shown with now is + // found here, now, whatever it has been replaced by since + if (!m_document) return false; + + // A recording can be undone while the analysis of it is still + // running. That result belongs to the state being left behind, so the + // run is abandoned rather than allowed to land on the take we are + // putting back; the command remembers the range, and analyses it again + // if it is redone + closeOpenTakeCommand(true); + + m_takes->restoreTake(state.path, state.coverage); + + QString error; + + if (state.path == "") { + // Undo of the first recording of a take: no audio to show, and no + // pitch or notes either, exactly as before that recording. The + // next Record makes them again (rebuildSingingTrackFromTake()) + teardownSingingTrackAnalyser(); + } else { + bool haveLayers = m_analyser2 && + m_analyser2->getLayer(Analyser::PitchTrack) && + m_analyser2->getLayer(Analyser::Notes); + error = (haveLayers ? swapSingingAudio(state.path) + : loadTakeAudio(state.path)); + if (error == "" && m_analyser2) { + error = m_analyser2->addEmptyAnalyses(); + } + } + + // After the swap, which puts the pane's selected layer back as it + // found it, and the strip is never to be that + syncCoverageStrip(); + + if (error == "") applyTakeEventChanges(state); + + if (error != "") { + QMessageBox::warning + (this, + tr("Failed to show the singing track"), + tr("The singing track could not be shown as it was" + "

%1

The take's audio is in the file \"%2\".

") + .arg(error).arg(state.path), + QMessageBox::Ok); + } + + // A range whose analysis never finished: its result is in no event + // list, so it is analysed again rather than restored, and the command + // is open once more until that merge lands + if (error == "" && state.analyse.length() > 0) { + m_openTakeCommand = command; + if (!startTakeAnalysis(state.analyse.start, state.analyse.end) && + m_openTakeCommand == command) { + m_openTakeCommand = nullptr; + } + } + + updateLayerStatuses(); + updateMenuStates(); + + emit activity(tr("The singing track is the take's file \"%1\" again") + .arg(state.path == "" ? tr("(none)") : state.path)); + + return error == ""; } void @@ -5740,7 +6032,7 @@ MainWindow::analyseNewMainModel() SVDEBUG << "MainWindow::analyseNewMainModel: Adding pane and selection strip (ruler)" << endl; pane = m_paneStack->addPane(); selectionStrip = m_paneStack->addPane(); - m_document->addLayerToView + m_document->attachLayerToView (selectionStrip, m_document->createMainModelLayer(LayerFactory::TimeRuler)); } else { diff --git a/main/MainWindow.h b/main/MainWindow.h index c99bfcc7..ee50e0c8 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -22,6 +22,7 @@ #include "AlternatePitchTrack.h" #include "CoverageStrip.h" #include "SingingTakes.h" +#include "TakeCommands.h" #include "TakeTiming.h" #include @@ -54,6 +55,20 @@ class MainWindow : public sv::MainWindowBase */ void loadSingingTrack(QString path); + /** + * Put the take into the state a SingingTakeCommand holds: its audio + * file, its coverage, the events of its pitch track and notes, and, if + * the state names a range whose analysis never finished, that + * analysis started again. Called by the command on undo and on redo; + * public only for that. False if the audio could not be shown, in + * which case the user has been told. + */ + bool applyTakeState(SingingTakeCommand *command, const TakeState &state); + + // A session that has been saved names the take's audio file as it was + // at the time, so that file must outlive every other reference to it + bool saveSessionFile(QString path) override; + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -309,8 +324,39 @@ protected slots: // Take the pitch events and the notes in these ranges out of the // take's layers, the erased audio having taken the singing they // describe with it. Straight on the models, as an analysis result - // is: phase 6 makes the erase as a whole undoable - void eraseTakeEvents(const Coverage::Ranges &erased); + // is; what was changed comes back in the two Changes, for the undo + // command of the erase as a whole + void eraseTakeEvents(const Coverage::Ranges &erased, + TakeEvents::Change *pitchChange = nullptr, + TakeEvents::Change *notesChange = nullptr); + + // Undo and redo of the singing of a take (spec 5.4). + // + // The command of the operation being made just now, from the splice + // or the erase until the analysis that follows a splice has been + // merged into the take's pitch and notes. That merge arrives seconds + // after the splice, and amends this command, so that one Undo takes + // the recording and its analysis back together. Null when no command + // is waiting for anything; never left pointing at a command the + // history may have deleted + SingingTakeCommand *m_openTakeCommand; + + // Put a finished take operation on the undo stack, its work already + // done (CommandHistory::addCommand(command, false)) + void addTakeCommand(SingingTakeCommand *command); + + // Stop waiting for a merge into the open command. With + // cancelAnalysis, a ranged analysis that is still running is + // abandoned first: its result belongs to a state that is being left + // behind, and must not land on the take afterwards + void closeOpenTakeCommand(bool cancelAnalysis); + + // The ranged analysis has been merged: the events it changed go into + // the command of the recording that asked for it + void takeAnalysisMerged(); + + // Apply one command's event changes to the take's pitch and notes + void applyTakeEventChanges(const TakeState &state); // Where on the reference's timeline the take being recorded, or the // one most recently recorded, starts: the playback position when @@ -500,6 +546,13 @@ protected slots: FileOpenStatus openSingingAudioFile(QString path, sv::ModelId &modelId, std::vector &extraPanes); + // Open a take's own audio file as a model of the document and nothing + // else: no pane, no layer and, above all, no undo command. Every + // change to a take swaps its audio, including an undo, and a command + // pushed while an undo is running destroys the command that is running + // (see the comment on the definition) + FileOpenStatus openTakeAudioFile(QString path, sv::ModelId &modelId); + // Remove an extra pane created by openAudio()/record() in // CreateAdditionalModel mode: delete the orphan layer(s) showing // ownedModelId, detach shared layers (the time ruler) without deleting diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 9dc18bca..30a80b2b 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -18,6 +18,7 @@ #include #include +#include #include #include @@ -39,6 +40,8 @@ SingingTakes::clear() m_audioPath = ""; m_coverage.clear(); m_superseded.clear(); + m_written.clear(); + m_protected.clear(); } void @@ -51,6 +54,48 @@ SingingTakes::setTake(QString path, const Coverage &coverage) m_coverage = coverage; } +void +SingingTakes::restoreTake(QString path, const Coverage &coverage) +{ + m_audioPath = path; + m_coverage = coverage; +} + +void +SingingTakes::protectPath(QString path) +{ + if (path != "" && !m_protected.contains(path)) m_protected.push_back(path); +} + +QStringList +SingingTakes::unusedWrittenFiles() const +{ + QStringList unused; + for (const QString &path : m_written) { + if (path == m_audioPath) continue; + if (m_protected.contains(path)) continue; + if (unused.contains(path)) continue; + unused.push_back(path); + } + return unused; +} + +QStringList +SingingTakes::removeUnusedFiles() +{ + QStringList gone; + for (const QString &path : unusedWrittenFiles()) { + if (QFile::remove(path)) { + gone.push_back(path); + m_written.removeAll(path); + } else if (!QFile::exists(path)) { + // Something else has taken it away; it is not ours any more + m_written.removeAll(path); + } + } + return gone; +} + void SingingTakes::setWholeFileTake(QString path, sv_frame_t frames) { @@ -81,6 +126,7 @@ SingingTakes::spliceRecording(QString recordingPath, if (m_audioPath != "") m_superseded.push_back(m_audioPath); m_audioPath = outPath; + m_written.push_back(outPath); m_coverage.add(range.start, range.end); if (placed) *placed = range; @@ -116,6 +162,7 @@ SingingTakes::eraseRanges(const Coverage::Ranges &ranges, QString directory, m_superseded.push_back(m_audioPath); m_audioPath = outPath; + m_written.push_back(outPath); for (const Coverage::Range &r : wanted.getRanges()) { m_coverage.remove(r.start, r.end); } diff --git a/main/SingingTakes.h b/main/SingingTakes.h index 0099de53..cf1d73a9 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -56,6 +56,16 @@ class SingingTakes : public QObject */ void setTake(QString path, const Coverage &coverage); + /** + * The take is this file with this coverage again, because an undo or + * a redo has said so. Unlike setTake(), nothing is added to the + * superseded list: both files were already known when the operation + * being undone was done, and undo and redo may swap between them any + * number of times. An empty path is the state before the first + * recording of a take: no take at all. + */ + void restoreTake(QString path, const Coverage &coverage); + /** * The take is the whole of this file: a singing track the user * loaded, or one restored from a session saved before coverage was @@ -109,6 +119,36 @@ class SingingTakes : public QObject */ const QStringList &getSupersededPaths() const { return m_superseded; } + /** + * The audio files this run wrote itself, oldest first. A file the + * user loaded as a singing track is not one of them, however + * thoroughly it has since been superseded, so this is the list Tony + * may delete from (see removeUnusedFiles()). + */ + const QStringList &getWrittenPaths() const { return m_written; } + + /** + * Keep this file whatever happens, because something outside this + * object refers to it: a session file that has been saved names the + * take's audio as it was at the time, and that file must still be + * there when the session is opened again. + */ + void protectPath(QString path); + + /** + * The files this run wrote that nothing refers to any more: every + * one but the take's own audio and the protected ones. Files the + * user brought are never in the list. + */ + QStringList unusedWrittenFiles() const; + + /** + * Delete the files unusedWrittenFiles() names and return those that + * really went. For the close of a session: until then a superseded + * file may be wanted again by undo. + */ + QStringList removeUnusedFiles(); + /// Recording from this frame on would record over material that is there bool coversPosition(sv::sv_frame_t position) const; @@ -130,6 +170,8 @@ class SingingTakes : public QObject QString m_audioPath; Coverage m_coverage; QStringList m_superseded; + QStringList m_written; + QStringList m_protected; }; #endif diff --git a/main/TakeCommands.cpp b/main/TakeCommands.cpp new file mode 100644 index 00000000..a73bc638 --- /dev/null +++ b/main/TakeCommands.cpp @@ -0,0 +1,80 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeCommands.h" + +#include "MainWindow.h" + +SingingTakeCommand::SingingTakeCommand(MainWindow *window, QString name, + QString pathBefore, + const Coverage &coverageBefore, + QString pathAfter, + const Coverage &coverageAfter) : + m_window(window), + m_name(name), + m_pathBefore(pathBefore), + m_pathAfter(pathAfter), + m_coverageBefore(coverageBefore), + m_coverageAfter(coverageAfter) +{ +} + +SingingTakeCommand::~SingingTakeCommand() +{ +} + +void +SingingTakeCommand::setEventChanges(const TakeEvents::Change &pitch, + const TakeEvents::Change ¬es) +{ + m_pitch = pitch; + m_notes = notes; + + // The analysis has landed, so its result is in the lists now and a + // redo has no reason to run it again + m_pendingAnalysis = Coverage::Range(); +} + +void +SingingTakeCommand::execute() +{ + TakeState state; + state.path = m_pathAfter; + state.coverage = m_coverageAfter; + state.pitchRemove = m_pitch.removed; + state.pitchAdd = m_pitch.added; + state.notesRemove = m_notes.removed; + state.notesAdd = m_notes.added; + state.analyse = m_pendingAnalysis; + + if (m_window) m_window->applyTakeState(this, state); +} + +void +SingingTakeCommand::unexecute() +{ + // The other way round: what the change added goes, and what it + // removed comes back. Nothing is analysed -- a range whose analysis + // is still running belongs to the state being left behind, and + // MainWindow abandons that run + TakeState state; + state.path = m_pathBefore; + state.coverage = m_coverageBefore; + state.pitchRemove = m_pitch.added; + state.pitchAdd = m_pitch.removed; + state.notesRemove = m_notes.added; + state.notesAdd = m_notes.removed; + + if (m_window) m_window->applyTakeState(this, state); +} diff --git a/main/TakeCommands.h b/main/TakeCommands.h new file mode 100644 index 00000000..407f5d25 --- /dev/null +++ b/main/TakeCommands.h @@ -0,0 +1,114 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_COMMANDS_H +#define TONY_TAKE_COMMANDS_H + +#include "Coverage.h" +#include "TakeEvents.h" + +#include "base/Command.h" + +#include + +class MainWindow; + +/** + * What the take is to be made into: its audio file, its coverage, and + * the events to take out of and put into its pitch track and its notes. + * An empty path is the state before the first recording of a take: no + * audio, and nothing to show it with. + * + * "analyse" is a range whose analysis has to be run again rather than + * restored from events, because its result never existed -- the run was + * still going when the command was undone. An empty range means there + * is nothing to analyse. + */ +struct TakeState +{ + QString path; + Coverage coverage; + sv::EventVector pitchRemove; + sv::EventVector pitchAdd; + sv::EventVector notesRemove; + sv::EventVector notesAdd; + Coverage::Range analyse; +}; + +/** + * One undoable change to the singing of a take: a recording spliced into + * it ("Record Singing", together with the analysis of the range that was + * recorded) or a part of it erased ("Erase Singing"). + * + * The command holds values only: the audio file before and after, the + * coverage before and after, and the events the change took out of and + * put into the pitch track and the notes. No layer and no model + * pointer -- by the time an undo runs, the take's audio model and its + * analyser have been made again several times over, and MainWindow finds + * the current ones. The audio files both stay on disk until the session + * closes, which is what makes the swap back possible. + * + * A recording is on the undo stack from the moment its audio is spliced, + * which is seconds before the analysis of the recorded range is merged + * into the take's pitch and notes. The command is "open" until then: + * MainWindow amends it with the events of the merge when it lands, and + * an undo pressed before that abandons the run and leaves the range in + * setPendingAnalysis(), for a redo to analyse again. + */ +class SingingTakeCommand : public sv::Command +{ +public: + SingingTakeCommand(MainWindow *window, QString name, + QString pathBefore, const Coverage &coverageBefore, + QString pathAfter, const Coverage &coverageAfter); + virtual ~SingingTakeCommand(); + + QString getName() const override { return m_name; } + + /// Do it again: the take as it was after the change + void execute() override; + + /// Undo: the take as it was before the change + void unexecute() override; + + /** + * The events the change took out of and put into the pitch track and + * the notes. For a recording this arrives with the merge of the + * ranged analysis, which is also the moment the range below stops + * needing to be analysed again. + */ + void setEventChanges(const TakeEvents::Change &pitch, + const TakeEvents::Change ¬es); + + /** + * This range was being analysed when the command was made, so its + * result is in no event list: a redo has to analyse it again. + */ + void setPendingAnalysis(const Coverage::Range &range) { + m_pendingAnalysis = range; + } + +private: + MainWindow *m_window; + QString m_name; + QString m_pathBefore; + QString m_pathAfter; + Coverage m_coverageBefore; + Coverage m_coverageAfter; + TakeEvents::Change m_pitch; + TakeEvents::Change m_notes; + Coverage::Range m_pendingAnalysis; +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 2fb338d0..798cb9f7 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -52,9 +52,11 @@ #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" +#include "base/Command.h" #include "base/PlayParameters.h" #include "base/RecordDirectory.h" #include "transform/ModelTransformerFactory.h" +#include "widgets/CommandHistory.h" #include "widgets/InteractiveFileFinder.h" #include @@ -543,6 +545,76 @@ class TestRecordWorkflow : public QObject m_window->analyser2()->getLayer(Analyser::Notes)); } + // Undo and redo, and what they say they did. CommandHistory has no + // accessor for the top of its stack, and the name of the command it + // unexecutes is the same thing; "" means there was nothing to undo + QString undoOnce() { + QString name; + auto *history = sv::CommandHistory::getInstance(); + auto conn = connect(history, &sv::CommandHistory::commandUnexecuted, + this, [&name](sv::Command *c) { + if (c) name = c->getName(); + }); + history->undo(); + disconnect(conn); + return name; + } + + QString redoOnce() { + QString name; + auto *history = sv::CommandHistory::getInstance(); + auto conn = connect(history, + qOverload + (&sv::CommandHistory::commandExecuted), + this, [&name](sv::Command *c) { + if (c) name = c->getName(); + }); + history->redo(); + disconnect(conn); + return name; + } + + // Everything about the singing of a take that an undo or a redo has + // to restore exactly + struct TakeSnapshot { + QString path; + Coverage::Ranges coverage; + sv::EventVector strip; + sv::EventVector pitch; + sv::EventVector notes; + sv::sv_frame_t frames = -1; + }; + + TakeSnapshot snapshotTake() { + TakeSnapshot s; + s.path = m_window->takes()->getAudioPath(); + s.coverage = m_window->takes()->getCoverage().getRanges(); + s.strip = stripEvents(); + Analyser *a2 = m_window->analyser2(); + s.pitch = pitchEvents(a2); + s.notes = a2 ? noteEvents(a2->getLayer(Analyser::Notes)) + : sv::EventVector(); + if (takeAudio()) s.frames = takeAudio()->getFrameCount(); + return s; + } + + // The same take again, down to every pitch event and note. A model + // of a file that has just been opened takes a moment to say how long + // it is, so the frame count is the one thing worth waiting for + void verifyTakeMatches(const TakeSnapshot &s) { + QCOMPARE(m_window->takes()->getAudioPath(), s.path); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), s.coverage); + QCOMPARE(stripEvents(), s.strip); + Analyser *a2 = m_window->analyser2(); + QCOMPARE(pitchEvents(a2), s.pitch); + QCOMPARE(a2 ? noteEvents(a2->getLayer(Analyser::Notes)) + : sv::EventVector(), s.notes); + if (s.frames >= 0) { + QTRY_VERIFY(takeAudio() && + takeAudio()->getFrameCount() == s.frames); + } + } + int alternateLayersInDocument() { int n = 0, octaves = 0; for (sv::Layer *layer : m_window->document()->getLayers()) { @@ -3258,6 +3330,277 @@ private slots: stopTake(); } + // Undo and redo of the singing of a take (spec 5.4) + + // A recording over material that is already there, undone and redone: + // the audio file, the coverage, the strip and every pitch event and + // note come back exactly as they were + void undo_redo_a_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + // The material the second recording is undone back to + take(1000); + if (QTest::currentTestFailed()) return; + TakeSnapshot before = snapshotTake(); + QVERIFY(!before.pitch.empty()); + QVERIFY(!before.notes.empty()); + QVERIFY(before.frames > 0); + + // A second recording, into the silence after the first + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + TakeSnapshot after = snapshotTake(); + QVERIFY(after.path != before.path); + QCOMPARE(int(after.coverage.size()), 2); + QVERIFY(after.frames > before.frames); + Coverage::Range recorded = after.coverage[1]; + QVERIFY(takeAudioRms(recorded.start + 1000, recorded.end - 1000) + > 0.01); + + QCOMPARE(undoOnce(), QString("Record Singing")); + verifyTakeMatches(before); + if (QTest::currentTestFailed()) return; + + // The file the take plays is the one from before, in which the + // second recording's range was never anything but silence + QVERIFY(takeAudioRms(recorded.start + 1000, + std::min(recorded.end, before.frames) - 1000) + < 1e-6); + QVERIFY(!m_window->analysingRange()); + verifyPlaySourceClean(); + + QCOMPARE(redoOnce(), QString("Record Singing")); + verifyTakeMatches(after); + if (QTest::currentTestFailed()) return; + QVERIFY(takeAudioRms(recorded.start + 1000, recorded.end - 1000) + > 0.01); + + // The redo restored the analysis from the command; it did not run + // pYIN again + QVERIFY(!m_window->analysingRange()); + verifyPlaySourceClean(); + } + + // An erase undone and redone, the same way + void undo_redo_an_erase() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(1200); + if (QTest::currentTestFailed()) return; + + TakeSnapshot before = snapshotTake(); + QCOMPARE(int(before.coverage.size()), 1); + QVERIFY(!before.pitch.empty()); + QVERIFY(!before.notes.empty()); + + Coverage::Range recorded = before.coverage[0]; + sv::sv_frame_t from = recorded.start + sv::sv_frame_t(0.4 * rate); + sv::sv_frame_t to = recorded.start + sv::sv_frame_t(0.8 * rate); + m_window->selectRange(from, to); + m_window->doEraseSingingInSelection(); + + TakeSnapshot after = snapshotTake(); + QVERIFY(after.path != before.path); + QCOMPARE(int(after.coverage.size()), 2); + QVERIFY(eventsBetween(after.pitch, from, to).empty()); + QVERIFY(takeAudioRms(from + 1000, to - 1000) < 1e-6); + + QCOMPARE(undoOnce(), QString("Erase Singing")); + verifyTakeMatches(before); + if (QTest::currentTestFailed()) return; + QVERIFY(takeAudioRms(from + 1000, to - 1000) > 0.01); + + QCOMPARE(redoOnce(), QString("Erase Singing")); + verifyTakeMatches(after); + if (QTest::currentTestFailed()) return; + QVERIFY(takeAudioRms(from + 1000, to - 1000) < 1e-6); + + // The selection is still there and the reference's own + // re-analysis of it may still be running: let it finish before + // the window goes + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + verifyPlaySourceClean(); + } + + // Undo pressed while the analysis of the recorded range is still + // running: the run is abandoned, so nothing of it lands on the take + // that has been put back. The redo has to run it again -- that + // result never existed to be restored + void undo_during_analysis_then_redo() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(1000); + if (QTest::currentTestFailed()) return; + TakeSnapshot before = snapshotTake(); + QVERIFY(!before.pitch.empty()); + + // Stop splices the recording in and starts the analysis of it + // there and then + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(700); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY2(m_window->analysingRange(), + "the race was not set up: no analysis was running after Stop"); + sv::sv_frame_t analysedStart = m_window->analysedRangeStart(); + sv::sv_frame_t analysedEnd = m_window->analysedRangeEnd(); + QVERIFY(analysedEnd > analysedStart); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QVERIFY2(!m_window->analysingRange(), + "the analysis was left running over the undone take"); + verifyTakeMatches(before); + if (QTest::currentTestFailed()) return; + + // The live dots were waiting for that analysis; nothing is + QVERIFY(!m_window->realtimeLayer()); + QVERIFY(m_window->eraseSingingAction()->isEnabled() || + m_window->selections().empty()); + + // Redo: the range is analysed again, and this time the result + // reaches the take's pitch track + QCOMPARE(redoOnce(), QString("Record Singing")); + QCOMPARE(m_window->analysedRangeStart(), analysedStart); + QCOMPARE(m_window->analysedRangeEnd(), analysedEnd); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + QVERIFY2(!eventsBetween(pitchEvents(m_window->analyser2()), + ranges[1].start, ranges[1].end).empty(), + "the redone recording was never analysed"); + verifyPlaySourceClean(); + } + + // The first recording of a take undone: there is no take and no + // singing track at all, as before it, and Record still works + void undo_a_first_take_then_record_again() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + + int panes = m_window->paneStack()->getPaneCount(); + take(700); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->takes()->haveTake()); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QVERIFY(!m_window->takes()->haveTake()); + QVERIFY(m_window->takes()->getCoverage().isEmpty()); + QVERIFY2(!m_window->analyser2(), + "the singing track was left behind with no audio"); + QVERIFY(!m_window->coverageStrip()->isShown()); + QCOMPARE(stripLayersInPane0(), 0); + QCOMPARE(noteLayersInPane0(), 1); // the reference's + QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + verifyPlaySourceClean(); + + // and the reference is untouched + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + + // Recording again makes a take from nothing, as the first + // recording did + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->takes()->haveTake()); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, sv::sv_frame_t(1.0 * rate)); + QVERIFY(m_window->analyser2()); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + QVERIFY(!noteEvents(m_window->analyser2() + ->getLayer(Analyser::Notes)).empty()); + verifyStripMatchesTake(); + verifyPlaySourceClean(); + } + + // What one take leaves on the undo stack: the take, and nothing of + // the layers and panes Tony makes for itself along the way + void undo_stack_top_after_a_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + // Nothing of loading and analysing the reference is undoable + // either: those layers are Tony's, not the user's + QCOMPARE(undoOnce(), QString()); + + take(700); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->isDocumentModified()); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QVERIFY(!m_window->takes()->haveTake()); + QCOMPARE(undoOnce(), QString()); + + QCOMPARE(redoOnce(), QString("Record Singing")); + QVERIFY(m_window->takes()->haveTake()); + QCOMPARE(redoOnce(), QString()); + } + + // The audio files a take has been through are kept until the session + // closes, and then only the ones nothing refers to any more go + void take_files_deleted_on_close() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + QStringList written = m_window->takes()->getWrittenPaths(); + QCOMPARE(int(written.size()), 2); + QCOMPARE(written.last(), m_window->takes()->getAudioPath()); + for (const QString &path : written) { + QVERIFY2(QFileInfo::exists(path), qPrintable(path)); + } + + // Only the superseded one is unused: the take's own file is kept + // even though this session was never saved + QCOMPARE(m_window->takes()->unusedWrittenFiles(), + QStringList { written.first() }); + + m_window->doCloseSession(); + + QVERIFY2(!QFileInfo::exists(written.first()), + "a superseded take audio file was left behind"); + QVERIFY2(QFileInfo::exists(written.last()), + "the take's own audio file was deleted"); + + // The commands that held those paths went with the session + QCOMPARE(undoOnce(), QString()); + } + // The alternate pitch track: the reference pitch track moved by // whole octaves, as a layer of its own diff --git a/main/test/TestSingingTakes.h b/main/test/TestSingingTakes.h index 3a6957c3..fe3031f9 100644 --- a/main/test/TestSingingTakes.h +++ b/main/test/TestSingingTakes.h @@ -387,6 +387,104 @@ private slots: QVERIFY(takes.getSupersededPaths().isEmpty()); } + // Undo and redo swap the take between two files it has already had: + // neither is superseded by the other again, however often they change + // places, and neither becomes a file this run wrote + void restore_take_supersedes_nothing() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QString first = takes.getAudioPath(); + QVERIFY(takes.spliceRecording(recording, 0, 5000, -1, + takeDirectory()).isEmpty()); + QString second = takes.getAudioPath(); + Coverage after = takes.getCoverage(); + QCOMPARE(takes.getSupersededPaths(), QStringList { first }); + + Coverage before; + before.add(0, 1000); + + takes.restoreTake(first, before); + QCOMPARE(takes.getAudioPath(), first); + QVERIFY(takes.getCoverage() == before); + QCOMPARE(takes.getSupersededPaths(), QStringList { first }); + QCOMPARE(takes.getWrittenPaths(), QStringList({ first, second })); + + takes.restoreTake(second, after); + QCOMPARE(takes.getAudioPath(), second); + QVERIFY(takes.getCoverage() == after); + QCOMPARE(takes.getSupersededPaths(), QStringList { first }); + QCOMPARE(takes.getWrittenPaths(), QStringList({ first, second })); + + // Undo of the very first recording of a take: no take at all + takes.restoreTake("", Coverage()); + QVERIFY(!takes.haveTake()); + QVERIFY(takes.getCoverage().isEmpty()); + } + + // What a session close may delete: the files this run wrote that + // nothing refers to any more. Never the file the take is in, and + // never a file the user brought, however thoroughly superseded + void only_unused_files_we_wrote_are_deleted() { + SingingTakes takes; + + // The user's own singing track, recorded over twice + QString loaded = writeRecording(1000, 0.5f); + takes.setWholeFileTake(loaded, 1000); + + QString recording = writeRecording(1000, 0.25f); + QVERIFY(takes.spliceRecording(recording, 0, 2000, -1, + takeDirectory()).isEmpty()); + QString first = takes.getAudioPath(); + QVERIFY(takes.spliceRecording(recording, 0, 5000, -1, + takeDirectory()).isEmpty()); + QString second = takes.getAudioPath(); + + QCOMPARE(takes.getWrittenPaths(), QStringList({ first, second })); + QVERIFY(takes.getSupersededPaths().contains(loaded)); + + QCOMPARE(takes.unusedWrittenFiles(), QStringList { first }); + QCOMPARE(takes.removeUnusedFiles(), QStringList { first }); + + QVERIFY2(!QFileInfo::exists(first), "a superseded file was kept"); + QVERIFY2(QFileInfo::exists(second), "the take's own file was deleted"); + QVERIFY2(QFileInfo::exists(loaded), "the user's own file was deleted"); + + // and nothing is deleted twice + QVERIFY(takes.unusedWrittenFiles().isEmpty()); + QVERIFY(takes.removeUnusedFiles().isEmpty()); + } + + // The file a saved session names must be there when that session is + // opened again, however many recordings have superseded it since + void a_saved_session_keeps_its_file() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QString saved = takes.getAudioPath(); + takes.protectPath(saved); + + QVERIFY(takes.spliceRecording(recording, 0, 5000, -1, + takeDirectory()).isEmpty()); + QString second = takes.getAudioPath(); + QVERIFY(takes.unusedWrittenFiles().isEmpty()); + + QVERIFY(takes.spliceRecording(recording, 0, 9000, -1, + takeDirectory()).isEmpty()); + QCOMPARE(takes.unusedWrittenFiles(), QStringList { second }); + + QCOMPARE(takes.removeUnusedFiles(), QStringList { second }); + QVERIFY(QFileInfo::exists(saved)); + QVERIFY(QFileInfo::exists(takes.getAudioPath())); + + // A new session starts with nothing to remember or protect + takes.clear(); + QVERIFY(takes.getWrittenPaths().isEmpty()); + QVERIFY(takes.unusedWrittenFiles().isEmpty()); + } + // The question before recording over something: asked inside the // covered ranges only, and not at all once the user has said so void overwrite_question() { diff --git a/meson.build b/meson.build index cc1b8ede..2008f31d 100644 --- a/meson.build +++ b/meson.build @@ -1109,6 +1109,7 @@ tony_app_files = [ 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', 'main/PaneUtils.cpp', + 'main/TakeCommands.cpp', ] tony_core_moc_files = qt.preprocess( From 60acc484f469ff479010ecb358aa91514b9665b9 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 16:58:38 +0300 Subject: [PATCH 046/275] fix: a take no longer clears the undo history The history was cleared for every take because svapp's record() left an "Import Recorded Audio" entry for a pane that Tony deletes. The recording is now made in svapp's new RecordCreateUnshownModel mode, which makes no pane and no entry, so takes and the edits before them undo in order. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 34 +++++++++++++---------------- main/test/TestRecordWorkflow.h | 39 ++++++++++++++++++++++++++++++++++ repoint-lock.json | 2 +- 3 files changed, 55 insertions(+), 20 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 7dad5d8b..9cef4a40 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3491,11 +3491,14 @@ MainWindow::record() m_recordingInProgress = false; m_recordingAsSingingTrack = true; - // Remember the pane count so we can prune the extra pane that - // MainWindowBase::record() creates via AddPaneCommand for the - // recording's waveform layer. We want everything in pane 0. + // The recording is only to be added to the document as a model: + // the pane, the layer and the "Import Recorded Audio" undo entry + // that RecordCreateAdditionalModel makes for it are all things we + // would delete at once, leaving a command on the undo stack whose + // pane has gone. The pane count is still remembered, so that the + // pruning code finds nothing to do rather than being taken out m_paneCountBeforeRecording = m_paneStack ? m_paneStack->getPaneCount() : 0; - setAudioRecordMode(RecordCreateAdditionalModel); + setAudioRecordMode(RecordCreateUnshownModel); } else { m_recordingAsSingingTrack = false; m_takePosition = 0; @@ -4435,23 +4438,16 @@ MainWindow::addTakeCommand(SingingTakeCommand *command) // Undo after a take must undo the take, not take away some layer of // Tony's own. Everything this application adds to a pane is kept off - // the undo stack (Document::attachLayerToView()), but two places in - // the svapp fork still push commands that Tony cannot reach: - // MainWindowBase::record() in RecordCreateAdditionalModel mode - // ("Import Recorded Audio") and MainWindowBase::openAudio() in - // CreateAdditionalModel mode ("Import \"...\""), which every swap of - // the take's audio goes through. Each makes a pane and a layer that - // Tony then deletes, so their unexecute() would work on freed memory. - // Until the fork stops making them, the history is cleared here: - // there is then one entry, and Undo is the take. Nothing of an - // earlier operation is lost from disk -- every audio file a take has - // had stays there until the session closes. + // the undo stack: its layers go in by Document::attachLayerToView(), + // a take's audio is opened by openTakeAudioFile(), and the recording + // itself is made in RecordCreateUnshownModel mode, which makes no + // pane and no "Import Recorded Audio" entry. So the history is left + // as it is, and the takes and the edits made before this one can + // still be undone after it. // - // Any command still waiting for a merge goes with it, so nothing is - // left pointing at a command that has been deleted (the command being - // added is not on the stack yet, so the clear cannot reach it) + // A command still waiting for a merge is not this one's business any + // more: the callers close it before they get here if (m_openTakeCommand != command) m_openTakeCommand = nullptr; - CommandHistory::getInstance()->clear(); // Already done: the take's audio has been written and is on screen. // CommandHistory marks the document modified, which is right for both diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 798cb9f7..fbfc9122 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -3563,6 +3563,45 @@ private slots: QCOMPARE(redoOnce(), QString()); } + // A take does not cost the takes before it their undo: each is an + // entry of its own, undone and redone in order + void undo_two_takes_in_order() { + FakeAudioIO::Config config; + config.input = tone(highHz, 5.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + if (QTest::currentTestFailed()) return; + QString firstPath = m_window->takes()->getAudioPath(); + Coverage firstCoverage = m_window->takes()->getCoverage(); + + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + if (QTest::currentTestFailed()) return; + QString secondPath = m_window->takes()->getAudioPath(); + QCOMPARE(int(m_window->takes()->getCoverage().getRanges().size()), 2); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QCOMPARE(m_window->takes()->getAudioPath(), firstPath); + QVERIFY(m_window->takes()->getCoverage() == firstCoverage); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QVERIFY(!m_window->takes()->haveTake()); + QCOMPARE(undoOnce(), QString()); + + QCOMPARE(redoOnce(), QString("Record Singing")); + QCOMPARE(m_window->takes()->getAudioPath(), firstPath); + QCOMPARE(redoOnce(), QString("Record Singing")); + QCOMPARE(m_window->takes()->getAudioPath(), secondPath); + QCOMPARE(int(m_window->takes()->getCoverage().getRanges().size()), 2); + } + // The audio files a take has been through are kept until the session // closes, and then only the ones nothing refers to any more go void take_files_deleted_on_close() { diff --git a/repoint-lock.json b/repoint-lock.json index f77c2839..9379cf98 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -10,7 +10,7 @@ "pin": "008441c137fd233d5d9ca4ea296acb7c8dde8dbb" }, "svapp": { - "pin": "2e54ae75f50a737b119defa47f8b3c1c4faf0244" + "pin": "8dfa3bc8d5ede210210ebba003c736c46e86405a" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" From d51ffa5ac3694a46b98a601ec94048fd3b2decf8 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 17:46:24 +0300 Subject: [PATCH 047/275] feat: several takes in memory, with per-take named layers SingingTakes holds a list of takes (name, audio path, coverage) with one of them active; every single-take call is about that one, so the callers from phases 2 to 6 are unchanged. Each take has its three layers in pane 0 named after it (TakeLayers: "Take 2 Pitch", "Take 2 Notes", "Take 2 Coverage"), and only the active take has an audio model and the singing analyser. Switching releases the analyser keeping the layers, puts the take's layers away -- hidden, silent, out of the play source and with no source model, so no analyser can claim them and they cannot hold playback open -- and opens the other take's audio under its layers, with no analysis. A Takes menu (New Empty, Duplicate, Rename, Delete) and a "Take:" combo box in the playback toolbar; every take operation but Rename clears the undo history, as does Load Singing Track, which now makes a take of its own. Co-Authored-By: Claude Opus 5 --- main/CoverageStrip.cpp | 54 ++- main/CoverageStrip.h | 43 +- main/MainWindow.cpp | 741 +++++++++++++++++++++++++++++++-- main/MainWindow.h | 92 ++++ main/SingingTakes.cpp | 221 +++++++++- main/SingingTakes.h | 120 +++++- main/TakeLayers.cpp | 108 +++++ main/TakeLayers.h | 78 ++++ main/test/TestRecordWorkflow.h | 632 +++++++++++++++++++++++++++- main/test/TestSingingTakes.h | 192 +++++++++ meson.build | 1 + 11 files changed, 2190 insertions(+), 92 deletions(-) create mode 100644 main/TakeLayers.cpp create mode 100644 main/TakeLayers.h diff --git a/main/CoverageStrip.cpp b/main/CoverageStrip.cpp index 2b56590c..11049257 100644 --- a/main/CoverageStrip.cpp +++ b/main/CoverageStrip.cpp @@ -14,6 +14,8 @@ #include "CoverageStrip.h" +#include "TakeLayers.h" + #include "framework/Document.h" #include "view/Pane.h" #include "layer/RegionLayer.h" @@ -51,18 +53,18 @@ CoverageStrip::~CoverageStrip() } QString -CoverageStrip::layerName() +CoverageStrip::layerName(QString takeName) { // Not translated: it is stored in the session file and adopt() looks - // it up. Phase 7a of the takes work numbers it after its take - return "Take 1 Coverage"; + // it up + return TakeLayers::nameFor(takeName, TakeLayers::Coverage); } bool -CoverageStrip::show(Document *document, Pane *pane) +CoverageStrip::show(Document *document, Pane *pane, QString takeName) { if (m_layer) return true; - if (!document || !pane) return false; + if (!document || !pane || takeName == "") return false; sv_samplerate_t rate = defaultSampleRate; if (auto main = ModelById::get(document->getMainModel())) { @@ -84,7 +86,7 @@ CoverageStrip::show(Document *document, Pane *pane) } document->setModel(layer, modelId); - takeLayer(document, pane, layer); + takeLayer(document, pane, layer, takeName); // Not addLayerToView(): the strip is part of the take, not something // the user added, so undo must not take it away and making it does @@ -95,16 +97,16 @@ CoverageStrip::show(Document *document, Pane *pane) } bool -CoverageStrip::adopt(Document *document, Pane *pane) +CoverageStrip::adopt(Document *document, Pane *pane, QString takeName) { if (m_layer) return true; - if (!document || !pane) return false; + if (!document || !pane || takeName == "") return false; for (int i = 0; i < pane->getLayerCount(); ++i) { auto layer = qobject_cast(pane->getLayer(i)); - if (!layer || layer->objectName() != layerName()) continue; + if (!layer || layer->objectName() != layerName(takeName)) continue; if (!ModelById::isa(layer->getModel())) continue; - takeLayer(document, pane, layer); + takeLayer(document, pane, layer, takeName); return true; } @@ -112,11 +114,13 @@ CoverageStrip::adopt(Document *document, Pane *pane) } void -CoverageStrip::takeLayer(Document *document, Pane *pane, RegionLayer *layer) +CoverageStrip::takeLayer(Document *document, Pane *pane, RegionLayer *layer, + QString takeName) { m_document = document; m_pane = pane; m_layer = layer; + m_takeName = takeName; connect(m_document, &Document::layerAboutToBeDeleted, this, &CoverageStrip::layerAboutToBeDeleted, @@ -130,8 +134,12 @@ CoverageStrip::configureLayer() { if (!m_layer) return; - m_layer->setObjectName(layerName()); - m_layer->setPresentationName(tr("Singing Coverage")); + m_layer->setObjectName(layerName(m_takeName)); + m_layer->setPresentationName(tr("%1 Coverage").arg(m_takeName)); + + // The strip of a take that is not the active one is hidden, and this + // one is the active take's + m_layer->setLayerDormant(m_pane, false); // A band along the bottom of the pane, filled where there is // singing: a plot style the svgui fork has for this. It has no @@ -168,6 +176,26 @@ CoverageStrip::hide() } m_document = nullptr; m_pane = nullptr; + m_takeName = ""; +} + +void +CoverageStrip::release() +{ + // The layer stays where it is, hidden: it is the stored coverage of a + // take that is still in the session, only not the active one + if (m_layer && m_pane) { + m_layer->setLayerDormant(m_pane, true); + } + + m_layer = nullptr; + + if (m_document) { + disconnect(m_document, nullptr, this, nullptr); + } + m_document = nullptr; + m_pane = nullptr; + m_takeName = ""; } void diff --git a/main/CoverageStrip.h b/main/CoverageStrip.h index a16cdb6d..a5940cc1 100644 --- a/main/CoverageStrip.h +++ b/main/CoverageStrip.h @@ -38,6 +38,11 @@ class RegionLayer; * session keeps them, and a session that has them is where the coverage * of its take comes from when it is loaded again. * + * Every take of the session has a strip of its own, under its own name, + * but only the active take's is held here: switching take lets one go + * (release(), which leaves it in the pane, hidden) and takes the other + * one up (adopt()). + * * The strip is display only. It is never the pane's selected layer (the * caller re-stacks the editable tracks after it is created), its model * cannot be played (a RegionModel has no play parameters), and its @@ -57,24 +62,36 @@ class CoverageStrip : public QObject virtual ~CoverageStrip(); /** - * Create the layer in the given pane, on top of whatever is there. - * Does nothing if the layer exists already. Returns false if it - * could not be created. + * Create the layer of the take of this name in the given pane, on top + * of whatever is there. Does nothing if the layer exists already. + * Returns false if it could not be created. */ - bool show(sv::Document *document, sv::Pane *pane); + bool show(sv::Document *document, sv::Pane *pane, QString takeName); /** - * Take over a layer that show() made in an earlier run, and that a - * session load has put back into the pane. Returns false if there - * is none; the coverage it holds is then read with getCoverage(). + * Take over the layer of the take of this name that show() made + * before -- in an earlier run, that a session load has put back into + * the pane, or for a take that has been switched away from and back. + * Returns false if there is none; the coverage it holds is then read + * with getCoverage(). */ - bool adopt(sv::Document *document, sv::Pane *pane); + bool adopt(sv::Document *document, sv::Pane *pane, QString takeName); /// Delete the layer and its model from the document void hide(); + /** + * Let go of the layer without deleting it: it stays in the pane, + * hidden, holding the coverage of a take that is no longer the active + * one. adopt() takes it up again. + */ + void release(); + bool isShown() const { return m_layer != nullptr; } + /// The name of the take whose strip this is, or "" when there is none + QString getTakeName() const { return m_takeName; } + /// The ranges the strip shows, which are the take's coverage void setCoverage(const Coverage &coverage); Coverage getCoverage() const; @@ -85,11 +102,10 @@ class CoverageStrip : public QObject sv::ModelId getModelId() const; /** - * The layer's object name, which is how adopt() knows it again. - * One function because phase 7a of the takes work names the layer - * after its take. + * The layer's object name, which is how adopt() knows it again: the + * name of its take and what it is (TakeLayers). */ - static QString layerName(); + static QString layerName(QString takeName); private slots: void layerAboutToBeDeleted(sv::Layer *); @@ -98,8 +114,9 @@ private slots: sv::Document *m_document; sv::Pane *m_pane; sv::RegionLayer *m_layer; + QString m_takeName; - void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *); + void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *, QString); void configureLayer(); }; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 9cef4a40..caf17acb 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -21,6 +21,7 @@ #include "LatencyUtils.h" #include "PaneUtils.h" #include "TakeEvents.h" +#include "TakeLayers.h" #include "framework/Document.h" #include "framework/VersionTester.h" @@ -40,6 +41,7 @@ #include "layer/WaveformLayer.h" #include "layer/TimeInstantLayer.h" #include "layer/TimeValueLayer.h" +#include "layer/RegionLayer.h" #include "layer/SpectrogramLayer.h" #include "widgets/Fader.h" #include "view/Overview.h" @@ -78,6 +80,7 @@ #include #include +#include #include #include #include @@ -138,6 +141,13 @@ MainWindow::MainWindow(AudioMode audioMode, m_referencePitchHiddenForTake(false), m_takes(nullptr), m_coverageStrip(nullptr), + m_takesMenu(nullptr), + m_takeCombo(nullptr), + m_newTakeAction(nullptr), + m_duplicateTakeAction(nullptr), + m_renameTakeAction(nullptr), + m_deleteTakeAction(nullptr), + m_updatingTakeCombo(false), m_eraseSingingAction(nullptr), m_selectRecordingAction(nullptr), m_openTakeCommand(nullptr), @@ -494,6 +504,7 @@ MainWindow::setupMenus() setupEditMenu(); setupViewMenu(); setupAnalysisMenu(); + setupTakesMenu(); m_mainMenusCreated = true; } @@ -991,6 +1002,63 @@ MainWindow::setupAnalysisMenu() updateAnalyseStates(); } +void +MainWindow::setupTakesMenu() +{ + if (m_mainMenusCreated) return; + + // The takes of the session: each is a whole singing performance of + // its own, with its own audio, pitch track and notes (spec 5.3). + // Switching between them is the combo box in the playback toolbar; + // making, copying, renaming and deleting them are here. None of it + // is undoable, and all of it clears the undo history but the rename + m_takesMenu = menuBar()->addMenu(tr("Ta&kes")); + m_takesMenu->setTearOffEnabled(true); + + m_keyReference->setCategory(tr("Takes")); + + m_newTakeAction = new QAction(tr("&New Empty Take"), this); + m_newTakeAction->setStatusTip + (tr("Start another take: an empty one, which the next recording " + "goes into, leaving this take as it is")); + connect(m_newTakeAction, SIGNAL(triggered()), this, SLOT(newEmptyTake())); + connect(this, SIGNAL(canChangeTakes(bool)), + m_newTakeAction, SLOT(setEnabled(bool))); + m_newTakeAction->setEnabled(false); + m_takesMenu->addAction(m_newTakeAction); + + m_duplicateTakeAction = new QAction(tr("&Duplicate Take"), this); + m_duplicateTakeAction->setStatusTip + (tr("Start another take holding a copy of this one, and carry on " + "in the copy")); + connect(m_duplicateTakeAction, SIGNAL(triggered()), + this, SLOT(duplicateTake())); + connect(this, SIGNAL(canActOnTake(bool)), + m_duplicateTakeAction, SLOT(setEnabled(bool))); + m_duplicateTakeAction->setEnabled(false); + m_takesMenu->addAction(m_duplicateTakeAction); + + m_takesMenu->addSeparator(); + + m_renameTakeAction = new QAction(tr("&Rename Take..."), this); + m_renameTakeAction->setStatusTip(tr("Give this take another name")); + connect(m_renameTakeAction, SIGNAL(triggered()), this, SLOT(renameTake())); + connect(this, SIGNAL(canActOnTake(bool)), + m_renameTakeAction, SLOT(setEnabled(bool))); + m_renameTakeAction->setEnabled(false); + m_takesMenu->addAction(m_renameTakeAction); + + m_deleteTakeAction = new QAction(tr("De&lete Take"), this); + m_deleteTakeAction->setStatusTip + (tr("Delete this take, with its pitch track and notes. Its audio " + "file is not deleted.")); + connect(m_deleteTakeAction, SIGNAL(triggered()), this, SLOT(deleteTake())); + connect(this, SIGNAL(canActOnTake(bool)), + m_deleteTakeAction, SLOT(setEnabled(bool))); + m_deleteTakeAction->setEnabled(false); + m_takesMenu->addAction(m_deleteTakeAction); +} + void MainWindow::resetAnalyseOptions() { @@ -1258,6 +1326,31 @@ MainWindow::setupToolbars() connect(this, SIGNAL(canRecord(bool)), recordAction, SLOT(setEnabled(bool))); + // The takes of the session, beside the recording controls: choosing + // one shows it, with its audio, pitch track and notes (spec 5.3). + // Making and deleting takes is the Takes menu + { + QLabel *takeLabel = new QLabel(tr(" Take:")); + QFont f = takeLabel->font(); + f.setPointSize(f.pointSize() - 1); + takeLabel->setFont(f); + takeLabel->setEnabled(false); // greyed out — purely decorative + toolbar->addWidget(takeLabel); + } + + m_takeCombo = new QComboBox; + m_takeCombo->setObjectName(tr("Take")); + m_takeCombo->setToolTip(tr("The take that is shown and played: its " + "audio, its pitch track and its notes")); + m_takeCombo->setMinimumContentsLength(8); + m_takeCombo->setSizeAdjustPolicy(QComboBox::AdjustToContents); + m_takeCombo->setEnabled(false); + connect(m_takeCombo, SIGNAL(currentIndexChanged(int)), + this, SLOT(takeChosenInCombo(int))); + connect(this, SIGNAL(canChangeTakes(bool)), + m_takeCombo, SLOT(setEnabled(bool))); + toolbar->addWidget(m_takeCombo); + toolbar = addToolBar(tr("Play Mode Toolbar")); QAction *psAction = toolbar->addAction(il.load("playselection"), @@ -1893,6 +1986,12 @@ MainWindow::updateMenuStates() emit canEraseSinging(haveCoverage && !inTake && haveSelection && !analysingRange); + // The takes of the session: switching and making one need a session + // and nothing running, and the rest need a take to act on as well + bool canChange = takeOperationsAllowed(); + emit canChangeTakes(canChange); + emit canActOnTake(canChange && m_takes->getActiveIndex() >= 0); + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -2258,6 +2357,7 @@ MainWindow::closeSession() m_takePreRoll = 0; m_takeEnd = -1; m_takeAnalysisRange = Coverage::Range(); + updateTakeCombo(); m_analyser->fileClosed(); @@ -2352,6 +2452,15 @@ MainWindow::loadSingingTrack(QString path) emit activity(tr("Load singing track \"%1\"").arg(path)); + // A track the user loads is a take of its own, analysed in full, and + // the take that was on show is put away as it is (spec 5.3) + if (m_takes->getActiveIndex() >= 0) { + closeOpenTakeCommand(true); + deactivateTake(); + m_takes->addTake(); + putOtherTakeLayersAway(); + } + ModelId singingModelId; std::vector extraPanes; FileOpenStatus status = @@ -2381,6 +2490,14 @@ MainWindow::loadSingingTrack(QString path) for (Pane *extra : extraPanes) { pruneExtraPane(extra, singingModelId); } + + // The history goes, as it does on any change of take (spec 5.4): its + // take commands are about a take that is not on show any more, and + // openPath() has just pushed an "Import" command of its own whose + // pane was pruned away again above + clearTakeHistory(); + + updateTakeCombo(); } MainWindow::FileOpenStatus @@ -2601,10 +2718,56 @@ MainWindow::analyseRestoredSingingModel() // of the file again if (m_pendingSingingModelId.isNone()) return; + // Which take this is has to be settled first: its layers are found by + // its name, so it must have one before they are looked for + if (m_takes->getActiveIndex() < 0) { + m_takes->addTake(restoredTakeName()); + } + adoptTakeLayers(m_pendingSingingModelId); analyseNewSingingModel(); } +QString +MainWindow::restoredTakeName() +{ + // The take that the singing track of a session being restored belongs + // to. Its pitch, notes and coverage are in the pane already, named + // after it, so where the pane holds one take's layers this is that + // take and it claims them. + // + // Where it holds several takes' layers, nothing in the session file + // says which of them the audio belongs to -- phase 7b writes that + // down -- so this is a take of its own, with a name none of them uses + // and an analysis of its own, and they are left in the pane hidden, + // silent and owned by nobody. + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane) return ""; + + QStringList names; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + QString name; + TakeLayers::Kind kind; + if (!TakeLayers::parse(pane->getLayer(i)->objectName(), name, kind)) { + continue; + } + if (!names.contains(name)) names.push_back(name); + m_takes->reserveTakeName(name); + } + + if (names.size() == 1) return names[0]; + + if (names.size() > 1) { + cerr << "MainWindow::restoredTakeName: the session holds the layers of " + << names.size() << " takes and does not say which of them the " + << "singing track belongs to: it becomes a take of its own" + << endl; + } + + return ""; +} + void MainWindow::openBackgroundMusic() { @@ -2870,8 +3033,19 @@ MainWindow::syncCoverageStrip() return; } + // The strip of the take that is active now. The strips of the other + // takes stay in the pane, hidden: each is the stored coverage of its + // own take, and release() is how this object lets go of one + QString name = m_takes->getActiveName(); + if (m_coverageStrip->isShown() && m_coverageStrip->getTakeName() != name) { + m_coverageStrip->release(); + } + bool wasShown = m_coverageStrip->isShown(); - if (!m_coverageStrip->show(m_document, pane)) return; + if (!m_coverageStrip->isShown()) { + m_coverageStrip->adopt(m_document, pane, name); + } + if (!m_coverageStrip->show(m_document, pane, name)) return; // The play source takes in the model of every layer that is in a // view, whether the model can be played or not, and the models it @@ -2972,15 +3146,17 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal // in the pane already, waiting to be taken over. if (!m_rebuildingTakeAudio) { if (auto wfm = ModelById::getAs(singingModelId)) { + // The take first, because its name is what the strip of a + // session that has one is stored under + m_takes->setWholeFileTake(wfm->getLocation(), + wfm->getFrameCount()); Coverage stored; if (!m_coverageStrip->isShown() && - m_coverageStrip->adopt(m_document, pane)) { + m_coverageStrip->adopt(m_document, pane, + m_takes->getActiveName())) { stored = m_coverageStrip->getCoverage(); } - if (stored.isEmpty()) { - m_takes->setWholeFileTake(wfm->getLocation(), - wfm->getFrameCount()); - } else { + if (!stored.isEmpty()) { cerr << "MainWindow::setupSingingTrackAnalyser: the session's " << "coverage strip has " << stored.getRanges().size() << " range(s) of recorded singing in it" << endl; @@ -2990,8 +3166,15 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal syncCoverageStrip(); } + // These layers are the active take's, and everything in the pane that + // is some other take's is put away: the takes that are not on show + // must not be claimed by an analyser, heard, or painted + nameActiveTakeLayers(); + putOtherTakeLayersAway(); + // Re-stack layers so the primary pitch track stays on top m_analyser->getLayer(Analyser::PitchTrack); // ensure primary is on top + updateTakeCombo(); updateLayerStatuses(); updateMenuStates(); @@ -4099,6 +4282,12 @@ MainWindow::rebuildSingingTrackFromTake(const Coverage::Range &placed) if (error == "" && m_analyser2) { error = m_analyser2->addEmptyAnalyses(); + + // The first recording of a take has just been given its layers: + // they are named after it, and they go above the takes that are + // put away, where the note tool looks for the layer to act on + nameActiveTakeLayers(); + raiseActiveTakeLayers(); } if (error != "") { @@ -4165,34 +4354,53 @@ MainWindow::adoptTakeLayers(ModelId audio) // (Analyser::claimExistingAnalyses()). The layers of a take that // has had audio swapped under it have no such link: the model they // were derived from is long gone, and a session file keeps them as - // ordinary layers. So the link is made here, for the layers in the - // pane that no analyser owns, just before the analyser that is to - // claim them is made. + // ordinary layers. So the link is made here, for the layers of the + // active take, just before the analyser that is to claim them is made. Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; if (!pane || audio.isNone()) return false; - TimeValueLayer *pitch = nullptr; - FlexiNoteLayer *notes = nullptr; + QString takeName = m_takes->getActiveName(); - for (int i = 0; i < pane->getLayerCount(); ++i) { + // By name, which is what says whose a take's layers are (spec 6.4). + // The layers of the takes that are put away have no live source model + // either, so nothing but the name can tell them apart + TakeLayers::Found found = TakeLayers::find(pane, takeName); + TimeValueLayer *pitch = found.pitch; + FlexiNoteLayer *notes = found.notes; - Layer *layer = pane->getLayer(i); + if (!pitch || !notes) { - // Ours, and not a take's: the live dots of a take being - // recorded, and the reference's pitch track moved by octaves - if (layer == m_realtimePitchLayer) continue; - if (m_alternatePitch && layer == m_alternatePitch->getLayer()) continue; + // A session saved before takes had names of their own: its take's + // layers are the ones in the pane that belong to no analyser + for (int i = 0; i < pane->getLayerCount(); ++i) { - auto model = ModelById::get(layer->getModel()); - if (!model) continue; + Layer *layer = pane->getLayer(i); - // A layer whose source model is still there has an analyser of - // its own: the reference's pitch, notes and candidates, or a - // take whose audio has not been swapped since it was analysed - if (ModelById::get(model->getSourceModel())) continue; + // Ours, and not a take's: the live dots of a take being + // recorded, and the reference's pitch track moved by octaves + if (layer == m_realtimePitchLayer) continue; + if (m_alternatePitch && + layer == m_alternatePitch->getLayer()) continue; - if (!pitch) pitch = qobject_cast(layer); - if (!notes) notes = qobject_cast(layer); + // Another take's, named as such: never ours to claim + QString otherName; + TakeLayers::Kind kind; + if (TakeLayers::parse(layer->objectName(), otherName, kind) && + otherName != takeName) { + continue; + } + + auto model = ModelById::get(layer->getModel()); + if (!model) continue; + + // A layer whose source model is still there has an analyser of + // its own: the reference's pitch, notes and candidates, or a + // take whose audio has not been swapped since it was analysed + if (ModelById::get(model->getSourceModel())) continue; + + if (!pitch) pitch = qobject_cast(layer); + if (!notes) notes = qobject_cast(layer); + } } // Half a pair is no use: the analyser claims both or neither @@ -4548,6 +4756,10 @@ MainWindow::applyTakeState(SingingTakeCommand *command, const TakeState &state) : loadTakeAudio(state.path)); if (error == "" && m_analyser2) { error = m_analyser2->addEmptyAnalyses(); + // A redo of the first recording of a take makes its layers + // again, and they are the take's + nameActiveTakeLayers(); + raiseActiveTakeLayers(); } } @@ -4633,6 +4845,485 @@ MainWindow::confirmRecordingOverTake() return yes; } +bool +MainWindow::confirmDeleteTake(QString name) +{ + return QMessageBox::question + (this, tr("Delete this take?"), + tr("Delete the take \"%1\"?

Its pitch track and its notes " + "go with it, and this cannot be undone. Its audio file is not " + "deleted.

").arg(name), + QMessageBox::Yes | QMessageBox::No, + QMessageBox::No) == QMessageBox::Yes; +} + +QString +MainWindow::askForTakeName(QString current) +{ + bool ok = false; + QString name = QInputDialog::getText + (this, tr("Rename take"), tr("Name for this take:"), + QLineEdit::Normal, current, &ok); + return ok ? name : QString(); +} + +bool +MainWindow::takeOperationsAllowed() const +{ + // Nothing about the takes of the session changes while one is being + // recorded, or while the analysis of a recorded range is running: that + // analysis is merged into the models a switch would hand over, and the + // switch would lose its result (as an erase would, see + // eraseSingingInSelection()) + if (!m_document) return false; + if (m_paneStack && m_paneStack->getPaneCount() < 1) return false; + if (!getMainModel()) return false; + if (m_recordTarget && m_recordTarget->isRecording()) return false; + if (m_analyser2 && m_analyser2->isAnalysingRange()) return false; + return true; +} + +void +MainWindow::clearTakeHistory() +{ + // A take command holds the state of one take, and after a switch, a + // new take, a duplicate or a delete it is not the take on show any + // more: undoing it would write one take's singing over another's. + // Spec 5.4: the whole history goes, with no prompt + closeOpenTakeCommand(true); + CommandHistory::getInstance()->clear(); +} + +void +MainWindow::nameActiveTakeLayers() +{ + // The pitch and notes layers the singing analyser holds are the active + // take's, and their object names say so: that is the only link between + // a take and the layers that show it (spec 6.4). A whole-file + // analysis and addEmptyAnalyses() both name them after the transform + // that made them, so this is done every time they change hands + QString name = m_takes->getActiveName(); + if (name == "" || !m_analyser2) return; + + const struct { Analyser::Component component; TakeLayers::Kind kind; } + wanted[] = { + { Analyser::PitchTrack, TakeLayers::Pitch }, + { Analyser::Notes, TakeLayers::Notes } + }; + + for (const auto &w : wanted) { + Layer *layer = m_analyser2->getLayer(w.component); + if (!layer) continue; + QString objectName = TakeLayers::nameFor(name, w.kind); + if (layer->objectName() != objectName) { + layer->setObjectName(objectName); + } + } +} + +void +MainWindow::putOtherTakeLayersAway() +{ + // Everything in pane 0 that belongs to a take other than the active + // one: hidden, silent, out of the play source and with no source + // model. Three things depend on this: + // + // - an Analyser claims a pitch or notes layer whose model's source + // model is its own audio, so a take that is put away must have no + // source model at all (and adoptTakeLayers() goes by name); + // - the play source takes in every model of a layer that is in a + // view, and what it holds says where playback ends, so a take + // longer than the active one would hold playback open past the end + // of what there is to hear (5a's finding, as for the strip); + // - a pitch track and a set of notes can be sonified in Tony, and a + // take that is not on show is not to be heard. + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane) return; + + QString active = m_takes->getActiveName(); + + for (int i = 0; i < pane->getLayerCount(); ++i) { + + Layer *layer = pane->getLayer(i); + + QString name; + TakeLayers::Kind kind; + if (!TakeLayers::parse(layer->objectName(), name, kind)) continue; + if (name == active) continue; + + layer->setLayerDormant(pane, true); + + if (auto params = layer->getPlayParameters()) { + params->setPlayAudible(false); + } + + ModelId modelId = layer->getModel(); + if (auto model = ModelById::get(modelId)) { + model->setSourceModel(ModelId()); + } + if (m_playSource && !modelId.isNone()) { + m_playSource->removeModel(modelId); + } + } +} + +void +MainWindow::raiseActiveTakeLayers() +{ + // The note tool acts on the pane's topmost note layer, whether that + // layer is dormant or not, so the active take's notes have to be above + // the ones of the takes that are put away. A take's layers are added + // to the pane when it is first recorded into, so without this the + // newest take would keep the tools to itself + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane || !m_analyser2) return; + + for (Analyser::Component c : { Analyser::PitchTrack, Analyser::Notes }) { + if (Layer *layer = m_analyser2->getLayer(c)) { + TakeLayers::raise(pane, layer); + } + } +} + +void +MainWindow::deactivateTake() +{ + // The take being put away keeps its layers, with every event in them: + // they are where it is stored (spec 6.4). What it loses is the + // analyser, the audio model under it, and its place in the mix + if (m_analyser2) { + m_analyser2->releaseLayers(); + delete m_analyser2; + m_analyser2 = nullptr; + } + + // The strip stays in the pane as well, holding this take's coverage + m_coverageStrip->release(); + + // Which leaves nothing belonging to the active take but its name: + // putOtherTakeLayersAway() does the rest once another take is active +} + +bool +MainWindow::activateTake() +{ + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (!pane) return false; + + QString name = m_takes->getActiveName(); + if (name == "") return false; + + QString path = m_takes->getAudioPath(); + QString error; + + if (path != "") { + // The take's audio under the take's layers, with no analysis: + // loadTakeAudio() finds them by name (adoptTakeLayers()) and the + // new analyser claims them. m_rebuildingTakeAudio: the coverage + // of this file is the take's own, not "the whole of it" + bool wasRebuilding = m_rebuildingTakeAudio; + m_rebuildingTakeAudio = true; + error = loadTakeAudio(path); + m_rebuildingTakeAudio = wasRebuilding; + + // A take that has audio but no pitch and notes -- one whose + // layers were lost with a session that could not be read back -- + // gets empty ones, so that the next recording has something to + // merge into + if (error == "" && m_analyser2) { + error = m_analyser2->addEmptyAnalyses(); + } + nameActiveTakeLayers(); + } + + // The models of the take that is on show now are in the play source + // again: they came out of it when the take was put away. Whether they + // are seen and heard is the analyser's business -- it has just loaded + // the show and play settings onto them, as it does for any file + if (m_analyser2 && m_playSource) { + for (Analyser::Component c : { Analyser::PitchTrack, Analyser::Notes }) { + Layer *layer = m_analyser2->getLayer(c); + if (!layer || layer->getModel().isNone()) continue; + m_playSource->addModel(layer->getModel()); + } + } + + putOtherTakeLayersAway(); + syncCoverageStrip(); + raiseActiveTakeLayers(); + + if (error != "") { + // Its pitch and notes are there; only the sound is missing + // (spec 6.4, a missing audio file) + QMessageBox::warning + (this, + tr("Failed to open the take's audio"), + tr("The take \"%1\" is shown without its audio

%2

") + .arg(name).arg(error), + QMessageBox::Ok); + } + + return error == ""; +} + +bool +MainWindow::switchToTake(int index) +{ + if (!takeOperationsAllowed()) return false; + if (index == m_takes->getActiveIndex()) return true; + if (!m_takes->getTake(index)) return false; + + cerr << "MainWindow::switchToTake: from take " + << m_takes->getActiveIndex() << " to " << index << endl; + + clearTakeHistory(); + + deactivateTake(); + m_takes->setActiveIndex(index); + bool ok = activateTake(); + + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + + emit activity(tr("Switched to the take \"%1\"") + .arg(m_takes->getActiveName())); + + return ok; +} + +void +MainWindow::takeChosenInCombo(int index) +{ + if (m_updatingTakeCombo) return; + if (index < 0) return; + if (index == m_takes->getActiveIndex()) return; + + if (!switchToTake(index)) { + // Whatever went wrong, the combo box must go on saying which take + // is the active one + updateTakeCombo(); + } +} + +void +MainWindow::updateTakeCombo() +{ + if (!m_takeCombo) return; + + m_updatingTakeCombo = true; + + QStringList names = m_takes->getTakeNames(); + QStringList shown; + for (int i = 0; i < m_takeCombo->count(); ++i) { + shown.push_back(m_takeCombo->itemText(i)); + } + + if (shown != names) { + m_takeCombo->clear(); + m_takeCombo->addItems(names); + } + m_takeCombo->setCurrentIndex(m_takes->getActiveIndex()); + + m_updatingTakeCombo = false; +} + +void +MainWindow::newEmptyTake() +{ + if (!takeOperationsAllowed()) return; + + clearTakeHistory(); + + // The take on show is put away as it is; the new one has no audio and + // no layers until the first recording goes into it + deactivateTake(); + QString name = m_takes->addTake(); + putOtherTakeLayersAway(); + syncCoverageStrip(); + + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + + // The session has a take it did not have before + documentModified(); + + emit activity(tr("Started the empty take \"%1\"").arg(name)); +} + +void +MainWindow::duplicateTake() +{ + if (!takeOperationsAllowed()) return; + if (m_takes->getActiveIndex() < 0) return; + + // The events of the take being copied, taken before anything moves: + // the copy gets models of its own holding the same events, and the + // same audio file, which neither take writes over (spec 5.4) + EventVector pitchEvents, notesEvents; + if (m_analyser2) { + if (Layer *layer = m_analyser2->getLayer(Analyser::PitchTrack)) { + if (auto model = ModelById::getAs + (layer->getModel())) { + pitchEvents = model->getAllEvents(); + } + } + if (Layer *layer = m_analyser2->getLayer(Analyser::Notes)) { + if (auto model = ModelById::getAs(layer->getModel())) { + notesEvents = model->getAllEvents(); + } + } + } + + clearTakeHistory(); + + QString from = m_takes->getActiveName(); + deactivateTake(); + QString name = m_takes->duplicateActiveTake(); + + // The copy has no layers of its own yet: activateTake() opens the + // audio, and addEmptyAnalyses() makes the pitch and notes that the + // events below go into + activateTake(); + + if (m_analyser2) { + if (Layer *layer = m_analyser2->getLayer(Analyser::PitchTrack)) { + if (auto model = ModelById::getAs + (layer->getModel())) { + for (const Event &e : pitchEvents) model->add(e); + } + } + if (Layer *layer = m_analyser2->getLayer(Analyser::Notes)) { + if (auto model = ModelById::getAs(layer->getModel())) { + for (const Event &e : notesEvents) model->add(e); + } + } + } + + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + documentModified(); + + emit activity(tr("Copied the take \"%1\" into \"%2\"") + .arg(from).arg(name)); +} + +void +MainWindow::renameTake() +{ + // The one take operation that leaves the undo history alone: nothing + // of the singing changes, only what the take is called + int index = m_takes->getActiveIndex(); + if (index < 0) return; + + QString current = m_takes->getActiveName(); + QString name = askForTakeName(current); + if (name == "" || name == current) return; + + if (!m_takes->renameTake(index, name)) { + QMessageBox::warning + (this, tr("Could not rename the take"), + tr("The take could not be renamed to \"%1\"

Another " + "take of this session has that name.

").arg(name), + QMessageBox::Ok); + return; + } + + name = m_takes->getActiveName(); + + // The take's layers are named after it, so they are renamed with it + nameActiveTakeLayers(); + + if (Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr) { + TakeLayers::Found found = TakeLayers::find(pane, current); + if (found.coverage) { + found.coverage->setObjectName + (TakeLayers::nameFor(name, TakeLayers::Coverage)); + } + // The strip remembers the name it was adopted under, so it takes + // its layer up again under the new one + m_coverageStrip->release(); + syncCoverageStrip(); + } + + updateTakeCombo(); + updateMenuStates(); + documentModified(); + + emit activity(tr("The take \"%1\" is called \"%2\" now") + .arg(current).arg(name)); +} + +void +MainWindow::deleteTake() +{ + if (!takeOperationsAllowed()) return; + + int index = m_takes->getActiveIndex(); + const SingingTakes::Take *take = m_takes->getTake(index); + if (!take) return; + + if (!confirmDeleteTake(take->name)) return; + + deleteTakeAt(index); +} + +bool +MainWindow::deleteTakeAt(int index) +{ + const SingingTakes::Take *take = m_takes->getTake(index); + if (!take) return false; + if (!m_document) return false; + + QString name = take->name; + bool wasActive = (index == m_takes->getActiveIndex()); + + cerr << "MainWindow::deleteTakeAt: deleting take \"" << name + << "\" (" << (wasActive ? "active" : "inactive") << ")" << endl; + + clearTakeHistory(); + + if (wasActive) { + // Its analyser and its audio go; the layers are deleted below + deactivateTake(); + } + + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + if (pane) { + TakeLayers::Found found = TakeLayers::find(pane, name); + for (Layer *layer : { static_cast(found.pitch), + static_cast(found.notes), + static_cast(found.coverage) }) { + if (!layer) continue; + // As the analyser and the strip take their own layers away: + // no command, and the model goes with the layer + if (m_playSource && !layer->getModel().isNone()) { + m_playSource->removeModel(layer->getModel()); + } + m_document->deleteLayer(layer, true); + } + } + + m_takes->removeTake(index); + + // A neighbour is the active take now, or there is none at all. Its + // audio has to be opened either way: the take that was deleted had it + if (wasActive && m_takes->getActiveIndex() >= 0) { + activateTake(); + } + + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + documentModified(); + + emit activity(tr("Deleted the take \"%1\"").arg(name)); + + return true; +} + void MainWindow::openLocation() { diff --git a/main/MainWindow.h b/main/MainWindow.h index ee50e0c8..4cea7718 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -31,6 +31,7 @@ #include "data/model/SparseTimeValueModel.h" class QTimer; +class QComboBox; namespace sv { class VersionTester; @@ -80,6 +81,11 @@ class MainWindow : public sv::MainWindowBase void canShowRealtimePitch(bool); void canEraseSinging(bool); void canSelectRecording(bool); + // Switching and making takes: not while a take is being recorded or a + // recorded range analysed. The second is the same with a take there + // to be copied, renamed or deleted + void canChangeTakes(bool); + void canActOnTake(bool); public slots: virtual bool commitData(bool mayAskUser); // on session shutdown @@ -125,6 +131,13 @@ protected slots: virtual void eraseSingingInSelection(); virtual void selectRecordingAtPlayhead(); + // The Takes menu and the "Take:" combo box (spec 5.3) + virtual void takeChosenInCombo(int index); + virtual void newEmptyTake(); + virtual void duplicateTake(); + virtual void renameTake(); + virtual void deleteTake(); + virtual void snapNotesToPitches(); virtual void splitNote(); virtual void mergeNotes(); @@ -316,6 +329,85 @@ protected slots: // is a take and none yet, take it away when the take goes void syncCoverageStrip(); + // --- Several takes (spec 5.3) --- + // + // Every take of the session has its three layers in pane 0, named + // after it (TakeLayers); only the active take has an audio model and + // the singing analyser. The take a menu or the combo box asks for is + // made the active one by switchToTake(), which is the swap of 6.2 + // with the layers of another take put in place of the ones on show. + + QMenu *m_takesMenu; + QComboBox *m_takeCombo; + QAction *m_newTakeAction; + QAction *m_duplicateTakeAction; + QAction *m_renameTakeAction; + QAction *m_deleteTakeAction; + + // Set while updateTakeCombo() fills the combo box, so that the + // currentIndexChanged it causes is not taken for the user's choice + bool m_updatingTakeCombo; + + void setupTakesMenu(); + + // The combo box lists the takes with the active one selected + void updateTakeCombo(); + + // Make the take at this index the active one: release the analyser + // keeping the layers, put the take's layers away, then show the other + // take's and open its audio underneath them. Nothing is analysed. + // The undo history goes, as spec 5.4 says it must: its commands hold + // the state of a take that is no longer on show + bool switchToTake(int index); + + // Release the singing analyser and the take's audio with it, and put + // the active take's layers away: still in the pane and in the + // session, hidden, silent and owned by nobody + void deactivateTake(); + + // Show the active take: its audio under its layers, an analyser that + // claims them (no analysis), its coverage strip, and the layers + // themselves visible, audible as the user asked and within reach of + // the editing tools. False if its audio could not be opened, in + // which case the user has been told + bool activateTake(); + + // The name for the take that the singing track of a session being + // restored belongs to: the name its layers in the pane are under, or + // "" where the session does not say which take that is. Reserves the + // names of the takes whose layers are left in the pane + QString restoredTakeName(); + + // Name the layers the singing analyser holds after the active take, + // which is what says whose they are (spec 6.4) + void nameActiveTakeLayers(); + + // Every take layer in pane 0 that is not the active take's: hidden, + // silent, out of the play source and with no source model, so that + // nothing claims it and nothing hears it. The one place that + // enforces it, for a switch and for a session load alike + void putOtherTakeLayersAway(); + + // The active take's pitch and notes to the top of the pane, where the + // note tool looks for the layer to act on + void raiseActiveTakeLayers(); + + // Forget the take at this index, with its layers; no audio file is + // touched. Deleting the active take activates a neighbour + bool deleteTakeAt(int index); + + // A take may be switched, made, copied or deleted only when nothing + // is being recorded or analysed + bool takeOperationsAllowed() const; + + // Every take operation but Rename clears the undo history (spec 5.4) + void clearTakeHistory(); + + // Ask before deleting a take, and for a take's new name. Overridden + // by the tests, which cannot answer a dialog + virtual bool confirmDeleteTake(QString name); + virtual QString askForTakeName(QString current); + // Erase Singing in Selection, and selecting the recording the // playhead is in QAction *m_eraseSingingAction; diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 30a80b2b..9dd918e6 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -26,7 +26,9 @@ using namespace sv; SingingTakes::SingingTakes(QObject *parent) : - QObject(parent) + QObject(parent), + m_active(-1), + m_named(0) { } @@ -37,28 +39,195 @@ SingingTakes::~SingingTakes() void SingingTakes::clear() { - m_audioPath = ""; - m_coverage.clear(); + m_takes.clear(); + m_active = -1; + m_named = 0; + m_reserved.clear(); m_superseded.clear(); m_written.clear(); m_protected.clear(); } +const SingingTakes::Take * +SingingTakes::activeTake() const +{ + if (m_active < 0 || m_active >= int(m_takes.size())) return nullptr; + return &m_takes[m_active]; +} + +SingingTakes::Take & +SingingTakes::takeForRecording() +{ + if (m_active < 0 || m_active >= int(m_takes.size())) { + // The first recording of a session records into a take of its own + // (spec 5.3), and so does one made after every take was deleted + addTake(); + } + return m_takes[m_active]; +} + +bool +SingingTakes::haveTake() const +{ + const Take *take = activeTake(); + return take && take->audioPath != ""; +} + +QString +SingingTakes::getAudioPath() const +{ + const Take *take = activeTake(); + return take ? take->audioPath : QString(); +} + +const Coverage & +SingingTakes::getCoverage() const +{ + static const Coverage empty; + const Take *take = activeTake(); + return take ? take->coverage : empty; +} + +QString +SingingTakes::getActiveName() const +{ + const Take *take = activeTake(); + return take ? take->name : QString(); +} + +QStringList +SingingTakes::getTakeNames() const +{ + QStringList names; + for (const Take &take : m_takes) names.push_back(take.name); + return names; +} + +int +SingingTakes::indexOf(QString name) const +{ + for (int i = 0; i < int(m_takes.size()); ++i) { + if (m_takes[i].name == name) return i; + } + return -1; +} + +const SingingTakes::Take * +SingingTakes::getTake(int index) const +{ + if (index < 0 || index >= int(m_takes.size())) return nullptr; + return &m_takes[index]; +} + +bool +SingingTakes::setActiveIndex(int index) +{ + if (index < 0 || index >= int(m_takes.size())) return false; + m_active = index; + return true; +} + +QString +SingingTakes::addTake(QString name) +{ + Take take; + + if (name != "" && indexOf(name) < 0) { + take.name = name; + } else { + // "Take N" with an N this session has not used, however many takes + // have been deleted since: the name is the take's identity, and + // the layers of a deleted take may still be in the document. Not + // translated: the names of the take's layers are built from it and + // stored in the session file + do { + take.name = QString("Take %1").arg(++m_named); + } while (indexOf(take.name) >= 0 || m_reserved.contains(take.name)); + } + + m_takes.push_back(take); + m_active = int(m_takes.size()) - 1; + return take.name; +} + +QString +SingingTakes::duplicateActiveTake(QString name) +{ + const Take *from = activeTake(); + if (!from) return ""; + + // Copied before addTake(), which may make the vector move + QString path = from->audioPath; + Coverage coverage = from->coverage; + + QString made = addTake(name); + m_takes[m_active].audioPath = path; + m_takes[m_active].coverage = coverage; + + // The audio file is not copied and not superseded: the two takes read + // the same one until one of them is recorded into or erased from, + // which writes a new file anyway (spec 5.4) + return made; +} + +bool +SingingTakes::renameTake(int index, QString name) +{ + if (index < 0 || index >= int(m_takes.size())) return false; + + name = name.trimmed(); + if (name == "") return false; + + int existing = indexOf(name); + if (existing >= 0 && existing != index) return false; + + m_takes[index].name = name; + return true; +} + +void +SingingTakes::reserveTakeName(QString name) +{ + if (name != "" && !m_reserved.contains(name)) m_reserved.push_back(name); +} + +bool +SingingTakes::removeTake(int index) +{ + if (index < 0 || index >= int(m_takes.size())) return false; + + m_takes.erase(m_takes.begin() + index); + + if (m_takes.empty()) { + m_active = -1; + } else if (index < m_active) { + --m_active; + } else if (index == m_active) { + // A neighbour takes over: the one before, or the first if this + // was it + m_active = (index > 0 ? index - 1 : 0); + } + + return true; +} + void SingingTakes::setTake(QString path, const Coverage &coverage) { - if (m_audioPath != "" && m_audioPath != path) { - m_superseded.push_back(m_audioPath); + Take &take = takeForRecording(); + if (take.audioPath != "" && take.audioPath != path) { + m_superseded.push_back(take.audioPath); } - m_audioPath = path; - m_coverage = coverage; + take.audioPath = path; + take.coverage = coverage; } void SingingTakes::restoreTake(QString path, const Coverage &coverage) { - m_audioPath = path; - m_coverage = coverage; + Take &take = takeForRecording(); + take.audioPath = path; + take.coverage = coverage; } void @@ -70,9 +239,14 @@ SingingTakes::protectPath(QString path) QStringList SingingTakes::unusedWrittenFiles() const { + QStringList inUse; + for (const Take &take : m_takes) { + if (take.audioPath != "") inUse.push_back(take.audioPath); + } + QStringList unused; for (const QString &path : m_written) { - if (path == m_audioPath) continue; + if (inUse.contains(path)) continue; if (m_protected.contains(path)) continue; if (unused.contains(path)) continue; unused.push_back(path); @@ -118,16 +292,18 @@ SingingTakes::spliceRecording(QString recordingPath, "in \"%1\"").arg(directory); } + Take &take = takeForRecording(); + Coverage::Range range; - QString error = TakeAudio::splice(m_audioPath, recordingPath, + QString error = TakeAudio::splice(take.audioPath, recordingPath, recordingOffset, position, length, outPath, &range); if (error != "") return error; - if (m_audioPath != "") m_superseded.push_back(m_audioPath); - m_audioPath = outPath; + if (take.audioPath != "") m_superseded.push_back(take.audioPath); + take.audioPath = outPath; m_written.push_back(outPath); - m_coverage.add(range.start, range.end); + take.coverage.add(range.start, range.end); if (placed) *placed = range; return ""; @@ -144,11 +320,13 @@ SingingTakes::eraseRanges(const Coverage::Ranges &ranges, QString directory, // the end of the singing must not make the file any longer Coverage wanted; for (const Coverage::Range &r : ranges) { - for (const Coverage::Range &covered : m_coverage.getRanges()) { + for (const Coverage::Range &covered : getCoverage().getRanges()) { wanted.add(std::max(r.start, covered.start), std::min(r.end, covered.end)); } } + // Nothing in the selection holds recorded singing, and there may be no + // take at all if (wanted.isEmpty()) return ""; QString outPath = nextAudioPath(directory); @@ -157,14 +335,17 @@ SingingTakes::eraseRanges(const Coverage::Ranges &ranges, QString directory, "in \"%1\"").arg(directory); } - QString error = TakeAudio::erase(m_audioPath, wanted.getRanges(), outPath); + Take &take = takeForRecording(); + + QString error = TakeAudio::erase(take.audioPath, wanted.getRanges(), + outPath); if (error != "") return error; - m_superseded.push_back(m_audioPath); - m_audioPath = outPath; + m_superseded.push_back(take.audioPath); + take.audioPath = outPath; m_written.push_back(outPath); for (const Coverage::Range &r : wanted.getRanges()) { - m_coverage.remove(r.start, r.end); + take.coverage.remove(r.start, r.end); } if (erased) *erased = wanted.getRanges(); @@ -174,7 +355,7 @@ SingingTakes::eraseRanges(const Coverage::Ranges &ranges, QString directory, bool SingingTakes::coversPosition(sv_frame_t position) const { - return m_coverage.contains(position); + return getCoverage().contains(position); } bool diff --git a/main/SingingTakes.h b/main/SingingTakes.h index cf1d73a9..ee5d949a 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -23,14 +23,20 @@ #include #include +#include + /** - * The state of the singing takes of a session: the audio file each - * take's singing lives in, and the ranges of that file that hold - * recorded material. Until takes proper arrive there is at most one. + * The state of the singing takes of a session: for each take, the audio + * file its singing lives in and the ranges of that file that hold + * recorded material. One take is the active one, and the single-take + * calls below -- getAudioPath(), spliceRecording() and the rest -- are + * all about that one. * * It knows nothing of layers, models or windows: MainWindow asks it * where a recording is to go and what the take is afterwards, and puts - * the answer on the screen itself. The audio files are written by + * the answer on the screen itself. A take's identity on the screen is + * its name, which is what the names of its layers are built from; no + * layer or model is named here. The audio files are written by * TakeAudio. */ class SingingTakes : public QObject @@ -41,18 +47,90 @@ class SingingTakes : public QObject SingingTakes(QObject *parent = nullptr); virtual ~SingingTakes(); - /// There is a take, with an audio file to play and analyse - bool haveTake() const { return m_audioPath != ""; } + /** + * One take: its name, the audio file that holds its singing (empty + * before its first recording) and the ranges of that file that hold + * recorded material. + */ + struct Take { + QString name; + QString audioPath; + Coverage coverage; + }; + + typedef std::vector Takes; - QString getAudioPath() const { return m_audioPath; } - const Coverage &getCoverage() const { return m_coverage; } + /// There is an active take, with an audio file to play and analyse + bool haveTake() const; + + QString getAudioPath() const; + const Coverage &getCoverage() const; /// Nothing recorded and nothing to go back to: a new session void clear(); + // --- The takes of the session (spec 5.3) --- + + const Takes &getTakes() const { return m_takes; } + int getTakeCount() const { return int(m_takes.size()); } + + /// Which take is the active one, or -1 when there is no take at all + int getActiveIndex() const { return m_active; } + + /// The name of the active take, or "" when there is none + QString getActiveName() const; + + QStringList getTakeNames() const; + + /// The take of this name, or -1 + int indexOf(QString name) const; + + /// The take at this index, or null + const Take *getTake(int index) const; + + /// Make the take at this index the active one. False if there is none + bool setActiveIndex(int index); + + /** + * Add a take with no audio yet and make it the active one; the + * return is its name. With no name given it is called "Take N", + * with an N that no take of this session has had. + */ + QString addTake(QString name = ""); + /** - * The take is this file, with the coverage a session kept for it in - * its coverage strip. + * Add a take that holds the same audio file and coverage as the + * active one, and make it the active one; the return is its name, or + * "" if there was no take to copy. The audio file is shared: both + * takes write a new one before they change anything in it. + */ + QString duplicateActiveTake(QString name = ""); + + /** + * Rename the take at this index. False if there is no such take, if + * the name is empty, or if another take has it already. + */ + bool renameTake(int index, QString name); + + /** + * Do not give this name to a take that is named by default, although + * no take of the session has it: the layers of a take that a session + * held and that this session has not taken up are in the document + * under it. A name asked for by name is still given. + */ + void reserveTakeName(QString name); + + /** + * Forget the take at this index; no audio file is touched. If it was + * the active one, the take before it becomes active, or the one after + * it if it was the first, or there is no active take left. False if + * there is no such take. + */ + bool removeTake(int index); + + /** + * The active take is this file, with the coverage a session kept for + * it in its coverage strip. With no take at all, one is added. */ void setTake(QString path, const Coverage &coverage); @@ -137,8 +215,9 @@ class SingingTakes : public QObject /** * The files this run wrote that nothing refers to any more: every - * one but the take's own audio and the protected ones. Files the - * user brought are never in the list. + * one but the audio of a take -- any take, since a duplicate shares + * its file with the take it was made from -- and the protected ones. + * Files the user brought are never in the list. */ QStringList unusedWrittenFiles() const; @@ -167,11 +246,24 @@ class SingingTakes : public QObject static QString nextAudioPath(QString directory); private: - QString m_audioPath; - Coverage m_coverage; + Takes m_takes; + int m_active; + + // Counts the takes this session has named, so that a name is never + // used twice even after the take that had it has gone + int m_named; + + // Names no take has, that are not to be given to one all the same + QStringList m_reserved; + QStringList m_superseded; QStringList m_written; QStringList m_protected; + + // The active take, or null. The non-const one adds a take if there + // is none: the first recording of a session makes "Take 1" + const Take *activeTake() const; + Take &takeForRecording(); }; #endif diff --git a/main/TakeLayers.cpp b/main/TakeLayers.cpp new file mode 100644 index 00000000..bb40b129 --- /dev/null +++ b/main/TakeLayers.cpp @@ -0,0 +1,108 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeLayers.h" + +#include "view/Pane.h" +#include "layer/TimeValueLayer.h" +#include "layer/FlexiNoteLayer.h" +#include "layer/RegionLayer.h" + +using namespace sv; + +// The word each kind of layer is named after its take +static QString +suffixFor(TakeLayers::Kind kind) +{ + switch (kind) { + case TakeLayers::Pitch: return "Pitch"; + case TakeLayers::Notes: return "Notes"; + case TakeLayers::Coverage: default: return "Coverage"; + } +} + +QString +TakeLayers::nameFor(QString takeName, Kind kind) +{ + if (takeName == "") return ""; + return takeName + " " + suffixFor(kind); +} + +bool +TakeLayers::parse(QString layerName, QString &takeName, Kind &kind) +{ + for (Kind k : { Pitch, Notes, Coverage }) { + QString suffix = " " + suffixFor(k); + if (!layerName.endsWith(suffix)) continue; + QString name = layerName.left(layerName.length() - suffix.length()); + if (name == "") continue; + takeName = name; + kind = k; + return true; + } + return false; +} + +TakeLayers::Found +TakeLayers::find(Pane *pane, QString takeName) +{ + Found found; + if (!pane || takeName == "") return found; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + + Layer *layer = pane->getLayer(i); + QString name = layer->objectName(); + + if (!found.pitch && name == nameFor(takeName, Pitch)) { + found.pitch = qobject_cast(layer); + } + if (!found.notes && name == nameFor(takeName, Notes)) { + found.notes = qobject_cast(layer); + } + if (!found.coverage && name == nameFor(takeName, Coverage)) { + found.coverage = qobject_cast(layer); + } + } + + return found; +} + +void +TakeLayers::raise(Pane *pane, Layer *layer) +{ + if (!pane || !layer) return; + + // The stacking order of a pane is the order its layers were added in: + // Analyser::stackLayers() does nothing in Tony, because it goes + // through PaneStack::setCurrentLayer(), which needs a PropertyStack + // that this pane stack has none of. So the only way to put a layer on + // top is to add it to the view again. + // + // View::removeLayer() and addLayer() are the view's own bookkeeping: + // Document::m_layerViewMap is untouched, so the layer stays a layer of + // this view as far as the document and the session file are concerned, + // and no command is made. If Pane::getTopFlexiNoteLayer() ever skips + // dormant layers (a fork change), the active take's notes would be on + // top without this. + // + // removeLayer() does not disconnect layerMeasurementRectsChanged, + // which addLayer() connects, so that one is undone here: without it + // the connection is made again on every switch. + QObject::disconnect(layer, SIGNAL(layerMeasurementRectsChanged()), + pane, nullptr); + + pane->removeLayer(layer); + pane->addLayer(layer); +} diff --git a/main/TakeLayers.h b/main/TakeLayers.h new file mode 100644 index 00000000..48885a2a --- /dev/null +++ b/main/TakeLayers.h @@ -0,0 +1,78 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_LAYERS_H +#define TONY_TAKE_LAYERS_H + +#include + +namespace sv { +class Layer; +class Pane; +class TimeValueLayer; +class FlexiNoteLayer; +class RegionLayer; +} + +/** + * The layers of a take, found by name. + * + * Every take of a session has three layers in pane 0 -- pitch, notes and + * coverage -- and their object names carry the identity of the take they + * belong to: "Take 2 Pitch", "Take 2 Notes", "Take 2 Coverage" (spec + * 6.4). That is the only link between a take and what shows it: no + * pointer to a layer is kept anywhere but in the analyser of the take + * that is active, so a take whose layers were put back by a session load + * is found the same way as one that has been switched away from. + * + * The names are not translated: they are stored in the session file. + */ +class TakeLayers +{ +public: + enum Kind { Pitch, Notes, Coverage }; + + /// The object name of that layer of the take of this name + static QString nameFor(QString takeName, Kind kind); + + /** + * Take a layer's object name apart again. False if it is not the + * name of a take's layer at all. + */ + static bool parse(QString layerName, QString &takeName, Kind &kind); + + /// The layers of the take of this name that are in the pane + struct Found { + sv::TimeValueLayer *pitch = nullptr; + sv::FlexiNoteLayer *notes = nullptr; + sv::RegionLayer *coverage = nullptr; + }; + + static Found find(sv::Pane *pane, QString takeName); + + /** + * Move a layer to the top of the pane's layer stack, which is where + * the editing tools look for the layer to act on: the note tool takes + * the pane's topmost note layer, dormant or not, so the active take's + * notes have to be above the takes that are put away. + * + * Done on the view alone -- the layer is not added to or removed from + * the document, so nothing is created, deleted or made undoable, and + * the layer goes on being saved with the session. See the note in + * the implementation about the fork change that would retire this. + */ + static void raise(sv::Pane *pane, sv::Layer *layer); +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index fbfc9122..19d84936 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -29,6 +29,7 @@ #include "../Analyser.h" #include "../CoverageStrip.h" #include "../SingingTakes.h" +#include "../TakeLayers.h" #include "version.h" @@ -63,6 +64,7 @@ #include #include #include +#include #include #include #include @@ -104,6 +106,26 @@ class TestMainWindow : public MainWindow QAction *selectRecordingAction() { return m_selectRecordingAction; } void doUpdateMenuStates() { updateMenuStates(); } + // The takes of the session, as the Takes menu and the combo box do + bool doSwitchToTake(int index) { return switchToTake(index); } + void doChooseTakeInCombo(int index) { m_takeCombo->setCurrentIndex(index); } + void doNewEmptyTake() { newEmptyTake(); } + void doDuplicateTake() { duplicateTake(); } + void doRenameTake() { renameTake(); } + void doDeleteTake() { deleteTake(); } + bool doDeleteTakeAt(int index) { return deleteTakeAt(index); } + QComboBox *takeCombo() { return m_takeCombo; } + QAction *newTakeAction() { return m_newTakeAction; } + QAction *duplicateTakeAction() { return m_duplicateTakeAction; } + QAction *renameTakeAction() { return m_renameTakeAction; } + QAction *deleteTakeAction() { return m_deleteTakeAction; } + + // The two questions the take operations ask, answered from here: the + // suite cannot answer a dialog + void setDeleteTakeAnswer(bool yes) { m_deleteTakeAnswer = yes; } + int deleteTakeQuestions() const { return m_deleteTakeQuestions; } + void setTakeNameAnswer(QString name) { m_takeNameAnswer = name; } + // True between the start of the analysis of a recorded range and the // merge of its result into the take's pitch and notes bool analysingRange() { @@ -208,6 +230,15 @@ class TestMainWindow : public MainWindow return m_recordOverAnswer; } + bool confirmDeleteTake(QString) override { + ++m_deleteTakeQuestions; + return m_deleteTakeAnswer; + } + + QString askForTakeName(QString current) override { + return m_takeNameAnswer == "" ? current : m_takeNameAnswer; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -216,6 +247,9 @@ class TestMainWindow : public MainWindow bool m_installDevice; bool m_recordOverAnswer = true; int m_recordOverQuestions = 0; + bool m_deleteTakeAnswer = true; + int m_deleteTakeQuestions = 0; + QString m_takeNameAnswer; }; class TestRecordWorkflow : public QObject @@ -492,17 +526,69 @@ class TestRecordWorkflow : public QObject return m_window->coverageStrip()->getLayer(); } + // The strips of the active take, and of every take: each take of the + // session has one, and only the active take's is on show int stripLayersInPane0() { + return stripLayersNamed + (CoverageStrip::layerName(m_window->takes()->getActiveName())); + } + + int allStripLayersInPane0() { + return stripLayersNamed(""); + } + + int stripLayersNamed(QString name) { int n = 0; sv::Pane *pane = m_window->paneStack()->getPane(0); if (!pane) return 0; for (int i = 0; i < pane->getLayerCount(); ++i) { - if (pane->getLayer(i)->objectName() == - CoverageStrip::layerName()) ++n; + QString takeName; + TakeLayers::Kind kind; + if (!TakeLayers::parse(pane->getLayer(i)->objectName(), + takeName, kind)) continue; + if (kind != TakeLayers::Coverage) continue; + if (name != "" && pane->getLayer(i)->objectName() != name) continue; + ++n; } return n; } + // The layers of the take of this name, found the way MainWindow and + // (from 7b) a session restore find them: by their object names + TakeLayers::Found takeLayers(QString name) { + return TakeLayers::find(m_window->paneStack()->getPane(0), name); + } + + // What a take that is not the active one has to be: hidden, silent and + // out of the play source, so that it is neither seen nor heard and + // cannot hold playback open past the end of the active take's audio + void verifyTakeIsPutAway(QString name) { + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane); + TakeLayers::Found found = takeLayers(name); + auto models = m_window->playSource()->getModels(); + for (sv::Layer *layer : { static_cast(found.pitch), + static_cast(found.notes), + static_cast(found.coverage) }) { + if (!layer) continue; + QVERIFY2(layer->isLayerDormant(pane), + qPrintable(QString("%1 is not hidden") + .arg(layer->objectName()))); + auto params = layer->getPlayParameters(); + QVERIFY2(!params || !params->isPlayAudible(), + qPrintable(QString("%1 can be heard") + .arg(layer->objectName()))); + QVERIFY2(!models.count(layer->getModel()), + qPrintable(QString("%1 is still in the play source") + .arg(layer->objectName()))); + auto model = sv::ModelById::get(layer->getModel()); + QVERIFY2(model && model->getSourceModel().isNone(), + qPrintable(QString("%1 still has a source model, so an " + "analyser could claim it") + .arg(layer->objectName()))); + } + } + // What the strip shows, from the regions of its model rather than // from the object that keeps it sv::EventVector stripEvents() { @@ -2382,8 +2468,10 @@ private slots: (singing)->getFrameCount())); } - // Loading over a singing track that is already there: the path - // through teardownSingingTrackAnalyser() that record() does not take + // Loading a singing track over one that is already there. Since + // phase 7a each load is a take of its own (spec 5.3): the first + // track's audio is released, but its pitch and notes are kept as the + // layers of the take that has been put away void reload_singing_track() { makeWindow(FakeAudioIO::Config()); openReference(writeWav(tone(lowHz, 1.0))); @@ -2401,22 +2489,32 @@ private slots: sv::ModelId second = m_window->analyser2()->getMainModelId(); QVERIFY(second != first); + // Two takes, and the first one's audio is gone: only the active + // take has an audio model (spec 6.4) + QCOMPARE(m_window->takes()->getTakeNames(), + QStringList({ "Take 1", "Take 2" })); QVERIFY2(!sv::ModelById::get(first), "the first singing track's model was not released"); - QVERIFY(!sv::ModelById::get(firstPitch)); + QVERIFY2(sv::ModelById::get(firstPitch), + "the first take's pitch track went with its audio"); QCOMPARE(layersOnModel(second), 1); QCOMPARE(m_window->paneStack()->getPaneCount(), panes); + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + auto playing = m_window->playSource()->getModels(); QVERIFY(!playing.count(first)); QVERIFY(!playing.count(firstPitch)); verifyPlaySourceClean(); if (QTest::currentTestFailed()) return; - // audio, pitch track and notes, of the reference and of the track + // audio, pitch track and notes, of the reference and of the take + // that is on show QCOMPARE(int(playing.size()), 6); // A stale id used to keep the end of playback where the longest - // model ever loaded had ended + // model ever loaded had ended, and the pitch track of the take + // that has been put away -- of the 2 s file -- would do the same QVERIFY(m_window->playSource()->getPlayEndFrame() < sv::sv_frame_t(1.2 * rate)); } @@ -3640,6 +3738,526 @@ private slots: QCOMPARE(undoOnce(), QString()); } + // Several takes (spec 5.3). Every take of the session has its three + // layers in pane 0, named after it; only the active take has an audio + // model and the singing analyser, and the rest are hidden and silent + + void first_recording_makes_take_1() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + // No take, nothing in the combo box, and nothing to act on + QCOMPARE(m_window->takes()->getTakeCount(), 0); + QCOMPARE(m_window->takeCombo()->count(), 0); + m_window->doUpdateMenuStates(); + QVERIFY(m_window->newTakeAction()->isEnabled()); + QVERIFY(!m_window->duplicateTakeAction()->isEnabled()); + QVERIFY(!m_window->deleteTakeAction()->isEnabled()); + + take(600); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 1" }); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + + // The layers of the take carry its name, and they are the ones the + // analyser holds + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + TakeLayers::Found found = takeLayers("Take 1"); + QCOMPARE(static_cast(found.pitch), + a2->getLayer(Analyser::PitchTrack)); + QCOMPARE(static_cast(found.notes), + a2->getLayer(Analyser::Notes)); + QCOMPARE(static_cast(found.coverage), + static_cast(stripLayer())); + + QCOMPARE(m_window->takeCombo()->count(), 1); + QCOMPARE(m_window->takeCombo()->currentText(), QString("Take 1")); + m_window->doUpdateMenuStates(); + QVERIFY(m_window->duplicateTakeAction()->isEnabled()); + QVERIFY(m_window->deleteTakeAction()->isEnabled()); + } + + // New Empty Take leaves the take that was on show as it is, and the + // next recording goes into the new one + void new_empty_take_then_record() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + QVERIFY(!first.pitch.empty()); + QVERIFY(!first.notes.empty()); + TakeLayers::Found one = takeLayers("Take 1"); + QVERIFY(one.pitch && one.notes && one.coverage); + + m_window->doNewEmptyTake(); + + // Two takes, the new one active with nothing in it and nothing to + // show it with + QCOMPARE(m_window->takes()->getTakeNames(), + QStringList({ "Take 1", "Take 2" })); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + QVERIFY(!m_window->takes()->haveTake()); + QVERIFY2(!m_window->analyser2(), + "the empty take was given the singing analyser"); + QVERIFY(!m_window->coverageStrip()->isShown()); + QCOMPARE(m_window->takeCombo()->currentText(), QString("Take 2")); + + // The first take's layers are the same objects, with their events, + // and they are put away + QCOMPARE(takeLayers("Take 1").pitch, one.pitch); + QCOMPARE(pitchEvents(one.pitch), first.pitch); + QCOMPARE(noteEvents(one.notes), first.notes); + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + + // Recording into the new take: its own audio file, its own layers + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + QVERIFY(m_window->takes()->getAudioPath() != first.path); + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QVERIFY(ranges[0].start >= sv::sv_frame_t(2.0 * rate)); + + TakeLayers::Found two = takeLayers("Take 2"); + QVERIFY(two.pitch && two.notes && two.coverage); + QVERIFY(two.pitch != one.pitch && two.notes != one.notes); + QCOMPARE(static_cast(two.notes), + m_window->analyser2()->getLayer(Analyser::Notes)); + QVERIFY(!pitchEvents(two.pitch).empty()); + + // The first take is untouched by all of it + QCOMPARE(pitchEvents(one.pitch), first.pitch); + QCOMPARE(noteEvents(one.notes), first.notes); + QCOMPARE(m_window->takes()->getTake(0)->audioPath, first.path); + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + + // Each take has its own strip, and one of them is on show + QCOMPARE(allStripLayersInPane0(), 2); + verifyStripMatchesTake(); + verifyPlaySourceClean(); + } + + // Switching back and forth: the take's audio, coverage, strip, pitch + // and notes come back exactly, the layers are the very same objects + // and nothing is analysed + void switch_between_takes() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + TakeLayers::Found one = takeLayers("Take 1"); + + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot second = snapshotTake(); + TakeLayers::Found two = takeLayers("Take 2"); + QVERIFY(second.path != first.path); + QVERIFY(second.frames > 0 && first.frames > 0); + + // Back to the first take, through the combo box as the user does + m_window->doChooseTakeInCombo(0); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + + verifyTakeMatches(first); + if (QTest::currentTestFailed()) return; + QCOMPARE(takeLayers("Take 1").pitch, one.pitch); + QCOMPARE(takeLayers("Take 1").notes, one.notes); + QCOMPARE(static_cast(one.pitch), + m_window->analyser2()->getLayer(Analyser::PitchTrack)); + verifyTakeIsPutAway("Take 2"); + if (QTest::currentTestFailed()) return; + + // Nothing was analysed, then or when the queued calls ran + QVERIFY(!m_window->analysingRange()); + QVERIFY(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers()); + QCoreApplication::processEvents(); + QVERIFY(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers()); + + // The note tool acts on the pane's topmost note layer: it has to + // be the take that is on show, not the one put away + QCOMPARE(topNoteLayerInPane0(), static_cast(one.notes)); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // The undo history goes with a switch (spec 5.4) + QCOMPARE(undoOnce(), QString()); + + // ... and back to the second + m_window->doChooseTakeInCombo(1); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + verifyTakeMatches(second); + if (QTest::currentTestFailed()) return; + QCOMPARE(takeLayers("Take 2").notes, two.notes); + QCOMPARE(topNoteLayerInPane0(), static_cast(two.notes)); + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->analysingRange()); + verifyStripMatchesTake(); + verifyPlaySourceClean(); + } + + // A take that is not on show is silent and cannot hold playback open + // past the end of the audio that is + void inactive_take_does_not_play() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + // A take that reaches well past the end of the reference + take(1500); + if (QTest::currentTestFailed()) return; + sv::sv_frame_t longest = m_window->takes()->getCoverage().getEndFrame(); + QVERIFY(longest > sv::sv_frame_t(1.2 * rate)); + QVERIFY(m_window->playSource()->getPlayEndFrame() >= longest); + + // ... and a short one beside it + m_window->doNewEmptyTake(); + m_window->seekTo(0); + take(400); + if (QTest::currentTestFailed()) return; + + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + + // Playback ends with the longest of what can be heard: the + // reference and the take on show, not the take put away + sv::sv_frame_t end = m_window->playSource()->getPlayEndFrame(); + QVERIFY2(end < longest, + qPrintable(QString("playback still runs to frame %1, the end " + "of the take that was put away (%2)") + .arg(end).arg(longest))); + QVERIFY(end >= sv::sv_frame_t(1.0 * rate)); + verifyPlaySourceClean(); + } + + // Duplicate Take: a copy of the take, sharing its audio file, and + // recording into the copy leaves the original alone + void duplicate_take_then_record() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + QVERIFY(!first.pitch.empty()); + QVERIFY(!first.notes.empty()); + TakeLayers::Found one = takeLayers("Take 1"); + + m_window->doDuplicateTake(); + + QCOMPARE(m_window->takes()->getTakeNames(), + QStringList({ "Take 1", "Take 2" })); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + + // The same audio file and coverage, and the same events in layers + // of its own + QCOMPARE(m_window->takes()->getAudioPath(), first.path); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), first.coverage); + TakeLayers::Found two = takeLayers("Take 2"); + QVERIFY(two.pitch && two.notes && two.coverage); + QVERIFY(two.pitch != one.pitch && two.notes != one.notes); + QCOMPARE(pitchEvents(two.pitch), first.pitch); + QCOMPARE(noteEvents(two.notes), first.notes); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + + // No audio file was written, and the shared one is not up for + // deletion however thoroughly either take supersedes it + QCOMPARE(m_window->takes()->getWrittenPaths(), + QStringList { first.path }); + + // Recording into the copy: its own file from now on, and the take + // it was copied from is exactly as it was + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->takes()->getAudioPath() != first.path); + QCOMPARE(int(m_window->takes()->getCoverage().getRanges().size()), 2); + QCOMPARE(m_window->takes()->getTake(0)->audioPath, first.path); + QCOMPARE(m_window->takes()->getTake(0)->coverage.getRanges(), + first.coverage); + QCOMPARE(pitchEvents(one.pitch), first.pitch); + QCOMPARE(noteEvents(one.notes), first.notes); + QVERIFY2(m_window->takes()->unusedWrittenFiles().isEmpty(), + "the file the first take still plays was up for deletion"); + QVERIFY(QFileInfo::exists(first.path)); + verifyPlaySourceClean(); + } + + // Delete Take asks first, and deletes the take's layers and nothing + // else: never an audio file + void delete_the_active_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + TakeLayers::Found one = takeLayers("Take 1"); + + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + QString secondPath = m_window->takes()->getAudioPath(); + TakeLayers::Found two = takeLayers("Take 2"); + + // Asked, and answered no: nothing happens + m_window->setDeleteTakeAnswer(false); + m_window->doDeleteTake(); + QCOMPARE(m_window->deleteTakeQuestions(), 1); + QCOMPARE(m_window->takes()->getTakeCount(), 2); + + m_window->setDeleteTakeAnswer(true); + m_window->doDeleteTake(); + QCOMPARE(m_window->deleteTakeQuestions(), 2); + + // The take before it is the active one, as it was + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 1" }); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + verifyTakeMatches(first); + if (QTest::currentTestFailed()) return; + QCOMPARE(takeLayers("Take 1").pitch, one.pitch); + QCOMPARE(static_cast(one.notes), + m_window->analyser2()->getLayer(Analyser::Notes)); + + // The deleted take's layers are gone from the pane and from the + // document, and its audio file is still on disk + QVERIFY(!takeLayers("Take 2").pitch); + QVERIFY(!takeLayers("Take 2").notes); + QVERIFY(!takeLayers("Take 2").coverage); + QVERIFY(!documentHasLayer(two.pitch)); + QVERIFY(!documentHasLayer(two.notes)); + QVERIFY(!documentHasLayer(two.coverage)); + QCOMPARE(noteLayersInPane0(), 2); // the reference's and Take 1's + QVERIFY2(QFileInfo::exists(secondPath), + "deleting a take deleted its audio file"); + verifyStripMatchesTake(); + verifyPlaySourceClean(); + if (QTest::currentTestFailed()) return; + + // And the last take can go too, leaving no take at all + m_window->doDeleteTake(); + QCOMPARE(m_window->takes()->getTakeCount(), 0); + QCOMPARE(m_window->takes()->getActiveIndex(), -1); + QVERIFY(!m_window->analyser2()); + QCOMPARE(allStripLayersInPane0(), 0); + QCOMPARE(noteLayersInPane0(), 1); // the reference's + QCOMPARE(m_window->takeCombo()->count(), 0); + QVERIFY(QFileInfo::exists(first.path)); + + // Recording again makes a take, as the first recording did + m_window->seekTo(0); + take(600); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 3" }); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + verifyStripMatchesTake(); + } + + // Deleting a take that is not the active one: the one on show does not + // move + void delete_an_inactive_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeLayers::Found one = takeLayers("Take 1"); + + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot second = snapshotTake(); + + QVERIFY(m_window->doDeleteTakeAt(0)); + + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 2" }); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + verifyTakeMatches(second); + if (QTest::currentTestFailed()) return; + QVERIFY(!takeLayers("Take 1").pitch); + QVERIFY(!documentHasLayer(one.pitch)); + QCOMPARE(noteLayersInPane0(), 2); + verifyStripMatchesTake(); + verifyPlaySourceClean(); + } + + // Rename Take: the take and its layers, and not the undo history + void rename_take() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + TakeLayers::Found before = takeLayers("Take 1"); + QVERIFY(before.pitch && before.notes && before.coverage); + + m_window->setTakeNameAnswer("Chorus"); + m_window->doRenameTake(); + + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Chorus" }); + QCOMPARE(m_window->takeCombo()->currentText(), QString("Chorus")); + + // The same layers, named after the take again + TakeLayers::Found after = takeLayers("Chorus"); + QCOMPARE(after.pitch, before.pitch); + QCOMPARE(after.notes, before.notes); + QCOMPARE(after.coverage, before.coverage); + QVERIFY(!takeLayers("Take 1").pitch); + QCOMPARE(m_window->coverageStrip()->getTakeName(), QString("Chorus")); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // A rename is not a change to the singing, so the recording can + // still be undone (spec 5.4) + QCOMPARE(undoOnce(), QString("Record Singing")); + QCOMPARE(redoOnce(), QString("Record Singing")); + + // The take's audio is still under its layers after all that. The + // layers are looked up again: an undo of the first recording of a + // take takes them away, and the redo makes them afresh + QVERIFY(m_window->analyser2()); + QCOMPARE(static_cast(takeLayers("Chorus").pitch), + m_window->analyser2()->getLayer(Analyser::PitchTrack)); + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Chorus" }); + } + + // The take operations are not to be had while a take is being + // recorded, and a new take then record is not the same as a switch + void take_actions_disabled_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + + m_window->doUpdateMenuStates(); + QVERIFY(m_window->takeCombo()->isEnabled()); + QVERIFY(m_window->newTakeAction()->isEnabled()); + + startTake(); + if (QTest::currentTestFailed()) return; + m_window->doUpdateMenuStates(); + QVERIFY2(!m_window->takeCombo()->isEnabled(), + "takes could be switched while one was being recorded"); + QVERIFY(!m_window->newTakeAction()->isEnabled()); + QVERIFY(!m_window->duplicateTakeAction()->isEnabled()); + QVERIFY(!m_window->deleteTakeAction()->isEnabled()); + + // And the calls themselves refuse, in case a script reaches them + QVERIFY(!m_window->doSwitchToTake(0)); + m_window->doNewEmptyTake(); + QCOMPARE(m_window->takes()->getTakeCount(), 1); + + stopTake(); + if (QTest::currentTestFailed()) return; + m_window->doUpdateMenuStates(); + QVERIFY(m_window->takeCombo()->isEnabled()); + } + + // A session with two takes saved and opened again. Phase 7b stores + // the takes properly; until then the session says only which audio + // file the active take was in, so that take comes back as a take of + // its own, analysed afresh, and the layers of both saved takes are + // left in the pane, hidden and owned by nobody + void two_takes_survive_a_session_opening() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(700); + if (QTest::currentTestFailed()) return; + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + + QString path = m_window->takes()->getAudioPath(); + auto pitch = pitchEvents(m_window->analyser2()); + QVERIFY(!pitch.empty()); + + QString session = m_dir.filePath("two-takes.ton"); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + // One take, in the audio file the active take was in, and named + // after neither of the takes whose layers the session holds + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 3" }); + // A restored model reports its own file's path, whose drive letter + // Windows may have changed the case of + QCOMPARE(m_window->takes()->getAudioPath().toLower(), path.toLower()); + QVERIFY(m_window->analyser2()); + QCOMPARE(static_cast(takeLayers("Take 3").pitch), + m_window->analyser2()->getLayer(Analyser::PitchTrack)); + + // Analysed afresh rather than mixed up with another take's pitch + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + + // Both saved takes' layers are still there, put away, and they are + // not the ones the take on show is using + QVERIFY(takeLayers("Take 1").pitch && takeLayers("Take 1").notes); + QVERIFY(takeLayers("Take 2").pitch && takeLayers("Take 2").notes); + QVERIFY(takeLayers("Take 3").pitch != takeLayers("Take 1").pitch); + QVERIFY(takeLayers("Take 3").pitch != takeLayers("Take 2").pitch); + verifyTakeIsPutAway("Take 1"); + verifyTakeIsPutAway("Take 2"); + if (QTest::currentTestFailed()) return; + verifyPlaySourceClean(); + } + // The alternate pitch track: the reference pitch track moved by // whole octaves, as a layer of its own diff --git a/main/test/TestSingingTakes.h b/main/test/TestSingingTakes.h index fe3031f9..a5f1afea 100644 --- a/main/test/TestSingingTakes.h +++ b/main/test/TestSingingTakes.h @@ -124,6 +124,198 @@ private slots: QVERIFY(takes.getSupersededPaths().isEmpty()); } + // Several takes (spec 5.3): a list, with one of them the active one, + // which is the take every single-take call is about + + void takes_are_a_list() { + SingingTakes takes; + QCOMPARE(takes.getTakeCount(), 0); + QCOMPARE(takes.getActiveIndex(), -1); + QCOMPARE(takes.getActiveName(), QString()); + QVERIFY(takes.getTakeNames().isEmpty()); + QVERIFY(!takes.getTake(0)); + + // An empty take: there, active, and with nothing in it + QCOMPARE(takes.addTake(), QString("Take 1")); + QCOMPARE(takes.getTakeCount(), 1); + QCOMPARE(takes.getActiveIndex(), 0); + QVERIFY(!takes.haveTake()); + QVERIFY(takes.getCoverage().isEmpty()); + + takes.setWholeFileTake("/somewhere/one.wav", 1000); + QVERIFY(takes.haveTake()); + + // A second take of its own: the first is left as it is + QCOMPARE(takes.addTake(), QString("Take 2")); + QCOMPARE(takes.getActiveIndex(), 1); + QVERIFY(!takes.haveTake()); + QVERIFY(takes.getCoverage().isEmpty()); + QCOMPARE(takes.getTakeNames(), QStringList({ "Take 1", "Take 2" })); + + takes.setWholeFileTake("/somewhere/two.wav", 2000); + + // Switching is the active index, and nothing else + QVERIFY(takes.setActiveIndex(0)); + QCOMPARE(takes.getAudioPath(), QString("/somewhere/one.wav")); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 1000)); + QVERIFY(takes.setActiveIndex(1)); + QCOMPARE(takes.getAudioPath(), QString("/somewhere/two.wav")); + QCOMPARE(takes.getCoverage().getRanges()[0], Coverage::Range(0, 2000)); + + QVERIFY(!takes.setActiveIndex(2)); + QVERIFY(!takes.setActiveIndex(-1)); + QCOMPARE(takes.getActiveIndex(), 1); + + QCOMPARE(takes.indexOf("Take 1"), 0); + QCOMPARE(takes.indexOf("Take 3"), -1); + + takes.clear(); + QCOMPARE(takes.getTakeCount(), 0); + QCOMPARE(takes.getActiveIndex(), -1); + } + + // The first recording of a session records into a take of its own, + // without anyone having to ask for one + void first_recording_makes_a_take() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QCOMPARE(takes.getTakeCount(), 1); + QCOMPARE(takes.getActiveName(), QString("Take 1")); + QVERIFY(takes.haveTake()); + } + + // A name is never used twice in a session, however many takes have + // been deleted since: the layers of a take that has gone may still be + // in the document, under the name it had + void take_names_are_not_reused() { + SingingTakes takes; + QCOMPARE(takes.addTake(), QString("Take 1")); + QCOMPARE(takes.addTake(), QString("Take 2")); + QVERIFY(takes.removeTake(1)); + QCOMPARE(takes.addTake(), QString("Take 3")); + + // A name of the caller's own, and one that is taken already + QCOMPARE(takes.addTake("Chorus"), QString("Chorus")); + QCOMPARE(takes.addTake("Chorus"), QString("Take 4")); + + takes.clear(); + QCOMPARE(takes.addTake(), QString("Take 1")); + } + + void rename_a_take() { + SingingTakes takes; + takes.addTake(); + takes.addTake(); + + QVERIFY(takes.renameTake(0, "Chorus")); + QCOMPARE(takes.getTakeNames(), QStringList({ "Chorus", "Take 2" })); + QCOMPARE(takes.indexOf("Chorus"), 0); + + // The name it has, which is not a change at all + QVERIFY(takes.renameTake(0, "Chorus")); + + // Empty, whitespace only, another take's, and no such take + QVERIFY(!takes.renameTake(0, "")); + QVERIFY(!takes.renameTake(0, " ")); + QVERIFY(!takes.renameTake(0, "Take 2")); + QVERIFY(!takes.renameTake(2, "Verse")); + QCOMPARE(takes.getTakeNames(), QStringList({ "Chorus", "Take 2" })); + + QVERIFY(takes.renameTake(1, " Verse ")); + QCOMPARE(takes.getTakeNames(), QStringList({ "Chorus", "Verse" })); + } + + // Deleting the active take leaves a neighbour active, and deleting the + // last of them leaves no take at all + void remove_take_activates_a_neighbour() { + SingingTakes takes; + takes.addTake(); + takes.addTake(); + takes.addTake(); + QCOMPARE(takes.getActiveIndex(), 2); + + QVERIFY(takes.removeTake(2)); + QCOMPARE(takes.getActiveIndex(), 1); + + // One before the active take: the same take stays active + QVERIFY(takes.setActiveIndex(1)); + QVERIFY(takes.removeTake(0)); + QCOMPARE(takes.getActiveIndex(), 0); + QCOMPARE(takes.getTakeNames(), QStringList { "Take 2" }); + + QVERIFY(takes.removeTake(0)); + QCOMPARE(takes.getTakeCount(), 0); + QCOMPARE(takes.getActiveIndex(), -1); + QVERIFY(!takes.haveTake()); + + QVERIFY(!takes.removeTake(0)); + } + + // A duplicate shares the audio file of the take it was made from + // until one of them is recorded into or erased from, which writes a + // new file anyway (spec 5.4). So a file is in use while any take + // refers to it + void duplicate_shares_the_audio_file() { + SingingTakes takes; + QString recording = writeRecording(1000, 0.5f); + QVERIFY(takes.spliceRecording(recording, 0, 0, -1, + takeDirectory()).isEmpty()); + QString shared = takes.getAudioPath(); + Coverage coverage = takes.getCoverage(); + + QCOMPARE(takes.duplicateActiveTake(), QString("Take 2")); + QCOMPARE(takes.getTakeCount(), 2); + QCOMPARE(takes.getActiveIndex(), 1); + QCOMPARE(takes.getAudioPath(), shared); + QVERIFY(takes.getCoverage() == coverage); + + // No file was written and none superseded + QCOMPARE(takes.getWrittenPaths(), QStringList { shared }); + QVERIFY(takes.getSupersededPaths().isEmpty()); + QVERIFY(takes.unusedWrittenFiles().isEmpty()); + + // Recording into the copy writes a file of its own; the take it + // was copied from is untouched, and still uses the shared file + QVERIFY(takes.spliceRecording(recording, 0, 5000, -1, + takeDirectory()).isEmpty()); + QVERIFY(takes.getAudioPath() != shared); + QCOMPARE(takes.getTake(0)->audioPath, shared); + QVERIFY(takes.getTake(0)->coverage == coverage); + + // The shared file has been superseded for the copy, but the first + // take still plays it, so it is not ours to delete + QVERIFY(takes.getSupersededPaths().contains(shared)); + QVERIFY2(takes.unusedWrittenFiles().isEmpty(), + "a file another take is using was up for deletion"); + + // ... and once that take has gone, it is + QVERIFY(takes.removeTake(0)); + QCOMPARE(takes.unusedWrittenFiles(), QStringList { shared }); + } + + // A take with no audio yet: its name is there, and the calls about the + // active take say there is nothing + void a_take_with_no_audio() { + SingingTakes takes; + takes.addTake(); + QVERIFY(!takes.haveTake()); + QCOMPARE(takes.getAudioPath(), QString()); + QVERIFY(takes.getCoverage().isEmpty()); + QVERIFY(!takes.coversPosition(0)); + QVERIFY(!takes.shouldConfirmRecordingAt(0)); + QVERIFY(takes.duplicateActiveTake() != ""); + QVERIFY(!takes.haveTake()); + + // Nothing to erase, and no file written trying + Coverage::Ranges erased { Coverage::Range(1, 2) }; + QCOMPARE(takes.eraseRanges(Coverage::Ranges { Coverage::Range(0, 100) }, + takeDirectory(), &erased), QString()); + QVERIFY(erased.empty()); + QVERIFY(takes.getWrittenPaths().isEmpty()); + } + // Every audio file of a take gets a name of its own, in the directory // asked for, and never one that is taken: TakeAudio refuses to write // over a file, so a name that collides would lose a recording diff --git a/meson.build b/meson.build index 2008f31d..0579baf1 100644 --- a/meson.build +++ b/meson.build @@ -1110,6 +1110,7 @@ tony_app_files = [ 'main/NetworkPermissionTester.cpp', 'main/PaneUtils.cpp', 'main/TakeCommands.cpp', + 'main/TakeLayers.cpp', ] tony_core_moc_files = qt.preprocess( From bef4aa4b10ad18ce3fb75e7bb36c487a416fe83f Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 17:58:31 +0300 Subject: [PATCH 048/275] build: svgui pin, the note tools leave hidden takes alone Co-Authored-By: Claude Fable 5.1 --- repoint-lock.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoint-lock.json b/repoint-lock.json index 9379cf98..4d990697 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "008441c137fd233d5d9ca4ea296acb7c8dde8dbb" + "pin": "573c939b9d0b1db3b756c6a133f64b2cba0a16a7" }, "svapp": { "pin": "8dfa3bc8d5ede210210ebba003c736c46e86405a" From f4369f07a66c1f8304b8e949056784a5626169b0 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 18:54:28 +0300 Subject: [PATCH 049/275] feat: the takes of a session are stored in the .ton A session file carries one element of Tony's own, written by MainWindow::toXml() and read back by a pass of its own over the same file (TakesFile, in tony_core): On load, restoreTakes() rebuilds SingingTakes from it -- names, audio paths, each take's coverage from its own coverage strip -- and shows the active take by the same path a switch uses: its audio opened from the element, its layers claimed by name, no analysis anywhere. The audio model that Document::toXml() wrote for the active take's waveform layer is dropped silently before that; a session with no element at all opens without its singing track, and the layers that showed one go with it (spec 3, "Old sessions"). The old scan for a second wave model and the "no live source model" guess in adoptTakeLayers() are gone with it. A save now waits for the analysis of a recorded range to be merged, so that no session holds the pre-merge events or the run's two temporary models, and it protects the audio file of every take it names, not only the active one's. Switching take marks the session modified. TakeAudio::checkOutPath() compared two QFileInfos, which are equal when neither file exists, so recording into a take whose audio had gone missing was refused with "file exists already" instead of naming the file that does not. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 435 +++++++++++++++++++--------- main/MainWindow.h | 56 +++- main/SingingTakes.cpp | 16 +- main/SingingTakes.h | 4 + main/TakeAudio.cpp | 16 +- main/TakesFile.cpp | 120 ++++++++ main/TakesFile.h | 81 ++++++ main/test/TestRecordWorkflow.h | 501 +++++++++++++++++++++++++++++++-- main/test/TestTakeAudio.h | 8 +- main/test/TestTakesFile.h | 179 ++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 12 files changed, 1256 insertions(+), 169 deletions(-) create mode 100644 main/TakesFile.cpp create mode 100644 main/TakesFile.h create mode 100644 main/test/TestTakesFile.h diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index caf17acb..55f86e33 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -22,6 +22,7 @@ #include "PaneUtils.h" #include "TakeEvents.h" #include "TakeLayers.h" +#include "TakesFile.h" #include "framework/Document.h" #include "framework/VersionTester.h" @@ -33,6 +34,7 @@ #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" +#include "data/model/RegionModel.h" #include "layer/FlexiNoteLayer.h" #include "view/ViewManager.h" #include "base/Preferences.h" @@ -102,6 +104,8 @@ #include #include #include +#include +#include #include #include @@ -148,6 +152,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_renameTakeAction(nullptr), m_deleteTakeAction(nullptr), m_updatingTakeCombo(false), + m_restoringSession(false), m_eraseSingingAction(nullptr), m_selectRecordingAction(nullptr), m_openTakeCommand(nullptr), @@ -2709,65 +2714,27 @@ MainWindow::analyseNewSingingModel() void MainWindow::analyseRestoredSingingModel() { - // The queued call from modelAdded(), which is how the singing track - // of a session being restored arrives: loadSingingTrack() and the - // take's own paths have set their analyser up synchronously by now - // and left nothing pending. A take that has been recorded into has - // pitch and notes layers in the pane that belong to no analyser, and - // the one about to be made is to claim them rather than analyse all - // of the file again + // The queued call from modelAdded(): an audio model has been added to + // the document by a route that has not set a singing analyser up for + // it. The routes that do -- loadSingingTrack() and the take's own + // openTakeAudioFile() -- have cleared the pending id by now, and a + // session being restored is dealt with by restoreTakes(), which drops + // the audio model the document carried and opens each take's audio + // itself. So this is the fallback, and what it finds is a take of its + // own, analysed in full + if (m_restoringSession) return; if (m_pendingSingingModelId.isNone()) return; // Which take this is has to be settled first: its layers are found by // its name, so it must have one before they are looked for if (m_takes->getActiveIndex() < 0) { - m_takes->addTake(restoredTakeName()); + m_takes->addTake(); } adoptTakeLayers(m_pendingSingingModelId); analyseNewSingingModel(); } -QString -MainWindow::restoredTakeName() -{ - // The take that the singing track of a session being restored belongs - // to. Its pitch, notes and coverage are in the pane already, named - // after it, so where the pane holds one take's layers this is that - // take and it claims them. - // - // Where it holds several takes' layers, nothing in the session file - // says which of them the audio belongs to -- phase 7b writes that - // down -- so this is a take of its own, with a name none of them uses - // and an analysis of its own, and they are left in the pane hidden, - // silent and owned by nobody. - Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; - if (!pane) return ""; - - QStringList names; - - for (int i = 0; i < pane->getLayerCount(); ++i) { - QString name; - TakeLayers::Kind kind; - if (!TakeLayers::parse(pane->getLayer(i)->objectName(), name, kind)) { - continue; - } - if (!names.contains(name)) names.push_back(name); - m_takes->reserveTakeName(name); - } - - if (names.size() == 1) return names[0]; - - if (names.size() > 1) { - cerr << "MainWindow::restoredTakeName: the session holds the layers of " - << names.size() << " takes and does not say which of them the " - << "singing track belongs to: it becomes a take of its own" - << endl; - } - - return ""; -} - void MainWindow::openBackgroundMusic() { @@ -4363,46 +4330,13 @@ MainWindow::adoptTakeLayers(ModelId audio) // By name, which is what says whose a take's layers are (spec 6.4). // The layers of the takes that are put away have no live source model - // either, so nothing but the name can tell them apart + // either, so nothing but the name can tell them apart. (Before 7b + // there was a fallback for a session whose takes had no names of their + // own; a session that does not name its takes now opens without them.) TakeLayers::Found found = TakeLayers::find(pane, takeName); TimeValueLayer *pitch = found.pitch; FlexiNoteLayer *notes = found.notes; - if (!pitch || !notes) { - - // A session saved before takes had names of their own: its take's - // layers are the ones in the pane that belong to no analyser - for (int i = 0; i < pane->getLayerCount(); ++i) { - - Layer *layer = pane->getLayer(i); - - // Ours, and not a take's: the live dots of a take being - // recorded, and the reference's pitch track moved by octaves - if (layer == m_realtimePitchLayer) continue; - if (m_alternatePitch && - layer == m_alternatePitch->getLayer()) continue; - - // Another take's, named as such: never ours to claim - QString otherName; - TakeLayers::Kind kind; - if (TakeLayers::parse(layer->objectName(), otherName, kind) && - otherName != takeName) { - continue; - } - - auto model = ModelById::get(layer->getModel()); - if (!model) continue; - - // A layer whose source model is still there has an analyser of - // its own: the reference's pitch, notes and candidates, or a - // take whose audio has not been swapped since it was analysed - if (ModelById::get(model->getSourceModel())) continue; - - if (!pitch) pitch = qobject_cast(layer); - if (!notes) notes = qobject_cast(layer); - } - } - // Half a pair is no use: the analyser claims both or neither if (!pitch || !notes) return false; @@ -4628,17 +4562,252 @@ MainWindow::eraseTakeEvents(const Coverage::Ranges &erased, bool MainWindow::saveSessionFile(QString path) { + // Not while the analysis of a recorded range runs: the session would + // hold neither the state before the merge nor the state after it + if (!waitForRangedAnalysis()) return false; + bool saved = MainWindowBase::saveSessionFile(path); - // The .ton that has just been written names the take's audio file. - // Recording into the take again writes another file and supersedes - // that one, but the saved session still needs it, so it is not ours to - // delete when the session closes - if (saved && m_takes) m_takes->protectPath(m_takes->getAudioPath()); + // The .ton that has just been written names the audio file of every + // take of the session. Recording into a take again writes another file + // and supersedes that one, but the saved session still needs it, so it + // is not ours to delete when the session closes + if (saved && m_takes) { + for (const SingingTakes::Take &take : m_takes->getTakes()) { + m_takes->protectPath(take.audioPath); + } + } return saved; } +void +MainWindow::toXml(QTextStream &out, bool asTemplate) +{ + // A template holds no session content, so it holds no takes either + if (asTemplate) { + MainWindowBase::toXml(out, asTemplate); + return; + } + + // The takes of the session are one element of Tony's own inside the + // document (spec 6.4), which SVFileReader passes over with a + // warning on the terminal and nothing else. The base class writes the + // document from to in one call and has no hook in the + // middle, so it writes into a string here and the element goes in + // before the closing tag. A hook in the fork would save the copy. + QString document; + { + QTextStream buffer(&document); + MainWindowBase::toXml(buffer, asTemplate); + buffer.flush(); + } + + int closing = document.lastIndexOf(""); + + if (closing < 0) { + // Not a document we know where to write in: better saved without + // its takes than not saved at all + cerr << "MainWindow::toXml: no to write the takes before" << endl; + out << document; + return; + } + + out << document.left(closing) + << TakesFile::toXml(*m_takes) + << document.mid(closing); +} + +MainWindow::FileOpenStatus +MainWindow::openSession(FileSource source) +{ + // m_restoringSession: modelAdded() queues a call that would make a take + // of the audio model the document carries, and restoreTakes() is what + // deals with that model instead + m_restoringSession = true; + FileOpenStatus status = MainWindowBase::openSession(source); + m_restoringSession = false; + + if (status != FileOpenSucceeded) return status; + + // The document is in the panes now, with every take's layers in pane 0, + // and nothing has been analysed + restoreTakes(source.getLocalFilename()); + + return status; +} + +void +MainWindow::restoreTakes(QString sessionPath) +{ + if (!m_document) return; + + TakesFile::Takes stored = TakesFile::read(sessionPath); + + // Opening a session is not a change to it, whatever is done below + bool wasModified = m_documentModified; + + // The audio model of the active take, which the document carried + // because the waveform layer showing it is in pane 0: of no use here, + // and not to be mistaken for a singing track of its own + dropRestoredSingingTrack(!stored.found); + + if (!stored.found) { + // A session saved before the takes were stored (spec 3, "Old + // sessions"): it opens without its singing track, and the layers + // that showed one have gone with it + cerr << "MainWindow::restoreTakes: the session has no takes element: " + << "it opens without a singing track" << endl; + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + if (!wasModified) documentRestored(); + return; + } + + Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; + + // Every name the session used is reserved before any take is made, so + // that the numbering carries on from it and no new take can be given + // the name of a take of the session -- or of one whose layers are in + // the pane although no take claims them any more + for (const TakesFile::Take &take : stored.takes) { + m_takes->reserveTakeName(take.name); + } + if (pane) { + for (int i = 0; i < pane->getLayerCount(); ++i) { + QString name; + TakeLayers::Kind kind; + if (TakeLayers::parse(pane->getLayer(i)->objectName(), + name, kind)) { + m_takes->reserveTakeName(name); + } + } + } + + for (const TakesFile::Take &take : stored.takes) { + + QString name = m_takes->addTake(take.name); + + // The coverage of a take is stored in the regions of its own + // coverage strip, which the document has put back into the pane + // (5a); a take with no recording in it yet has neither + Coverage coverage; + if (pane) { + TakeLayers::Found found = TakeLayers::find(pane, name); + if (found.coverage) { + if (auto model = ModelById::getAs + (found.coverage->getModel())) { + coverage = Coverage::fromEvents(model->getAllEvents()); + } + } + } + + // restoreTake() and not setTake(): nothing here supersedes a file + m_takes->restoreTake(take.audioPath, coverage); + } + + int active = m_takes->indexOf(stored.active); + if (active < 0 && m_takes->getTakeCount() > 0) active = 0; + + cerr << "MainWindow::restoreTakes: " << m_takes->getTakeCount() + << " take(s) restored, active is \"" << stored.active << "\"" << endl; + + if (active >= 0) { + // The same path a switch uses: the take's audio under the take's + // own layers, an analyser that claims them with no analysis, its + // coverage strip, and every other take put away + m_takes->setActiveIndex(active); + activateTake(); + } else { + putOtherTakeLayersAway(); + } + + updateTakeCombo(); + updateLayerStatuses(); + updateMenuStates(); + + // Whatever was done above, the session is as it was saved + if (!wasModified) documentRestored(); +} + +void +MainWindow::dropRestoredSingingTrack(bool withTakeLayers) +{ + // Nothing is to make a take of the audio model the session carried: + // restoreTakes() opens each take's audio from the path in + m_pendingSingingModelId = {}; + + if (!m_document) return; + + // The reference is the main model; every other audio model in a + // restored document belongs to a singing track + ModelId mainId = getMainModelId(); + std::vector audio; + for (ModelId id : m_document->getModels()) { + if (id == mainId) continue; + if (ModelById::isa(id)) audio.push_back(id); + } + if (audio.empty()) return; + + auto isAudio = [&audio](ModelId id) { + return std::find(audio.begin(), audio.end(), id) != audio.end(); + }; + + // Collected before anything is deleted: releasing a model clears the + // source of what was derived from it + std::vector going; + + for (Layer *layer : m_document->getLayers()) { + + ModelId modelId = layer->getModel(); + + if (isAudio(modelId)) { + going.push_back(layer); + continue; + } + + if (!withTakeLayers) continue; + + // The pitch and notes of that singing track, whether they are + // named after a take or (in a session from before this feature) + // simply derived from its audio + auto model = ModelById::get(modelId); + if (model && isAudio(model->getSourceModel())) { + going.push_back(layer); + continue; + } + + QString takeName; + TakeLayers::Kind kind; + if (TakeLayers::parse(layer->objectName(), takeName, kind)) { + going.push_back(layer); + } + } + + cerr << "MainWindow::dropRestoredSingingTrack: dropping " << going.size() + << " restored layer(s) of the singing track the document carried" + << endl; + + for (Layer *layer : going) dropLayerSilently(layer); +} + +void +MainWindow::dropLayerSilently(Layer *layer) +{ + if (!layer || !m_document) return; + + // The play source takes in the model of every layer that is in a view, + // and a forced delete does not fire layerInAView() to take it out again + if (m_playSource && !layer->getModel().isNone()) { + m_playSource->removeModel(layer->getModel()); + } + + // deleteLayer(force) and nothing else: no undo command, out of every + // view it is in, and the model goes with it if nothing else uses it + m_document->deleteLayer(layer, true); +} + void MainWindow::addTakeCommand(SingingTakeCommand *command) { @@ -5086,6 +5255,10 @@ MainWindow::switchToTake(int index) updateLayerStatuses(); updateMenuStates(); + // Which take is the active one is stored in the session (spec 6.4), so + // a switch is a change to it + documentModified(); + emit activity(tr("Switched to the take \"%1\"") .arg(m_takes->getActiveName())); @@ -5580,6 +5753,48 @@ MainWindow::waitForInitialAnalysis() } } +bool +MainWindow::waitForRangedAnalysis() +{ + // A session must not be saved in the middle of the analysis of a + // recorded range (spec 6.3): the take's pitch and notes still hold the + // state before the merge, and the two models the run works in are in + // the document, in no pane, so both would be written. The run takes a + // fraction of the recording it follows -- a second or so -- and its + // merge is driven by the event loop, so it is waited for here rather + // than with the dialog waitForInitialAnalysis() puts up for the + // reference's first analysis, which can take as long as the song. + if (!m_analyser2 || !m_analyser2->isAnalysingRange()) return true; + + cerr << "MainWindow::waitForRangedAnalysis: waiting for the analysis of [" + << m_takeAnalysisRange.start << "," << m_takeAnalysisRange.end + << ") to be merged" << endl; + + QEventLoop loop; + + QTimer poll; + connect(&poll, &QTimer::timeout, &loop, [this, &loop]() { + if (!m_analyser2 || !m_analyser2->isAnalysingRange()) loop.quit(); + }); + poll.start(10); + + // A run that never finishes must not hold the save for ever + QTimer::singleShot(30000, &loop, &QEventLoop::quit); + loop.exec(); + + if (m_analyser2 && m_analyser2->isAnalysingRange()) { + // Abandoning it loses the analysis of that range, which the user + // can ask for again with Analyse Now; writing its temporary models + // into the session would leave the session itself wrong + cerr << "MainWindow::waitForRangedAnalysis: the analysis has not " + << "finished; abandoning it so that the session can be saved" + << endl; + closeOpenTakeCommand(true); + } + + return true; +} + void MainWindow::saveSession() { @@ -6773,44 +6988,10 @@ MainWindow::analyseNewMainModel() m_analyser->setAudible(Analyser::Notes, false); } - // Session restore: if the loaded session contained a second audio model - // (i.e. a previously loaded singing track), set up the secondary analyser - // for it now. We scan all document models for a WaveFileModel that is - // not the main model and not already being tracked as a singing model. - // We only do this if we don't already have a secondary analyser (it may - // have been set up already e.g. via modelAdded() during session load). - // Skip the scan while a singing model is pending: openAudio() emits - // audioFileLoaded() for CreateAdditionalModel too, and loadSingingTrack() - // is about to set up that model itself. A second, queued setup would - // tear down m_analyser2's layers — the only references to the singing - // model — releasing it before the re-setup. - if (!m_analyser2 && m_document && m_pendingSingingModelId.isNone()) { - ModelId mainId = getMainModelId(); - ModelId foundSinging; - for (ModelId mid : m_document->getModels()) { - if (mid == mainId) continue; - if (mid == m_realtimePitchModelId) continue; - if (mid == m_backgroundMusicModelId) continue; - if (ModelById::isa(mid)) { - foundSinging = mid; - break; - } - } - if (!foundSinging.isNone()) { - cerr << "analyseNewMainModel: found existing singing track model " - << foundSinging << " in session, setting up secondary analyser" << endl; - // Defer so that the primary analyser's layers are fully in place - // before the secondary analyser tries to share the same pane. - QTimer::singleShot(0, this, [this, foundSinging]() { - // A take recorded into is saved with pitch and notes - // layers that belong to no analyser (the model they were - // derived from went with the swap that put this audio - // under them): they are this analyser's to claim - adoptTakeLayers(foundSinging); - setupSingingTrackAnalyser(foundSinging); - }); - } - } + // A session used to be searched here for a second WaveFileModel, which + // was then set up as the singing track. The takes of a session come + // from its element now (spec 7), and restoreTakes() opens the + // audio of the active take itself, so there is nothing to look for. updateLayerStatuses(); documentRestored(); diff --git a/main/MainWindow.h b/main/MainWindow.h index 4cea7718..c28f3996 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -66,10 +66,20 @@ class MainWindow : public sv::MainWindowBase */ bool applyTakeState(SingingTakeCommand *command, const TakeState &state); - // A session that has been saved names the take's audio file as it was - // at the time, so that file must outlive every other reference to it + // A session that has been saved names the audio file of every take as + // it was at the time, so those files must outlive every other + // reference to them. This is also where a save waits for the analysis + // of a recorded range, so that no session holds the state half way + // through one (see waitForRangedAnalysis()) bool saveSessionFile(QString path) override; + // The takes of the session are written into the document as one + // element of Tony's own (spec 6.4), which the base class knows nothing + // of; and after the base class has read a session back, the same + // element is read from the file and the takes put back from it + void toXml(QTextStream &out, bool asTemplate) override; + FileOpenStatus openSession(sv::FileSource source) override; + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -372,11 +382,36 @@ protected slots: // which case the user has been told bool activateTake(); - // The name for the take that the singing track of a session being - // restored belongs to: the name its layers in the pane are under, or - // "" where the session does not say which take that is. Reserves the - // names of the takes whose layers are left in the pane - QString restoredTakeName(); + // The takes of a session that has just been read back: the + // element of the file says which takes there are, what their audio + // files are and which of them was on show, and the layers the document + // restored say what is in them (spec 6.4). Each take's coverage comes + // from its own coverage strip; the active take is shown by the same + // path a switch uses, and nothing is analysed. A session with no + // element opens without a singing track at all (spec 3). + void restoreTakes(QString sessionPath); + + // Drop the audio model of the singing track that a session restored: + // Document::toXml() writes it because the active take's waveform layer + // is in pane 0, and it is of no use here -- the take's audio is opened + // from the path in , as it is for a switch. Silently: no undo + // entry, no pane, nothing left in the play source, and + // m_pendingSingingModelId cleared so that nothing makes a take of it. + // + // withTakeLayers: its pitch, notes and coverage layers as well, for a + // session that has no element -- it opens without its singing + // track, and nothing of one is left in the pane to be mistaken for a + // take. + void dropRestoredSingingTrack(bool withTakeLayers); + + // Take a layer and its model out of the document with no trace: no + // undo entry, and nothing left in the play source + void dropLayerSilently(sv::Layer *layer); + + // Set while a session is being read, so that the queued call which + // makes a take of a newly added audio model (modelAdded()) leaves the + // restored one alone: restoreTakes() deals with it + bool m_restoringSession; // Name the layers the singing analyser holds after the active take, // which is what says whose they are (spec 6.4) @@ -778,6 +813,13 @@ protected slots: bool checkSaveModified(); bool waitForInitialAnalysis(); + // A session must not be saved in the middle of the analysis of a + // recorded range: the take's pitch and notes still hold the state + // before the merge, and the two models the run works in are in the + // document. Waits for the merge, as waitForInitialAnalysis() waits + // for the reference's first analysis + bool waitForRangedAnalysis(); + virtual void updateVisibleRangeDisplay(sv::Pane *p) const; virtual void updatePositionStatusDisplays() const; diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 9dd918e6..32f7edbe 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -188,7 +189,20 @@ SingingTakes::renameTake(int index, QString name) void SingingTakes::reserveTakeName(QString name) { - if (name != "" && !m_reserved.contains(name)) m_reserved.push_back(name); + if (name == "") return; + + if (!m_reserved.contains(name)) m_reserved.push_back(name); + + // A session that had a "Take 7" in it goes on at "Take 8", however + // many of the takes before it have been deleted since: the default + // names carry on where the session left off rather than starting again + // at the first number no take happens to be using + static const QRegularExpression pattern("^Take (\\d+)$"); + QRegularExpressionMatch match = pattern.match(name); + if (match.hasMatch()) { + int number = match.captured(1).toInt(); + if (number > m_named) m_named = number; + } } bool diff --git a/main/SingingTakes.h b/main/SingingTakes.h index ee5d949a..56fe9c31 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -117,6 +117,10 @@ class SingingTakes : public QObject * no take of the session has it: the layers of a take that a session * held and that this session has not taken up are in the document * under it. A name asked for by name is still given. + * + * A name of the "Take N" form also carries the numbering on: after + * "Take 7" has been reserved the next take named by default is "Take + * 8", whatever has happened to the takes before it. */ void reserveTakeName(QString name); diff --git a/main/TakeAudio.cpp b/main/TakeAudio.cpp index 8c885b42..5dd4041d 100644 --- a/main/TakeAudio.cpp +++ b/main/TakeAudio.cpp @@ -168,11 +168,23 @@ QString write(WavFileReader *old, sv_samplerate_t rate, int channels, QString checkOutPath(QString outPath, QString oldPath) { if (outPath == "") return tr("No file name given to write to"); - if (QFileInfo(outPath) == QFileInfo(oldPath) || - QFileInfo::exists(outPath)) { + + // Not QFileInfo == QFileInfo: that compares canonical paths, which are + // both empty for two files that do not exist, so a take whose audio + // file has gone missing was refused with this message instead of the + // one that says what is really wrong + if (oldPath != "" && + QFileInfo(outPath).absoluteFilePath() == + QFileInfo(oldPath).absoluteFilePath()) { + return tr("File \"%1\" is the one being read from, and is not to be " + "overwritten").arg(outPath); + } + + if (QFileInfo::exists(outPath)) { return tr("File \"%1\" exists already, and is not to be overwritten") .arg(outPath); } + return ""; } diff --git a/main/TakesFile.cpp b/main/TakesFile.cpp new file mode 100644 index 00000000..7b377cdf --- /dev/null +++ b/main/TakesFile.cpp @@ -0,0 +1,120 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakesFile.h" + +#include "SingingTakes.h" + +#include "base/XmlExportable.h" +#include "data/fileio/BZipFileDevice.h" + +#include +#include + +using namespace sv; + +QString +TakesFile::toXml(const SingingTakes &takes, QString indent) +{ + QString xml; + + xml += QString("%1\n") + .arg(indent) + .arg(XmlExportable::encodeEntities(takes.getActiveName())); + + for (const SingingTakes::Take &take : takes.getTakes()) { + xml += QString("%1 \n") + .arg(indent) + .arg(XmlExportable::encodeEntities(take.name)) + .arg(XmlExportable::encodeEntities(take.audioPath)); + } + + xml += QString("%1\n").arg(indent); + + return xml; +} + +TakesFile::Takes +TakesFile::read(QString sessionPath) +{ + Takes takes; + if (sessionPath == "") return takes; + + // A session file is bzip2, but the session reader takes a plain XML + // one as well, so the first bytes say which this is rather than the + // extension + bool compressed = false; + { + QFile probe(sessionPath); + if (!probe.open(QIODevice::ReadOnly)) return takes; + compressed = (probe.read(3) == QByteArray("BZh")); + } + + if (compressed) { + BZipFileDevice file(sessionPath); + if (!file.open(QIODevice::ReadOnly)) return takes; + QXmlStreamReader reader(&file); + takes = readFrom(reader); + file.close(); + } else { + QFile file(sessionPath); + if (!file.open(QIODevice::ReadOnly)) return takes; + QXmlStreamReader reader(&file); + takes = readFrom(reader); + } + + return takes; +} + +TakesFile::Takes +TakesFile::readFrom(QXmlStreamReader &reader) +{ + Takes takes; + bool inTakes = false; + + // Everything of the document but our own element is passed over: it + // has been read once already, by the session reader + while (!reader.atEnd()) { + + switch (reader.readNext()) { + + case QXmlStreamReader::StartElement: + { + QString name = reader.name().toString().toLower(); + + if (name == "takes") { + inTakes = true; + takes.found = true; + takes.active = reader.attributes().value("active").toString(); + } else if (inTakes && name == "take") { + Take take; + take.name = reader.attributes().value("name").toString(); + take.audioPath = reader.attributes().value("audio").toString(); + // A take is its name: one without a name is no take + if (take.name != "") takes.takes.push_back(take); + } + break; + } + + case QXmlStreamReader::EndElement: + if (reader.name().toString().toLower() == "takes") inTakes = false; + break; + + default: + break; + } + } + + return takes; +} diff --git a/main/TakesFile.h b/main/TakesFile.h new file mode 100644 index 00000000..265ae7ff --- /dev/null +++ b/main/TakesFile.h @@ -0,0 +1,81 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKES_FILE_H +#define TONY_TAKES_FILE_H + +#include + +#include + +class SingingTakes; +class QXmlStreamReader; + +/** + * The takes of a session in the session file: the one element of Tony's + * own that a `.ton` carries (spec 6.4). + * + * + * + * + * + * + * Everything else about a take -- its pitch, its notes and the coverage + * strip that holds its coverage -- is in the document as ordinary layers, + * named after the take (TakeLayers), and comes back with it. This element + * says which takes there are, which of them was on show, and where each + * one's audio is; a take with no recording in it yet has an empty audio + * path. + * + * `SVFileReader` does not know the element and only warns about it, so it + * is written inside the `` document and read by a pass of Tony's own + * over the same file afterwards. The element being absent is what tells a + * session saved before this feature from one saved with no takes in it. + */ +class TakesFile +{ +public: + struct Take { + QString name; + QString audioPath; // "" for a take with no recording in it yet + }; + + struct Takes { + std::vector takes; + + /// The name of the take that was on show + QString active; + + /// The file had a `` element at all + bool found = false; + }; + + /** + * The element for the takes of this session, indented with indent and + * ending in a newline. Names and paths are escaped. + */ + static QString toXml(const SingingTakes &takes, QString indent = " "); + + /** + * Read the element from a session file: bzip2, as a `.ton` is, or + * plain XML, which the session reader also accepts. A file that + * cannot be read, or that has no element, gives found == false. + */ + static Takes read(QString sessionPath); + + /// The pass itself, over a document that is open already + static Takes readFrom(QXmlStreamReader &reader); +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 19d84936..374df3f1 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -50,6 +50,7 @@ #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" #include "data/model/RegionModel.h" +#include "data/fileio/BZipFileDevice.h" #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" @@ -62,6 +63,7 @@ #include #include +#include #include #include #include @@ -73,6 +75,7 @@ #include #include +#include #include /** @@ -511,6 +514,117 @@ class TestRecordWorkflow : public QObject QVERIFY(paneHasLayer(1, m_window->timeRuler())); } + // Open a session that has just been saved, with the reference analysed + // (claimed, in fact: a restored session has its layers) and the takes + // of the session read from the file + void reopenSession(QString path) { + m_window->doCloseSession(); + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + } + + // Take the element out of a saved session file, making it what + // a .ton written before phase 7b is: the same document with no takes in + // it. The file is bzip2, as the session reader and writer leave it + // "" on success, else what went wrong + QString removeTakesElement(QString path) { + QByteArray document; + { + sv::BZipFileDevice file(path); + if (!file.open(QIODevice::ReadOnly)) { + return "could not read " + path + ": " + file.errorString(); + } + document = file.readAll(); + file.close(); + } + if (document.isEmpty()) return "read nothing from " + path; + + int from = document.indexOf(""); + if (from < 0 || to < from) return "no takes element in " + path; + document.remove(from, to + int(strlen("")) - from); + + sv::BZipFileDevice out(path); + if (!out.open(QIODevice::WriteOnly)) { + return "could not write " + path + ": " + out.errorString(); + } + qint64 written = out.write(document); + out.close(); + if (written != document.size()) { + return QString("wrote %1 of %2 bytes to ").arg(written) + .arg(document.size()) + path; + } + return ""; + } + + // The dialogs the watchdog has dismissed so far, taken off the list so + // that cleanup() does not fail the test with them: for a test that + // expects one + QStringList takeDialogs() { + QStringList dialogs = m_dialogs; + m_dialogs.clear(); + return dialogs; + } + + // The dialogs seen so far whose text holds this, taken off the list + // along with the ones before them: for a test that expects one of its + // own among the dialogs another part of the application showed first + QStringList dialogsMatching(QString text) { + QStringList matching; + for (const QString &dialog : takeDialogs()) { + if (dialog.contains(text)) matching.push_back(dialog); + } + return matching; + } + + // Audio in the session besides the reference: one model per take that + // is on show, and nothing left over from a session load + int audioModelsBesidesReference() { + int n = 0; + for (sv::ModelId id : m_window->document()->getModels()) { + if (id == m_window->mainModelId()) continue; + if (sv::ModelById::isa(id)) ++n; + } + return n; + } + + int waveformLayersInPane0() { + int n = 0; + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return 0; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (qobject_cast(pane->getLayer(i))) ++n; + } + return n; + } + + // The same events after a session round trip. Frames, durations and + // labels come back exactly; a value goes through QString::arg(float) + // in Event::toXml(), which keeps six significant figures + void verifyEventsSurvived(const sv::EventVector &before, + const sv::EventVector &after, + const char *what) { + QVERIFY2(before.size() == after.size(), + qPrintable(QString("%1: %2 events before, %3 after") + .arg(what).arg(before.size()).arg(after.size()))); + for (size_t i = 0; i < before.size(); ++i) { + QVERIFY2(before[i].getFrame() == after[i].getFrame(), + qPrintable(QString("%1: event %2 was at frame %3, is at %4") + .arg(what).arg(i) + .arg(before[i].getFrame()) + .arg(after[i].getFrame()))); + QCOMPARE(after[i].getDuration(), before[i].getDuration()); + double value = before[i].getValue(); + QVERIFY2(std::fabs(after[i].getValue() - value) <= + 1e-5 * std::fabs(value) + 1e-6, + qPrintable(QString("%1: event %2 had the value %3, has %4") + .arg(what).arg(i).arg(value) + .arg(after[i].getValue()))); + } + } + void verifyRulerIntact() { QVERIFY(m_window->timeRuler()); QVERIFY2(documentHasLayer(m_window->timeRuler()), @@ -754,8 +868,20 @@ class TestRecordWorkflow : public QObject QString description = modal->windowTitle(); if (auto box = qobject_cast(modal)) { description += ": " + box->text(); + m_dialogs.push_back(description); + // A question with buttons of its own is not answered by + // rejecting it: the session reader's "do you want to locate + // this file?" asks again until one of its buttons is pressed + // (InteractiveFileFinder::locateInteractive()). The last + // button is Cancel in every question Tony can show + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); } - m_dialogs.push_back(description); if (auto dialog = qobject_cast(modal)) { dialog->reject(); } else { @@ -4200,11 +4326,12 @@ private slots: QVERIFY(m_window->takeCombo()->isEnabled()); } - // A session with two takes saved and opened again. Phase 7b stores - // the takes properly; until then the session says only which audio - // file the active take was in, so that take comes back as a take of - // its own, analysed afresh, and the layers of both saved takes are - // left in the pane, hidden and owned by nobody + // The takes of a session in the .ton (spec 6.4): the element + // says which takes there are, where their audio is and which was on + // show; their pitch, notes and coverage come back as the layers they + // are stored in, and nothing is analysed + + // Two takes, the second of them active, saved and opened again void two_takes_survive_a_session_opening() { FakeAudioIO::Config config; config.input = tone(highHz, 6.0); @@ -4212,49 +4339,363 @@ private slots: openReference(writeWav(tone(lowHz, 4.0))); if (QTest::currentTestFailed()) return; + // A take with a gap in its coverage, and a second take beside it take(700); if (QTest::currentTestFailed()) return; - m_window->doNewEmptyTake(); m_window->seekTo(sv::sv_frame_t(2.0 * rate)); take(700); if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + QCOMPARE(int(first.coverage.size()), 2); + QVERIFY(!first.pitch.empty() && !first.notes.empty()); - QString path = m_window->takes()->getAudioPath(); - auto pitch = pitchEvents(m_window->analyser2()); - QVERIFY(!pitch.empty()); + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(700); + if (QTest::currentTestFailed()) return; + TakeSnapshot second = snapshotTake(); + QVERIFY(second.path != first.path); + QVERIFY(!second.pitch.empty() && !second.notes.empty()); QString session = m_dir.filePath("two-takes.ton"); QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + // Both takes, under their own names, with the one that was on show + // active again + QCOMPARE(m_window->takes()->getTakeNames(), + QStringList({ "Take 1", "Take 2" })); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + QCOMPARE(m_window->takes()->getAudioPath(), second.path); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), second.coverage); + QCOMPARE(m_window->takes()->getTake(0)->audioPath, first.path); + QCOMPARE(m_window->takes()->getTake(0)->coverage.getRanges(), + first.coverage); + + // The active take's own layers are the ones the analyser holds, and + // its audio is under them + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + QCOMPARE(static_cast(takeLayers("Take 2").pitch), + a2->getLayer(Analyser::PitchTrack)); + QVERIFY(takeAudio()); + QCOMPARE(takeAudio()->getStartFrame(), sv::sv_frame_t(0)); + + // The pitch and the notes of both takes came back as they were: a + // session file keeps a value to six figures, nothing else changes + verifyEventsSurvived(second.pitch, pitchEvents(a2), "the active take's pitch"); + verifyEventsSurvived(second.notes, + noteEvents(a2->getLayer(Analyser::Notes)), + "the active take's notes"); + verifyEventsSurvived(first.pitch, pitchEvents(takeLayers("Take 1").pitch), + "the stored take's pitch"); + verifyEventsSurvived(first.notes, noteEvents(takeLayers("Take 1").notes), + "the stored take's notes"); + if (QTest::currentTestFailed()) return; + + // Nothing was analysed on the way in, then or when the queued + // calls of the load ran + QVERIFY(!m_window->analysingRange()); + QCoreApplication::processEvents(); + QTest::qWait(100); + QVERIFY2(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), + "the session load started an analysis of a take"); + QCOMPARE(noteLayersInPane0(), 3); // the reference's and the two takes' + + // The take that is not on show is hidden, silent and owned by + // nobody; the one that is can be heard, and playback runs to the + // end of it + verifyTakeIsPutAway("Take 1"); + if (QTest::currentTestFailed()) return; + QVERIFY(a2->isAudible(Analyser::Audio)); + QVERIFY(m_window->playSource()->getModels() + .count(a2->getMainModelId())); + QVERIFY(m_window->playSource()->getPlayEndFrame() >= + m_window->takes()->getCoverage().getEndFrame()); + + // Each take has its strip; the active one's is on show and is the + // coverage the take came back with + QCOMPARE(allStripLayersInPane0(), 2); + verifyStripMatchesTake(); + if (QTest::currentTestFailed()) return; + + // One copy of the take's audio and no leftovers of the load: the + // waveform layer and model that Document::toXml() wrote for the + // active take were dropped (spec 6.4) + QCOMPARE(m_window->paneStack()->getPaneCount(), 2); + QCOMPARE(m_window->paneStack()->getHiddenPaneCount(), 0); + QCOMPARE(audioModelsBesidesReference(), 1); + QCOMPARE(waveformLayersInPane0(), 2); // the reference's and the take's + verifyPlaySourceClean(); + QVERIFY2(!m_window->isDocumentModified(), + "opening a session left it looking modified"); + QCOMPARE(undoOnce(), QString()); // and nothing on the undo stack + if (QTest::currentTestFailed()) return; + + // Switching takes works after a load like any other time + QVERIFY(m_window->doSwitchToTake(0)); + QCOMPARE(m_window->takes()->getAudioPath(), first.path); + verifyEventsSurvived(first.pitch, pitchEvents(m_window->analyser2()), + "the take switched to after a load"); + if (QTest::currentTestFailed()) return; + verifyTakeIsPutAway("Take 2"); + if (QTest::currentTestFailed()) return; + verifyStripMatchesTake(); + QVERIFY(!m_window->analysingRange()); + QVERIFY2(m_window->isDocumentModified(), + "switching take did not mark the session modified"); + + // A take made now carries the numbering on from the session + m_window->doNewEmptyTake(); + QCOMPARE(m_window->takes()->getActiveName(), QString("Take 3")); + } + + // A saved session names the audio file of every take, so none of them + // is deleted on close, however thoroughly it has been superseded since + void saved_session_protects_every_take_audio() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + QString firstSaved = m_window->takes()->getAudioPath(); + + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(600); + if (QTest::currentTestFailed()) return; + QString secondSaved = m_window->takes()->getAudioPath(); + + QString session = m_dir.filePath("protected.ton"); + QVERIFY(m_window->saveSessionFile(session)); + + // Recording into the first take again supersedes the file the saved + // session names for it + QVERIFY(m_window->doSwitchToTake(0)); + m_window->seekTo(sv::sv_frame_t(1.0 * rate)); + take(600); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->takes()->getAudioPath() != firstSaved); + QVERIFY2(m_window->takes()->unusedWrittenFiles().isEmpty(), + "an audio file the saved session names was up for deletion"); + + m_window->doCloseSession(); + + QVERIFY2(QFileInfo::exists(firstSaved), + "the audio the saved session names for the first take was " + "deleted when the session closed"); + QVERIFY2(QFileInfo::exists(secondSaved), + "the audio the saved session names for the second take was " + "deleted when the session closed"); + } + + // A take name with characters that XML cares about + void take_name_with_entities_survives_a_session() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + + const QString name = "Rock & \"Roll\" <2>"; + m_window->setTakeNameAnswer(name); + m_window->doRenameTake(); + QCOMPARE(m_window->takes()->getActiveName(), name); + TakeSnapshot before = snapshotTake(); + + QString session = m_dir.filePath("entities.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { name }); + QCOMPARE(m_window->takes()->getAudioPath(), before.path); + QVERIFY(m_window->analyser2()); + QCOMPARE(static_cast(takeLayers(name).pitch), + m_window->analyser2()->getLayer(Analyser::PitchTrack)); + verifyEventsSurvived(before.pitch, pitchEvents(m_window->analyser2()), + "the pitch of a take with an escaped name"); + if (QTest::currentTestFailed()) return; + verifyStripMatchesTake(); + } + + // A session from before the takes were stored opens without its + // singing track, and nothing of one is left in the pane (spec 3) + void session_without_takes_opens_without_singing() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + QString takePath = m_window->takes()->getAudioPath(); + + QString session = m_dir.filePath("old.ton"); + QVERIFY(m_window->saveSessionFile(session)); + + // As a .ton written before this phase: the same document without + // the element Tony's own pass reads + QString stripped = removeTakesElement(session); + QVERIFY2(stripped == "", qPrintable(stripped)); + + reopenSession(session); + if (QTest::currentTestFailed()) return; + + // No take, and no analyser or layers of one + QCOMPARE(m_window->takes()->getTakeCount(), 0); + QVERIFY(!m_window->takes()->haveTake()); + QVERIFY2(!m_window->analyser2(), + "the singing track of an old session was set up as a take"); + QVERIFY(!m_window->coverageStrip()->isShown()); + QCOMPARE(allStripLayersInPane0(), 0); + QCOMPARE(noteLayersInPane0(), 1); // the reference's + QCOMPARE(waveformLayersInPane0(), 1); // the reference's + QCOMPARE(audioModelsBesidesReference(), 0); + QCOMPARE(m_window->paneStack()->getPaneCount(), 2); + verifyPlaySourceClean(); + QVERIFY(!m_window->isDocumentModified()); + + // The audio file was not touched, and the reference is intact + QVERIFY(QFileInfo::exists(takePath)); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser())), + lowHz)) < 10.0); + + // Recording makes the session's first take, as in a new session + m_window->seekTo(0); + take(600); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 1" }); + verifyStripMatchesTake(); + } + + // The audio file of a take is gone when the session is opened: one + // warning, and the take is shown without its sound (spec 6.4) + void session_take_with_missing_audio() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + TakeSnapshot before = snapshotTake(); + QVERIFY(!before.pitch.empty()); + + QString session = m_dir.filePath("missing.ton"); + QVERIFY(m_window->saveSessionFile(session)); m_window->doCloseSession(); + QVERIFY(QFile::remove(before.path)); m_window->discardModifications(); QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), MainWindow::FileOpenSucceeded); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); - QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); - // One take, in the audio file the active take was in, and named - // after neither of the takes whose layers the session holds - QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 3" }); - // A restored model reports its own file's path, whose drive letter - // Windows may have changed the case of - QCOMPARE(m_window->takes()->getAudioPath().toLower(), path.toLower()); - QVERIFY(m_window->analyser2()); - QCOMPARE(static_cast(takeLayers("Take 3").pitch), - m_window->analyser2()->getLayer(Analyser::PitchTrack)); + // One warning of ours, naming the take and the file that is + // missing. The session reader has its own two to say first -- it + // asks whether to locate the audio model that Document::toXml() + // wrote for the take's waveform layer, and then says the session is + // incomplete -- which is the reason for the fork change asked for + // in the phase's notes; the takes themselves need neither + QStringList ours = dialogsMatching("shown without its audio"); + QCOMPARE(ours.size(), 1); + QVERIFY2(ours[0].contains(QFileInfo(before.path).fileName()), + qPrintable(ours[0])); + + // The take is there, with its pitch, its notes and its coverage, + // and no audio + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 1" }); + QCOMPARE(m_window->takes()->getAudioPath(), before.path); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), before.coverage); + QVERIFY(!m_window->analyser2()); + QVERIFY(!takeAudio()); + TakeLayers::Found found = takeLayers("Take 1"); + QVERIFY(found.pitch && found.notes && found.coverage); + verifyEventsSurvived(before.pitch, pitchEvents(found.pitch), + "the pitch of a take whose audio is missing"); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->coverageStrip()->isShown()); + QCOMPARE(stripEvents(), before.strip); + verifyPlaySourceClean(); - // Analysed afresh rather than mixed up with another take's pitch - QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + // Recording into it is refused, with one warning, and the take is + // left exactly as it was: splicing needs the file it adds to + m_window->seekTo(0); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(400); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); - // Both saved takes' layers are still there, put away, and they are - // not the ones the take on show is using - QVERIFY(takeLayers("Take 1").pitch && takeLayers("Take 1").notes); - QVERIFY(takeLayers("Take 2").pitch && takeLayers("Take 2").notes); - QVERIFY(takeLayers("Take 3").pitch != takeLayers("Take 1").pitch); - QVERIFY(takeLayers("Take 3").pitch != takeLayers("Take 2").pitch); - verifyTakeIsPutAway("Take 1"); - verifyTakeIsPutAway("Take 2"); + QStringList refusals = dialogsMatching("could not be added"); + QCOMPARE(refusals.size(), 1); + QVERIFY2(refusals[0].contains(QFileInfo(before.path).fileName()), + qPrintable(refusals[0])); + QVERIFY(takeDialogs().isEmpty()); + QCOMPARE(m_window->takes()->getAudioPath(), before.path); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), before.coverage); + QCOMPARE(stripEvents(), before.strip); + } + + // Saving while the analysis of a recorded range runs: the save waits + // for the merge, so the session holds the take's own pitch and notes + // and neither the state before the merge nor the run's two temporary + // models + void save_during_ranged_analysis() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + startTake(); if (QTest::currentTestFailed()) return; + QTest::qWait(700); + + // Stop splices the recording in and starts the analysis of it there + // and then, so it is running when the session is saved + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY2(m_window->analysingRange(), + "the test shows nothing: no analysis was running when the " + "session was saved"); + + QString session = m_dir.filePath("mid-analysis.ton"); + QVERIFY(m_window->saveSessionFile(session)); + + QVERIFY2(!m_window->analysingRange(), + "the save did not wait for the analysis of the recording"); + TakeSnapshot before = snapshotTake(); + QVERIFY(!before.pitch.empty()); + + reopenSession(session); + if (QTest::currentTestFailed()) return; + + // The merged pitch and notes are what the session holds, in the + // take's own layers and no others + QVERIFY(m_window->analyser2()); + verifyEventsSurvived(before.pitch, pitchEvents(m_window->analyser2()), + "the pitch saved during an analysis"); + verifyEventsSurvived(before.notes, + noteEvents(m_window->analyser2() + ->getLayer(Analyser::Notes)), + "the notes saved during an analysis"); + if (QTest::currentTestFailed()) return; + QCOMPARE(noteLayersInPane0(), 2); + QCOMPARE(audioModelsBesidesReference(), 1); + verifyStripMatchesTake(); verifyPlaySourceClean(); } diff --git a/main/test/TestTakeAudio.h b/main/test/TestTakeAudio.h index f2ec65a5..cde5eed1 100644 --- a/main/test/TestTakeAudio.h +++ b/main/test/TestTakeAudio.h @@ -368,8 +368,12 @@ private slots: 0, 0, -1, out) != ""); QVERIFY(!QFile::exists(out)); - QVERIFY(TakeAudio::splice(m_dir.filePath("absent.wav"), recording, - 0, 0, -1, out) != ""); + // The take's own file gone missing: the message says so, and does + // not claim that the file being written exists already + QString missing = m_dir.filePath("absent.wav"); + QString error = TakeAudio::splice(missing, recording, 0, 0, -1, out); + QVERIFY2(error.contains("absent.wav") && !error.contains(out), + qPrintable(error)); QVERIFY(!QFile::exists(out)); QVERIFY(TakeAudio::splice(old, other, 0, 0, -1, out) != ""); diff --git a/main/test/TestTakesFile.h b/main/test/TestTakesFile.h new file mode 100644 index 00000000..b84f88f8 --- /dev/null +++ b/main/test/TestTakesFile.h @@ -0,0 +1,179 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TAKES_FILE_H +#define TEST_TAKES_FILE_H + +// Tier 2: the element of a session file, written from the takes of +// a session and read back. No window: the element is written to a string +// and to a file here, as a session's own reader would find it. + +#include "../TakesFile.h" +#include "../SingingTakes.h" + +#include "data/fileio/BZipFileDevice.h" + +#include +#include +#include +#include +#include + +class TestTakesFile : public QObject +{ + Q_OBJECT + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + // The element as it appears in a session file: inside an + // document, which is what the reading pass has to pick it out of + static QString inDocument(QString element) { + return "\n" + "\n\n\n" + "\n" + "\n\n\n" + "\n" + element + "\n"; + } + + static TakesFile::Takes readString(QString xml) { + QXmlStreamReader reader(xml); + return TakesFile::readFrom(reader); + } + + QString writeFile(QString content, bool compressed) { + QString path = m_dir.filePath + (QString("session-%1.ton").arg(++m_fileCounter)); + if (compressed) { + sv::BZipFileDevice file(path); + if (!file.open(QIODevice::WriteOnly)) return ""; + file.write(content.toUtf8()); + file.close(); + } else { + QFile file(path); + if (!file.open(QIODevice::WriteOnly)) return ""; + file.write(content.toUtf8()); + file.close(); + } + return path; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + // Two takes and the active one, through a string and back + void round_trip() { + SingingTakes takes; + QCOMPARE(takes.addTake(), QString("Take 1")); + takes.restoreTake("C:/songs/one.wav", Coverage()); + QCOMPARE(takes.addTake(), QString("Take 2")); + takes.restoreTake("C:/songs/two.wav", Coverage()); + QVERIFY(takes.setActiveIndex(0)); + + QString element = TakesFile::toXml(takes); + TakesFile::Takes read = readString(inDocument(element)); + + QVERIFY(read.found); + QCOMPARE(read.active, QString("Take 1")); + QCOMPARE(int(read.takes.size()), 2); + QCOMPARE(read.takes[0].name, QString("Take 1")); + QCOMPARE(read.takes[0].audioPath, QString("C:/songs/one.wav")); + QCOMPARE(read.takes[1].name, QString("Take 2")); + QCOMPARE(read.takes[1].audioPath, QString("C:/songs/two.wav")); + } + + // A take with no recording in it yet has no audio file + void empty_take() { + SingingTakes takes; + takes.addTake(); + + TakesFile::Takes read = readString(inDocument(TakesFile::toXml(takes))); + + QVERIFY(read.found); + QCOMPARE(int(read.takes.size()), 1); + QCOMPARE(read.takes[0].audioPath, QString()); + QCOMPARE(read.active, QString("Take 1")); + } + + // A session with no take at all still says so, which is what tells it + // from a session saved before the takes were stored + void no_takes() { + SingingTakes takes; + + TakesFile::Takes read = readString(inDocument(TakesFile::toXml(takes))); + QVERIFY(read.found); + QVERIFY(read.takes.empty()); + QCOMPARE(read.active, QString()); + + TakesFile::Takes none = readString(inDocument("")); + QVERIFY2(!none.found, "a document with no takes element was read as " + "having one"); + } + + // A name the user gave with characters that XML cares about + void escaped_name() { + SingingTakes takes; + takes.addTake("Rock & \"Roll\" <2>"); + takes.restoreTake("C:/songs/a & b/take's.wav", Coverage()); + + QString element = TakesFile::toXml(takes); + QVERIFY2(!element.contains("& \""), qPrintable(element)); + + TakesFile::Takes read = readString(inDocument(element)); + QCOMPARE(int(read.takes.size()), 1); + QCOMPARE(read.takes[0].name, QString("Rock & \"Roll\" <2>")); + QCOMPARE(read.takes[0].audioPath, + QString("C:/songs/a & b/take's.wav")); + QCOMPARE(read.active, QString("Rock & \"Roll\" <2>")); + } + + // From a file: bzip2, as a .ton is, and plain XML, which the session + // reader also accepts + void from_a_file() { + SingingTakes takes; + takes.addTake("Chorus"); + takes.restoreTake("C:/songs/chorus.wav", Coverage()); + + QString document = inDocument(TakesFile::toXml(takes)); + + for (bool compressed : { true, false }) { + QString path = writeFile(document, compressed); + QVERIFY(!path.isEmpty()); + TakesFile::Takes read = TakesFile::read(path); + QVERIFY2(read.found, compressed ? "bzip2" : "plain"); + QCOMPARE(int(read.takes.size()), 1); + QCOMPARE(read.takes[0].name, QString("Chorus")); + QCOMPARE(read.takes[0].audioPath, QString("C:/songs/chorus.wav")); + } + + // Nothing there to read + QVERIFY(!TakesFile::read(m_dir.filePath("nothing.ton")).found); + QVERIFY(!TakesFile::read("").found); + } + + // The numbering carries on from the takes a session had, whatever has + // happened to them since + void reserved_names_carry_the_numbering_on() { + SingingTakes takes; + takes.reserveTakeName("Take 1"); + takes.reserveTakeName("Take 7"); + takes.reserveTakeName("Chorus"); + + QCOMPARE(takes.addTake("Take 7"), QString("Take 7")); + QCOMPARE(takes.addTake(), QString("Take 8")); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 0adb6fdb..4a7ed2b5 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -18,6 +18,7 @@ #include "TestTakeAudio.h" #include "TestTakeEvents.h" #include "TestSingingTakes.h" +#include "TestTakesFile.h" #include "TestTakeTiming.h" #include "RunSuite.h" @@ -85,6 +86,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTakesFile t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestTakeTiming t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 0579baf1..1e3a4704 100644 --- a/meson.build +++ b/meson.build @@ -1099,6 +1099,7 @@ tony_core_files = [ 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', 'main/TakeEvents.cpp', + 'main/TakesFile.cpp', 'main/TakeTiming.cpp', ] @@ -1353,6 +1354,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakeAudio.h', 'main/test/TestTakeEvents.h', 'main/test/TestSingingTakes.h', + 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', ]) From 3b6c506d705850e6e53f0af9235a8f400634a7b0 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 19:13:58 +0300 Subject: [PATCH 050/275] test: take_analyses_only_the_new_range no longer races the merge The range a ranged analysis was asked for is remembered only until its result is merged (m_takeAnalysisRange), and a run short enough to finish before the splice call returns never records one. The test read it after waiting for the analysis to complete, so it failed about one full run in three with "0 expected 110250". It reads it now between the Stop and the wait, and only while the run is still going. Co-Authored-By: Claude Opus 5 --- main/test/TestRecordWorkflow.h | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 374df3f1..65cbb683 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1600,8 +1600,20 @@ private slots: // analysis of the second recording takes for itself const sv::sv_frame_t P = sv::sv_frame_t(2.5 * rate); m_window->seekTo(P); - take(700); + startTake(); if (QTest::currentTestFailed()) return; + QTest::qWait(700); + + // Stop splices the recording in and asks for the analysis of the + // range it went into. The range is read here, while that run is + // still going: it is remembered only until the merge, so a run that + // finishes before the splice call returns never records one at all + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + if (m_window->analysingRange()) { + QCOMPARE(m_window->analysedRangeStart(), P); + } + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); // The analysis of the take was not thrown away and run again: the // layers, and the models under them, are the same objects @@ -1638,7 +1650,6 @@ private slots: // The second recording was analysed, and in the right place QVERIFY(!eventsBetween(pitchEvents(pitch), P, P + sv::sv_frame_t(0.6 * rate)).empty()); - QCOMPARE(m_window->analysedRangeStart(), P); verifyPlaySourceClean(); } From b05dde78e04e3e706de9ce519aa481bcec9a923e Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 19:38:09 +0300 Subject: [PATCH 051/275] fix: a take's audio is named in the session once, by the takes element The active take's waveform layer, and so its audio model with an absolute path, were written to the .ton by the document as well, and dropped again on load; with the file gone the session reader asked the user to locate it before Tony could say its one line. The layer is marked as not saved (svgui/svapp forks), and so is the raw recording's layer during a take. Co-Authored-By: Claude Fable 5.1 --- main/MainWindow.cpp | 18 +++++++++++++++++- main/test/TestRecordWorkflow.h | 14 ++++++++++++++ repoint-lock.json | 4 ++-- 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 55f86e33..dba36a5b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3093,6 +3093,15 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal return; } + // A take's audio is named in the session's element and opened + // from there, so neither the waveform layer nor the audio model is to + // be written to the session as well: a second copy, with an absolute + // path, that the session reader would ask the user to locate if the + // file had gone + if (Layer *audio = m_analyser2->getLayer(Analyser::Audio)) { + audio->setSavedInSession(false); + } + // m_analyser2->newFileLoaded() has now created its own WaveformLayer // referencing singingModelId. This means it is safe to delete the orphan // WaveformLayer that MainWindowBase::record() put in the extra pane: @@ -3485,6 +3494,8 @@ MainWindow::setupRecordingLayer() m_document->setModel(m_recordingLayer, m_currentRecordingModelId); m_document->attachLayerToView(pane, m_recordingLayer); + // Raw material of a take in progress: no part of a session + m_recordingLayer->setSavedInSession(false); m_recordingLayer->showLayer(pane, false); if (auto params = m_recordingLayer->getPlayParameters()) { params->setPlayAudible(false); @@ -4748,7 +4759,12 @@ MainWindow::dropRestoredSingingTrack(bool withTakeLayers) if (id == mainId) continue; if (ModelById::isa(id)) audio.push_back(id); } - if (audio.empty()) return; + + // (None, in a session saved since the take's waveform layer was kept + // out of the file -- Layer::setSavedInSession() -- but sessions saved + // before that carry one, and layers named after a take may be there + // to drop either way) + if (audio.empty() && !withTakeLayers) return; auto isAudio = [&audio](ModelId id) { return std::find(audio.begin(), audio.end(), id) != audio.end(); diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 65cbb683..fb32e04b 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -4606,6 +4606,20 @@ private slots: QString session = m_dir.filePath("missing.ton"); QVERIFY(m_window->saveSessionFile(session)); + + // The session names the take's audio once, in the takes element: + // the document does not carry the audio model as well (the take's + // waveform layer is not saved), which the session reader would + // ask the user to locate when the file has gone + { + sv::BZipFileDevice file(session); + QVERIFY(file.open(QIODevice::ReadOnly)); + QByteArray document = file.readAll(); + file.close(); + // (one wave file model: the reference) + QCOMPARE(int(document.count("type=\"wavefile\"")), 1); + QVERIFY(document.contains("doCloseSession(); QVERIFY(QFile::remove(before.path)); diff --git a/repoint-lock.json b/repoint-lock.json index 4d990697..4b22ff82 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,10 +7,10 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "573c939b9d0b1db3b756c6a133f64b2cba0a16a7" + "pin": "1852b930f77cb54e9fc9af39218bc8c012ba0626" }, "svapp": { - "pin": "8dfa3bc8d5ede210210ebba003c736c46e86405a" + "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" }, "checker": { "pin": "fae540cf4a79ac5ed5a4d4dc0df680b1acbe8628" From 84cb39d31058e19857c9a71f0d83d43e076f5ddd Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 20:11:20 +0300 Subject: [PATCH 052/275] feat: a take's audio lives in the session's .takes folder TakesFile grows the path arithmetic of spec 6.4: takesFolder(), relativeAudioPath(), resolveAudioPath(), isInFolder(), freeCopyPath() and copyTakeAudioInto(), which copies the audio of every take that is not in the folder already and points the takes at the copies, one copy per file for takes that share one. Copied and not moved: the active take's model has its file open. MainWindow::takeAudioDirectory() is where a new combined file goes: the session's folder, made on demand, once the session has a file, and the record directory before the first save. saveSessionFile() copies the takes' audio into the folder of the file it is about to write and refuses the save if that cannot be done; toXml() then names each take's audio relative to the .ton, and restoreTakes() resolves it against the session's own directory, an absolute path from a 7b-era session being left alone. A .ton opened without its folder gives one warning for the session, naming the folder, instead of one per take. An empty takes folder this run made goes when the session closes. Save As and Save In Audio Path share saveSessionToPath(), which is also the tests' way in to a save that sets the session's file. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 186 ++++++++++++--- main/MainWindow.h | 41 +++- main/SingingTakes.cpp | 10 + main/SingingTakes.h | 14 ++ main/TakesFile.cpp | 209 ++++++++++++++++- main/TakesFile.h | 67 +++++- main/test/TestRecordWorkflow.h | 400 +++++++++++++++++++++++++++++++-- main/test/TestTakesFile.h | 272 ++++++++++++++++++++++ 8 files changed, 1154 insertions(+), 45 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index dba36a5b..6b16fc48 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -2357,6 +2357,21 @@ MainWindow::closeSession() << " superseded take audio file(s)" << endl; } + // A takes folder this run made and that has nothing left in it goes as + // well (spec 6.4). One with anything at all in it stays: what is in + // it was not necessarily put there by us + for (const QString &folder : m_takeFoldersMade) { + QDir dir(folder); + if (!dir.exists()) continue; + if (!dir.isEmpty(QDir::AllEntries | QDir::Hidden | QDir::System | + QDir::NoDotAndDotDot)) continue; + if (QDir().rmdir(folder)) { + cerr << "MainWindow::closeSession: removed the empty takes folder " + << folder << endl; + } + } + m_takeFoldersMade.clear(); + m_takes->clear(); m_takePosition = 0; m_takePreRoll = 0; @@ -4158,7 +4173,7 @@ MainWindow::finishSingingTake() if (recordingPath == "") { error = tr("The recording is no longer there to be used"); } else { - QString directory = RecordDirectory::getRecordDirectory(); + QString directory = takeAudioDirectory(); if (directory == "") { error = tr("Could not find a directory to write the singing " "track into"); @@ -4450,7 +4465,7 @@ MainWindow::eraseSingingInSelection() } if (selected.empty()) return; - QString directory = RecordDirectory::getRecordDirectory(); + QString directory = takeAudioDirectory(); QString error; if (directory == "") { @@ -4570,6 +4585,84 @@ MainWindow::eraseTakeEvents(const Coverage::Ranges &erased, } } +QString +MainWindow::takeAudioDirectory() +{ + // A take's combined audio belongs to the song, in the session's own + // folder beside the .ton (spec 6.4). Before the first save there is + // no folder to put it in, so it goes to the record directory with the + // raw recordings, and the save copies it across + if (m_sessionFile != "") { + QString folder = ensureTakesFolder(m_sessionFile); + if (folder != "") return folder; + + // Nowhere to write it beside the session: better in the record + // directory than nowhere at all, and the next save says so when it + // cannot copy it either + cerr << "MainWindow::takeAudioDirectory: could not make the takes " + << "folder of \"" << m_sessionFile << "\"; writing to the record " + << "directory instead" << endl; + } + + return RecordDirectory::getRecordDirectory(); +} + +QString +MainWindow::ensureTakesFolder(QString sessionPath) +{ + QString folder = TakesFile::takesFolder(sessionPath); + if (folder == "") return ""; + + if (QFileInfo(folder).isDir()) return folder; + + if (!QDir().mkpath(folder)) return ""; + + // Made here and empty so far: if nothing is left in it when the + // session closes, it goes again + if (!m_takeFoldersMade.contains(folder)) m_takeFoldersMade.push_back(folder); + + return folder; +} + +bool +MainWindow::copyTakeAudioForSave(QString sessionPath) +{ + if (!m_takes || m_takes->getTakeCount() == 0) return true; + + // Nothing to do for a session whose takes are all in its folder + // already, and no folder to make for one that has no audio at all + bool anyOutside = false; + QString folder = TakesFile::takesFolder(sessionPath); + for (const SingingTakes::Take &take : m_takes->getTakes()) { + if (take.audioPath == "") continue; + if (!TakesFile::isInFolder(folder, take.audioPath)) anyOutside = true; + } + if (!anyOutside) return true; + + QString error; + QString made = ensureTakesFolder(sessionPath); + + if (made == "") { + error = tr("The folder \"%1\", where the takes' audio belongs, could " + "not be made").arg(folder); + } else { + error = TakesFile::copyTakeAudioInto(*m_takes, made); + } + + if (error != "") { + QMessageBox::critical + (this, + tr("Failed to save the session"), + tr("The session was not saved

The audio of its takes " + "must be in the folder beside it, and could not be put " + "there.

%1

").arg(error), + QMessageBox::Ok); + return false; + } + + return true; +} + bool MainWindow::saveSessionFile(QString path) { @@ -4577,7 +4670,16 @@ MainWindow::saveSessionFile(QString path) // hold neither the state before the merge nor the state after it if (!waitForRangedAnalysis()) return false; + // The takes' audio goes into the folder of the session being saved + // before the file is written, so that the file names it where it is. + // A Save As copies into the new session's folder and leaves the old + // one alone: it belongs to the .ton that is still there + if (!copyTakeAudioForSave(path)) return false; + + // What toXml() names the takes' audio relative to + m_savingSessionPath = path; bool saved = MainWindowBase::saveSessionFile(path); + m_savingSessionPath = ""; // The .ton that has just been written names the audio file of every // take of the session. Recording into a take again writes another file @@ -4625,7 +4727,7 @@ MainWindow::toXml(QTextStream &out, bool asTemplate) } out << document.left(closing) - << TakesFile::toXml(*m_takes) + << TakesFile::toXml(*m_takes, m_savingSessionPath) << document.mid(closing); } @@ -4714,8 +4816,14 @@ MainWindow::restoreTakes(QString sessionPath) } } + // The stored path is relative to the session file, so that a song + // and its takes folder can be moved together (spec 6.4); a session + // saved before the folder names its audio absolutely + QString audioPath = TakesFile::resolveAudioPath(sessionPath, + take.audioPath); + // restoreTake() and not setTake(): nothing here supersedes a file - m_takes->restoreTake(take.audioPath, coverage); + m_takes->restoreTake(audioPath, coverage); } int active = m_takes->indexOf(stored.active); @@ -4724,16 +4832,39 @@ MainWindow::restoreTakes(QString sessionPath) cerr << "MainWindow::restoreTakes: " << m_takes->getTakeCount() << " take(s) restored, active is \"" << stored.active << "\"" << endl; + // The .ton moved without its folder: every take is silent, and the + // user is told once, about the folder, rather than once per take + // (spec 6.4). Only the active take opens its audio, so this is also + // the only place the other takes' files are ever looked for + QStringList missing; + for (const SingingTakes::Take &take : m_takes->getTakes()) { + if (take.audioPath == "" || QFileInfo::exists(take.audioPath)) continue; + missing.push_back(take.audioPath); + } + if (active >= 0) { // The same path a switch uses: the take's audio under the take's // own layers, an analyser that claims them with no analysis, its // coverage strip, and every other take put away m_takes->setActiveIndex(active); - activateTake(); + activateTake(missing.isEmpty()); } else { putOtherTakeLayersAway(); } + if (!missing.isEmpty()) { + QMessageBox::warning + (this, + tr("Takes without their audio"), + tr("The takes of this session are shown without their " + "audio

%1 audio file(s) were expected in \"%2\", and " + "are not there. Each take still has its pitch, its notes and " + "its coverage; only the sound is missing.

") + .arg(missing.size()) + .arg(TakesFile::takesFolder(sessionPath)), + QMessageBox::Ok); + } + updateTakeCombo(); updateLayerStatuses(); updateMenuStates(); @@ -5190,7 +5321,7 @@ MainWindow::deactivateTake() } bool -MainWindow::activateTake() +MainWindow::activateTake(bool warnIfNoAudio) { Pane *pane = m_paneStack ? m_paneStack->getPane(0) : nullptr; if (!pane) return false; @@ -5237,7 +5368,7 @@ MainWindow::activateTake() syncCoverageStrip(); raiseActiveTakeLayers(); - if (error != "") { + if (error != "" && warnIfNoAudio) { // Its pitch and notes are there; only the sound is missing // (spec 6.4, a missing audio file) QMessageBox::warning @@ -5873,18 +6004,30 @@ MainWindow::saveSessionInAudioPath() tr("Wait cancelled: the session has not been saved.")); } + saveSessionToPath(path); +} + +bool +MainWindow::saveSessionToPath(QString path) +{ if (!saveSessionFile(path)) { QMessageBox::critical(this, tr("Failed to save file"), tr("Session file \"%1\" could not be saved.").arg(path)); - } else { - setWindowTitle(tr("%1: %2") - .arg(QApplication::applicationName()) - .arg(QFileInfo(path).fileName())); - m_sessionFile = path; - CommandHistory::getInstance()->documentSaved(); - documentRestored(); - m_recentFiles.addFile(path); + return false; } + + setWindowTitle(tr("%1: %2") + .arg(QApplication::applicationName()) + .arg(QFileInfo(path).fileName())); + + // From here on the session has a file, so the audio of a take recorded + // from now on is written into its folder (spec 6.4) + m_sessionFile = path; + + CommandHistory::getInstance()->documentSaved(); + documentRestored(); + m_recentFiles.addFile(path); + return true; } void @@ -5907,18 +6050,7 @@ MainWindow::saveSessionAs() return; } - if (!saveSessionFile(path)) { - QMessageBox::critical(this, tr("Failed to save file"), - tr("Session file \"%1\" could not be saved.").arg(path)); - } else { - setWindowTitle(tr("%1: %2") - .arg(QApplication::applicationName()) - .arg(QFileInfo(path).fileName())); - m_sessionFile = path; - CommandHistory::getInstance()->documentSaved(); - documentRestored(); - m_recentFiles.addFile(path); - } + saveSessionToPath(path); } QString diff --git a/main/MainWindow.h b/main/MainWindow.h index c28f3996..2944c9b1 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -119,6 +119,11 @@ protected slots: virtual void saveSession(); virtual void saveSessionInAudioPath(); virtual void saveSessionAs(); + + // Save the session to this file and make it the session's own file: + // what Save As and Save In Audio Path do once they have a path. False + // if it could not be saved, in which case the user has been told + bool saveSessionToPath(QString path); virtual void exportPitchLayer(); virtual void exportNoteLayer(); virtual void importPitchLayer(); @@ -339,6 +344,36 @@ protected slots: // is a take and none yet, take it away when the take goes void syncCoverageStrip(); + // --- The audio folder of the session (spec 6.4) --- + + // Where the next combined audio file of a take is to be written: the + // session's own ".takes" folder once the session has a file, and + // the record directory before its first save, the save copying what is + // there into the folder. "" if neither could be had + QString takeAudioDirectory(); + + // The session's takes folder, made if it is not there yet. "" if it + // could not be made. A folder this run made and left empty is removed + // again when the session closes + QString ensureTakesFolder(QString sessionPath); + + // The takes folders this run made, so that an empty one can be taken + // away again on close. A folder with anything in it is never removed: + // what is in it is not necessarily ours + QStringList m_takeFoldersMade; + + // The session file being written, from the start of saveSessionFile() + // to its end: toXml() names each take's audio relative to it + QString m_savingSessionPath; + + // Copy the audio of every take that is not in the folder of the + // session being saved to into it, and point the takes at the copies + // (TakesFile::copyTakeAudioInto()). Copied, not moved: the active + // take's model has its file open. False if a copy failed, in which + // case the user has been told and nothing has changed: the session is + // not to be saved naming files that are not there + bool copyTakeAudioForSave(QString sessionPath); + // --- Several takes (spec 5.3) --- // // Every take of the session has its three layers in pane 0, named @@ -379,8 +414,10 @@ protected slots: // claims them (no analysis), its coverage strip, and the layers // themselves visible, audible as the user asked and within reach of // the editing tools. False if its audio could not be opened, in - // which case the user has been told - bool activateTake(); + // which case the user has been told -- unless warnIfNoAudio is false, + // for a session load that has one warning of its own covering all of + // its takes + bool activateTake(bool warnIfNoAudio = true); // The takes of a session that has just been read back: the // element of the file says which takes there are, what their audio diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index 32f7edbe..a7764f57 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -244,6 +244,16 @@ SingingTakes::restoreTake(QString path, const Coverage &coverage) take.coverage = coverage; } +void +SingingTakes::relocateTake(int index, QString path) +{ + if (index < 0 || index >= int(m_takes.size())) return; + if (path == "" || m_takes[index].audioPath == path) return; + + m_takes[index].audioPath = path; + if (!m_written.contains(path)) m_written.push_back(path); +} + void SingingTakes::protectPath(QString path) { diff --git a/main/SingingTakes.h b/main/SingingTakes.h index 56fe9c31..22090c3e 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -148,6 +148,20 @@ class SingingTakes : public QObject */ void restoreTake(QString path, const Coverage &coverage); + /** + * The audio of the take at this index is this file instead: a copy of + * the same sound somewhere else, made because the session was saved + * and its takes' audio belongs beside it (spec 6.4, TakesFile). + * + * The file the take had is not superseded -- nothing about the take + * has changed, and the audio model that is showing it goes on + * reading it -- but it is no longer the take's, so the cleanup on + * close may take it away if this run wrote it and no saved session + * names it. The copy counts as a file this run wrote, for the same + * reason. + */ + void relocateTake(int index, QString path); + /** * The take is the whole of this file: a singing track the user * loaded, or one restored from a session saved before coverage was diff --git a/main/TakesFile.cpp b/main/TakesFile.cpp index 7b377cdf..7f034228 100644 --- a/main/TakesFile.cpp +++ b/main/TakesFile.cpp @@ -19,13 +19,60 @@ #include "base/XmlExportable.h" #include "data/fileio/BZipFileDevice.h" +#include +#include #include +#include #include +#include + using namespace sv; +namespace { + +QString tr(const char *text) +{ + return QCoreApplication::translate("TakesFile", text); +} + +// Windows tells no two file names apart by their case, and a path +// written with back slashes names the same file as one with forward +// slashes. The session file stores forward slashes throughout +#ifdef Q_OS_WIN +const Qt::CaseSensitivity nameCase = Qt::CaseInsensitive; +#else +const Qt::CaseSensitivity nameCase = Qt::CaseSensitive; +#endif + +QString cleaned(QString path) +{ + if (path == "") return path; + return QDir::cleanPath(QDir::fromNativeSeparators(path)); +} + +// The directory part of a cleaned path, without its trailing slash +// ("C:/songs/a.ton" gives "C:", "/songs/a.ton" gives "/songs") +QString directoryOf(QString cleanedPath) +{ + int slash = cleanedPath.lastIndexOf('/'); + if (slash < 0) return ""; + if (slash == 0) return "/"; + return cleanedPath.left(slash); +} + +// One file, under one name: for telling whether two takes are sharing +// their audio +QString fileKey(QString path) +{ + QString key = cleaned(path); + return nameCase == Qt::CaseInsensitive ? key.toLower() : key; +} + +} + QString -TakesFile::toXml(const SingingTakes &takes, QString indent) +TakesFile::toXml(const SingingTakes &takes, QString sessionPath, QString indent) { QString xml; @@ -37,7 +84,8 @@ TakesFile::toXml(const SingingTakes &takes, QString indent) xml += QString("%1 \n") .arg(indent) .arg(XmlExportable::encodeEntities(take.name)) - .arg(XmlExportable::encodeEntities(take.audioPath)); + .arg(XmlExportable::encodeEntities + (relativeAudioPath(sessionPath, take.audioPath))); } xml += QString("%1\n").arg(indent); @@ -118,3 +166,160 @@ TakesFile::readFrom(QXmlStreamReader &reader) return takes; } + +QString +TakesFile::takesFolder(QString sessionPath) +{ + if (sessionPath == "") return ""; + + QString path = cleaned(sessionPath); + QString directory = directoryOf(path); + QString name = path.mid(path.lastIndexOf('/') + 1); + + // The base name of the session file: everything but the extension, + // so that "My Song.ton" gives "My Song.takes" and a name with dots + // of its own keeps them + int dot = name.lastIndexOf('.'); + if (dot > 0) name = name.left(dot); + if (name == "") return ""; + + name += ".takes"; + + if (directory == "") return name; + if (directory == "/") return "/" + name; + return directory + "/" + name; +} + +QString +TakesFile::relativeAudioPath(QString sessionPath, QString audioPath) +{ + if (audioPath == "") return ""; + + QString audio = cleaned(audioPath); + if (sessionPath == "") return audio; + + QString directory = directoryOf(cleaned(sessionPath)); + if (directory == "") return audio; + + // Anything outside the session's own directory is named where it is: + // a relative path there would be a trail of "../.." that says nothing + if (!isInFolder(directory, audio)) return audio; + + int from = (directory == "/" ? 1 : directory.length() + 1); + return audio.mid(from); +} + +QString +TakesFile::resolveAudioPath(QString sessionPath, QString storedPath) +{ + if (storedPath == "") return ""; + + QString stored = cleaned(storedPath); + + // A session saved before the takes folder names its audio absolutely + // (phase 7b), and that is where the file is + if (QFileInfo(stored).isAbsolute()) return stored; + + QString directory = directoryOf(cleaned(sessionPath)); + if (directory == "") return stored; + if (directory == "/") return cleaned("/" + stored); + return cleaned(directory + "/" + stored); +} + +bool +TakesFile::isInFolder(QString folder, QString path) +{ + if (folder == "" || path == "") return false; + + QString within = cleaned(folder); + if (!within.endsWith('/')) within += '/'; + + QString file = cleaned(path); + return file.length() > within.length() && + file.startsWith(within, nameCase); +} + +QString +TakesFile::freeCopyPath(QString folder, QString fileName) +{ + if (folder == "" || fileName == "") return ""; + + QDir dir(folder); + if (!dir.exists(fileName)) return cleaned(dir.filePath(fileName)); + + // A file of that name is there already. Whatever it is, it is not + // ours to write over: the copy gets a name of its own + QString base = fileName, suffix; + int dot = fileName.lastIndexOf('.'); + if (dot > 0) { + base = fileName.left(dot); + suffix = fileName.mid(dot); + } + + for (int i = 2; i < 1000; ++i) { + QString name = QString("%1-%2%3").arg(base).arg(i).arg(suffix); + if (!dir.exists(name)) return cleaned(dir.filePath(name)); + } + + return ""; +} + +QString +TakesFile::copyTakeAudioInto(SingingTakes &takes, QString folder) +{ + if (folder == "") return tr("No folder to copy the takes' audio into"); + + QString error; + + // The copies made here, to be undone if one of them fails, and which + // source file each of them came from: two takes sharing a file share + // the copy as well + QStringList made; + std::map copyOf; + std::vector> relocations; + + for (int i = 0; i < takes.getTakeCount(); ++i) { + + const SingingTakes::Take *take = takes.getTake(i); + if (!take || take->audioPath == "") continue; + if (isInFolder(folder, take->audioPath)) continue; + + QString key = fileKey(take->audioPath); + auto known = copyOf.find(key); + if (known != copyOf.end()) { + relocations.push_back({ i, known->second }); + continue; + } + + QString target = + freeCopyPath(folder, QFileInfo(take->audioPath).fileName()); + if (target == "") { + error = tr("Could not find a name to copy the audio of the take " + "\"%1\" under, in \"%2\"").arg(take->name).arg(folder); + break; + } + + if (!QFile::copy(take->audioPath, target)) { + error = tr("Could not copy the audio of the take \"%1\" to " + "\"%2\"").arg(take->name).arg(target); + break; + } + + made.push_back(target); + copyOf[key] = target; + relocations.push_back({ i, target }); + } + + if (error != "") { + // Nothing has been changed and nothing is left behind: a session + // that cannot have all of its audio beside it is not saved at all + for (const QString &path : made) QFile::remove(path); + return error; + } + + for (const auto &relocation : relocations) { + takes.relocateTake(relocation.first, relocation.second); + } + + return ""; +} diff --git a/main/TakesFile.h b/main/TakesFile.h index 265ae7ff..716df2cb 100644 --- a/main/TakesFile.h +++ b/main/TakesFile.h @@ -64,8 +64,73 @@ class TakesFile /** * The element for the takes of this session, indented with indent and * ending in a newline. Names and paths are escaped. + * + * sessionPath is the file being written: a take's audio inside that + * session's own directory is named relative to it (spec 6.4), which + * is what lets a song and its takes folder be moved together. With + * no session path the paths are written as they are. */ - static QString toXml(const SingingTakes &takes, QString indent = " "); + static QString toXml(const SingingTakes &takes, + QString sessionPath = "", + QString indent = " "); + + // --- Where a session's take audio lives (spec 6.4) --- + + /** + * The folder the combined audio files of a session's takes belong + * in: the session file's name without its extension, plus + * ".takes", beside the session file. "C:/songs/My Song.ton" gives + * "C:/songs/My Song.takes". Forward slashes, no trailing one; "" + * for an empty path. + */ + static QString takesFolder(QString sessionPath); + + /** + * The path to store in the session file for this audio file: the + * path relative to the session file's own directory when the audio + * is inside it, and the absolute path otherwise -- an audio file + * somewhere else is named where it is. Forward slashes. + */ + static QString relativeAudioPath(QString sessionPath, QString audioPath); + + /** + * The audio file a stored path means, for a session read from + * sessionPath: a relative path is taken against the session file's + * own directory, so that the song and its folder may be moved + * together; an absolute one -- as a session saved before the folder + * has -- is where it says it is. + */ + static QString resolveAudioPath(QString sessionPath, QString storedPath); + + /// path is inside folder, at any depth + static bool isInFolder(QString folder, QString path); + + /** + * A path in folder for a copy of a file of this name, under a name + * no file there has: the name itself if it is free, and otherwise + * the name with "-2", "-3" and so on before the extension. "" + * if folder or fileName is empty, or every name tried was taken. + */ + static QString freeCopyPath(QString folder, QString fileName); + + /** + * Copy the audio of every take that is not in folder already into + * it, and point the take at the copy: what a session save does + * before it writes the file, so that the paths it writes are inside + * the session's own folder (spec 6.4). + * + * Copied and not moved: the active take's audio model has its file + * open, which Windows will not let us move, and it can go on + * reading the old, identical copy until the next swap. A file two + * takes share -- a duplicated take -- is copied once and both take + * the copy. No file in folder is ever written over. + * + * "" on success. Otherwise the return is a message for the user, + * every take still points where it did, and the copies this call + * made have been removed again: a session must not be saved naming + * files that are not there. + */ + static QString copyTakeAudioInto(SingingTakes &takes, QString folder); /** * Read the element from a session file: bzip2, as a `.ton` is, or diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index fb32e04b..555eb3f9 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -30,6 +30,7 @@ #include "../CoverageStrip.h" #include "../SingingTakes.h" #include "../TakeLayers.h" +#include "../TakesFile.h" #include "version.h" @@ -137,6 +138,12 @@ class TestMainWindow : public MainWindow sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } + // Save As, with the file name given here instead of by a dialog: the + // session's own file is set, so that what is recorded next goes into + // its takes folder + bool doSaveSessionAs(QString path) { return saveSessionToPath(path); } + QString sessionFile() { return m_sessionFile; } + // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } bool isDocumentModified() { return m_documentModified; } @@ -745,6 +752,17 @@ class TestRecordWorkflow : public QObject m_window->analyser2()->getLayer(Analyser::Notes)); } + // The take has its sound: there is audio under it, and the file the + // take names holds the singing + void verifyTakeHasSound() { + QVERIFY(m_window->takes()->haveTake()); + QVERIFY(takeAudio()); + Coverage::Ranges ranges = m_window->takes()->getCoverage().getRanges(); + QVERIFY(!ranges.empty()); + QVERIFY2(takeAudioRms(ranges[0].start + 1000, ranges[0].end - 1000) > + 0.01, "the file the take names holds no sound"); + } + // Undo and redo, and what they say they did. CommandHistory has no // accessor for the top of its stack, and the name of the command it // unexecutes is the same thing; "" means there was nothing to undo @@ -4370,6 +4388,15 @@ private slots: QString session = m_dir.filePath("two-takes.ton"); QVERIFY(m_window->saveSessionFile(session)); + + // The save copied both takes' audio into the session's own folder + // and the file names it there (spec 6.4), so that is where the + // takes come back from + first.path = m_window->takes()->getTake(0)->audioPath; + second.path = m_window->takes()->getTake(1)->audioPath; + QVERIFY2(TakesFile::isInFolder(TakesFile::takesFolder(session), + first.path), qPrintable(first.path)); + reopenSession(session); if (QTest::currentTestFailed()) return; @@ -4474,16 +4501,21 @@ private slots: take(600); if (QTest::currentTestFailed()) return; - QString firstSaved = m_window->takes()->getAudioPath(); + QString firstRecorded = m_window->takes()->getAudioPath(); m_window->doNewEmptyTake(); m_window->seekTo(sv::sv_frame_t(2.0 * rate)); take(600); if (QTest::currentTestFailed()) return; - QString secondSaved = m_window->takes()->getAudioPath(); QString session = m_dir.filePath("protected.ton"); - QVERIFY(m_window->saveSessionFile(session)); + QVERIFY(m_window->doSaveSessionAs(session)); + + // What the saved session names is the copy of each take's audio in + // its own folder (spec 6.4) + QString firstSaved = m_window->takes()->getTake(0)->audioPath; + QString secondSaved = m_window->takes()->getTake(1)->audioPath; + QVERIFY(firstSaved != firstRecorded); // Recording into the first take again supersedes the file the saved // session names for it @@ -4492,8 +4524,13 @@ private slots: take(600); if (QTest::currentTestFailed()) return; QVERIFY(m_window->takes()->getAudioPath() != firstSaved); - QVERIFY2(m_window->takes()->unusedWrittenFiles().isEmpty(), + + QStringList unused = m_window->takes()->unusedWrittenFiles(); + QVERIFY2(!unused.contains(firstSaved) && !unused.contains(secondSaved), "an audio file the saved session names was up for deletion"); + QVERIFY2(unused.contains(firstRecorded), + "the recording the save copied into the folder is still " + "referred to by something"); m_window->doCloseSession(); @@ -4503,6 +4540,12 @@ private slots: QVERIFY2(QFileInfo::exists(secondSaved), "the audio the saved session names for the second take was " "deleted when the session closed"); + + // The recording the save copied into the folder is nobody's now: + // that copy is the take's audio, and no undo is left to want this + QVERIFY2(!QFileInfo::exists(firstRecorded), + "the recording that the save copied into the session's " + "folder was left behind in the record directory"); } // A take name with characters that XML cares about @@ -4524,6 +4567,8 @@ private slots: QString session = m_dir.filePath("entities.ton"); QVERIFY(m_window->saveSessionFile(session)); + // The take's audio is in the session's folder now (spec 6.4) + before.path = m_window->takes()->getAudioPath(); reopenSession(session); if (QTest::currentTestFailed()) return; @@ -4549,10 +4594,12 @@ private slots: take(600); if (QTest::currentTestFailed()) return; - QString takePath = m_window->takes()->getAudioPath(); QString session = m_dir.filePath("old.ton"); QVERIFY(m_window->saveSessionFile(session)); + // The save copied the take's audio into the session's folder; that + // copy is what the file names, and what must be left alone below + QString takePath = m_window->takes()->getAudioPath(); // As a .ton written before this phase: the same document without // the element Tony's own pass reads @@ -4606,6 +4653,9 @@ private slots: QString session = m_dir.filePath("missing.ton"); QVERIFY(m_window->saveSessionFile(session)); + // The save copied the take's audio into the session's folder, and + // that copy is the file the session names (spec 6.4) + before.path = m_window->takes()->getAudioPath(); // The session names the take's audio once, in the takes element: // the document does not carry the audio model as well (the take's @@ -4628,15 +4678,12 @@ private slots: MainWindow::FileOpenSucceeded); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); - // One warning of ours, naming the take and the file that is - // missing. The session reader has its own two to say first -- it - // asks whether to locate the audio model that Document::toXml() - // wrote for the take's waveform layer, and then says the session is - // incomplete -- which is the reason for the fork change asked for - // in the phase's notes; the takes themselves need neither - QStringList ours = dialogsMatching("shown without its audio"); + // One warning of ours for the session, naming the folder the takes' + // audio was expected in rather than one warning per take (spec 6.4) + QStringList ours = dialogsMatching("without their audio"); QCOMPARE(ours.size(), 1); - QVERIFY2(ours[0].contains(QFileInfo(before.path).fileName()), + QVERIFY2(ours[0].contains(QFileInfo(TakesFile::takesFolder(session)) + .fileName()), qPrintable(ours[0])); // The take is there, with its pitch, its notes and its coverage, @@ -4674,6 +4721,333 @@ private slots: QCOMPARE(stripEvents(), before.strip); } + // --- The takes folder of a session (spec 6.4) --- + // + // A take's combined audio belongs to the song: it lives in + // ".takes" beside the .ton, which names it relative to + // itself, so that the two can be moved together. Before the first + // save there is no folder, so the files are written among the raw + // recordings and the save copies them across. + + // The first save copies the take's audio into the session's folder and + // names it there, relative to the .ton. The recording it was copied + // from belongs to nobody afterwards, and goes when the session closes + void first_save_copies_the_take_audio() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + QString recorded = m_window->takes()->getAudioPath(); + + QString session = m_dir.filePath("copied.ton"); + QString folder = TakesFile::takesFolder(session); + QVERIFY(!QFileInfo::exists(folder)); + + QVERIFY(m_window->doSaveSessionAs(session)); + QCOMPARE(m_window->sessionFile(), session); + + // The take's audio is the copy in the folder, under the name it had + QString copied = m_window->takes()->getAudioPath(); + QVERIFY2(TakesFile::isInFolder(folder, copied), qPrintable(copied)); + QCOMPARE(QFileInfo(copied).fileName(), QFileInfo(recorded).fileName()); + QVERIFY(QFileInfo::exists(copied)); + + // Copied and not moved: the audio model has the old file open and + // goes on reading it + QVERIFY2(QFileInfo::exists(recorded), + "the recording was moved out from under the open model"); + QVERIFY(takeAudio()); + + // And the file names it relative to itself + { + sv::BZipFileDevice file(session); + QVERIFY(file.open(QIODevice::ReadOnly)); + QByteArray document = file.readAll(); + file.close(); + QByteArray wanted = "audio=\"copied.takes/" + + QFileInfo(copied).fileName().toUtf8() + "\""; + QVERIFY2(document.contains(wanted), qPrintable(document)); + } + + // Nothing refers to the recording now, and it is up for deletion + // when the session closes + QCOMPARE(m_window->takes()->unusedWrittenFiles(), + QStringList { recorded }); + + // An undo and a redo after the save still find the files the + // commands hold: the recordings are there until the session closes. + // The redo leaves the take pointing outside the folder again, so the + // next save copies it in again -- beside the copy that is there + // already, never over it + QCOMPARE(undoOnce(), QString("Record Singing")); + QVERIFY(!m_window->takes()->haveTake()); + QCOMPARE(redoOnce(), QString("Record Singing")); + QCOMPARE(m_window->takes()->getAudioPath(), recorded); + + QVERIFY(m_window->doSaveSessionAs(session)); + QString again = m_window->takes()->getAudioPath(); + QVERIFY2(TakesFile::isInFolder(folder, again), qPrintable(again)); + QVERIFY2(again != copied, "the second copy was written over the first"); + QVERIFY(QFileInfo::exists(copied)); + + m_window->doCloseSession(); + QVERIFY2(!QFileInfo::exists(recorded), + "the recording the save copied was left behind"); + QVERIFY2(QFileInfo::exists(copied) && QFileInfo::exists(again), + "audio that a saved session named was deleted on close"); + } + + // Once the session has a file, the next recording is written straight + // into its folder + void record_after_saving_writes_into_the_folder() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + + QString session = m_dir.filePath("recorded-into.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + QString folder = TakesFile::takesFolder(session); + + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(600); + if (QTest::currentTestFailed()) return; + + QString path = m_window->takes()->getAudioPath(); + QVERIFY2(TakesFile::isInFolder(folder, path), qPrintable(path)); + verifyTakeHasSound(); + if (QTest::currentTestFailed()) return; + + // The raw recordings are still where they were: only the combined + // files moved house + QVERIFY(QDir(sv::RecordDirectory::getRecordDirectory()) + .entryList(QStringList { "*.wav" }, QDir::Files).size() > 0); + } + + // Save As to another place copies the takes into the new session's + // folder and leaves the old one alone: the old .ton is still good + void save_as_copies_into_the_new_folder() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + + QString first = m_dir.filePath("first.ton"); + QVERIFY(m_window->doSaveSessionAs(first)); + QString firstAudio = m_window->takes()->getAudioPath(); + + QVERIFY(QDir().mkpath(m_dir.filePath("other"))); + QString second = QDir(m_dir.filePath("other")).filePath("second.ton"); + QVERIFY(m_window->doSaveSessionAs(second)); + QString secondAudio = m_window->takes()->getAudioPath(); + + QVERIFY2(TakesFile::isInFolder(TakesFile::takesFolder(second), + secondAudio), qPrintable(secondAudio)); + QVERIFY2(QFileInfo::exists(firstAudio), + "Save As took the audio of the session it was saved from"); + + // Both sessions open, with their own copy of the singing + for (QString session : { first, second }) { + reopenSession(session); + if (QTest::currentTestFailed()) return; + verifyTakeHasSound(); + if (QTest::currentTestFailed()) return; + QVERIFY2(TakesFile::isInFolder(TakesFile::takesFolder(session), + m_window->takes()->getAudioPath()), + qPrintable(m_window->takes()->getAudioPath())); + } + } + + // Two takes sharing one audio file (a duplicate) share the copy of it + void duplicated_take_copies_its_audio_once() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + m_window->doDuplicateTake(); + QCOMPARE(m_window->takes()->getTakeCount(), 2); + QCOMPARE(m_window->takes()->getTake(1)->audioPath, + m_window->takes()->getTake(0)->audioPath); + + QString session = m_dir.filePath("shared.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + + QString folder = TakesFile::takesFolder(session); + QString copied = m_window->takes()->getTake(0)->audioPath; + QCOMPARE(m_window->takes()->getTake(1)->audioPath, copied); + QVERIFY2(TakesFile::isInFolder(folder, copied), qPrintable(copied)); + QCOMPARE(QDir(folder).entryList(QStringList { "*.wav" }, + QDir::Files).size(), 1); + } + + // The .ton and its folder moved together: the paths in the file are + // relative, so the takes are found in the new place + void session_folder_moved_as_a_whole() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + + QString home = m_dir.filePath("in-place"); + QVERIFY(QDir().mkpath(home)); + QString session = QDir(home).filePath("song.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + m_window->doCloseSession(); + + // The .ton and its "song.takes" folder, moved as one + QString moved = m_dir.filePath("moved-house"); + QVERIFY2(QDir().rename(home, moved), "could not move the session"); + QString movedSession = QDir(moved).filePath("song.ton"); + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(movedSession, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + + verifyTakeHasSound(); + if (QTest::currentTestFailed()) return; + QVERIFY2(TakesFile::isInFolder(TakesFile::takesFolder(movedSession), + m_window->takes()->getAudioPath()), + qPrintable(m_window->takes()->getAudioPath())); + QVERIFY(!pitchEvents(m_window->analyser2()).empty()); + } + + // The .ton moved without its folder: one warning for the session, + // naming the folder, and the take shows its pitch and notes with no + // sound + void session_moved_without_its_folder() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + TakeSnapshot before = snapshotTake(); + + QString session = m_dir.filePath("left-behind.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + m_window->doCloseSession(); + + // The file on its own, in a directory with no takes folder + QString elsewhere = m_dir.filePath("without-folder"); + QVERIFY(QDir().mkpath(elsewhere)); + QString moved = QDir(elsewhere).filePath("left-behind.ton"); + QVERIFY(QFile::copy(session, moved)); + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(moved, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + + // One warning, for the session and not for the take, naming the + // folder the audio was expected in + QStringList warnings = dialogsMatching("without their audio"); + QCOMPARE(warnings.size(), 1); + QVERIFY2(warnings[0].contains("left-behind.takes"), + qPrintable(warnings[0])); + QVERIFY(takeDialogs().isEmpty()); + + // The take is there with everything but its sound + QCOMPARE(m_window->takes()->getTakeNames(), QStringList { "Take 1" }); + QCOMPARE(m_window->takes()->getCoverage().getRanges(), before.coverage); + QVERIFY(!m_window->analyser2()); + QVERIFY(!takeAudio()); + TakeLayers::Found found = takeLayers("Take 1"); + QVERIFY(found.pitch && found.notes && found.coverage); + verifyEventsSurvived(before.pitch, pitchEvents(found.pitch), + "the pitch of a take whose folder was left " + "behind"); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->coverageStrip()->isShown()); + } + + // A copy that cannot be made fails the save: the session is not written + // at all, and nothing about the takes changes + void a_failed_copy_fails_the_save() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(600); + if (QTest::currentTestFailed()) return; + QString firstAudio = m_window->takes()->getAudioPath(); + + m_window->doNewEmptyTake(); + m_window->seekTo(sv::sv_frame_t(2.0 * rate)); + take(600); + if (QTest::currentTestFailed()) return; + QString secondAudio = m_window->takes()->getAudioPath(); + + QString session = m_dir.filePath("blocked.ton"); + QString folder = TakesFile::takesFolder(session); + + // There is a file where the folder would go, so it cannot be made + { + QFile blocker(folder); + QVERIFY(blocker.open(QIODevice::WriteOnly)); + blocker.write("not a folder"); + blocker.close(); + } + + QVERIFY2(!m_window->saveSessionFile(session), + "the session was saved although its takes' audio could not " + "be put beside it"); + QCOMPARE(dialogsMatching("was not saved").size(), 1); + QVERIFY2(!QFileInfo::exists(session), + "a session file was written that names audio which is not " + "in its folder"); + QCOMPARE(m_window->takes()->getTake(0)->audioPath, firstAudio); + QCOMPARE(m_window->takes()->getTake(1)->audioPath, secondAudio); + + // Now the folder can be made, but the audio of the take that is not + // on show has gone from under us: the copy of the first take, which + // was made before the failure, is taken back again and the folder + // made for them is left empty + QVERIFY(QFile::remove(folder)); + QVERIFY(m_window->doSwitchToTake(0)); + QVERIFY2(QFile::remove(secondAudio), + "the audio of the take that was put away is still open"); + + QVERIFY2(!m_window->saveSessionFile(session), + "the session was saved although one take's audio was gone"); + QCOMPARE(dialogsMatching("was not saved").size(), 1); + QVERIFY(!QFileInfo::exists(session)); + QCOMPARE(m_window->takes()->getTake(0)->audioPath, firstAudio); + QCOMPARE(m_window->takes()->getTake(1)->audioPath, secondAudio); + QVERIFY(QDir(folder).exists()); + QVERIFY2(QDir(folder).isEmpty(), + "the copy made before the failure was left in the folder"); + + // An empty folder this run made goes with the session + m_window->doCloseSession(); + QVERIFY2(!QFileInfo::exists(folder), + "the empty takes folder was left behind"); + } + // Saving while the analysis of a recorded range runs: the save waits // for the merge, so the session holds the take's own pitch and notes // and neither the state before the merge nor the run's two temporary diff --git a/main/test/TestTakesFile.h b/main/test/TestTakesFile.h index b84f88f8..223466cd 100644 --- a/main/test/TestTakesFile.h +++ b/main/test/TestTakesFile.h @@ -25,7 +25,9 @@ #include #include +#include #include +#include #include #include @@ -51,6 +53,15 @@ class TestTakesFile : public QObject return TakesFile::readFrom(reader); } + // A file for the copying to work on; nothing here reads what is in it + bool writeAudio(QString path) { + QFile file(path); + if (!file.open(QIODevice::WriteOnly)) return false; + bool ok = (file.write(QByteArray(64, 'x')) == 64); + file.close(); + return ok; + } + QString writeFile(QString content, bool compressed) { QString path = m_dir.filePath (QString("session-%1.ton").arg(++m_fileCounter)); @@ -163,6 +174,267 @@ private slots: QVERIFY(!TakesFile::read("").found); } + // The element names a take's audio relative to the session file when + // it is in the session's own folder (spec 6.4), and where it is + // otherwise -- before the first save, the take is in the record + // directory + void paths_relative_to_the_session() { + SingingTakes takes; + takes.addTake("Take 1"); + takes.restoreTake("C:/songs/My Song.takes/take-1.wav", Coverage()); + takes.addTake("Take 2"); + takes.restoreTake("C:/recorded/take-2.wav", Coverage()); + + QString element = TakesFile::toXml(takes, "C:/songs/My Song.ton"); + TakesFile::Takes read = readString(inDocument(element)); + + QCOMPARE(read.takes[0].audioPath, + QString("My Song.takes/take-1.wav")); + QCOMPARE(read.takes[1].audioPath, QString("C:/recorded/take-2.wav")); + + // What a session load makes of them + QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", + read.takes[0].audioPath), + QString("C:/songs/My Song.takes/take-1.wav")); + + // With no session path the paths are written as they are, which is + // what a session saved before the folder holds + QString absolute = TakesFile::toXml(takes); + QCOMPARE(readString(inDocument(absolute)).takes[0].audioPath, + QString("C:/songs/My Song.takes/take-1.wav")); + } + + // --- The audio folder of a session (spec 6.4) --- + + // ".takes" beside the session file, whatever the session is + // called and whichever slashes its path is written with + void takes_folder() { + QCOMPARE(TakesFile::takesFolder("C:/songs/My Song.ton"), + QString("C:/songs/My Song.takes")); + QCOMPARE(TakesFile::takesFolder("C:\\songs\\My Song.ton"), + QString("C:/songs/My Song.takes")); + QCOMPARE(TakesFile::takesFolder("C:/Käännös/Säkeistö 2.ton"), + QString("C:/Käännös/Säkeistö 2.takes")); + // Only the extension goes, so a name with dots of its own keeps them + QCOMPARE(TakesFile::takesFolder("C:/songs/take.2.ton"), + QString("C:/songs/take.2.takes")); + // A session file at the root of a drive, and one with no directory + QCOMPARE(TakesFile::takesFolder("C:/song.ton"), + QString("C:/song.takes")); + QCOMPARE(TakesFile::takesFolder("song.ton"), QString("song.takes")); + QCOMPARE(TakesFile::takesFolder(""), QString()); + } + + // What goes in the file: relative to the session file when the audio is + // inside its directory, and the path as it is when it is not + void relative_audio_path() { + QCOMPARE(TakesFile::relativeAudioPath + ("C:/songs/My Song.ton", + "C:/songs/My Song.takes/take-1.wav"), + QString("My Song.takes/take-1.wav")); + QCOMPARE(TakesFile::relativeAudioPath + ("C:/songs/My Song.ton", + "C:\\songs\\My Song.takes\\take-1.wav"), + QString("My Song.takes/take-1.wav")); + QCOMPARE(TakesFile::relativeAudioPath + ("C:/Käännös/Säkeistö.ton", + "C:/Käännös/Säkeistö.takes/take-1.wav"), + QString("Säkeistö.takes/take-1.wav")); + + // Not below the session: the record directory before the first + // save, or another drive altogether + QCOMPARE(TakesFile::relativeAudioPath + ("C:/songs/My Song.ton", "C:/recorded/take-1.wav"), + QString("C:/recorded/take-1.wav")); + QCOMPARE(TakesFile::relativeAudioPath + ("C:/songs/My Song.ton", "D:/songs/My Song.takes/take-1.wav"), + QString("D:/songs/My Song.takes/take-1.wav")); + + // A take with no audio in it yet, and no session to be relative to + QCOMPARE(TakesFile::relativeAudioPath("C:/songs/My Song.ton", ""), + QString()); + QCOMPARE(TakesFile::relativeAudioPath("", "C:/recorded/take-1.wav"), + QString("C:/recorded/take-1.wav")); + + // A session file at the root of a drive + QCOMPARE(TakesFile::relativeAudioPath + ("C:/song.ton", "C:/song.takes/take-1.wav"), + QString("song.takes/take-1.wav")); + } + + // And back: a relative path against the session's directory, an + // absolute one -- as a session saved before the folder has -- as it is + void resolve_audio_path() { + QCOMPARE(TakesFile::resolveAudioPath + ("C:/songs/My Song.ton", "My Song.takes/take-1.wav"), + QString("C:/songs/My Song.takes/take-1.wav")); + QCOMPARE(TakesFile::resolveAudioPath + ("C:\\songs\\My Song.ton", "My Song.takes\\take-1.wav"), + QString("C:/songs/My Song.takes/take-1.wav")); + QCOMPARE(TakesFile::resolveAudioPath + ("C:/Käännös/Säkeistö.ton", "Säkeistö.takes/take-1.wav"), + QString("C:/Käännös/Säkeistö.takes/take-1.wav")); + QCOMPARE(TakesFile::resolveAudioPath + ("C:/songs/My Song.ton", "C:/recorded/take-1.wav"), + QString("C:/recorded/take-1.wav")); + QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", ""), + QString()); + + // The session moved, with its folder: the same relative path finds + // the audio in the new place + QCOMPARE(TakesFile::resolveAudioPath + ("D:/backup/songs/My Song.ton", "My Song.takes/take-1.wav"), + QString("D:/backup/songs/My Song.takes/take-1.wav")); + + // Written and read back, for every shape of path + for (QString session : { "C:/songs/My Song.ton", "C:/song.ton", + "C:/Käännös/Säkeistö 2.ton" }) { + for (QString audio : { "take-1.wav", "deeper/take-2.wav" }) { + QString path = QDir::cleanPath + (TakesFile::takesFolder(session) + "/" + audio); + QString stored = TakesFile::relativeAudioPath(session, path); + QVERIFY2(!QFileInfo(stored).isAbsolute(), qPrintable(stored)); + QCOMPARE(TakesFile::resolveAudioPath(session, stored), path); + } + } + } + + void in_folder() { + QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes", + "C:/songs/My Song.takes/take-1.wav")); + // Windows tells no two names apart by their case + QVERIFY(TakesFile::isInFolder("C:/songs/my song.takes", + "C:\\Songs\\My Song.takes\\take-1.wav")); + QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes/", + "C:/songs/My Song.takes/in/take-1.wav")); + QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", + "C:/songs/My Song.takes")); + QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", + "C:/songs/My Song.takes-2/take-1.wav")); + QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", + "C:/recorded/take-1.wav")); + QVERIFY(!TakesFile::isInFolder("", "C:/recorded/take-1.wav")); + QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", "")); + } + + // A copy is never written over a file that is there already + void free_copy_path() { + QString folder = m_dir.filePath("free"); + QVERIFY(QDir().mkpath(folder)); + + QString first = TakesFile::freeCopyPath(folder, "take-1.wav"); + QCOMPARE(first, QDir::cleanPath(folder + "/take-1.wav")); + + QVERIFY(writeAudio(first)); + QString second = TakesFile::freeCopyPath(folder, "take-1.wav"); + QCOMPARE(second, QDir::cleanPath(folder + "/take-1-2.wav")); + + QVERIFY(writeAudio(second)); + QCOMPARE(TakesFile::freeCopyPath(folder, "take-1.wav"), + QDir::cleanPath(folder + "/take-1-3.wav")); + + QCOMPARE(TakesFile::freeCopyPath("", "take-1.wav"), QString()); + QCOMPARE(TakesFile::freeCopyPath(folder, ""), QString()); + } + + // The copy a save makes: one per file, the takes pointing at the + // copies, and the copies counted as files this run wrote + void copy_take_audio_into_the_folder() { + QString record = m_dir.filePath("rec"); + QString folder = m_dir.filePath("copied.takes"); + QVERIFY(QDir().mkpath(record)); + QVERIFY(QDir().mkpath(folder)); + + QString one = QDir::cleanPath(record + "/take-1.wav"); + QString two = QDir::cleanPath(record + "/take-2.wav"); + QString inside = QDir::cleanPath(folder + "/take-3.wav"); + QVERIFY(writeAudio(one)); + QVERIFY(writeAudio(two)); + QVERIFY(writeAudio(inside)); + + SingingTakes takes; + takes.addTake("A"); + takes.restoreTake(one, Coverage()); + // Two takes sharing one file, as a duplicated take does + takes.addTake("B"); + takes.restoreTake(one, Coverage()); + takes.addTake("C"); + takes.restoreTake(two, Coverage()); + // And one whose audio is in the folder already + takes.addTake("D"); + takes.restoreTake(inside, Coverage()); + + QCOMPARE(TakesFile::copyTakeAudioInto(takes, folder), QString()); + + QString copyOfOne = QDir::cleanPath(folder + "/take-1.wav"); + QString copyOfTwo = QDir::cleanPath(folder + "/take-2.wav"); + QCOMPARE(takes.getTake(0)->audioPath, copyOfOne); + QCOMPARE(takes.getTake(1)->audioPath, copyOfOne); + QCOMPARE(takes.getTake(2)->audioPath, copyOfTwo); + QCOMPARE(takes.getTake(3)->audioPath, inside); + QVERIFY(QFileInfo::exists(copyOfOne)); + QVERIFY(QFileInfo::exists(copyOfTwo)); + + // The shared file was copied once: a second copy would have been + // "take-1-2.wav" + QVERIFY2(!QFileInfo::exists(QDir::cleanPath(folder + "/take-1-2.wav")), + "a file two takes share was copied twice"); + + // The originals are still there, for the models that have them open + QVERIFY(QFileInfo::exists(one)); + QVERIFY(QFileInfo::exists(two)); + + // The copies are ours, so the cleanup on close may take them away + // again if no saved session names them + QCOMPARE(takes.getWrittenPaths(), QStringList({ copyOfOne, copyOfTwo })); + + // Saving again copies nothing: everything is in the folder + QCOMPARE(TakesFile::copyTakeAudioInto(takes, folder), QString()); + QCOMPARE(takes.getTake(0)->audioPath, copyOfOne); + QCOMPARE(takes.getWrittenPaths(), QStringList({ copyOfOne, copyOfTwo })); + } + + // A copy that fails leaves the takes exactly as they were, and takes + // the copies that were made with it: better no saved session than one + // that names files which are not there + void a_failed_copy_changes_nothing() { + QString record = m_dir.filePath("rec-failing"); + QVERIFY(QDir().mkpath(record)); + + QString one = QDir::cleanPath(record + "/take-1.wav"); + QVERIFY(writeAudio(one)); + QString missing = QDir::cleanPath(record + "/not-there.wav"); + + SingingTakes takes; + takes.addTake("A"); + takes.restoreTake(one, Coverage()); + takes.addTake("B"); + takes.restoreTake(missing, Coverage()); + + // No such folder to copy into + QString folder = m_dir.filePath("nowhere.takes"); + QString error = TakesFile::copyTakeAudioInto(takes, folder); + QVERIFY2(error != "", "copying into a folder that is not there " + "reported no error"); + QCOMPARE(takes.getTake(0)->audioPath, one); + QCOMPARE(takes.getTake(1)->audioPath, missing); + QVERIFY(takes.getWrittenPaths().isEmpty()); + + // The folder is there, but the second take's audio is not: the + // first take's copy is removed again + QVERIFY(QDir().mkpath(folder)); + error = TakesFile::copyTakeAudioInto(takes, folder); + QVERIFY2(error != "", "copying a file that is not there reported no " + "error"); + QCOMPARE(takes.getTake(0)->audioPath, one); + QCOMPARE(takes.getTake(1)->audioPath, missing); + QVERIFY(takes.getWrittenPaths().isEmpty()); + QVERIFY2(!QFileInfo::exists(QDir::cleanPath(folder + "/take-1.wav")), + "a copy made before the failure was left behind"); + + QVERIFY(TakesFile::copyTakeAudioInto(takes, "") != ""); + } + // The numbering carries on from the takes a session had, whatever has // happened to them since void reserved_names_carry_the_numbering_on() { From aff04a6328fcf1349b3c05ed1fa8bfae2ec86d6b Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 20:36:14 +0300 Subject: [PATCH 053/275] docs: the singing practice features in the README The fork's singing practice mode and the partial recordings and takes work were described only in the untracked notes under tmp/. Say in the README what a user gets: recording from the playhead, the pre-roll, recording into a selection, the coverage strip and erasing, undo, several takes and the session's takes folder. Co-Authored-By: Claude Opus 5 --- README.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/README.md b/README.md index f8352299..ad8ed9ca 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,31 @@ Features * import/export of pitch track and note track +Singing practice +---------------- + +This fork adds a singing practice mode: a reference recording is loaded and +analysed as usual, and your own singing is recorded alongside it as a second, +orange pitch track to compare with it. + + * record from the playback position, not only from the start of the song, so + you can practise one phrase without singing everything before it + * an optional pre-roll: the reference starts a few seconds early and the + status bar counts you in, and nothing sung during the lead-in is kept + * an optional "record into the selection only": select the phrase, and the + recording starts and stops at the ends of the selection by itself + * a take is one recording or many. A strip along the bottom of the pane shows + where there is singing and where there is not; recording over a part of it + replaces just that part, and only the part that changed is analysed again + * Edit -> Select Recording at Playhead and Edit -> Erase Singing in Selection + remove, trim or split what has been recorded + * recordings and erases can be undone and redone + * several takes of a song in one session (the Takes menu and the "Take:" box in + the toolbar): each keeps its own audio, pitch track and notes, and switching + between them needs no re-analysis. The audio of a session's takes is kept in + a folder named after the session beside it, so the two can be moved together + + Authors, Citation, License and Use ---------------------------------- From e6d03c07c29240bcbab64c770558b689178747ea Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 20:48:42 +0300 Subject: [PATCH 054/275] feat: choose the audio device, and see ones connected since startup Tony never passed a device name to bqaudioio, so playback always went to whatever PortAudio reported as the default. Worse, PortAudio enumerates devices once, at Pa_Initialize, and bqaudioio holds it initialised for as long as an audio IO object exists -- which Tony creates on the first file open and never rebuilds. A Bluetooth speaker connected mid-session was therefore not in the device list at all, and no restart-free route to it existed. Add Audio Output Device and Audio Input Device submenus under Playback, writing the audio-playback-device and audio-record-device settings that createAudioIO() already read but nothing ever wrote. Opening either menu tears the IO down, re-enumerates and rebuilds it, so the list reflects what is connected now; that is skipped while recording, so a take cannot be dropped. A chosen device that has since disappeared stays listed as "(not connected)" rather than silently reverting to the default. Co-Authored-By: Claude Opus 5 --- main/MainWindow.cpp | 200 ++++++++++++++++++++++++++++++++++++++++++++ main/MainWindow.h | 13 +++ 2 files changed, 213 insertions(+) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 6b16fc48..6c263d79 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -27,6 +27,8 @@ #include "framework/Document.h" #include "framework/VersionTester.h" +#include + #include "view/Pane.h" #include "view/PaneStack.h" #include "data/model/WaveFileModel.h" @@ -171,6 +173,10 @@ MainWindow::MainWindow(AudioMode audioMode, m_recentFilesMenu(0), m_rightButtonMenu(0), m_rightButtonPlaybackMenu(0), + m_audioDeviceMenu(0), + m_audioDeviceGroup(0), + m_audioInputDeviceMenu(0), + m_audioInputDeviceGroup(0), m_deleteSelectedAction(0), m_ffwdAction(0), m_rwdAction(0), @@ -1264,6 +1270,176 @@ MainWindow::setupRecentFilesMenu() } } +namespace { + +// The audio-device preference keys that MainWindowBase::createAudioIO() +// reads. The key is suffixed with the preferred driver when one has been +// pinned, so that a device name chosen for (say) JACK is not then offered +// to PortAudio. +static QString +audioDeviceSettingKey(QString base) +{ + QSettings settings; + settings.beginGroup("Preferences"); + QString implementation = settings.value("audio-target", "").toString(); + settings.endGroup(); + if (implementation == "" || implementation == "auto") return base; + return base + "-" + implementation; +} + +// Which bqaudioio implementation the device names should come from. The +// factory can only enumerate devices for a named implementation, so when +// the driver is left on "auto" we ask the only one that was built in -- +// which on Windows and macOS is always PortAudio. +static std::string +audioImplementationName() +{ + QSettings settings; + settings.beginGroup("Preferences"); + QString implementation = settings.value("audio-target", "").toString(); + settings.endGroup(); + + if (implementation != "" && implementation != "auto") { + return implementation.toStdString(); + } + + std::vector available = + breakfastquay::AudioFactory::getImplementationNames(); + if (available.size() == 1) return available[0]; + return {}; +} + +} + +void +MainWindow::buildAudioDeviceMenu(QMenu *menu, + QActionGroup *group, + const std::vector &names, + QString settingKey) +{ + QSettings settings; + settings.beginGroup("Preferences"); + QString current = settings.value(settingKey, "").toString(); + settings.endGroup(); + + for (QAction *a: group->actions()) { + group->removeAction(a); + } + menu->clear(); + + QAction *defaultAction = menu->addAction(tr("(System Default)")); + defaultAction->setCheckable(true); + defaultAction->setData(QString()); + defaultAction->setChecked(current == ""); + group->addAction(defaultAction); + + if (names.empty()) { + menu->addSeparator(); + menu->addAction(tr("(No devices found)"))->setEnabled(false); + return; + } + + menu->addSeparator(); + + bool haveCurrent = false; + + for (const std::string &name: names) { + QString qname = QString::fromStdString(name); + QAction *action = menu->addAction(qname); + action->setCheckable(true); + action->setData(qname); + if (qname == current) { + action->setChecked(true); + haveCurrent = true; + } + group->addAction(action); + } + + if (current != "" && !haveCurrent) { + // The chosen device has gone away. Show it anyway, marked as + // absent, rather than silently reverting the tick to the system + // default -- the setting is still in force and will take effect + // again when the device comes back. + menu->addSeparator(); + QAction *missing = menu->addAction + (tr("%1 (not connected)").arg(current)); + missing->setCheckable(true); + missing->setChecked(true); + missing->setData(current); + group->addAction(missing); + } +} + +void +MainWindow::rescanAudioDevices() +{ + if (!m_audioDeviceMenu || !m_audioInputDeviceMenu) return; + + // PortAudio enumerates the system's devices once, when it is + // initialised, and bqaudioio keeps it initialised for as long as an + // audio IO object exists. So a device that appeared after Tony started + // -- a Bluetooth speaker connected mid-session, typically -- is not in + // the list at all until the IO is torn down and rebuilt. Do that here, + // around the enumeration, so that opening either of these menus always + // shows what is actually connected now. + // + // Deleting the IO mid-recording would drop the take, so in that case + // leave the list as it stands. With no IO open there is nothing to + // tear down: the enumeration below initialises PortAudio itself and so + // gets a current list anyway. + bool canRebuild = (m_playTarget || m_audioIO) && + !(m_recordTarget && m_recordTarget->isRecording()); + + if (canRebuild) { + if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + deleteAudioIO(); + } + + std::string implementation = audioImplementationName(); + + std::vector playbackNames = + breakfastquay::AudioFactory::getPlaybackDeviceNames(implementation); + std::vector recordNames = + breakfastquay::AudioFactory::getRecordDeviceNames(implementation); + + if (canRebuild) { + createAudioIO(); + } + + buildAudioDeviceMenu(m_audioDeviceMenu, + m_audioDeviceGroup, + playbackNames, + audioDeviceSettingKey("audio-playback-device")); + + buildAudioDeviceMenu(m_audioInputDeviceMenu, + m_audioInputDeviceGroup, + recordNames, + audioDeviceSettingKey("audio-record-device")); +} + +void +MainWindow::audioDeviceSelected(QAction *action) +{ + if (!action) return; + + QString key = audioDeviceSettingKey + (m_audioInputDeviceGroup->actions().contains(action) ? + "audio-record-device" : "audio-playback-device"); + + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(key, action->data().toString()); + settings.endGroup(); + + if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + + recreateAudioIO(); +} + void MainWindow::setupToolbars() { @@ -1436,6 +1612,30 @@ MainWindow::setupToolbars() menu->addAction(recordAction); menu->addSeparator(); + m_audioDeviceMenu = menu->addMenu(tr("Audio Output &Device")); + m_audioDeviceMenu->setStatusTip(tr("Choose which device Tony plays through")); + m_audioDeviceGroup = new QActionGroup(this); + m_audioDeviceGroup->setExclusive(true); + + m_audioInputDeviceMenu = menu->addMenu(tr("Audio &Input Device")); + m_audioInputDeviceMenu->setStatusTip(tr("Choose which device Tony records from")); + m_audioInputDeviceGroup = new QActionGroup(this); + m_audioInputDeviceGroup->setExclusive(true); + + for (QMenu *m: { m_audioDeviceMenu, m_audioInputDeviceMenu }) { + connect(m, SIGNAL(aboutToShow()), this, SLOT(rescanAudioDevices())); + // Placeholder so that the submenu is not empty: an empty submenu is + // drawn disabled and never emits aboutToShow, which is where the + // real device list is built. + m->addAction(tr("(Scanning...)"))->setEnabled(false); + } + + for (QActionGroup *g: { m_audioDeviceGroup, m_audioInputDeviceGroup }) { + connect(g, SIGNAL(triggered(QAction *)), + this, SLOT(audioDeviceSelected(QAction *))); + } + menu->addSeparator(); + m_rightButtonPlaybackMenu->addAction(playAction); m_rightButtonPlaybackMenu->addAction(psAction); m_rightButtonPlaybackMenu->addAction(plAction); diff --git a/main/MainWindow.h b/main/MainWindow.h index 2944c9b1..f8cb3c11 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -26,12 +26,14 @@ #include "TakeTiming.h" #include +#include #include #include "data/model/SparseTimeValueModel.h" class QTimer; class QComboBox; +class QActionGroup; namespace sv { class VersionTester; @@ -251,6 +253,9 @@ protected slots: virtual void setupRecentFilesMenu(); + virtual void rescanAudioDevices(); + virtual void audioDeviceSelected(QAction *); + virtual void handleOSCMessage(const sv::OSCMessage &); virtual void mouseEnteredWidget(); @@ -619,6 +624,10 @@ protected slots: QMenu *m_recentFilesMenu; QMenu *m_rightButtonMenu; QMenu *m_rightButtonPlaybackMenu; + QMenu *m_audioDeviceMenu; + QActionGroup *m_audioDeviceGroup; + QMenu *m_audioInputDeviceMenu; + QActionGroup *m_audioInputDeviceGroup; QAction *m_deleteSelectedAction; QAction *m_ffwdAction; @@ -669,6 +678,10 @@ protected slots: virtual void setupAnalysisMenu(); virtual void setupHelpMenu(); virtual void setupToolbars(); + virtual void buildAudioDeviceMenu(QMenu *menu, + QActionGroup *group, + const std::vector &names, + QString settingKey); // Helpers for the singing / second-track workflow. // deferAnalysis=true skips pYIN: swapSingingAudio() uses it, as the From e5c26e236c0c2927cb89cfcf145737760b379858 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 21:33:32 +0300 Subject: [PATCH 055/275] docs: agent instructions and developer docs for the singing features AGENTS.md holds the build and test commands and the working rules, with CLAUDE.md importing it and .github/copilot-instructions.md pointing at it. docs/ keeps what the code cannot say for itself, distilled from the working notes in the gitignored tmp/: why the record and Stop paths are ordered as they are, the rules of the SV libraries the code rests on, the takes design and its limitations, the library forks, the test fakes and pitfalls, open points and the manual checklist. Co-Authored-By: Claude Fable 5.1 --- .github/copilot-instructions.md | 19 +++ AGENTS.md | 125 +++++++++++++++ CLAUDE.md | 1 + docs/README.md | 21 +++ docs/architecture.md | 193 +++++++++++++++++++++++ docs/building.md | 81 ++++++++++ docs/forks.md | 96 ++++++++++++ docs/manual-checklist.md | 103 +++++++++++++ docs/open-points.md | 44 ++++++ docs/recording.md | 147 ++++++++++++++++++ docs/takes.md | 261 ++++++++++++++++++++++++++++++++ docs/testing.md | 174 +++++++++++++++++++++ 12 files changed, 1265 insertions(+) create mode 100644 .github/copilot-instructions.md create mode 100644 AGENTS.md create mode 100644 CLAUDE.md create mode 100644 docs/README.md create mode 100644 docs/architecture.md create mode 100644 docs/building.md create mode 100644 docs/forks.md create mode 100644 docs/manual-checklist.md create mode 100644 docs/open-points.md create mode 100644 docs/recording.md create mode 100644 docs/takes.md create mode 100644 docs/testing.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..fbb18bf2 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,19 @@ +# Copilot instructions + +The instructions for AI agents in this repository are in [`AGENTS.md`](../AGENTS.md) at +the repository root, and the documentation they point to is in [`docs/`](../docs/). Read +`AGENTS.md` first and follow it; this file exists only so that tools which look here find +their way there. + +The essentials, for a context that cannot follow the link: + +- Fork of Tony (Qt6 / C++17, meson + ninja, Sonic Visualiser libraries) adding singing + practice features. The fork's own code is in `main/`; tests are in `main/test/`. +- Build from Git Bash with + `export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64"` and + `ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe`, logging to a + file under `tmp/`. From PowerShell use `.\build.bat` and `.\build.bat test`. +- `svcore/`, `svgui/`, `svapp/` and the other top-level library directories are separate, + gitignored repositories. See `docs/forks.md` before changing one. +- Match the surrounding code, keep changes in scope, and give every behaviour a test that + can fail. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..11556e7e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,125 @@ +# Instructions for AI agents + +This is a fork of [Tony](https://github.com/sonic-visualiser/tony) (Qt6 / C++17, meson + +ninja, built on the Sonic Visualiser libraries) that turns it into a **singing practice +aid**: a reference pitch track and a singing pitch track in one pane, live pitch dots +while recording, partial recordings, takes, undo. Development happens on Windows with +MSYS2 MinGW-w64. The default branch is `default`. + +All of the fork's own code is in `main/`. Detailed documentation is in [`docs/`](docs/); +**read the page for the area you are about to change before changing it** — the rules +there were each learned from a crash or a wrong result, and most cannot be seen in the +code. + +| Read | Before | +| --- | --- | +| [docs/building.md](docs/building.md) | building for the first time in a session | +| [docs/testing.md](docs/testing.md) | running or writing tests | +| [docs/architecture.md](docs/architecture.md) | touching layers, models, the document, commands, playback or the session file | +| [docs/recording.md](docs/recording.md) | touching `record()`, the Stop path, latency, pre-roll, the live tracker | +| [docs/takes.md](docs/takes.md) | touching takes, the audio swap, ranged analysis, undo, the coverage strip, save/restore | +| [docs/forks.md](docs/forks.md) | needing a change in `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` | +| [docs/open-points.md](docs/open-points.md), [docs/manual-checklist.md](docs/manual-checklist.md) | choosing what to do next, or saying what the user should try by hand | + +## Build and test + +From **Git Bash** (the usual agent shell). `build.bat` does not run from sh. + +```sh +export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe > tmp/build.log 2>&1 +echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log +``` + +- Only `mingw64/bin` on PATH (never `/c/msys64/usr/bin`); always set `MINGW_PREFIX`, + spelled exactly so; always `-j 3`; always log to a file and never pipe ninja; targets + need `.exe`. The reasons are in [docs/building.md](docs/building.md). +- Incremental builds take under a minute, a few minutes after `MainWindow.cpp`; a clean + build up to 30 minutes. If `cc1plus.exe` runs out of memory, run the command again. + +Run tests from `build_mingw/` with the same environment: + +```sh +mkdir -p ../tmp/tl +TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core.exe > ../tmp/test.log 2>&1; echo "exit:$?" +TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe > ../tmp/test.log 2>&1; echo "exit:$?" # ~5 min, real time +TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe undo_two_takes_in_order > ../tmp/test.log 2>&1 +grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt +``` + +- Read results from the per-suite files in `TONY_TEST_LOG_DIR`, not from stdout. +- A test name on the command line goes to **every** suite in the executable; the suites + that lack it fail, so the exit status is only meaningful for a run with no names. +- Run named tests while working; run **both whole suites** before calling anything done. +- From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. Its + 300 s timeout for `tony-app` is nearly used up; a "timeout" there is not a test failure + ([docs/testing.md](docs/testing.md)). +- Give the app suite a tool timeout of 10 minutes. + +## Rules for working here + +### Scope + +- Build what was asked. Where the request is silent, take the simpler option and say so. + No drive-by refactors. If the plan turns out wrong or impossible, do not improvise + another design: finish what can be finished, leave the tree building and green, report. +- `main/MainWindow.cpp` is over 6000 lines and `main/test/TestRecordWorkflow.h` over 2000. + Never read them whole: search, then read a range. +- `svcore/`, `svgui/`, `svapp/`, `pyin/` and the other top-level library directories are + **separate git repositories, gitignored here**, so ripgrep-based search tools skip them + unless given the directory explicitly. Four are forks that may be changed + ([docs/forks.md](docs/forks.md)); the rest are upstream and stay untouched. A sub-agent + does not edit a fork: it reports the change it needs. + +### Code + +- Match the surrounding code: naming, comment density, idiom. Comments say *why*, in plain + words. Every source file starts with the project's GPL header. +- State gets a class and files of its own; `MainWindow` only wires it + (`AlternatePitchTrack`, `CoverageStrip`, `TakeLayers`, `TakeCommands` are the pattern). +- Logic that needs no window goes in `tony_core` as pure functions or plain structs + (`TakeTiming`, `TakeEvents`), where it is cheap to test. +- The handful of rules most often broken — each explained in + [docs/architecture.md](docs/architecture.md): + - Tear a layer down with `m_document->deleteLayer(layer, true)` and nothing else. Never + `removeLayerFromView()` first; never `ModelById::release()` a model the document knows. + - Add Tony's own layers with `Document::attachLayerToView()`, never `addLayerToView()`. + - Never push a command, or call `openPath()`, from anything reachable from an undo or + redo. Commands hold values, never layer or model pointers. + - Remove a pane only with `pruneExtraPane()`, and only after another layer holds the model. + - Temporary mute/hide goes straight to the play parameters / `showLayer()`; + `Analyser::setAudible()` / `setVisible()` write settings shared by both analysers. + - Use member-pointer `connect`; string-based connects with `sv::` types fail silently. + - Stop `RealtimePitchTracker` before releasing the model it reads. All model writes + happen on the GUI thread. + +### Tests + +- Every behaviour gets a test **that can fail**. For the ones that matter, prove it: break + the code briefly, see the test fail, put the code back, and say which you checked. +- Never weaken or delete a test to get green. If the behaviour was meant to change, change + the test and say so. +- App tests run in real time against a fake audio device: keep them short. + +### Git + +- Commit only when asked, and only with both whole suites green. One commit per coherent + step, staged by file name (never `git add -A`; `.claude/` and `tmp/` stay out). +- Messages: `feat:` / `fix:` / `test:` / `docs:`, lower case, then a short what-and-why + body. End with a `Co-Authored-By` trailer naming the model that wrote the code. +- Use the `gh` CLI for anything on GitHub. + +### Reporting + +Say what was built, by file; copy the `Totals` lines of the final full test runs verbatim; +say which tests were seen to fail; list deviations and anything fragile or unfinished. A +problem reported is cheap, one found later is not. If a change needs a real device or real +eyes to judge, name the items of [docs/manual-checklist.md](docs/manual-checklist.md) the +user should try. + +### Keeping the docs true + +`docs/` records what the code cannot say: why, in what order, and what must not be done. +When a change makes a statement there false, fix it in the same commit. Do not add +narratives of fixed bugs, lists of members or methods, or commit hashes — the code and +`git log` have those. `tmp/` is gitignored scratch space for logs and working notes. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..43c994c2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..6eea96bc --- /dev/null +++ b/docs/README.md @@ -0,0 +1,21 @@ +# Developer documentation + +For people and AI agents developing the singing-practice features of this Tony fork. What +the features *are*, for a user, is in the [top-level README](../README.md); how to work in +the repository is in [AGENTS.md](../AGENTS.md). + +These pages hold what the code cannot say for itself: why things are in the order they are +in, which rules of the Sonic Visualiser libraries the code depends on, what was decided +and rejected, and what is known to be weak. They do not list classes, members or +methods, and they do not tell the story of fixed bugs. + +| Page | What is in it | +| --- | --- | +| [building.md](building.md) | The MinGW build, and every way the environment has gone wrong | +| [testing.md](testing.md) | The two test executables, the fakes and helpers, how tests turned out to be worthless, races | +| [architecture.md](architecture.md) | What the fork adds, `tony_core` / `tony_app`, who owns what, and the rules for layers, models, commands, playback and session files | +| [recording.md](recording.md) | Record to Stop, step by step and why in that order; latency; pre-roll; record into selection; the live tracker | +| [takes.md](takes.md) | Takes: decisions, the audio swap, ranged analysis and merge, undo, the coverage strip, files and sessions, limitations | +| [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | +| [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | +| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..64af7bdc --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,193 @@ +# Architecture of the singing-practice fork + +What this fork adds to upstream Tony, who owns what, and the rules of the Sonic Visualiser +(SV) libraries that the code depends on and that are not visible from Tony's own sources. +Member lists, slots and method names are in the headers; they are not repeated here. + +## What the fork is for + +Upstream Tony analyses the pitch of one recording. This fork makes it a singing practice aid: + +1. **Two pitch tracks in one pane**: the reference (black) and the singing (orange). +2. **Live pitch dots** while recording, from a fast YIN tracker on its own thread. +3. **pYIN analysis of what was just recorded** replaces the dots when the take stops. +4. **Partial recordings and takes**: record from the playhead into part of the song, keep + the rest, erase, undo, and keep several takes. See [takes.md](takes.md). +5. Around that: play the reference while recording, latency compensation, pre-roll, + record into selection, an octave-shifted "alternate" pitch track to follow, and a + background music track that is played but never analysed. + +The user-facing description is in the [README](../README.md). + +## Where things are + +Everything of Tony's own is in `main/`. The sibling directories (`svcore/`, `svgui/`, +`svapp/`, `pyin/`, `bq*/` ...) are separate repositories checked out by repoint and +**gitignored here**; see [forks.md](forks.md). + +`meson.build` splits `main/` into two static libraries so the two test executables link +only what they need: + +| Library | Rule | Contents | +| --- | --- | --- | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h` | +| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `TakeCommands`, `TakeLayers`, `PaneUtils` | + +When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` +list **only if** it has `Q_OBJECT`. Logic that can be written as pure functions or a plain +struct goes in `tony_core` so that it can be tested cheaply — `TakeTiming` (all the frame +arithmetic of a take) and `TakeEvents` (what an erase does to events) are the models to +follow. `MainWindow` then only fills the struct in and puts the answer on screen. + +## Who owns what + +- `MainWindow` has two `Analyser`s: `m_analyser` for the reference (the document's main + model) and `m_analyser2` for the **active** singing take. `Analyser` takes a + `ColorScheme`; the secondary one makes no spectrogram and mutes its pitch and notes. +- **Both analysers share pane 0.** Pane 1 is the time ruler. Never put the singing track + in a pane of its own. +- An `Analyser` recognises a pitch or notes layer as its own when the layer's model has + the analyser's audio model as its **source model**. This one rule is what session + restore, the audio swap and take switching all rest on, and it is why layers of + inactive takes have their source model cleared (see [takes.md](takes.md)). +- `Analyser::fileClosed()` clears the layers but **not** `m_fileModel`. +- Helper objects that own one layer each and are only wired by `MainWindow`: + `AlternatePitchTrack`, `CoverageStrip`. Both watch `Document::layerAboutToBeDeleted` + in case someone else deletes their layer, and both must be deleted in `~MainWindow` + **before** the base class deletes the document (as must `m_analyser2`). +- `RealtimePitchTracker` is a `QThread` that only **reads** the recording's + `WritableWaveFileModel` and emits `pitchDetected(frame, hz)`. It never touches the pitch + model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued + connection). Stop the tracker **before** releasing the model it reads. + +Colours: reference pitch black / notes bright blue; singing and live dots orange / notes +bright purple; alternate pitch faded brown, dark brown while a take is recorded. + +## Rules of the SV libraries + +These were all learned from crashes or wrong behaviour. They hold for any new code. + +### Models and layers + +- `Document` owns all models and layers. Register a model with `addNonDerivedModel(id)`. + **Never call `ModelById::release()` on a model the document knows** — double free. +- `Document::releaseModel()` silently does nothing while any layer still references the + model. So *some layer must hold a model* for as long as it is wanted: that is the only + reason the recording and the background music have hidden waveform layers. +- Silent teardown is **`deleteLayer(layer, true)` and nothing else**. It removes the layer + from every view, releases the model if unreferenced and deletes the layer. +- **Never `removeLayerFromView()` followed by `deleteLayer()`** on the same layer: + the first pushes a `RemoveLayerCommand` holding a raw pointer onto the undo stack. +- `Pane::removeLayer()` / `View::removeLayer()` do **not** update + `Document::m_layerViewMap`. Deleting a pane whose layers are still in that map leaves + a dangling `View*` that the next `deleteLayer()` dereferences. Remove a pane only with + `pruneExtraPane()` (`main/PaneUtils.cpp`), which force-deletes the layer that owns the + given model and `detachLayerFromView()`s the shared ones (time ruler) first. + Precondition: another layer already references that model. +- `createLayer()` + `setModel()`, not `createEmptyLayer()`: the latter makes a throwaway + model that enters the play source. +- `View::removeLayer()` fails to disconnect `layerMeasurementRectsChanged`; + `TakeLayers::raise()` does it by hand. + +### Tony's own layers make no undo commands + +Every layer Tony makes for itself — analysers' layers, live dots, the recording's hidden +waveform, the alternate pitch track, the coverage strip, background music — is added with +**`Document::attachLayerToView()`** (svapp fork): in the view and in the layer-view map, so +the session keeps it, but no command and no modified flag. `addLayerToView()` (the +undoable Add Layer) must not be used for these: Undo after a take has to find the take. +Whoever attaches the layer calls `documentModified()` if the change should count. + +### Commands + +- `CommandHistory::addCommand()` clears **and deletes** the redo stack. A command pushed + while an undo or redo is running therefore destroys the command that is running. + Nothing reachable from `execute()` / `unexecute()` may push a command. +- `openPath()` pushes an `AddPaneCommand`. That is why a take's audio is opened with + `MainWindow::openTakeAudioFile()` (a `ReadOnlyWaveFileModel` + `addNonDerivedModel()`, + no pane, no layer, no command, no Recent Files entry) and never `openPath()`. +- A command whose work is already done is pushed with `addCommand(command, false)`. +- Commands hold **values only** (paths, `Coverage`, event lists), never layer or model + pointers: by the time an undo runs the models have been made again several times. + +### Playback + +- The play source takes in the model of **every layer in a view**, playable or not, and + what it holds decides where playback ends. Models that must not extend playback + (coverage strip, layers of inactive takes) are taken out with `m_playSource->removeModel()`. +- The svapp fork emits `Document::modelAboutToBeReleased(ModelId)` and `MainWindowBase` + removes the model from the play source on it. Upstream only did so from + `RemoveLayerCommand`, which forced deletes never run. +- Mute with `getPlayParameters()->setPlayAudible(false)` directly. `Analyser::setAudible()` + and `Analyser::setVisible()` **write QSettings keys that both analysers share**; use them + only for the user's own toggles, never for temporary states such as "during a take". + For temporary hiding use `showLayer(pane, false)`. + +### Selection and tools + +- Tony's pane stack is built with `NoPropertyStacks`, so `Pane::getSelectedLayer()` is + always null and **`Analyser::stackLayers()` / `PaneStack::setCurrentLayer()` do nothing**. + A tool acts on the topmost layer of its kind (`Pane::getTopFlexiNoteLayer()`, which + skips dormant layers in the svgui fork). Layer order is changed with `TakeLayers::raise()`. +- Making any selection starts `Analyser::reAnalyseSelection()` on the reference, so a + transformer is usually running afterwards. Tests cannot assert + `!haveRunningTransformers()` after selecting. + +### Signals + +- Connect with **member pointers**, not `SIGNAL()`/`SLOT()` strings, for anything whose + signature has `sv::` types when the receiving class is outside namespace `sv`: the + string form never matches and fails silently at run time (this kept + `Analyser::layerAboutToBeDeleted` from ever being called). +- `audioFileLoaded()` is emitted for `CreateAdditionalModel` too (singing track, background + music). `analyseNewMainModel()` returns early if the main model is the one it already + analysed (`m_analysedMainModelId`); handing the reference to `m_analyser` twice forgets + its pitch candidates and double-connects `regionOutlined()`. +- `recordStatusChanged(true)` fires inside `startRecording()`, **before** the base class + has added the recording's model to the document. Work that needs the model is deferred + with `QTimer::singleShot(0)`. + +### Session files + +- `Document::toXml()` writes a model only if a layer **in a view** uses it. Layers that + store data (inactive takes) therefore stay in pane 0, hidden. +- A layer with `setSavedInSession(false)` (svgui/svapp forks) is left out, as is a model + that only such layers show. Used for layers Tony makes again by itself on load. +- `SVFileReader` only warns about unknown elements, which is what lets the `.ton` carry + Tony's `` element. +- An event's value is written with six significant figures. After a round trip compare + frames exactly and values with a tolerance. +- Layer **object names carry identity** across a save: `"Alternate Pitch Track -1"` holds + the octave count, `"Take 2 Pitch"` links a layer to its take. They are not translated. + +### Miscellaneous + +- `floatvec_t` (bqvec allocator) is not `std::vector`; copy before handing to code + that wants the latter. +- `ModelById` is in `data/model/Model.h` in the pinned svcore. +- `WritableWaveFileModel::addSamples()` does not update the read view; the record target + calls `updateModel()` on a timer (10 ms in the svapp fork, ~200 ms upstream). Readers + of a recording in progress poll and wait. +- The build force-includes `main/mingw_byte_fix.h` to resolve the MinGW C++17 + `std::byte` / `byte` clash; do not remove it. + +## Smaller features + +**Alternate pitch track** (`AlternatePitchTrack`): a copy of the reference pitch model +shifted by -3..+3 octaves (never 0), in a model of its own with **no source model** so no +analyser claims it. It rebuilds 50 ms after a burst of changes to the source (queued: +pYIN writes from its own thread), and `MainWindow::syncAlternatePitchTrack()` re-points +it on `m_analyser::layersChanged()`, because Analyse Now replaces the reference's pitch +layer and model. Found again after a session load by its object name (`adopt()`). During +a take the reference pitch layer is hidden with `showLayer()` and comes back the moment +recording stops. + +**Background music**: loaded with `openPath(CreateAdditionalModel)` under the +`m_loadingBackgroundMusic` flag so `modelAdded()` does not take it for a singing track; +given a hidden waveform layer of Tony's own **before** the extra pane is pruned; not +saved in the session. + +**Load Singing Track** follows the same order: `analyseNewSingingModel()` synchronously +after `openPath()`, and only then prune the extra pane — the imported waveform in that pane +is the only reference to the model until `m_analyser2` has a layer of its own. It ends with +`clearTakeHistory()`, which also disposes of the "Import" command for the pruned pane. diff --git a/docs/building.md b/docs/building.md new file mode 100644 index 00000000..8d77e1d7 --- /dev/null +++ b/docs/building.md @@ -0,0 +1,81 @@ +# Building on Windows (MSYS2 MinGW-w64) + +The development machine builds with meson + ninja under MSYS2's `mingw64` toolchain +(default prefix `C:\msys64\mingw64`) into `build_mingw/`. The CI workflows in +`.github/workflows/` build the upstream way on Linux, macOS and MSVC and are not what is +described here. + +## From cmd or PowerShell: `build.bat` + +``` +.\build.bat build Tony.exe +.\build.bat run build, then launch +.\build.bat launch launch without building +.\build.bat test meson test: tony-core, tony-app and four svcore suites +.\build.bat clean wipe build_mingw and reconfigure (only for a broken build directory) +``` + +It reads `MSYS2_MINGW` and puts `mingw64\bin` and msys2's `usr\bin` on `PATH` itself. +From PowerShell its output is safe to capture: `.\build.bat *> tmp\build.log`. + +## From Git Bash (what an agent's Bash tool usually is) + +`build.bat` cannot be called from sh. Call ninja directly: + +```sh +export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe > tmp/build.log 2>&1 +echo "exit:$?" >> tmp/build.log +tail -20 tmp/build.log +``` + +Each part of that is there because of something that went wrong: + +- **Only `mingw64/bin` on PATH, not `/c/msys64/usr/bin`.** With the latter, msys2's + coreutils shadow Git's and run under a different msys runtime: quoted arguments held in + variables arrive empty (`grep: Usage...`, `mkdir: missing operand`). ninja, meson, gcc, + moc and the Qt DLLs are all under `mingw64`. +- **`MINGW_PREFIX` must be set.** `meson.build` reads it, and Git Bash sets it to Git's + own `/mingw64`, which fails with `Include dir C:/Program Files/Git/mingw64/include/opus + does not exist`. ninja reconfigures by itself whenever `meson.build` changed (a new + source file is enough), so this bites on ordinary incremental builds. +- **Spell `MINGW_PREFIX` exactly `C:/msys64/mingw64`.** A reconfigure with a different + spelling than the build directory was set up with changes the include flags and + rebuilds everything (about 560 steps). +- **`-j 3`.** At ninja's default parallelism a large rebuild runs this machine out of + memory (`cc1plus.exe: out of memory`, bash cannot fork). If it happens, run the same + command again; ninja carries on where it stopped. +- **Redirect to a log and never pipe ninja.** The output is large and can stall or time out + the tool. `tmp/` is gitignored and is the place for logs. +- **Write ninja's exit status into the log.** The status of a `ninja ...; tail ...` chain + is that of `tail`. +- **Targets carry the extension**: `test-tony-app.exe`, not `test-tony-app`. +- `Tony.exe` cannot be relinked while it is running. Close the app first. + +Success is `[N/N] Linking target ...` and `exit:0`; nothing to do is `ninja: no work to +do.` and `exit:0`. + +Timings: incremental under a minute, a few minutes after touching `MainWindow.cpp` or a +widely included header, up to 30 minutes from clean. Give long builds a 30-minute timeout. + +Reconfigure from scratch (rarely needed): + +```sh +export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" +meson setup --wipe build_mingw > tmp/build.log 2>&1 && ninja -j 3 -C build_mingw Tony.exe >> tmp/build.log 2>&1 +echo "exit:$?" >> tmp/build.log +``` + +## What is particular about this `meson.build` + +- A MinGW/GCC win64 branch that upstream does not have: msys2 system libraries, + `-include main/mingw_byte_fix.h` on every translation unit (the C++17 `std::byte` versus + `rpcndr.h`'s `byte` clash), `--export-dynamic-symbol` for the vamp entry point, explicit + include directories for `opus`, `sord-0`, `serd-0`. +- `-DHAVE_MEDIAFOUNDATION` with `-lmfplat -lmfreadwrite -lmfuuid -lpropsys`; needs the + `bqaudiostream` fork. +- `tony_core` / `tony_app` static libraries and the two test executables; see + [architecture.md](architecture.md) for what goes where. A new source file goes into + `tony_core_files` or `tony_app_files`, and its header into the matching `*_moc_files` + only if it declares `Q_OBJECT`. +- Windows headers define `near` and `far` as macros. Do not use them as identifiers. diff --git a/docs/forks.md b/docs/forks.md new file mode 100644 index 00000000..96b29f45 --- /dev/null +++ b/docs/forks.md @@ -0,0 +1,96 @@ +# Dependency forks + +Tony's libraries are separate repositories checked out into the top-level directories by +[repoint](../repoint-project.json) and **gitignored in this repository**. Four of them are +forks under `github.com/jhhr` that exist only for this Tony fork: + +| Directory | Fork branch | Why it is forked | +| --- | --- | --- | +| `svcore/` | `jhhr/svcore` `tony-customizations` | Qt 6.11 build fixes only. No behaviour change. | +| `svgui/` | `jhhr/svgui` `tony-customizations` | See below. | +| `svapp/` | `jhhr/svapp` `tony-customizations` | See below. | +| `bqaudiostream/` | `jhhr/bqaudiostream` `master` | `` instead of `` under MinGW, needed for `-DHAVE_MEDIAFOUNDATION`. | + +`pyin/` and the rest are upstream and must stay untouched. + +## Changing a fork + +The forks are free to change when Tony needs it; prefer a small, general addition to the +library over a workaround in `main/`. + +1. Edit and commit inside the library's directory (it is its own git repository, on the + fork branch). Commit messages there follow that repository's style: `area: what`. +2. Push to the remote named **`jhhr`**. In `svcore`, `svgui` and `svapp`, `origin` is + upstream sonic-visualiser — do not push there. +3. Put the new commit hash in `repoint-lock.json` as that library's `pin`, and commit that + in Tony together with the code that needs it. +4. A sub-agent that was told to work only in `main/` does not edit a fork: it reports + exactly which change it needs, and the lead session makes it. + +**repoint does not run on the development machine** (it needs an SML compiler and none is +installed). The checkouts are managed with plain git, and `repoint-project.json` / +`repoint-lock.json` are edited by hand. Keep the lock file's pins equal to what is checked +out: CI and anyone else's checkout get exactly what the lock file says. + +**Searching**: ripgrep-based search tools skip these directories because they are +gitignored. Pass the directory as the search path explicitly, or use `grep -rn` in Bash. + +## What the forks add (and why Tony needs it) + +### svapp + +- `Document::attachLayerToView(View*, Layer*)`: layer into a view and the layer-view map, + no undo command, document not marked modified. For every layer that is Tony's own + furniture. Without it Undo after a take found Add Layer commands. +- `Document::detachLayerFromView(View*, Layer*)`: the reverse, used by `pruneExtraPane()` + to keep shared layers (the time ruler) alive when an extra pane is removed. +- `Document::modelAboutToBeReleased(ModelId)`, and `MainWindowBase` removing the model + from the play source on it. Upstream reached `removeModel()` only from + `RemoveLayerCommand`, which forced deletes never run. +- `Document::toXml()` leaves out layers with `setSavedInSession(false)` and any model only + such layers show; what was derived from it is written as not derived. +- `MainWindowBase::RecordCreateUnshownModel`: record into a model with no pane, layer or + "Import Recorded Audio" command. Before it, that command referred to a pane Tony had + deleted, so the history had to be cleared before every take (undo depth one). +- `AudioCallbackRecordTarget`: `recordUpdateTimeout` 10 ms (upstream ~200 ms) so the live + tracker sees audio promptly; `setSystemRecordLatency()` actually stores its value + (upstream: a no-op); `getSystemRecordLatency()`; `getFramesReceived()`. +- `AudioCallbackPlaySource`: `setPlayStartCallback(std::function)`, run from + the audio callback with the first block after `play()` (start-gap measurement); + `getModels()` for tests; `removeModel()` tolerates a model already gone. + `AudioGenerator::removeModel()`/`clearModels()` also delete the continuous synth. +- `SVFileReader` restores the `start` attribute of wave file models (upstream wrote it and + never read it). Only older `.ton` files need it now. + +### svgui + +- `ViewManager`: `checkPlayStatus` timer 500 ms → 20 ms, so the cursor keeps up with the + live dots. +- `ViewManager::setRecordStartFrame()` / `getRecordStartFrame()`: while recording, the + playback frame is this plus the recorded duration, not the duration alone. Without it a + take recorded at P > 0 showed the cursor crawling from frame 0 and the pane scrolling + away from the dots. +- `RegionLayer::PlotStrip` plot style: the coverage strip. Saved through the existing + `plotStyle` attribute. +- `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the + notes of a take that is put away. +- `Layer::setSavedInSession(false)`: `View::toXml()` leaves the layer out. + +## Known defects in the forks, not fixed + +- `svapp/audio/AudioCallbackRecordTarget.cpp` connects to `SIGNAL(aboutToBeDeleted())`, + which the model class does not have, so its `modelAboutToBeDeleted()` never runs. The + signal to use would be `Document::modelAboutToBeReleased(ModelId)`. Tony avoids the + consequence by stopping the recording and releasing the model in a fixed order (see + [recording.md](recording.md)); whether `m_model` can dangle otherwise was not looked into. +- The play-start callback is passed the frames actually got, not the requested block size. +- `View::removeLayer()` does not disconnect `layerMeasurementRectsChanged`. + +## Changes that would tidy Tony up but were not made + +- `Document::setModelSource()` (or any way to set or clear a derivation record) would + replace `MainWindow::adoptTakeLayers()` setting source models by hand. +- A hook in `MainWindowBase::toXml()` would save `MainWindow::toXml()` buffering the whole + document to insert ``. +- Fixing pYIN's doubled frame in fixed-lag mode would remove the workaround in + `Analyser::mergeRangedAnalysis()`; `pyin` is not a fork today. diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md new file mode 100644 index 00000000..96d57e22 --- /dev/null +++ b/docs/manual-checklist.md @@ -0,0 +1,103 @@ +# Manual checklist: what no automated test can tell + +The automated suites run against a fake audio device and an offscreen window. Everything +below needs a real device, real ears or real eyes. **As of 2026-09-20 none of it has been +tried by hand.** When an item has been checked, note the date and the result next to it; +when a change touches an area, the items of that area are what to ask the user to try. + +Launch with `.\build.bat run`. + +## Latency and live feedback + +1. **Latency on this machine.** Play Reference While Recording on, headphones, clap along + with a reference with a clear onset. Afterwards the take lines up with the reference by + eye and by ear; still does after save and reopen. +2. **Several phrases in one take.** Record two or three phrases at different positions: + every one sits in time, not just the first (each recording measures its start gap). +3. **Live dots** appear under the playback cursor, not behind it; stay after Stop until the + orange pitch track replaces them; the status bar stops changing when the take stops. +4. **Nothing of the take in the speakers while recording**: with speakers on, neither your + voice nor a synth tone comes back. Play Singing Audio keeps its state through the take. +5. **Stereo interface with the mic on input 2**: dots appear. +6. **No input device / device in use**: Record does nothing harmful, and the next file + opened is analysed as usual. + +## Recording from a position + +7. Seek into the song, Record, sing, Stop: the singing is where it was sung and the part + before it is untouched. Record again inside it: the overwrite question comes; No records + nothing. "Don't ask again" with Yes holds across sessions. +8. During a take at P > 0 the cursor starts at P, the pane follows it, and cursor, + reference and dots are in the same place. +9. **How long Stop takes on a 4-minute song**: a moment for the file copy, then new pitch + only where the singing was. A pause like a whole-song analysis means the ranged path + did not happen. +10. **The joins**: no click at the edges of a new range; the pitch track runs through the + join without a hole, a doubled dot or one note showing as two. Pitch and notes outside + the recorded range (± about 0.25 s) must not flicker or move at all. + +## Pre-roll and Record into Selection + +11. **Is 3 s right, is the countdown readable while singing?** (QSettings + `MainWindow/prerollseconds`; there is deliberately no UI yet.) +12. Sing through the lead-in: nothing of it is heard back, and nothing before P changed. +13. Pre-roll less than 3 s from the start of the song: shorter countdown, no attempt to + run from before frame 0. +14. Record into Selection stops by itself about 0.25 s after the end has been sung; what is + added is exactly the selection; no overwrite question. +15. **Both together — the practice loop this is all for**: select, Record, hear the + lead-in, sing, and be back with nothing to press; Play hears it. Is anything else + needed to make repeating that pleasant? +16. Constrain Playback to Selection together with a pre-roll: the lead-in is probably cut + short. Should the two be kept apart? + +## Coverage strip and erase + +17. The band is readable over waveform and dots at every zoom: height, colour, gaps. +18. It cannot be touched: clicking and dragging on it with either tool creates, moves or + selects nothing and does not change the pane's scale. +19. Select Recording at Playhead then Erase: audio silent, bar gone, pitch and notes gone, + no analysis afterwards. Erasing the middle of a long note leaves two. +20. Erase and Select Recording are greyed out with no take, no selection, while recording, + and for the second or two of analysis after Stop — and come back by themselves. + +## Undo + +21. Three recordings, Ctrl+Z three times: each takes back exactly one (audio, coverage, + band, pitch together). The menu says "Record Singing" / "Erase Singing", never anything + about a layer or pane. +22. Ctrl+Z immediately after Stop, before the pitch appears: nothing of that analysis + lands later; redo analyses again. +23. Undo of the very first recording leaves no singing track at all, and Record still + works. + +## Takes + +24. New Empty Take, record, switch back and forth: fast, no analysis, each take with its + own audio, pitch, notes and band. The inactive take is silent, and playback stops at + the end of the take on show even when another is longer. +25. Duplicate, record into the copy: the original is untouched. +26. Delete asks first and never deletes an audio file. Rename keeps the undo history — + every other take operation clears it without a prompt: acceptable in use? +27. Combo and Takes menu are greyed out during a take. +28. Analyse Now on a take re-analyses all of its coverage in place. + +## Sessions and files + +29. Before the first save, take files go to the record directory; after it, to + `.takes/`. Closing leaves each take's file, files any saved session named, + and nothing else Tony wrote. `recorded-*.wav` are never deleted. +30. Move `.ton` and folder together: everything plays. Move the `.ton` alone: exactly one + warning naming the folder, takes shown without sound, no "locate it?" question. +31. Save As copies the takes; the old `.ton` still opens and plays. +32. A `.ton` from before the takes work opens with the reference only and no dialog. +33. Open a `.ton`, Load Singing Track or Load Background Music, then Record: no crash, time + ruler still there. Closing afterwards asks whether to save. +34. Stop a take and close the window at once: no crash. +35. Log out with unsaved takes: what `commitData` writes into `~/.sv1` is playable. + +## Looks + +36. Alternate pitch track: faded brown is readable but secondary; dark brown during a take + is distinct from black; `8vb` / `8va` buttons look acceptable; the track stays in view + after an octave step. diff --git a/docs/open-points.md b/docs/open-points.md new file mode 100644 index 00000000..7776f247 --- /dev/null +++ b/docs/open-points.md @@ -0,0 +1,44 @@ +# Open points + +Decisions waiting for the user, ideas not built, and known weak spots. Limitations that +belong to the takes design are in [takes.md](takes.md#known-limitations); defects in the +library forks are in [forks.md](forks.md). Remove an item when it is dealt with. + +## For the user to decide + +- **Pre-roll length** is fixed at 3 s (QSettings `MainWindow/prerollseconds`) with no UI. + Is 3 s right, and should there be a control? +- **No overwrite question when recording into a selection**: the selection is taken as the + consent. Right in use? +- **Constrain Playback to Selection + pre-roll**: the play source constrains playback to + the selection, the lead-in is outside it, so it is cut short. Nothing keeps the two apart. +- **Take operations clear the undo history with no prompt** (all but Rename). +- None of the [manual checklist](manual-checklist.md) has been run. + +## Not built + +- Showing two takes at once, or any comparison of takes other than switching. +- Singing track gain and pan are not saved in the session. +- Background music is not saved in the session; it is reloaded by hand. +- An old session (before takes) loses its singing track without telling the user why. +- Recording that starts before frame 0 of the reference. + +## Weak spots + +- **If pYIN fails part-way, the live dots wait for ever**: they are removed on + `initialAnalysisCompleted`, which then never comes. +- **`Analyser::newFileLoaded()` error path for the singing track** (pYIN plugin missing): + not verified that no layers or models are leaked before the error return. +- **Play Singing Audio turned off during a take is not remembered for the next take**: it + lives in `m_singingAudioAfterTake`, not in the settings. +- `Analyser::cancelAnalyses()` cannot see transforms whose layers are held only by the undo + history. +- The dead pane-pruning fallback in `record()` (`m_pendingExtraPanes`, + `drainPendingExtraPanes()` from `teardownRecordingLayer()`) is kept as a safety net since + `RecordCreateUnshownModel`; it can go once that has proved itself. +- `Coverage::regionLabel()` gives every region a blank label, for a stock `RegionLayer` + that printed the value otherwise. With `PlotStrip` it is no longer needed. +- Untested by any suite: removal of dots placed before the latency was measured; the + deferred and error paths of the dot teardown; `ContinuousSynth` deletion in the svapp + fork; the 30 s give-up of `waitForRangedAnalysis()`; `commitData()` relocating takes; + the two other ways `MainWindowBase::record()` can fail. diff --git a/docs/recording.md b/docs/recording.md new file mode 100644 index 00000000..e819c4de --- /dev/null +++ b/docs/recording.md @@ -0,0 +1,147 @@ +# Recording a singing take: order of events, latency, timing + +The Stop and Start paths of `MainWindow::record()` are long and their **order** is the +hard part. This page explains why each step is where it is. Read it before changing +`record()`, `recordingStarted()`, `finishSingingTake()` or `recordingFinishedFull()`. +What happens to the recording afterwards (splice, swap, ranged analysis, undo) is in +[takes.md](takes.md). + +Symbols used throughout, all in frames on the reference's timeline +(`main/TakeTiming.h` holds every formula, unit-tested in `TestTakeTiming`): + +| | | +| --- | --- | +| **P** | where the take starts (`m_takePosition`): the playhead, or the start of the selection | +| **E** | where it stops itself (`m_takeEnd`), -1 when the user presses Stop | +| **R** | pre-roll lead-in actually available before P (`m_takePreRoll`), 0 without one | +| **S** | where playback starts: P − R | +| **L** | recording latency: output + input latency + start gap (`m_recordingLatencyFrames`) | + +The recording's own file always begins at the press of Record. What was sung in answer to +the reference at P is therefore at file frame **L + R**, and the splice reads from there. + +## Start click + +1. **A Stop click returns early** into `MainWindowBase::record()` at the top of `record()`. + Everything below is for a Start. The latency fields are zeroed only on Start: the + splice on Stop still needs them. +2. **P is read before the base call.** `MainWindowBase::record()` calls + `setGlobalCentreFrame(0)`, and from the moment recording starts + `ViewManager::getPlaybackFrame()` reports *record start frame + recorded duration*, + not the position. +3. With Record into Selection on and a selection present, P/E come from + `TakeTiming::chooseRange()` (the range the playhead is in, else the first). No + overwrite question is asked in this mode: the selection is the consent. Otherwise, if + P is inside the take's coverage, `confirmRecordingOverTake()` asks (virtual, so tests + answer it; QSettings `MainWindow/confirmrecordover`). +4. Leftovers of an unfinished take are cleared (`teardownRealtimePitchLayer()`, + `teardownRecordingLayer()`). The existing singing track is **not** torn down: a + recording adds to the take. +5. The take's existing audio is muted for the duration (`muteSingingAudioForTake()`, + directly on the play parameters). What the Play Singing Audio button asks for meanwhile + is kept in `m_singingAudioAfterTake` and applied when the take is over. The take's + pitch and notes stay on show. +6. Record mode is switched to **`RecordCreateUnshownModel`** (svapp fork) around the base + call: the recording becomes a model of the document with no pane, no layer and no + "Import Recorded Audio" undo entry. `ViewManager::setRecordStartFrame(S)` (svgui fork) + makes the cursor run with the reference instead of crawling from frame 0. +7. `recordStatusChanged(true)` → `recordingStarted()` fires *inside* the base call, before + the model is in the document. It defers with `QTimer::singleShot(0)`: + `setupRealtimePitchLayer()`, and, if Play Reference While Recording is on, the latency + estimate and `m_playSource->play(S)`. +8. `modelAdded()` sees `m_recordingAsSingingTrack`, stores `m_currentRecordingModelId` and + returns. The recording is raw material, not the singing track; no analyser is made. +9. After the base call: if `isRecording()` is false (no device, device busy) the take + flags are cleared and the singing unmuted. Otherwise the playback and centre frames are + restored to S, `setupRecordingLayer()` gives the recording a hidden, muted waveform + layer (`attachLayerToView`, `setSavedInSession(false)`) — the document needs *some* + layer to hold the model — and `startTakePolling()` starts the 100 ms timer if there is + an E to reach. + +The recording's waveform is never shown: it starts at frame 0 of its own file, not at P. +The live dots are the feedback. + +The pane-pruning loop and the `m_pendingExtraPanes` fallback after the base call find +nothing to do since `RecordCreateUnshownModel`. They are kept as a safety net. + +## Stop + +`recordCompleted()` → `analyseNow()` → `finishSingingTake()`. (Analyse Now is ignored while +`isRecording()`; `stopRecording()` clears that flag before emitting `recordCompleted`.) + +1. `refineRecordingLatency()`; read the recording's path and length from the model. +2. **Stop the tracker, then** `teardownRecordingLayer()`. The tracker goes first so the + model is never released under it. `stopRecording()` has already called + `writeComplete()`; releasing the model closes the reader too, so the WAV is whole and + closed before the splice reads it. +3. Playhead back to P, so Play hears what was sung and Record records the same part again. +4. `m_takes->spliceRecording(...)` — see [takes.md](takes.md). A recording no longer than + L + R is dropped quietly; a real failure is a dialog and leaves the track as it was. +5. The undo command is made, the audio swapped, the ranged analysis started, the command + pushed, and `syncCoverageStrip()` called **after** the swap. +6. `recordingFinishedFull(analysing ? m_analyser2 : nullptr)` clears flags, restores + audibility, stops reference playback. With an analyser, the live dots stay until it + emits `initialAnalysisCompleted` (`m_realtimeLayerTeardownConnection`); without one + they go at once. + +`recordingStarted(false)` only calls `updateAlternatePitchForTake()` and +`updateLayerStatuses()`, so the reference pitch track is back the moment the take stops, +whatever happens to the analysis. + +`onRealtimePitchDetected()` begins with `if (!m_recordingInProgress) return;` because the +dot model outlives the take. + +## Latency + +- **Estimate** (GUI thread, in the deferred lambda): + `computeRecordingLatency(getTargetPlayLatency(), getSystemRecordLatency())` plus + `m_recordTarget->getFramesReceived()` just before `play()` — the *start gap*, the part + of the recording made before the reference began to play. +- **Measurement** (audio callback): the lambda given to + `m_playSource->setPlayStartCallback()` runs with the first block after `play()` and + stores `getFramesReceived() − blockFrames` in an atomic. Drivers deliver a block's input + before asking for its output; `FakeAudioIO` does the same. No Qt calls in there. +- `refineRecordingLatency()` swaps the estimate for the measurement. When the figure + changes, the dots placed so far are cleared: they belong to sound from before the + reference started. +- `currentRecordingLatency()` reads the measurement without consuming it; only + `refineRecordingLatency()` consumes it. `currentTakeTiming()` is therefore built afresh + wherever an answer is wanted — never cache a `TakeTiming`. +- Live dots are drawn at `P + TakeTiming::liveFrameIntoTake(frame)`; a negative result + (lead-in, or sound from before the reference started) means the dot is dropped. +- The compensation ends up **in the audio**: a take's file always starts at frame 0. + `setStartFrame(-L)` on the model is the old route; `TestLatencyShift` and + `shift_aligns_onset` still cover it, and the svapp fork still restores the `start` + attribute so that older `.ton` files open right. +- L is per take, not per device: each recording measures its start gap afresh. + +## Pre-roll and Record into Selection + +- **Pre-roll**: R = `MainWindow/prerollseconds` (3 s, no UI on purpose) clipped to the + start of the song. The device records from the press of Record as always; the splice + simply starts R frames later. With Play Reference off it is just a pause. +- **Record into Selection**: the take stops itself when `getFramesReceived()` reaches + `L + R + (E − P) + 0.25 s` (`autoStopFrames()`). `pollTakeProgress()` calls `record()` — + the same path as the Stop button, so everything that ends a take is in one place. The + splice keeps at most E − P frames. +- `stopTakePolling()` is called from every place a take can end (`record()`'s Stop branch, + `finishSingingTake()`, `recordingFinishedFull()`, `closeSession()`, `~MainWindow()`), and + the poll stops itself when it finds no take. +- **The countdown**: three things in `MainWindowBase` write the status bar during a take + (recorded duration every 10 ms, playback position every 20 ms, visible range on + scroll). All three are routed through `MainWindow::showTakeCountdown()` first. Anything + written to the status bar from a timer of your own will be overwritten before it can be + read. + +## The live tracker + +`RealtimePitchTracker::run()`: read a 2048-frame window of the **mixdown** +(`getData(-1, ...)`, so a mic on input 2 works), YIN, emit, advance 256; sleep 5 ms when +there is not a full window yet. `kHopSize` is also the resolution of the dot model, whose +unit must be `"Hz"` for the layer to align to the pane's log-frequency scale. + +Correct as they are, though they look wrong: + +- The `1/frameSize` scale in the FFT difference function: bqfft's inverse is unscaled. + It matches pyin's `fastDifference`, which `test-tony-core` links as the reference. +- The mixdown is a sum, not an average: YIN's normalised difference is scale-free. diff --git a/docs/takes.md b/docs/takes.md new file mode 100644 index 00000000..f83af74e --- /dev/null +++ b/docs/takes.md @@ -0,0 +1,261 @@ +# Partial recordings and takes: design + +How a singing take is stored, changed, analysed, undone and saved, and why. The recording +itself (what happens between Record and Stop) is in [recording.md](recording.md); the rules +of the SV libraries this leans on are in [architecture.md](architecture.md). + +## What it is for + +1. Skip intros and intermissions: record only where there is singing. +2. Practise one part repeatedly: re-record a portion and keep the rest. +3. Keep several takes of a song. + +## Terms + +- **Recording**: one press of Record to one Stop. A raw `recorded-*.wav` in the record + directory. Tony never deletes these and no session refers to them. +- **Take**: what the singing track shows and plays — one **combined audio file**, a pitch + layer, a notes layer and a coverage list. Built from one or more recordings. +- **Combined file**: a WAV that starts at **frame 0 of the reference's timeline**, silent + wherever nothing was recorded. Named `take-[-n].wav`. +- **Coverage**: the frame ranges of the combined file that hold singing (`main/Coverage.*`). + +## Decisions, and what was rejected + +| Question | Decision | +| --- | --- | +| A take made of several recordings | One combined file per take, so there is one singing model and one analyser as before. A new file is written for every change; nothing is edited in place. | +| Analysis after a partial recording | Only the recorded range is analysed and merged in. | +| Inactive takes | Their pitch, notes and coverage stay in pane 0 as hidden, dormant layers. **Only the active take has an audio model and an analyser** (an audio model costs a file handle, a peak cache and a place in the play source). Rejected: an analysis file per take. | +| Showing two takes at once | Not supported. | +| Where audio lives | `.takes/` beside the `.ton`, relative paths in the file. | +| Sessions from before takes | No migration: they open without their singing track, silently. | +| Editing | One operation, Erase Singing in Selection, covers remove, trim and split. No hand-editing of singing pitch or notes exists, so replacing a range loses nothing the user made. | +| Undo | Recordings and erases are undoable to any depth. Take operations (new, duplicate, delete, switch, Load Singing Track) are not, and **clear the undo history** without a prompt; Rename does not. | + +## The pieces + +`tony_core` (no window, unit-tested): `Coverage`, `TakeAudio` (`splice()` / `erase()`, +streaming, ~5 ms edge fades, refuse to overwrite a file, refuse mismatched sample rates), +`TakeEvents` (what an erase does to pitch and note events), `SingingTakes` (the list: +name, audio path, coverage, active index; and the bookkeeping of files), `TakesFile` (the +`` element and the folder), `TakeTiming`. + +App side: `TakeLayers` (layers by name, `raise()`), `CoverageStrip`, `TakeCommands` +(`SingingTakeCommand`), and the wiring in `MainWindow`. + +`SingingTakes` knows nothing of layers or models on purpose. Keep it that way. + +## A take is linked to its layers by name only + +`"Take 2 Pitch"`, `"Take 2 Notes"`, `"Take 2 Coverage"` (`TakeLayers::nameFor()` / +`parse()` / `find()`). No pointer to a take's layer is kept anywhere except in the analyser +of the active take. Consequences: + +- The names are stored in the session file and are **not translated**. +- A take name is never used twice in a session, even after its take is deleted + (`reserveTakeName()`): that take's layers may still be in the document. +- After a session load, names found on layers in the pane are reserved as well. + +## An inactive take is owned by no analyser + +An `Analyser` claims any pitch or notes layer whose model's source model is its audio +model. So `MainWindow::putOtherTakeLayersAway()` — **the one enforcement point**, called +from a switch and from `setupSingingTrackAnalyser()` — clears the source model of every +inactive take's layers, makes them dormant and inaudible, and takes their models out of the +play source (a longer inactive take would otherwise hold playback open past the end). + +The active take's layers are raised (`raiseActiveTakeLayers()`), because tools act on the +topmost layer of a kind. + +## The swap: new audio under existing pitch and notes + +`MainWindow::swapSingingAudio(path)`, used after every splice and erase, by undo/redo and +by a take switch: + +1. Open the new file **first** with `openTakeAudioFile()` — a file that cannot be read then + disturbs nothing. +2. `m_analyser2->releaseLayers()`: cancels analyses, deletes only the waveform layer (which + releases the old audio model), forgets pitch and notes without deleting them. Delete + the analyser. +3. `setSourceModel(newAudio)` on the pitch and notes models. `Document::releaseModel()` + clears the derivation of dependent models, so after step 2 they belong to nobody. + **Nothing may read a layer's source model between 2 and 3**, and anything that held the + old audio model's id is stale. +4. `setupSingingTrackAnalyser(newAudio, deferAnalysis=true)`: its scan + (`claimExistingAnalyses()`) claims the two layers and `connectAnalysisLayers()` wires + them; no pYIN runs. **`m_rebuildingTakeAudio` must be set around this**, or the coverage + becomes "the whole of the new file" (the rule for a file the user loaded). +5. Put back what settings do not know: visibility, audibility, stacking. + +The first recording of a take has no layers to keep: `loadTakeAudio()` opens the file and +`Analyser::addEmptyAnalyses()` makes an empty pitch and notes pair indistinguishable from an +analysed one. Both layers must exist before a ranged run. + +`adoptTakeLayers(audio)` does step 3 for a restored or switched-to take, by name, just +before the analyser is made. Without it the take is analysed from scratch beside its own +layers. + +## Ranged analysis and merge (`Analyser::analyseRange`) + +The same two pYIN transforms as a whole-file analysis (`buildAnalysisTransforms()` serves +both), over the recorded range **widened by 0.5 s each side**, clipped to the coverage +range it sits in, aligned to the 256-frame grid. The temporary layers are in the document +but in **no view**, so nothing shows, selects or claims them. + +When both are complete the result is merged into the claimed models directly (no command, +as a transform's output never was) and `rangedAnalysisMerged()` then +`initialAnalysisCompleted()` are emitted. + +- **Only the middle of the run is merged.** W = the range asked for ± 0.25 s. The ends of + a run are where pYIN has least context (it cannot stamp its first two hops at all), so + what the models already hold there is better. Merging the whole run left a two-hop hole + 0.5 s before every recording and split notes in unchanged audio. Where the run's far + edge is the coverage's edge (`m_rangedClippedEnd`), W reaches it and takes in the two + hops the run stamped past it. +- **Time stamps.** The smoothed pitch output is fixed-sample-rate: the host rounds it onto + the whole file's grid, no correction. The **notes** output is variable-rate and pYIN + times a note by frame number *within the run*: `m_rangedStart` is added back. That is + what the grid alignment is for. +- **Pitch**: old events in W go, new events in W are added. +- **Notes, by onset**: old notes with onset in W go; new notes with onset in W are added. + An old note from before W is cut back only if a new note overlaps it. A new note that + would overlap an old note starting at or after the end of W is cut back to that onset. A + new note cut off by the *end of the run* (ends within four hops of it) takes the end of + the old note that ran past, if there is one. +- **pYIN stamps one frame of every run twice** in fixed-lag mode: the last frame of + `process()` comes again first from `getRemainingFeatures()`, 100 hops before the end. + Whole-file tracks have it too, harmlessly. In a ranged run it falls inside W, so the + merge drops the second copy. `pyin` is upstream and untouched. +- Every remove and add is recorded (`getRangedPitchChange()` / `getRangedNotesChange()`) + for undo. +- One ranged run at a time. A second `analyseRange()` abandons the first; so do + `cancelAnalyses()`, `releaseLayers()`, `fileClosed()` and `~Analyser()`. If a run is + still going when the next take stops, the new run is widened to take in + `m_takeAnalysisRange`, since the swap kills the old run's result. +- Do not clip a range to a model's frame count that is still growing: it is 0 just after a + splice. +- **`getInitialAnalysisCompletion()` knows nothing of a ranged run.** Anything that means + "is analysis finished?" must also ask `isAnalysingRange()`. + +Analyse Now on a take re-analyses all coverage as **one** run from the first range to the +last (silence in between is cheaper than a queue). It is not undoable and closes the open +command first. + +## Undo and redo (`SingingTakeCommand`) + +The command holds **values only**: take name, audio path and `Coverage` before and after, +the `TakeEvents::Change` of pitch and of notes, and a range that still has to be analysed. +`execute()` / `unexecute()` hand a `TakeState` to `MainWindow::applyTakeState()`, which +finds the analyser and layers *then*. + +- A recording's command is pushed at once, work already done, and stays **open** + (`m_openTakeCommand`) until its analysis merges: `takeAnalysisMerged()` adds the two + event changes and closes it. One Undo therefore covers the splice and its analysis. +- Undo before the merge abandons the run; the command remembers the range + (`setPendingAnalysis()`), and a redo analyses it again. +- `applyTakeState()` uses `SingingTakes::restoreTake()`, not `setTake()`: neither file + supersedes the other however often they are swapped. +- Event changes are applied removals-of-both-models first, then additions: where audio did + not change, an analysis can put back the very event it removed, and it must be there + once. +- Undo of a take's first recording has an empty path: the singing analyser is torn down, + leaving no singing track at all. +- Erase is **refused while a ranged analysis runs** (the swap would lose its result); the + action re-enables on `initialAnalysisCompleted()`. +- The history is never cleared by a recording or an erase. + +## Erase rules (`TakeEvents`) + +A pitch event in an erased range goes. A note wholly inside goes; a note running *into* +the range keeps its onset and is cut back; a note whose **onset** was erased begins again +at the end of the range, shorter; a range through the middle of a note leaves two notes. No +analysis runs. Erasing all coverage leaves the take, with an all-silent file. + +## The coverage strip is the stored coverage + +One `RegionLayer` (svgui fork's `PlotStrip` style: a 6 px band along the bottom, +display-only, `EqualSpaced` scale so the pane's scale is untouched) per take, in pane 0. +`Coverage::toEvents()` / `fromEvents()` are the entire file format: after a session load a +take's coverage is read back out of its strip. + +`MainWindow::syncCoverageStrip()` is the one place it is kept in step. Call it **after** a +swap, never before: the swap restores the pane's state as it found it. It also removes the +strip's model from the play source. + +## Files on disk + +`SingingTakes` tracks three sets: **superseded** paths (kept until the session closes, for +undo), paths **this run wrote**, and paths a save has **protected**. `closeSession()` +deletes only files this run wrote that no take refers to and no save protected. The +superseded list can contain a file the user loaded; that is never ours to delete. + +- `takeAudioDirectory()` is the record directory until the session has a file, and + `.takes/` from then on. +- **Every** save — not only the first and Save As — copies into the folder whatever takes + refer to outside it (`copyTakeAudioForSave()` → `TakesFile::copyTakeAudioInto()`), + because an undo after a save can point a take back at the record directory. Copied, not + moved: Windows will not move an open file. Never overwrites (`-2`, `-3`). If a copy fails + the copies made are removed and **the save is refused**. +- Paths are **absolute in memory** (takes, undo commands, protect list) and relative only + in the file. +- `saveSessionFile()` first calls `waitForRangedAnalysis()` (polls; after 30 s it abandons + the run rather than write two temporary models into the file). +- A takes folder this run made and left empty is removed on close. +- `QFileInfo` equality is true when neither file exists; do not use it to compare an output + path that has not been written yet. + +## The session file + +```xml + + + + +``` + +`MainWindowBase::toXml()` writes the whole document in one call with no hook, so +`MainWindow::toXml()` buffers it and inserts the element before ``. A take's waveform +layer has `setSavedInSession(false)`, so the `.ton` names the audio once and holds no wave +model but the reference's. Without that, a missing take file produced the session +reader's repeating "locate it?" question plus an "incomplete session" warning before +Tony's own. + +**Restore** (`openSession()` → `restoreTakes()`, with `m_restoringSession` set around the +base call so the queued `analyseRestoredSingingModel()` stands aside): + +1. `dropRestoredSingingTrack()` drops every audio model but the reference. Current + sessions carry none, older ones do — keep it. +2. No `` element: an old session. Derived layers and take-named layers go too. +3. Reserve names, add each take, read coverage from its strip, resolve its audio path + against the `.ton`'s directory. +4. `activateTake()` — the same path a switch uses. **Nothing is analysed.** +5. Missing audio is reported in **one** warning naming the folder. The take still shows + pitch, notes and coverage. Recording into it is refused: the splice needs the file. +6. Opening is not a change: `documentRestored()` unless already modified. + +A file the user loads with Load Singing Track becomes a take covering the whole file +(`setWholeFileTake()`) and is analysed in full. + +## Known limitations + +Things to know, none of which stops the feature being used. See also +[open-points.md](open-points.md). + +- A save during a ranged analysis longer than 30 s loses that range's analysis (Analyse + Now brings it back). +- An undo after a save followed by another save leaves the same audio in the takes folder + twice. Harmless. +- `commitData()` (crash/logout save to `~/.sv1`) goes through `saveSessionFile()`, so it + copies the takes there **and points the running session's takes at the copies**. The + next ordinary save brings them back. +- About 11 ms of pitch at the very start of a coverage range cannot be produced (pYIN's + first two hops). +- One sung note can still become two when a note runs into W from before it *and* its + audio changed. Deliberate trade for not splitting notes in unchanged audio. +- A splice or erase that succeeded on disk but could not be shown is not rolled back; the + user gets a dialog naming the file. +- Playing a wave model with a **positive** start frame plays up to a block early and + without an edge fade. This design avoids it: a take's file always starts at frame 0. +- A recording device whose sample rate differs from the reference's is unexercised; + `TakeAudio` refuses the splice. diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 00000000..125bbbec --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,174 @@ +# Testing + +QtTest suites in `main/test/`, in two executables that mirror the two libraries +(see [architecture.md](architecture.md)). The commands are in [AGENTS.md](../AGENTS.md). + +| Executable | Links | Suites | Time | +| --- | --- | --- | --- | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming` | seconds | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow` | about 4.5 minutes (measured 2026-09-20), nearly all of it `TestRecordWorkflow`: takes are recorded in real time | + +`meson test` / `build.bat test` runs both plus four svcore suites. + +- **The `tony-app` meson test has `timeout: 300` and the suite takes about 277 s unloaded.** + A few more real-time tests, or a busy machine, and `meson test` reports a timeout although + every test passes. Raise the timeout in `meson.build` when adding workflow tests. Running + the executable by hand has no timeout. +- `main()` of the app suite replaces `VAMP_PATH` with the executable's directory, so an + installed pYIN is never the one tested; the meson test `depends:` on `pyin_plugin` + because nothing else builds `pyin.dll`. Build `pyin.dll` too when running by hand after + a clean. +- Both mains set the organisation/application names to `tony-tests` / `test-tony-*` and + every suite works in a `QTemporaryDir`, so the user's QSettings and record directory + are never touched. +- `Tony.exe` links both libraries with `link_whole:`. A new source file that is in neither + `tony_core_files` nor `tony_app_files` is invisible to the tests. +- `build_mingw/meson-logs/testlog.txt` contains a dump of the whole inherited environment. + Do not print it or grep it loosely. + +Suites are header-only classes (`TestX.h`). A new suite needs: the header, an `#include` +and a `runSuite()` block in `tony-core-test.cpp` or `tony-app-test.cpp`, and the header in +the matching `*_test_moc_files` list in `meson.build`. A new test function in an existing +suite needs nothing but itself (a private slot). **Every private slot runs as a test**, so +helpers must not be slots; connect to lambdas instead. For access to private statics use +`friend class TestX;`, as `RealtimePitchTracker.h` does. + +## Running + +- `RunSuite.h` writes each suite's results to `$TONY_TEST_LOG_DIR/.txt`. + Read those with `grep -a` (the files can contain odd bytes); stdout is unreliable when + redirected. +- Test function names on the command line are passed to **every** suite in the + executable. The ones that do not have the function report it as unknown and fail, so + the exit status of a run with names is always 1. Only a run with no names has a + meaningful exit status. +- `QT_QPA_PLATFORM=offscreen` is set by `main()` when not given. + +## Design principles + +- **Split a number from what is done with it.** Latency is tested as arithmetic + (`LatencyUtils.h`, `TakeTiming` — core suite) and, separately, as "the application applies + the number it was given" (app suite, with `FakeAudioIO` reporting latencies chosen by the + test and delaying its input by exactly that much). The real figure of a real device is + for the [manual checklist](manual-checklist.md). +- **Pure logic goes in `tony_core`** so that it can have many cheap tests. The app suite is + for order-of-events and ownership: what is in the document, the pane, the play source + and the undo history after a workflow. +- **Ranged analysis is judged against a whole-file analysis of the same audio**, over the + whole file, not just around the range (`ranged_leaves_the_rest_alone`): that is what + caught the merge damaging unchanged audio half a second away. + +## What is there to reuse (`TestRecordWorkflow.h`) + +- `FakeAudioIO` (`FakeAudioIO.h`): a duplex device with a worker thread that runs the + callback in real time, input first and then output, as PortAudio and JACK do. `Config` + sets rate, block size, reported latencies, a programmed mono input, its delay, and + whether the input clock starts at the first audible output sample ("a singer exactly on + time"). It captures the output, so tests can assert what reached the speakers. +- `TestMainWindow`: subclass of `MainWindow` that exposes protected operations as + `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the + fake device through `createAudioIO()`, and **answers dialogs through virtual seams**: + `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, each with a + `set...Answer()` and a counter of questions asked. A test cannot answer a real dialog: + anything new that asks the user needs such a virtual. +- Fixture helpers: `makeWindow(config)`, `writeWav()`, `openReference()`, `startTake()` / + `stopTake()` / `take(ms)`, `verifyPlaySourceClean()`, `layersOnModel()`, + `paneHasLayer()`, `documentHasLayer()`, `reopenAsSession()` / `reopenSession()`, + `verifyEventsSurvived()`. +- A **dialog watchdog**: a 50 ms timer closes any modal dialog and records it, and + `cleanup()` fails the test for one that was not expected. `dialogsMatching()` is for the + dialogs a test does expect. +- `analysed()` waits for analysis completion, no running transformers **and** no ranged + run. `snapshotTake()`, `verifyStripMatchesTake()`, `takeLayers()`. +- `TestSignals.h`: sine, sawtooth, seeded noise, comparison in cents. + `TestSingingAnalysis.h` works the expected ranges out for itself (`widenRange()`, + `mergeWindow()`, `comparePitchAcrossFile()`, `verifyNothingLeftOver()`). +- `PyinReference.*`: pYIN's YIN as the reference for the live tracker. A translation unit + of its own because the Vamp *plugin* SDK headers must not meet the *host* SDK headers + svcore uses. +- `testdata/happy_birthday_gp_masked.wav`: a real sung recording. + +Prefer signals that describe themselves: `TestTakeAudio` uses constants and ramps so that +every sample says where it came from. Assert **identity** as well as equality where the +point is that something survived: the same layer and model objects before and after. + +### Setup that fails confusingly when it is missing + +- An `Analyser` without a `MainWindow` needs `qRegisterMetaType` for `"ModelId"`, + `"sv_frame_t"` and `"sv_samplerate_t"` (else "No such slot" and completion never + fires), the QSettings value `Transformer/use-flexi-note-model=true`, and the named + colours in `ColourDatabase`. Delete the `Document` before the `PaneStack`. +- A `MainWindow` needs the network-permission setting (else a modal dialog), + `setApplicationSessionExtension("ton")` and the record directory. +- `QSignalSpy` connects directly; for a signal from another thread use a receiver object + on the test thread. +- In `TestMainWindow` override only `createAudioIO()`: `~MainWindowBase` calls + `deleteAudioIO()` non-virtually. +- With `FakeAudioIO`'s `inputFollowsPlayback`, `inputDelay` must be at least one block. + The device delivers about two blocks before the application sets its recording flag; + `inputIsKept` counts only input that was kept, which exact start-gap tests need. + +Copy the shape of a neighbouring test. Use `QVERIFY2` with a message that says what went +wrong, and `if (QTest::currentTestFailed()) return;` after a helper that asserts. + +## How tests turned out to be worthless, and how to avoid it + +A review found five tests that could not fail. So: **see each important test fail** — write +it first, or break the code for a moment (mark the line `MUTATION`, and check +`git diff | grep -c MUTATION` is 0 afterwards; rebuild after putting it back). + +- An assertion made vacuous by an earlier guard or early return in the test. +- Setup that resets the state under test: `openReference()` goes through `ReplaceSession` + → `closeSession()`, which clears the very flags a test may have just set. +- Passing by accident: the play source reads about **3 s ahead**, so a take that is + wrongly audible is still not heard in a short test. Only a test that re-seeks + (`take_silent_in_output_after_reseek`) proves a mute at the device output. +- A comparison that is true for the wrong reason: `QFileInfo` equality compares canonical + paths, which are both empty when neither file exists. +- **Test tones need a whole number of samples per period** (220.5 Hz = 200 samples, + 294 Hz = 150, at 44.1 kHz). Otherwise pYIN reports a subharmonic: 220, 330 and 440 Hz all + came out as 110 Hz. +- `MainWindowBase::m_timeRulerLayer` is set only by a `.ton` load. Bugs about the ruler or + pane pruning show only in the `_after_session` variants; the plain-wav tests passed + against the broken code. +- Re-analysis does not change an event count. Detect it by the layer or model object + having been replaced, or with a spy on `layersChanged()`. +- Sparse models are inaudible by default (`getDefaultPlayAudible()`), so asserting that + one is muted proves nothing. +- `RealtimePitchTracker`'s FFT difference shares pYIN's off-by-one in the power term on + purpose. "Fixing" one side makes the live dots and the pYIN track disagree. + +A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming +it; the marker goes in the commit that fixes it. There are none at present. + +## Timing and races + +- Wait for conditions (`QTRY_VERIFY`, the helpers' waits), never for fixed times. Wait for + `ModelTransformerFactory::haveRunningTransformers()` to go false before closing anything + — but not after making a selection: a selection starts `reAnalyseSelection()` on the + reference, so a transformer is then usually running and nothing of the take depends on it. +- A take's analysis lands in two steps, `rangedAnalysisMerged()` then + `initialAnalysisCompleted()`. Read results after the merge (`analysingRange()` false), + not after some other signal that happens to come at about the same time. +- The status bar is written by three base-class timers; a test that reads it must go + through what `showTakeCountdown()` controls. +- Deleting a derived layer does not stop its transform; only + `Analyser::cancelAnalyses()` does, and the transformer still counts as running for one + more turn of the event loop. The live dots likewise go one turn after the models report + completion. +- The evidence for a fixed race is N clean runs **under CPU load** (run `test-tony-core` + alongside, say): a regression shows as an intermittent crash of the executable, often + several tests later, not as a failed assertion. The tests this matters for: + `swap_during_analysis`, `rerecord_during_analysis`, + `range_analysis_torn_down_while_running`, `close_session_during_analysis`. +- If a worker thread drops the last reference to a `WritableWaveFileModel`, it dies off the + GUI thread with its `QTimer` ("Timers cannot be stopped from another thread", then an + access violation). That message in a test log means an analysis was not cancelled before + its model was released. +- After a `.ton` round trip compare frames exactly and values with a tolerance (six + significant figures in the file). + +## What stays manual + +Anything about a real device, real timing by ear, or how something looks: +[manual-checklist.md](manual-checklist.md). From 70143dd636564aacaa9bbb0958e2a71a3d45530f Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 21:34:16 +0300 Subject: [PATCH 056/275] test: tony-app gets 900 s under meson test The suite took about 277 s of its 300 s, nearly all of it TestRecordWorkflow recording takes in real time, so a few more workflow tests or a busy machine would have had meson report a timeout with every test passing. Co-Authored-By: Claude Fable 5.1 --- AGENTS.md | 4 +--- docs/testing.md | 8 ++++---- meson.build | 2 +- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 11556e7e..44e4d87a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -51,9 +51,7 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - A test name on the command line goes to **every** suite in the executable; the suites that lack it fail, so the exit status is only meaningful for a run with no names. - Run named tests while working; run **both whole suites** before calling anything done. -- From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. Its - 300 s timeout for `tony-app` is nearly used up; a "timeout" there is not a test failure - ([docs/testing.md](docs/testing.md)). +- From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. - Give the app suite a tool timeout of 10 minutes. ## Rules for working here diff --git a/docs/testing.md b/docs/testing.md index 125bbbec..0de32575 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -10,10 +10,10 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries `meson test` / `build.bat test` runs both plus four svcore suites. -- **The `tony-app` meson test has `timeout: 300` and the suite takes about 277 s unloaded.** - A few more real-time tests, or a busy machine, and `meson test` reports a timeout although - every test passes. Raise the timeout in `meson.build` when adding workflow tests. Running - the executable by hand has no timeout. +- The `tony-app` meson test has `timeout: 900`; the suite took about 277 s unloaded when + that was set. Every workflow test adds real time, so if the suite comes near it, raise it + in `meson.build`: `meson test` reports a timeout even when every test passes. Running the + executable by hand has no timeout. - `main()` of the app suite replaces `VAMP_PATH` with the executable's directory, so an installed pYIN is never the one tested; the meson test `depends:` on `pyin_plugin` because nothing else builds `pyin.dll`. Build `pyin.dll` too when running by hand after diff --git a/meson.build b/meson.build index 1e3a4704..8f8b0e6e 100644 --- a/meson.build +++ b/meson.build @@ -1417,7 +1417,7 @@ tony_app_test_exe = executable( test('tony-core', tony_core_test_exe) test('tony-app', tony_app_test_exe, depends: pyin_plugin, - timeout: 300, + timeout: 900, env: [ 'QT_QPA_PLATFORM=offscreen' ]) test('svcore-base', svcore_base_test_exe) From 8a31176324e9b2ad5c02c06d99a06d738fc244a7 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 23:24:02 +0300 Subject: [PATCH 057/275] feat: Ctrl+D erases the singing in the selection Erase Singing in Selection had no shortcut and sat at the bottom of the Edit menu, which made a feature meant for repeated practice awkward to reach. Ctrl+Backspace, the obvious partner to the Backspace of Delete Notes, is upstream Tony's Remove Pitches, so Ctrl+D it is; the key reference lists it under Singing Track. The README now also says where a range is selected, since nothing in the fork's own documentation said that it is a drag in the ruler strip below the pane. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 3 ++- main/MainWindow.cpp | 9 +++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index ad8ed9ca..099d7680 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,8 @@ orange pitch track to compare with it. where there is singing and where there is not; recording over a part of it replaces just that part, and only the part that changed is analysed again * Edit -> Select Recording at Playhead and Edit -> Erase Singing in Selection - remove, trim or split what has been recorded + (Ctrl+D) remove, trim or split what has been recorded. A range is selected by + dragging in the thin ruler strip below the pane * recordings and erases can be undone and redone * several takes of a song in one session (the Takes menu and the "Take:" box in the toolbar): each keeps its own audio, pitch track and notes, and switching diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 6c263d79..85adcd6d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -853,8 +853,9 @@ MainWindow::setupEditMenu() m_keyReference->setCategory(tr("Singing Track")); - // No shortcuts for these two: every key worth having in this menu is - // taken, and an erase is not something to reach by accident + // No shortcut for this one: the keys worth having in this menu are + // taken, and it is a step on the way to the erase rather than + // something to reach for on its own m_selectRecordingAction = new QAction(tr("Select Recording at Playhead"), this); m_selectRecordingAction->setStatusTip @@ -868,8 +869,12 @@ MainWindow::setupEditMenu() m_rightButtonMenu->addAction(m_selectRecordingAction); m_eraseSingingAction = new QAction(tr("Erase Singing in Selection"), this); + // Ctrl+Backspace, the obvious partner to the Backspace of Delete + // Notes, is upstream Tony's Remove Pitches + m_eraseSingingAction->setShortcut(tr("Ctrl+D")); m_eraseSingingAction->setStatusTip (tr("Remove the recorded singing within the selected region, leaving silence")); + m_keyReference->registerShortcut(m_eraseSingingAction); connect(m_eraseSingingAction, SIGNAL(triggered()), this, SLOT(eraseSingingInSelection())); connect(this, SIGNAL(canEraseSinging(bool)), From 9a9d946c3028de5a7dc73fe153a2dd0a58c9674b Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 23:24:48 +0300 Subject: [PATCH 058/275] feat: the take's pitch and notes make way while it is recorded into Recording over singing that is already there left that take's own pitch track and notes on the pane, drawn over the same part of it as what is being sung now and, for the pitch, in the same orange as the live dots. The singer could not tell the one from the other, and neither helps them follow the track they are singing to. updateSingingTrackForTake() hides both for the duration and shows them again when the take stops, beside updateAlternatePitchForTake(), which does the same for the reference pitch track. Only what was on show is hidden and only what was hidden here is shown again, so a track the user had already switched off stays off. Layer::showLayer(), not Analyser::setVisible(), which would write the state to the settings the two analysers share. Co-Authored-By: Claude Opus 5 (1M context) --- docs/manual-checklist.md | 3 ++ docs/recording.md | 15 +++++--- main/MainWindow.cpp | 52 ++++++++++++++++++++++++--- main/MainWindow.h | 10 ++++++ main/test/TestRecordWorkflow.h | 64 ++++++++++++++++++++++++++++++++-- 5 files changed, 134 insertions(+), 10 deletions(-) diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 96d57e22..9fa49878 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -16,6 +16,9 @@ Launch with `.\build.bat run`. every one sits in time, not just the first (each recording measures its start gap). 3. **Live dots** appear under the playback cursor, not behind it; stay after Stop until the orange pitch track replaces them; the status bar stops changing when the take stops. + Recording over singing that is there: that take's own pitch track and notes are out of + sight for the take, so only the dots and the track being followed are on the pane, and + they are back when the take stops. 4. **Nothing of the take in the speakers while recording**: with speakers on, neither your voice nor a synth tone comes back. Play Singing Audio keeps its state through the take. 5. **Stereo interface with the mic on input 2**: dots appear. diff --git a/docs/recording.md b/docs/recording.md index e819c4de..653f2bb4 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -40,7 +40,13 @@ the reference at P is therefore at file frame **L + R**, and the splice reads fr 5. The take's existing audio is muted for the duration (`muteSingingAudioForTake()`, directly on the play parameters). What the Play Singing Audio button asks for meanwhile is kept in `m_singingAudioAfterTake` and applied when the take is over. The take's - pitch and notes stay on show. + pitch track and notes are hidden for the duration too (`updateSingingTrackForTake()`, + called after the base call and again when the take stops): they are drawn over the same + part of the pane as what is being sung now, the pitch in the same orange as the live + dots, so with them on show the singer cannot tell what they are singing from what they + sang before. Hidden with `Layer::showLayer()`, not `Analyser::setVisible()`, which would + write the state to the shared settings. Only what was on show is hidden, and only what + was hidden here is shown again. 6. Record mode is switched to **`RecordCreateUnshownModel`** (svapp fork) around the base call: the recording becomes a model of the document with no pane, no layer and no "Import Recorded Audio" undo entry. `ViewManager::setRecordStartFrame(S)` (svgui fork) @@ -84,9 +90,10 @@ nothing to do since `RecordCreateUnshownModel`. They are kept as a safety net. emits `initialAnalysisCompleted` (`m_realtimeLayerTeardownConnection`); without one they go at once. -`recordingStarted(false)` only calls `updateAlternatePitchForTake()` and -`updateLayerStatuses()`, so the reference pitch track is back the moment the take stops, -whatever happens to the analysis. +`recordingStarted(false)` only calls `updateAlternatePitchForTake()`, +`updateSingingTrackForTake()` and `updateLayerStatuses()`, so the reference pitch track +and the take's own pitch and notes are back the moment the take stops, whatever happens to +the analysis. `onRealtimePitchDetected()` begins with `if (!m_recordingInProgress) return;` because the dot model outlives the take. diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 85adcd6d..872ca8fd 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -145,6 +145,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_alternatePitchUpAction(nullptr), m_alternatePitchDownAction(nullptr), m_referencePitchHiddenForTake(false), + m_singingPitchHiddenForTake(false), + m_singingNotesHiddenForTake(false), m_takes(nullptr), m_coverageStrip(nullptr), m_takesMenu(nullptr), @@ -2541,6 +2543,8 @@ MainWindow::closeSession() m_alternatePitch->hide(); m_coverageStrip->hide(); m_referencePitchHiddenForTake = false; + m_singingPitchHiddenForTake = false; + m_singingNotesHiddenForTake = false; m_pendingSingingModelId = {}; m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; @@ -3206,6 +3210,40 @@ MainWindow::updateAlternatePitchForTake() } } +void +MainWindow::updateSingingTrackForTake() +{ + // The singing that is already there is drawn over the same part of + // the pane as what is being sung now, and its pitch in the same + // orange as the live dots: during a take the singer cannot tell the + // one from the other, and neither helps them follow the track they + // are singing to. So the stored pitch and notes make way, and come + // back when the take stops. Not with Analyser::setVisible(), which + // would write the state to the settings the reference shares. + bool inTake = (m_recordingAsSingingTrack && + m_recordTarget && m_recordTarget->isRecording()); + + Pane *pane = m_analyser2 ? m_analyser2->getPane() : nullptr; + + // Only what was on show is hidden, and only what we hid is shown + // again: the user may have had either of them off already + auto update = [&](Analyser::Component c, bool &hidden) { + Layer *layer = m_analyser2 ? m_analyser2->getLayer(c) : nullptr; + if (inTake) { + if (!hidden && layer && pane && !layer->isLayerDormant(pane)) { + layer->showLayer(pane, false); + hidden = true; + } + } else if (hidden) { + hidden = false; + if (layer && pane) layer->showLayer(pane, true); + } + }; + + update(Analyser::PitchTrack, m_singingPitchHiddenForTake); + update(Analyser::Notes, m_singingNotesHiddenForTake); +} + void MainWindow::syncCoverageStrip() { @@ -3863,9 +3901,11 @@ MainWindow::record() teardownRecordingLayer(); } - // The singing track itself stays as it is: its pitch and notes are - // what the singer is adding to, and they stay on show for the take. - // Only its audio is kept out of the mix (spec 5.1). + // The singing track itself stays as it is: its pitch and notes + // are what the singer is adding to. Its audio is kept out of the + // mix (spec 5.1), and its pitch and notes out of sight -- see + // updateSingingTrackForTake(), called below once the device has + // either started or failed to. muteSingingAudioForTake(); m_pendingSingingModelId = {}; @@ -3922,6 +3962,7 @@ MainWindow::record() } updateAlternatePitchForTake(); + updateSingingTrackForTake(); updateLayerStatuses(); // Restore the default mode so that a subsequent "standalone" recording @@ -4067,8 +4108,10 @@ MainWindow::recordingStarted() if (!m_recordTarget->isRecording()) { // Recording stopped - recordingFinishedFull() is called from // analyseNow() once pYIN completes. The reference pitch track - // comes back now, though: the singer has stopped following. + // comes back now, though: the singer has stopped following, and + // so do the take's own pitch and notes. updateAlternatePitchForTake(); + updateSingingTrackForTake(); updateLayerStatuses(); return; } @@ -4285,6 +4328,7 @@ MainWindow::recordingFinishedFull(Analyser *analysing) m_recordingAsSingingTrack = false; m_currentRecordingModelId = {}; restoreSingingAudioAfterTake(); + updateSingingTrackForTake(); if (analysing && m_realtimePitchLayer) { stopRealtimePitchTracker(); diff --git a/main/MainWindow.h b/main/MainWindow.h index f8cb3c11..d27a635d 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -336,6 +336,16 @@ protected slots: void stepAlternatePitch(bool up); void updateAlternatePitchForTake(); + // The take's own pitch and notes are put out of sight while it is + // being recorded into, as the reference is for the alternate above: + // the pitch is the orange of the live dots and the notes are drawn + // over the same part of the pane, and a singer recording over + // singing that is there cannot tell either of them from what they + // are singing now. The two flags say which of them we hid. + bool m_singingPitchHiddenForTake; + bool m_singingNotesHiddenForTake; + void updateSingingTrackForTake(); + // The singing takes of the session: the audio file of each take and // the ranges of it that hold recorded singing. MainWindow only // wires it: it decides where a recording goes and writes the files. diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 555eb3f9..2c054178 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -2172,7 +2172,8 @@ private slots: QVERIFY(m_window->playSingingAudioAction()->isChecked()); // Recording into the take that is now there: its pitch and notes - // stay on show, and its audio is kept out of the mix + // are kept (put out of sight, see the next test), and its audio + // is kept out of the mix auto events = pitchEvents(m_window->analyser2()); QVERIFY(!events.empty()); @@ -2181,7 +2182,7 @@ private slots: QVERIFY2(m_window->analyser2(), "the singing track was torn down for the take"); QVERIFY2(pitchEvents(m_window->analyser2()).size() == events.size(), - "the singing pitch track did not stay on show for the take"); + "the singing pitch track lost its events during the take"); QVERIFY2(!takeParams()->isPlayAudible(), "the singing that is there is audible while it is being " "recorded into"); @@ -2200,6 +2201,65 @@ private slots: QVERIFY(m_window->analyser()->isAudible(Analyser::Audio)); } + // The take's stored pitch track and notes sit over the same part of + // the pane as what is being sung now, the pitch in the same orange + // as the live dots. While a take is being recorded they are out of + // sight, so that the only pitch the singer sees beside the track + // they are following is the one they are singing now; they come back + // when the take stops. + void singing_track_hidden_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2); + sv::Layer *pitch = a2->getLayer(Analyser::PitchTrack); + sv::Layer *notes = a2->getLayer(Analyser::Notes); + sv::Pane *pane = a2->getPane(); + QVERIFY(pitch && notes && pane); + QVERIFY2(!pitch->isLayerDormant(pane), + "the take's pitch track is not shown after the take"); + QVERIFY2(!notes->isLayerDormant(pane), + "the take's notes are not shown after the take"); + QVERIFY(!noteEvents(notes).empty()); + + m_window->setRecordOverAnswer(true); + m_window->seekTo(0); + + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY2(pitch->isLayerDormant(pane), + "the take's own orange pitch track is still shown while it " + "is being recorded over"); + QVERIFY2(notes->isLayerDormant(pane), + "the take's own notes are still shown while it is being " + "recorded over"); + // The live dots are the pitch the singer does see + QTRY_VERIFY_WITH_TIMEOUT(m_window->realtimeLayer(), 2000); + QVERIFY(!m_window->realtimeLayer()->isLayerDormant(pane)); + + QTest::qWait(800); + stopTake(); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->analyser2()); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY2(!pitch->isLayerDormant(pane), + "the take's pitch track did not come back when the take " + "stopped"); + QVERIFY2(!notes->isLayerDormant(pane), + "the take's notes did not come back when the take stopped"); + } + // Review finding 3, at the device. The read-ahead that keeps the // take out of the output in no_self_monitoring is taken away here: // once more has been recorded than the play source buffers, playback From ea95a8def5080466c9952e32968ec42e196d1248 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sun, 20 Sep 2026 23:25:03 +0300 Subject: [PATCH 059/275] fix: the pane no longer greys itself out from the end of a take The pane blocks off everything past the end of its "work model" with a pale wash and a vertical line, to show where the audio ends. It chose that model as the one of the topmost layer that had one, and Tony's pane holds three audio models: the reference, the take's own file, which stops where the singing stopped, and, during a take, the recording being written, whose end frame crawls along behind the playback cursor. So the pane greyed itself out from one of those, which tells the user nothing, and during a take it did so from a line that moved. MainWindow names the reference with the new Pane::setWorkModel() (svgui bdac54f, which also stops the scan choosing a layer that is dormant in the pane -- the recording is held by one of those). Co-Authored-By: Claude Opus 5 (1M context) --- docs/architecture.md | 4 +++ docs/forks.md | 7 +++++ main/MainWindow.cpp | 8 +++++ main/test/TestRecordWorkflow.h | 54 ++++++++++++++++++++++++++++++++++ repoint-lock.json | 2 +- 5 files changed, 74 insertions(+), 1 deletion(-) diff --git a/docs/architecture.md b/docs/architecture.md index 64af7bdc..fa0ba02c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -60,6 +60,10 @@ follow. `MainWindow` then only fills the struct in and puts the answer on screen model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued connection). Stop the tracker **before** releasing the model it reads. +The reference is the pane's **work model** (`Pane::setWorkModel()`, svgui fork, set in +`analyseNewMainModel()`). Without that the pane greys itself out from the end of the +take's audio file, or of the recording being written. + Colours: reference pitch black / notes bright blue; singing and live dots orange / notes bright purple; alternate pitch faded brown, dark brown while a take is recorded. diff --git a/docs/forks.md b/docs/forks.md index 96b29f45..cc25196c 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -74,6 +74,13 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` `plotStyle` attribute. - `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the notes of a take that is put away. +- `Pane::setWorkModel()` / `getWorkModel()`: which model's extents are blocked off at the + ends of the pane (a pale wash and a line), and whose duration, title and alignment are + reported. The scan that chooses one now skips layers dormant in that pane. Tony's pane + holds three audio models, and the pane used to block itself off at the end of whichever + was topmost -- the take's file, which stops where the singing did, or the recording in + progress, whose end crawls along behind the playback cursor. `MainWindow` names the + reference instead. - `Layer::setSavedInSession(false)`: `View::toXml()` leaves the layer out. ## Known defects in the forks, not fixed diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 872ca8fd..534cf8a7 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -7352,6 +7352,14 @@ MainWindow::analyseNewMainModel() if (pane) { + // The reference is the work of this pane, and its end is the end + // of the song. Without saying so, the pane blocks itself off at + // the end of whichever audio model happens to be topmost: the + // take's file, which stops where the singing did, or the + // recording in progress, whose end crawls along behind the + // playback cursor. Either greys out the pane from there on. + pane->setWorkModel(getMainModelId()); + disconnect(pane, SIGNAL(regionOutlined(QRect)), pane, SLOT(zoomToRegion(QRect))); connect(pane, SIGNAL(regionOutlined(QRect)), diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 2c054178..d1614237 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -2260,6 +2260,60 @@ private slots: "the take's notes did not come back when the take stopped"); } + // The recording is held in the pane by a hidden waveform layer + // (setupRecordingLayer), because the document needs a layer to hold + // the model. A hidden layer must not be the pane's work model: the + // pane blocks off everything past that model's end frame with a pale + // wash and a vertical line, and a recording's end frame crawls along + // behind the playback cursor as it is written, so the pane would be + // greyed out from there to the right for the whole take. + void recording_does_not_grey_out_the_pane() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane); + QCOMPARE(pane->getWorkModel(), m_window->mainModelId()); + + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->recordingLayer()); + QVERIFY(!m_window->currentRecordingModelId().isNone()); + QVERIFY2(pane->getWorkModel() != m_window->currentRecordingModelId(), + "the pane blocks itself off at the end of the recording"); + QCOMPARE(pane->getWorkModel(), m_window->mainModelId()); + + QTest::qWait(600); + stopTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(pane->getWorkModel(), m_window->mainModelId()); + + // And again with a take already there, whose own audio layer is + // in the pane as well + m_window->setRecordOverAnswer(true); + m_window->seekTo(0); + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY2(pane->getWorkModel() != m_window->currentRecordingModelId(), + "the pane blocks itself off at the end of the recording"); + + // Even with the pin taken off, the hidden layer that holds the + // recording must not be the one the pane chooses: a layer that + // is not shown is no part of what the pane is showing + pane->setWorkModel({}); + QVERIFY2(pane->getWorkModel() != m_window->currentRecordingModelId(), + "the model of a hidden layer was chosen as the work model"); + pane->setWorkModel(m_window->mainModelId()); + + QTest::qWait(600); + stopTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(pane->getWorkModel(), m_window->mainModelId()); + } + // Review finding 3, at the device. The read-ahead that keeps the // take out of the output in no_self_monitoring is taken away here: // once more has been recorded than the play source buffers, playback diff --git a/repoint-lock.json b/repoint-lock.json index 4b22ff82..ab78ee55 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "1852b930f77cb54e9fc9af39218bc8c012ba0626" + "pin": "bdac54f488f4d0346eaf692ec4a46d436db0ca87" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 1242f37e3dc971a70b88e513089e341c10af8a9c Mon Sep 17 00:00:00 2001 From: jhhr Date: Mon, 21 Sep 2026 01:54:40 +0300 Subject: [PATCH 060/275] build: bqaudioio pin, so that the record side can be suppressed FakeAudioIO overrides suppressRecordSide, which bqaudioio declares as a pure virtual on SystemAudioIO. That arrived upstream on the toggle-record-in-io branch and was merged to default at 017ab3ed3a33, five revisions past the pinned 88520ab5b8e5 -- so the test suite did not build from a fresh checkout, failing with "marked 'override', but does not override". Only the tests were affected; Tony itself never calls it. With the pin moved on, all six suites pass. Co-Authored-By: Claude Opus 5 --- repoint-lock.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoint-lock.json b/repoint-lock.json index ab78ee55..e495809b 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -31,7 +31,7 @@ "pin": "38c3e524416a" }, "bqaudioio": { - "pin": "88520ab5b8e5" + "pin": "017ab3ed3a33" }, "bqaudiostream": { "pin": "9b59e706f9dda8507c5e1b07ad619c060f65bd29" From 10333c8dafe01d9f82ba420f2690fc73811adbe8 Mon Sep 17 00:00:00 2001 From: jhhr Date: Mon, 21 Sep 2026 02:44:59 +0300 Subject: [PATCH 061/275] fix: tony stays light when windows is in dark app mode Qt follows the system's dark app mode on Windows 11, but Tony's icons and several of its widgets are drawn in black for a light background and vanished on the dark one. The colour scheme is now pinned to light at startup, on Qt 6.8 and later where the call exists. Co-Authored-By: Claude Fable 5.1 --- main/main.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/main/main.cpp b/main/main.cpp index ebb20b5f..ac90b39b 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -36,6 +36,7 @@ #include #include #include +#include #include #include @@ -216,6 +217,14 @@ main(int argc, char **argv) TonyApplication application(argc, argv); +#if QT_VERSION >= QT_VERSION_CHECK(6, 8, 0) + // Qt follows the system's dark app mode, but Tony's icons and + // several of its widgets are drawn in black for a light + // background and vanish on a dark one. Stay light whatever the + // system says. + QGuiApplication::styleHints()->setColorScheme(Qt::ColorScheme::Light); +#endif + QApplication::setOrganizationName("sonic-visualiser"); QApplication::setOrganizationDomain("sonicvisualiser.org"); QApplication::setApplicationName("Tony"); From 37a7e27ce8f1e7d49d3017e2da295e0fac81e156 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:28:10 +0000 Subject: [PATCH 062/275] fix: connect slots taking sv types by member pointer Four connections in Analyser and one in MainWindow named their slots in SIGNAL()/SLOT() strings with ModelId or sv_frame_t, while moc records the slots as taking sv::ModelId and sv::sv_frame_t. Qt 6.4 does not match the two ("No such slot"), so the analyser never heard that pYIN had finished and TestSingingAnalysis timed out; the Qt the project is developed with happens to match them. Member pointers are checked at compile time and match under any version, as the rule in docs/architecture.md already asks. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/Analyser.cpp | 25 +++++++++++++++---------- main/MainWindow.cpp | 6 ++++-- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index e7cbf079..9b8cd77e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -908,19 +908,23 @@ Analyser::connectAnalysisLayers() // Claimed layers need these as much as ones we made ourselves: the // analyser they belonged to before is gone, and with it its // connections. Unique, because an analyser handed the same layers - // twice would otherwise hear each signal twice + // twice would otherwise hear each signal twice. By member pointer + // where the arguments are sv types: moc records this class's slots + // as taking sv::ModelId and sv::sv_frame_t, and whether a SLOT() + // string saying ModelId matches that depends on the Qt version (6.4 + // says "No such slot") if (auto pitchLayer = qobject_cast(m_layers[PitchTrack])) { - connect(pitchLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId)), + connect(pitchLayer, &Layer::modelCompletionChanged, + this, &Analyser::layerCompletionChanged, Qt::UniqueConnection); } if (auto noteLayer = qobject_cast(m_layers[Notes])) { - connect(noteLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId)), + connect(noteLayer, &Layer::modelCompletionChanged, + this, &Analyser::layerCompletionChanged, Qt::UniqueConnection); - connect(noteLayer, SIGNAL(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), - this, SLOT(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), + connect(noteLayer, &FlexiNoteLayer::reAnalyseRegion, + this, &Analyser::reAnalyseRegion, Qt::UniqueConnection); connect(noteLayer, SIGNAL(materialiseReAnalysis()), this, SLOT(materialiseReAnalysis()), @@ -1170,9 +1174,10 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, auto model = ModelById::get(id); if (!model) continue; // Emitted on the transform's own thread, so delivered here as a - // queued call: the merge happens on this thread like any other - connect(model.get(), SIGNAL(completionChanged(ModelId)), - this, SLOT(rangedAnalysisCompletionChanged(ModelId))); + // queued call: the merge happens on this thread like any other. + // By member pointer, as in connectAnalysisLayers() + connect(model.get(), &Model::completionChanged, + this, &Analyser::rangedAnalysisCompletionChanged); } // createDerivedLayers() returns only once the transform has set both diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 534cf8a7..b1e52653 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -291,8 +291,10 @@ MainWindow::MainWindow(AudioMode audioMode, // We have a pane stack: it comes with the territory. However, we // have a fixed and known number of panes in it -- it isn't // variable - connect(m_paneStack, SIGNAL(doubleClickSelectInvoked(sv_frame_t)), - this, SLOT(doubleClickSelectInvoked(sv_frame_t))); + // By member pointer: the slot takes sv::sv_frame_t, which a SLOT() + // string saying sv_frame_t does not match under every Qt version + connect(m_paneStack, &PaneStack::doubleClickSelectInvoked, + this, &MainWindow::doubleClickSelectInvoked); scroll->setWidget(m_paneStack); m_overview = new Overview(frame); From 838a2eadad37fd2482a9566f64fb37e02e598447 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:53:26 +0000 Subject: [PATCH 063/275] feat: lrc lyrics parser Timed lyrics are to come from LRC files: the Moises lyrics exporter writes them, line by line or word by word, and so do many other tools. parseLrc() reads both kinds into words with a start, an end and a line, and lyricsToEvents() makes them the events of a region model, one region per word with the line as its value, on the reference's timeline. The exporter writes no end times, so a word ends where the next one starts and the last word of a line is capped at 2 s unless a gap line (a time tag with no text, or only a note sign) says where it ends. Its words are split on their tags, never on spaces: it leaves the space out before a word with punctuation in it. Text that XML 1.0 cannot hold is removed on the way in, so that no label can make a session unreadable. Files that are not UTF-8 are read as Latin-1, the same on every system. The core suite reads the fixtures in testdata/lyrics through a define of the source directory; the fixture text is invented. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/Lyrics.cpp | 568 ++++++++++++++++ main/Lyrics.h | 126 ++++ main/test/TestLyrics.h | 793 ++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 7 + testdata/lyrics/lrc-with-ends.lrc | 11 + testdata/lyrics/moises-exporter-lines.lrc | 10 + testdata/lyrics/moises-exporter-words.lrc | 10 + 8 files changed, 1532 insertions(+) create mode 100644 main/Lyrics.cpp create mode 100644 main/Lyrics.h create mode 100644 main/test/TestLyrics.h create mode 100644 testdata/lyrics/lrc-with-ends.lrc create mode 100644 testdata/lyrics/moises-exporter-lines.lrc create mode 100644 testdata/lyrics/moises-exporter-words.lrc diff --git a/main/Lyrics.cpp b/main/Lyrics.cpp new file mode 100644 index 00000000..3f3c09e3 --- /dev/null +++ b/main/Lyrics.cpp @@ -0,0 +1,568 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "Lyrics.h" + +#include +#include + +#include +#include +#include + +using namespace sv; + +namespace { + +QString tr(const char *text) +{ + return QCoreApplication::translate("Lyrics", text); +} + +// "1 line was" or "3 lines were": the status bar shows these +QString counted(int n, const char *one, const char *many) +{ + if (n == 1) return tr(one); + return tr(many).arg(n); +} + +// Every time in an LRC file is a whole number of milliseconds, and so +// is the offset, so they stay that way until the words are made: no +// rounding can then put a word before the one it follows +typedef qint64 Ms; + +const Ms msPerSecond = 1000; + +// Only to keep the arithmetic far from overflowing: no reference is a +// day long +const qint64 maxMinutes = 24 * 60; + +bool isAsciiDigit(QChar c) +{ + return c.unicode() >= '0' && c.unicode() <= '9'; +} + +int digitValue(QChar c) +{ + return c.unicode() - '0'; +} + +/** + * The length of the time tag at pos, or 0 if there is none there. A + * tag is [m:ss], [m:ss.f], [m:ss.ff] or [m:ss.fff], with ':' allowed in + * place of '.', and with open and close in place of the brackets: '<' + * and '>' for a word. Seconds of 60 or more make it no tag. + */ +int readTimeTag(const QString &s, int pos, char open, char close, Ms &time) +{ + const int n = s.size(); + int i = pos; + if (i >= n || s[i] != QLatin1Char(open)) return 0; + ++i; + + qint64 minutes = 0; + int minuteDigits = 0; + while (i < n && isAsciiDigit(s[i])) { + minutes = minutes * 10 + digitValue(s[i]); + if (minutes > maxMinutes) return 0; + ++minuteDigits; + ++i; + } + if (minuteDigits == 0 || i >= n || s[i] != QLatin1Char(':')) return 0; + ++i; + + if (i + 1 >= n || !isAsciiDigit(s[i]) || !isAsciiDigit(s[i + 1])) { + return 0; + } + int seconds = digitValue(s[i]) * 10 + digitValue(s[i + 1]); + if (seconds >= 60) return 0; + i += 2; + + Ms fraction = 0; + if (i < n && (s[i] == QLatin1Char('.') || s[i] == QLatin1Char(':'))) { + ++i; + int fractionDigits = 0; + while (i < n && isAsciiDigit(s[i]) && fractionDigits < 3) { + fraction = fraction * 10 + digitValue(s[i]); + ++fractionDigits; + ++i; + } + if (fractionDigits == 0) return 0; + // One digit is tenths, two hundredths, three thousandths + for (int d = fractionDigits; d < 3; ++d) fraction *= 10; + } + + if (i >= n || s[i] != QLatin1Char(close)) return 0; + ++i; + + time = (minutes * 60 + seconds) * msPerSecond + fraction; + return i - pos; +} + +/** + * A [key:value] line. A key of digits only is a time tag gone wrong, + * not metadata. The value runs to the last ']', so that it can hold + * brackets of its own: [ti:Song [Live]]. + */ +bool readMetadata(const QString &row, QString &key, QString &value) +{ + if (!row.startsWith(QLatin1Char('[')) || !row.endsWith(QLatin1Char(']'))) { + return false; + } + int colon = row.indexOf(QLatin1Char(':')); + if (colon < 0) return false; + + QString k = row.mid(1, colon - 1).trimmed(); + if (k.isEmpty() || k.contains(QLatin1Char('[')) || + k.contains(QLatin1Char(']'))) { + return false; + } + if (std::all_of(k.begin(), k.end(), isAsciiDigit)) return false; + + key = k.toLower(); + value = row.mid(colon + 1, row.size() - colon - 2).trimmed(); + return true; +} + +/** + * XML 1.0 cannot hold the C0 controls other than tab (and the line + * breaks, which never get this far), nor U+FFFE and U+FFFF, which the + * UTF-8 decoder lets through: one in a label would make the session + * file unreadable. DEL is never meant as text either. A tab becomes + * a space, which is what it comes back as from a session file anyway: + * an XML attribute value is read with its tabs as spaces. + */ +QString withoutControls(const QString &s) +{ + QString out; + out.reserve(s.size()); + for (QChar c : s) { + const ushort u = c.unicode(); + if (u == '\t') { + out += QLatin1Char(' '); + continue; + } + if (u < 0x20 || u == 0x7F || u == 0xFFFE || u == 0xFFFF) { + continue; + } + out += c; + } + return out; +} + +// A label as it is shown and saved: trimmed, and not too long +QString labelFrom(const QString &text) +{ + QString s = text.trimmed(); + if (s.size() > Lyrics::maxLabelLength) { + int n = Lyrics::maxLabelLength; + // Never half of a surrogate pair + if (s.at(n - 1).isHighSurrogate()) --n; + s = s.left(n).trimmed(); + } + return s; +} + +// Nothing but notes and space: the exporter's mark for a gap +bool isOnlyMusic(const QString &text) +{ + for (QChar c : text) { + if (c != QChar(0x266A) && c != QChar(0x266B) && !c.isSpace()) { + return false; + } + } + return true; +} + +/** + * The text of the file. A BOM says what it is; without one it ought + * to be UTF-8, and anything that is not is read as Latin-1. That + * decodes every byte, keeps the ä and ö of an older Finnish file, and + * gives the same answer on every system, as the system's own codec + * would not. Stateless, so that a sequence cut off at the end counts + * as an error too. + */ +QString decoded(const QByteArray &bytes, QStringList &warnings) +{ + const QStringConverter::Flags flags = QStringConverter::Flag::Stateless; + + std::optional bom = + QStringConverter::encodingForData(bytes); + if (bom) { + QStringDecoder decoder(*bom, flags); + QString text = decoder.decode(bytes); + if (decoder.hasError()) { + warnings << tr("Some characters in the file could not be read " + "and were replaced."); + } + return text; + } + + QStringDecoder utf8(QStringConverter::Utf8, flags); + QString text = utf8.decode(bytes); + if (!utf8.hasError()) return text; + + warnings << tr("The file is not UTF-8, so it was read as Latin-1."); + QStringDecoder latin1(QStringConverter::Latin1, flags); + return latin1.decode(bytes); +} + +// A word, or a whole line, before it has a line index +struct Entry +{ + Ms start = 0; + Ms end = 0; + bool haveEnd = false; + bool endGiven = false; + QString text; +}; + +struct TimedLine +{ + Ms stamp = 0; + + /// Its entries are words, not the whole line + bool hasWordTags = false; + + /// None: the line only marks where the one before it ends + QVector entries; +}; + +/** + * The entries of the text after a line's time tags. Words are split + * on word tags only, never on spaces: the exporter leaves the space + * out before a word with punctuation in it (onzestilo,). Each tag + * starts a word and ends the one before it; a tag with no text after + * it only ends the one before. Text before the first tag starts at + * the line's own time. + */ +TimedLine readLineText(const QString &text, Ms stamp, int &backwards) +{ + TimedLine line; + line.stamp = stamp; + + struct Piece { + Ms time; + bool tagged; + QString text; + }; + + QVector pieces; + Piece current { stamp, false, QString() }; + const int n = text.size(); + int i = 0; + while (i < n) { + Ms time = 0; + int length = readTimeTag(text, i, '<', '>', time); + if (length > 0) { + pieces.push_back(current); + current = Piece { time, true, QString() }; + line.hasWordTags = true; + i += length; + } else { + // Anything else, a '<' that starts no tag included, is text + current.text += text[i]; + ++i; + } + } + pieces.push_back(current); + + for (const Piece &piece : pieces) { + Ms time = piece.time; + if (piece.tagged && !line.entries.isEmpty()) { + Entry &previous = line.entries.last(); + if (time < previous.start) { + time = previous.start; + ++backwards; + } + if (!previous.haveEnd) { + previous.end = time; + previous.haveEnd = true; + previous.endGiven = true; + } + } + QString label = labelFrom(piece.text); + if (label.isEmpty()) continue; + Entry entry; + entry.start = time; + entry.text = label; + line.entries.push_back(entry); + } + + bool onlyMusic = true; + for (const Entry &e : line.entries) { + if (!isOnlyMusic(e.text)) onlyMusic = false; + } + if (onlyMusic) line.entries.clear(); + + return line; +} + +TimedLine shifted(TimedLine line, Ms by) +{ + line.stamp += by; + for (Entry &e : line.entries) { + e.start += by; + e.end += by; + } + return line; +} + +/** + * Where each line's last word ends when the line itself does not say: + * where a following empty or music-only line puts it, else at the + * next line, but a word no more than a short while after its start. + * The lines must be in time order. + */ +void inferEnds(QVector &lines) +{ + const Ms wordMs = Ms(std::llround(Lyrics::inferredWordSeconds * + msPerSecond)); + const Ms lastLineMs = Ms(std::llround(Lyrics::inferredLastLineSeconds * + msPerSecond)); + + for (int k = 0; k < lines.size(); ++k) { + + if (lines[k].entries.isEmpty()) continue; + + // The words before it all end where the next word tag is + Entry &last = lines[k].entries.last(); + if (last.haveEnd) continue; + + const bool isWord = lines[k].hasWordTags; + + if (k + 1 < lines.size()) { + const TimedLine &next = lines[k + 1]; + if (next.entries.isEmpty()) { + last.end = next.stamp; + last.endGiven = true; + } else if (isWord) { + last.end = std::min(next.stamp, last.start + wordMs); + } else { + last.end = next.stamp; + } + } else { + last.end = last.start + (isWord ? wordMs : lastLineMs); + } + + // A line whose words run past the next line's start: this one + // gets no length, which conversion makes one frame + last.end = std::max(last.end, last.start); + last.haveEnd = true; + } +} + +} // namespace + +int +Lyrics::lineCount() const +{ + // Words need not be in line order: lines can overlap + int count = 0; + for (const LyricWord &w : words) count = std::max(count, w.line + 1); + return count; +} + +LyricsParseResult +parseLrc(const QByteArray &bytes) +{ + LyricsParseResult result; + + if (bytes.isEmpty()) { + result.error = tr("The file is empty."); + return result; + } + if (bytes.size() > Lyrics::maxFileBytes) { + result.error = tr("The file is over 1 MB, too big to be an LRC file."); + return result; + } + + QString text = decoded(bytes, result.warnings); + text.replace(QLatin1String("\r\n"), QLatin1String("\n")); + text.replace(QLatin1Char('\r'), QLatin1Char('\n')); + + Lyrics &lyrics = result.lyrics; + QVector lines; + Ms offset = 0; + int unrecognised = 0; + int backwards = 0; + + const QStringList rows = text.split(QLatin1Char('\n')); + for (const QString &raw : rows) { + + const QString row = withoutControls(raw).trimmed(); + if (row.isEmpty()) continue; + + QVector stamps; + int pos = 0; + Ms stamp = 0; + while (int length = readTimeTag(row, pos, '[', ']', stamp)) { + stamps.push_back(stamp); + pos += length; + } + + if (!stamps.isEmpty()) { + TimedLine line = readLineText(row.mid(pos), stamps[0], backwards); + if (line.hasWordTags) lyrics.wordTimed = true; + // [00:12.00][01:30.00]chorus: the line again at each stamp, + // its word tags moving with it + for (Ms s : stamps) { + lines.push_back(shifted(line, s - stamps[0])); + } + continue; + } + + QString key, value; + if (readMetadata(row, key, value)) { + if (key == QLatin1String("offset")) { + bool ok = false; + int ms = value.toInt(&ok); + if (ok) { + offset = ms; + } else { + result.warnings << tr("The offset \"%1\" is not a whole " + "number of milliseconds and was " + "ignored.").arg(labelFrom(value)); + } + } else if (key == QLatin1String("ti")) { + lyrics.title = labelFrom(value); + } else if (key == QLatin1String("ar")) { + lyrics.artist = labelFrom(value); + } + continue; + } + + ++unrecognised; + } + + // The exporter does not promise stamps in order, and it clamps + // negative times to 0, so that several lines can share stamp 0: + // those stay in file order + std::stable_sort(lines.begin(), lines.end(), + [](const TimedLine &a, const TimedLine &b) { + return a.stamp < b.stamp; + }); + + inferEnds(lines); + + // A positive offset makes the lyrics appear sooner, as LRC has it + const Ms shift = -offset; + + int dropped = 0; + int lineIndex = 0; + for (const TimedLine &line : lines) { + bool any = false; + for (const Entry &e : line.entries) { + if (e.start + shift < 0) { + ++dropped; + continue; + } + LyricWord word; + word.start = double(e.start + shift) / msPerSecond; + word.end = double(e.end + shift) / msPerSecond; + word.text = e.text; + word.line = lineIndex; + word.endGiven = e.endGiven; + lyrics.words.push_back(word); + any = true; + } + if (any) ++lineIndex; + } + + std::stable_sort(lyrics.words.begin(), lyrics.words.end(), + [](const LyricWord &a, const LyricWord &b) { + return a.start < b.start; + }); + + if (unrecognised > 0) { + result.warnings << counted + (unrecognised, + "1 line was not LRC and was skipped.", + "%1 lines were not LRC and were skipped."); + } + if (backwards > 0) { + result.warnings << counted + (backwards, + "1 word tag went back in time and was moved up to the word " + "before it.", + "%1 word tags went back in time and were moved up to the " + "word before them."); + } + if (dropped > 0) { + result.warnings << counted + (dropped, + "1 word came before the start of the song after the offset " + "and was dropped.", + "%1 words came before the start of the song after the offset " + "and were dropped."); + } + + if (lyrics.words.isEmpty()) { + result.lyrics = Lyrics(); + result.error = tr("No timed lyrics were found: this is not an LRC " + "file, or it has no timed lines."); + } + + return result; +} + +EventVector +lyricsToEvents(const Lyrics &lyrics, sv_samplerate_t rate) +{ + EventVector events; + for (const LyricWord &w : lyrics.words) { + sv_frame_t frame = sv_frame_t(std::llround(w.start * rate)); + sv_frame_t end = sv_frame_t(std::llround(w.end * rate)); + sv_frame_t duration = std::max(sv_frame_t(1), end - frame); + // Two words at the same frame are two events: a model keeps + // both, whether or not their labels differ + events.push_back(Event(frame, float(w.line), duration, w.text)); + } + std::sort(events.begin(), events.end()); + return events; +} + +Lyrics +lyricsFromEvents(const EventVector &events, sv_samplerate_t rate) +{ + Lyrics lyrics; + if (rate <= 0) return lyrics; + + // In time order, then in line order, as the parser leaves them. + // Words of one line at one frame go as a model orders them, + // shortest first: all but the last of them end at that frame, so + // this is the parser's order too, but for a tie. The order the + // events come in makes no difference. + EventVector sorted(events); + std::sort(sorted.begin(), sorted.end(), + [](const Event &a, const Event &b) { + if (a.getFrame() != b.getFrame()) { + return a.getFrame() < b.getFrame(); + } + if (a.getValue() != b.getValue()) { + return a.getValue() < b.getValue(); + } + return a < b; + }); + + for (const Event &e : sorted) { + LyricWord word; + word.start = double(e.getFrame()) / rate; + word.end = double(e.getFrame() + e.getDuration()) / rate; + word.text = e.getLabel(); + word.line = int(std::lround(e.getValue())); + lyrics.words.push_back(word); + } + return lyrics; +} diff --git a/main/Lyrics.h b/main/Lyrics.h new file mode 100644 index 00000000..da57ab83 --- /dev/null +++ b/main/Lyrics.h @@ -0,0 +1,126 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_H +#define TONY_LYRICS_H + +#include "base/BaseTypes.h" +#include "base/Event.h" + +#include +#include +#include +#include + +/** + * Timed lyrics read from an LRC file, and the events of the region + * model that holds them in a session: one region per word (or per + * line, for a file that times only lines), with the line's index as + * its value. + * + * These are pure functions over bytes and event lists: they touch no + * model, so they can be tested without a window (TestLyrics). + */ + +/** + * One word of the lyrics, or a whole line when the file times only + * lines. + */ +struct LyricWord +{ + /// Seconds on the reference's timeline, after the file's [offset:] + double start = 0.0; + + /// Seconds; never before start, but it can be start itself, for a + /// word the next one starts at the same time as. Conversion gives + /// such a word one frame + double end = 0.0; + + /// One word, or a whole line + QString text; + + /// 0-based index of the line, in time order + int line = 0; + + /// The file gave the end; false if it was inferred + bool endGiven = false; +}; + +struct Lyrics +{ + /// Sorted by start + QVector words; + + /// From [ti:] and [ar:], for the layer's name + QString title; + QString artist; + + /// Any word tags were seen + bool wordTimed = false; + + bool isEmpty() const { return words.isEmpty(); } + + /// How many lines have words: the lines are numbered from 0 up + int lineCount() const; + + /** + * How long a word lasts, at most, when the file does not say + * where it ends: the next line's start, or the end of the file, + * would otherwise draw its bar through an instrumental break. + */ + static constexpr double inferredWordSeconds = 2.0; + + /// How long the last line lasts in a file that times only lines + static constexpr double inferredLastLineSeconds = 5.0; + + /// A longer word or line is cut to this many characters + static constexpr int maxLabelLength = 200; + + /// LRC files are a few kB; a bigger file is not read at all + static constexpr qint64 maxFileBytes = 1024 * 1024; +}; + +struct LyricsParseResult +{ + Lyrics lyrics; + + /// Non-empty if there is nothing usable (not LRC, no timed lines, + /// too big); lyrics is then empty + QString error; + + /// Short sentences on what was skipped or changed, for the status bar + QStringList warnings; +}; + +/** + * Read an LRC file, with line timing ([mm:ss.xx]text) or word timing + * ([mm:ss.xx]word word ...). + */ +LyricsParseResult parseLrc(const QByteArray &bytes); + +/** + * The events of a region model holding the lyrics: frame = start, + * duration = end - start (at least 1 frame), value = line, label = + * text. Sorted as a model holds them, so that the two compare equal. + */ +sv::EventVector lyricsToEvents(const Lyrics &lyrics, sv::sv_samplerate_t rate); + +/** + * The words of those events, back again. The events keep only the + * words: title, artist, wordTimed and each endGiven are left empty + * and false. + */ +Lyrics lyricsFromEvents(const sv::EventVector &events, sv::sv_samplerate_t rate); + +#endif diff --git a/main/test/TestLyrics.h b/main/test/TestLyrics.h new file mode 100644 index 00000000..8b4e9d97 --- /dev/null +++ b/main/test/TestLyrics.h @@ -0,0 +1,793 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LYRICS_H +#define TEST_LYRICS_H + +// Tier 2: reading LRC files, and the events the lyrics become. No +// window and no model: what the application does with them is Tier +// 5's business (TestRecordWorkflow). The files in testdata/lyrics are +// written as Moises-Lyric-Exporter writes them, with invented text. + +#include "../Lyrics.h" + +#include "base/EventSeries.h" +#include "base/XmlExportable.h" + +#include +#include +#include +#include +#include + +#include + +class TestLyrics : public QObject +{ + Q_OBJECT + + // The inferred lengths, as the parser has them + static constexpr double W = Lyrics::inferredWordSeconds; + static constexpr double L = Lyrics::inferredLastLineSeconds; + + static LyricsParseResult parse(const char *text) { + return parseLrc(QByteArray(text)); + } + + static QByteArray fixture(const char *name) { + QFile file(QString(TONY_TEST_DATA_DIR) + "/lyrics/" + name); + if (!file.open(QIODevice::ReadOnly)) return QByteArray(); + return file.readAll(); + } + + static int wordCount(const LyricsParseResult &r) { + return int(r.lyrics.words.size()); + } + + struct Expected { + double start; + double end; + const char *text; + int line; + bool endGiven; + }; + + static QString describe(const LyricWord &w) { + return QString("\"%1\" %2-%3 line %4%5") + .arg(w.text).arg(w.start, 0, 'f', 3).arg(w.end, 0, 'f', 3) + .arg(w.line).arg(w.endGiven ? ", end given" : ""); + } + + static QString describe(const Lyrics &lyrics) { + QStringList list; + for (const LyricWord &w : lyrics.words) list << describe(w); + return list.join("; "); + } + + // Every LRC time is whole milliseconds, and so is every expected + // time here: this only absorbs the last bit of a double + static bool sameTime(double a, double b) { + return std::abs(a - b) < 1e-9; + } + + // All the words, in order. A failure names the word that differs. + static void compareWords(const Lyrics &lyrics, + const QVector &expected) { + QVERIFY2(lyrics.words.size() == expected.size(), + qPrintable(QString("%1 words, expected %2: %3") + .arg(lyrics.words.size()).arg(expected.size()) + .arg(describe(lyrics)))); + for (int i = 0; i < expected.size(); ++i) { + const LyricWord &w = lyrics.words[i]; + LyricWord want; + want.start = expected[i].start; + want.end = expected[i].end; + want.text = QString::fromUtf8(expected[i].text); + want.line = expected[i].line; + want.endGiven = expected[i].endGiven; + bool same = (w.text == want.text && + sameTime(w.start, want.start) && + sameTime(w.end, want.end) && + w.line == want.line && + w.endGiven == want.endGiven); + QVERIFY2(same, qPrintable(QString("word %1 is %2, expected %3") + .arg(i).arg(describe(w)) + .arg(describe(want)))); + } + } + +private slots: + // Line timing: a line lasts until the next one starts, however far + // away that is, and the last one for a default length + void line_lrc_starts_and_ends() { + LyricsParseResult r = parse("[00:01.00]Ensimmäinen rivi\n" + "[00:04.50]Toinen rivi\n" + "[00:30.00]Kolmas rivi\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QVERIFY(!r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 1.0, 4.5, "Ensimmäinen rivi", 0, false }, + { 4.5, 30.0, "Toinen rivi", 1, false }, + { 30.0, 30.0 + L, "Kolmas rivi", 2, false }, + }); + } + + // One fraction digit is tenths, two hundredths, three thousandths, + // and ':' does as well as '.' + void time_tag_precision() { + const char *tags[] = { + "[01:02.5]", "[01:02.50]", "[01:02.500]", "[01:02:50]", + "[01:02:5]", "[01:02:500]" + }; + for (const char *tag : tags) { + LyricsParseResult r = parseLrc(QByteArray(tag) + "sana\n"); + QVERIFY2(r.error.isEmpty(), tag); + QCOMPARE(wordCount(r), 1); + QVERIFY2(r.lyrics.words[0].start == 62.5, tag); + } + + LyricsParseResult r = parse("[01:02]sana\n"); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].start, 62.0); + + r = parse("[00:01.234]a\n[00:01.3]b\n"); + QCOMPARE(wordCount(r), 2); + QCOMPARE(r.lyrics.words[0].start, 1.234); + QCOMPARE(r.lyrics.words[1].start, 1.3); + + // Word tags are read the same way + r = parse("[01:00.00]<01:02.5>a <01:03:25>b <01:04.125>c\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 62.5, 63.25, "a", 0, true }, + { 63.25, 64.125, "b", 0, true }, + { 64.125, 64.125 + W, "c", 0, false }, + }); + + // Four fraction digits make no tag + r = parse("[00:01.2345]ei\n[00:02.00]kyllä\n"); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, QString("kyllä")); + QCOMPARE(r.warnings.size(), qsizetype(1)); + } + + // Songs over an hour: minutes have as many digits as they need + void minutes_over_59() { + LyricsParseResult r = parse("[75:00.00]pitkä\n" + "[123:04.5]<123:04.5>hyvin <123:05.25>pitkä\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 4500.0, 7384.5, "pitkä", 0, false }, + { 7384.5, 7385.25, "hyvin", 1, true }, + { 7385.25, 7385.25 + W, "pitkä", 1, false }, + }); + + // but not so many that the arithmetic overflows: that is no + // time tag, and the line is skipped + r = parse("[00:01.00]sana\n[99999999999999999999:00.00]ei\n"); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.warnings.size(), qsizetype(1)); + } + + // Seconds of 60 or more make no time tag: the line is skipped + void seconds_of_60_or_more_skip_the_line() { + LyricsParseResult r = parse("[00:60.00]ei\n" + "[00:01.00]kyllä\n" + "[01:75.50]ei tämäkään\n"); + QCOMPARE(r.error, QString()); + compareWords(r.lyrics, { { 1.0, 1.0 + L, "kyllä", 0, false } }); + QCOMPARE(r.warnings.size(), qsizetype(1)); + QVERIFY2(r.warnings[0].startsWith("2 "), qPrintable(r.warnings[0])); + + // Inside a line such a word tag is just text + r = parse("[00:01.00]<00:01.00>a <00:61.00>b\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { { 1.0, 1.0 + W, "a <00:61.00>b", 0, false } }); + } + + // [00:12.00][01:30.00]chorus puts the line in at both times, each + // sorted into place + void several_leading_time_tags_repeat_the_line() { + LyricsParseResult r = parse("[00:30.00]säkeistö\n" + "[00:12.00][01:30.00]kertosäe\n" + "[01:00.00]väliosa\n"); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.lineCount(), 4); + compareWords(r.lyrics, { + { 12.0, 30.0, "kertosäe", 0, false }, + { 30.0, 60.0, "säkeistö", 1, false }, + { 60.0, 90.0, "väliosa", 2, false }, + { 90.0, 90.0 + L, "kertosäe", 3, false }, + }); + + // Word tags move with the stamp the line is repeated at + r = parse("[00:10.00][00:20.00]<00:10.00>la <00:10.50>lu\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 10.0, 10.5, "la", 0, true }, + { 10.5, 10.5 + W, "lu", 0, false }, + { 20.0, 20.5, "la", 1, true }, + { 20.5, 20.5 + W, "lu", 1, false }, + }); + } + + // Only the title and the artist are kept; other metadata, whatever + // the case of its key, is neither lyrics nor a problem + void metadata_ignored_title_and_artist_kept() { + LyricsParseResult r = parse("[ti: Kesäyö ]\n" + "[AR:Testiryhmä]\n" + "[al:Levy]\n" + "[by:Joku]\n" + "[au:Tekijä]\n" + "[length: 03:25]\n" + "[#:kommentti]\n" + "[00:01.00]sana\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString("Kesäyö")); + QCOMPARE(r.lyrics.artist, QString("Testiryhmä")); + compareWords(r.lyrics, { { 1.0, 1.0 + L, "sana", 0, false } }); + } + + // A positive [offset:] makes the lyrics appear sooner, as LRC has + // it; a negative one later + void offset_sign() { + LyricsParseResult r = parse("[offset:+500]\n" + "[00:01.00]yksi\n" + "[00:03.00]<00:03.00>kaksi <00:03.40>kolme\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 0.5, 2.5, "yksi", 0, false }, + { 2.5, 2.9, "kaksi", 1, true }, + { 2.9, 2.9 + W, "kolme", 1, false }, + }); + + r = parse("[offset:-500]\n[00:01.00]yksi\n"); + compareWords(r.lyrics, { { 1.5, 1.5 + L, "yksi", 0, false } }); + + // Wherever it is in the file, it counts for all of it + r = parse("[00:01.00]yksi\n[offset:250]\n"); + compareWords(r.lyrics, { { 0.75, 0.75 + L, "yksi", 0, false } }); + + // A word it pushes below 0 is dropped, and the lines are + // numbered without it + r = parse("[offset:+1500]\n[00:01.00]yksi\n[00:03.00]kaksi\n"); + compareWords(r.lyrics, { { 1.5, 1.5 + L, "kaksi", 0, false } }); + QCOMPARE(r.warnings.size(), qsizetype(1)); + + // Exactly 0 is not below it + r = parse("[offset:1000]\n[00:01.00]yksi\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { { 0.0, L, "yksi", 0, false } }); + + // Not a number: no offset, and a warning + r = parse("[offset:puoli sekuntia]\n[00:01.00]yksi\n"); + compareWords(r.lyrics, { { 1.0, 1.0 + L, "yksi", 0, false } }); + QCOMPARE(r.warnings.size(), qsizetype(1)); + } + + // Word timing: a word ends where the next word tag is, or where a + // trailing tag says; the last word of a line with neither ends at + // the next line, or a default length after its start + void word_lrc_starts_and_ends() { + LyricsParseResult r = parse + ("[00:01.00]<00:01.00>Päivä <00:01.50>paistaa <00:02.20>ja<00:02.60>\n" + "[00:05.00]<00:05.00>kuu <00:05.40>nousee\n" + "[00:06.00]<00:06.00>yö\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QVERIFY(r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 1.0, 1.5, "Päivä", 0, true }, + { 1.5, 2.2, "paistaa", 0, true }, + { 2.2, 2.6, "ja", 0, true }, + { 5.0, 5.4, "kuu", 1, true }, + { 5.4, 6.0, "nousee", 1, false }, + { 6.0, 6.0 + W, "yö", 2, false }, + }); + + // Text before the first word tag starts at the line's own time + r = parse("[00:10.00] Alku <00:10.50>loppu\n"); + compareWords(r.lyrics, { + { 10.0, 10.5, "Alku", 0, true }, + { 10.5, 10.5 + W, "loppu", 0, false }, + }); + } + + // A timed line with no text, or only notes, ends the line before + // it and is no line itself. With neither, the last word of a line + // lasts until the next line, but no longer than the cap. + void empty_or_music_line_ends_the_line_before() { + LyricsParseResult r = parse("[00:01.00]<00:01.00>yksi <00:01.50>kaksi\n" + "[00:04.00]\n" + "[00:10.00]<00:10.00>kolme\n" + "[00:12.50] ♪\n" + "[00:15.00]<00:15.00>neljä\n" + "[00:15.80] ♫ ♪ \n" + "[00:20.00]<00:20.00>viisi <00:20.40>kuusi\n" + "[00:30.00]<00:30.00>seitsemän\n"); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.lineCount(), 5); + compareWords(r.lyrics, { + { 1.0, 1.5, "yksi", 0, true }, + { 1.5, 4.0, "kaksi", 0, true }, + { 10.0, 12.5, "kolme", 1, true }, + { 15.0, 15.8, "neljä", 2, true }, + { 20.0, 20.4, "viisi", 3, true }, + { 20.4, 20.4 + W, "kuusi", 3, false }, + { 30.0, 30.0 + W, "seitsemän", 4, false }, + }); + + // The same for line timing, where nothing but such a line caps + // how long a line lasts + r = parse("[00:01.00]rivi yksi\n" + "[00:03.00]\n" + "[00:10.00]rivi kaksi\n" + "[00:12.00]♫\n" + "[00:20.00]rivi kolme\n" + "[00:40.00]rivi neljä\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 3.0, "rivi yksi", 0, true }, + { 10.0, 12.0, "rivi kaksi", 1, true }, + { 20.0, 40.0, "rivi kolme", 2, false }, + { 40.0, 40.0 + L, "rivi neljä", 3, false }, + }); + } + + // The exporter leaves out the space before a word with punctuation + // in it: words are split on the tags, never on spaces, and the + // spaces around them are not part of them + void glued_words_split_on_tags() { + LyricsParseResult r = parse + ("[00:04.640] <00:04.640>Tämä <00:04.920>on <00:05.160>keksitty<00:05.620>laulu,\n" + "[00:07.000] <00:07.000> väli <00:07.500> lopussa \n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 4.64, 4.92, "Tämä", 0, true }, + { 4.92, 5.16, "on", 0, true }, + { 5.16, 5.62, "keksitty", 0, true }, + { 5.62, 7.0, "laulu,", 0, false }, + { 7.0, 7.5, "väli", 1, true }, + { 7.5, 7.5 + W, "lopussa", 1, false }, + }); + + // Nor is one tag's text split at its spaces: it is one label + r = parse("[00:01.00]<00:01.00>kaksi sanaa<00:02.00>\n"); + compareWords(r.lyrics, { { 1.0, 2.0, "kaksi sanaa", 0, true } }); + } + + // The exporter clamps negative times to 0, so several lines can + // share that stamp: all of them stay, in file order + void lines_stamped_zero_kept_in_file_order() { + LyricsParseResult r = parse("[00:00.000] <00:00.000>eka\n" + "[00:00.000] <00:00.000>toka\n" + "[00:00.000] <00:00.000>kolmas\n" + "[00:02.500] <00:02.500>neljäs\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 0.0, 0.0, "eka", 0, false }, + { 0.0, 0.0, "toka", 1, false }, + { 0.0, W, "kolmas", 2, false }, + { 2.5, 2.5 + W, "neljäs", 3, false }, + }); + + // and all of them become events, one frame long at least + sv::EventVector events = lyricsToEvents(r.lyrics, 44100); + QCOMPARE(int(events.size()), 4); + for (int i = 0; i < 3; ++i) { + QCOMPARE(events[i].getFrame(), sv::sv_frame_t(0)); + QVERIFY(events[i].getDuration() >= 1); + } + + r = parse("[00:00.00]eka\n[00:00.00]toka\n[00:01.00]kolmas\n"); + compareWords(r.lyrics, { + { 0.0, 0.0, "eka", 0, false }, + { 0.0, 1.0, "toka", 1, false }, + { 1.0, 1.0 + L, "kolmas", 2, false }, + }); + } + + // A metadata value runs to the last ']'; the exporter's own header + // lines are no trouble + void bracketed_title_and_exporter_header() { + LyricsParseResult r = parse + ("[ti:Laulu [Live]]\n" + "[re:Moises Lyrics Exporter Pro]\n" + "[ve:2.0]\n" + "[co:avg_confidence=1.000; low_confidence_count=0]\n" + "\n" + "[00:01.000] <00:01.000>sana\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString("Laulu [Live]")); + QCOMPARE(r.lyrics.artist, QString()); + compareWords(r.lyrics, { { 1.0, 1.0 + W, "sana", 0, false } }); + } + + // A word tag earlier than the word before it is moved up to that + // word's start, with one warning that counts them + void backward_word_tags_clamped() { + LyricsParseResult r = parse + ("[00:10.00]<00:10.00>a <00:09.50>b <00:10.50>c<00:10.20>\n"); + compareWords(r.lyrics, { + { 10.0, 10.0, "a", 0, true }, + { 10.0, 10.5, "b", 0, true }, + { 10.5, 10.5, "c", 0, true }, + }); + QCOMPARE(r.warnings.size(), qsizetype(1)); + QVERIFY2(r.warnings[0].startsWith("2 "), qPrintable(r.warnings[0])); + } + + // Lines are put in time order, and numbered in it; a line's words + // go with it + void unsorted_lines_sorted() { + LyricsParseResult r = parse("[00:20.00]kolmas\n" + "[00:05.00]ensimmäinen\n" + "[00:10.00]toinen\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 5.0, 10.0, "ensimmäinen", 0, false }, + { 10.0, 20.0, "toinen", 1, false }, + { 20.0, 20.0 + L, "kolmas", 2, false }, + }); + + r = parse("[00:08.00]<00:08.00>c <00:08.50>d\n" + "[00:02.00]<00:02.00>a <00:02.50>b\n"); + compareWords(r.lyrics, { + { 2.0, 2.5, "a", 0, true }, + { 2.5, 2.5 + W, "b", 0, false }, + { 8.0, 8.5, "c", 1, true }, + { 8.5, 8.5 + W, "d", 1, false }, + }); + } + + // Finnish text survives UTF-8 with and without a BOM, and UTF-16 + // and UTF-32 with one + void unicode_with_and_without_bom() { + const QString title = QString::fromUtf8("Hämärä yö"); + const QString line = QString::fromUtf8("Äiti öisin söi jäätelöä"); + const QString text = "[ti:" + title + "]\n[00:01.00]" + line + "\n"; + + QVector files; + files << text.toUtf8(); + files << QByteArray("\xEF\xBB\xBF") + text.toUtf8(); + const QStringConverter::Encoding encodings[] = { + QStringConverter::Utf16LE, QStringConverter::Utf16BE, + QStringConverter::Utf32LE, QStringConverter::Utf32BE + }; + for (QStringConverter::Encoding e : encodings) { + QStringEncoder encoder(e, QStringConverter::Flag::WriteBom); + QByteArray bytes = encoder.encode(text); + QVERIFY(QStringConverter::encodingForData(bytes) == e); + files << bytes; + } + + for (int i = 0; i < files.size(); ++i) { + LyricsParseResult r = parseLrc(files[i]); + QVERIFY2(r.error.isEmpty(), qPrintable(QString::number(i))); + QVERIFY2(r.warnings.isEmpty(), + qPrintable(QString("%1: %2").arg(i) + .arg(r.warnings.join(" ")))); + QCOMPARE(r.lyrics.title, title); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, line); + } + } + + // Bytes that are not UTF-8 are read as Latin-1, with a warning, so + // that the ä and ö of an older file survive + void non_utf8_read_as_latin1() { + LyricsParseResult r = parse("[ti:H\xE4m\xE4r\xE4 y\xF6]\n" + "[00:01.00]\xC4iti \xF6isin\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.lyrics.title, QString::fromUtf8("Hämärä yö")); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, QString::fromUtf8("Äiti öisin")); + QCOMPARE(r.warnings.size(), qsizetype(1)); + + // Even when the only such byte is the file's last, where a + // decoder that waits for more would say nothing + r = parse("[00:01.00]Hyv\xE4"); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, QString::fromUtf8("Hyvä")); + QCOMPARE(r.warnings.size(), qsizetype(1)); + + // With a BOM the file says it is UTF-8: what cannot be read is + // replaced, with a warning + r = parse("\xEF\xBB\xBF[00:01.00]a\xFF" "b\n"); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, QString("a") + QChar(0xFFFD) + "b"); + QCOMPARE(r.warnings.size(), qsizetype(1)); + } + + // CRLF, LF and CR, with or without a newline at the end + void line_endings() { + const char *files[] = { + "[00:01.00]yksi\r\n[00:02.00]kaksi\r\n[00:03.00]kolme", + "[00:01.00]yksi\r\n[00:02.00]kaksi\r\n[00:03.00]kolme\r\n", + "[00:01.00]yksi\r[00:02.00]kaksi\r[00:03.00]kolme\r", + "[00:01.00]yksi\n[00:02.00]kaksi\n[00:03.00]kolme", + }; + for (const char *file : files) { + LyricsParseResult r = parse(file); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 2.0, "yksi", 0, false }, + { 2.0, 3.0, "kaksi", 1, false }, + { 3.0, 3.0 + L, "kolme", 2, false }, + }); + if (QTest::currentTestFailed()) return; + } + } + + // Control characters go, as the session file (XML 1.0) could not + // be read back with one in a label; a tab becomes a space, as a + // session file would give it back. A long word is cut. + void control_characters_stripped_and_long_words_cut() { + QByteArray bytes("[ti:Ni\x01mi]\n[00:01.00]a"); + bytes += '\0'; + bytes += "b\x1F" "c\x7F" "d\te"; + bytes += "\xEF\xBF\xBF" "f\xEF\xBF\xBE" "g\n"; // U+FFFF, U+FFFE + bytes += "[00:02.00]<00:02.00>h\x02i <00:02.50>\x03 <00:03.00>j\n"; + LyricsParseResult r = parseLrc(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + + // First what it is for: the labels and the title, as a session + // holds them, are well-formed XML + QString xml = "\n" + "\n"; + for (const sv::Event &e : lyricsToEvents(r.lyrics, 44100)) { + xml += e.toXmlString(" "); + } + xml += "\n"; + QXmlStreamReader reader(xml.toUtf8()); + int points = 0; + while (!reader.atEnd()) { + if (reader.readNext() == QXmlStreamReader::StartElement && + reader.name() == QLatin1String("point")) { + ++points; + } + } + QVERIFY2(!reader.hasError(), qPrintable(reader.errorString())); + QCOMPARE(points, 3); + + QCOMPARE(r.lyrics.title, QString("Nimi")); + compareWords(r.lyrics, { + { 1.0, 2.0, "abcd efg", 0, false }, + { 2.0, 2.5, "hi", 1, true }, + { 3.0, 3.0 + W, "j", 1, false }, + }); + if (QTest::currentTestFailed()) return; + + // A word, or a line, of 1000 characters is cut to the limit + r = parseLrc("[00:01.00]<00:01.00>" + QByteArray(1000, 'x') + + " <00:02.00>lyhyt\n[00:03.00]" + QByteArray(1000, 'y')); + QCOMPARE(wordCount(r), 3); + QCOMPARE(r.lyrics.words[0].text, QString(Lyrics::maxLabelLength, 'x')); + QCOMPARE(r.lyrics.words[1].text, QString("lyhyt")); + QCOMPARE(r.lyrics.words[2].text, QString(Lyrics::maxLabelLength, 'y')); + + // never through the middle of a character that needs two + const char32_t clef[] = { 0x1D11E }; + QString text = QString(Lyrics::maxLabelLength - 1, 'z') + + QString::fromUcs4(clef, 1) + "zz"; + r = parseLrc(("[00:01.00]" + text).toUtf8()); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, + QString(Lyrics::maxLabelLength - 1, 'z')); + } + + // Brackets and ampersands that are not part of a tag are text + void markup_characters_kept() { + LyricsParseResult r = parse + ("[00:01.00]rock & roll [x] ad <00:6> <1:02.00\n" + "[00:03.00]<00:03.00>a&b <00:03.50><3 <00:04.00>[x]<00:04.50>x>y <00:05.00>&\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 3.0, "rock & roll [x] ad <00:6> <1:02.00", 0, false }, + { 3.0, 3.5, "a&b", 1, true }, + { 3.5, 4.0, "<3", 1, true }, + { 4.0, 4.5, "[x]", 1, true }, + { 4.5, 5.0, "x>y", 1, true }, + { 5.0, 5.0 + W, "&", 1, false }, + }); + } + + // Nothing usable is an error, and gives no lyrics + void unusable_files_are_errors() { + LyricsParseResult r = parse("Tämä on vain tekstiä\nilman aikoja\n"); + QVERIFY(!r.error.isEmpty()); + QVERIFY(r.lyrics.isEmpty()); + + // Timed lines, but none with words + r = parse("[ti:Nimi]\n[00:01.00]\n[00:02.00] ♪\n"); + QVERIFY(!r.error.isEmpty()); + QVERIFY(r.lyrics.isEmpty()); + QCOMPARE(r.lyrics.title, QString()); + + // Not text at all + QByteArray binary; + for (int i = 0; i < 4096; ++i) binary += char((i * 37) % 256); + r = parseLrc(binary); + QVERIFY(!r.error.isEmpty()); + + r = parseLrc(QByteArray()); + QVERIFY(!r.error.isEmpty()); + + // Exactly the limit is read; one byte more is not + QByteArray big("[00:01.00]sana\n"); + big += QByteArray(Lyrics::maxFileBytes - big.size(), '\n'); + r = parseLrc(big); + QCOMPARE(r.error, QString()); + QCOMPARE(wordCount(r), 1); + big += '\n'; + r = parseLrc(big); + QVERIFY(!r.error.isEmpty()); + QVERIFY(r.lyrics.isEmpty()); + } + + // Frames are the seconds times the rate, rounded; a word with no + // length gets one frame; the value is the line. Back again, the + // same words come out, to the frame. + void events_at_44100_and_48000() { + LyricsParseResult r = parse("[00:01.234]<00:01.234>a <00:01.234>b <00:02.000>c\n" + "[01:02.500]<01:02.500>d\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.234, 1.234, "a", 0, true }, + { 1.234, 2.0, "b", 0, true }, + { 2.0, 2.0 + W, "c", 0, false }, + { 62.5, 62.5 + W, "d", 1, false }, + }); + if (QTest::currentTestFailed()) return; + + // 1.234 s is 54419.4 frames at 44100, and 59232 at 48000 + const sv::sv_frame_t w44 = sv::sv_frame_t(std::llround(W * 44100)); + const sv::sv_frame_t w48 = sv::sv_frame_t(std::llround(W * 48000)); + struct Case { + double rate; + sv::EventVector expected; + } cases[] = { + { 44100, { + sv::Event(54419, 0.f, 1, "a"), + sv::Event(54419, 0.f, 88200 - 54419, "b"), + sv::Event(88200, 0.f, w44, "c"), + sv::Event(2756250, 1.f, w44, "d"), + } }, + { 48000, { + sv::Event(59232, 0.f, 1, "a"), + sv::Event(59232, 0.f, 96000 - 59232, "b"), + sv::Event(96000, 0.f, w48, "c"), + sv::Event(3000000, 1.f, w48, "d"), + } }, + }; + + for (const Case &c : cases) { + sv::EventVector events = lyricsToEvents(r.lyrics, c.rate); + QCOMPARE(int(events.size()), int(c.expected.size())); + for (int i = 0; i < int(events.size()); ++i) { + QVERIFY2(events[i] == c.expected[i], + qPrintable(events[i].toXmlString() + " expected " + + c.expected[i].toXmlString())); + } + + // Back from the events in any order, as a model would give + // them: the same words, and the same events again + sv::EventVector reversed(events.rbegin(), events.rend()); + Lyrics back = lyricsFromEvents(reversed, c.rate); + QCOMPARE(back.words.size(), r.lyrics.words.size()); + for (int i = 0; i < int(back.words.size()); ++i) { + const LyricWord &w = back.words[i]; + const LyricWord &o = r.lyrics.words[i]; + QCOMPARE(w.text, o.text); + QCOMPARE(w.line, o.line); + QVERIFY(std::abs(w.start - o.start) <= 0.5 / c.rate); + } + QVERIFY(lyricsToEvents(back, c.rate) == events); + } + } + + // Two words at the same frame are two events, even when nothing + // tells them apart, and a model's event series keeps both + void identical_words_at_one_frame_both_kept() { + LyricsParseResult r = parse("[00:01.00]<00:01.00>la<00:01.00>la<00:01.00>\n"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 1.0, "la", 0, true }, + { 1.0, 1.0, "la", 0, true }, + }); + + sv::EventVector events = lyricsToEvents(r.lyrics, 44100); + QCOMPARE(int(events.size()), 2); + QVERIFY(events[0] == events[1]); + + sv::EventSeries series; + for (const sv::Event &e : events) series.add(e); + QCOMPARE(series.count(), 2); + } + + // What Moises-Lyric-Exporter writes in word mode: no end times, + // clamped stamps at 0, glued words and a gap marker + void exporter_word_fixture() { + QByteArray bytes = fixture("moises-exporter-words.lrc"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseLrc(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString::fromUtf8("Kesäyön testilaulu")); + QVERIFY(r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 4); + compareWords(r.lyrics, { + { 0.0, 0.0, "Alku", 0, true }, + { 0.0, 0.31, "ennen", 0, true }, + { 0.31, 0.31 + W, "nollaa", 0, false }, + { 4.64, 4.92, "Tämä", 1, true }, + { 4.92, 5.16, "on", 1, true }, + { 5.16, 5.62, "keksitty", 1, true }, + { 5.62, 6.0, "laulu,", 1, true }, + { 21.5, 21.8, "Yö", 2, true }, + { 21.8, 22.05, "on", 2, true }, + { 22.05, 23.4, "hämärä", 2, false }, + { 23.4, 23.7, "Nyt", 3, true }, + { 23.7, 24.0, "se", 3, true }, + { 24.0, 24.0 + W, "loppuu!", 3, false }, + }); + } + + // The same in line mode, with two decimals + void exporter_line_fixture() { + QByteArray bytes = fixture("moises-exporter-lines.lrc"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseLrc(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString::fromUtf8("Kesäyön testilaulu")); + QVERIFY(!r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 4); + compareWords(r.lyrics, { + { 0.0, 4.64, "Alku ennen nollaa", 0, false }, + { 4.64, 6.0, "Tämä on keksittylaulu,", 1, true }, + { 21.5, 23.4, "Yö on hämärä", 2, false }, + { 23.4, 23.4 + L, "Nyt seloppuu!", 3, false }, + }); + } + + // An enhanced LRC that gives its ends: trailing word tags and + // empty lines + void fixture_with_ends() { + QByteArray bytes = fixture("lrc-with-ends.lrc"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseLrc(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString::fromUtf8("Päivä [Live]")); + QCOMPARE(r.lyrics.artist, QString::fromUtf8("Testiryhmä")); + QVERIFY(r.lyrics.wordTimed); + compareWords(r.lyrics, { + { 1.0, 1.5, "Päivä", 0, true }, + { 1.5, 2.2, "paistaa", 0, true }, + { 3.0, 3.4, "Pöllö", 1, true }, + { 3.4, 4.8, "huhuilee", 1, true }, + { 6.0, 8.5, "Koko rivi yhdellä leimalla", 2, true }, + }); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 4a7ed2b5..df63a235 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -20,6 +20,7 @@ #include "TestSingingTakes.h" #include "TestTakesFile.h" #include "TestTakeTiming.h" +#include "TestLyrics.h" #include "RunSuite.h" @@ -98,6 +99,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLyrics t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 8f8b0e6e..597bb75e 100644 --- a/meson.build +++ b/meson.build @@ -1095,6 +1095,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/Coverage.cpp', + 'main/Lyrics.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', @@ -1356,8 +1357,13 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestSingingTakes.h', 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', + 'main/test/TestLyrics.h', ]) +# Where the suites find the files in testdata/. Forward slashes: a +# backslash of a Windows path would start an escape in the C string. +tony_test_data_dir = import('fs').as_posix(meson.current_source_dir() / 'testdata') + tony_core_test_exe = executable( 'test-tony-core', tony_core_test_moc_files, @@ -1375,6 +1381,7 @@ tony_core_test_exe = executable( cpp_args: [ feature_defines, general_defines, + '-DTONY_TEST_DATA_DIR="' + tony_test_data_dir + '"', ], link_args: [ feature_additional_libs, diff --git a/testdata/lyrics/lrc-with-ends.lrc b/testdata/lyrics/lrc-with-ends.lrc new file mode 100644 index 00000000..3792903f --- /dev/null +++ b/testdata/lyrics/lrc-with-ends.lrc @@ -0,0 +1,11 @@ +[ar:Testiryhmä] +[ti:Päivä [Live]] +[al:Keksitty levy] +[by:Tonyn testit] +[offset:0] + +[00:01.00]<00:01.00>Päivä <00:01.50>paistaa<00:02.20> +[00:03.00]<00:03.00>Pöllö <00:03.40>huhuilee +[00:04.80] +[00:06.00]Koko rivi yhdellä leimalla +[00:08.50] diff --git a/testdata/lyrics/moises-exporter-lines.lrc b/testdata/lyrics/moises-exporter-lines.lrc new file mode 100644 index 00000000..b97444ab --- /dev/null +++ b/testdata/lyrics/moises-exporter-lines.lrc @@ -0,0 +1,10 @@ +[ti:Kesäyön testilaulu] +[re:Moises Lyrics Exporter Pro] +[ve:2.0] +[co:avg_confidence=1.000; low_confidence_count=0] + +[00:00.00] Alku ennen nollaa +[00:04.64] Tämä on keksittylaulu, +[00:06.00] ♪ +[00:21.50] Yö on hämärä +[00:23.40] Nyt seloppuu! diff --git a/testdata/lyrics/moises-exporter-words.lrc b/testdata/lyrics/moises-exporter-words.lrc new file mode 100644 index 00000000..6a934a84 --- /dev/null +++ b/testdata/lyrics/moises-exporter-words.lrc @@ -0,0 +1,10 @@ +[ti:Kesäyön testilaulu] +[re:Moises Lyrics Exporter Pro] +[ve:2.0] +[co:avg_confidence=1.000; low_confidence_count=0] + +[00:00.000] <00:00.000>Alku <00:00.000>ennen <00:00.310>nollaa +[00:04.640] <00:04.640>Tämä <00:04.920>on <00:05.160>keksitty<00:05.620>laulu, +[00:06.000] ♪ +[00:21.500] <00:21.500>Yö <00:21.800>on <00:22.050>hämärä +[00:23.400] <00:23.400>Nyt <00:23.700>se<00:24.000>loppuu! From bdb0489f38246b0a6e1bd905b5bea1da98e0506c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 20:00:59 +0000 Subject: [PATCH 064/275] test: the lyrics plot style of the svgui fork svgui gains RegionLayer::PlotLyrics (62af60f, on the fork's branch claude/nifty-goodall-4kocmz): each region's label along the top of the view at its start, over a bar as long as the region, in up to two rows, for the words of a song. The pin moves to it. TestLyricsLayer, a new app suite, tests where labels go (assignLabelRows), that nothing is drawn below the band, and that the layer painted strip by strip is the same as the layer painted whole: a view that scrolls repaints only what comes into sight, and a layout that depended on the strip broke the text at the seams. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/test/TestLyricsLayer.h | 234 ++++++++++++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 7 ++ meson.build | 1 + repoint-lock.json | 2 +- 4 files changed, 243 insertions(+), 1 deletion(-) create mode 100644 main/test/TestLyricsLayer.h diff --git a/main/test/TestLyricsLayer.h b/main/test/TestLyricsLayer.h new file mode 100644 index 00000000..372231a3 --- /dev/null +++ b/main/test/TestLyricsLayer.h @@ -0,0 +1,234 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LYRICS_LAYER_H +#define TEST_LYRICS_LAYER_H + +// Tier 3: the lyrics plot style of RegionLayer (svgui fork), which +// draws the words of the lyrics along the top of pane 0. Where each +// label goes is worked out by a pure function, tested first; then the +// layer is painted into images, with no MainWindow. + +#include "framework/Document.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/ViewManager.h" +#include "layer/RegionLayer.h" +#include "layer/LayerFactory.h" +#include "data/model/RegionModel.h" +#include "data/model/WritableWaveFileModel.h" + +#include +#include +#include +#include +#include + +#include +#include + +class TestLyricsLayer : public QObject +{ + Q_OBJECT + + typedef std::vector> Spans; + + QTemporaryDir m_dir; + + sv::ViewManager *m_viewManager = nullptr; + sv::PaneStack *m_paneStack = nullptr; + sv::Document *m_document = nullptr; + sv::Pane *m_pane = nullptr; + sv::RegionLayer *m_layer = nullptr; + + static constexpr double kRate = 44100.0; + static constexpr int kWidth = 800; + static constexpr int kHeight = 200; + + sv::ModelId makeAudioModel() { + auto model = std::make_shared + (m_dir.filePath("audio.wav"), kRate, 1, + sv::WritableWaveFileModel::Normalisation::None); + std::vector data(size_t(kRate) * 10, 0.25f); + const float *ptr = data.data(); + model->addSamples(&ptr, sv::sv_frame_t(data.size())); + model->writeComplete(); + return sv::ModelById::add(model); + } + + // Words a quarter of a second apart, each longer than that at the + // zoom the tests use (100 pixels a second), so that they do not + // all fit in one row; a new line every four words + void addWords(int count) { + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + for (int i = 0; i < count; ++i) { + sv::sv_frame_t frame = sv::sv_frame_t(kRate * 0.25 * i); + model->add(sv::Event(frame, float(i / 4), sv::sv_frame_t(kRate * 0.2), + QString("sanaseppo%1").arg(i))); + } + } + + // The layer painted into a white image, clip rect by clip rect, as + // the view paints what has scrolled into sight + QImage render(const std::vector &rects) { + QImage image(kWidth, kHeight, QImage::Format_ARGB32); + image.fill(Qt::white); + QPainter painter(&image); + for (const QRect &rect : rects) { + painter.save(); + painter.setClipRect(rect); + m_layer->paint(m_pane, painter, rect); + painter.restore(); + } + painter.end(); + return image; + } + + static bool rowIsWhite(const QImage &image, int y) { + for (int x = 0; x < image.width(); ++x) { + if (image.pixel(x, y) != qRgb(255, 255, 255)) return false; + } + return true; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void init() { + m_viewManager = new sv::ViewManager; + m_paneStack = new sv::PaneStack(nullptr, m_viewManager); + m_document = new sv::Document; + m_document->setMainModel(makeAudioModel()); + m_pane = m_paneStack->addPane(); + m_pane->resize(kWidth, kHeight); + m_pane->setZoomLevel(sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, + int(kRate / 100))); + m_pane->setStartFrame(0); + + auto model = std::make_shared(kRate, 1); + sv::ModelId id = sv::ModelById::add(model); + m_document->addNonDerivedModel(id); + m_layer = qobject_cast + (m_document->createLayer(sv::LayerFactory::Regions)); + QVERIFY(m_layer); + m_document->setModel(m_layer, id); + m_layer->setPlotStyle(sv::RegionLayer::PlotLyrics); + m_document->attachLayerToView(m_pane, m_layer); + } + + void cleanup() { + // The document force-deletes its layers from their views, so + // it has to go while the panes still exist + delete m_document; + delete m_paneStack; + delete m_viewManager; + m_document = nullptr; + m_paneStack = nullptr; + m_viewManager = nullptr; + m_pane = nullptr; + m_layer = nullptr; + } + + // --- Where labels go ------------------------------------------------- + + void labels_that_fit_share_the_first_row() { + Spans spans { { 0, 10 }, { 20, 10 }, { 40, 10 } }; + QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), + std::vector({ 0, 0, 0 })); + } + + void a_label_that_overlaps_goes_to_the_next_row() { + // The second runs into the first; the third clears the first + Spans spans { { 0, 30 }, { 10, 30 }, { 45, 10 } }; + QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), + std::vector({ 0, 1, 0 })); + } + + void a_label_with_no_room_in_any_row_is_left_out() { + Spans spans { { 0, 50 }, { 10, 50 }, { 20, 50 }, { 70, 10 } }; + QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), + std::vector({ 0, 1, -1, 0 })); + } + + void the_gap_is_the_least_space_between_labels() { + QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 }, { 14, 10 } }, 2, 4), + std::vector({ 0, 0 })); + QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 }, { 13.5, 10 } }, 2, 4), + std::vector({ 0, 1 })); + } + + void no_labels_and_no_rows() { + QCOMPARE(sv::RegionLayer::assignLabelRows({}, 2, 4), std::vector()); + QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 } }, 0, 4), + std::vector({ -1 })); + } + + // --- The layer -------------------------------------------------------- + + void lyrics_take_no_vertical_scale_and_no_edits() { + // Like the coverage strip: nothing in the pane may align its + // scale to the lyrics, and no tool may edit them + QVERIFY(!m_layer->isLayerEditable()); + QVERIFY(m_layer->getVerticalExtents().first == + sv::Layer::NO_VERTICAL_EXTENTS.first); + addWords(4); + QPoint pos(10, 10); + QCOMPARE(m_layer->getFeatureDescription(m_pane, pos), QString()); + } + + void lyrics_are_drawn_only_along_the_top() { + addWords(20); + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + + bool drawn = false; + for (int y = 0; y < kHeight / 4; ++y) { + if (!rowIsWhite(image, y)) drawn = true; + } + QVERIFY2(drawn, "nothing was drawn along the top of the pane"); + + for (int y = kHeight / 2; y < kHeight; ++y) { + QVERIFY2(rowIsWhite(image, y), + qPrintable(QString("something was drawn at y = %1, " + "below the band of the lyrics").arg(y))); + } + } + + void painting_in_strips_matches_painting_whole() { + // A view that scrolls repaints only the strip that comes into + // sight; the labels in it must be where they were when the + // whole view was painted, or the text breaks at the seams. + // With words this close some go to the second row and some are + // left out, which is what depends on the neighbours + addWords(24); + QImage whole = render({ QRect(0, 0, kWidth, kHeight) }); + QImage strips = render({ QRect(0, 0, 230, kHeight), + QRect(230, 0, 310, kHeight), + QRect(540, 0, kWidth - 540, kHeight) }); + QVERIFY2(whole == strips, + "the layer painted in strips differs from the layer painted whole"); + + // And again after the view has scrolled on by part of a label + m_pane->setStartFrame(sv::sv_frame_t(kRate * 0.37)); + whole = render({ QRect(0, 0, kWidth, kHeight) }); + strips = render({ QRect(0, 0, 400, kHeight), + QRect(400, 0, kWidth - 400, kHeight) }); + QVERIFY2(whole == strips, + "after scrolling, the layer painted in strips differs from " + "the layer painted whole"); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 6ad3eaf0..df354e9b 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -14,6 +14,7 @@ #include "TestSingingDocument.h" #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" +#include "TestLyricsLayer.h" #include "RunSuite.h" @@ -64,6 +65,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLyricsLayer t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestRecordWorkflow t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 597bb75e..52a100c9 100644 --- a/meson.build +++ b/meson.build @@ -1395,6 +1395,7 @@ tony_app_test_moc_files = qt.preprocess( 'main/test/TestSingingDocument.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', + 'main/test/TestLyricsLayer.h', ]) tony_app_test_exe = executable( diff --git a/repoint-lock.json b/repoint-lock.json index e495809b..d8911c46 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "bdac54f488f4d0346eaf692ec4a46d436db0ca87" + "pin": "62af60f4e746b87b9a4896a0569603f8cb56694c" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 61ffb7d1fd4556ae192b33cfca28940690a213e7 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 20:35:06 +0000 Subject: [PATCH 065/275] feat: import timed lyrics onto the reference File > Import Lyrics... reads an LRC file and shows its words along the top of pane 0, on the reference's timeline, in the lyrics plot style of the svgui fork. LyricsTrack owns the one layer, as the coverage strip's class owns its own: one set of lyrics per session, a new import in place of the last, File > Remove Lyrics to take them away and View > Show Lyrics to hide them without losing them. Like loading background music, an import is not undoable and pushes no command, but marks the session modified. The layer goes in under the pane's top layer, whose hover readout and scale the pane uses, and its model is taken out of the play source, so that words after the end of the reference do not hold playback open. A file that cannot be read or holds no timed lyrics is refused with a dialog and changes nothing; what was skipped goes to the status bar with the counts. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsTrack.cpp | 198 ++++++++++++++++ main/LyricsTrack.h | 104 +++++++++ main/MainWindow.cpp | 205 ++++++++++++++++ main/MainWindow.h | 29 +++ main/test/TestRecordWorkflow.h | 411 +++++++++++++++++++++++++++++++++ meson.build | 3 + 6 files changed, 950 insertions(+) create mode 100644 main/LyricsTrack.cpp create mode 100644 main/LyricsTrack.h diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp new file mode 100644 index 00000000..adb39051 --- /dev/null +++ b/main/LyricsTrack.cpp @@ -0,0 +1,198 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LyricsTrack.h" + +#include "TakeLayers.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "layer/RegionLayer.h" +#include "layer/LayerFactory.h" +#include "layer/ColourDatabase.h" +#include "data/model/RegionModel.h" +#include "base/PlayParameters.h" + +#include + +using namespace sv; + +using std::cerr; +using std::endl; + +// The words are in frames of the reference's timeline, as the parser +// worked them out: a coarser resolution would move them +static const int lyricsResolution = 1; + +// Used only if the session has no main model to take the rate from, +// which cannot happen while there is a reference to time the words by +static const sv_samplerate_t defaultSampleRate = 44100; + +LyricsTrack::LyricsTrack(QObject *parent) : + QObject(parent), + m_document(nullptr), + m_pane(nullptr), + m_layer(nullptr) +{ +} + +LyricsTrack::~LyricsTrack() +{ +} + +QString +LyricsTrack::layerName() +{ + return "Lyrics"; +} + +bool +LyricsTrack::show(Document *document, Pane *pane, const EventVector &events, + QString presentationName) +{ + if (!document || !pane) return false; + + // One set of lyrics per session: a new import takes the place of the + // last, whose layer and model go with it + if (m_layer) hide(); + + sv_samplerate_t rate = defaultSampleRate; + if (auto main = ModelById::get(document->getMainModel())) { + rate = main->getSampleRate(); + } + + auto model = std::make_shared(rate, lyricsResolution); + model->setObjectName(tr("Lyrics")); + for (const Event &e : events) model->add(e); + ModelId modelId = ModelById::add(model); + + // Not createEmptyLayer(): see MainWindow::setupRealtimePitchLayer(). + // Before the model goes to the document, so that a failure here can + // release it: the document must never hold a released model + auto layer = qobject_cast + (document->createLayer(LayerFactory::Regions)); + if (!layer) { + cerr << "LyricsTrack::show: failed to create layer" << endl; + ModelById::release(modelId); + return false; + } + + document->addNonDerivedModel(modelId); + document->setModel(layer, modelId); + takeLayer(document, pane, layer); + m_layer->setPresentationName(presentationName); + + // The pane takes its hover readout and its vertical scale from its + // top layer, and this one has neither to give: whatever is on top + // now stays there + Layer *previousTop = pane->getTopLayer(); + + // Not addLayerToView(): an import is not undoable, and hide() takes + // the layer away with deleteLayer(force), which would leave an + // AddLayerCommand holding a deleted layer + document->attachLayerToView(pane, layer); + + if (previousTop) TakeLayers::raise(pane, previousTop); + + return true; +} + +void +LyricsTrack::takeLayer(Document *document, Pane *pane, RegionLayer *layer) +{ + m_document = document; + m_pane = pane; + m_layer = layer; + + connect(m_document, &Document::layerAboutToBeDeleted, + this, &LyricsTrack::layerAboutToBeDeleted, + Qt::UniqueConnection); + + configureLayer(); +} + +void +LyricsTrack::configureLayer() +{ + if (!m_layer) return; + + m_layer->setObjectName(layerName()); + + // Words along the top of the pane: a plot style the svgui fork has + // for this. It has no vertical scale and takes no edits. + // EqualSpaced as well, so that nothing in the pane can align its + // scale to this layer + m_layer->setVerticalScale(RegionLayer::EqualSpaced); + m_layer->setPlotStyle(RegionLayer::PlotLyrics); + + // The bar under each word; the words themselves are in the view's + // own colours. The layer's default would be black + m_layer->setBaseColour + (ColourDatabase::getInstance()->getColourIndex(tr("Grey"))); + + // A RegionModel cannot play, so there are none; if that ever + // changes, the words are still not something to hear + if (auto params = m_layer->getPlayParameters()) { + params->setPlayAudible(false); + } +} + +void +LyricsTrack::hide() +{ + RegionLayer *layer = m_layer; + m_layer = nullptr; + + // deleteLayer(force) and nothing else: see the notes on tearing a + // layer down silently in MainWindow::teardownRealtimePitchLayer(). + // The model goes with the layer, which is its only user + if (layer && m_document) { + m_document->deleteLayer(layer, true); + } + + if (m_document) { + disconnect(m_document, nullptr, this, nullptr); + } + m_document = nullptr; + m_pane = nullptr; +} + +void +LyricsTrack::setVisible(bool visible) +{ + if (!m_layer || !m_pane) return; + m_layer->showLayer(m_pane, visible); +} + +bool +LyricsTrack::isVisible() const +{ + return m_layer && m_pane && !m_layer->isLayerDormant(m_pane); +} + +void +LyricsTrack::layerAboutToBeDeleted(Layer *layer) +{ + // Someone else's doing, the document being closed for instance + if (layer && layer == m_layer) { + m_layer = nullptr; + hide(); + } +} + +ModelId +LyricsTrack::getModelId() const +{ + return m_layer ? m_layer->getModel() : ModelId(); +} diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h new file mode 100644 index 00000000..50c9beaa --- /dev/null +++ b/main/LyricsTrack.h @@ -0,0 +1,104 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_TRACK_H +#define TONY_LYRICS_TRACK_H + +#include "base/ById.h" +#include "base/Event.h" +#include "data/model/Model.h" + +#include +#include + +namespace sv { +class Document; +class Pane; +class Layer; +class RegionLayer; +} + +/** + * The timed lyrics of the session: a RegionLayer in pane 0 with one + * region per word (lyricsToEvents()), drawn as words along the top of + * the pane by the lyrics plot style of the svgui fork. + * + * The lyrics belong to the song, not to a take: one set per session, + * which only an import replaces. The layer and its model are ordinary + * document contents, so a session keeps them, and the layer's object + * name is how it is known again. + * + * The layer is display only. It is never the pane's top layer, because + * the pane takes the hover readout and the vertical scale from that one + * and this style has neither; it cannot be played (a RegionModel has no + * play parameters), and it takes no edits. + * + * It looks like the coverage strip's class and is used the same way: + * MainWindow only wires it. + */ +class LyricsTrack : public QObject +{ + Q_OBJECT + +public: + LyricsTrack(QObject *parent = nullptr); + virtual ~LyricsTrack(); + + /** + * Make a layer holding these events in the given pane, under the + * layer that is on top of the pane now. Lyrics already shown are + * replaced: their layer and model go first. The presentation name + * is what the user sees the layer called. Returns false if the + * layer could not be created. + */ + bool show(sv::Document *document, sv::Pane *pane, + const sv::EventVector &events, QString presentationName); + + /// Delete the layer and its model from the document + void hide(); + + bool isShown() const { return m_layer != nullptr; } + + /** + * The user's Show Lyrics toggle: the layer stays, hidden. Not kept + * in the settings: the pane writes each layer's visibility into the + * session, which is where it belongs. + */ + void setVisible(bool visible); + bool isVisible() const; + + sv::RegionLayer *getLayer() const { return m_layer; } + + /// The model the words are in + sv::ModelId getModelId() const; + + /** + * The layer's object name, which is how it is known after a session + * load. Not translated: it is stored in the session file. + */ + static QString layerName(); + +private slots: + void layerAboutToBeDeleted(sv::Layer *); + +private: + sv::Document *m_document; + sv::Pane *m_pane; + sv::RegionLayer *m_layer; + + void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *); + void configureLayer(); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index b1e52653..a321a5aa 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -19,6 +19,7 @@ #include "NetworkPermissionTester.h" #include "Analyser.h" #include "LatencyUtils.h" +#include "Lyrics.h" #include "PaneUtils.h" #include "TakeEvents.h" #include "TakeLayers.h" @@ -92,6 +93,7 @@ #include #include #include +#include #include #include #include @@ -149,6 +151,10 @@ MainWindow::MainWindow(AudioMode audioMode, m_singingNotesHiddenForTake(false), m_takes(nullptr), m_coverageStrip(nullptr), + m_lyrics(nullptr), + m_importLyricsAction(nullptr), + m_removeLyricsAction(nullptr), + m_showLyrics(nullptr), m_takesMenu(nullptr), m_takeCombo(nullptr), m_newTakeAction(nullptr), @@ -393,6 +399,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_takes = new SingingTakes(this); m_coverageStrip = new CoverageStrip(this); + m_lyrics = new LyricsTrack(this); // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end @@ -487,6 +494,8 @@ MainWindow::~MainWindow() m_alternatePitch = nullptr; delete m_coverageStrip; m_coverageStrip = nullptr; + delete m_lyrics; + m_lyrics = nullptr; delete m_analyser; delete m_keyReference; Profiles::getInstance()->dump(); @@ -605,6 +614,19 @@ MainWindow::setupFileMenu() connect(this, SIGNAL(canPlay(bool)), m_loadBackgroundMusicAction, SLOT(setEnabled(bool))); menu->addAction(m_loadBackgroundMusicAction); + // Enabled in updateMenuStates() + m_importLyricsAction = new QAction(il.load("fileopen"), tr("Import &Lyrics..."), this); + m_importLyricsAction->setStatusTip(tr("Import timed lyrics from an LRC file, to be shown along the top of the pane")); + m_importLyricsAction->setEnabled(false); + connect(m_importLyricsAction, &QAction::triggered, this, &MainWindow::importLyrics); + menu->addAction(m_importLyricsAction); + + m_removeLyricsAction = new QAction(tr("Remove Lyrics"), this); + m_removeLyricsAction->setStatusTip(tr("Take the imported lyrics out of the session")); + m_removeLyricsAction->setEnabled(false); + connect(m_removeLyricsAction, &QAction::triggered, this, &MainWindow::removeLyrics); + menu->addAction(m_removeLyricsAction); + menu->addSeparator(); action = new QAction(tr("I&mport Pitch Track Data..."), this); @@ -960,6 +982,17 @@ MainWindow::setupViewMenu() action->setStatusTip(tr("Set the minimum and maximum frequencies in the visible display")); connect(action, SIGNAL(triggered()), this, SLOT(editDisplayExtents())); menu->addAction(action); + + menu->addSeparator(); + + // Enabled and checked in updateLayerStatuses(). Not "Show &Lyrics": + // Peek Left has the L + m_showLyrics = new QAction(tr("Show L&yrics"), this); + m_showLyrics->setCheckable(true); + m_showLyrics->setStatusTip(tr("Show or hide the imported lyrics along the top of the pane")); + m_showLyrics->setEnabled(false); + connect(m_showLyrics, &QAction::triggered, this, &MainWindow::showLyricsToggled); + menu->addAction(m_showLyrics); } void @@ -2206,6 +2239,13 @@ MainWindow::updateMenuStates() emit canChangeTakes(canChange); emit canActOnTake(canChange && m_takes->getActiveIndex() >= 0); + if (m_importLyricsAction) { + m_importLyricsAction->setEnabled(lyricsImportAllowed()); + } + if (m_removeLyricsAction) { + m_removeLyricsAction->setEnabled(m_lyrics && m_lyrics->isShown()); + } + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -2420,6 +2460,12 @@ MainWindow::updateLayerStatuses() (shown && !inTake && m_alternatePitch->canStep(false)); } + // Lyrics: shown or hidden once there are some + if (m_showLyrics && m_lyrics) { + m_showLyrics->setEnabled(m_lyrics->isShown()); + m_showLyrics->setChecked(m_lyrics->isVisible()); + } + // Background music toggle: enabled when a background music track is loaded if (m_playBackgroundMusic) { bool haveBgMusic = (m_backgroundMusicLayer != nullptr); @@ -2544,6 +2590,7 @@ MainWindow::closeSession() teardownBackgroundMusic(); m_alternatePitch->hide(); m_coverageStrip->hide(); + m_lyrics->hide(); m_referencePitchHiddenForTake = false; m_singingPitchHiddenForTake = false; m_singingNotesHiddenForTake = false; @@ -3300,6 +3347,164 @@ MainWindow::syncCoverageStrip() } } +bool +MainWindow::lyricsImportAllowed() const +{ + // The words are put on the reference's timeline, in pane 0. Not + // while a take is being recorded: the singer is reading the words + // that are there + if (!m_document || !getMainModel()) return false; + if (!m_paneStack || m_paneStack->getPaneCount() < 1) return false; + if (m_recordTarget && m_recordTarget->isRecording()) return false; + return true; +} + +QString +MainWindow::askForLyricsFile() +{ + // Lyrics are for the reference, and likely to be kept next to it + QString dir; + if (auto reference = getMainModel()) { + QFileInfo info(reference->getLocation()); + if (info.exists()) dir = info.absolutePath(); + } + + return QFileDialog::getOpenFileName + (this, tr("Import Lyrics"), dir, + tr("LRC lyrics (*.lrc)") + ";;" + tr("Text files (*.txt)") + ";;" + + tr("All files (*)")); +} + +void +MainWindow::importLyrics() +{ + if (!lyricsImportAllowed()) return; + QString path = askForLyricsFile(); + if (path.isEmpty()) return; + importLyricsFrom(path); +} + +bool +MainWindow::importLyricsFrom(QString path) +{ + if (!lyricsImportAllowed()) return false; + + Pane *pane = m_paneStack->getPane(0); + auto reference = getMainModel(); + + emit activity(tr("Import lyrics \"%1\"").arg(path)); + + // An LRC file is a few kB. One chosen by mistake, a recording say, is + // not read at all; and the read stops just past the limit, which + // parseLrc() enforces too, in case the file grows in the meantime + QString error; + QByteArray bytes; + QFileInfo info(path); + if (!info.isFile()) { + error = tr("File \"%1\" could not be found.").arg(path); + } else if (info.size() > Lyrics::maxFileBytes) { + error = tr("The file is over 1 MB, too big to be an LRC file."); + } else { + QFile file(path); + if (!file.open(QIODevice::ReadOnly)) { + error = tr("File \"%1\" could not be opened: %2") + .arg(path).arg(file.errorString()); + } else { + bytes = file.read(Lyrics::maxFileBytes + 1); + } + } + + LyricsParseResult parsed; + if (error == "") { + parsed = parseLrc(bytes); + error = parsed.error; + } + + // Nothing has changed yet, and nothing does: the lyrics there are + // stay + if (error != "") { + QMessageBox::warning(this, tr("Could not import lyrics"), error); + return false; + } + + const Lyrics &lyrics = parsed.lyrics; + EventVector events = lyricsToEvents(lyrics, reference->getSampleRate()); + + // Words after the end of the reference are shown where there is + // nothing to hear: the lyrics may be of another recording of the song + sv_frame_t end = reference->getEndFrame(); + int pastEnd = int(std::count_if(events.begin(), events.end(), + [end](const Event &e) { + return e.getFrame() >= end; + })); + + QString name = (lyrics.title != "" ? lyrics.title : tr("Lyrics")); + if (!m_lyrics->show(m_document, pane, events, name)) { + // Only if the layer could not be made + updateMenuStates(); + updateLayerStatuses(); + return false; + } + + // The play source takes in the model of every layer that is in a + // view, whether the model can be played or not, and the models it + // holds are what say where playback ends. Words past the end of the + // reference would hold playback open, with nothing to hear + if (m_playSource && !m_lyrics->getModelId().isNone()) { + m_playSource->removeModel(m_lyrics->getModelId()); + } + + // The layer arrived without a command, as it must, and an import is + // not undoable; but the session has changed + documentModified(); + updateMenuStates(); + updateLayerStatuses(); + + // Kept as the status message, so that the pane's context help, when + // it has nothing to say, gives this back rather than clearing it + int wordCount = int(lyrics.words.size()); + int lineCount = lyrics.lineCount(); + QStringList messages; + messages << tr("Imported %1 in %2.") + .arg(wordCount == 1 ? tr("1 word") : tr("%1 words").arg(wordCount), + lineCount == 1 ? tr("1 line") : tr("%1 lines").arg(lineCount)); + messages << parsed.warnings; + if (pastEnd == 1) { + messages << tr("1 word starts after the end of the reference."); + } else if (pastEnd > 1) { + messages << tr("%1 words start after the end of the reference.") + .arg(pastEnd); + } + m_myStatusMessage = messages.join(" "); + getStatusLabel()->setText(m_myStatusMessage); + + return true; +} + +void +MainWindow::removeLyrics() +{ + if (!m_lyrics->isShown()) return; + m_lyrics->hide(); + // As for the import: no command, but the session has changed + documentModified(); + updateMenuStates(); + updateLayerStatuses(); +} + +void +MainWindow::showLyricsToggled() +{ + // Straight on the layer, with no command and nothing in the settings: + // the pane writes the layer's visibility into the session, which is + // where it belongs + if (m_lyrics->isShown()) { + m_lyrics->setVisible(!m_lyrics->isVisible()); + documentModified(); + } + updateLayerStatuses(); +} + void MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { diff --git a/main/MainWindow.h b/main/MainWindow.h index d27a635d..a5ad5368 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -21,6 +21,7 @@ #include "RealtimePitchTracker.h" #include "AlternatePitchTrack.h" #include "CoverageStrip.h" +#include "LyricsTrack.h" #include "SingingTakes.h" #include "TakeCommands.h" #include "TakeTiming.h" @@ -179,6 +180,10 @@ protected slots: virtual void alternatePitchDown(); virtual void syncAlternatePitchTrack(); + virtual void importLyrics(); + virtual void removeLyrics(); + virtual void showLyricsToggled(); + virtual void editDisplayExtents(); virtual void analyseNow(); @@ -359,6 +364,30 @@ protected slots: // is a take and none yet, take it away when the take goes void syncCoverageStrip(); + // The timed lyrics of the session, drawn along the top of pane 0 and + // stored in the session with the layer that draws them. They belong + // to the song, not to a take. Display only + LyricsTrack *m_lyrics; + QAction *m_importLyricsAction; + QAction *m_removeLyricsAction; + QAction *m_showLyrics; + + // Put the lyrics of this LRC file on the reference's timeline, in + // place of any there are. Not undoable, as loading background music + // is not, and nothing goes onto the undo stack. False if the file + // could not be read or holds no timed lyrics, which the user is told + // in a dialog, or if lyricsImportAllowed() says no; nothing has + // changed then + bool importLyricsFrom(QString path); + + // Lyrics can be imported once there is a reference, and not while a + // take is being recorded + bool lyricsImportAllowed() const; + + // Ask for the LRC file to import, "" if the user cancelled. + // Overridden by the tests, which cannot answer a dialog + virtual QString askForLyricsFile(); + // --- The audio folder of the session (spec 6.4) --- // Where the next combined audio file of a take is to be written: the diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d1614237..c298e7c8 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -28,6 +28,8 @@ #include "../MainWindow.h" #include "../Analyser.h" #include "../CoverageStrip.h" +#include "../Lyrics.h" +#include "../LyricsTrack.h" #include "../SingingTakes.h" #include "../TakeLayers.h" #include "../TakesFile.h" @@ -68,6 +70,7 @@ #include #include #include +#include #include #include #include @@ -216,6 +219,17 @@ class TestMainWindow : public MainWindow QAction *alternatePitchUpAction() { return m_alternatePitchUpAction; } QAction *alternatePitchDownAction() { return m_alternatePitchDownAction; } + // The timed lyrics, and the three menu actions that act on them + LyricsTrack *lyrics() { return m_lyrics; } + bool doImportLyricsFrom(QString path) { return importLyricsFrom(path); } + QAction *importLyricsAction() { return m_importLyricsAction; } + QAction *removeLyricsAction() { return m_removeLyricsAction; } + QAction *showLyricsAction() { return m_showLyrics; } + + // The file Import Lyrics asks for, answered from here: "" is Cancel + void setLyricsFileAnswer(QString path) { m_lyricsFileAnswer = path; } + int lyricsFileQuestions() const { return m_lyricsFileQuestions; } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { onRealtimePitchDetected(frame, hz); } @@ -249,6 +263,11 @@ class TestMainWindow : public MainWindow return m_takeNameAnswer == "" ? current : m_takeNameAnswer; } + QString askForLyricsFile() override { + ++m_lyricsFileQuestions; + return m_lyricsFileAnswer; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -260,6 +279,8 @@ class TestMainWindow : public MainWindow bool m_deleteTakeAnswer = true; int m_deleteTakeQuestions = 0; QString m_takeNameAnswer; + QString m_lyricsFileAnswer; + int m_lyricsFileQuestions = 0; }; class TestRecordWorkflow : public QObject @@ -859,6 +880,63 @@ class TestRecordWorkflow : public QObject return true; } + // The timed lyrics. The layers are counted by the name a session + // knows the lyrics by, spelled out here: it is part of the file format + + int lyricsLayersInDocument() { + int n = 0; + for (sv::Layer *layer : m_window->document()->getLayers()) { + if (layer->objectName() == "Lyrics") ++n; + } + return n; + } + + int lyricsLayersInPane0() { + int n = 0; + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return 0; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i)->objectName() == "Lyrics") ++n; + } + return n; + } + + sv::EventVector lyricsEvents() { + sv::RegionLayer *layer = m_window->lyrics()->getLayer(); + if (!layer) return {}; + auto model = sv::ModelById::getAs(layer->getModel()); + return model ? model->getAllEvents() : sv::EventVector(); + } + + static QString lyricsFixture(const char *name) { + return QString(TONY_TEST_DATA_DIR) + "/lyrics/" + name; + } + + static Lyrics lyricsIn(QString path) { + QFile file(path); + if (!file.open(QIODevice::ReadOnly)) return {}; + return parseLrc(file.readAll()).lyrics; + } + + // What an import of this file has to put in the model: the words on + // the reference's timeline + sv::EventVector expectedLyricsEvents(QString path) { + auto reference = sv::ModelById::get(m_window->mainModelId()); + if (!reference) return {}; + return lyricsToEvents(lyricsIn(path), reference->getSampleRate()); + } + + QString writeLrc(const QByteArray &text) { + QString path = m_dir.filePath + (QString("lyrics-%1.lrc").arg(++m_fileCounter)); + QFile file(path); + if (!file.open(QIODevice::WriteOnly) || + file.write(text) != text.size()) { + return {}; + } + return path; + } + // The pre-roll's length has no UI: it is read from the settings when // Record is pressed. These are the test suite's own settings // (tony-app-test), not the user's @@ -5428,6 +5506,339 @@ private slots: != alt->getLayer()); } + // Timed lyrics: an LRC file imported onto the reference's timeline, + // drawn along the top of pane 0 by LyricsTrack's layer + + void lyrics_import_shows_words() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + QString path = lyricsFixture("moises-exporter-words.lrc"); + QVERIFY(m_window->doImportLyricsFrom(path)); + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(lyrics->isShown()); + QVERIFY(lyrics->isVisible()); + QCOMPARE(lyricsLayersInDocument(), 1); + QCOMPARE(lyricsLayersInPane0(), 1); + sv::RegionLayer *layer = lyrics->getLayer(); + QVERIFY(paneHasLayer(0, layer)); + QCOMPARE(int(layer->getPlotStyle()), int(sv::RegionLayer::PlotLyrics)); + QCOMPARE(int(layer->getVerticalScale()), + int(sv::RegionLayer::EqualSpaced)); + QCOMPARE(colourOf(layer), colourNamed("Grey")); + QVERIFY(!layer->isLayerEditable()); + QCOMPARE(layer->getLayerPresentationName(), + QString("Kesäyön testilaulu")); + + // Every word, at its frame, with its label + sv::EventVector expected = expectedLyricsEvents(path); + QCOMPARE(int(expected.size()), 13); + QCOMPARE(lyricsEvents(), expected); + + Lyrics parsed = lyricsIn(path); + QString counts = QString("Imported %1 words in %2 lines.") + .arg(parsed.words.size()).arg(parsed.lineCount()); + QVERIFY2(m_window->statusText().startsWith(counts), + qPrintable(m_window->statusText())); + + QVERIFY(m_window->removeLyricsAction()->isEnabled()); + QVERIFY(m_window->showLyricsAction()->isEnabled()); + QVERIFY(m_window->showLyricsAction()->isChecked()); + } + + // Like loading background music: not undoable, and nothing goes onto + // the undo stack or comes off it, but the session has changed + void lyrics_import_leaves_history_alone() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + auto *history = sv::CommandHistory::getInstance(); + history->addCommand(new sv::GenericCommand + ("Earlier Edit", []() {}, []() {}), false); + m_window->discardModifications(); + QSignalSpy commands(history, qOverload<> + (&sv::CommandHistory::commandExecuted)); + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + QCOMPARE(int(commands.count()), 0); + QVERIFY(m_window->isDocumentModified()); + + sv::Layer *layer = m_window->lyrics()->getLayer(); + QCOMPARE(undoOnce(), QString("Earlier Edit")); + QCOMPARE(undoOnce(), QString()); + QVERIFY(m_window->lyrics()->isShown()); + QVERIFY(m_window->lyrics()->getLayer() == layer); + QVERIFY(paneHasLayer(0, layer)); + } + + // The play source takes in the model of every layer in a view, and + // the last end of them is where playback ends + void lyrics_do_not_extend_playback() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + sv::AudioCallbackPlaySource *playSource = m_window->playSource(); + sv::sv_frame_t playEnd = playSource->getPlayEndFrame(); + QVERIFY(playEnd > 0); + + // Two words within the second of reference, and one well after it + QVERIFY(m_window->doImportLyricsFrom + (writeLrc("[00:00.20]<00:00.20>Yksi <00:00.60>kaksi\n" + "[00:03.00]<00:03.00>Kolme\n"))); + sv::ModelId model = m_window->lyrics()->getModelId(); + QVERIFY(!model.isNone()); + QVERIFY2(sv::ModelById::get(model)->getEndFrame() > playEnd, + "the lyrics end within the reference: this shows nothing"); + + QVERIFY2(playSource->getModels().count(model) == 0, + "the play source holds the lyrics"); + // Not equal: taking a model out works the end out again from the + // models' ends as they are now, and one of pYIN's may be shorter + // than it was when it came in + QVERIFY2(playSource->getPlayEndFrame() <= playEnd, + qPrintable(QString("playback ends at frame %1, later than " + "%2 before the import") + .arg(playSource->getPlayEndFrame()).arg(playEnd))); + verifyPlaySourceClean(); + + QVERIFY2(m_window->statusText().contains + ("1 word starts after the end of the reference."), + qPrintable(m_window->statusText())); + } + + void lyrics_reimport_replaces() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::ModelId first = lyrics->getModelId(); + + QString path = lyricsFixture("lrc-with-ends.lrc"); + QVERIFY(m_window->doImportLyricsFrom(path)); + QCOMPARE(lyricsLayersInDocument(), 1); + QCOMPARE(lyricsLayersInPane0(), 1); + QVERIFY(lyrics->getModelId() != first); + QCOMPARE(lyricsEvents(), expectedLyricsEvents(path)); + QCOMPARE(lyrics->getLayer()->getLayerPresentationName(), + QString("Päivä [Live]")); + + // A model id is never used twice, unlike the address of a layer + QVERIFY2(!sv::ModelById::get(first), + "the first lyrics' model was not released"); + QVERIFY(m_window->document()->getModels().count(first) == 0); + verifyPlaySourceClean(); + } + + void lyrics_remove() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::ModelId model = lyrics->getModelId(); + m_window->discardModifications(); + + m_window->removeLyricsAction()->trigger(); + QVERIFY(!lyrics->isShown()); + QCOMPARE(lyricsLayersInDocument(), 0); + QVERIFY2(!sv::ModelById::get(model), "the lyrics model was not released"); + QVERIFY(m_window->document()->getModels().count(model) == 0); + QVERIFY(m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString()); + verifyPlaySourceClean(); + + QVERIFY(!m_window->removeLyricsAction()->isEnabled()); + QVERIFY(!m_window->showLyricsAction()->isEnabled()); + QVERIFY(!m_window->showLyricsAction()->isChecked()); + QVERIFY(m_window->importLyricsAction()->isEnabled()); + } + + // Hidden, not removed; not in the settings, and no command + void lyrics_show_toggle() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Layer *layer = lyrics->getLayer(); + sv::Pane *pane = m_window->paneStack()->getPane(0); + m_window->discardModifications(); + auto *history = sv::CommandHistory::getInstance(); + QSignalSpy commands(history, qOverload<> + (&sv::CommandHistory::commandExecuted)); + + QAction *show = m_window->showLyricsAction(); + show->trigger(); + QVERIFY(layer->isLayerDormant(pane)); + QVERIFY(!lyrics->isVisible()); + QVERIFY(!show->isChecked()); + QVERIFY(show->isEnabled()); + QVERIFY(lyrics->getLayer() == layer); + QVERIFY(paneHasLayer(0, layer)); + QVERIFY(m_window->isDocumentModified()); + + show->trigger(); + QVERIFY(!layer->isLayerDormant(pane)); + QVERIFY(lyrics->isVisible()); + QVERIFY(show->isChecked()); + + QCOMPARE(int(commands.count()), 0); + QCOMPARE(undoOnce(), QString()); + for (const QString &key : QSettings().allKeys()) { + QVERIFY2(!key.contains("lyrics", Qt::CaseInsensitive), + qPrintable("in the settings: " + key)); + } + } + + // The pane takes its hover readout and its vertical scale from its + // top layer, and the lyrics have neither + void lyrics_keep_top_layer() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + sv::Pane *pane = m_window->paneStack()->getPane(0); + sv::Layer *top = pane->getTopLayer(); + QVERIFY(top); + sv::Layer *reference = + m_window->analyser()->getLayer(Analyser::PitchTrack); + double r0 = 0, r1 = 0; + bool referenceScale = reference->getDisplayExtents(r0, r1); + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Layer *layer = m_window->lyrics()->getLayer(); + QVERIFY2(pane->getTopLayer() == top, + "the lyrics are the pane's top layer"); + QVERIFY(pane->getLayer(pane->getLayerCount() - 2) == layer); + + double s0 = 0, s1 = 0; + QCOMPARE(reference->getDisplayExtents(s0, s1), referenceScale); + QCOMPARE(s0, r0); + QCOMPARE(s1, r1); + } + + // One dialog each, and nothing changes: not even lyrics that are there + void lyrics_import_failure() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + m_window->discardModifications(); + + LyricsTrack *lyrics = m_window->lyrics(); + QString untimed = writeLrc("Just some words\nwith no times at all\n"); + QVector> failures { + { untimed, "No timed lyrics were found" }, + { m_dir.filePath("no-such-lyrics.lrc"), "could not be found" }, + { writeLrc(QByteArray(int(Lyrics::maxFileBytes) + 1, 'a')), + "over 1 MB" }, + }; + for (const auto &f : failures) { + QVERIFY2(!m_window->doImportLyricsFrom(f.first), + qPrintable(f.first)); + QStringList dialogs = dialogsMatching("Could not import lyrics"); + QCOMPARE(dialogs.size(), 1); + QVERIFY2(dialogs[0].contains(f.second), qPrintable(dialogs[0])); + QVERIFY(!lyrics->isShown()); + QCOMPARE(lyricsLayersInDocument(), 0); + QVERIFY(!m_window->isDocumentModified()); + } + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::ModelId model = lyrics->getModelId(); + sv::EventVector events = lyricsEvents(); + m_window->discardModifications(); + + QVERIFY(!m_window->doImportLyricsFrom(untimed)); + QCOMPARE(dialogsMatching("Could not import lyrics").size(), 1); + QVERIFY(lyrics->getModelId() == model); + QCOMPARE(lyricsEvents(), events); + QCOMPARE(lyricsLayersInDocument(), 1); + QVERIFY(!m_window->isDocumentModified()); + } + + // File > Import Lyrics..., with the file dialog answered from here + void lyrics_import_through_the_menu() { + makeWindow(FakeAudioIO::Config()); + QAction *import = m_window->importLyricsAction(); + QVERIFY2(!import->isEnabled(), "there is no reference yet"); + QVERIFY(!m_window->removeLyricsAction()->isEnabled()); + QVERIFY(!m_window->showLyricsAction()->isEnabled()); + + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(import->isEnabled()); + + // Line timing, as the exporter writes it + QString path = lyricsFixture("moises-exporter-lines.lrc"); + m_window->setLyricsFileAnswer(path); + import->trigger(); + QCOMPARE(m_window->lyricsFileQuestions(), 1); + QVERIFY(m_window->lyrics()->isShown()); + QCOMPARE(lyricsEvents(), expectedLyricsEvents(path)); + QVERIFY(m_window->isDocumentModified()); + } + + void lyrics_import_cancelled() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::ModelId model = lyrics->getModelId(); + sv::EventVector events = lyricsEvents(); + m_window->discardModifications(); + + m_window->setLyricsFileAnswer(""); + m_window->importLyricsAction()->trigger(); + QCOMPARE(m_window->lyricsFileQuestions(), 1); + QVERIFY(lyrics->getModelId() == model); + QCOMPARE(lyricsEvents(), events); + QCOMPARE(lyricsLayersInDocument(), 1); + QVERIFY(!m_window->isDocumentModified()); + } + + // The singer is reading the lyrics there are + void lyrics_import_disabled_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + QAction *import = m_window->importLyricsAction(); + QVERIFY(import->isEnabled()); + QString path = lyricsFixture("moises-exporter-words.lrc"); + m_window->setLyricsFileAnswer(path); + + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(!import->isEnabled(), 2000); + import->trigger(); + QCOMPARE(m_window->lyricsFileQuestions(), 0); + QVERIFY(!m_window->doImportLyricsFrom(path)); + QVERIFY(!m_window->lyrics()->isShown()); + + stopTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(import->isEnabled(), 2000); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform diff --git a/meson.build b/meson.build index 52a100c9..afe58372 100644 --- a/meson.build +++ b/meson.build @@ -1107,6 +1107,7 @@ tony_core_files = [ tony_app_files = [ 'main/AlternatePitchTrack.cpp', 'main/CoverageStrip.cpp', + 'main/LyricsTrack.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', @@ -1127,6 +1128,7 @@ tony_app_moc_files = qt.preprocess( 'main/Analyser.h', 'main/AlternatePitchTrack.h', 'main/CoverageStrip.h', + 'main/LyricsTrack.h', ]) qt_resource_files = qt.preprocess( @@ -1414,6 +1416,7 @@ tony_app_test_exe = executable( cpp_args: [ feature_defines, general_defines, + '-DTONY_TEST_DATA_DIR="' + tony_test_data_dir + '"', ], link_args: [ feature_additional_libs, From 716ac5e814cfbba63c5d00c12b7b0e8b9875ae5c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 20:54:44 +0000 Subject: [PATCH 066/275] feat: lyrics are kept in the session The lyrics' layer and model are document contents, so a session file already held them; what a load lacked was LyricsTrack taking the layer up again. analyseNewMainModel() now adopts it by its name, next to the alternate pitch track, with the visibility and the name it was saved with, and takes its model out of the play source, which the load had put it into. A session saved with the lyrics on top of the pane comes back with them under the layer that was below them. Tests show the lyrics coming through a save and reopen exactly, hidden or shown; left alone by recording, undo, redo, new takes, switching and Load Singing Track; and gone with their session. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsTrack.cpp | 27 +++ main/LyricsTrack.h | 8 + main/MainWindow.cpp | 13 ++ main/test/TestRecordWorkflow.h | 303 +++++++++++++++++++++++++++++++++ 4 files changed, 351 insertions(+) diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index adb39051..8af25132 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -108,6 +108,33 @@ LyricsTrack::show(Document *document, Pane *pane, const EventVector &events, return true; } +bool +LyricsTrack::adopt(Document *document, Pane *pane) +{ + if (m_layer) return true; + if (!document || !pane) return false; + + for (int i = 0; i < pane->getLayerCount(); ++i) { + auto layer = qobject_cast(pane->getLayer(i)); + if (!layer || layer->objectName() != layerName()) continue; + if (!ModelById::isa(layer->getModel())) continue; + takeLayer(document, pane, layer); + + // The session puts the layers back in the order they were saved + // in, and the lyrics end up on top if the layer that was above + // them went before the save. Not to stay there, for the reason + // show() leaves the top layer where it is + int count = pane->getLayerCount(); + if (count > 1 && pane->getTopLayer() == layer) { + TakeLayers::raise(pane, pane->getLayer(count - 2)); + } + + return true; + } + + return false; +} + void LyricsTrack::takeLayer(Document *document, Pane *pane, RegionLayer *layer) { diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index 50c9beaa..e054312e 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -65,6 +65,14 @@ class LyricsTrack : public QObject bool show(sv::Document *document, sv::Pane *pane, const sv::EventVector &events, QString presentationName); + /** + * Take over a layer that show() made in an earlier run, and that a + * session load has put back into the pane, with the visibility and + * the presentation name it was saved with. Returns false if there + * is none. + */ + bool adopt(sv::Document *document, sv::Pane *pane); + /// Delete the layer and its model from the document void hide(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index a321a5aa..795d2f11 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -7591,6 +7591,19 @@ MainWindow::analyseNewMainModel() syncAlternatePitchTrack(); } + // Lyrics saved with the session are there too. The load put their + // model into the play source when it added the layer to the view, as + // it does for every layer, and words past the end of the reference + // would hold playback open: out again, as after an import + if (pane && m_lyrics->adopt(m_document, pane)) { + cerr << "analyseNewMainModel: found the lyrics of the session" << endl; + if (m_playSource && !m_lyrics->getModelId().isNone()) { + m_playSource->removeModel(m_lyrics->getModelId()); + } + // Remove Lyrics; Show Lyrics is set by updateLayerStatuses() below + updateMenuStates(); + } + if (!m_withSpectrogram) { m_analyser->setVisible(Analyser::Spectrogram, false); } diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index c298e7c8..9e6277ed 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -901,6 +901,19 @@ class TestRecordWorkflow : public QObject return n; } + // Found by name, as a session load finds it, and not from LyricsTrack: + // the point is often whether that still has the layer the pane shows + sv::Layer *lyricsLayerInPane0() { + sv::Pane *pane = m_window->paneStack()->getPane(0); + if (!pane) return nullptr; + for (int i = 0; i < pane->getLayerCount(); ++i) { + if (pane->getLayer(i)->objectName() == "Lyrics") { + return pane->getLayer(i); + } + } + return nullptr; + } + sv::EventVector lyricsEvents() { sv::RegionLayer *layer = m_window->lyrics()->getLayer(); if (!layer) return {}; @@ -908,6 +921,31 @@ class TestRecordWorkflow : public QObject return model ? model->getAllEvents() : sv::EventVector(); } + // The lyrics are the very layer and model they were, with the same + // words, on show, under the top layer and out of the play source. The + // model id is what proves the layer is the same one: a model id is + // never used twice, unlike the address of a layer + void verifyLyricsUntouched(sv::Layer *layer, sv::ModelId model, + const sv::EventVector &events, QString when) { + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane); + QVERIFY2(lyricsLayersInDocument() == 1 && lyricsLayersInPane0() == 1, + qPrintable(when + ": not one lyrics layer")); + sv::Layer *found = lyricsLayerInPane0(); + QVERIFY2(found && found == layer && found->getModel() == model, + qPrintable(when + ": the lyrics layer was replaced")); + QVERIFY2(m_window->lyrics()->getLayer() == found, + qPrintable(when + ": LyricsTrack has let go of the layer")); + QVERIFY2(lyricsEvents() == events, + qPrintable(when + ": the words have changed")); + QVERIFY2(!found->isLayerDormant(pane) && m_window->lyrics()->isVisible(), + qPrintable(when + ": the lyrics are hidden")); + QVERIFY2(pane->getTopLayer() != found, + qPrintable(when + ": the lyrics are the pane's top layer")); + QVERIFY2(m_window->playSource()->getModels().count(model) == 0, + qPrintable(when + ": the lyrics are in the play source")); + } + static QString lyricsFixture(const char *name) { return QString(TONY_TEST_DATA_DIR) + "/lyrics/" + name; } @@ -5609,6 +5647,27 @@ private slots: QVERIFY2(m_window->statusText().contains ("1 word starts after the end of the reference."), qPrintable(m_window->statusText())); + + // The same once the session is opened again: the load puts the + // model of every layer it adds to a view into the play source + QString session = m_dir.filePath("lyrics-past-the-end.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->lyrics()->isShown()); + model = m_window->lyrics()->getModelId(); + QVERIFY(!model.isNone()); + sv::sv_frame_t lyricsEnd = sv::ModelById::get(model)->getEndFrame(); + playSource = m_window->playSource(); + QVERIFY2(playSource->getModels().count(model) == 0, + "the play source holds the lyrics of the session"); + QVERIFY2(playSource->getPlayEndFrame() < lyricsEnd, + qPrintable(QString("playback ends at frame %1, with the " + "lyrics, which end at %2") + .arg(playSource->getPlayEndFrame()) + .arg(lyricsEnd))); + verifyPlaySourceClean(); } void lyrics_reimport_replaces() { @@ -5839,6 +5898,250 @@ private slots: QTRY_VERIFY_WITH_TIMEOUT(import->isEnabled(), 2000); } + // Saved with the session and found again by name when it is opened, + // as it was: hidden if it was hidden + void lyrics_session_round_trip() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::EventVector before = lyricsEvents(); + QCOMPARE(int(before.size()), 13); + m_window->showLyricsAction()->trigger(); + QVERIFY(!lyrics->isVisible()); + + QString session = m_dir.filePath("lyrics-hidden.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + // One layer, and LyricsTrack has it + QVERIFY2(lyrics->isShown(), "the lyrics of the session were not adopted"); + QCOMPARE(lyricsLayersInDocument(), 1); + QCOMPARE(lyricsLayersInPane0(), 1); + sv::RegionLayer *layer = lyrics->getLayer(); + QCOMPARE(static_cast(layer), lyricsLayerInPane0()); + + // Every word at its frame and for as long, and its label letter for + // letter: the file is UTF-8, and a label is escaped in it + sv::EventVector after = lyricsEvents(); + verifyEventsSurvived(before, after, "the lyrics"); + if (QTest::currentTestFailed()) return; + QStringList labels; + for (size_t i = 0; i < after.size(); ++i) { + QCOMPARE(after[i].getLabel(), before[i].getLabel()); + labels << after[i].getLabel(); + } + QVERIFY2(labels.contains(QString("Tämä")) && + labels.contains(QString("hämärä")) && + labels.contains(QString("Yö")), qPrintable(labels.join(" "))); + + // Still hidden, and the menu says so + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(layer->isLayerDormant(pane)); + QVERIFY(!lyrics->isVisible()); + QVERIFY(m_window->showLyricsAction()->isEnabled()); + QVERIFY(!m_window->showLyricsAction()->isChecked()); + QVERIFY(m_window->removeLyricsAction()->isEnabled()); + + // Called what the file's [ti:] tag says, and drawn as before + QCOMPARE(layer->getLayerPresentationName(), + QString("Kesäyön testilaulu")); + QCOMPARE(int(layer->getPlotStyle()), int(sv::RegionLayer::PlotLyrics)); + QCOMPARE(colourOf(layer), colourNamed("Grey")); + QVERIFY2(pane->getTopLayer() != layer, + "the lyrics are the pane's top layer"); + + // Opening a session is not a change to it + QVERIFY(!m_window->isDocumentModified()); + + // Shown again, saved and opened again: shown + m_window->showLyricsAction()->trigger(); + QVERIFY(lyrics->isVisible()); + session = m_dir.filePath("lyrics-shown.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + QVERIFY(lyrics->isShown()); + QCOMPARE(lyricsLayersInDocument(), 1); + pane = m_window->paneStack()->getPane(0); + QVERIFY(!lyrics->getLayer()->isLayerDormant(pane)); + QVERIFY(lyrics->isVisible()); + QVERIFY(m_window->showLyricsAction()->isChecked()); + verifyEventsSurvived(before, lyricsEvents(), "the lyrics, saved twice"); + } + + // A session can have the lyrics on top of pane 0, if the layer that + // was above them went before it was saved. They do not stay on top + // once it is opened + void lyrics_not_top_after_reopen() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Pane *pane = m_window->paneStack()->getPane(0); + sv::Layer *layer = m_window->lyrics()->getLayer(); + TakeLayers::raise(pane, layer); + QVERIFY(pane->getTopLayer() == layer); + int count = pane->getLayerCount(); + QString below = pane->getLayer(count - 2)->objectName(); + QVERIFY(below != ""); + + QString session = m_dir.filePath("lyrics-on-top.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + // The layer that was just under them is over them, and nothing + // else has moved + pane = m_window->paneStack()->getPane(0); + layer = m_window->lyrics()->getLayer(); + QVERIFY(layer); + QCOMPARE(pane->getLayerCount(), count); + QVERIFY2(pane->getTopLayer() != layer, + "the lyrics are the pane's top layer"); + QCOMPARE(pane->getTopLayer()->objectName(), below); + QVERIFY(pane->getLayer(count - 2) == layer); + } + + // The lyrics belong to the song, not to a take: nothing a take does + // touches them, and the singer reads them while recording + void lyrics_survive_takes() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Layer *layer = lyricsLayerInPane0(); + QVERIFY(layer); + sv::ModelId model = m_window->lyrics()->getModelId(); + sv::EventVector events = lyricsEvents(); + QVERIFY(!events.empty()); + + startTake(); + if (QTest::currentTestFailed()) return; + verifyLyricsUntouched(layer, model, events, "at the start of the take"); + if (QTest::currentTestFailed()) return; + QTest::qWait(600); + verifyLyricsUntouched(layer, model, events, "while recording"); + if (QTest::currentTestFailed()) return; + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->takes()->haveTake()); + verifyLyricsUntouched(layer, model, events, "after the take"); + if (QTest::currentTestFailed()) return; + + QCOMPARE(undoOnce(), QString("Record Singing")); + verifyLyricsUntouched(layer, model, events, "after the undo"); + if (QTest::currentTestFailed()) return; + QCOMPARE(redoOnce(), QString("Record Singing")); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + verifyLyricsUntouched(layer, model, events, "after the redo"); + if (QTest::currentTestFailed()) return; + + // The first take's layers are put away for the new one, and it has + // its audio swapped in under them on the way back + m_window->doNewEmptyTake(); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + verifyLyricsUntouched(layer, model, events, "in a new take"); + if (QTest::currentTestFailed()) return; + m_window->doChooseTakeInCombo(0); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + verifyLyricsUntouched(layer, model, events, "back in the first take"); + if (QTest::currentTestFailed()) return; + verifyPlaySourceClean(); + + // A session with takes is restored after its lyrics are found, + // and that leaves them alone as well + QString session = m_dir.filePath("lyrics-takes.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takes()->getTakeCount(), 2); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + sv::EventVector reopened = lyricsEvents(); + verifyEventsSurvived(events, reopened, "the lyrics"); + if (QTest::currentTestFailed()) return; + verifyLyricsUntouched(lyricsLayerInPane0(), + m_window->lyrics()->getModelId(), reopened, + "after the session was opened"); + if (QTest::currentTestFailed()) return; + verifyPlaySourceClean(); + } + + // Load Singing Track opens its file with openPath(), prunes the pane + // that makes and clears the history: none of it is the lyrics' business + void lyrics_survive_load_singing_track() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Layer *layer = lyricsLayerInPane0(); + sv::ModelId model = m_window->lyrics()->getModelId(); + sv::EventVector events = lyricsEvents(); + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + verifyLyricsUntouched(layer, model, events, "after Load Singing Track"); + if (QTest::currentTestFailed()) return; + verifyPlaySourceClean(); + } + + // The lyrics go with their session, and no other session is given them + void lyrics_gone_with_session() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QString plain = m_dir.filePath("no-lyrics.ton"); + QVERIFY(m_window->saveSessionFile(plain)); + + LyricsTrack *lyrics = m_window->lyrics(); + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::ModelId model = lyrics->getModelId(); + QString withLyrics = m_dir.filePath("with-lyrics.ton"); + QVERIFY(m_window->saveSessionFile(withLyrics)); + + m_window->doCloseSession(); + QVERIFY(!lyrics->isShown()); + QVERIFY(!lyrics->getLayer()); + QVERIFY2(!sv::ModelById::get(model), + "the lyrics model outlived its session"); + + auto noLyrics = [&](const char *what) { + QVERIFY2(!lyrics->isShown(), what); + QVERIFY2(lyricsLayersInDocument() == 0, what); + QVERIFY2(!m_window->removeLyricsAction()->isEnabled(), what); + QVERIFY2(!m_window->showLyricsAction()->isEnabled(), what); + QVERIFY2(!m_window->showLyricsAction()->isChecked(), what); + QVERIFY2(m_window->importLyricsAction()->isEnabled(), what); + }; + + openReference(writeWav(tone(highHz, 1.0))); + if (QTest::currentTestFailed()) return; + noLyrics("a new reference was given the lyrics"); + if (QTest::currentTestFailed()) return; + + // A session without lyrics, opened after one with them + openReference(withLyrics); + if (QTest::currentTestFailed()) return; + QVERIFY(lyrics->isShown()); + openReference(plain); + if (QTest::currentTestFailed()) return; + noLyrics("a session without lyrics was given the last one's"); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform From 870c7d37ec6adec8b8ac71441073221ca3f0d799 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 21:01:18 +0000 Subject: [PATCH 067/275] fix: lyrics stay under the top layer when the layer above them goes The pane takes its hover readout and its vertical scale from its top layer, which the lyrics never are: an import puts them under the layer on top, and a session load puts them back under it. But deleting the layer just above them, by turning the alternate pitch track off or deleting a take, left them on top until the session was loaded again. LyricsTrack now looks again once any other layer has gone, and puts the layer under the lyrics back on top if they are there. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsTrack.cpp | 28 ++++++++++++++++++++++++---- main/LyricsTrack.h | 4 ++++ main/test/TestRecordWorkflow.h | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 4 deletions(-) diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index 8af25132..5ff862b5 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -24,6 +24,9 @@ #include "data/model/RegionModel.h" #include "base/PlayParameters.h" +#include +#include + #include using namespace sv; @@ -124,10 +127,7 @@ LyricsTrack::adopt(Document *document, Pane *pane) // in, and the lyrics end up on top if the layer that was above // them went before the save. Not to stay there, for the reason // show() leaves the top layer where it is - int count = pane->getLayerCount(); - if (count > 1 && pane->getTopLayer() == layer) { - TakeLayers::raise(pane, pane->getLayer(count - 2)); - } + keepUnderTop(); return true; } @@ -208,6 +208,16 @@ LyricsTrack::isVisible() const return m_layer && m_pane && !m_layer->isLayerDormant(m_pane); } +void +LyricsTrack::keepUnderTop() +{ + if (!m_layer || !m_pane) return; + int count = m_pane->getLayerCount(); + if (count > 1 && m_pane->getTopLayer() == m_layer) { + TakeLayers::raise(m_pane, m_pane->getLayer(count - 2)); + } +} + void LyricsTrack::layerAboutToBeDeleted(Layer *layer) { @@ -215,7 +225,17 @@ LyricsTrack::layerAboutToBeDeleted(Layer *layer) if (layer && layer == m_layer) { m_layer = nullptr; hide(); + return; } + + // Another layer going can leave the lyrics on top of the pane: the + // alternate pitch track turned off, a take deleted. It is still in + // the pane now, so this looks again once it has gone. A layer and a + // pane that have gone by then are not ours to look at + QPointer pane(m_pane); + QTimer::singleShot(0, this, [this, pane]() { + if (pane && pane == m_pane) keepUnderTop(); + }); } ModelId diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index e054312e..9d73c077 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -107,6 +107,10 @@ private slots: void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *); void configureLayer(); + + // If the lyrics are the pane's top layer, put the one under them on + // top: see show() + void keepUnderTop(); }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 9e6277ed..f5dd9a38 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -5789,6 +5789,38 @@ private slots: QCOMPARE(s1, r1); } + // The layer over the lyrics can go while they stay: the alternate + // pitch track when it is turned off, a take's layers when the take + // is deleted. The lyrics must not be left on top then either + void lyrics_stay_under_top_when_the_layer_above_goes() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + m_window->doToggleAlternatePitch(); + QVERIFY(m_window->alternatePitch()->isShown()); + sv::Pane *pane = m_window->paneStack()->getPane(0); + QVERIFY(pane->getTopLayer() == m_window->alternatePitch()->getLayer()); + sv::Layer *under = pane->getLayer(pane->getLayerCount() - 2); + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::Layer *layer = m_window->lyrics()->getLayer(); + QVERIFY(pane->getLayer(pane->getLayerCount() - 2) == layer); + + // The alternate track's layer is deleted, and the lyrics were + // just under it + m_window->doToggleAlternatePitch(); + QVERIFY(!m_window->alternatePitch()->isShown()); + QTRY_VERIFY2(pane->getTopLayer() != layer, + "the lyrics were left the pane's top layer"); + QVERIFY2(pane->getTopLayer() == under, + "the layer that was under the lyrics is not on top"); + QVERIFY(pane->getLayer(pane->getLayerCount() - 2) == layer); + QVERIFY(m_window->lyrics()->isShown()); + QVERIFY(m_window->lyrics()->isVisible()); + } + // One dialog each, and nothing changes: not even lyrics that are there void lyrics_import_failure() { makeWindow(FakeAudioIO::Config()); From b51afa682d347df37118864581ec9d0a78848b43 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 20:34:08 +0000 Subject: [PATCH 068/275] docs: record the android and sailfish os port research Feasibility research for running Tony on a phone, kept so that a session starting either port need not repeat it. mobile-port.md holds the decisions, the facts about Tony and its libraries that any port depends on, and a comparison; port-android.md and port-sailfish.md hold the dated platform facts with sources, the work, and a first test port. The research also found that a recording device not at 44.1 kHz probably places take audio at the wrong scale, since the splice does not resample; noted in open-points.md as unverified. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- AGENTS.md | 1 + docs/README.md | 2 + docs/mobile-port.md | 226 +++++++++++++++++++++++++++++++ docs/open-points.md | 6 + docs/port-android.md | 271 +++++++++++++++++++++++++++++++++++++ docs/port-sailfish.md | 308 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 814 insertions(+) create mode 100644 docs/mobile-port.md create mode 100644 docs/port-android.md create mode 100644 docs/port-sailfish.md diff --git a/AGENTS.md b/AGENTS.md index 44e4d87a..70d6a39f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,7 @@ code. | [docs/takes.md](docs/takes.md) | touching takes, the audio swap, ranged analysis, undo, the coverage strip, save/restore | | [docs/forks.md](docs/forks.md) | needing a change in `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` | | [docs/open-points.md](docs/open-points.md), [docs/manual-checklist.md](docs/manual-checklist.md) | choosing what to do next, or saying what the user should try by hand | +| [docs/mobile-port.md](docs/mobile-port.md), then [docs/port-android.md](docs/port-android.md) or [docs/port-sailfish.md](docs/port-sailfish.md) | starting or working on a phone port | ## Build and test diff --git a/docs/README.md b/docs/README.md index 6eea96bc..64a29e57 100644 --- a/docs/README.md +++ b/docs/README.md @@ -19,3 +19,5 @@ methods, and they do not tell the story of fixed bugs. | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | | [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | +| [mobile-port.md](mobile-port.md) | Porting to a phone: decisions, what in the code any port depends on, Android against Sailfish OS | +| [port-android.md](port-android.md), [port-sailfish.md](port-sailfish.md) | Platform facts with sources, the work, and the first test port for each | diff --git a/docs/mobile-port.md b/docs/mobile-port.md new file mode 100644 index 00000000..a13ca925 --- /dev/null +++ b/docs/mobile-port.md @@ -0,0 +1,226 @@ +# Porting Tony to a phone: Android or Sailfish OS + +**Status: researched 2026-09-24 and 2026-09-25; nothing built.** This page and its two +platform pages hold what that research found, so that a session starting a port does not +have to find it again: + +- [port-android.md](port-android.md): Android facts, the work, the first test port. +- [port-sailfish.md](port-sailfish.md): Sailfish OS (Jolla phones), the same. + +Facts about Tony and its libraries below were read from the code and hold until the code +changes. Facts about the platforms come from the web and are dated. Anything marked +*(snippet)* was seen only in search results, because the proxy blocked the page. Check a +web fact again before a step depends on it. + +## Decisions so far + +- **Port the existing Qt Widgets app; do not rewrite it.** Every rewrite considered amounts + to a new application and loses the `.ton` compatibility with the desktop (see + [Rejected](#rejected-alternatives)). +- **On the phone the app is for practice**: open a session, choose a take, select a phrase, + record, listen, erase, undo. Editing notes and the reference stays on the desktop. +- **No platform has been chosen.** In short: + - Sailfish needs less new code: its audio backend, file access, pYIN loading and build + are mostly there already. + - Sailfish carries more risk: Qt 6 is community-packaged there, window-manager support + for such apps only appeared in April 2026, and apps are routed to a high-latency audio + path. + - Android needs more code: an audio backend, file import and export, packaging. + - Android's unknowns are amounts of work, not open questions. +- **Before either**, run [manual-checklist.md](manual-checklist.md) items 1 to 3 on the + desktop. As of this research the latency compensation had never been checked by hand; do + not add a phone's variability to an unverified mechanism. +- Each port starts with a **test port** that answers the platform's open questions before + anything else is built. The platform pages describe it. + +## Comparison by area + +| Area | Android | Sailfish OS | Easier on | +| --- | --- | --- | --- | +| Qt 6 | Official Qt 6.11 for Android | System Qt is 5.6; community Qt 6.8.4 from the Chum repository | Android | +| Windows and dialogs | One full-screen window, dialogs inside it | Every window and dialog shown maximised, no title bar | Android | +| Build | NDK cross-compile, androiddeployqt, no meson precedent | Sailfish SDK builds RPMs; meson is in the platform | Sailfish | +| C libraries | All cross-compiled | Four in the OS, six to build | Sailfish | +| Audio backend | New Oboe backend in a bqaudioio fork | Existing PulseAudio backend | Sailfish | +| Latency | AAudio low-latency path; latency estimated from timestamps | Deep-buffer sink; fixed latency estimate from the HAL | Android | +| pYIN loading | Packaging change or svcore fork change | Unchanged | Sailfish | +| File access | `content://` URIs: copy in and out | Ordinary paths | Sailfish | +| Sharing with the desktop | Sync app plus "All files access", or the system picker | Syncthing, or rclone | Tie | +| Touch layout | Compact mode and gestures | The same, plus edge-swipe conflicts | Android | +| Distribution | Sideloaded APK | Sideloaded RPM or Chum; not the Jolla Store | Tie | +| Platform risk | Mainstream | Small, active ecosystem; Qt 6 is community-maintained | Android | + +An Android APK would also run on a Jolla phone through Android App Support. Its audio goes +through the same PulseAudio, with unmeasured latency. See +[port-sailfish.md](port-sailfish.md#android-app-support). + +## What in Tony and its libraries matters to any port + +The library directories are separate repositories (see [forks.md](forks.md)). To read +them, clone `jhhr/svcore`, `jhhr/svgui` and `jhhr/svapp` (branch `tony-customizations`), +`jhhr/bqaudiostream`, `breakfastquay/bqaudioio` and `c4dm/pyin` from GitHub. That worked +from a cloud session. + +### Qt 6 only + +- svgui and `main/` use Qt 6-only API with no Qt 5 fallback: + - `QMouseEvent::position()`: about 160 uses in 12 files. + - `enterEvent(QEnterEvent *)` overrides: about 30 files. +- svcore has `QT_VERSION` guards around `QStringConverter`. +- A Qt 5.15 build would need a mechanical backport in the forks. Qt 5.6 is out of reach + (`Qt::SkipEmptyParts`, `horizontalAdvance`, `QOverload` and more). +- Built with Qt 6.11 on the development machine. `main.cpp` guards only the colour-scheme + call, at 6.8, so 6.8 is plausible but untried. + +### Audio I/O + +- **bqaudioio is upstream, not a fork.** It is on sourcehut (Mercurial), mirrored at + `github.com/breakfastquay/bqaudioio`. `AudioFactory.cpp` knows JACK, PulseAudio and + PortAudio only. A new backend, or any buffer change, means forking it by the procedure + in [forks.md](forks.md) and adding it to `repoint-project.json`. +- `PortAudioIO.cpp` (about 760 lines) is the model for a new backend: + - one duplex stream, input and output in the same callback; + - input and output latency from the stream info, handed to `setSystemRecordLatency()` + and `setSystemPlaybackLatency()`. +- `PulseAudioIO.cpp` (about 840 lines): + - separate capture and playback streams on one `pa_mainloop` thread; + - flags `PA_STREAM_INTERPOLATE_TIMING | PA_STREAM_AUTO_TIMING_UPDATE` and **no buffer + attributes**, so the server's default buffering applies, which can be large; + - latency from `pa_stream_get_latency()`. +- Tony's compensation ([recording.md](recording.md#latency)) is reported play latency plus + reported record latency plus the measured start gap. + - The start-gap measurement assumes the driver hands over a block's input before asking + for its output. A duplex stream does that; two PulseAudio streams need not. + - Wherever the reported latency is only an estimate, the compensation is only as good as + the estimate. +- **There is no manual latency offset or calibration setting.** It is the fallback on any + platform whose reported latency is wrong, and it would help the desktop too. Build it if a + test port shows takes landing off the beat. +- Development has used PortAudio on Windows only. The PulseAudio recording path has never + been exercised with takes. + +### Sample rate + +What the code does: +- `MainWindow` sets `Preferences` to resample on load, at a fixed 44100 Hz. So the + reference, and every file opened, is a 44.1 kHz model. +- Neither side of the audio device asks for a rate: + - `AudioCallbackPlaySource::getApplicationSampleRate()` and + `AudioCallbackRecordTarget::getApplicationSampleRate()` both return 0. + - Playback is resampled to whatever the device runs at. +- The backends then choose: + - `PortAudioIO` opens at the output device's **default** rate; + - `PulseAudioIO` asks PulseAudio for **44100**, which resamples to the hardware. +- Recordings are written at the device's rate. +- Take timing uses the main model's rate: `currentTakeTiming()` sets `TakeTiming::rate` + from `getMainModel()`, and the pre-roll is computed the same way. +- `TakeAudio::splice()` does not resample. It refuses only a recording whose rate differs + from the take so far. + +So with a device that is not at 44.1 kHz, take audio would presumably be placed at the +wrong scale. Nothing verifies this; [takes.md](takes.md#known-limitations) calls a device +rate different from the reference's unexercised. It may already affect a Windows machine +whose default output device runs at 48 kHz. + +Phones run at 48 kHz natively. **Check this on the desktop with a 48 kHz device before a +port.** The fixes: +- resample the recording to the main model's rate before the splice, in `tony_core`, + which is testable; +- or open the device at 44.1 kHz (Oboe can convert, at some cost in latency). + +### The pYIN plugin + +- `meson.build` builds `pyin` and `chp` as shared libraries with `name_prefix: ''`, which + gives `pyin.so` and `chp.so`. +- `setupTonyVampPath()` in `main.cpp`: + - `TONY_VAMP_PATH` overrides the default; + - otherwise, off Windows and macOS, `VAMP_PATH` is `/../lib/`, + `/../lib/` and ``. +- `HAVE_PLUGIN_CHECKER_HELPER` is not defined, so svcore's `NativeVampPluginFactory` + scans the `VAMP_PATH` directories for `*.so` itself and `dlopen`s them. No helper process + runs; the `checker/` sources are compiled but not used for Vamp plugins. + +### Files and sessions + +- `FileSource` handles local files, `http`, `https` and `ftp`. Readers open paths: + `WavFileReader` through `sf_open()`, bqaudiostream's WAV reader through `std::ifstream`. + **A `content://` URI does not open.** +- Opening goes through svgui's `InteractiveFileFinder`, which runs its own `QFileDialog` + instance and then checks the result with `QFileInfo`. +- **Sessions already move between machines:** + - The reference is saved as an absolute `file=` path. + - On load, `SVFileReader` calls `FileFinder::find()` with the session's location, and + `InteractiveFileFinder::findRelative()` finds a file of the same name next to the + `.ton`. + - Take audio is in `.takes/` with relative paths. + - So `Song.ton`, `Song.mp3` and `Song.takes/` move together without prompts, provided the + reference sits next to the `.ton`. +- Take files are uncompressed WAV, several MB per minute, which matters for syncing. + +### The window + +- `MainWindow` has 7 menus: File, Edit, View, Analysis, Takes, Playback, Help. +- Its toolbars are File, Tools, Playback, Play Mode, Playback Controls (speed and gain) and + the bottom "Show and Play" bar. +- Size: about 190 `addAction` calls, about 47 shortcuts and about 88 message-box or dialog + calls. +- `menuBar()->setNativeMenuBar(false)` sits under `#ifdef Q_OS_LINUX`. **Qt defines + `Q_OS_LINUX` on Android too**, so on Android this forces the in-window menu bar instead + of Qt's ⋮ options menu. Exclude `Q_OS_ANDROID` there if the ⋮ menu is wanted. +- **svgui has no touch or gesture code** (no `QGesture` or `QTouchEvent`). + - Qt synthesises mouse events from unhandled touches. Tapping, one-finger drag to pan in + Navigate mode (`Pane::dragTopLayer()`) and dragging in the selection strip (SelectMode + through `setToolModeFor()`) should therefore work. + - Pinch to zoom, two-finger scrolling and long-press for the right-button menu (the pane + emits `rightButtonMenuRequested` on a right click) need adding in the svgui fork. +- `main.cpp`: + - `--no-audio` selects `AUDIO_NONE`, which is useful for a first test port; + - the window is sized from the screen; + - the colour scheme is forced light. + +### Build and tests + +- `meson.build` has `linux`, `darwin` and `windows` branches. + - The Linux branch (`if system == 'linux'`) lists the C libraries and their `HAVE_*` + defines: fftw3, sndfile, samplerate, rubberband ≥ 3, sord/serd, oggz, fishsound, mad, + id3tag, opusfile, optional opusenc, jack, libpulse, alsa and optional portaudio. + - RtMidi uses the ALSA defines. + - Qt modules: Core, Gui, Widgets, Xml, Network, Svg, Test. +- For a phone: + - drop JACK, ALSA (and RtMidi's `__LINUX_ALSA*__` defines), oggz and fishsound; + - keep the rest, or leave out a library's `HAVE_*` define where svcore allows it. +- `.github/workflows/linux.yml` is the model for a CI job: Ubuntu, apt packages, meson from + a release tarball, Rubber Band from source, `./repoint install`. +- The test suites use `FakeAudioIO` and keep running on the desktop. A phone audio backend + can only be tested on the phone. + +## Work common to both ports + +- **Compact touch mode.** Following the rules in [AGENTS.md](../AGENTS.md), it is a class of + its own and `MainWindow` only wires it. + - Landscape only. + - One toolbar: Play, Record, Record into Selection, the Take box, Undo, Redo, Erase (the + Ctrl+D action as a button), zoom. + - The Show and Play toggles and gains in a slide-out panel. + - Hidden: the note-editing tools, the audio device menus, perhaps the spectrogram. +- **Gestures in the svgui fork**: pinch zoom, two-finger scroll, long-press menu. +- **The latency calibration setting** described above, if a test port needs it. +- **The sample-rate check** described above. +- **Headphones.** + - Wired or USB-C headphones with the phone's own microphone are the safe setup. + - Bluetooth output adds a large, variable latency. + - On Android, using a Bluetooth headset's microphone switches to call-quality audio. + +## Rejected alternatives + +- **A Qt Quick (QML) interface.** svgui's panes are `QWidget`s painted with `QPainter`, + and `MainWindow` builds on svapp's `MainWindowBase`, a `QMainWindow`. It would be a new + application. +- **A native Kotlin app calling pYIN through JNI.** It would feel best on a phone, but + takes, undo and sessions would be rewritten, and Sonic Visualiser's session XML is hard to + write from outside, so desktop compatibility would be lost. Months of work. +- **A web app, or Qt for WebAssembly.** + - Up to Qt 6.11, microphone input sits on deprecated browser API. + - An AudioWorklet backend landed on Qt's dev branch in May 2026 (QTBUG-115191), + probably for 6.12. + - It has the same interface problems, plus browser limits. diff --git a/docs/open-points.md b/docs/open-points.md index 7776f247..c18da1ba 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -22,9 +22,15 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - Background music is not saved in the session; it is reloaded by hand. - An old session (before takes) loses its singing track without telling the user why. - Recording that starts before frame 0 of the reference. +- A phone version. Android and Sailfish OS were researched and nothing was built; see + [mobile-port.md](mobile-port.md). ## Weak spots +- **A recording device not at 44.1 kHz** probably places take audio at the wrong scale: + recordings are written at the device's rate, take timing uses the reference's 44.1 kHz, + and the splice does not resample. Unverified; see + [mobile-port.md](mobile-port.md#sample-rate). - **If pYIN fails part-way, the live dots wait for ever**: they are removed on `initialAnalysisCompleted`, which then never comes. - **`Analyser::newFileLoaded()` error path for the singing track** (pYIN plugin missing): diff --git a/docs/port-android.md b/docs/port-android.md new file mode 100644 index 00000000..df246500 --- /dev/null +++ b/docs/port-android.md @@ -0,0 +1,271 @@ +# Android port: findings and first steps + +Researched 2026-09-24; nothing built. Read [mobile-port.md](mobile-port.md) first: it has +the decisions, the facts about Tony's own code that any port depends on, and the work +common to both platforms. *(snippet)* marks a fact seen only in search results. + +## Platform facts + +### Qt for Android + +- **Versions, from Qt's `macros.qdocconf`:** + + | Qt | NDK | JDK | Android API | Build tools, Gradle / AGP | + | --- | --- | --- | --- | --- | + | 6.11 | r27c (27.2.12479018) | 21 | min 28 (Android 9), target 36 | 36.0.0, 9.3.1 / 9.0.0 | + | 6.8 | r26b, r27c | 17 | 28 to 35 | not noted | + | 6.12 (dev) | r27c | not noted | 28 to 36 | Gradle 9.5.1 | + + Qt 6.12 LTS was due around 2026-09-22; whether it has shipped was not confirmed. +- **Qt Widgets are supported, but QML is the expected route.** Qt's Android page describes + Widgets as something to add "if needed". There is no official list of Widgets + limitations. From the Android platform plugin's source: + - `QMenuBar` becomes the Android options (⋮) menu (`QAndroidPlatformMenuBar`) unless + `setNativeMenuBar(false)` is called. Tony calls it under `Q_OS_LINUX`, which Android + also defines (see [mobile-port.md](mobile-port.md#the-window)). + - `QFileDialog` is always the native system picker. + - `QMessageBox` is native only with the environment variable + `QT_USE_ANDROID_NATIVE_DIALOGS=1`; otherwise Qt draws it. + - Widgets use the "android" style plugin or fall back to Fusion. + - The back key is a navigation request, not a close, and `closeEvent()` does not run. + Since Qt 6.12 it sends the app to the background. + - There is no hover, so tooltips and hover states are lost. + - Kinetic scrolling of item views needs `QScroller`, reported janky *(snippet)*. + - High-DPI scaling is always on in Qt 6. +- **16 KB page alignment** matters only for Google Play: + - NDK r28 and later align native libraries by default; with r27 add + `-Wl,-z,max-page-size=16384`. + - Qt fixed its own libraries in 6.8.6, 6.9.3 and 6.10.0. + +### Build and packaging + +- **Packaging is done by androiddeployqt.** It reads + `android--deployment-settings.json`. CMake and qmake write that file, and Qt's + documentation says not to edit it by hand. +- **Without CMake, the file must be written by the build.** The pcons build tool does this + (`examples/79_qt_android_apk`, `tests/toolchains/test_qt_android_settings.py`). + - androiddeployqt will not start without eight keys: `qt`, `sdk`, `ndk`, `ndk-host`, + `architectures`, `application-binary`, `toolchain-prefix`, `stdcpp-path`. + - The application must be a shared library named like `lib_arm64-v8a.so`, not an + executable. +- **Meson:** + - No public example of a meson-built Qt Android app was found. + - meson's Qt 6 module has trouble finding the host `moc` and `rcc` when + cross-compiling (meson issue 13018, open; issue 6089, the build machine's qmake picked + up). + - Mesa's Android documentation shows the usual NDK cross-file. +- **Two routes; decide in the test port:** + - (a) meson with an NDK cross file and host Qt tools, a script that writes the deployment + JSON, then androiddeployqt. + - (b) A `CMakeLists.txt` for Android only, using `qt_add_executable`. This is the + supported route, but it duplicates the source lists of `meson.build`, which must then + be kept in step. +- **Libraries:** + - libsndfile documents Android builds (`Building-for-Android.md`, autotools or CMake), + without its codec libraries. + - The others (libsamplerate, fftw3, Rubber Band, opusfile, serd/sord, mad/id3tag) are to + be cross-compiled. + - Drop JACK, PulseAudio, ALSA, PortAudio, oggz and fishsound. +- **Build in CI, not on the MSYS2 machine**: a GitHub Actions job on Linux producing an APK + to sideload, modelled on `.github/workflows/linux.yml`. The development machine has no + Android toolchain. + +### Audio + +- **Do not use Qt Multimedia for this.** + - Up to Qt 6.9, `QAudioSource` and `QAudioSink` used OpenSL ES. From 6.10 they use AAudio + (QTBUG-132951). + - Output streams are low-latency, with a three-burst buffer. + - Input is deliberately opened with `AAUDIO_PERFORMANCE_MODE_NONE`, because some devices + limit how many low-latency streams can be open. No input preset is set yet. + - There is no latency or timestamp API. + - Google measured 205 ms round trip without low-latency mode, so Qt's input path may be + slow. That is an inference, not measured. +- **Google recommends Oboe.** + - Oboe uses AAudio on API 27 and later and OpenSL ES below that. OpenSL ES is "not + recommended for new designs". + - Recent releases are 1.10 and 1.11; the repository is active. + - `oboe::FullDuplexStream` gives input and output in one callback, which is what the + start-gap measurement assumes (see [recording.md](recording.md#latency)). + - Devices run at 48 kHz natively, and Tony's models at 44.1 kHz. Either open the streams + at 44.1 kHz and let Oboe convert (which may lose the lowest-latency path), or fix the + splice to resample; see [mobile-port.md](mobile-port.md#sample-rate). +- **Latency figures:** + - About 20 ms round trip when every recommendation is followed; 205 ms when the + performance mode is not low-latency. + - Popular phones averaged about 39 ms in 2021, down from 109 ms in 2017. + - The feature flag `android.hardware.audio.low_latency` means output latency of 45 ms or + less; `android.hardware.audio.pro` means round trip of 20 ms or less. + - Android has "no API to determine audio latency at runtime". Latency is estimated from + `AAudioStream_getTimestamp()`, which the compatibility requirements say is accurate to + ±2 ms. Oboe's `calculateLatencyMillis()` does the estimate. +- **Alternative to a new bqaudioio backend: PortAudio with Oboe.** + - Upstream PortAudio has no Android host API. + - A pull request adding one (PortAudio pull request 1084, by the Mixxx developer + acolombier) was still open in September 2026. + - Forks exist: `NetResultsIT/portaudio-oboe`, and `croissanne/portaudio_opensles` + (deprecated OpenSL ES). + - This route would keep bqaudioio's existing `PortAudioIO` and need no bqaudioio fork, + but it depends on unmerged code. Compare the two in the audio step. + +### Files, storage and cloud apps + +- **The file dialog is the Storage Access Framework** (`qandroidplatformfiledialoghelper.cpp`): + - Open uses `ACTION_OPEN_DOCUMENT` with `CATEGORY_OPENABLE`, plus `EXTRA_ALLOW_MULTIPLE` + for several files. + - Save uses `ACTION_CREATE_DOCUMENT`, with the suggested name in `EXTRA_TITLE`. + - Directory mode uses `ACTION_OPEN_DOCUMENT_TREE`. + - Name filters are turned into MIME types. + - Every result gets a persistable URI permission. + - `exec()` blocks in a nested event loop. + - Results are `content://` URIs, also from `getOpenFileName()`. +- **`QFile` opens `content://` URIs** through Qt's Android content file engine + (`androidcontentfileengine.cpp`, using `ContentResolver.openFileDescriptor`). It also + supports size, time, MIME type, iterating a tree URI, mkdir, remove, rename and creating + files under a tree. +- **Pitfalls:** + - Writing uses mode "w", which since Android 10 may not truncate, depending on the + provider. Overwriting a longer file can leave old bytes at the end. + - Cloud providers may return a pipe, which cannot seek. + - `QFileInfo::path()` and `absolutePath()` return the URI string, so code that builds + paths, changes suffixes, or writes a temporary file and renames it breaks. + - Picking one file grants no access to the files next to it. + - C libraries that open by path need a local copy. For Tony: **copy into app storage on + open, write locally and copy out on export**, never edit through the URI. +- **Cloud apps in the picker:** + - Google Drive, Dropbox and OneDrive each have a document provider, so they appear when + opening a file. + - Whether each supports creating files and picking a folder is **unconfirmed**; reports + say Drive does not support folder picking. + - KeePassDX issues report data lost writing through the Dropbox provider (2023) and the + OneDrive provider (2023). Export a new copy rather than overwriting. + - A whole session (`.ton`, reference audio, takes folder) therefore needs a one-file + bundle if it is to travel through the picker, for example a zip with Export and Import + Session Bundle on both desktop and phone. +- **Sync apps** mirror a real folder: + - FolderSync (many clouds); Dropsync (Dropbox) and OneSync (OneDrive), both from MetaCtrl. + - Syncthing-Fork (researchxxl, v2.1.5.0 on 2026-09-08): phone to PC directly, no cloud. + The official Syncthing Android app was discontinued in December 2024. +- **Reading such a folder by path needs a permission:** + - Reading non-media files (sessions) in shared storage needs either a folder grant, which + gives `content://` URIs again, or `MANAGE_EXTERNAL_STORAGE` ("All files access"). + Google Play restricts that permission; a sideloaded APK can use it. + - Audio alone can be read through MediaStore with `READ_MEDIA_AUDIO` (API 33 and later). + - With "All files access" the existing path-based code works unchanged. That is the + cheapest first sharing option. + +### Permissions and lifecycle + +- **Microphone:** `QMicrophonePermission` (Qt 6.5 and later), through + `qApp->checkPermission()` and the asynchronous `qApp->requestPermission()`. + - It maps to `RECORD_AUDIO`, which must also be in the manifest. + - androiddeployqt inserts it only when Qt's multimedia plugins are deployed. Tony does + not use them, so add it at the `` placeholder of a custom + manifest. +- **Background:** the system may stop a backgrounded app. On + `Qt::ApplicationSuspended`, stop audio and save; Tony has no autosave today. + +### The pYIN plugin + +- Apps targeting API 29 or later may still `dlopen` libraries; they may not `exec` files. + Libraries in the app's native library directory load normally. +- Reportedly only files named `lib*.so` are packaged *(unconfirmed)*, so `pyin.so` needs a + `lib` prefix. +- With modern packaging the libraries stay inside the APK and the native library directory + can be empty, so svcore's scan of `VAMP_PATH` finds nothing. Two fixes: + - Turn on legacy packaging (`android-legacy-packaging` for androiddeployqt, or + `useLegacyPackaging` in Qt's gradle template) and name the plugin `libpyin.so`. + - Link pYIN into the app and have svcore load it without a scan (a small svcore fork + change). +- Precedents link Vamp plugin code straight in rather than loading it, for example + `recifra/cordova-plugin-chordino`. +- No Android port of Sonic Visualiser or Tony exists. + +## What has to be built + +| Item | Size | Notes | +| --- | --- | --- | +| Android build and APK in CI | Medium, fiddly | The largest risk to the schedule | +| Audio: Oboe backend in a bqaudioio fork, or PortAudio with Oboe | Medium | Must report latency and deliver input before output | +| pYIN loading | Small | Renaming and legacy packaging, or linking it in | +| File import and export through app storage | Small to medium | Or "All files access" and a sync app first | +| Menu bar under `Q_OS_ANDROID`, microphone permission, save on suspend | Small | | +| Compact touch mode and gestures | Medium to large | Common to both ports | +| Sample-rate check, latency calibration | Small each | Common to both ports | + +## Test port + +This step decides whether to go on. + +1. **CI job** (Linux runner): + - Qt 6.11 for `android_arm64_v8a` plus the host Qt, NDK r27c, JDK 21, API 28 to 36. + - Cross-compile the C libraries. + - Build the app as `libTony_arm64-v8a.so` and package it with androiddeployqt, by route + (a) or (b) above. + - Upload the APK as an artifact. +2. **No audio**: `AUDIO_NONE`, by `#ifdef Q_OS_ANDROID` for the test, or `--no-audio` + through the manifest's `android.app.arguments` metadata (check Qt's manifest + template; this was not verified). +3. **pYIN** found through one of the two fixes above; check the log line "Setting VAMP_PATH + to ...". +4. **On a phone**, with a WAV pushed into app storage with `adb`: + - open it, and see pYIN run and the pitch track drawn; + - pan with one finger, drag a selection in the ruler strip, try menus and dialogs by + touch. + +If that works, the rest is known work, in this order: +- the audio backend: playback, then recording, then checklist items 1 to 3 on the phone; +- sharing: "All files access" and a sync app, then "Open reference audio from..." through + the picker; +- the compact touch mode and gestures; +- lifecycle and permissions; +- a session bundle, if whole sessions are to travel through the picker. + +## Unconfirmed + +- Whether Google Drive, Dropbox and OneDrive support creating files and picking folders. +- Whether only `lib*.so` files are packaged. +- Whether Qt 6.12 has shipped. +- Google Play's deadline for 16 KB page alignment: 1 Nov 2025 or 1 Feb 2027; sources + disagree. + +## Sources + +- Qt: + - https://raw.githubusercontent.com/qt/qtdoc/dev/doc/src/platforms/android/android.qdoc + - https://raw.githubusercontent.com/qt/qtdoc/dev/doc/src/platforms/android/android-platform-notes.qdoc + - https://raw.githubusercontent.com/qt/qtdoc/dev/doc/src/platforms/android/android-deploying-application.qdoc + - https://raw.githubusercontent.com/qt/qtbase/6.11/doc/global/macros.qdocconf + - https://raw.githubusercontent.com/qt/qtbase/dev/src/plugins/platforms/android/qandroidplatformtheme.cpp + - https://raw.githubusercontent.com/qt/qtbase/dev/src/plugins/platforms/android/qandroidplatformfiledialoghelper.cpp + - https://raw.githubusercontent.com/qt/qtbase/dev/src/plugins/platforms/android/androidcontentfileengine.cpp + - https://raw.githubusercontent.com/qt/qtbase/dev/src/corelib/kernel/qpermissions.cpp + - https://raw.githubusercontent.com/qt/qtbase/dev/src/android/templates/AndroidManifest.xml + - https://raw.githubusercontent.com/qt/qtbase/dev/src/android/templates/build.gradle + - https://github.com/qt/qtmultimedia/tree/6.10/src/multimedia/android +- Build: + - https://github.com/DarkStarSystems/pcons + - https://github.com/mesonbuild/meson/issues/13018 + - https://github.com/mesonbuild/meson/issues/6089 + - https://docs.mesa3d.org/android.html + - https://developer.android.com/guide/practices/page-sizes + - https://github.com/libsndfile/libsndfile/blob/master/Building-for-Android.md +- Audio: + - https://developer.android.com/ndk/guides/audio + - https://developer.android.com/ndk/guides/audio/audio-latency + - https://developer.android.com/games/sdk/oboe/low-latency-audio + - https://github.com/google/oboe/releases + - https://github.com/PortAudio/portaudio/pull/1084 + - https://github.com/NetResultsIT/portaudio-oboe +- Files and sync: + - https://github.com/Kunzisoft/KeePassDX/issues/1594 + - https://github.com/Kunzisoft/KeePassDX/issues/1487 + - https://github.com/OneDrive/onedrive-api-docs/issues/1134 + - https://github.com/PhilippC/keepass2android/wiki/Keepass2Android-file-handling + - https://github.com/researchxxl/syncthing-android/releases + - https://foldersync.io/ + - https://metactrl.com/ + - https://developer.android.com/training/data-storage/manage-all-files +- Vamp plugins: + - https://github.com/recifra/cordova-plugin-chordino + - https://github.com/vamp-plugins/jvamp diff --git a/docs/port-sailfish.md b/docs/port-sailfish.md new file mode 100644 index 00000000..5b3f3b4b --- /dev/null +++ b/docs/port-sailfish.md @@ -0,0 +1,308 @@ +# Sailfish OS port: findings and first steps + +Researched 2026-09-25; nothing built. Read [mobile-port.md](mobile-port.md) first: it has +the decisions, the facts about Tony's own code that any port depends on, and the work +common to both platforms. + +The research proxy blocked forum.sailfishos.org, docs.sailfishos.org, jolla.com, +openrepos.net and build.sailfishos.org. The facts below come from Sailfish's source +repositories on GitHub, including `sailfishos/docs.sailfishos.org`, the documentation's +source. *(snippet)* marks a fact seen only in search results. + +## Platform facts + +### The phones + +- **Jolla Phone (2026)** *(specs, dates and prices snippet only)*: + - Announced 2025-12-05 as a community pre-order; deliveries began 2026-07-08, and an + October 2026 batch was on sale. + - MediaTek Dimensity 7100 (4× Cortex-A78, 4× A55, Mali-G610 MC2), so aarch64 + (inferred). + - 8 or 12 GB RAM, 256 GB storage, microSD; 6.36" AMOLED at 1080×2260. + - **No 3.5 mm jack**, USB-C only. A forum thread lists USB-C adapters and DACs known to + work, and an official USB-C headset was announced for August 2026. + - A community analysis of a MediaTek device (possibly this one) says USB headsets go + through the Android USB audio HAL, and the headset mic works everywhere except in calls + (`github.com/smatkovi/usb-headset-call-mic`). + - Runs **Sailfish OS 5.2 "Finlayson"**, a Jolla Phone-only branch (5.2.0.11 to .17). +- **Jolla C2 (2024)**: Unisoc T606, 8 GB, 720×1600, **has a 3.5 mm jack**. Android App + Support 13 (API 33). + +### Sailfish OS and Qt + +- **Releases:** 5.0 "Tampella" (2024-2025), 5.1 "Pispala" (June 2026 *(snippet)*), 5.2 for + the Jolla Phone only (July 2026). There is no 6.x. +- **The system Qt is 5.6.** + - `sailfishos/qtbase` is 5.6.3 on branch `mer-5.6`, and the docs say "Sailfish currently + uses Qt version 5.6". + - No official plan for Qt 5.15 or 6, and no Silica (Sailfish's own QML components) for + Qt 6, was found. + - Tony cannot use Qt 5.6 (see [mobile-port.md](mobile-port.md#qt-6-only)). +- **Community Qt 6 from Chum** (Sailfish's community repository, built on the Sailfish + OBS): + - Qt 6.8.4 LTS, updated in July 2026, published to `sailfishos:chum` and + `sailfishos:chum:testing`. + - Installed system-wide: `/usr/lib64`, plugins in `/usr/lib64/qt6/plugins`. + - `qt6-qtbase-gui` includes `libQt6Widgets`. It is Wayland only, with no XCB. + - Also packaged: qtwayland, qtmultimedia, qtsvg, qtdeclarative, qtwebengine, and a + Maliit on-screen keyboard plugin (`qt6-sfos-maliit-platforminputcontext`). + - Apps using it: NeoChat, Angelfish, Kirigami gallery. All are QML or Kirigami; none is a + Qt Widgets desktop app. + - Community Qt 5.15 ("opt-qt5", under `/opt/qt5`) also exists *(snippet)*. It would need + the backport described in mobile-port.md, and is not a good target. +- **How a Qt 6 app is shown on screen:** + - **Lipstick**, Sailfish's compositor, gained xdg-shell support in April 2026. It + implements "only the basic parts needed to show maximized toplevel windows and position + popups". + - NeoChat (in Chum since July 2026) is launched through `qt6-start.sh`, which only sets + the environment and runs the program: + - `QT_WAYLAND_DISABLE_WINDOWDECORATION=1`; + - `QT_WAYLAND_FORCE_DPI=260`, which users can override. + - The older route, `qt-runner`, is a nested compositor that supports wl-shell only: one + window, no dialogs. Angelfish still uses it. + - Whether the xdg-shell support shipped in 5.1 or first in 5.2 is unconfirmed. NeoChat's + direct launch suggests current releases have it. +- **What that means for Tony's window:** + - Every top-level window, including every dialog (about 88 call sites in `MainWindow`), + will be shown maximised with no title bar. + - Menus and pop-ups are positioned. + - A developer reported that xdg-shell reports scale 1.0 on the Jolla Phone *(snippet)*. + - Conflicts between Sailfish's edge swipes (system navigation) and panning in the pane + have not been reported, nor tested. +- **Qt Widgets and the Jolla Store (Harbour):** + - `libQt5Widgets` is not on Harbour's allowed-library list, and the Harbour FAQ calls + Widgets not touch-optimised and software-rendered *(snippet)*. + - A bundled Qt 6 may link only whitelisted system libraries. freetype, harfbuzz, ICU and + xkbcommon are not on that list, so they would have to be bundled too. + - No precedent was found. **Tony would be distributed as a sideloaded RPM or through + Chum, not the store.** + +### Audio + +- **PulseAudio 17.0**, not PipeWire, over the Android audio HAL through + `mer-hybris/pulseaudio-modules-droid` (libhybris), actively maintained. There is no + official PipeWire plan. +- **bqaudioio's existing PulseAudio backend should work unchanged.** It is compiled into + Tony's Linux build already. What it does is described in + [mobile-port.md](mobile-port.md#audio-io). +- It asks PulseAudio for 44.1 kHz, the rate Tony's models run at, and PulseAudio resamples + to the hardware. The sample-rate question in + [mobile-port.md](mobile-port.md#sample-rate) is therefore probably moot here. +- **Routing** (`xpolicy.conf` in `mer-hybris/droid-hal-configs`): + - Ordinary media and app streams go to the **"media_latency" sink**. That is the + deep-buffer output when the HAL has one, otherwise the primary. + - The droid card's low-latency ("FAST") sink is used only for keyboard feedback, voice + UI, VoIP and calls. + - No documented way for an app to request it was found. +- **Reported latency is an estimate.** + - The droid sink sets a fixed latency equal to the HAL's `get_latency()`; the source + reports one HAL buffer (`droid-sink.c`, `droid-source.c`). + - `pa_stream_get_latency()` therefore returns the HAL's figure, not a measurement. + - No published round-trip figures for any Sailfish device were found. Expect to + calibrate on the device. + - Possible levers, all untried: + - buffer attributes and `PA_STREAM_ADJUST_LATENCY` in a bqaudioio fork; + - a stream role that the policy routes to the low-latency sink; + - Tony's missing latency calibration setting. +- **Microphone:** the Sailjail `Microphone` permission includes `Audio` ("playback and + record streams cannot be separated on pulseaudio"). `libpulse` is on Harbour's allowed + list. +- **Existing play-and-record apps:** SailTuner (`LouJo/SailTuner`, `pa_simple`, 16 kHz, + last committed 2021) and Sounder (a tuner). No multitrack recorder or synth was found. + +### Sandboxing (Sailjail) + +- Firejail-based; apps have been sandboxed by default since Sailfish 4.4. +- Permissions are declared in the `.desktop` file: + + ``` + [X-Sailjail] + Permissions=Audio;Microphone;Documents;Music + OrganizationName=org.foo + ApplicationName=Tony + ``` + +- Each app may write `~/.local/share`, `~/.cache` and `~/.config` under + `/`. +- File paths each permission opens: + - `Documents`: `~/Documents` and `~/android_storage/Documents`. + - `Music`: `~/Music`, `~/Playlists`, `~/android_storage/Music` and + `~/android_storage/Podcasts`. + - `Downloads`: `~/Downloads` and `~/android_storage/Download`. + - `UserDirs`: all of these plus Pictures, Videos and Public. + - `RemovableMedia`: memory cards and USB storage. +- An app with no `[X-Sailjail]` section gets a default profile that includes Audio, + Microphone, Internet and UserDirs. +- **`Sandboxing=Disabled` works for a sideloaded RPM** (not allowed in the store). NeoChat, + Angelfish and Ferry Sync ship with it. +- Tony's QSettings, record directory and temporary files then land in ordinary Linux + locations. + +### Build tooling and libraries + +- **Sailfish SDK 3.13.5** (July 2026): + - Build targets for 5.1.0 on aarch64, armv7hl and i486. + - GCC 13.4.0, so C++17 is fine. + - The build engine runs in Docker or VirtualBox, on Linux and Windows; macOS has + VirtualBox only. The command-line tool is `sfdk`. +- **Meson works inside an RPM spec.** + - The IDE knows qmake and CMake only, but any build system can run from a hand-written + spec `%build` section (docs: "Building packages – advanced techniques"). + - Meson 1.11.1 is packaged in Sailfish, and platform packages such as PulseAudio build + with it. + - The build engine compiles in an emulated target, not by cross-compiling, so the + meson-plus-Qt cross-compiling problems on the Android page should not arise (not + tried). +- **Libraries:** + - In Sailfish's own repositories: libsndfile 1.2.2, pulseaudio 17.0 (with -devel), boost + 1.81, opus 1.6.1, libvorbis, libogg, flac, mpg123. + - **Not found** in the sailfishos or sailfishos-chum organisations: libsamplerate, fftw3, + Rubber Band, opusfile, serd/sord, libmad (id3tag was not checked). Build them in Tony's + spec, or package them for Chum. PulseAudio itself is built with fftw disabled. +- **Changes to `meson.build`:** a Sailfish branch, or options on the Linux branch, that + drop JACK, ALSA (and RtMidi's ALSA defines), oggz and fishsound. +- **pYIN:** the plugin loads unchanged. Install it where `setupTonyVampPath()` looks + (`/../lib/tony`, that is `/usr/lib/tony` even on aarch64, where libraries are in + `/usr/lib64`), or set `TONY_VAMP_PATH` in the launcher. +- **Installing on the phone:** + - Settings > System > Untrusted software > Allow untrusted software, then tap the RPM in + Transfers or the File manager. + - Or, in developer mode, `devel-su` and `rpm -i`. + - Chum's Qt 6 packages must be installed first; Chum has its own installer app. + +### Files and sharing + +- **Native apps use ordinary paths**, gated by the Sailjail permissions above. The + existing file code, and the session layout described in + [mobile-port.md](mobile-port.md#files-and-sessions), work as they are. +- **Built-in accounts do not sync folders.** + - Dropbox and OneDrive are for backup and "share to" upload only. + - Nextcloud covers gallery images, backup, calendar and contacts. + - Google covers contacts, calendar and mail. + - There is no system picker that reaches cloud storage. +- **Sync options:** + - harbour-Syncthing (`ilpianista/harbour-Syncthing`, bundles Syncthing 2.1.3, 2026-08-26, + on OpenRepos): phone to PC directly, no cloud. The simplest pairing with a Windows + desktop running Syncthing. + - rclone: the official aarch64 RPM works; it is also on OpenRepos. It reaches Google + Drive, Dropbox and OneDrive, but is configured on the command line; a sync would run + from a systemd timer. + - Ferry Sync (`Dominik-h-hub/harbour-ferry`): an rclone front end for Nextcloud, WebDAV, + Seafile, pCloud, SFTP and FTP. + - GhostCloud, and a Chum package of the Nextcloud desktop client. +- **Android sync apps through App Support** (FolderSync, Dropsync) write under + `~/android_storage`. The `Music` and `Documents` permissions include + `~/android_storage/Music` and `~/android_storage/Documents`, so a native Tony should be + able to read what they sync there. This is inferred, not tested, and file ownership is + unverified. + +### Android App Support + +- An LXC container running AOSP: Android 13 (API 33) on the C2 and the Xperia 10 IV and V. + Assumed, not confirmed, for the Jolla Phone. +- Android audio goes through the host's PulseAudio; audio passthrough improved in 5.1 + *(snippet)*. +- No latency measurements, and no statement about AAudio low-latency or MMAP paths, were + found. Expect it to be no faster than native. +- **An APK from the Android port would run here.** It gives Jolla users a fallback without a + native port, with unknown audio quality. + +### Ecosystem + +- In November 2023 Jolla's business moved to Jollyboys Ltd, owned by the former + management, through a court-approved restructuring. +- Releases roughly yearly with frequent point releases. Community News appears every two + weeks through 2026. +- Chum has about 312 package repositories. Lipstick and the droid audio modules had commits + in September 2026. +- Small, but active. + +## What has to be built + +| Item | Size | Notes | +| --- | --- | --- | +| RPM spec with meson, the six missing libraries, and a Sailfish branch in `meson.build` | Medium | Build engine on the Windows machine through Docker, or in CI | +| `.desktop` file with `[X-Sailjail]`, and a launcher that sets the Qt 6 environment | Small | Model: `sailfishos-chum/neochat` and `qt6-sailfishos-util` | +| Qt 6 on 6.8.4 | Unknown | Tony has only been built with 6.11 | +| Audio | None to medium | The backend exists; latency may need buffer settings (a bqaudioio fork) or calibration | +| Dialogs that suit a maximised, undecorated window | Unknown | Depends on how the test port looks | +| Compact touch mode and gestures | Medium to large | Common to both ports, plus the edge-swipe check | +| Sample-rate check, latency calibration | Small each | Common to both ports | + +## Test port + +This step decides whether to go on. Its two questions cannot be answered from Tony's +side: whether the windows are usable, and whether the latency is usable and correctly +reported. + +1. **RPM spec** building with meson in the Sailfish SDK for the aarch64 5.1 target: + - `BuildRequires` on Chum's Qt 6 packages and on the libraries in the Sailfish + repositories; + - the six missing libraries built in the spec, or as sub-packages. +2. **Install** on the phone after Chum's Qt 6. + - `Sandboxing=Disabled` for now. + - Launch with the same environment as `qt6-start.sh`. +3. **Without audio first** (`--no-audio` on the launcher's command line): + - open a WAV from `~/Music`, analyse it, see the pitch track; + - pan with one finger, drag a selection in the ruler strip; + - open the menus, trigger a message box and the preferences dialog; + - check the scaling. +4. **With audio**: + - play the reference; + - do the clap test (manual checklist item 1) with a USB-C headset on the Jolla Phone, + or wired on the C2; + - read what latency PulseAudio reports (bqaudioio logs it: "playback latency = ... usec", + "record latency = ...") and compare it with where the claps land. + +If the windows are unusable, stop; nothing in Tony fixes lipstick. If only the latency is +wrong, the fix is buffer settings or the calibration setting. + +## Unconfirmed + +- The Jolla Phone's specs, dates and prices (search snippets only), and which Android level + its App Support runs. +- Whether lipstick's xdg-shell support is in 5.1 or only 5.2. +- Whether a third-party app can get the low-latency sink, and what the real latency is. +- That libsamplerate, fftw3, Rubber Band, opusfile, serd/sord and libmad are absent from + Chum. Only GitHub was searched; Chum packages can live in any repository. +- Whether files that Android apps write under `~/android_storage` are readable by native + apps. + +## Sources + +- Documentation source (docs.sailfishos.org), `github.com/sailfishos/docs.sailfishos.org`: + - `Reference/Qt/README.md` (Qt 5.6) + - `Support/Supported_Devices/README.md` + - `Support/Releases/README.md` + - `Support/Help_Articles/Android_App_Support/README.md` + - `Support/Help_Articles/Backup/Backup_and_Restore/README.md` + - `Tools/Sailfish_SDK/` + - `Develop/Apps/Tutorials/Building_packages_-_advanced_techniques/README.md` +- Qt: + - https://github.com/sailfishos/qtbase + - https://github.com/sailfishos-chum/qt6 + - https://github.com/sailfishos-chum/qt6-qtbase + - https://github.com/sailfishos-chum/qt6-sailfishos-util (`qt6-start-env.sh`) + - https://github.com/sailfishos-chum/neochat (`rpm/neochat.spec`) + - https://github.com/rinigus/qt-runner + - https://github.com/sailfishos/lipstick/tree/master/src/compositor/xdgshell + - https://github.com/sailfishos/sdk-harbour-rpmvalidator/blob/master/allowed_libraries.conf +- Audio: + - https://github.com/sailfishos/pulseaudio + - https://github.com/mer-hybris/pulseaudio-modules-droid + - https://github.com/mer-hybris/droid-hal-configs/blob/master/sparse/etc/pulse/xpolicy.conf + - https://github.com/sailfishos/sailjail-permissions + - https://github.com/LouJo/SailTuner +- Build and packages: + - https://github.com/sailfishos/meson + - https://github.com/sailfishos/libsndfile + - https://github.com/sailfishos-chum/main +- Sync: + - https://github.com/ilpianista/harbour-Syncthing + - https://github.com/Dominik-h-hub/harbour-ferry + - https://github.com/sailfishos-chum/nextcloud-client +- Forum *(snippets only)*: + - https://forum.sailfishos.org/t/release-notes-finlayson-5-2-0-15-jolla-phone-only/30793 + - https://forum.sailfishos.org/t/usb-c-to-3-5mm-jack-adapters-known-to-work-with-sailfish-os/30685 + - https://forum.sailfishos.org/t/lipstick-questions/31318 + - https://forum.sailfishos.org/t/guide-setup-mount-webdav-resource-with-rclone-on-sailfishos/14518 From be790f38d8bc9ab62eaee7af6f25ba4413aef88e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 21:08:17 +0000 Subject: [PATCH 069/275] docs: timed lyrics What the lyrics are made of and why (architecture), the lyrics plot style of the svgui fork and why it stays scrollable (forks), the new suites, the test data they read and the tests expected to fail when built on Linux with Qt 6.4 (testing), what is not built and what is weak (open points), what only a person can judge (manual checklist, items 37-48), and how to import lyrics and get them out of Moises (README). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- README.md | 13 +++++++++ docs/architecture.md | 60 ++++++++++++++++++++++++++++++---------- docs/forks.md | 11 ++++++++ docs/manual-checklist.md | 33 ++++++++++++++++++++++ docs/open-points.md | 23 +++++++++++++++ docs/testing.md | 35 +++++++++++++++++++---- 6 files changed, 154 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 099d7680..a7792582 100644 --- a/README.md +++ b/README.md @@ -47,6 +47,19 @@ orange pitch track to compare with it. the toolbar): each keeps its own audio, pitch track and notes, and switching between them needs no re-analysis. The audio of a session's takes is kept in a folder named after the session beside it, so the two can be moved together + * timed lyrics: File -> Import Lyrics... reads an LRC file, timed by line or + by word, and shows the words along the top of the pane, each over a bar for + as long as it is sung. They are saved with the session; View -> Show Lyrics + hides them and File -> Remove Lyrics takes them out. The reference must be + the recording the lyrics were timed to (a Moises stem and its original mix + share a timeline); to change a word or its time, edit the file and import it + again + * LRC files can be exported from Moises with the Moises-Lyric-Exporter browser + extension. Set its offset to 0 (otherwise every line is 0.2 s early, and + Tony cannot tell) and its gap threshold as low as it goes (so that it marks + where lines end). It is unofficial and not made by Moises: install it + unpacked from a commit that has been reviewed, and do not update it without + reviewing the change Authors, Citation, License and Use diff --git a/docs/architecture.md b/docs/architecture.md index fa0ba02c..a9b6c6e7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -14,8 +14,9 @@ Upstream Tony analyses the pitch of one recording. This fork makes it a singing 4. **Partial recordings and takes**: record from the playhead into part of the song, keep the rest, erase, undo, and keep several takes. See [takes.md](takes.md). 5. Around that: play the reference while recording, latency compensation, pre-roll, - record into selection, an octave-shifted "alternate" pitch track to follow, and a - background music track that is played but never analysed. + record into selection, an octave-shifted "alternate" pitch track to follow, timed + lyrics along the top of the pane, and a background music track that is played but + never analysed. The user-facing description is in the [README](../README.md). @@ -30,8 +31,8 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h` | -| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `TakeCommands`, `TakeLayers`, `PaneUtils` | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LatencyUtils.h` | +| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `TakeCommands`, `TakeLayers`, `PaneUtils` | When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` list **only if** it has `Q_OBJECT`. Logic that can be written as pure functions or a plain @@ -52,9 +53,10 @@ follow. `MainWindow` then only fills the struct in and puts the answer on screen inactive takes have their source model cleared (see [takes.md](takes.md)). - `Analyser::fileClosed()` clears the layers but **not** `m_fileModel`. - Helper objects that own one layer each and are only wired by `MainWindow`: - `AlternatePitchTrack`, `CoverageStrip`. Both watch `Document::layerAboutToBeDeleted` - in case someone else deletes their layer, and both must be deleted in `~MainWindow` - **before** the base class deletes the document (as must `m_analyser2`). + `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`. All three watch + `Document::layerAboutToBeDeleted` in case someone else deletes their layer, and all + three must be deleted in `~MainWindow` **before** the base class deletes the document + (as must `m_analyser2`). - `RealtimePitchTracker` is a `QThread` that only **reads** the recording's `WritableWaveFileModel` and emits `pitchDetected(frame, hz)`. It never touches the pitch model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued @@ -96,10 +98,11 @@ These were all learned from crashes or wrong behaviour. They hold for any new co ### Tony's own layers make no undo commands Every layer Tony makes for itself — analysers' layers, live dots, the recording's hidden -waveform, the alternate pitch track, the coverage strip, background music — is added with -**`Document::attachLayerToView()`** (svapp fork): in the view and in the layer-view map, so -the session keeps it, but no command and no modified flag. `addLayerToView()` (the -undoable Add Layer) must not be used for these: Undo after a take has to find the take. +waveform, the alternate pitch track, the coverage strip, the lyrics, background music — is +added with **`Document::attachLayerToView()`** (svapp fork): in the view and in the +layer-view map, so the session keeps it, but no command and no modified flag. +`addLayerToView()` (the undoable Add Layer) must not be used for these: Undo after a take +has to find the take. Whoever attaches the layer calls `documentModified()` if the change should count. ### Commands @@ -118,7 +121,9 @@ Whoever attaches the layer calls `documentModified()` if the change should count - The play source takes in the model of **every layer in a view**, playable or not, and what it holds decides where playback ends. Models that must not extend playback - (coverage strip, layers of inactive takes) are taken out with `m_playSource->removeModel()`. + (coverage strip, lyrics, layers of inactive takes) are taken out with + `m_playSource->removeModel()`. A session load adds each layer to its view and so puts + their models back in: take them out after a load too. - The svapp fork emits `Document::modelAboutToBeReleased(ModelId)` and `MainWindowBase` removes the model from the play source on it. Upstream only did so from `RemoveLayerCommand`, which forced deletes never run. @@ -141,8 +146,10 @@ Whoever attaches the layer calls `documentModified()` if the change should count - Connect with **member pointers**, not `SIGNAL()`/`SLOT()` strings, for anything whose signature has `sv::` types when the receiving class is outside namespace `sv`: the - string form never matches and fails silently at run time (this kept - `Analyser::layerAboutToBeDeleted` from ever being called). + string form need not match, and then fails silently at run time (this kept + `Analyser::layerAboutToBeDeleted` from ever being called). Whether it matches can + depend on the Qt version: Qt 6.4 does not match `ModelId` in a string against a slot + moc recorded as taking `sv::ModelId`, where the Qt used for development does. - `audioFileLoaded()` is emitted for `CreateAdditionalModel` too (singing track, background music). `analyseNewMainModel()` returns early if the main model is the one it already analysed (`m_analysedMainModelId`); handing the reference to `m_analyser` twice forgets @@ -162,7 +169,8 @@ Whoever attaches the layer calls `documentModified()` if the change should count - An event's value is written with six significant figures. After a round trip compare frames exactly and values with a tolerance. - Layer **object names carry identity** across a save: `"Alternate Pitch Track -1"` holds - the octave count, `"Take 2 Pitch"` links a layer to its take. They are not translated. + the octave count, `"Take 2 Pitch"` links a layer to its take, `"Lyrics"` marks the + lyrics. They are not translated. ### Miscellaneous @@ -195,3 +203,25 @@ saved in the session. after `openPath()`, and only then prune the extra pane — the imported waveform in that pane is the only reference to the model until `m_analyser2` has a layer of its own. It ends with `clearTakeHistory()`, which also disposes of the "Import" command for the pruned pane. + +**Lyrics** (`Lyrics` parses the LRC file, `LyricsTrack` owns the layer): one `RegionModel` +in pane 0 on the reference's timeline, a region per word (or per line, for a line with no +word times): frame = start, duration = end - start (at least one frame), label = the word, +value = the line's index, which is what the bold line starts go by. It is drawn by the +svgui fork's `PlotLyrics` style ([forks.md](forks.md)), because a session restores only +layers `LayerFactory` can make. Found again after a session load by its untranslated object +name `"Lyrics"`, in `analyseNewMainModel()` after the alternate pitch track. Its model is +taken out of the play source after an import and again after a load: a word past the end +of the reference would hold playback open. It is **never the pane's top layer**, because +the pane takes its hover readout and vertical scale from the top layer and this one has +neither. `show()` raises the layer that was on top before; `adopt()` raises the one under +the lyrics if the session was saved with them on top; and when any other layer is deleted +(the alternate pitch track turned off, a take deleted) a zero-time timer does the same, +because that layer is still in the pane when `layerAboutToBeDeleted` arrives. Not tied to +takes: the take code finds layers by take name, source model or extra pane, so it never +finds this one, and it stays on show during a take, when the singer needs the words most. +Import and Remove push no command and leave the undo history alone, like Load Background +Music: the simpler option, and the file is still there to import again. Show Lyrics is the layer's own +visibility, which the session saves, not a QSettings key. The parser strips control +characters (and U+FFFE, U+FFFF, which the UTF-8 decoder lets through) from every label: +XML 1.0 cannot hold them, and one in a label would make the `.ton` unreadable. diff --git a/docs/forks.md b/docs/forks.md index cc25196c..0b99deed 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -72,6 +72,17 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` away from the dots. - `RegionLayer::PlotStrip` plot style: the coverage strip. Saved through the existing `plotStyle` attribute. +- `RegionLayer::PlotLyrics` plot style, after `PlotStrip` so saved numbers keep their + meaning: the lyrics. Each region's label along the top of the view over a bar as long as + the region, in two rows, the first word of a line (where the value changes) in bold; no + vertical scale, no feature description, not editable. The static, pure + `assignLabelRows()` places the labels (Tony's app suite tests it): a label that fits in + no row is left out. Where a label goes depends on the labels before it, so the layout is + made for the **whole model** at once and cached per zoom level and font; a strip newly + scrolled into sight then agrees with what is already on show. That is what lets the + layer stay **scrollable**: `View::getNonScrollableFrontLayers()` treats every layer in + front of a non-scrollable one as non-scrollable too, so the pitch tracks above the + lyrics would repaint on every cursor update. - `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the notes of a take that is put away. - `Pane::setWorkModel()` / `getWorkModel()`: which model's extents are blocked off at the diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 9fa49878..35b7ee7d 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -104,3 +104,36 @@ Launch with `.\build.bat run`. 36. Alternate pitch track: faded brown is readable but secondary; dark brown during a take is distinct from black; `8vb` / `8va` buttons look acceptable; the track stays in view after an octave step. + +## Lyrics + +37. **Legibility**: the words are readable over the reference and singing pitch tracks, + the alternate pitch track and the live dots, and the pitch shows through between them. + Grey bars: right colour? +38. **Band height and font** at the zoom used while singing: can the words be read while + singing, and does the band hide too much of the top of the pitch range? +39. **Density**: zoom out until words drop out (they are left out, never drawn over each + other) and back in (they return). At the usual zoom on a fast song, how many drop out: + are two rows enough? +40. **Bold line starts**: do they read as the start of a phrase, or as noise? +41. **The left edge**: a word in the first ~30 px of the view (at 0 s, with the view at the + start) is under the pane's vertical scale. How much does that matter in use? +42. **Inferred ends**: the exporter writes no end times. With word timing the last word of + a line ends at the next line but at most 2 s after it starts, unless a `♪` line marks + the end; with line timing a line lasts until the next one, and the last line 5 s. Do + those bars mislead? The start times are exact. +43. **Hover readout**: with the lyrics shown, hovering over the pitch tracks gives the same + readout and the same vertical scale as without them, also after turning the alternate + pitch track off and after deleting a take. +44. **A real Moises export** of one of your songs (exporter offset 0, gap threshold low), + imported onto the Moises stem or the original mix: the words line up with the vocal, by + eye and while playing. All early or late by the same amount means the reference is not + the recording Moises timed; an `[offset:]` line in the file moves them. +45. **Finnish text**: ä and ö come out right in the pane, and again after save and reopen. +46. **Show Lyrics and Remove Lyrics**: hiding keeps the words for later, Remove takes them + out, a second import replaces the first; each makes Close ask whether to save. The + status bar after an import counts words and lines and names anything skipped. +47. **Session**: save and reopen: the same words, hidden or shown as saved; playback still + ends at the end of the song, even with words past it. +48. **During a take**: the words stay on show and readable while recording, with the + countdown and the live dots; Import Lyrics is greyed out while recording. diff --git a/docs/open-points.md b/docs/open-points.md index 7776f247..39befd18 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -13,6 +13,8 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - **Constrain Playback to Selection + pre-roll**: the play source constrains playback to the selection, the lead-in is outside it, so it is cut short. Nothing keeps the two apart. - **Take operations clear the undo history with no prompt** (all but Rename). +- **A shortcut for Show Lyrics?** There is none; one would have to be checked against + `KeyReference` for clashes first. - None of the [manual checklist](manual-checklist.md) has been run. ## Not built @@ -22,6 +24,18 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - Background music is not saved in the session; it is reloaded by hand. - An old session (before takes) loses its singing track without telling the user why. - Recording that starts before frame 0 of the reference. +- **Lyrics cannot be shifted or edited in Tony.** The remedy is to edit the LRC file and + import it again; its `[offset:]` tag is the only shift. That includes lyrics that are + all off by the same amount because the reference is not the recording they were timed + to. Import and Remove are not undoable, and a new import replaces the lyrics without + asking. +- **No highlight of the word being sung**: the playback cursor crosses the words. A + highlight would tie the lyrics layer's painting to the play position, which defeats the + view's paint cache. +- **LRC only**: no SRT, TTML or Moises JSON. The exporter's TTML carries real word (and + syllable) end times where its LRC has none, so it is the natural second format if the + inferred ends turn out misleading; another format is another function beside + `parseLrc()`. ## Weak spots @@ -38,6 +52,15 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. `RecordCreateUnshownModel`; it can go once that has proved itself. - `Coverage::regionLabel()` gives every region a blank label, for a stock `RegionLayer` that printed the value otherwise. With `PlotStrip` it is no longer needed. +- **A word in the first ~30 px of the view is hidden** under the pane's vertical scale, + which the pane draws over the left edge for its top layer. Seen with a word at 0 s and + the view at the start. +- **The lyrics' layout is a guess at what reads well**: two rows (a word with no room is + left out), the bold line starts, and the 2 s / 5 s caps on inferred ends are all + constants to be judged by eye ([manual checklist](manual-checklist.md)). +- **Right after `closeSession()`, Show Lyrics and the alternate pitch actions keep their + enabled and checked states** until the next reference or session opens: nothing there + calls `updateLayerStatuses()`. Show Lyrics then does nothing when chosen. - Untested by any suite: removal of dots placed before the latency was measured; the deferred and error paths of the dot teardown; `ContinuousSynth` deletion in the svapp fork; the 30 s give-up of `waitForRangedAnalysis()`; `commitData()` relocating takes; diff --git a/docs/testing.md b/docs/testing.md index 0de32575..2b81e9b9 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,8 +5,8 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow` | about 4.5 minutes (measured 2026-09-20), nearly all of it `TestRecordWorkflow`: takes are recorded in real time | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics` | seconds | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow` | about 4.5 minutes (measured 2026-09-20), nearly all of it `TestRecordWorkflow`: takes are recorded in real time | `meson test` / `build.bat test` runs both plus four svcore suites. @@ -33,6 +33,10 @@ suite needs nothing but itself (a private slot). **Every private slot runs as a helpers must not be slots; connect to lambdas instead. For access to private statics use `friend class TestX;`, as `RealtimePitchTracker.h` does. +Suites find the files in `testdata/` through `TONY_TEST_DATA_DIR`, which `meson.build` +defines for both test executables as a path with forward slashes: the backslash of a +Windows path would start an escape in the C string. + ## Running - `RunSuite.h` writes each suite's results to `$TONY_TEST_LOG_DIR/.txt`. @@ -43,6 +47,14 @@ helpers must not be slots; connect to lambdas instead. For access to private sta the exit status of a run with names is always 1. Only a run with no names has a meaningful exit status. - `QT_QPA_PLATFORM=offscreen` is set by `main()` when not given. +- **Built on Linux with Qt 6.4**, some tests are expected to fail, whatever the change. + Core: `TestTakesFile`'s `takes_folder`, `relative_audio_path`, `resolve_audio_path` and + `in_folder`, which test Windows paths (`C:\...`, case-insensitive). App, all + `TestRecordWorkflow`: `stale_pitch_event_ignored`, whose string-based `invokeMethod` + with an `sv::` type Qt 6.4 cannot match; and `take_analysis_covers_the_range_it_lost`, + `range_analysis_torn_down_while_running`, `save_during_ranged_analysis` and + `undo_during_analysis_then_redo`, where the analysis finishes before the race they need + can be set up. Which of those four fail changes from run to run. ## Design principles @@ -57,6 +69,12 @@ helpers must not be slots; connect to lambdas instead. For access to private sta - **Ranged analysis is judged against a whole-file analysis of the same audio**, over the whole file, not just around the range (`ranged_leaves_the_rest_alone`): that is what caught the merge damaging unchanged audio half a second away. +- **A layer painted in strips must equal the layer painted whole** + (`painting_in_strips_matches_painting_whole`, `TestLyricsLayer`). A view that scrolls + repaints only the strip that comes into sight, so anything a layer lays out from its + neighbours must not depend on the rect being painted. The test paints into an image once + whole and once strip by strip, and compares the pixels; a layout worked out from the + painted rect fails it. ## What is there to reuse (`TestRecordWorkflow.h`) @@ -68,13 +86,14 @@ helpers must not be slots; connect to lambdas instead. For access to private sta - `TestMainWindow`: subclass of `MainWindow` that exposes protected operations as `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the fake device through `createAudioIO()`, and **answers dialogs through virtual seams**: - `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, each with a - `set...Answer()` and a counter of questions asked. A test cannot answer a real dialog: - anything new that asks the user needs such a virtual. + `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, + `askForLyricsFile()`, each with a `set...Answer()` and a counter of questions asked. A + test cannot answer a real dialog: anything new that asks the user needs such a virtual. - Fixture helpers: `makeWindow(config)`, `writeWav()`, `openReference()`, `startTake()` / `stopTake()` / `take(ms)`, `verifyPlaySourceClean()`, `layersOnModel()`, `paneHasLayer()`, `documentHasLayer()`, `reopenAsSession()` / `reopenSession()`, - `verifyEventsSurvived()`. + `verifyEventsSurvived()`; for the lyrics `lyricsFixture()`, `writeLrc()`, + `verifyLyricsUntouched()`. - A **dialog watchdog**: a 50 ms timer closes any modal dialog and records it, and `cleanup()` fails the test for one that was not expected. `dialogsMatching()` is for the dialogs a test does expect. @@ -87,6 +106,10 @@ helpers must not be slots; connect to lambdas instead. For access to private sta of its own because the Vamp *plugin* SDK headers must not meet the *host* SDK headers svcore uses. - `testdata/happy_birthday_gp_masked.wav`: a real sung recording. +- `testdata/lyrics/`: LRC files with invented text. Two are in the exact format of the + Moises lyrics exporter (word timing and line timing: no end times, a `♪` gap line, a + word with punctuation glued to the one before, a line its clamp stamped 0); the third is + a generic LRC that does give ends. Prefer signals that describe themselves: `TestTakeAudio` uses constants and ramps so that every sample says where it came from. Assert **identity** as well as equality where the From 23e19af2f4e5e893a0b127b8258dcead66de3e9f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 21:08:48 +0000 Subject: [PATCH 070/275] docs: work orders for the android port's phase agents The phases, decisions and rules that the agents building the android port read. Tracked on the branch because cloud containers are thrown away between sessions; the last phase folds it into the other pages and deletes it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 253 ++++++++++++++++++++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 docs/android-work-orders.md diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md new file mode 100644 index 00000000..a4a69d89 --- /dev/null +++ b/docs/android-work-orders.md @@ -0,0 +1,253 @@ +# Android port — work orders for phase agents + +Working document for the `feat/tonyandroid` branch. It is removed, and whatever is worth +keeping folded into the other pages, by the last phase. It is tracked only because the +cloud containers that run the work are thrown away between sessions. + +You are one of a line of agents, each building **one small phase** of the Android port. A +lead reviews your work when you report back. You have no memory of earlier phases; what +you need is here. + +**Your context is the budget.** Aim to finish well under 200k tokens. The rules below say +how. They are about not reading huge files whole and not maintaining big documents; they +are **not** a licence to skip what you need to understand. Careful, correct work comes +first. + +## 1. What to read (and what not to) + +1. This file, all of it. +2. [AGENTS.md](../AGENTS.md), all of it (short). Its rules hold, except where section 2 + below replaces them for this container (build commands, who commits). +3. The spec is [mobile-port.md](mobile-port.md) plus [port-android.md](port-android.md). + Read the sections your work order names; search for the heading, read that range. +4. The docs page for the area you touch, as the table in AGENTS.md says. Named sections + only, unless your work order says otherwise. +5. Code: `main/MainWindow.cpp` (over 6000 lines) and `main/test/TestRecordWorkflow.h` + (over 2000) are **never read whole**: search, then read the range. The same goes for + `meson.build` (about 1400 lines). Read an existing test next to where yours will go and + copy its shape. + +## 2. Rules + +**Scope** + +- Build your phase only. If something from a later phase is needed, build the smallest + part of it and say so. +- The spec and the decisions in section 3 are agreed. Where they are silent, choose the + simpler option and note it. Where they are **wrong or impossible**, do not improvise + another design: finish what can be finished, leave the tree building and green, report. +- Do not edit the library directories (`svcore/`, `svgui/`, `svapp/`, `bqaudiostream/`, + `bqaudioio/`, `pyin/`, ...). They are separate repositories. If one needs a change, + report exactly what; the lead decides. +- Do not edit `.github/` unless your work order says so. Never push, never touch the + remote, never amend or rebase. +- Match the surrounding code: naming, comment density, idiom, the GPL header. State gets a + class of its own and `MainWindow` only wires it; window-free logic goes in `tony_core`. +- Android-only code sits behind `#ifdef Q_OS_ANDROID` (or in files the Android build alone + compiles) and must not change the desktop build's behaviour. + +**Build and test in this container** (Linux; the lead fills in the exact commands after +phase A0) + +- Not the Windows commands in AGENTS.md: this is an Ubuntu 24.04 container, 4 cores, + 15 GB memory, no swap. ``. +- Send build output to a log file with the exit status written into it; look at the tail + or grep it for errors, never read it whole. +- Tests: read results from the per-suite files in `TONY_TEST_LOG_DIR` (see AGENTS.md). + While working run **only your tests** by name; run **both whole suites once** at the + end, and again only if something failed. The app suite runs in real time (minutes): + give it a 10-minute tool timeout. +- Network: GitHub (git and release downloads), the Ubuntu archive, PyPI and conda-forge + are reachable. `download.qt.io`, `dl.google.com`, `hg.sr.ht` and `breakfastquay.com` + are blocked by the environment's policy: do not look for mirrors of them; report if you + need them. +- Every behaviour gets a test that can fail. Show it for the two or three that matter + most by breaking the code for a moment. Undo the break **by hand**: never + `git checkout`/`git restore` a file to revert an experiment. +- Do not weaken or delete an existing test to get green. If one is wrong because the + behaviour was meant to change, change it and say so. + +**Docs: almost none.** The docs pages are brought up to date once, by the last phase. You +write only: + +- one entry in the log (section 6), **25 lines at most**, appended at the end; +- "Done" on your phase's line in section 5, and a correction of any statement in the spec + or in section 3 that your work proved wrong. + +Edit documents with the Edit/Write tools only: a shell heredoc or one-liner containing +backticks, `$` or non-ASCII text gets mangled and has corrupted documents before. + +**Commit: the lead commits.** Leave your work in the working tree, uncommitted. In the +report, list the files to stage and propose a message (`feat:` / `fix:` / `test:` / +`build:` / `docs:`, lower case, a short what-and-why body). + +**Report** (your final message; all the lead sees; under 60 lines) + +- What was built, by file, briefly. +- The `Totals` lines of the final full test runs, copied, not paraphrased. +- Which tests you saw fail without the change. +- Choices made, deviations, anything fragile or unfinished. Say it plainly: a problem + reported is cheap, one found later is not. + +## 3. Decisions for this port (made by the lead, 2026-09-25) + +| Decision | Why | +| --- | --- | +| Android, on branch `feat/tonyandroid`; Sailfish is not being done | The user's request | +| Tony's code is built and tested on **desktop Linux in the container**; Android code is built by the Android toolchain once it is reachable | No Android toolchain here yet (blocked hosts); the desktop suites catch regressions | +| Audio backend: an `OboeAudioIO` class in `main/`, installed by overriding `createAudioIO()` in `MainWindow` on Android, not a bqaudioio fork | `createAudioIO()` is virtual in svapp's `MainWindowBase`; the test suite already installs `FakeAudioIO` that way (`TestRecordWorkflow.h`) | +| Gestures: an event filter class in `main/` on the panes, not a change in the svgui fork | Keeps the forks untouched; testable with synthetic touch events | +| Build route: first try (a), meson with an NDK cross file, a script writing the deployment JSON, then androiddeployqt; fall back to (b), an Android-only `CMakeLists.txt`, only if (a) fails for a reason that cannot be worked around | `meson.build` holds about 1400 lines of source lists that a CMake copy would have to keep in step | +| No Qt Multimedia | Its input path is not low-latency and reports no latency (port-android.md, Audio) | +| Features beyond the spec (latency calibration setting, session bundle) are not built unless a phone test shows they are needed | The spec says so | + +## 4. State of the code (kept by the lead; as of 2026-09-25, before phase A0) + +- Nothing of the port exists yet. The branch holds `default` plus the research docs. +- The library directories are not checked out in a fresh container; A0 sets them up. + +## 5. Phases + +Order: A0, A1; then A2 and A3 **if the Android toolchain is reachable** (the lead says +so in the prompt), otherwise A4 and A5 first. A6 needs the result of the user's phone test +of A3. A8 is last. + +- A0 — Desktop build and tests in the container. +- A1 — Sample rate: a device that is not at 44.1 kHz. +- A2 — Android toolchain and C libraries. +- A3 — Tony as an APK (no audio): the test port. +- A4 — Touch gestures on the panes. +- A5 — Compact touch mode. +- A6 — Oboe audio backend. +- A7 — Android files, permission and lifecycle. +- A8 — Documentation pass. + +### A0 — Desktop build and tests in the container + +Read: [building.md](building.md) "What is particular about this `meson.build`"; +[testing.md](testing.md) "Running"; `.github/workflows/linux.yml`; `repoint-project.json` +and `repoint-lock.json`. + +- A script, `deploy/linux/container-setup.sh`, that makes a fresh Ubuntu 24.04 cloud + container able to build Tony and run both test suites: apt packages, meson, and the + library directories at their pinned revisions. Idempotent (safe to run twice). +- `./repoint install` cannot work here: the Mercurial libraries (`dataquay`, `bqvec`, + `bqfft`, `bqresample`, `bqaudioio`, `bqthingfactory`) are on `hg.sr.ht`, which is + blocked. Their Git mirrors are at `github.com/breakfastquay/`. The pins in + `repoint-lock.json` are Mercurial hashes that the mirrors do not carry. `bqaudioio`'s pin + `017ab3ed3a33` is the merge of `toggle-record-in-io` into default, which is the mirror's + commit `7ab6de9` ("Merge from branch toggle-record-in-io"). For the others take the + mirror commit that matches the pin by date and message if you can tell, else the + mirror's head, and say which in the script. The Git libraries clone from GitHub at + their pins. +- Then `meson setup` and a build of `Tony` and the two test executables, and both suites + run headless (`QT_QPA_PLATFORM=offscreen` is the likely need). +- The suites have only ever run on Windows. A test that fails on Linux only is a finding: + work out why. If the cause is Tony's code or a test's assumption about the platform, fix + it in `main/` and say so; if it is in a library, report it. Do not skip or weaken tests. +- Ubuntu 24.04 ships Qt 6.4. If Tony or the forks need a newer Qt, say what fails, and + try conda-forge's `qt6-main` as the Qt instead (the script then installs that). +- Not in this phase: any Android work, docs pages (A8 writes the building.md section). +- Report: the exact commands for setup, build and the two test runs, for section 2. + +### A1 — Sample rate: a device that is not at 44.1 kHz + +Read: [mobile-port.md](mobile-port.md) "Sample rate"; [recording.md](recording.md) +"Latency" and "Pre-roll and Record into Selection"; [takes.md](takes.md) "The swap", "Files +on disk", "Known limitations"; [testing.md](testing.md) "What is there to reuse" and +"How tests turned out to be worthless". + +- First an app test with `FakeAudioIO` at 48000 Hz (its `Config::sampleRate`) against a + 44.1 kHz reference: record a take from a known position and check where its audio, its + pitch and its coverage land and how long they are. Also a second take spliced into the + first. This settles whether the suspected misplacement is real. +- If it is: make take audio match the main model's rate before it is spliced (resample + the recording, in `tony_core`, with core tests of the resampling itself), and check the + live tracker, the pre-roll and the latency shift at 48 kHz too. +- If it is not: keep the tests, and correct the spec's "Sample rate" section and the + "Unverified" weak spot in [open-points.md](open-points.md) (that line only). + +### A2 — Android toolchain and C libraries + +Only when the lead says the toolchain is reachable. Read: [port-android.md](port-android.md) +"Qt for Android", "Build and packaging"; [mobile-port.md](mobile-port.md) "Build and +tests". + +- A script that fetches or checks the Android toolchain (Qt 6.11 for `android_arm64_v8a` + plus the matching host Qt, NDK r27c, JDK 21, SDK platform 36 and build tools) and + cross-compiles the C libraries into one prefix for `arm64-v8a`, API 28: libsndfile + (without its codec libraries), libsamplerate, fftw3 (float), Rubber Band 3, libogg, + opus, opusfile, serd and sord, libmad, libid3tag (with the NDK's zlib). Pin versions. +- Leave out JACK, PulseAudio, ALSA, PortAudio, oggz and fishsound; note any other library + that turns out to be needed. +- A CI workflow `.github/workflows/android.yml` only if the lead says Actions are enabled. + +### A3 — Tony as an APK, without audio: the test port + +Read: [port-android.md](port-android.md) all of "Platform facts" and "Test port"; +[mobile-port.md](mobile-port.md) "The pYIN plugin", "The window", "Build and tests". + +- Route (a) from section 3: an `android` branch in `meson.build`, an NDK cross file, Tony + built as `libTony_arm64-v8a.so`, a script that writes the deployment JSON and runs + androiddeployqt, a custom `AndroidManifest.xml` (with `RECORD_AUDIO` for later). +- Audio: `AUDIO_NONE` under `Q_OS_ANDROID` for now. +- The menu bar: exclude `Q_OS_ANDROID` from the `Q_OS_LINUX` `setNativeMenuBar(false)` + only if the ⋮ options menu works better on a phone; otherwise keep the in-window bar and + say why. +- pYIN found on the phone: `libpyin.so` naming plus legacy packaging, or linking it in. + The log must show "Setting VAMP_PATH to ...". +- The desktop build and suites unchanged and green. +- Result: an APK the user can sideload; the lead hands it over. + +### A4 — Touch gestures on the panes + +Read: [architecture.md](architecture.md) "Selection and tools", "Signals"; +[mobile-port.md](mobile-port.md) "The window". + +- A class in `main/` that filters the panes' touch events: pinch zooms the time axis + about the pinch centre, two-finger drag scrolls, long-press opens the pane's right-button + menu. One-finger touches stay as today (Qt's synthesised mouse events). +- Desktop mouse behaviour unchanged. App tests with synthetic touch events + (`QTest::touchEvent` and a `QPointingDevice`). + +### A5 — Compact touch mode + +Read: [mobile-port.md](mobile-port.md) "The window", "Work common to both ports"; +[architecture.md](architecture.md) "Where things are", "Who owns what". + +- A class in `main/`, wired by `MainWindow`; on by default on Android, and switchable on + the desktop (a View menu toggle and a `--compact` option) so it can be tested there. +- One toolbar: Play, Record, Record into Selection, the Take box, Undo, Redo, Erase (the + Ctrl+D action), zoom in and out, and one menu button holding the menus. The Show and + Play toggles and gains in a panel that slides out. The note-editing tools and the audio + device menus hidden. +- Switching back restores the desktop layout exactly. Tests for both directions. + +### A6 — Oboe audio backend + +After the user's phone test of A3. Detailed when it starts. + +### A7 — Android files, permission and lifecycle + +Detailed when it starts: microphone permission, stopping audio and saving on suspend, +opening a reference or session through the picker by copying it into app storage, and +exporting. + +### A8 — Documentation pass + +- Bring the docs pages up to date from the code and the log: building.md (the container + script, the Android build), testing.md, architecture.md (new classes), + manual-checklist.md (phone items), mobile-port.md and port-android.md (what was built, + known limitations, open points), open-points.md. Fold anything still useful from this + file into those pages, then delete this file and its row in docs/README.md if any. No + code: suspected bugs go in the report. + +## 6. Log (newest last; 25 lines at most per entry) + +Template: + + ### Phase — + Built: ... + Choices / deviations: ... + The next phase must know: ... + Left open: ... From d1ddcb50f53072134de8f6b54457223f6bb5cbeb Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 21:39:38 +0000 Subject: [PATCH 071/275] docs: the android toolchain is reachable, so a2 and a3 follow a1 download.qt.io and dl.google.com are now allowed from the cloud container and actions is enabled on the fork, so the toolchain phase installs locally and the apk phase adds the ci workflow. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index a4a69d89..b85ce95b 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -57,10 +57,10 @@ phase A0) While working run **only your tests** by name; run **both whole suites once** at the end, and again only if something failed. The app suite runs in real time (minutes): give it a 10-minute tool timeout. -- Network: GitHub (git and release downloads), the Ubuntu archive, PyPI and conda-forge - are reachable. `download.qt.io`, `dl.google.com`, `hg.sr.ht` and `breakfastquay.com` - are blocked by the environment's policy: do not look for mirrors of them; report if you - need them. +- Network: GitHub (git and release downloads), the Ubuntu archive, PyPI, conda-forge, + `download.qt.io` and `dl.google.com` are reachable. `breakfastquay.com` and + `ppa.launchpadcontent.net` are blocked by the environment's policy, and `hg.sr.ht` + answers 502: do not look for mirrors of blocked hosts; report if you need them. - Every behaviour gets a test that can fail. Show it for the two or three that matter most by breaking the code for a moment. Undo the break **by hand**: never `git checkout`/`git restore` a file to revert an experiment. @@ -108,9 +108,10 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: ## 5. Phases -Order: A0, A1; then A2 and A3 **if the Android toolchain is reachable** (the lead says -so in the prompt), otherwise A4 and A5 first. A6 needs the result of the user's phone test -of A3. A8 is last. +Order: A0, A1, A2, A3, then A4 and A5 while the user tries the APK on the phone. A6 +needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt.io` and +`dl.google.com` are reachable from the container, and GitHub Actions is enabled on +`jhhr/tony`.) - A0 — Desktop build and tests in the container. - A1 — Sample rate: a device that is not at 44.1 kHz. @@ -169,7 +170,7 @@ on disk", "Known limitations"; [testing.md](testing.md) "What is there to reuse" ### A2 — Android toolchain and C libraries -Only when the lead says the toolchain is reachable. Read: [port-android.md](port-android.md) +Read: [port-android.md](port-android.md) "Qt for Android", "Build and packaging"; [mobile-port.md](mobile-port.md) "Build and tests". @@ -180,7 +181,8 @@ tests". opus, opusfile, serd and sord, libmad, libid3tag (with the NDK's zlib). Pin versions. - Leave out JACK, PulseAudio, ALSA, PortAudio, oggz and fishsound; note any other library that turns out to be needed. -- A CI workflow `.github/workflows/android.yml` only if the lead says Actions are enabled. +- The toolchain is installed into the container (outside the repo, e.g. under `/opt`), so + that A3 can iterate locally. The CI workflow comes in A3, once there is an APK to build. ### A3 — Tony as an APK, without audio: the test port @@ -197,6 +199,9 @@ Read: [port-android.md](port-android.md) all of "Platform facts" and "Test port" - pYIN found on the phone: `libpyin.so` naming plus legacy packaging, or linking it in. The log must show "Setting VAMP_PATH to ...". - The desktop build and suites unchanged and green. +- A CI workflow `.github/workflows/android.yml` (Linux runner, triggered by pushes to + `feat/tonyandroid` and by hand) that builds the APK the same way and uploads it as an + artifact. The lead pushes and reads the run; write it so it can only be judged there. - Result: an APK the user can sideload; the lead hands it over. ### A4 — Touch gestures on the panes From a0ab4ad7c67f650144724088c9f2efc708389961 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:22:44 +0000 Subject: [PATCH 072/275] docs: plan for a calibrate audio button and dev checks Tony shifts every take by the latency PortAudio reports, which on MME, DirectSound and WASAPI is buffer sizes only, and most of the manual checklist has never been run. The plan measures the round trip through the ordinary take path with the mic hearing the output, stores it per device, and in development builds runs the checklist items a loopback can settle. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/README.md | 1 + docs/calibrate-audio.md | 320 ++++++++++++++++++++++++++++++++++++++++ docs/open-points.md | 2 + 3 files changed, 323 insertions(+) create mode 100644 docs/calibrate-audio.md diff --git a/docs/README.md b/docs/README.md index 6eea96bc..2ced9f0c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -19,3 +19,4 @@ methods, and they do not tell the story of fixed bugs. | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | | [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | +| [calibrate-audio.md](calibrate-audio.md) | Plan, not built: a Calibrate Audio button that measures the round trip through a speaker-to-mic loopback, and dev checks that automate most of the manual checklist | diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md new file mode 100644 index 00000000..362a3984 --- /dev/null +++ b/docs/calibrate-audio.md @@ -0,0 +1,320 @@ +# Calibrate Audio: plan + +Version 2. One button that measures the audio path. In development builds it also runs +every manual-checklist item that a speaker-to-mic loopback can settle. Nothing is built +yet. The research behind the figures quoted here (OboeTester, Bucket Brigade, Ardour's +MTDM, PortAudio's latency reporting, Audacity's measurements) was a separate report, +not kept in the repository. + +Setup assumed: wired headphones and a wired mic. For the run, hold one earcup against the +mic, off your ears. No cable is needed. A 3.5 mm loop cable would give the same result +with less noise. + +## 1. Why + +- **Every take is shifted by the wrong amount.** Tony shifts it by *reported output + latency + reported input latency + start gap* (`MainWindow::recordingStarted()`, + `LatencyUtils.h`). The start gap is measured and right. The reported pair comes from + `Pa_GetStreamInfo()`, which on MME, DirectSound and WASAPI is buffer sizes only. + Audacity measured it off by −5 to +155 ms. bqaudioio opens the stream with + `suggestedLatency = 0.2` on both sides. +- **The device's sample rate is not checked.** The device opens at PortAudio's default + rate. For "(System Default)" through MME that is most likely 44.1 kHz. For a device + chosen from the menu whose name exists only under WASAPI or WDM-KS, it is often 48 kHz. + `TakeAudio::splice()` writes the first recording of a take at that rate and places it + with frames of the 44.1 kHz reference. +- **Most of the manual checklist has never been run** (`docs/manual-checklist.md`, + 36 items). Many items ask whether logic that the app suite proves on `FakeAudioIO` + also holds on a real device. A loopback run in the real app can answer that without a + person listening. + +## 2. The button + +**Playback ▸ Calibrate Audio…** is in every build. + +1. **Instructions:** + - an earcup against the mic, off your ears; + - moderate volume, quiet room; + - names the output and input device and the latency now in use (reported or + measured). +2. **Test session.** Tony writes a generated test reference and opens it the way + File ▸ Open does. You are asked to save your work first. The run never touches your + song's takes or undo history. +3. **Calibration, about 30 s.** Four punch-ins at about 2, 8, 14 and 20 s, using the + ordinary take path with Play Reference While Recording on. Every punch-in restarts the + stream, as a real take does. +4. **Result page:** + - **Round trip:** measured, next to the driver's figure. + - **Spread between punch-ins:** how much the driver's timing moves from one stream + start to the next. + - **Sample rates:** the recording's rate against the reference's 44100 Hz. + - **Mic channel:** which input channel carries the mic. + - **Levels:** noise floor, and the input peak (clipping). + - **Verdict:** in plain words, with the fix for each failure. + + **Use this latency** stores the figure. +5. **Development builds only:** the dialog carries on into the **dev checks**, about + 4 minutes (section 5). It ends with a report page and a report file. + +The test session stays open afterwards, so the reference's and the takes' pitch tracks +can be looked at. You go back to your song through Recent Files. + +## 3. Dev mode + +`meson.build` already switches on the build type (`WANT_TIMING` versus `NO_TIMING`). Add: + +- `-DTONY_DEV_CHECKS` when the build type does not start with `release`; +- the dev-check source files, compiled only then. + +`build.bat` sets up `build_mingw` as `debugoptimized`, so your builds have the checks. +`meson.build`'s default and the deploy scripts use `release`, so packages have none of +the code. + +A runtime flag, so that a release build on someone else's PC could run the checks, is +possible later. It would ship the check code in every package, so it is not part of this +plan. + +## 4. What the loopback run settles, item by item + +The numbers are those of `docs/manual-checklist.md`. + +- **Automated:** the dev checks pass or fail it. +- **Measured:** the dev checks report numbers; a person still judges how it looks or + feels. +- **Smoke:** the logic is already covered by the app suite; the dev checks repeat it on + the real device and real timing. Optional, last. +- **Manual:** stays on the checklist. + +| # | Item | Verdict | How | +| --- | --- | --- | --- | +| 1 | Latency on this machine, also after save and reopen | **Automated** | Every sweep of every punch-in lands within ±2 ms (§6). Save the test session to a temp `.ton`, reopen it, analyse again: unchanged. | +| 2 | Several phrases in one take | **Automated** | Punch-ins at four positions of one take, each placed right. Each measures its own start gap. | +| 3 | Live dots | **Measured** | *Automated:* dots sit on the reference's tones on the timeline (±1 hop); they stay after Stop until the pitch track arrives, then go; the status bar stops changing; the take's own pitch and notes are hidden during the take and back after. *Reported:* how far behind the cursor a dot appears, in ms. *Eyes:* does it look right. | +| 4 | Nothing of the take in the speakers | **Automated** | Re-record over earlier material. Tony's output peak (`getOutputLevels()`) is exactly zero in the reference's silent gaps, so no take audio and no synth. The mic hears no second arrival of each sweep; one would mean the input is monitored somewhere (Windows "Listen to this device", or an interface's direct monitor). Play Singing Audio keeps its state. | +| 5 | Mic on input 2 of a stereo interface | **Measured** | Per-channel input peaks show which input the mic is on. If it is input 2, dots appearing is the check; otherwise "not applicable here". | +| 6 | No input device / device in use | **Manual** | Needs the device gone or busy. | +| 7 | Record from a position; overwrite question | **Automated** (placement) | Placement as in 1. Outside the new range, the take's audio is bit-identical and pitch and notes are unchanged beyond ±0.25 s. The question, No, and "Don't ask again" stay with the app suite (`record_over_existing_question`) and a glance. | +| 8 | Cursor from P, pane follows, all in one place | **Measured** | *Automated:* the cursor is at S when the take starts, and inside the visible range throughout. *Reported:* the dot-to-cursor offset. *Eyes:* the rest. | +| 9 | Stop on a 4-minute song | **Automated** | A generated 4-minute reference, punch-in near the end. Time from Stop to new pitch, against a threshold. Pitch outside the range unchanged, which proves the ranged path ran. | +| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. | +| 11 | Is 3 s right, is the countdown readable | **Manual** | A judgement. | +| 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. | +| 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown starts at 1, placement is right. | +| 14 | Record into Selection stops by itself | **Automated** | Stops within 0.25 s plus one poll of the end. The coverage added is exactly the selection. No dialog. | +| 15 | The practice loop | **Measured** | *Automated:* it stops by itself, the playhead is back at P, and Play plays the take. *Yours:* "is anything else needed?". | +| 16 | Constrain Playback to Selection with a pre-roll | **Measured** | Reports whether the lead-in was played in full. The decision is yours. | +| 17 | Coverage band readable | **Manual** | Eyes. | +| 18 | Band cannot be touched | **Manual** | Could become an app-suite test with synthetic mouse events; it needs no device. | +| 19 | Select Recording at Playhead, then Erase | Smoke | Audio zero in the range, band and events gone, no analysis. | +| 20 | Erase/Select greyed out when they should be | Smoke | Action states sampled through a real take and its real analysis time. | +| 21 | Ctrl+Z three times | Smoke | On real takes, with the menu texts. | +| 22 | Ctrl+Z straight after Stop | Smoke | With real analysis timing. | +| 23 | Undo of the very first recording | Smoke | | +| 24 | New Empty Take, switching | Smoke | Switch time measured. The inactive take is silent: output peaks over its region while the active take is empty there. | +| 25 | Duplicate, record into the copy | Smoke | The original's file is bit-identical afterwards. | +| 26 | Delete asks first; undo clearing acceptable? | **Manual** | Dialogs and a judgement. | +| 27 | Combo and Takes menu greyed during a take | Smoke | Sampled during a real take. | +| 28 | Analyse Now on a take | Smoke | | +| 29–35 | Sessions and files | **Manual** | File system and OS. Mostly covered by the app suite already; item 35 needs a log-out. | +| 36 | Looks | **Manual** | Eyes. | + +**Tally:** + +| Category | Items | Count | +| --- | --- | --- | +| Automated | 1, 2, 4, 7, 9, 10, 12, 13, 14 | 9 | +| Measured, judgement left | 3, 5, 8, 15, 16 | 5 | +| Smoke | 19–25, 27, 28 | 9 | +| Manual | 6, 11, 17, 18, 26, 29–36 | 13 | + +Also newly checked, though not a checklist item yet: the device's sample rate. + +## 5. Architecture + +Following `AGENTS.md`: state gets its own class, `MainWindow` only wires, and pure logic +goes in `tony_core`. + +### `tony_core` (pure, core suite) + +- **`LatencyCheck`** + - **Generator.** Fixed layouts: + - *calibration*: 26 s; + - *dev*: adds held tones of 3 s for the join check; + - *long*: 4 minutes. + + Each event is a sweep of 1 → 8 kHz, 200 ms, with 10 ms edges and a −12 dBFS peak, + followed by a tone at a pitch pYIN tracks (196, 220.5, 245 or 294 Hz; see + `docs/testing.md`). Gaps between events are irregular (1.6–2.6 s, all different). + The generator returns every sweep's exact frame. + - **Analysis:** + - FFT matched filter in the sweep band (bqfft), then its envelope; + - the **earliest** peak within 6 dB of the largest; + - confidence: ≥ 15 dB over the window's median, ≥ 6 dB over the second peak outside + ±10 ms (starting values, to tune); + - error per sweep; median and spread per punch-in and across punch-ins; slope over + position; + - **second-arrival detection**, for monitoring echo; + - verdicts: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent. + - **Calibration arithmetic:** `newRoundTrip = usedRoundTrip + median offset`. A take + that lands late was spliced from too early a frame. +- **`TakeDiff`** + - audio bit-identity outside a range; + - pitch and note events unchanged outside a range ± margin; + - pitch continuity across a join (gaps, doubled frames); + - notes spanning a frame; + - sample step at a join. +- **`LatencyCalibration`** + - **Key:** the Preferences values `createAudioIO()` reads, plus the recording rate. + - **Value:** round trip in seconds, spread, date, and the reported pair at the time, + which works as a staleness fingerprint. + +### App, every build + +- **`AudioCheckRunner`.** Primitives on the live window: + - generate and open a test reference; + - `punchIn(P, E)`, recorded with Record into Selection, Play Reference While Recording + on and a 1 s pre-roll, **without writing the user's settings** (the three actions + write QSettings when toggled, so this needs an override inside `record()`, not + `setChecked()`); + - wait for the splice and for the analysis; + - read the take's audio, coverage and events. + + A **`TakeObserver`** polls every 20 ms during a punch-in and records: + - output and input peaks per channel; + - each live dot, with the cursor frame when it was added; + - playback frame, pane centre, status text, action states; + - frames received, and when the take stopped. +- **`CalibrateAudioDialog`.** Instructions, progress, the result page, Use this latency. +- **`MainWindow`.** + - The menu entry. + - In `recordingStarted()`: "stored round trip if valid for this key, else the reported + sum", logged either way; converted at the recording's rate. + - A record, kept at take start, of the round trip each take used. + - "Forget Measured Latency". + +### App, development builds only (`main/dev/`) + +- **`DevChecks`.** A list of checks. Each returns a plain + `CheckResult { item, name, verdict (Pass/Fail/Measured/Skipped), numbers, message }`. + Waiting is done with a small `waitUntil(predicate, timeout)`, a `QEventLoop` with a + timer, behind the modal progress dialog. Cancel stops the take and closes the test + session cleanly. + + They are **not** QtTest functions. A QVERIFY failure cannot be asserted from inside + another QtTest, and the app suite has to prove each check can fail. +- **Access.** `DevChecks` is a `friend` of `MainWindow` under `#ifdef TONY_DEV_CHECKS`, + so there is no public surface in release builds. +- **Report.** A page in the dialog, grouped by checklist item, with the measured numbers. + A text file goes to `TONY_TEST_LOG_DIR` if that is set, else to the app data directory, + and ends with a `Totals:` line like the suites. +- **Scratch files.** The run saves its test session into a temporary folder, so every + take file lands in `.takes/`. It deletes the folder at the end, unless + something failed; then the report names it. + +### The dev run, one scripted sequence (~4 min) + +Calibration first. The dev checks then use the new figure for the run only; your stored +setting changes only through Use this latency. + +| Step | What it does | Items | +| --- | --- | --- | +| 1 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 5, 8 | +| 2 | Re-record over an earlier punch-in, through its lead-in | 4, 7, 12, 14 | +| 3 | Punch-in at P = 1 s | 13 | +| 4 | Two adjacent punch-ins meeting inside a held tone | 10 | +| 5 | Constrain Playback to Selection on for one punch-in, then off | 16 | +| 6 | Play from P | 15 | +| 7 | Save to the temp `.ton`, reopen, analyse again | 1 | +| 8 | Long reference, punch-in near the end | 9 | +| 9 | Smoke group, optional | 19–25, 27, 28 | + +## 6. Tests + +- **Core suite, `TestLatencyCheck` and `TestTakeDiff`.** Synthetic takes: + - every shift from −0.75 to +0.75 s, including a fractional one; + - noise at 0 and −10 dB SNR; + - small-speaker band limiting; + - polarity inverted; + - a stronger reflection 7 ms after the direct sound; + - an event missing, silence, fading events, clipping; + - resampled by 48000/44100; + - two punch-ins 20 ms apart; + - a monitoring echo; + - the join cases. +- **App suite.** `TestMainWindow` with `FakeAudioIO` `loopback = true`: + - **Calibration.** Wrong reported latencies are measured right. With the stored + figure, `latency_end_to_end`'s recipe lines up; without it, the same test fails. A + stale key falls back to the reported sum. The user's three toggles and their + settings are untouched. A 48 kHz fake reports PositionDependent and names the rate. + - **Every dev check at least twice.** Once passing on a calibrated fake, and once + **failing** under an injected fault: + - an uncalibrated offset, for items 1, 2, 7 and 13; + - a monitoring echo, from a second loopback tap in the fake, for item 4; + - a splice offset broken on purpose, for item 10; + - a 48 kHz device. + + That is the "can fail" proof for each one. + - **`FakeAudioIO` additions**, all new fields that change no existing meaning: + loopback gain, a second echo tap and noise. + - **Where they run.** If the dev-check tests add more than about a minute of real time + to the app suite, they move to a third executable, `test-tony-dev`. That would + change the "run both whole suites" rule in `AGENTS.md`, so ask first. + +## 7. Order of work + +Every step ends with both whole suites green; commits only when asked. + +1. **Core:** `LatencyCheck`, `TakeDiff`, `LatencyCalibration` and their tests. +2. **Runner, dialog and calibration page** (every build), with app tests. **Then you run + it on your PC.** Its numbers settle three things before the rest is built: how wrong + the driver's figure is, whether the offset holds across stream restarts on MME, and + whether your device's rate hits the takes. +3. **Calibration in use:** `recordingStarted()`, Use this latency, Forget, staleness, + app tests. +4. **Dev-check framework:** build flag, `DevChecks`, `TakeObserver`, report, friend + access. First group: items 1, 2, 7, 12, 13, 14. +5. **Observer group:** items 3, 4, 5, 8, 15, 16. +6. **Join and long-song group:** items 9 and 10. +7. **Smoke group:** optional. +8. **Docs,** in the same commit as the code they describe: + - `manual-checklist.md`: an automated item keeps its text and gets "*automated: + dev check ``*"; a measured item keeps only the question for a person. The + list becomes what a person must do after a dev run. + - `testing.md`: a Dev checks section. + - `recording.md`: the latency section. + - `open-points.md`. + +**Separate task:** fix the device-rate mismatch. For example the record target could ask +for the session's rate, a one-line change to +`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork; or the splice +could convert. The button then shows the fix working on each device. + +## 8. Risks + +- **Restart jitter.** If the spread across punch-ins is ≥ 10 ms, no stored figure fits + every take. The remedy would be to keep the stream running between takes instead of + suspending it in `stop()`, an svapp change. Decide on step 2's numbers. +- **Windows enhancements or echo cancellation** can remove the sweeps. Tony cannot ask for + raw capture: bqaudioio is upstream and MME has no raw mode. The NoSignal and Fading + verdicts point to *Sound settings ▸ device ▸ Audio enhancements: Off*. +- **Thresholds are guesses** until real runs exist. Every check reports its numbers as + well as pass or fail, and the report file is what tunes them. +- **Nested event loops in the live app.** The run stays behind a modal progress dialog, + and Cancel must always leave a clean state. `closeSession()` already stops take + polling. +- **Loudness.** The sweeps are −12 dBFS with earcups off the ears; the dialog says so + before starting. +- **Cursor versus dots.** The cursor subtracts the *reported* output latency. With a + measured round trip the dots move to the right place and may sit off the cursor. Item + 8's number will show how much. Fixing it needs the round trip split between output + and input, which is not in this plan. +- **Release builds must stay clean.** A `release`-type build must compile with no + `main/dev/` file, so build one before calling step 4 done. + +## 9. Later candidates for the button + +- **A noise gate for live dots.** `RealtimePitchTracker` has no level gate (YIN is + scale-free), so room noise can make dots. The measured noise floor could set one. +- **A quick re-measure** after a Bluetooth reconnect, without a test session. +- **Items 18 and 6**, as app-suite tests: synthetic mouse events, and a fake device that + fails to open. diff --git a/docs/open-points.md b/docs/open-points.md index 7776f247..4025ee13 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -17,6 +17,8 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Not built +- **Calibrate Audio**: a measured round trip in place of PortAudio's reported latency, + and dev checks for the manual checklist. Planned in [calibrate-audio.md](calibrate-audio.md). - Showing two takes at once, or any comparison of takes other than switching. - Singing track gain and pan are not saved in the session. - Background music is not saved in the session; it is reloaded by hand. From a1db45208348004d88e41331ee6d69b7d3e5bf00 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:25:00 +0000 Subject: [PATCH 073/275] build: a setup script for building and testing in the ubuntu cloud container repoint cannot reach hg.sr.ht from the cloud container, so the mercurial libraries come from their github mirrors, matched to the pins by date. qt 6.11 comes from conda-forge: with ubuntu's 6.4 the analyser's string connects naming sv:: types never match, and the analysis never completes. the work orders record phase a0 and the container's build and test commands. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/linux/container-setup.sh | 289 ++++++++++++++++++++++++++++++++ docs/android-work-orders.md | 61 ++++++- 2 files changed, 343 insertions(+), 7 deletions(-) create mode 100755 deploy/linux/container-setup.sh diff --git a/deploy/linux/container-setup.sh b/deploy/linux/container-setup.sh new file mode 100755 index 00000000..65b21c5a --- /dev/null +++ b/deploy/linux/container-setup.sh @@ -0,0 +1,289 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Makes a fresh Ubuntu 24.04 cloud container able to build Tony and run +# both test suites: installs the apt packages and Qt, checks out the +# library directories at the revisions pinned in repoint-lock.json, and +# configures build/ with meson. +# +# It exists because "./repoint install" cannot work in the cloud +# container: the Mercurial libraries are on hg.sr.ht, which cannot be +# reached from there. Their Git mirrors on GitHub are +# checked out instead, at the commits in the table below. The Git +# libraries are cloned from GitHub at their pins, as repoint would. +# sv-dependency-builds is skipped: only the macOS and Windows branches +# of meson.build use it. +# +# Qt is conda-forge's qt6-main, not Ubuntu's Qt 6.4. Tony builds with +# 6.4, but its analysis never completes there: Analyser connects by +# SIGNAL()/SLOT() strings naming ModelId and sv_frame_t to slots that +# moc records as sv::ModelId and sv::sv_frame_t, and only Qt 6.5 and +# later match those by their registered metatypes rather than by name. +# The development machine and the Android build use Qt 6.11. +# +# Safe to run again. Packages already installed, a Qt of the right +# version and libraries already at their pins are left alone, and a +# library with local changes or local commits is never moved. +# +# Usage, from anywhere: +# deploy/linux/container-setup.sh set up and configure build/ +# deploy/linux/container-setup.sh --build the same, then build Tony, +# the pYIN plugin and both +# test executables + +set -eu -o pipefail + +build=no +case "${1:-}" in + "") ;; + --build) build=yes ;; + *) echo "Usage: $0 [--build]" 1>&2; exit 2 ;; +esac + +cd "$(dirname "$0")/../.." +root=$(pwd) +echo "Setting up $root" + +sudo="" +if [ "$(id -u)" -ne 0 ]; then + sudo=sudo +fi + +qt_version=6.11.2 +qt_prefix=/opt/qt6-conda +micromamba_version=2.9.0-0 +micromamba=/opt/micromamba/bin/micromamba + +# Only Qt's own .pc files, so that pkg-config finds everything else +# (alsa, for one, which the conda prefix has too) on the system +qt_pkgconfig=$qt_prefix/tony-pkgconfig + +# 1. Packages. The names are those of .github/workflows/linux.yml where +# meson.build needs them, as spelled on Ubuntu 24.04, less Qt. Rubber +# Band is Ubuntu's librubberband-dev (3.3), which meets meson.build's +# ">= 3.0.0", so the CI's tarball from breakfastquay.com (blocked here +# too) is not needed. + +packages=" +build-essential pkg-config ninja-build meson git python3 curl ca-certificates +libboost-dev libbz2-dev libfftw3-dev libsndfile1-dev libsamplerate0-dev +librubberband-dev libsord-dev libserd-dev liboggz2-dev libfishsound1-dev +libmad0-dev libid3tag0-dev libopus-dev libopusfile-dev libopusenc-dev +libjack-jackd2-dev libpulse-dev libasound2-dev portaudio19-dev +fonts-dejavu-core +" + +missing="" +for p in $packages; do + if ! dpkg-query -W -f='${Status}' "$p" 2>/dev/null | grep -q "install ok installed"; then + missing="$missing $p" + fi +done + +if [ -n "$missing" ]; then + echo + echo "Installing packages:$missing" + # Some of the container's own apt sources (PPAs) are blocked too; + # apt-get update warns about them and carries on. + $sudo apt-get update -q + $sudo env DEBIAN_FRONTEND=noninteractive \ + apt-get install -y -q --no-install-recommends $missing +else + echo + echo "Packages: all installed" +fi + +# 2. Qt, from conda-forge through micromamba (both from hosts the +# container can reach: GitHub releases and conda.anaconda.org). + +echo +if [ -x "$micromamba" ]; then + echo "micromamba: installed already" +else + echo "Installing micromamba $micromamba_version in $(dirname "$micromamba")" + $sudo mkdir -p "$(dirname "$micromamba")" + # The container's proxy now and then answers 502 for a moment + $sudo curl -sSfL --retry 5 --retry-all-errors -o "$micromamba.part" \ + "https://github.com/mamba-org/micromamba-releases/releases/download/$micromamba_version/micromamba-linux-64" + $sudo chmod +x "$micromamba.part" + $sudo mv "$micromamba.part" "$micromamba" +fi + +installed_qt="" +if [ -f "$qt_prefix/lib/pkgconfig/Qt6Core.pc" ]; then + installed_qt=$(PKG_CONFIG_PATH="$qt_prefix/lib/pkgconfig" pkg-config --modversion Qt6Core) +fi + +if [ "$installed_qt" = "$qt_version" ]; then + echo "Qt: $qt_version in $qt_prefix already" +else + echo "Installing Qt $qt_version (conda-forge qt6-main) in $qt_prefix" + if [ -d "$qt_prefix/conda-meta" ]; then + action=install + else + action=create + fi + $sudo env MAMBA_ROOT_PREFIX="$(dirname "$(dirname "$micromamba")")" \ + "$micromamba" $action -y -q -p "$qt_prefix" -c conda-forge \ + "qt6-main=$qt_version" +fi + +$sudo mkdir -p "$qt_pkgconfig" +for pc in "$qt_prefix"/lib/pkgconfig/Qt6*.pc; do + $sudo ln -sf "$pc" "$qt_pkgconfig/" +done + +# 3. Libraries. +# +# repoint-lock.json pins the Mercurial libraries by Mercurial hash, +# which their Git mirrors (github.com/breakfastquay/) do not +# carry, and a conversion back to Mercurial does not reproduce it. Each +# pin was matched to a mirror commit by hand, by date and message: +# +# - bqaudioio 017ab3ed3a33 is the merge of toggle-record-in-io into +# default: the mirror's "Merge from branch toggle-record-in-io". +# - The other pins were taken by upstream Tony's "Update Repoint +# locations and revisions" (2024-06-25). For each, the commit is the +# last one on the mirror's master before that date, and Sonic +# Visualiser's repoint-lock.json took the same Mercurial pin shortly +# after that commit's date: +# dataquay "Fix warning", 2024-01-04 (SV pinned it the same +# day; the mirror's head is newer, from 2024-09) +# bqvec "Adjust local include policy", 2023-06-27 (head) +# bqfft "Update CI for SLEEF", 2022-08-09 (head) +# bqresample "Adjust local include policy", 2023-06-27 (head) +# bqthingfactory "Copyright dates", 2021-01-08 (head) +# +# When a pin in repoint-lock.json changes, this table must change with +# it; the script stops if they disagree. + +mirror_commit() { + case "$1 $2" in + "bqaudioio 017ab3ed3a33") echo 7ab6de96b44d2f0c8ce58a16ce1a9831724dd29f ;; + "dataquay 79623fb778da") echo 2dbf1bed112c1a7eaaf43335abbe5ddb5c03d0ff ;; + "bqvec 291cde50db9d") echo ddfcd1716576c6bb44218c5f5696bf24a248960a ;; + "bqfft d41a117b8cbe") echo 68dc4c5735c1e0da099e8473fc4562acf2895cb8 ;; + "bqresample 38c3e524416a") echo 6cef06961f15399f8cecc414f1364dac7d83c3db ;; + "bqthingfactory 2e4bd170f57f") echo 8468b3f98d9d1768561d6f1fc95d1689af193a3e ;; + *) echo "" ;; + esac +} + +warnings=0 + +checkout() { + local name="$1" url="$2" branch="$3" commit="$4" + local short="${commit:0:12}" + if [ -e "$name/.git" ]; then + local head + head=$(git -C "$name" rev-parse HEAD) + if [ "$head" = "$commit" ]; then + echo " $name: at $short already" + return + fi + if [ -n "$(git -C "$name" status --porcelain --untracked-files=no)" ]; then + echo " $name: WARNING: has local changes and is not at $short; left alone" + warnings=1 + return + fi + if ! git -C "$name" cat-file -e "$commit^{commit}" 2>/dev/null; then + echo " $name: fetching from $url" + git -C "$name" fetch -q origin + fi + # Local commits that are on no remote branch would be lost from + # sight by moving the checkout + if [ -z "$(git -C "$name" branch -r --contains HEAD)" ]; then + echo " $name: WARNING: has local commits and is not at $short; left alone" + warnings=1 + return + fi + # Detached, so that no local branch is moved + git -C "$name" checkout -q --detach "$commit" + elif [ -e "$name" ] && [ -n "$(ls -A "$name")" ]; then + echo "ERROR: $name exists but is not a Git checkout; move it away and run again" 1>&2 + exit 1 + else + echo " $name: cloning $url" + git clone -q ${branch:+--branch "$branch"} "$url" "$name" + # As repoint does: the local branch at the pin, where there is one + if [ -n "$branch" ]; then + git -C "$name" checkout -q -B "$branch" "$commit" + else + git -C "$name" checkout -q --detach "$commit" + fi + fi + echo " $name: checked out $short" +} + +echo +echo "Libraries:" + +# One line per library: name, vcs, owner, repository, branch, pin +libraries=$(python3 - <<'EOF' +import json +project = json.load(open("repoint-project.json"))["libraries"] +lock = json.load(open("repoint-lock.json"))["libraries"] +for name, lib in project.items(): + print(name, lib["vcs"], lib["owner"], lib.get("repository", name.split("/")[-1]), + lib.get("branch", "-"), lock[name]["pin"]) +EOF +) + +while read -r name vcs owner repository branch pin; do + [ "$branch" = "-" ] && branch="" + if [ "$name" = "sv-dependency-builds" ]; then + echo " $name: skipped, not used on Linux" + continue + fi + if [ "$vcs" = "git" ]; then + checkout "$name" "https://github.com/$owner/$repository.git" "$branch" "$pin" + else + commit=$(mirror_commit "$name" "$pin") + if [ -z "$commit" ]; then + echo "ERROR: repoint-lock.json pins $name at $pin, which this script does not map to a commit of its Git mirror; update the table in $0" 1>&2 + exit 1 + fi + checkout "$name" "https://github.com/$owner/$repository.git" "" "$commit" + fi +done <<< "$libraries" + +# 4. Configure. The same build type as the Windows build (build.bat): +# optimised, with asserts. meson keeps the pkg-config path in build/, so +# the reconfigures ninja runs by itself find the same Qt, and it puts +# Qt's library directory in the executables' RPATH. + +setup_args="--buildtype=debugoptimized --pkg-config-path=$qt_pkgconfig" + +echo +if [ -f build/build.ninja ] && + grep -q "$qt_pkgconfig" build/meson-info/intro-buildoptions.json; then + echo "build/: configured already" +elif [ -f build/build.ninja ]; then + echo "build/ is configured with another Qt: configuring it again from scratch" + meson setup --wipe build $setup_args +else + echo "Configuring build/" + meson setup build $setup_args +fi + +if [ "$build" = "yes" ]; then + echo + echo "Building" + ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app +fi + +echo +if [ "$warnings" -ne 0 ]; then + echo "Done, with warnings above" +else + echo "Done" +fi diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index b85ce95b..8346e877 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -46,11 +46,31 @@ first. - Android-only code sits behind `#ifdef Q_OS_ANDROID` (or in files the Android build alone compiles) and must not change the desktop build's behaviour. -**Build and test in this container** (Linux; the lead fills in the exact commands after -phase A0) +**Build and test in this container** (Linux) - Not the Windows commands in AGENTS.md: this is an Ubuntu 24.04 container, 4 cores, - 15 GB memory, no swap. ``. + 15 GB memory, no swap, running as root. Qt 6.11 is conda-forge's, in `/opt/qt6-conda`. + In a fresh container run `deploy/linux/container-setup.sh` first (apt, Qt, the library + directories, `meson setup build`); it is idempotent. From the repo root: + + ```sh + mkdir -p tmp + ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app > tmp/build.log 2>&1 + echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log + ``` + + From `build/` (no environment needed: the tests set the offscreen platform themselves): + + ```sh + mkdir -p ../tmp/tl + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core > ../tmp/test.log 2>&1; echo "exit:$?" + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app > ../tmp/test.log 2>&1; echo "exit:$?" # 4.5 min + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app undo_two_takes_in_order > ../tmp/test.log 2>&1 + grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt + ``` + + No `.exe` on Linux; the plugin target is `pyin.so`. Tony needs Qt 6.5 or later at run + time (string connects with `sv::` types, see the A0 log entry). - Send build output to a log file with the exit status written into it; look at the tail or grep it for errors, never read it whole. - Tests: read results from the per-suite files in `TONY_TEST_LOG_DIR` (see AGENTS.md). @@ -101,10 +121,12 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: | No Qt Multimedia | Its input path is not low-latency and reports no latency (port-android.md, Audio) | | Features beyond the spec (latency calibration setting, session bundle) are not built unless a phone test shows they are needed | The spec says so | -## 4. State of the code (kept by the lead; as of 2026-09-25, before phase A0) +## 4. State of the code (kept by the lead; as of 2026-09-25, after phase A0) -- Nothing of the port exists yet. The branch holds `default` plus the research docs. -- The library directories are not checked out in a fresh container; A0 sets them up. +- Nothing of the port exists yet. The branch holds `default`, the research docs, and the + container setup script `deploy/linux/container-setup.sh`. +- Both suites pass on Linux (Qt 6.11.2 from conda-forge). `TestTakesFile`'s Windows path + assertions run on Windows only. ## 5. Phases @@ -113,7 +135,7 @@ needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt. `dl.google.com` are reachable from the container, and GitHub Actions is enabled on `jhhr/tony`.) -- A0 — Desktop build and tests in the container. +- A0 — Desktop build and tests in the container. Done. - A1 — Sample rate: a device that is not at 44.1 kHz. - A2 — Android toolchain and C libraries. - A3 — Tony as an APK (no audio): the test port. @@ -256,3 +278,28 @@ Template: Choices / deviations: ... The next phase must know: ... Left open: ... + +### Phase A0 — 2026-09-25 +Built: `deploy/linux/container-setup.sh` (apt packages, conda-forge `qt6-main` 6.11.2 in +`/opt/qt6-conda` through micromamba, the libraries at their pins, `meson setup build`; +`--build` also builds). `main/test/TestTakesFile.h`: its back-slash and case-insensitive +path assertions now run on Windows only, with Linux counterparts. +Choices / deviations: +- Qt 6.11 from conda-forge, not Ubuntu's 6.4. Tony compiles with 6.4, but pYIN results + never reach `Analyser`: its `SIGNAL()`/`SLOT()` strings say `ModelId` and `sv_frame_t`, + moc records `sv::ModelId`, and only Qt 6.5 and later match the two through their + registered metatypes (`methodMatch()` in Qt's `qmetaobject.cpp`). `MainWindow.cpp`'s + `doubleClickSelectInvoked(sv_frame_t)` connect is the same. Tony needs Qt >= 6.5 at run + time; nothing in `meson.build` says so. +- Mercurial pins matched to mirror commits by date (reasons in the script): dataquay is + `2dbf1be`, not the mirror's head; the other four are their heads. sv-dependency-builds + is not cloned (Linux does not use it). Rubber Band is Ubuntu's 3.3.0. +The next phase must know: only `Qt6*.pc` are on meson's pkg-config path; the RUNPATH is +`/opt/qt6-conda/lib`, so libasound and libstdc++ come from there at run time. No `.exe`; +the plugin target is `pyin.so`. Clean build about 5 min at `-j 4`, 0.8 GB per compile job; +app suite 4.5 min. +Left open: +- `svapp/audio/AudioCallbackRecordTarget.cpp:291` connects to `aboutToBeDeleted()`, which + no model has: a warning in every recording test, on every platform. Not touched. +- architecture.md "Signals" says such string connects never match: true for pointers + (`Layer *`), but registered types match from Qt 6.5. For A8. From bcbb5c9aa9a38bb3a9bfee4638ecea9a52a7e336 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:25:00 +0000 Subject: [PATCH 074/275] test: the takes file suite's windows path assertions run on windows only back slashes as separators and names that ignore case are windows behaviour by design; on other systems the suite checks that names are case-sensitive and uses a posix absolute path. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/test/TestTakesFile.h | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/main/test/TestTakesFile.h b/main/test/TestTakesFile.h index 223466cd..c968a3d0 100644 --- a/main/test/TestTakesFile.h +++ b/main/test/TestTakesFile.h @@ -211,8 +211,12 @@ private slots: void takes_folder() { QCOMPARE(TakesFile::takesFolder("C:/songs/My Song.ton"), QString("C:/songs/My Song.takes")); +#ifdef Q_OS_WIN + // Back slashes separate the parts of a path on Windows only; + // elsewhere they are part of a name QCOMPARE(TakesFile::takesFolder("C:\\songs\\My Song.ton"), QString("C:/songs/My Song.takes")); +#endif QCOMPARE(TakesFile::takesFolder("C:/Käännös/Säkeistö 2.ton"), QString("C:/Käännös/Säkeistö 2.takes")); // Only the extension goes, so a name with dots of its own keeps them @@ -232,10 +236,12 @@ private slots: ("C:/songs/My Song.ton", "C:/songs/My Song.takes/take-1.wav"), QString("My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN QCOMPARE(TakesFile::relativeAudioPath ("C:/songs/My Song.ton", "C:\\songs\\My Song.takes\\take-1.wav"), QString("My Song.takes/take-1.wav")); +#endif QCOMPARE(TakesFile::relativeAudioPath ("C:/Käännös/Säkeistö.ton", "C:/Käännös/Säkeistö.takes/take-1.wav"), @@ -268,15 +274,23 @@ private slots: QCOMPARE(TakesFile::resolveAudioPath ("C:/songs/My Song.ton", "My Song.takes/take-1.wav"), QString("C:/songs/My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN QCOMPARE(TakesFile::resolveAudioPath ("C:\\songs\\My Song.ton", "My Song.takes\\take-1.wav"), QString("C:/songs/My Song.takes/take-1.wav")); +#endif QCOMPARE(TakesFile::resolveAudioPath ("C:/Käännös/Säkeistö.ton", "Säkeistö.takes/take-1.wav"), QString("C:/Käännös/Säkeistö.takes/take-1.wav")); - QCOMPARE(TakesFile::resolveAudioPath - ("C:/songs/My Song.ton", "C:/recorded/take-1.wav"), - QString("C:/recorded/take-1.wav")); + // An absolute path is one in the form of the system the session + // is read on: a drive letter makes one only on Windows +#ifdef Q_OS_WIN + QString elsewhere = "C:/recorded/take-1.wav"; +#else + QString elsewhere = "/recorded/take-1.wav"; +#endif + QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", elsewhere), + elsewhere); QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", ""), QString()); @@ -302,9 +316,15 @@ private slots: void in_folder() { QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes", "C:/songs/My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN // Windows tells no two names apart by their case QVERIFY(TakesFile::isInFolder("C:/songs/my song.takes", "C:\\Songs\\My Song.takes\\take-1.wav")); +#else + // Other systems do + QVERIFY(!TakesFile::isInFolder("/songs/my song.takes", + "/Songs/My Song.takes/take-1.wav")); +#endif QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes/", "C:/songs/My Song.takes/in/take-1.wav")); QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", From a9cd3babccfec71a1757aa7e05cc427e4f522349 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:26:55 +0000 Subject: [PATCH 075/275] docs: calibrate audio phases, decisions and facts checked in the code The plan is split into phases for a line of agents. The work orders say what each reads, how to build and test in this session, and what it may not touch; the spec gains the decisions made and the facts about today's code the phases rest on. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 228 ++++++++++++++++++++++++++++ docs/calibrate-audio.md | 113 ++++++++++++-- 2 files changed, 326 insertions(+), 15 deletions(-) create mode 100644 docs/calibrate-audio-work-orders.md diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md new file mode 100644 index 00000000..ae63b45a --- /dev/null +++ b/docs/calibrate-audio-work-orders.md @@ -0,0 +1,228 @@ +# Calibrate Audio: work orders for phase agents + +You are one of a line of agents, each building **one small phase** of the Calibrate +Audio work. A lead reviews your work when you report back. You have no memory of earlier +phases; what you need is here. This file is working memory for the feature branch, and +the documentation phase removes it. + +**Your context is the budget.** Aim to finish well under 200k tokens. The rules below +say how. They are about not reading huge files whole and not maintaining big documents; +they are **not** a licence to skip what you need to understand. Careful, correct work +comes first. + +## 1. What to read, and what not to + +1. This file, all of it. +2. `AGENTS.md` at the repository root. Its rules apply, except that the **build and test + commands in section 2 below replace its Windows ones**. +3. `docs/calibrate-audio.md` (the spec, about 400 lines): always §1, §5, §10 and §11, + plus the sections your work order names. Search for the heading and read that range. +4. `docs/testing.md`, "What there is to reuse" and "Design principles", if you write app + tests. `docs/recording.md`, "Latency", if you touch the take path. +5. Code: + - `main/MainWindow.cpp` is over 6000 lines and `main/test/TestRecordWorkflow.h` over + 2000. **Never read them whole**: search for the function, then read that range. + - Before writing a test, read an existing test next to where yours will go and copy + its shape. + +## 2. Rules + +**Scope** + +- Build your phase only. If something from a later phase is needed, build the smallest + part of it and say so. +- The spec is agreed with the user. Where it is silent, choose the simpler option and say + so. Where it is **wrong or impossible**, do not improvise another design: finish what + can be finished, leave the tree building and green, and report. +- **Do not edit** `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` (forks) or any other + top-level library directory (`bqaudioio/`, `pyin/`, `vamp-plugin-sdk/`, …). These are + separate repositories, gitignored here. If one needs a change, report exactly which + change; the lead makes it. +- Match the surrounding code: naming, comment density, idiom. Comments say why, in plain + words. Every new source file starts with the project's GPL header (copy it from + `main/TakeTiming.h`). +- Put pure logic in `tony_core` (listed in `meson.build` as `tony_core_files`), as plain + structs and functions like `TakeTiming`. State gets a class and files of its own; + `MainWindow` only wires it. The layer, model and command rules in `AGENTS.md` apply. +- **Qt here is 6.4.2; the user builds with Qt 6.11 on Windows.** Use no Qt API newer than + 6.4, and nothing specific to Linux. + +**Build and test.** This session builds on Linux in `build_linux/`, not the user's +MinGW setup. + +- Build: + + cd /home/user/tony + ninja -j 4 -C build_linux tony test-tony-core test-tony-app > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log + + Search the log for `error:`; never read it whole. meson reconfigures by itself after + `meson.build` changes. Targets have no `.exe`. +- Test, from `build_linux/`: + + mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core > ../tmp/test.log 2>&1; echo "exit:$?" + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app > ../tmp/test.log 2>&1; echo "exit:$?" + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app some_test_name > ../tmp/test.log 2>&1 + grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt + + - Read results from the per-suite files, not stdout. + - A test name on the command line goes to every suite in the executable, so the exit + status of a run with names is meaningless. + - New test classes are registered in `main/test/tony-core-test.cpp` or + `tony-app-test.cpp` and added to `meson.build`. +- While working, run **only your tests**. The app suite takes several minutes of real + time: run both whole suites **once**, at the end, and again only if something failed. + Give the app suite a tool timeout of 10 minutes. +- App tests run in real time against `FakeAudioIO`: keep them short (seconds, not tens + of seconds). +- Every behaviour gets a test that can fail. Show it for the two or three that matter + most by breaking the code for a moment. Undo the break **by hand**: never + `git checkout` or `git restore` a file to revert an experiment. +- Do not weaken or delete an existing test to get green. If one is wrong because the + behaviour was meant to change, change it and say so. +- **Linux traps:** see section 3 for suite results known before this work started. + +**Docs: almost none.** The documentation phase (D) brings `docs/` up to date. You +write only: + +- one entry in the log (section 5), **25 lines at most**, appended at the end; +- "Done" on your phase's line in spec §7, and a correction of any spec statement your + work proved wrong. + +Edit documents with the Edit/Write tools only. A shell heredoc or one-liner containing +backticks, `$` or non-ASCII text gets mangled, and has corrupted documents before. + +**Git.** The lead commits. Leave your work uncommitted, building and green. Never commit, +push, amend, stash, or `git add -A`. + +**Report** (your final message, and all the lead sees; under 60 lines) + +- What was built, by file, briefly. +- The `Totals` lines of the final full runs of both suites, copied, not paraphrased. +- Which tests you saw fail without the change. +- Choices made, deviations, anything fragile or unfinished. Say it plainly: a problem + reported is cheap, one found later is not. + +## 3. State of the code (kept by the lead; as of 2026-09-25, before phase A1) + +- Nothing of the feature exists yet. The spec's §11 lists the facts about today's code + that the phases build on. +- **Linux baseline:** (filled in by the lead after the first build) + +## 4. Phases + +Done: none. + +### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) + +- New `main/LatencyCheck.{h,cpp}` in `tony_core`, pure. +- **Generator.** + - A layout gives the length and the events. Make three: *calibration* (about 26 s), + *dev* (adds 3 s held tones) and *long* (4 minutes). + - Each event: a sweep of 1 → 8 kHz, 200 ms, 10 ms raised-cosine edges, −12 dBFS + peak; then a tone at a pitch with a whole number of samples per period at 44.1 kHz + (196, 220.5, 245, 294 Hz; see `docs/testing.md` on pYIN subharmonics). + - Gaps between events are irregular, 1.6–2.6 s, all different by at least 0.1 s. + - Deterministic. It returns the samples and each sweep's exact frame. + - Exponential or linear sweep: choose one and say why. The spec leans neither way. +- **Finder.** Given take audio (samples and rate) and an expected event time, search + ±0.8 s: + - FFT matched filter against the sweep (bqfft; see how `RealtimePitchTracker` uses + it), weighted to the sweep's band; + - the envelope of the result; + - the **earliest** peak within 6 dB of the largest; + - two confidences: the peak over the window's median in dB, and the peak over the + second-best peak outside ±10 ms in dB; + - the error in frames and in ms. + + The thresholds are named constants (15 dB and 6 dB to start). +- **Not in this phase:** aggregation over events and punch-ins, verdicts, second arrivals. +- **Tests,** in a new core test class `TestLatencyCheck`: + - generator: deterministic, events where it says, gaps all different, peak level; + - finder on synthetic takes, made by shifting the reference: + - shifts across ±0.75 s, plus one fractional shift (resample or interpolate); + - white noise at 0 and −10 dB SNR; + - high-pass at 1 kHz and low-pass at 4 kHz; + - polarity inverted; + - a reflection 6 dB **stronger** 7 ms after the direct sound: the direct sound + must be found; + - silence: no confident peak. + - **Show that the reflection test fails** when the finder takes the largest peak. + +### A2 — Verdicts and calibration arithmetic (spec §5 "tony_core") + +- In `LatencyCheck`: given the events inside a take's coverage ranges (one range per + punch-in), and the finder's results, compute: + - median and spread per punch-in and across punch-ins; + - the slope of offset over position; + - the input peak, for clipping; + - whether confidences fall steadily over time, for fading; + - a **second arrival**: a second confident peak at a consistent extra delay across + events, which means monitoring echo. +- `Verdict`: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent, with + the thresholds of spec §5 as named constants. +- The calibration arithmetic as a pure function: new round trip = used round trip + + median offset, in seconds. A take that lands late was spliced from too early a frame. +- **Tests:** + - one event missing, the rest found; + - fading; + - clipped; + - resampled by 48000/44100 → PositionDependent; + - two punch-ins 20 ms apart → Scattered or Unsteady by threshold; + - monitoring echo detected; + - the arithmetic with both signs. **Show that the sign test fails** when flipped. + +### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) + +To be refined by the lead after A2. + +### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") + +To be refined by the lead after B1. + +### B3 — Calibrate Audio dialog and menu (spec §2) + +To be refined by the lead after B2. + +### C0 — TakeDiff (spec §5 "tony_core") + +To be refined by the lead. + +### C1 — Dev-check framework and first group (spec §3, §4, §5 "development builds only") + +To be refined by the lead. + +### C2 — Observer group (spec §4 items 3, 4, 5, 8, 15, 16) + +To be refined by the lead. + +### C3 — Joins and long song (spec §4 items 9, 10) + +To be refined by the lead. + +### C4 — Smoke group (spec §4 items 19–25, 27, 28) + +To be refined by the lead. + +### D — Documentation pass + +- Bring `docs/` up to date from the code, this log and the spec: + - `recording.md`: the latency section; + - `testing.md`: a Dev checks section, and the loopback fake; + - `manual-checklist.md`: automated items marked with their check's name; + - `architecture.md`, if new classes change who owns what; + - `open-points.md`: remove the item, and add what is left open. +- Make `docs/calibrate-audio.md` describe what was built, with a "Known limitations and + open points" section. +- Delete this work-orders file and remove the spec's link to it. +- No code. Suspected bugs go in the report. + +## 5. Log (newest last; 25 lines at most per entry) + +Template: + + ### Phase — + Built: ... + Choices / deviations: ... + The next phase must know: ... + Left open: ... diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 362a3984..cf099880 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -262,21 +262,35 @@ setting changes only through Use this latency. ## 7. Order of work -Every step ends with both whole suites green; commits only when asked. - -1. **Core:** `LatencyCheck`, `TakeDiff`, `LatencyCalibration` and their tests. -2. **Runner, dialog and calibration page** (every build), with app tests. **Then you run - it on your PC.** Its numbers settle three things before the rest is built: how wrong - the driver's figure is, whether the offset holds across stream restarts on MME, and - whether your device's rate hits the takes. -3. **Calibration in use:** `recordingStarted()`, Use this latency, Forget, staleness, - app tests. -4. **Dev-check framework:** build flag, `DevChecks`, `TakeObserver`, report, friend - access. First group: items 1, 2, 7, 12, 13, 14. -5. **Observer group:** items 3, 4, 5, 8, 15, 16. -6. **Join and long-song group:** items 9 and 10. -7. **Smoke group:** optional. -8. **Docs,** in the same commit as the code they describe: +Every step ends with both whole suites green. The work is split into phases for a line +of agents; their work orders are in +[calibrate-audio-work-orders.md](calibrate-audio-work-orders.md). Each phase's line is +marked "Done" when it is committed. + +1. **Core.** + - **A1** Test reference and sweep finder: `LatencyCheck` generator and per-event + analysis. + - **A2** Verdicts and calibration arithmetic: aggregation over events and punch-ins. +2. **Runner, dialog and calibration page** (every build), with app tests. + - **B1** The alignment check runner and its app tests. + - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, + `recordingStarted()`, staleness). This was step 3 below; it moved up because the + dialog needs it. + - **B3** The Calibrate Audio dialog and menu entry. + + **Then you run it on your PC.** Its numbers settle three things: how wrong the + driver's figure is, whether the offset holds across stream restarts on MME, and + whether your device's rate hits the takes. Work goes on meanwhile: only the + thresholds and the restart-jitter remedy wait on those numbers. +3. **Calibration in use:** built in B2 (Use this latency and Forget in B3). +4. **Dev-check framework:** + - **C0** `TakeDiff`, pure. + - **C1** Build flag, `DevChecks`, `TakeObserver`, report, friend access. First + group: items 1, 2, 7, 12, 13, 14. +5. **C2** Observer group: items 3, 4, 5, 8, 15, 16. +6. **C3** Join and long-song group: items 9 and 10. +7. **C4** Smoke group. +8. **D** Docs, from the code and the phase log: - `manual-checklist.md`: an automated item keeps its text and gets "*automated: dev check ``*"; a measured item keeps only the question for a person. The list becomes what a person must do after a dev run. @@ -318,3 +332,72 @@ could convert. The button then shows the fix working on each device. - **A quick re-measure** after a Bluetooth reconnect, without a test session. - **Items 18 and 6**, as app-suite tests: synthetic mouse events, and a fake device that fails to open. + +## 10. Decisions + +| Question | Decision | +| --- | --- | +| Where the check runs | In a session of its own, opened from a generated WAV; the user is asked to save first | +| How the round trip is measured | Through ordinary takes (§2), not a separate audio IO | +| When a measured figure is used | After **Use this latency**; a dev run uses the new figure for itself only | +| Dev mode | Any build type that does not start with `release` (`TONY_DEV_CHECKS`) | +| Form of a dev check | A function returning a plain `CheckResult`, not a QtTest function | +| Checkpoint after B3 | The user runs it on Windows when they can; C0 onwards does not wait | +| Commits | The lead commits each phase after review and pushes `feat/calibrateaudiotests` | + +## 11. Facts checked in the code + +Checked on 2026-09-25, so that phases do not re-derive them. + +- **The latency today.** In `MainWindow::recordingStarted()`'s deferred lambda: L = + `computeRecordingLatency(getTargetPlayLatency(), getSystemRecordLatency())` + start + gap. It applies only with Play Reference While Recording on; otherwise L = 0. The + start gap comes from the play-start callback set in `MainWindow`'s constructor: + `getFramesReceived() − blockFrames` on the first output block with audio. + `refineRecordingLatency()` and `currentRecordingLatency()` swap the estimate for the + measurement. +- **bqaudioio `PortAudioIO`** (upstream, not a fork): + - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; + - input goes to the record target **before** output is asked for, in the same + callback; + - `suspend()`/`resume()` are `Pa_StopStream`/`Pa_StartStream`. `MainWindowBase::stop()` + suspends and `record()` resumes, so **every take restarts the stream**; + - it exposes no device names and ignores PortAudio's callback time info. +- **Device rate.** + - `AudioCallbackPlaySource::getApplicationSampleRate()` and + `AudioCallbackRecordTarget::getApplicationSampleRate()` both return 0, so the device + opens at PortAudio's default rate. + - `ResamplerWrapper` resamples the play source to it. The record target records at it. + - `MainWindow` sets `Preferences::setFixedSampleRate(44100)`. + - `TakeAudio::splice()` writes a take's first recording at the recording's rate + without converting positions. Later recordings are refused only when their rate + differs from the take file's. + - PortAudio's MME default rate is the first of {44100, 48000, …} that the device + accepts. +- **Device choice.** `getDeviceIndex()` takes the first PortAudio device with the given + name, across host APIs. MME names are cut to 31 characters. +- **Settings the check must not write.** The toggles `m_recordIntoSelection` + (`MainWindow/recordintoselection`), `m_playRefWhileRecording` and `m_preRoll` write + QSettings when toggled. `wantedPreRollFrames()` reads `MainWindow/prerollseconds`. +- **Opening a reference.** + - The tests use `openPath(path, MainWindow::ReplaceSession)` after + `discardModifications()`. + - `checkSaveModified()` is what asks the user to save. + - The reference is analysed when `Analyser::getInitialAnalysisCompletion() >= 100` and + the layers exist. See `analysed()` in `TestRecordWorkflow.h`. +- **The take after Stop.** + - Its audio is the model `analyser2()->getMainModelId()`, and its file is + `m_takes->getAudioPath()`. + - Coverage is `m_takes->getCoverage().getRanges()`. + - The take is analysed when `analysed(analyser2())` holds. +- **Levels.** `getOutputLevels()` and `getInputLevels()`, on the play source and record + target, return per-channel peaks since the last call. +- **Fake device.** `FakeAudioIO::Config::loopback` adds the output to the input + `inputDelay` frames late; no test uses it yet. The reported latencies are independent + of the real delay. `TestMainWindow::createAudioIO()` installs the fake. +- **Menus.** The Playback menu is built in `MainWindow::setupToolbars()` + (`m_playbackMenu`). The audio device submenus are there too. +- **Build types.** `build.bat` uses `debugoptimized`; `meson.build` defaults to + `release`, which the deploy scripts use. `meson.build` already switches on + `buildtype.startswith('release')` for `WANT_TIMING` / `NO_TIMING`. +- **Qt Test** is already in `qt_dep`'s modules, so every target links it. From e2cf7c0cd8162dfe266c7abe2650827a5e25648c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 19:28:10 +0000 Subject: [PATCH 076/275] fix: connect slots taking sv types by member pointer Four connections in Analyser and one in MainWindow named their slots in SIGNAL()/SLOT() strings with ModelId or sv_frame_t, while moc records the slots as taking sv::ModelId and sv::sv_frame_t. Qt 6.4 does not match the two ("No such slot"), so the analyser never heard that pYIN had finished and TestSingingAnalysis timed out; the Qt the project is developed with happens to match them. Member pointers are checked at compile time and match under any version, as the rule in docs/architecture.md already asks. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/Analyser.cpp | 25 +++++++++++++++---------- main/MainWindow.cpp | 6 ++++-- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index e7cbf079..9b8cd77e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -908,19 +908,23 @@ Analyser::connectAnalysisLayers() // Claimed layers need these as much as ones we made ourselves: the // analyser they belonged to before is gone, and with it its // connections. Unique, because an analyser handed the same layers - // twice would otherwise hear each signal twice + // twice would otherwise hear each signal twice. By member pointer + // where the arguments are sv types: moc records this class's slots + // as taking sv::ModelId and sv::sv_frame_t, and whether a SLOT() + // string saying ModelId matches that depends on the Qt version (6.4 + // says "No such slot") if (auto pitchLayer = qobject_cast(m_layers[PitchTrack])) { - connect(pitchLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId)), + connect(pitchLayer, &Layer::modelCompletionChanged, + this, &Analyser::layerCompletionChanged, Qt::UniqueConnection); } if (auto noteLayer = qobject_cast(m_layers[Notes])) { - connect(noteLayer, SIGNAL(modelCompletionChanged(ModelId)), - this, SLOT(layerCompletionChanged(ModelId)), + connect(noteLayer, &Layer::modelCompletionChanged, + this, &Analyser::layerCompletionChanged, Qt::UniqueConnection); - connect(noteLayer, SIGNAL(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), - this, SLOT(reAnalyseRegion(sv_frame_t, sv_frame_t, float, float)), + connect(noteLayer, &FlexiNoteLayer::reAnalyseRegion, + this, &Analyser::reAnalyseRegion, Qt::UniqueConnection); connect(noteLayer, SIGNAL(materialiseReAnalysis()), this, SLOT(materialiseReAnalysis()), @@ -1170,9 +1174,10 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, auto model = ModelById::get(id); if (!model) continue; // Emitted on the transform's own thread, so delivered here as a - // queued call: the merge happens on this thread like any other - connect(model.get(), SIGNAL(completionChanged(ModelId)), - this, SLOT(rangedAnalysisCompletionChanged(ModelId))); + // queued call: the merge happens on this thread like any other. + // By member pointer, as in connectAnalysisLayers() + connect(model.get(), &Model::completionChanged, + this, &Analyser::rangedAnalysisCompletionChanged); } // createDerivedLayers() returns only once the transform has set both diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 534cf8a7..b1e52653 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -291,8 +291,10 @@ MainWindow::MainWindow(AudioMode audioMode, // We have a pane stack: it comes with the territory. However, we // have a fixed and known number of panes in it -- it isn't // variable - connect(m_paneStack, SIGNAL(doubleClickSelectInvoked(sv_frame_t)), - this, SLOT(doubleClickSelectInvoked(sv_frame_t))); + // By member pointer: the slot takes sv::sv_frame_t, which a SLOT() + // string saying sv_frame_t does not match under every Qt version + connect(m_paneStack, &PaneStack::doubleClickSelectInvoked, + this, &MainWindow::doubleClickSelectInvoked); scroll->setWidget(m_paneStack); m_overview = new Overview(frame); From b2ae57c1b87097ab32afbb6deddf513279f8da3d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:43:22 +0000 Subject: [PATCH 077/275] docs: calibrate audio work orders, linux build traps and baseline The app suite needs pyin.so beside it or it hangs until the QtTest watchdog fires; Qt 6.4 does not match string connects naming sv types; four TestTakesFile tests use Windows paths and fail on Linux only. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index ae63b45a..0a2d72fe 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -53,10 +53,13 @@ MinGW setup. - Build: cd /home/user/tony - ninja -j 4 -C build_linux tony test-tony-core test-tony-app > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log + ninja -j 4 -C build_linux tony test-tony-core test-tony-app pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log Search the log for `error:`; never read it whole. meson reconfigures by itself after `meson.build` changes. Targets have no `.exe`. +- **`pyin.so` must be built too.** The app suite sets `VAMP_PATH` to its own directory. + Without the plugin, every test that waits for pitch analysis hangs until QtTest's + 5-minute watchdog aborts the run. - Test, from `build_linux/`: mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt @@ -80,7 +83,12 @@ MinGW setup. `git checkout` or `git restore` a file to revert an experiment. - Do not weaken or delete an existing test to get green. If one is wrong because the behaviour was meant to change, change it and say so. -- **Linux traps:** see section 3 for suite results known before this work started. +- **Linux traps:** + - Section 3 lists suite results known before this work started. + - Qt 6.4 does not match a `SIGNAL()`/`SLOT()` string naming `ModelId` or `sv_frame_t` + with moc's `sv::` names: the connection fails at run time with "No such slot". The + user's Qt happens to match them, so a string connect can pass there and fail here. + Use member-pointer `connect` only, and grep test output for "No such slot". **Docs: almost none.** The documentation phase (D) brings `docs/` up to date. You write only: @@ -107,7 +115,13 @@ push, amend, stash, or `git add -A`. - Nothing of the feature exists yet. The spec's §11 lists the facts about today's code that the phases build on. -- **Linux baseline:** (filled in by the lead after the first build) +- **Linux baseline** (after the lead cherry-picked the member-pointer connect fix from + the lyrics branch, `e2cf7c0`): + - core suite: all green except 4 tests in `TestTakesFile` (`takes_folder`, + `relative_audio_path`, `resolve_audio_path`, `in_folder`). They use Windows paths + (`C:\Songs\...`, case-insensitive) and fail on Linux only. Not yours to fix; your + final runs must show exactly these 4 and nothing else. + - app suite: see the log's lead entry after the first full run. ## 4. Phases From e13fb9a0809958ca628a7e592ee89e19d46600c4 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:49:37 +0000 Subject: [PATCH 078/275] test: queue the stale pitch event as a functor stale_pitch_event_ignored queued onRealtimePitchDetected by name with Q_ARG(sv::sv_frame_t), which Qt 6.4 does not match against the slot moc recorded, so invokeMethod returned false and the test failed before testing anything. A queued functor calling the same slot is checked at compile time. Removing the slot's guard still makes it fail. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- main/test/TestRecordWorkflow.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d1614237..9e6b68ce 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1161,10 +1161,14 @@ private slots: // it, so it would make a dot. The tracker may have queued // events of its own as well; the same goes for them QCOMPARE(m_window->recordingLatencyFrames(), sv::sv_frame_t(0)); + // Queued as a functor: invoking the slot by name depends on the + // Qt version matching "sv::sv_frame_t" against what moc recorded, + // and Qt 6.4 does not + TestMainWindow *window = m_window; QVERIFY2(QMetaObject::invokeMethod - (m_window, "onRealtimePitchDetected", Qt::QueuedConnection, - Q_ARG(sv::sv_frame_t, sv::sv_frame_t(20000)), - Q_ARG(double, 440.0)), + (m_window, [window]() { + window->doRealtimePitchDetected(20000, 440.0); + }, Qt::QueuedConnection), "the pitch event could not be queued"); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); From 3318c026afbffa521fb189368879a653be95c982 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 22:49:45 +0000 Subject: [PATCH 079/275] docs: calibrate audio work orders, the app suite baseline on linux Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 0a2d72fe..fb1641e5 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -121,7 +121,9 @@ push, amend, stash, or `git add -A`. `relative_audio_path`, `resolve_audio_path`, `in_folder`). They use Windows paths (`C:\Songs\...`, case-insensitive) and fail on Linux only. Not yours to fix; your final runs must show exactly these 4 and nothing else. - - app suite: see the log's lead entry after the first full run. + - app suite: all green, `TestRecordWorkflow` 92 tests in about 4.5 minutes. The lead + also made `stale_pitch_event_ignored` queue its event as a functor (`e13fb9a`), + because invoking a slot by name with an `sv::` argument type fails under Qt 6.4. ## 4. Phases From 8b9f07b602e03695572dd58b417efb470f32d240 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:09:30 +0000 Subject: [PATCH 080/275] docs: building on linux Two cloud sessions have had to work out how to build and test Tony on Linux. The steps are now in docs/building.md: the packages, the libraries at their pins (the six Mercurial ones from their GitHub mirrors, as the cloud proxy refuses hg.sr.ht), the targets, and the traps of Qt 6.4. build_linux/ is ignored like the other build folders. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- .gitignore | 1 + AGENTS.md | 4 ++- docs/README.md | 2 +- docs/building.md | 94 ++++++++++++++++++++++++++++++++++++++++++++++-- 4 files changed, 97 insertions(+), 4 deletions(-) diff --git a/.gitignore b/.gitignore index 96440733..a19192f4 100644 --- a/.gitignore +++ b/.gitignore @@ -37,6 +37,7 @@ build build_win32 build_win64 build_win64_debug +build_linux *.AppImage *.AppDir Dockerfile*.gen diff --git a/AGENTS.md b/AGENTS.md index 44e4d87a..1c7b1e54 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -23,7 +23,9 @@ code. ## Build and test -From **Git Bash** (the usual agent shell). `build.bat` does not run from sh. +From **Git Bash** (the usual agent shell) on the Windows machine. `build.bat` does not run +from sh. In a Linux cloud session, set up and build as in +[docs/building.md](docs/building.md#building-on-linux) instead, into `build_linux/`. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" diff --git a/docs/README.md b/docs/README.md index 6eea96bc..8b19d1af 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,7 +11,7 @@ methods, and they do not tell the story of fixed bugs. | Page | What is in it | | --- | --- | -| [building.md](building.md) | The MinGW build, and every way the environment has gone wrong | +| [building.md](building.md) | The MinGW build, and every way the environment has gone wrong; the Linux build of a cloud session | | [testing.md](testing.md) | The two test executables, the fakes and helpers, how tests turned out to be worthless, races | | [architecture.md](architecture.md) | What the fork adds, `tony_core` / `tony_app`, who owns what, and the rules for layers, models, commands, playback and session files | | [recording.md](recording.md) | Record to Stop, step by step and why in that order; latency; pre-roll; record into selection; the live tracker | diff --git a/docs/building.md b/docs/building.md index 8d77e1d7..95626691 100644 --- a/docs/building.md +++ b/docs/building.md @@ -1,10 +1,14 @@ -# Building on Windows (MSYS2 MinGW-w64) +# Building The development machine builds with meson + ninja under MSYS2's `mingw64` toolchain -(default prefix `C:\msys64\mingw64`) into `build_mingw/`. The CI workflows in +(default prefix `C:\msys64\mingw64`) into `build_mingw/`: that is most of this page. An +agent in a cloud session builds on Linux instead, into `build_linux/`: see +[Building on Linux](#building-on-linux) at the end. The CI workflows in `.github/workflows/` build the upstream way on Linux, macOS and MSVC and are not what is described here. +# Building on Windows (MSYS2 MinGW-w64) + ## From cmd or PowerShell: `build.bat` ``` @@ -79,3 +83,89 @@ echo "exit:$?" >> tmp/build.log `tony_core_files` or `tony_app_files`, and its header into the matching `*_moc_files` only if it declares `Q_OBJECT`. - Windows headers define `near` and `far` as macros. Do not use them as identifiers. + +# Building on Linux + +For a cloud session (Ubuntu 24.04, root, no Windows): a fresh container has none of the +libraries, and repoint does not run there either, so everything below is done by hand. +A clean build takes 15 to 20 minutes with `-j 4`. + +## Packages + +```sh +apt-get update +DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential meson ninja-build \ + qt6-base-dev qt6-base-dev-tools qt6-tools-dev-tools qt6-pdf-dev libqt6svg6-dev \ + libboost-all-dev libbz2-dev libfftw3-dev libsndfile-dev libsamplerate-dev \ + librubberband-dev libsord-dev raptor2-utils liboggz2-dev libfishsound1-dev \ + libmad0-dev libid3tag0-dev libopus-dev libopusfile-dev libopusenc-dev liblo-dev \ + liblrdf0-dev libjack-jackd2-dev libpulse-dev libasound2-dev portaudio19-dev \ + capnproto libcapnp-dev libglib2.0-dev libxml2-utils mercurial +``` + +Ubuntu 24.04 gives Qt **6.4** and rubberband 3.3; the Windows machine has Qt 6.11. See the +traps below. + +## The libraries + +Check out every library of `repoint-project.json` at its pin in `repoint-lock.json`, into +the directory of the same name at the top of the repository (`icons/scalable` included). +The git ones: + +```sh +cd /path/to/tony +python3 -c ' +import json +p = json.load(open("repoint-project.json"))["libraries"] +l = json.load(open("repoint-lock.json"))["libraries"] +for name, lib in p.items(): + if lib["vcs"] == "git": + repo = lib.get("repository", name.split("/")[-1]) + print(name, "https://github.com/%s/%s" % (lib["owner"], repo), l[name]["pin"]) +' | while read dir url pin; do + [ -d "$dir/.git" ] || git clone -q "$url" "$dir" + git -C "$dir" checkout -q "$pin" && echo "$dir $pin" +done +``` + +The six Mercurial ones (`dataquay`, `bqvec`, `bqfft`, `bqresample`, `bqaudioio`, +`bqthingfactory`) live on `hg.sr.ht/~breakfastquay`. **A cloud session's network proxy +refuses that host**, so use the git mirrors at `github.com/breakfastquay/` instead. +The mirrors' hashes are not the hg pins: take each mirror's `HEAD`, which matched the pins +when this was written (for `bqaudioio`, its "Merge from branch toggle-record-in-io" is +hg `017ab3ed3a33`, which `FakeAudioIO` needs). If a pin moves past a mirror's `HEAD`, the +build or the tests will say so. + +```sh +for r in dataquay bqvec bqfft bqresample bqaudioio bqthingfactory; do + [ -d "$r" ] || git clone -q "https://github.com/breakfastquay/$r" "$r" +done +``` + +Where `hg.sr.ht` can be reached, `hg clone` and `hg update -r ` are the real thing. If +`hg` fails with `ImportError: cannot import name parsers`, a Python on `PATH` other than the +system's is picking it up: run it as `/usr/bin/python3.12 /usr/bin/hg`. + +## Configure, build, test + +```sh +meson setup build_linux --buildtype release > tmp/configure.log 2>&1 +ninja -j 4 -C build_linux tony test-tony-core test-tony-app pyin.so > tmp/build.log 2>&1 +echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log +``` + +Targets have no `.exe`, and `pyin.so` has to be named: nothing else builds the plugin the +app suite loads. The rules above still hold: log to a file, never pipe ninja, write the exit +status into the log. Tests run from `build_linux/` exactly as [testing.md](testing.md) +says, with `./test-tony-core` and `./test-tony-app`. + +## Traps + +- **Some tests fail on Linux whatever the change**: the list is in + [testing.md](testing.md#running). Record the baseline before changing anything. +- **Qt 6.4 does not match a `SIGNAL()`/`SLOT()` string saying `ModelId` or `sv_frame_t` + against a slot moc recorded with `sv::`**, where Qt 6.11 does. Such a connection fails + silently here and works on Windows. Use member-pointer `connect` (AGENTS.md asks for it + anyway), and use no Qt API newer than 6.4. +- Several tests race the analysis against the take; this machine finishes the analysis + sooner than the Windows one does, which is why some of them fail here. From d4546ebd717e0023c40a6e1479fb7d38d5a98bfc Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:09:41 +0000 Subject: [PATCH 081/275] docs: the android apk is built in the container, not by a workflow the repository's workflows are turned off, so phase a3 writes a build script and the lead hands the apk over. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 8346e877..64043b1f 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -132,8 +132,8 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: Order: A0, A1, A2, A3, then A4 and A5 while the user tries the APK on the phone. A6 needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt.io` and -`dl.google.com` are reachable from the container, and GitHub Actions is enabled on -`jhhr/tony`.) +`dl.google.com` are reachable from the container. GitHub workflows are turned off: all +builds happen in the container.) - A0 — Desktop build and tests in the container. Done. - A1 — Sample rate: a device that is not at 44.1 kHz. @@ -204,7 +204,7 @@ tests". - Leave out JACK, PulseAudio, ALSA, PortAudio, oggz and fishsound; note any other library that turns out to be needed. - The toolchain is installed into the container (outside the repo, e.g. under `/opt`), so - that A3 can iterate locally. The CI workflow comes in A3, once there is an APK to build. + that A3 can iterate locally. ### A3 — Tony as an APK, without audio: the test port @@ -221,9 +221,8 @@ Read: [port-android.md](port-android.md) all of "Platform facts" and "Test port" - pYIN found on the phone: `libpyin.so` naming plus legacy packaging, or linking it in. The log must show "Setting VAMP_PATH to ...". - The desktop build and suites unchanged and green. -- A CI workflow `.github/workflows/android.yml` (Linux runner, triggered by pushes to - `feat/tonyandroid` and by hand) that builds the APK the same way and uploads it as an - artifact. The lead pushes and reads the run; write it so it can only be judged there. +- No CI workflow: the user has turned the repository's workflows off. The build is a + script that runs in the container. - Result: an APK the user can sideload; the lead hands it over. ### A4 — Touch gestures on the panes From d4756491f66d8de849266ecc7e2deb67cecdde61 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:20:15 +0000 Subject: [PATCH 082/275] test: lyrics along the bottom, in boxes, with the word being sung The svgui pin moves to e2c736d, the second version of the lyrics plot style: light boxes as long as each word along the bottom of the pane, text centred, a font at least twice the view's that grows as the view zooms in, and the word at a given frame in amber, shown over the others if it has no room of its own. TestLyricsLayer now checks the band is along the bottom and leaves the coverage strip's pixels alone, where a box goes, how the font follows the zoom, that a label is centred in its box, and that the highlight repaints only when the word changes, changes nothing but its own box, and is shown even for a word that had no row. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/test/TestLyricsLayer.h | 198 ++++++++++++++++++++++++++++++++++-- repoint-lock.json | 2 +- 2 files changed, 190 insertions(+), 10 deletions(-) diff --git a/main/test/TestLyricsLayer.h b/main/test/TestLyricsLayer.h index 372231a3..46f7d407 100644 --- a/main/test/TestLyricsLayer.h +++ b/main/test/TestLyricsLayer.h @@ -15,9 +15,10 @@ #define TEST_LYRICS_LAYER_H // Tier 3: the lyrics plot style of RegionLayer (svgui fork), which -// draws the words of the lyrics along the top of pane 0. Where each -// label goes is worked out by a pure function, tested first; then the -// layer is painted into images, with no MainWindow. +// draws the words of the lyrics along the bottom of pane 0. Where each +// box goes and how big its font is are worked out by pure functions, +// tested first; then the layer is painted into images, with no +// MainWindow. #include "framework/Document.h" #include "view/Pane.h" @@ -30,10 +31,12 @@ #include #include +#include #include #include #include +#include #include #include @@ -53,7 +56,7 @@ class TestLyricsLayer : public QObject static constexpr double kRate = 44100.0; static constexpr int kWidth = 800; - static constexpr int kHeight = 200; + static constexpr int kHeight = 300; sv::ModelId makeAudioModel() { auto model = std::make_shared @@ -102,6 +105,40 @@ class TestLyricsLayer : public QObject return true; } + // Words two seconds apart and a second and a half long, with short + // labels: at the tests' zoom each box is 150 pixels of region with + // its label well inside it, and nothing is left out + void addSpacedWords(int count) { + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + for (int i = 0; i < count; ++i) { + model->add(sv::Event(sv::sv_frame_t(kRate * 2.0 * i), 0.f, + sv::sv_frame_t(kRate * 1.5), + QString("w%1").arg(i))); + } + } + + sv::Event spacedWord(int i) { + return sv::Event(sv::sv_frame_t(kRate * 2.0 * i), 0.f, + sv::sv_frame_t(kRate * 1.5), QString("w%1").arg(i)); + } + + // The columns in which two images differ, as [first, last], or + // (-1, -1) if they are the same + static std::pair differingColumns(const QImage &a, const QImage &b) { + int first = -1, last = -1; + for (int x = 0; x < a.width(); ++x) { + for (int y = 0; y < a.height(); ++y) { + if (a.pixel(x, y) != b.pixel(x, y)) { + if (first < 0) first = x; + last = x; + break; + } + } + } + return { first, last }; + } + private slots: void initTestCase() { QVERIFY(m_dir.isValid()); @@ -170,6 +207,41 @@ private slots: std::vector({ 0, 1 })); } + void a_label_reaching_back_over_the_one_before_goes_to_the_next_row() { + // A long label centred on a short region can start left of the + // label of the region before it + Spans spans { { 100, 10 }, { 60, 100 }, { 115, 10 } }; + QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), + std::vector({ 0, 1, 0 })); + } + + void a_box_is_its_region_or_its_label_centred_on_the_region() { + typedef std::pair Span; + QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 200, 50), Span(100, 100)); + QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 200, 100), Span(100, 100)); + QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 120, 60), Span(80, 60)); + } + + void the_font_grows_with_the_zoom_from_twice_to_four_times() { + // Base 13 pixels in a tall view: 26 to 52, a pixel for every 10 + // pixels per second in between + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(50, 13, 800), 26); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(300, 13, 800), 30); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(400, 13, 800), 40); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(5000, 13, 800), 52); + // Never more than an eighth of the view, nor less than its font + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(5000, 13, 200), 25); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(50, 13, 200), 25); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(50, 13, 40), 13); + + int last = 0; + for (double pps = 10; pps < 2000; pps *= 1.3) { + int size = sv::RegionLayer::getLyricsFontPixelSize(pps, 13, 600); + QVERIFY2(size >= last, "the font got smaller as the view zoomed in"); + last = size; + } + } + void no_labels_and_no_rows() { QCOMPARE(sv::RegionLayer::assignLabelRows({}, 2, 4), std::vector()); QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 } }, 0, 4), @@ -189,21 +261,129 @@ private slots: QCOMPARE(m_layer->getFeatureDescription(m_pane, pos), QString()); } - void lyrics_are_drawn_only_along_the_top() { + void lyrics_are_drawn_only_along_the_bottom() { addWords(20); QImage image = render({ QRect(0, 0, kWidth, kHeight) }); bool drawn = false; - for (int y = 0; y < kHeight / 4; ++y) { + for (int y = kHeight * 3 / 4; y < kHeight; ++y) { if (!rowIsWhite(image, y)) drawn = true; } - QVERIFY2(drawn, "nothing was drawn along the top of the pane"); + QVERIFY2(drawn, "nothing was drawn along the bottom of the pane"); + + for (int y = 0; y < kHeight / 2; ++y) { + QVERIFY2(rowIsWhite(image, y), + qPrintable(QString("something was drawn at y = %1, " + "above the band of the lyrics").arg(y))); + } - for (int y = kHeight / 2; y < kHeight; ++y) { + // The coverage strip's band, the bottom six pixels, is left to it + for (int y = kHeight - m_pane->scalePixelSize(6); y < kHeight; ++y) { QVERIFY2(rowIsWhite(image, y), qPrintable(QString("something was drawn at y = %1, " - "below the band of the lyrics").arg(y))); + "where the coverage strip goes").arg(y))); + } + } + + void a_label_is_centred_in_its_box() { + addSpacedWords(3); + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + + // The dark pixels of the text of the second word, which spans + // x = 200 to 350, above the bars + int x0 = m_pane->getXForFrame(spacedWord(1).getFrame()); + int x1 = m_pane->getXForFrame(spacedWord(1).getFrame() + + spacedWord(1).getDuration()); + int left = -1, right = -1; + for (int x = x0; x <= x1; ++x) { + for (int y = kHeight / 2; y < kHeight - 12; ++y) { + if (qGray(image.pixel(x, y)) < 100) { + if (left < 0) left = x; + right = x; + break; + } + } } + QVERIFY2(left >= 0, "no text was found in the box"); + double textMiddle = (left + right) / 2.0; + double boxMiddle = (x0 + x1) / 2.0; + QVERIFY2(std::fabs(textMiddle - boxMiddle) <= 3.0, + qPrintable(QString("the text is centred on x = %1, the box " + "on x = %2").arg(textMiddle).arg(boxMiddle))); + } + + void the_highlight_follows_the_frame_and_repaints_only_for_a_new_word() { + addSpacedWords(4); + QSignalSpy repaints(m_layer, &sv::Layer::layerParametersChanged); + sv::Event e(0); + + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 2.1)); + QCOMPARE(int(repaints.count()), 1); + QVERIFY(m_layer->getHighlightedEvent(e)); + QVERIFY(e == spacedWord(1)); + + // Within the same word: nothing to paint again + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 3.0)); + QCOMPARE(int(repaints.count()), 1); + + // Between words: none + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 3.7)); + QCOMPARE(int(repaints.count()), 2); + QVERIFY(!m_layer->getHighlightedEvent(e)); + m_layer->setHighlightFrame(-1); + QCOMPARE(int(repaints.count()), 2); + + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 4.0)); + QCOMPARE(int(repaints.count()), 3); + QVERIFY(m_layer->getHighlightedEvent(e)); + QVERIFY(e == spacedWord(2)); + } + + void the_highlighted_word_is_drawn_differently_and_nothing_else_is() { + addSpacedWords(4); + QImage plain = render({ QRect(0, 0, kWidth, kHeight) }); + + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 2.1)); + QImage highlighted = render({ QRect(0, 0, kWidth, kHeight) }); + + auto columns = differingColumns(plain, highlighted); + QVERIFY2(columns.first >= 0, "the highlighted word looks the same"); + int x0 = m_pane->getXForFrame(spacedWord(1).getFrame()); + int x1 = m_pane->getXForFrame(spacedWord(1).getFrame() + + spacedWord(1).getDuration()); + QVERIFY2(columns.first >= x0 && columns.second <= x1, + qPrintable(QString("pixels changed from x = %1 to %2, outside " + "the word's box at %3 to %4") + .arg(columns.first).arg(columns.second) + .arg(x0).arg(x1))); + } + + // The highlight's box colour, and nothing else drawn is like it + static int highlightPixels(const QImage &image) { + int n = 0; + for (int y = 0; y < image.height(); ++y) { + for (int x = 0; x < image.width(); ++x) { + QRgb p = image.pixel(x, y); + if (qRed(p) > 240 && qGreen(p) > 195 && qGreen(p) < 225 && + qBlue(p) > 100 && qBlue(p) < 150) ++n; + } + } + return n; + } + + void the_highlighted_word_is_shown_even_with_no_room_of_its_own() { + // Words this close fill both rows, and the third has no room + addWords(3); + QImage plain = render({ QRect(0, 0, kWidth, kHeight) }); + QCOMPARE(highlightPixels(plain), 0); + + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 0.55)); + sv::Event e(0); + QVERIFY(m_layer->getHighlightedEvent(e)); + QCOMPARE(e.getLabel(), QString("sanaseppo2")); + QImage highlighted = render({ QRect(0, 0, kWidth, kHeight) }); + QVERIFY2(highlightPixels(highlighted) > 100, + "the word being sung was not drawn: it had no row"); } void painting_in_strips_matches_painting_whole() { diff --git a/repoint-lock.json b/repoint-lock.json index d8911c46..bef369da 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "62af60f4e746b87b9a4896a0569603f8cb56694c" + "pin": "e2c736d08ec5150b1d74c5e8b6052360a966777c" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 944df7c4c019dd40af339d2c881e84dfe3813602 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:22:29 +0000 Subject: [PATCH 083/275] feat: the calibrate audio test reference and sweep finder LatencyCheck makes the reference a calibration plays: linear sweeps of 1 to 8 kHz at irregular spacings, each followed by a tone pYIN can track. findSweep() locates a sweep in a take with an FFT matched filter, takes the earliest peak within 6 dB of the largest so that a stronger reflection does not win, and says how sure it is. It works at the take's own rate. Pure functions, tested in TestLatencyCheck; the reflection, 48 kHz, polarity and notched-band tests were seen to fail with the code broken. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 17 + docs/calibrate-audio.md | 4 +- main/LatencyCheck.cpp | 335 +++++++++++++++++ main/LatencyCheck.h | 206 ++++++++++ main/test/TestLatencyCheck.h | 564 ++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 7 files changed, 1134 insertions(+), 1 deletion(-) create mode 100644 main/LatencyCheck.cpp create mode 100644 main/LatencyCheck.h create mode 100644 main/test/TestLatencyCheck.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index fb1641e5..db8401dd 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -242,3 +242,20 @@ Template: Choices / deviations: ... The next phase must know: ... Left open: ... + +### Phase A1 — 2026-09-25 +Built: `main/LatencyCheck.{h,cpp}` in `tony_core`: `calibrationLayout()`, `devLayout()`, `longLayout()` (rate defaults to 44100), `sweep(rate)`, `generate(layout)`, `findSweep(samples, count, rate, expectedSeconds)` → `Arrival {found, errorFrames, errorSeconds, peakOverMedianDb, peakOverSecondDb, levelDb, inputPeak}`. Thresholds are `k…` constants in the header. `main/test/TestLatencyCheck.h`: 16 tests, 1.4 s. +Choices / deviations: +- Linear sweep: flat spectrum, narrowest peak; an exponential sweep's harmonics match it 67/106 ms *early*, where the earliest-peak rule looks. +- Envelope = magnitude of the analytic signal (a second inverse FFT gives the Hilbert part). +- Earlier peak counts if it is a local maximum, within 6 dB, and ≥ 1 ms before the largest. No dip rule: one arrival with a hole in its band beats, with deep dips, so only time tells arrivals apart (`finder_takes_one_arrival_as_one`). +- No band mask beyond the matched filter itself: a 0 dBFS 100 Hz hum already comes through 100 dB down; a mask changed nothing measurable. +- Confidences are the chosen (earliest) peak's; "second" is the envelope's maximum more than 10 ms from it. +- A gap is sweep to sweep. Calibration sweeps at 1.0, 3.1, 4.7, 7.2, 9.1, 11.4, 13.1, 15.7, 17.7, 19.5, 21.9, 24.1 s; each tone starts 0.3 s after its sweep, 0.8 s long. Dev adds 3 s held tones after sweeps at 26.9, 30.9, 35.2 s (40 s). Long: 113 events, 240 s. +- Reflection test at 5.5 dB (direct found) and 7 dB (reflection taken): exactly 6 dB passes here only by rounding (6.1 dB flips). +The next phase must know: +- Pass the take's samples at their own rate, and the expected time in seconds (layout frame / layout rate). +- A take of 48 kHz samples read as 44.1 kHz (sweeps stretched 8.8%) finds nothing: level −22 dB, 0.1 dB over the second. `findSweep` at 48000 finds them. A2's "resampled by 48000/44100" case must be built as misplaced frames, or try both rates. +- Measured: noise alone 6–12 dB over the median (threshold 15); SNR 0 / −10 dB: 38 / 28 dB over the median, 26 / 16 dB over the second. In digital silence the median is ~0, so over-the-median reads up to the 200 dB clamp. +- About 20 ms per call. The second peak's position is computed but not returned; second arrivals (A2) need it. +Left open: every threshold untuned; nothing reads `inputPeak` yet. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index cf099880..47c891eb 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -145,6 +145,8 @@ goes in `tony_core`. Each event is a sweep of 1 → 8 kHz, 200 ms, with 10 ms edges and a −12 dBFS peak, followed by a tone at a pitch pYIN tracks (196, 220.5, 245 or 294 Hz; see `docs/testing.md`). Gaps between events are irregular (1.6–2.6 s, all different). + Only eleven gaps can differ by 0.1 s in that range, so the *long* layout repeats + the calibration's eleven, and the gaps around *dev*'s held tones are longer. The generator returns every sweep's exact frame. - **Analysis:** - FFT matched filter in the sweep band (bqfft), then its envelope; @@ -269,7 +271,7 @@ marked "Done" when it is committed. 1. **Core.** - **A1** Test reference and sweep finder: `LatencyCheck` generator and per-event - analysis. + analysis. Done. - **A2** Verdicts and calibration arithmetic: aggregation over events and punch-ins. 2. **Runner, dialog and calibration page** (every build), with app tests. - **B1** The alignment check runner and its app tests. diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp new file mode 100644 index 00000000..308849fc --- /dev/null +++ b/main/LatencyCheck.cpp @@ -0,0 +1,335 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LatencyCheck.h" + +#include "bqfft/FFT.h" + +#include +#include + +using namespace sv; +using std::vector; + +namespace LatencyCheck { +namespace { + +const double pi = 3.14159265358979323846; + +// Sweep to sweep, in tenths of a second so that the times are the same +// at every rate: each value from 1.6 to 2.6 s once, in an irregular order +const int calibrationSpacings[] = { 21, 16, 25, 19, 23, 17, 26, 20, 18, 24, 22 }; +const int calibrationSpacingCount = + int(sizeof(calibrationSpacings) / sizeof(calibrationSpacings[0])); + +// On from the last calibration event to the held tones and between +// them. None of these can be a calibration spacing, which are all +// taken, and after a held tone the next event has to wait for its end +const int heldSpacings[] = { 28, 40, 43 }; + +const int firstSweepTenths = 10; + +const double calibrationSeconds = 26.0; +const double devSeconds = 40.0; +const double longSeconds = 240.0; + +// The silence after the last event, as the calibration layout has it +const double tailSeconds = 0.8; + +const double pitches[] = { 220.5, 294.0, 196.0, 245.0 }; +const int pitchCount = int(sizeof(pitches) / sizeof(pitches[0])); + +sv_frame_t +framesAt(double seconds, sv_samplerate_t rate) +{ + return sv_frame_t(std::llround(seconds * rate)); +} + +double +ratioOf(double db) +{ + return std::pow(10.0, db / 20.0); +} + +// A ratio in dB that stays finite: silence against silence is 0 dB, +// and anything against silence is +/-200 +double +decibels(double value, double against) +{ + const double limit = 200.0; + if (value <= 0.0 && against <= 0.0) return 0.0; + if (value <= 0.0) return -limit; + if (against <= 0.0) return limit; + double db = 20.0 * std::log10(value / against); + return std::max(-limit, std::min(limit, db)); +} + +// A function of time rather than of frames, like everything else the +// reference is made of, so that it is the same reference at any rate +double +fade(double t, double duration) +{ + if (t < kFadeSeconds) { + return 0.5 * (1.0 - std::cos(pi * t / kFadeSeconds)); + } + if (t > duration - kFadeSeconds) { + return 0.5 * (1.0 - std::cos(pi * (duration - t) / kFadeSeconds)); + } + return 1.0; +} + +void +addEvent(Layout &layout, int sweepTenths, double toneSeconds) +{ + double at = sweepTenths / 10.0; + Event e; + e.sweepStart = framesAt(at, layout.rate); + e.toneStart = framesAt(at + kSweepSeconds + kPauseSeconds, layout.rate); + e.toneLength = framesAt(toneSeconds, layout.rate); + e.toneHz = pitches[layout.events.size() % pitchCount]; + layout.events.push_back(e); +} + +} // namespace +} // namespace LatencyCheck + +LatencyCheck::Layout +LatencyCheck::calibrationLayout(sv_samplerate_t rate) +{ + Layout layout; + layout.rate = rate; + + int at = firstSweepTenths; + addEvent(layout, at, kToneSeconds); + for (int spacing : calibrationSpacings) { + at += spacing; + addEvent(layout, at, kToneSeconds); + } + + layout.length = framesAt(calibrationSeconds, rate); + return layout; +} + +LatencyCheck::Layout +LatencyCheck::devLayout(sv_samplerate_t rate) +{ + Layout layout = calibrationLayout(rate); + + int at = firstSweepTenths; + for (int spacing : calibrationSpacings) at += spacing; + for (int spacing : heldSpacings) { + at += spacing; + addEvent(layout, at, kHeldToneSeconds); + } + + layout.length = framesAt(devSeconds, rate); + return layout; +} + +LatencyCheck::Layout +LatencyCheck::longLayout(sv_samplerate_t rate) +{ + Layout layout; + layout.rate = rate; + + const double eventSeconds = kSweepSeconds + kPauseSeconds + kToneSeconds; + + int at = firstSweepTenths; + for (int i = 0; at / 10.0 + eventSeconds + tailSeconds <= longSeconds; + ++i) { + addEvent(layout, at, kToneSeconds); + at += calibrationSpacings[i % calibrationSpacingCount]; + } + + layout.length = framesAt(longSeconds, rate); + return layout; +} + +vector +LatencyCheck::sweep(sv_samplerate_t rate) +{ + const sv_frame_t length = framesAt(kSweepSeconds, rate); + const double amplitude = ratioOf(kPeakDbfs); + const double rise = (kSweepEndHz - kSweepStartHz) / kSweepSeconds; + + vector out(length); + for (sv_frame_t i = 0; i < length; ++i) { + double t = double(i) / rate; + double phase = 2.0 * pi * (kSweepStartHz * t + 0.5 * rise * t * t); + out[i] = float(amplitude * fade(t, kSweepSeconds) * std::sin(phase)); + } + return out; +} + +vector +LatencyCheck::generate(const Layout &layout) +{ + vector out(std::max(layout.length, sv_frame_t(0)), 0.f); + + const vector s = sweep(layout.rate); + const double amplitude = ratioOf(kPeakDbfs); + + for (const Event &e : layout.events) { + + for (sv_frame_t i = 0; i < sv_frame_t(s.size()); ++i) { + sv_frame_t f = e.sweepStart + i; + if (f >= 0 && f < layout.length) out[f] += s[i]; + } + + const double duration = double(e.toneLength) / layout.rate; + for (sv_frame_t i = 0; i < e.toneLength; ++i) { + sv_frame_t f = e.toneStart + i; + if (f < 0 || f >= layout.length) continue; + double t = double(i) / layout.rate; + out[f] += float(amplitude * fade(t, duration) * + std::sin(2.0 * pi * e.toneHz * t)); + } + } + + return out; +} + +LatencyCheck::Arrival +LatencyCheck::findSweep(const float *take, sv_frame_t count, + sv_samplerate_t rate, double expectedSeconds) +{ + Arrival arrival; + if (!take || count <= 0 || rate <= 0) return arrival; + + // In frames of the take, at its own rate: the reference's frames + // would put the window in the wrong place in a take at any other + const sv_frame_t expected = framesAt(expectedSeconds, rate); + const sv_frame_t reach = framesAt(kSearchSeconds, rate); + const sv_frame_t first = std::max(sv_frame_t(0), expected - reach); + const sv_frame_t last = std::min(count - 1, expected + reach); + if (first > last) return arrival; + + // The sweep at the take's rate too: at the reference's, it would be + // the wrong length, and would match a stretched sweep, not this one + const vector reference = sweep(rate); + const sv_frame_t sweepLength = sv_frame_t(reference.size()); + + // The correlation at a lag reads a sweep's length on from it, so the + // stretch read runs that far past the last lag + const sv_frame_t end = std::min(count, last + sweepLength); + const int lags = int(last - first + 1); + + for (sv_frame_t i = first; i < end; ++i) { + arrival.inputPeak = std::max(arrival.inputPeak, + double(std::fabs(take[i]))); + } + + // Padded with zeros to a sweep's length beyond the stretch at least, + // so that no lag looked at wraps round the end + int size = 1; + while (size < (end - first) + sweepLength) size *= 2; + + vector x(size, 0.0), s(size, 0.0); + for (sv_frame_t i = first; i < end; ++i) x[i - first] = take[i]; + for (sv_frame_t i = 0; i < sweepLength; ++i) s[i] = reference[i]; + + breakfastquay::FFT fft(size); + const int bins = size / 2 + 1; + vector xre(bins), xim(bins), sre(bins), sim(bins); + fft.forward(x.data(), xre.data(), xim.data()); + fft.forward(s.data(), sre.data(), sim.data()); + + // The take's spectrum times the conjugate of the sweep's is their + // correlation: the matched filter. That product is itself the + // weighting to the sweep's band, since the sweep's spectrum is flat + // from 1 to 8 kHz and falls some 100 dB below that at the tones' + // pitches and at mains hum: a full-scale 100 Hz hum under the + // sweep comes through 100 dB down. A band mask on top of it + // changed nothing that could be measured, so there is none. + // The second spectrum is the first turned by -90 degrees, whose + // inverse is the correlation's Hilbert transform: the two together + // give its envelope (the magnitude of the analytic signal), which + // peaks where the sweep begins whatever the polarity or the phase + // of the carrier. The correlation itself oscillates at the band's + // centre, and its largest sample can be half a cycle off. + vector cre(bins), cim(bins), hre(bins), him(bins); + for (int k = 0; k < bins; ++k) { + cre[k] = xre[k] * sre[k] + xim[k] * sim[k]; + cim[k] = xim[k] * sre[k] - xre[k] * sim[k]; + hre[k] = cim[k]; + him[k] = -cre[k]; + } + hre[0] = him[0] = hre[bins-1] = him[bins-1] = 0.0; + + vector c(size), h(size); + fft.inverse(cre.data(), cim.data(), c.data()); + fft.inverse(hre.data(), him.data(), h.data()); + + // bqfft's inverse is unscaled. Scaled, the envelope of a take + // holding the reference's own sweep peaks at the sweep's energy + vector envelope(lags); + for (int j = 0; j < lags; ++j) { + envelope[j] = std::hypot(c[j], h[j]) / size; + } + + const int largest = int(std::max_element(envelope.begin(), envelope.end()) + - envelope.begin()); + + // The earliest peak within kEarliestPeakDb of the largest: in a + // room a reflection can be stronger than the direct sound, and the + // direct sound's time is the one wanted. A peak is a local maximum + // of the envelope, and only one at least kSeparateArrivalSeconds + // before the largest counts. One arrival has no earlier peak that + // high unless its spectrum has a hole in it: then the envelope + // beats, a row of near-equal peaks with deep dips between them, so + // a dip between two peaks does not tell arrivals apart, but the + // time does, since those peaks lie within about one over the width + // of what is left of the band. A reflection closer than the + // spacing is too close to matter: the larger peak is taken. + int chosen = largest; + const int apart = std::max(1, int(framesAt(kSeparateArrivalSeconds, rate))); + const double within = envelope[largest] * ratioOf(-kEarliestPeakDb); + for (int j = 1; j + apart <= largest; ++j) { + if (envelope[j] >= within && + envelope[j] > envelope[j-1] && envelope[j] >= envelope[j+1]) { + chosen = j; + break; + } + } + + vector ordered(envelope); + std::nth_element(ordered.begin(), ordered.begin() + lags / 2, + ordered.end()); + const double median = ordered[lags / 2]; + + // The chosen peak's own reflections are near it, and are not + // another candidate for where the sweep is + const int closeBy = int(framesAt(kSecondPeakSeconds, rate)); + double second = 0.0; + for (int j = 0; j < lags; ++j) { + if (std::abs(j - chosen) > closeBy) { + second = std::max(second, envelope[j]); + } + } + + double energy = 0.0; + for (float v : reference) energy += double(v) * double(v); + + const sv_frame_t at = first + chosen; + arrival.errorFrames = at - expected; + arrival.errorSeconds = double(at) / rate - expectedSeconds; + arrival.peakOverMedianDb = decibels(envelope[chosen], median); + arrival.peakOverSecondDb = decibels(envelope[chosen], second); + arrival.levelDb = decibels(envelope[chosen], energy); + arrival.found = + arrival.peakOverMedianDb >= kMinPeakOverMedianDb && + arrival.peakOverSecondDb >= kMinPeakOverSecondDb; + + return arrival; +} diff --git a/main/LatencyCheck.h b/main/LatencyCheck.h new file mode 100644 index 00000000..3b1cc263 --- /dev/null +++ b/main/LatencyCheck.h @@ -0,0 +1,206 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LATENCY_CHECK_H +#define TONY_LATENCY_CHECK_H + +#include "base/BaseTypes.h" + +#include + +/** + * The test reference that Calibrate Audio plays, and the finder that + * locates its sweeps in what the microphone recorded. + * + * The reference is a row of events: a short sweep, which is what the + * finder looks for, then a tone that pYIN can track, then silence. A + * take recorded against it through a loopback (an earcup held to the + * mic) holds every sweep where the take path put it, so how far a + * sweep is found from where the reference has it is how far the take + * was misplaced. + * + * Pure functions over sample buffers: no model, no window and no + * device, so all of it is tested without them (TestLatencyCheck). + * Judging a take from its events is a later step. + */ +namespace LatencyCheck +{ + /// The session's rate, which the reference is made at unless asked + constexpr sv::sv_samplerate_t kReferenceRate = 44100; + + /** + * The sweep: linear from 1 to 8 kHz. Linear rather than + * exponential for two reasons. Its spectrum is flat over the + * band, so its matched filter gives the narrowest, most symmetric + * peak that band allows. And the harmonics a small speaker adds + * to a linear sweep rise at a different rate from it, so they do + * not match it; those of an exponential sweep match the sweep + * itself shifted earlier (by 67 ms for the second harmonic and + * 106 ms for the third, here), which is just where the + * earliest-peak rule below looks. + */ + constexpr double kSweepStartHz = 1000.0; + constexpr double kSweepEndHz = 8000.0; + constexpr double kSweepSeconds = 0.2; + + /// Raised-cosine fades at both ends of every sweep and tone, so + /// that no edge is a click + constexpr double kFadeSeconds = 0.01; + + /// The peak of every sweep and tone: well clear of room noise, and + /// no louder than that, since the earcups are held near the ears + constexpr double kPeakDbfs = -12.0; + + /// Silence between a sweep and its tone, so that the tone begins + /// as a note of its own + constexpr double kPauseSeconds = 0.1; + + /// How long the tone after a sweep lasts; the dev layout's held + /// tones are long enough for two punch-ins to meet inside one + constexpr double kToneSeconds = 0.8; + constexpr double kHeldToneSeconds = 3.0; + + /** + * The finder's thresholds. Starting values: the report of every + * real run gives the numbers they are to be tuned from. + */ + + /// How far either side of the expected time a sweep is looked + /// for. Less than half the smallest spacing between events, so a + /// neighbour's sweep cannot be taken for this one + constexpr double kSearchSeconds = 0.8; + + /// A peak this much below the largest can still be the direct + /// sound, the largest being a reflection of it + constexpr double kEarliestPeakDb = 6.0; + + /// An earlier peak counts as an arrival of its own only if it is + /// at least this far before the largest; see findSweep() + constexpr double kSeparateArrivalSeconds = 0.001; + + /// The sweep is found when its peak stands this far above the + /// median of the window ... + constexpr double kMinPeakOverMedianDb = 15.0; + + /// ... and this far above anything in the window more than + /// kSecondPeakSeconds away from it + constexpr double kMinPeakOverSecondDb = 6.0; + constexpr double kSecondPeakSeconds = 0.01; + + /** + * One event of the reference, in frames of its layout's rate: a + * sweep, a pause, a tone, then silence until the next event. + */ + struct Event { + sv::sv_frame_t sweepStart; + sv::sv_frame_t toneStart; + sv::sv_frame_t toneLength; + double toneHz; + + Event() : sweepStart(0), toneStart(0), toneLength(0), toneHz(0) { } + }; + + /** + * A reference's length and events, at a rate. The events are in + * order, and each ends before the next begins. + */ + struct Layout { + sv::sv_samplerate_t rate; + sv::sv_frame_t length; + std::vector events; + + Layout() : rate(kReferenceRate), length(0) { } + }; + + /** + * The fixed layouts. The events are at irregular spacings (sweep + * to sweep) between 1.6 and 2.6 s, each spacing used once, so that + * no stretch of the reference matches another one some events + * along: a take misplaced by whole events cannot look right. The + * spacing is also what keeps the finder off a neighbour's sweep + * (kSearchSeconds). The tones take turns at 196, 220.5, 245 and + * 294 Hz: a whole number of samples per period at 44.1 kHz, which + * pYIN needs to report the pitch rather than a subharmonic + * (docs/testing.md). + * + * All three give their events the same times in seconds at any + * rate, so a reference made at another rate is the same reference. + */ + + /// 26 s: twelve events, for the four punch-ins of a calibration + Layout calibrationLayout(sv::sv_samplerate_t rate = kReferenceRate); + + /// 40 s: the calibration events, then three with held tones. The + /// spacings into and between those are longer than any calibration + /// spacing, so all still differ, and a held tone's event (3.3 s) + /// ends before the next sweep + Layout devLayout(sv::sv_samplerate_t rate = kReferenceRate); + + /// 4 minutes of events at the calibration spacings, over and over. + /// No more than eleven spacings can differ by 0.1 s inside 1.6 to + /// 2.6 s, so here any eleven in a row do + Layout longLayout(sv::sv_samplerate_t rate = kReferenceRate); + + /// One sweep at the given rate, as the reference has it + std::vector sweep(sv::sv_samplerate_t rate); + + /// The reference: the layout's events, silence everywhere else + std::vector generate(const Layout &layout); + + /** + * Where a sweep was found in a take, and how sure the finder is. + * The error and the levels describe the best candidate even when + * it was not found; they mean something only when it was. + */ + struct Arrival { + /// Both confidences cleared their thresholds + bool found; + + /// Where the sweep arrived minus where it was expected, in + /// whole frames of the take counted from the frame nearest the + /// expected time. Positive is late + sv::sv_frame_t errorFrames; + + /// The same, in seconds, from the expected time itself + double errorSeconds; + + /// The two confidences, in dB + double peakOverMedianDb; + double peakOverSecondDb; + + /// How loud the sweep arrived against how it was generated: + /// 0 dB if the take holds the reference's sweep unchanged + double levelDb; + + /// The largest sample of the take in the stretch searched, + /// full scale being 1 + double inputPeak; + + Arrival() : found(false), errorFrames(0), errorSeconds(0), + peakOverMedianDb(0), peakOverSecondDb(0), + levelDb(0), inputPeak(0) { } + }; + + /** + * Look for the reference's sweep in a take, within kSearchSeconds + * of where it is expected; the window is cut short at the ends of + * the take. The take's samples are at the given rate, which need + * not be the reference's: the expected time is in seconds, and + * the sweep is matched at the take's own rate. + */ + Arrival findSweep(const float *take, sv::sv_frame_t count, + sv::sv_samplerate_t rate, double expectedSeconds); +} + +#endif diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h new file mode 100644 index 00000000..b26bdd98 --- /dev/null +++ b/main/test/TestLatencyCheck.h @@ -0,0 +1,564 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LATENCY_CHECK_H +#define TEST_LATENCY_CHECK_H + +// Tier 2: the test reference that Calibrate Audio plays, and the finder +// that locates its sweeps in a recording. The takes are made here from +// the reference itself: shifted, filtered, with noise or a reflection +// added. No window, no device and no files. + +#include "../LatencyCheck.h" + +#include "TestSignals.h" + +#include +#include + +#include +#include +#include +#include + +class TestLatencyCheck : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + typedef std::vector samples_t; + + static constexpr double kRate = 44100.0; + + static frame_t framesOf(double seconds, double rate = kRate) { + return frame_t(std::llround(seconds * rate)); + } + + // The first n events of the calibration layout, and the reference up + // to where the next would begin: as much as a finder test needs + static LatencyCheck::Layout firstEvents(int n, double rate = kRate) { + LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(rate); + layout.length = layout.events[n].sweepStart; + layout.events.resize(n); + return layout; + } + + static double expectedAt(const LatencyCheck::Layout &layout, int i) { + return double(layout.events[i].sweepStart) / layout.rate; + } + + // The take as it would be if all of the reference arrived this many + // frames late (early, if negative) + static samples_t shifted(const samples_t &x, frame_t by) { + const frame_t n = frame_t(x.size()); + samples_t y(x.size(), 0.f); + for (frame_t i = std::max(frame_t(0), by); i < n && i - by < n; ++i) { + y[i] = x[i - by]; + } + return y; + } + + static samples_t mixed(const samples_t &x, const samples_t &y, + double gain) { + samples_t out(x); + for (size_t i = 0; i < out.size() && i < y.size(); ++i) { + out[i] += float(gain * y[i]); + } + return out; + } + + static double ratioOf(double db) { return std::pow(10.0, db / 20.0); } + + static double rms(const samples_t &x) { + double sum = 0.0; + for (float v : x) sum += double(v) * v; + return x.empty() ? 0.0 : std::sqrt(sum / double(x.size())); + } + + static double peakOf(const samples_t &x, frame_t from, frame_t to) { + double peak = 0.0; + for (frame_t i = from; i < to; ++i) { + peak = std::max(peak, double(std::fabs(x[i]))); + } + return peak; + } + + // Sign changes: twice the cycles of a sine, whatever its level + static int crossings(const samples_t &x, frame_t from, frame_t to) { + int n = 0; + for (frame_t i = from + 1; i < to; ++i) { + if ((x[i-1] < 0.f) != (x[i] < 0.f)) ++n; + } + return n; + } + + // A second-order Butterworth section (the Audio EQ Cookbook's), as a + // small speaker's roll-off + static samples_t filtered(const samples_t &x, double rate, double hz, + bool highPass) { + const double w = 2.0 * TestSignals::kPi * hz / rate; + const double alpha = std::sin(w) / std::sqrt(2.0); + const double cosw = std::cos(w); + const double b0 = highPass ? (1.0 + cosw) / 2.0 : (1.0 - cosw) / 2.0; + const double b1 = highPass ? -(1.0 + cosw) : (1.0 - cosw); + const double b2 = b0; + const double a0 = 1.0 + alpha, a1 = -2.0 * cosw, a2 = 1.0 - alpha; + samples_t y(x.size()); + double x1 = 0.0, x2 = 0.0, y1 = 0.0, y2 = 0.0; + for (size_t i = 0; i < x.size(); ++i) { + double in = x[i]; + double out = (b0 * in + b1 * x1 + b2 * x2 - a1 * y1 - a2 * y2) / a0; + x2 = x1; x1 = in; + y2 = y1; y1 = out; + y[i] = float(out); + } + return y; + } + + static LatencyCheck::Arrival find(const samples_t &take, double rate, + double expectedSeconds) { + return LatencyCheck::findSweep(take.data(), frame_t(take.size()), + rate, expectedSeconds); + } + + static QByteArray describe(const LatencyCheck::Arrival &a) { + return QString("found %1, error %2 frames (%3 ms), %4 dB over the " + "median, %5 dB over the second, level %6 dB") + .arg(a.found ? "yes" : "no") + .arg(qint64(a.errorFrames)) + .arg(a.errorSeconds * 1000.0) + .arg(a.peakOverMedianDb) + .arg(a.peakOverSecondDb) + .arg(a.levelDb) + .toUtf8(); + } + +private slots: + // The same layout and the same samples every time: nothing in the + // reference depends on a clock or a random source + void generator_is_deterministic() { + const LatencyCheck::Layout a = LatencyCheck::devLayout(); + const LatencyCheck::Layout b = LatencyCheck::devLayout(); + QCOMPARE(a.length, b.length); + QCOMPARE(a.events.size(), b.events.size()); + for (size_t i = 0; i < a.events.size(); ++i) { + QCOMPARE(a.events[i].sweepStart, b.events[i].sweepStart); + QCOMPARE(a.events[i].toneStart, b.events[i].toneStart); + QCOMPARE(a.events[i].toneLength, b.events[i].toneLength); + QCOMPARE(a.events[i].toneHz, b.events[i].toneHz); + } + QVERIFY(LatencyCheck::generate(a) == LatencyCheck::generate(b)); + } + + // Each event is where the layout says and what it says: silence, a + // sweep rising from 1 to 8 kHz, a pause, a tone at its pitch, silence + void generator_puts_events_where_it_says() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t x = LatencyCheck::generate(layout); + const samples_t sweep = LatencyCheck::sweep(kRate); + const frame_t sweepLength = frame_t(sweep.size()); + + QCOMPARE(layout.rate, kRate); + QCOMPARE(layout.length, framesOf(26.0)); + QCOMPARE(frame_t(x.size()), layout.length); + QCOMPARE(int(layout.events.size()), 12); + QCOMPARE(sweepLength, framesOf(0.2)); + + std::set pitches; + frame_t silentFrom = 0; + + for (const LatencyCheck::Event &e : layout.events) { + + QCOMPARE(peakOf(x, silentFrom, e.sweepStart), 0.0); + + int misplaced = 0; + for (frame_t i = 0; i < sweepLength; ++i) { + if (x[e.sweepStart + i] != sweep[i]) ++misplaced; + } + QCOMPARE(misplaced, 0); + + // Rising: 1 to 4.5 kHz in the first 0.1 s is 550 sign + // changes, 1 to 8 kHz in 0.2 s is 1800. Falling, the first + // half would have 1250 + int half = crossings(x, e.sweepStart, e.sweepStart + sweepLength / 2); + int whole = crossings(x, e.sweepStart, e.sweepStart + sweepLength); + QVERIFY2(std::abs(half - 550) <= 3, qPrintable(QString::number(half))); + QVERIFY2(std::abs(whole - 1800) <= 3, qPrintable(QString::number(whole))); + + QCOMPARE(e.toneStart - e.sweepStart, framesOf(0.3)); + QCOMPARE(peakOf(x, e.sweepStart + sweepLength, e.toneStart), 0.0); + + // A whole number of samples per period at 44.1 kHz, and + // that pitch in the samples + QCOMPARE(e.toneLength, framesOf(0.8)); + const double period = kRate / e.toneHz; + QCOMPARE(period, std::round(period)); + const int tone = crossings(x, e.toneStart, e.toneStart + e.toneLength); + const double cycles = e.toneHz * double(e.toneLength) / kRate; + QVERIFY2(std::fabs(tone - 2.0 * cycles) <= 2.0, + qPrintable(QString("%1 at %2 Hz").arg(tone).arg(e.toneHz))); + pitches.insert(e.toneHz); + + silentFrom = e.toneStart + e.toneLength; + } + + QCOMPARE(peakOf(x, silentFrom, frame_t(x.size())), 0.0); + QCOMPARE(pitches, (std::set { 196.0, 220.5, 245.0, 294.0 })); + } + + // Sweep to sweep: never less than twice the finder's reach, so that + // a neighbour's sweep is never in its window; 1.6 to 2.6 s between + // events with short tones; and all different, or in the long + // layout, any eleven in a row + void generator_spacings_are_irregular() { + struct Case { + LatencyCheck::Layout layout; + double seconds; + int held; + int run; + }; + const Case cases[] = { + { LatencyCheck::calibrationLayout(), 26.0, 0, 0 }, + { LatencyCheck::devLayout(), 40.0, 3, 0 }, + { LatencyCheck::longLayout(), 240.0, 0, 11 }, + }; + + for (const Case &c : cases) { + const LatencyCheck::Layout &layout = c.layout; + const int n = int(layout.events.size()); + QCOMPARE(layout.length, framesOf(c.seconds)); + + std::vector spacings; + int held = 0; + + for (int i = 0; i < n; ++i) { + const LatencyCheck::Event &e = layout.events[i]; + const frame_t end = e.toneStart + e.toneLength; + QVERIFY(end <= layout.length); + const bool isHeld = (e.toneLength == framesOf(3.0)); + if (isHeld) ++held; + if (i + 1 == n) break; + + const LatencyCheck::Event &next = layout.events[i + 1]; + QVERIFY(end < next.sweepStart); + const double spacing = + double(next.sweepStart - e.sweepStart) / layout.rate; + QVERIFY2(spacing >= 2.0 * LatencyCheck::kSearchSeconds - 1e-9, + qPrintable(QString::number(spacing))); + if (!isHeld && next.toneLength == framesOf(0.8)) { + QVERIFY2(spacing <= 2.6 + 1e-9, + qPrintable(QString::number(spacing))); + } + spacings.push_back(spacing); + } + + QCOMPARE(held, c.held); + + const int count = int(spacings.size()); + const int run = c.run > 0 ? c.run : count; + for (int i = 0; i < count; ++i) { + for (int j = i + 1; j < count && j < i + run; ++j) { + QVERIFY2(std::fabs(spacings[i] - spacings[j]) >= 0.1 - 1e-9, + qPrintable(QString("%1 and %2").arg(i).arg(j))); + } + } + } + + QVERIFY(LatencyCheck::longLayout().events.size() > 100); + } + + // Every sweep and every tone peaks at -12 dBFS, and nothing is louder + void generator_peaks_at_minus_12_dbfs() { + const LatencyCheck::Layout layout = LatencyCheck::devLayout(); + const samples_t x = LatencyCheck::generate(layout); + const double peak = ratioOf(LatencyCheck::kPeakDbfs); + const frame_t sweepLength = framesOf(LatencyCheck::kSweepSeconds); + + QVERIFY(peakOf(x, 0, frame_t(x.size())) <= peak * (1.0 + 1e-6)); + for (const LatencyCheck::Event &e : layout.events) { + QVERIFY(peakOf(x, e.sweepStart, e.sweepStart + sweepLength) >= + peak * ratioOf(-0.05)); + QVERIFY(peakOf(x, e.toneStart, e.toneStart + e.toneLength) >= + peak * ratioOf(-0.05)); + } + } + + // The reference as generated: every event where it is, at the level + // it was made at + void finder_finds_every_event_of_the_reference() { + const LatencyCheck::Layout layout = LatencyCheck::devLayout(); + const samples_t x = LatencyCheck::generate(layout); + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(x, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, frame_t(0)); + QVERIFY2(std::fabs(a.errorSeconds) < 1e-9, describe(a).constData()); + QVERIFY2(std::fabs(a.levelDb) < 0.01, describe(a).constData()); + } + } + + // Takes that land early and late, by up to nearly the finder's + // reach. At 0.75 s, the neighbour across the smallest spacing (1.6 + // s, between the second and third events) is 0.85 s from where this + // sweep is expected: just out of reach, which is what the spacing + // is for + void finder_finds_shifts_across_the_window() { + const LatencyCheck::Layout layout = firstEvents(4); + const samples_t reference = LatencyCheck::generate(layout); + + const frame_t shifts[] = { + -framesOf(0.75), -framesOf(0.5), -8821, -1, 0, 1, + framesOf(0.1), 13231, framesOf(0.75) + }; + for (frame_t shift : shifts) { + const samples_t take = shifted(reference, shift); + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(take, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift); + QVERIFY2(std::fabs(a.errorSeconds - shift / kRate) < 1e-9, + describe(a).constData()); + } + } + } + + // Half a frame late. The reference made at twice the rate, read at + // every other sample, is the reference itself from an even sample + // and the reference half a frame late from an odd one + void finder_finds_a_fractional_shift() { + const LatencyCheck::Layout layout = firstEvents(4); + const samples_t reference = LatencyCheck::generate(layout); + const samples_t doubled = LatencyCheck::generate(firstEvents(4, 2 * kRate)); + QCOMPARE(doubled.size(), 2 * reference.size()); + + int different = 0; + for (size_t i = 0; i < reference.size(); ++i) { + if (doubled[2 * i] != reference[i]) ++different; + } + QCOMPARE(different, 0); + + const frame_t whole = 100; + samples_t take(reference.size(), 0.f); + for (frame_t i = 0; i < frame_t(take.size()); ++i) { + frame_t j = 2 * i - (2 * whole + 1); + if (j >= 0) take[i] = doubled[j]; + } + + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(take, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QVERIFY2(a.errorFrames == whole || a.errorFrames == whole + 1, + describe(a).constData()); + QVERIFY2(std::fabs(a.errorSeconds - (whole + 0.5) / kRate) + <= 0.5 / kRate + 1e-9, describe(a).constData()); + } + } + + // A device at 48 kHz: the take's frames are 48 kHz frames, and the + // expected times come from the session's 44.1 kHz layout. The take + // is the same reference made at 48 kHz, the same times in seconds + void finder_works_at_the_take_rate() { + const double deviceRate = 48000.0; + const LatencyCheck::Layout session = firstEvents(4); + const LatencyCheck::Layout device = firstEvents(4, deviceRate); + for (int i = 0; i < 4; ++i) { + QCOMPARE(device.events[i].sweepStart * 441, + session.events[i].sweepStart * 480); + } + + const frame_t shift = 4801; + const samples_t take = shifted(LatencyCheck::generate(device), shift); + for (int i = 0; i < 4; ++i) { + LatencyCheck::Arrival a = + find(take, deviceRate, expectedAt(session, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift); + QVERIFY2(std::fabs(a.errorSeconds - shift / deviceRate) < 1e-9, + describe(a).constData()); + QVERIFY2(std::fabs(a.levelDb) < 0.01, describe(a).constData()); + } + } + + // White noise as loud as the sweep, and ten times louder in power. + // The sweep is still found to the frame; what the noise changes is + // how sure the finder is + void finder_hears_through_noise() { + const LatencyCheck::Layout layout = firstEvents(4); + const frame_t shift = 12345; + const samples_t take = shifted(LatencyCheck::generate(layout), shift); + const double sweepRms = rms(LatencyCheck::sweep(kRate)); + + std::vector sureness(layout.events.size(), 1000.0); + for (double snr : { 0.0, -10.0 }) { + // Uniform noise: its RMS is its amplitude over root 3 + const double amplitude = sweepRms * ratioOf(-snr) * std::sqrt(3.0); + const samples_t noisy = + mixed(take, TestSignals::whiteNoise(int(take.size()), 20260925, + amplitude), 1.0); + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(noisy, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QVERIFY2(std::abs(a.errorFrames - shift) <= 1, describe(a).constData()); + QVERIFY2(a.peakOverMedianDb < sureness[i], describe(a).constData()); + sureness[i] = a.peakOverMedianDb; + } + } + } + + // A small speaker, or an earcup held to a mic: nothing much below + // 1 kHz or above 4 kHz. The filters delay the band a little + // themselves (the 4 kHz low-pass by about 60 microseconds), which + // is all the error there is + void finder_hears_a_band_limited_path() { + const LatencyCheck::Layout layout = firstEvents(4); + const frame_t shift = 12345; + const samples_t take = shifted(LatencyCheck::generate(layout), shift); + + const samples_t highPassed = filtered(take, kRate, 1000.0, true); + const samples_t lowPassed = filtered(take, kRate, 4000.0, false); + const samples_t both = filtered(highPassed, kRate, 4000.0, false); + + for (const samples_t *path : { &highPassed, &lowPassed, &both }) { + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(*path, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QVERIFY2(std::fabs(a.errorSeconds - shift / kRate) < 0.0001, + describe(a).constData()); + } + } + } + + // A speaker or mic wired the other way round + void finder_ignores_polarity() { + const LatencyCheck::Layout layout = firstEvents(4); + const frame_t shift = 12345; + samples_t take = shifted(LatencyCheck::generate(layout), shift); + for (float &v : take) v = -v; + + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(take, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift); + QVERIFY2(std::fabs(a.levelDb) < 0.01, describe(a).constData()); + } + } + + // A reflection 7 ms after the direct sound, and stronger than it: + // the direct sound's time is the one wanted. Exactly kEarliestPeakDb + // stronger is the rule's edge, where rounding decides, so the test + // is half a dB inside it; a dB outside it the reflection is taken, + // which is where the rule gives up + void finder_takes_the_direct_sound_before_a_stronger_reflection() { + const LatencyCheck::Layout layout = firstEvents(4); + const samples_t reference = LatencyCheck::generate(layout); + const frame_t shift = 12345; + const frame_t reflection = framesOf(0.007); + const samples_t direct = shifted(reference, shift); + const samples_t late = shifted(reference, shift + reflection); + + const samples_t inside = + mixed(direct, late, ratioOf(LatencyCheck::kEarliestPeakDb - 0.5)); + const samples_t outside = + mixed(direct, late, ratioOf(LatencyCheck::kEarliestPeakDb + 1.0)); + + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(inside, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift); + + a = find(outside, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift + reflection); + } + } + + // A path that loses the middle of the band (2.75 to 6.25 kHz: the + // middle 100 ms of each sweep) makes the envelope beat, with + // near-equal peaks a fraction of a millisecond apart. Those are one + // arrival, not an earlier one + void finder_takes_one_arrival_as_one() { + const LatencyCheck::Layout layout = firstEvents(4); + const frame_t shift = 12345; + samples_t take = shifted(LatencyCheck::generate(layout), shift); + for (const LatencyCheck::Event &e : layout.events) { + for (frame_t f = framesOf(0.05); f < framesOf(0.15); ++f) { + take[e.sweepStart + shift + f] = 0.f; + } + } + + for (int i = 0; i < int(layout.events.size()); ++i) { + LatencyCheck::Arrival a = find(take, kRate, expectedAt(layout, i)); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, shift); + } + } + + // Nothing but silence, or nothing but room noise: no sweep there + void finder_finds_nothing_in_silence() { + const samples_t silence(size_t(framesOf(5.0)), 0.f); + const samples_t noise = + TestSignals::whiteNoise(int(framesOf(5.0)), 7, 0.05); + + for (double expected : { 1.0, 2.5, 4.0 }) { + LatencyCheck::Arrival a = find(silence, kRate, expected); + QVERIFY2(!a.found, describe(a).constData()); + QVERIFY2(a.peakOverMedianDb < LatencyCheck::kMinPeakOverMedianDb, + describe(a).constData()); + + a = find(noise, kRate, expected); + QVERIFY2(!a.found, describe(a).constData()); + } + } + + // The sweep's level as it arrived, and the loudest sample around it + void finder_reports_levels() { + const LatencyCheck::Layout layout = firstEvents(1); + samples_t take = shifted(LatencyCheck::generate(layout), 12345); + for (float &v : take) v *= 0.5f; + + LatencyCheck::Arrival a = find(take, kRate, expectedAt(layout, 0)); + QVERIFY2(a.found, describe(a).constData()); + QVERIFY2(std::fabs(a.levelDb - 20.0 * std::log10(0.5)) < 0.01, + describe(a).constData()); + QVERIFY(std::fabs(a.inputPeak - + 0.5 * ratioOf(LatencyCheck::kPeakDbfs)) < 1e-5); + } + + // The window stops at the ends of the take + void finder_window_stops_at_the_ends_of_the_take() { + const LatencyCheck::Layout layout = firstEvents(1); + const samples_t reference = LatencyCheck::generate(layout); + const frame_t sweepStart = layout.events[0].sweepStart; + + // From 0.1 s before the sweep to 50 ms after it: less than the + // reach on either side + const samples_t take(reference.begin() + (sweepStart - framesOf(0.1)), + reference.begin() + (sweepStart + framesOf(0.25))); + + LatencyCheck::Arrival a = find(take, kRate, 0.1); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, frame_t(0)); + + // Expected before the take begins, and still within reach + a = find(take, kRate, -0.5); + QVERIFY2(a.found, describe(a).constData()); + QCOMPARE(a.errorFrames, framesOf(0.6)); + + // Out of reach either side, or no take at all + QVERIFY(!find(take, kRate, 5.0).found); + QVERIFY(!find(take, kRate, -1.0).found); + QVERIFY(!find(samples_t(), kRate, 0.1).found); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 4a7ed2b5..850a47b1 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -20,6 +20,7 @@ #include "TestSingingTakes.h" #include "TestTakesFile.h" #include "TestTakeTiming.h" +#include "TestLatencyCheck.h" #include "RunSuite.h" @@ -98,6 +99,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLatencyCheck t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 8f8b0e6e..88d76863 100644 --- a/meson.build +++ b/meson.build @@ -1095,6 +1095,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/Coverage.cpp', + 'main/LatencyCheck.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', @@ -1356,6 +1357,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestSingingTakes.h', 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', + 'main/test/TestLatencyCheck.h', ]) tony_core_test_exe = executable( From 12a13b831e0e3c2580cad025ca7216c3a27ced9d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:23:34 +0000 Subject: [PATCH 084/275] docs: calibrate audio work orders, A2 refined from what A1 found Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index db8401dd..bf72950c 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: none. +Done: A1 (`944df7c`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -179,14 +179,36 @@ Done: none. the thresholds of spec §5 as named constants. - The calibration arithmetic as a pure function: new round trip = used round trip + median offset, in seconds. A take that lands late was spliced from too early a frame. +- **Refined by the lead after A1:** + - **Entry point for B1.** One function takes a layout, the take's samples and rate, + and the punch-ins. Each punch-in is its timeline range in seconds, in the order + recorded. The function returns a summary with the verdict, all flags that applied, + per-punch-in figures, and per-event results. + - **Which events are judged.** An event is *judged* in a punch-in when its sweep and + the finder's window sit inside the range with a margin, a named constant. The splice + cuts content at the range ends, and a sweep cut in half is not a failure of the + path. Count judged and found separately; NoSignal is about found out of judged. + - **Second arrivals.** `findSweep()` computes the second peak but does not return + its position; add it, and its level against the chosen one, to `Arrival`. Monitoring + echo is a second arrival at a consistent extra delay (a few ms of spread) across + most found events, not more than some named dB below the direct sound. + - **Verdict order.** When several apply, pick one and document it, with all flags kept. + Suggested: NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok. + - **PositionDependent against Scattered.** Fit offset over punch-in position. + PositionDependent is a slope above 0.5 % whose fit leaves little residual; large + residuals are Scattered. - **Tests:** - one event missing, the rest found; - fading; - clipped; - - resampled by 48000/44100 → PositionDependent; + - misplacement that grows with position (the 48000/44100 case) → PositionDependent; - two punch-ins 20 ms apart → Scattered or Unsteady by threshold; - monitoring echo detected; + - an event cut by a range end is not judged; - the arithmetic with both signs. **Show that the sign test fails** when flipped. + - A1 found that a take of 48 kHz frames read as 44.1 kHz finds nothing, because the + sweeps are stretched. Build the rate case as punch-ins displaced by + P·(1 − 44100/48000), not as a stretch. ### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) From 0419d36beb0688d8cc490242bbd51e7c0208d779 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:24:45 +0000 Subject: [PATCH 085/275] fix: a take from a device not at 44.1 khz lands where it was sung tony's models are at 44.1 khz, but a recording is written at the device's rate, and phones run at 48 khz, as some desktop devices do. the take's audio went in at the wrong scale and place, punch-out stopped early, and the live dots used the wrong timeline. a recording is now resampled to the reference's rate before the splice, and taketiming converts between device frames (latency, frames received, the live tracker) and the reference's. the 48 khz placement, latency and punch-out tests failed before. the play cursor still runs fast while recording at 48 khz, which needs the svgui fork; an expected failure marks it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/mobile-port.md | 18 +- main/MainWindow.cpp | 64 +++-- main/MainWindow.h | 20 +- main/SingingTakes.cpp | 24 +- main/SingingTakes.h | 10 +- main/TakeAudio.cpp | 113 +++++++++ main/TakeAudio.h | 21 +- main/TakeTiming.cpp | 34 ++- main/TakeTiming.h | 36 ++- main/test/TestRecordWorkflow.h | 424 +++++++++++++++++++++++++++++++++ main/test/TestSingingTakes.h | 59 ++++- main/test/TestTakeAudio.h | 118 +++++++++ main/test/TestTakeTiming.h | 57 +++++ 13 files changed, 945 insertions(+), 53 deletions(-) diff --git a/docs/mobile-port.md b/docs/mobile-port.md index a13ca925..d9af2dd9 100644 --- a/docs/mobile-port.md +++ b/docs/mobile-port.md @@ -117,16 +117,14 @@ What the code does: - `TakeAudio::splice()` does not resample. It refuses only a recording whose rate differs from the take so far. -So with a device that is not at 44.1 kHz, take audio would presumably be placed at the -wrong scale. Nothing verifies this; [takes.md](takes.md#known-limitations) calls a device -rate different from the reference's unexercised. It may already affect a Windows machine -whose default output device runs at 48 kHz. - -Phones run at 48 kHz natively. **Check this on the desktop with a 48 kHz device before a -port.** The fixes: -- resample the recording to the main model's rate before the splice, in `tony_core`, - which is testable; -- or open the device at 44.1 kHz (Oboe can convert, at some cost in latency). +Checked in phase A1 with `FakeAudioIO` at 48 kHz: take audio, pitch and coverage were +misplaced and mis-scaled by 48/44.1 (as they were for a desktop device opened at 48 kHz). +Fixed in `tony_core`, without opening the device at 44.1 kHz: +`SingingTakes::spliceRecording()` resamples the recording to the reference's rate first +(`TakeAudio::resample()`), so a take's WAV is always at the reference's rate, and +`TakeTiming` converts between the device's frames (the latency, the frames received, the +live tracker's frames) and the reference's. Left: the cursor during a take runs at the +device's rate (svgui's `ViewManager`, an expected failure in `TestRecordWorkflow`). ### The pYIN plugin diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 534cf8a7..4edab375 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -199,7 +199,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_recordingLatencyFrames(0), m_recordingStartGapEstimate(0), m_recordingStartGapMeasured(-1), - m_awaitingReferenceStart(false) + m_awaitingReferenceStart(false), + m_recordFramesPerPlayFrame(1.0) { setWindowTitle(QApplication::applicationName()); @@ -210,8 +211,13 @@ MainWindow::MainWindow(AudioMode audioMode, if (!m_recordTarget || !m_recordTarget->isRecording()) return; // The device drivers deliver the input of a block before they // ask for its output (PortAudioIO, JACKAudioIO), so the count - // already includes the input that goes with this first block - sv_frame_t gap = m_recordTarget->getFramesReceived() - blockFrames; + // already includes the input that goes with this first block. + // The block is counted at the play source's rate, the + // reference's, and the frames received at the device's + sv_frame_t block = sv_frame_t(std::llround + (blockFrames * + m_recordFramesPerPlayFrame.load())); + sv_frame_t gap = m_recordTarget->getFramesReceived() - block; m_recordingStartGapMeasured = (gap > 0 ? gap : 0); }); } @@ -3604,12 +3610,16 @@ MainWindow::setupRealtimePitchLayer() return; } - // Determine sample rate from the audio source model. + // The dots go on the main model's timeline, so their model is at its + // rate. The device may record at another (a phone runs at 48 kHz): + // the tracker works in the recording's own frames and rate, and + // onRealtimePitchDetected() converts where the dot goes sv_samplerate_t sr = 44100; - if (auto audioModel = ModelById::getAs(audioSourceId)) { - sr = audioModel->getSampleRate(); - } else if (auto wfm = getMainModel()) { + if (auto wfm = getMainModel()) { sr = wfm->getSampleRate(); + } else if (auto audioModel = + ModelById::getAs(audioSourceId)) { + sr = audioModel->getSampleRate(); } // Create a SparseTimeValueModel to receive pitch estimates. @@ -4158,9 +4168,15 @@ MainWindow::recordingStarted() // With a pre-roll, playback starts at the beginning of the // lead-in rather than at the take's position, and the splice // skips the lead-in as well (TakeTiming::spliceOffset()). - sv_frame_t playbackStart = currentTakeTiming().playbackStart(); - - sv_frame_t outputLatency = m_playSource->getTargetPlayLatency(); + TakeTiming timing = currentTakeTiming(); + sv_frame_t playbackStart = timing.playbackStart(); + + // All of it in frames of the recording, at the device's rate. + // The play source counts at the reference's rate: bqaudioio's + // ResamplerWrapper converts the device's output latency to + // that rate when it hands it on + sv_frame_t outputLatency = timing.referenceToRecorded + (m_playSource->getTargetPlayLatency()); sv_frame_t inputLatency = m_recordTarget ? m_recordTarget->getSystemRecordLatency() : 0; // // The take is already running by now: record() started it, and @@ -4182,6 +4198,9 @@ MainWindow::recordingStarted() << " total compensation=" << m_recordingLatencyFrames << " frames" << endl; m_recordingStartGapMeasured = -1; + m_recordFramesPerPlayFrame = + (timing.rate > 0 && timing.recordRate > 0 ? + double(timing.recordRate) / double(timing.rate) : 1.0); m_awaitingReferenceStart = true; m_viewManager->setPlaybackFrame(playbackStart); @@ -4222,6 +4241,12 @@ MainWindow::currentTakeTiming() const TakeTiming timing; auto model = getMainModel(); timing.rate = model ? model->getSampleRate() : 0; + // The device's rate, which the recording is made at; not known until + // the recording has been started, and not needed before + if (auto recording = ModelById::getAs + (m_currentRecordingModelId)) { + timing.recordRate = recording->getSampleRate(); + } timing.position = m_takePosition; timing.end = m_takeEnd; timing.preRoll = m_takePreRoll; @@ -4272,8 +4297,9 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) for (const Event &e : m->getAllEvents()) m->remove(e); } - // A negative answer is sound sung during the lead-in of a pre-roll, or - // before the reference started at all: no dot for it + // The frame is the recording's, at the device's rate, and the answer + // the reference's. A negative answer is sound sung during the lead-in + // of a pre-roll, or before the reference started at all: no dot for it sv_frame_t intoTake = currentTakeTiming().liveFrameIntoTake(frame); if (intoTake < 0) return; sv_frame_t dotFrame = m_takePosition + intoTake; @@ -4403,8 +4429,9 @@ MainWindow::finishSingingTake() // A take stopped the moment it was started, or one no longer than the // latency and the lead-in together, has nothing in it to add. Nothing - // has gone wrong; there is simply nothing to do - if (recordingPath != "" && recorded <= offset) { + // has gone wrong; there is simply nothing to do. (The recording is at + // the device's rate, the offset at the reference's.) + if (recordingPath != "" && timing.recordedToReference(recorded) <= offset) { cerr << "MainWindow::finishSingingTake: nothing to use: " << recorded << " frames recorded, the first " << offset << " of which are the latency and the lead-in" << endl; @@ -4430,9 +4457,14 @@ MainWindow::finishSingingTake() // Everything before the offset is sound from before the singer // could have heard the reference at the take's position: the // round trip, and the lead-in of a pre-roll before it. The - // length is what a punch-out allows, or all there is + // length is what a punch-out allows, or all there is. A + // device that does not run at the reference's rate made the + // recording at its own, and it is converted to the + // reference's on the way in: the take's frames are the + // reference's, as all three figures are error = m_takes->spliceRecording(recordingPath, offset, position, - length, directory, &placed); + length, directory, &placed, + timing.rate); } } diff --git a/main/MainWindow.h b/main/MainWindow.h index d27a635d..33aff270 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -821,12 +821,14 @@ protected slots: // what the singer sang would be left unanalysed. Coverage::Range m_takeAnalysisRange; - // Round-trip hardware latency (output + input, in frames at the model - // sample rate) stored when a singing-track recording is made with the - // "play reference while recording" toggle on. The recording is read - // from this frame on when it is spliced into the take's audio, so that - // what the singer sang in answer to the reference at m_takePosition - // lands there; and the live dots are placed with it during the take. + // Round-trip hardware latency (output + input, in frames of the + // recording, at the device's rate) stored when a singing-track + // recording is made with the "play reference while recording" toggle + // on. The recording is read from this point on when it is spliced + // into the take's audio, so that what the singer sang in answer to + // the reference at m_takePosition lands there; and the live dots are + // placed with it during the take. TakeTiming converts it to the + // reference's frames. // Reset to 0 in record() at the start of every take, standalone ones // included, but not by a Stop: the splice needs it after that. // @@ -843,6 +845,12 @@ protected slots: std::atomic m_recordingStartGapMeasured; std::atomic m_awaitingReferenceStart; + // The play source counts at the reference's rate, and the device may + // run at another: frames of the recording per frame of the play + // source, set before the reference is started, for the audio + // callback that measures the start gap + std::atomic m_recordFramesPerPlayFrame; + void refineRecordingLatency(); // The best figure for the latency as things stand, measurement diff --git a/main/SingingTakes.cpp b/main/SingingTakes.cpp index a7764f57..beb764cc 100644 --- a/main/SingingTakes.cpp +++ b/main/SingingTakes.cpp @@ -21,8 +21,10 @@ #include #include #include +#include #include +#include using namespace sv; @@ -308,7 +310,8 @@ SingingTakes::spliceRecording(QString recordingPath, sv_frame_t position, sv_frame_t length, QString directory, - Coverage::Range *placed) + Coverage::Range *placed, + sv_samplerate_t rate) { QString outPath = nextAudioPath(directory); if (outPath == "") { @@ -316,10 +319,27 @@ SingingTakes::spliceRecording(QString recordingPath, "in \"%1\"").arg(directory); } + // A recording at another rate than the take's is converted first, + // into a folder of its own beside the take's files that goes again, + // with what is in it, as soon as the splice has read it + QString source = recordingPath; + std::unique_ptr scratch; + if (rate > 0 && TakeAudio::sampleRate(recordingPath) != rate) { + scratch.reset(new QTemporaryDir + (QDir(directory).filePath("resampling-XXXXXX"))); + if (!scratch->isValid()) { + return tr("Could not make a folder to convert the recording in, " + "in \"%1\"").arg(directory); + } + source = scratch->filePath("recording.wav"); + QString error = TakeAudio::resample(recordingPath, rate, source); + if (error != "") return error; + } + Take &take = takeForRecording(); Coverage::Range range; - QString error = TakeAudio::splice(take.audioPath, recordingPath, + QString error = TakeAudio::splice(take.audioPath, source, recordingOffset, position, length, outPath, &range); if (error != "") return error; diff --git a/main/SingingTakes.h b/main/SingingTakes.h index 22090c3e..09586557 100644 --- a/main/SingingTakes.h +++ b/main/SingingTakes.h @@ -178,6 +178,13 @@ class SingingTakes : public QObject * its end if length is negative. The file is written into * directory, under a name that nothing else is using. * + * The take's audio is at the given rate, the reference's, so that + * its frames are the reference's frames. A recording at any other + * rate -- from a device that runs at 48 kHz, as phones do -- is + * converted to it before it goes in, and recordingOffset and length + * count frames of it as converted. With a rate of 0 the recording + * goes in at its own rate. + * * On success returns "" and the take's audio is the new file, its * coverage takes in what was recorded, and the file before is * remembered as superseded. Otherwise the take is as it was and @@ -188,7 +195,8 @@ class SingingTakes : public QObject sv::sv_frame_t position, sv::sv_frame_t length, QString directory, - Coverage::Range *placed = nullptr); + Coverage::Range *placed = nullptr, + sv::sv_samplerate_t rate = 0); /** * Write the next audio file of the take with the given ranges made diff --git a/main/TakeAudio.cpp b/main/TakeAudio.cpp index 5dd4041d..0c608889 100644 --- a/main/TakeAudio.cpp +++ b/main/TakeAudio.cpp @@ -18,11 +18,14 @@ #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" +#include + #include #include #include #include +#include #include #include @@ -278,3 +281,113 @@ TakeAudio::erase(QString oldPath, const Coverage::Ranges &ranges, return write(old.get(), old->getSampleRate(), old->getChannelCount(), total, patches, fadeFrames, outPath); } + +QString +TakeAudio::resample(QString inPath, sv_samplerate_t rate, QString outPath) +{ + QString error = checkOutPath(outPath, inPath); + if (error != "") return error; + + if (rate <= 0) { + return tr("No sample rate given to convert \"%1\" to").arg(inPath); + } + + auto in = openWav(inPath, error); + if (!in) return error; + + int channels = in->getChannelCount(); + sv_frame_t inFrames = in->getFrameCount(); + double ratio = rate / in->getSampleRate(); + + // As long as the original in seconds, to the frame + sv_frame_t outFrames = sv_frame_t(std::llround(double(inFrames) * ratio)); + + // The quality svcore uses when it opens a file at another rate than + // its own, as it does a reference: libsamplerate's medium sinc + breakfastquay::Resampler::Parameters params; + params.quality = breakfastquay::Resampler::FastestTolerable; + params.initialSampleRate = in->getSampleRate(); + params.maxBufferSize = int(blockFrames); + + // The resampler holds back the last few frames it is given until it + // has seen what follows them, so the end of the file comes out only + // once some silence has gone in after it. One block of silence is + // far more than it holds back; this many is a resampler that is not + // working + const int maxSilentBlocks = 4; + int silentBlocks = 0; + + int outSpace = int(std::ceil(double(blockFrames) * ratio)) + 16; + floatvec_t out(size_t(outSpace) * channels, 0.f); + + { + // To a temporary file that is moved into place on close + WavFileWriter writer(outPath, rate, channels, + WavFileWriter::WriteToTemporary); + if (!writer.isOK()) { + error = writer.getError(); + } + + try { + breakfastquay::Resampler resampler(params, channels); + + sv_frame_t readFrom = 0; + sv_frame_t written = 0; + + while (written < outFrames && error == "") { + + floatvec_t block; + if (readFrom < inFrames) { + sv_frame_t count = std::min(blockFrames, inFrames - readFrom); + block = in->getInterleavedFrames(readFrom, count); + block.resize(size_t(count) * channels, 0.f); + readFrom += count; + } else if (++silentBlocks <= maxSilentBlocks) { + block.assign(size_t(blockFrames) * channels, 0.f); + } else { + error = tr("The resampler stopped short of the end"); + break; + } + + int got = resampler.resampleInterleaved + (out.data(), outSpace, block.data(), + int(block.size() / channels), ratio, false); + + sv_frame_t keep = std::min(sv_frame_t(got), outFrames - written); + if (keep <= 0) continue; + + floatvec_t part(out.begin(), out.begin() + keep * channels); + if (!writer.putInterleavedFrames(part)) { + error = writer.getError(); + if (error == "") error = tr("Failed to write audio data"); + } + written += keep; + } + } catch (const breakfastquay::Resampler::Exception &) { + error = tr("The resampler failed"); + } + + if (error == "" && !writer.close()) { + error = writer.getError(); + if (error == "") error = tr("Failed to finish writing audio file"); + } + } + + // The writer puts its file in place even when it is abandoned + if (error != "") { + QFile::remove(outPath); + return tr("Failed to convert \"%1\" to %2 Hz: %3") + .arg(inPath).arg(rate).arg(error); + } + + return ""; +} + +sv_samplerate_t +TakeAudio::sampleRate(QString path) +{ + QString error; + auto reader = openWav(path, error); + if (!reader) return 0; + return reader->getSampleRate(); +} diff --git a/main/TakeAudio.h b/main/TakeAudio.h index a78f99f1..fd853774 100644 --- a/main/TakeAudio.h +++ b/main/TakeAudio.h @@ -27,9 +27,9 @@ * These functions make the next such file from the last: nothing is * ever changed in place, so the file before is there to go back to. * - * Both work through the files a block at a time, and both return an - * empty string on success and a message for the user otherwise. If - * they fail, there is no file at outPath. + * All of them work through the files a block at a time, and all + * return an empty string on success and a message for the user + * otherwise. If they fail, there is no file at outPath. */ namespace TakeAudio { @@ -78,6 +78,21 @@ namespace TakeAudio const Coverage::Ranges &ranges, QString outPath, sv::sv_frame_t fadeFrames = -1); + + /** + * Write to outPath the audio in inPath at another sample rate: as + * long as it was in seconds, with the same channels, and what was + * at a time in the one at the same time in the other. For a + * recording from a device that does not run at the reference's + * rate, whose frames have to be the reference's before it can go + * into a take. + */ + QString resample(QString inPath, + sv::sv_samplerate_t rate, + QString outPath); + + /// The sample rate of the audio file at path, or 0 if it cannot be read + sv::sv_samplerate_t sampleRate(QString path); } #endif diff --git a/main/TakeTiming.cpp b/main/TakeTiming.cpp index 76281c5c..817e14e4 100644 --- a/main/TakeTiming.cpp +++ b/main/TakeTiming.cpp @@ -32,6 +32,24 @@ QString tr(const char *text) } // namespace +sv_frame_t +TakeTiming::recordedToReference(sv_frame_t recordedFrames) const +{ + if (rate <= 0 || recordRate <= 0 || recordRate == rate) { + return recordedFrames; + } + return sv_frame_t(std::llround(double(recordedFrames) * rate / recordRate)); +} + +sv_frame_t +TakeTiming::referenceToRecorded(sv_frame_t referenceFrames) const +{ + if (rate <= 0 || recordRate <= 0 || recordRate == rate) { + return referenceFrames; + } + return sv_frame_t(std::llround(double(referenceFrames) * recordRate / rate)); +} + sv_frame_t TakeTiming::preRollBefore(sv_frame_t position, sv_frame_t wanted) { @@ -58,7 +76,7 @@ TakeTiming::spliceOffset() const // Everything before this was recorded before the singer could have // heard the reference at P: the round trip, and the lead-in they // were listening to before it - sv_frame_t offset = latency + preRoll; + sv_frame_t offset = recordedToReference(latency) + preRoll; return offset > 0 ? offset : 0; } @@ -73,8 +91,10 @@ sv_frame_t TakeTiming::autoStopFrames() const { if (!havePunchOut()) return 0; - sv_frame_t margin = sv_frame_t(rate * autoStopMarginSeconds()); - return spliceOffset() + (end - position) + margin; + // Counted as the record target counts, in frames of the recording + sv_samplerate_t deviceRate = (recordRate > 0 ? recordRate : rate); + sv_frame_t margin = sv_frame_t(deviceRate * autoStopMarginSeconds()); + return referenceToRecorded(spliceOffset() + (end - position)) + margin; } bool @@ -89,7 +109,8 @@ TakeTiming::liveFrameIntoTake(sv_frame_t recordedFrame) const { // The same shift the splice applies to the audio, so that a dot sits // where the finished pitch track will put the sound it stands for - return compensatedLiveFrame(recordedFrame, spliceOffset()); + return compensatedLiveFrame(recordedToReference(recordedFrame), + spliceOffset()); } bool @@ -97,14 +118,15 @@ TakeTiming::isInLeadIn(sv_frame_t framesReceived) const { // Without a pre-roll there is no lead-in to wait through: what // little comes before the latency is not worth counting down - return preRoll > 0 && framesReceived < spliceOffset(); + return preRoll > 0 && recordedToReference(framesReceived) < spliceOffset(); } int TakeTiming::countdownSeconds(sv_frame_t framesReceived) const { if (rate <= 0 || !isInLeadIn(framesReceived)) return 0; - double seconds = double(spliceOffset() - framesReceived) / rate; + double seconds = + double(spliceOffset() - recordedToReference(framesReceived)) / rate; return int(std::ceil(seconds)); } diff --git a/main/TakeTiming.h b/main/TakeTiming.h index 01bda619..d6d508a1 100644 --- a/main/TakeTiming.h +++ b/main/TakeTiming.h @@ -34,15 +34,26 @@ * Record whatever else happens: pre-roll and punch-out only change * which part of the recording is used and when it stops. * + * The device need not run at the reference's rate (phones run at + * 48 kHz, the reference is always a 44.1 kHz model). P, E and R, and + * anything placed on the reference's timeline, are frames at rate; L + * and anything counted off the record target are frames of the + * recording, at recordRate. The recording is converted to the + * reference's rate before it is spliced, so the splice counts in the + * reference's frames. + * * Nothing here touches a model, a window or the settings, so all of * it is tested without either (TestTakeTiming): MainWindow fills the * fields in and does as the answers say. */ struct TakeTiming { - /// Of the reference and of the recording alike; they must match + /// Of the reference, and of the take's audio sv::sv_samplerate_t rate; + /// Of the recording: the device's. 0 means the same as rate + sv::sv_samplerate_t recordRate; + /// P: where the material recorded from now on is to land sv::sv_frame_t position; @@ -52,11 +63,19 @@ struct TakeTiming /// R: the lead-in played before P, never taking S below frame 0 sv::sv_frame_t preRoll; - /// L: what was recorded before the singer could hear the reference at S + /// L: what was recorded before the singer could hear the reference + /// at S, in frames of the recording sv::sv_frame_t latency; TakeTiming() : - rate(0), position(0), end(-1), preRoll(0), latency(0) { } + rate(0), recordRate(0), position(0), end(-1), preRoll(0), + latency(0) { } + + /// Frames of the recording as frames of the reference's timeline + sv::sv_frame_t recordedToReference(sv::sv_frame_t recordedFrames) const; + + /// Frames of the reference's timeline as frames of the recording + sv::sv_frame_t referenceToRecorded(sv::sv_frame_t referenceFrames) const; /** * The lead-in there is room for before position: the pre-roll @@ -81,7 +100,10 @@ struct TakeTiming /// The take has an end to stop itself at bool havePunchOut() const; - /// The frame of the recording that the take's new material starts at + /** + * The frame of the recording that the take's new material starts + * at, once the recording is at the reference's rate + */ sv::sv_frame_t spliceOffset() const; /// How much of the recording to use, or -1 for all there is of it @@ -99,9 +121,9 @@ struct TakeTiming /** * Where sound found at this frame of the recording belongs, - * counted from P. Negative means it was sung during the lead-in, - * before the take's own material begins, and has no place on the - * reference's timeline. + * counted in the reference's frames from P. Negative means it was + * sung during the lead-in, before the take's own material begins, + * and has no place on the reference's timeline. */ sv::sv_frame_t liveFrameIntoTake(sv::sv_frame_t recordedFrame) const; diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d1614237..06da55b7 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -879,6 +879,243 @@ class TestRecordWorkflow : public QObject return signal; } + // --- A device that does not run at the reference's rate --- + + // What phones run at. The reference is a model at 44.1 kHz whatever + // its file was, and the device records at its own rate + static constexpr int otherDeviceRate = 48000; + + // Sung at the device's rate: lowHz and highHz, as sines. pYIN gets + // the take at 44.1 kHz, where these two have whole numbers of samples + // per period; tones without (240 and 320 Hz, sines or sawtooths) came + // out as subharmonics. Sines, because a sawtooth made at 48 kHz that + // is not whole there aliases into partials that are not harmonics, + // and the sawtooths whole at both rates (100, 150, 300 Hz) came out + // an octave low after a step, or not, as the step fell between frames + static std::vector toneAt(double hz, double seconds, + int sampleRate) { + return TestSignals::sine(hz, sampleRate, int(seconds * sampleRate)); + } + + static std::vector twoTonesAt(double firstHz, double first, + double thenHz, double then, + int sampleRate) { + auto signal = toneAt(firstHz, first, sampleRate); + auto second = toneAt(thenHz, then, sampleRate); + signal.insert(signal.end(), second.begin(), second.end()); + return signal; + } + + // Record from P for ms milliseconds. recordedFrames receives the + // length of the recording the device made, which must be at its + // rate + void recordAt(sv::sv_frame_t P, int ms, int deviceRate, + sv::sv_frame_t &recordedFrames) { + recordedFrames = -1; + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QString path; + { + // Not held past this: the recording is to be released, and + // its file closed, when the take stops + auto recording = sv::ModelById::getAs + (m_window->currentRecordingModelId()); + QVERIFY(recording); + path = recording->getLocation(); + } + QTest::qWait(ms); + stopTake(); + if (QTest::currentTestFailed()) return; + sv::WavFileReader reader { sv::FileSource(path) }; + QVERIFY(reader.isOK()); + QCOMPARE(int(reader.getSampleRate()), deviceRate); + recordedFrames = reader.getFrameCount(); + } + + // The first and the last frame of [from, to) at which the take's + // audio model is louder than a whisper, or -1 for both. The model, + // not the file: it is what is played and what pYIN is given, and it + // is on the reference's timeline whatever the file is + void soundInTake(sv::sv_frame_t from, sv::sv_frame_t to, + sv::sv_frame_t &first, sv::sv_frame_t &last) { + first = last = -1; + auto wave = takeAudio(); + if (!wave || to <= from) return; + auto data = wave->getData(0, from, to - from); + for (size_t i = 0; i < data.size(); ++i) { + if (std::fabs(data[i]) > 0.05f) { + if (first < 0) first = from + sv::sv_frame_t(i); + last = from + sv::sv_frame_t(i); + } + } + } + + // Where the singing of a recording made at deviceRate has to be: + // from P, for as long as it was sung, in the take's coverage, audio + // and pitch alike. The coverage range holding P is returned in range + void verifyRecordingPlaced(sv::sv_frame_t P, sv::sv_frame_t recordedFrames, + int deviceRate, Coverage::Range &range) { + const sv::sv_frame_t margin = sv::sv_frame_t(0.1 * rate); + const sv::sv_frame_t close = sv::sv_frame_t(0.01 * rate); + + range = Coverage::Range(); + for (const auto &r : m_window->takes()->getCoverage().getRanges()) { + if (r.start <= P && P < r.end) range = r; + } + QVERIFY2(range.length() > 0, + qPrintable(QString("no coverage holds the position %1") + .arg(P))); + + // As many seconds as the device recorded, at the reference's rate + sv::sv_frame_t want = sv::sv_frame_t + (std::llround(double(recordedFrames) * rate / deviceRate)); + QCOMPARE(range.start, P); + QVERIFY2(std::llabs(range.length() - want) <= 1, + qPrintable(QString("%1 frames were recorded at %2 Hz, which " + "is %3 at %4 Hz; the take covers %5") + .arg(recordedFrames).arg(deviceRate).arg(want) + .arg(rate).arg(range.length()))); + + // The file is at the reference's rate, so its frames are the + // reference's frames + QString path = m_window->takes()->getAudioPath(); + sv::WavFileReader file { sv::FileSource(path) }; + QVERIFY(file.isOK()); + QVERIFY2(file.getSampleRate() == rate, + qPrintable(QString("the take's audio file is at %1 Hz, not " + "the reference's %2") + .arg(file.getSampleRate()).arg(rate))); + + // The sound is where the coverage says it is + auto wave = takeAudio(); + QVERIFY(wave); + QTRY_VERIFY(wave->isReady()); + sv::sv_frame_t first = -1, last = -1; + soundInTake(std::max(sv::sv_frame_t(0), P - margin), + range.end + margin, first, last); + QVERIFY2(std::llabs(first - range.start) <= close && + std::llabs(last - range.end) <= close, + qPrintable(QString("the take's audio is loud over [%1,%2]; " + "the recording went into [%3,%4)") + .arg(first).arg(last) + .arg(range.start).arg(range.end))); + + // and so is the pitch, at the pitch that was sung + auto events = eventsBetween(pitchEvents(m_window->analyser2()), + range.start - margin, range.end + margin); + QVERIFY(!events.empty()); + QVERIFY2(std::llabs(events.front().getFrame() - range.start) <= margin && + std::llabs(events.back().getFrame() - range.end) <= margin, + qPrintable(QString("the take's pitch runs from %1 to %2; the " + "recording went into [%3,%4)") + .arg(events.front().getFrame()) + .arg(events.back().getFrame()) + .arg(range.start).arg(range.end))); + } + + // A first recording at P1 and a second in the gap after it, from a + // device at deviceRate against the reference at 44.1 kHz + void verifyTakesPlacedFromDeviceAt(int deviceRate) { + const double sungHz = highHz; + FakeAudioIO::Config config; + config.sampleRate = deviceRate; + config.input = toneAt(sungHz, 3.0, deviceRate); + makeWindow(config); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P1 = sv::sv_frame_t(1.0 * rate); + const sv::sv_frame_t P2 = sv::sv_frame_t(3.0 * rate); + const sv::sv_frame_t margin = sv::sv_frame_t(0.1 * rate); + + // As recordAt() does, with a look at the cursor on the way: while + // the take records, it runs from the position at the reference's + // rate + m_window->seekTo(P1); + startTake(); + if (QTest::currentTestFailed()) return; + QString firstPath; + { + auto recording = sv::ModelById::getAs + (m_window->currentRecordingModelId()); + QVERIFY(recording); + firstPath = recording->getLocation(); + } + QTest::qWait(500); + sv::sv_frame_t during = m_window->playbackFrame(); + sv::sv_frame_t recordedSoFar = + m_window->recordTarget()->getRecordDuration(); + sv::sv_frame_t cursorWant = P1 + sv::sv_frame_t + (std::llround(double(recordedSoFar) * rate / deviceRate)); + if (deviceRate != int(rate)) { + QEXPECT_FAIL("", "ViewManager::getPlaybackFrame() adds the " + "recorded duration in the device's frames to the " + "record start frame (svgui fork)", Continue); + } + QVERIFY2(std::llabs(during - cursorWant) <= 1, + qPrintable(QString("%1 frames at %2 Hz into a take from " + "frame %3, the cursor is at %4, not %5") + .arg(recordedSoFar).arg(deviceRate).arg(P1) + .arg(during).arg(cursorWant))); + QTest::qWait(500); + stopTake(); + if (QTest::currentTestFailed()) return; + + sv::sv_frame_t firstFrames = -1; + { + sv::WavFileReader reader { sv::FileSource(firstPath) }; + QVERIFY(reader.isOK()); + QCOMPARE(int(reader.getSampleRate()), deviceRate); + firstFrames = reader.getFrameCount(); + } + + Coverage::Range first; + verifyRecordingPlaced(P1, firstFrames, deviceRate, first); + if (QTest::currentTestFailed()) return; + + auto wave = takeAudio(); + QVERIFY(wave); + QCOMPARE(wave->getFrameCount(), first.end); + QVERIFY2(std::fabs(TestSignals::centsBetween + (medianHz(pitchEvents(m_window->analyser2())), + sungHz)) < 10.0, + qPrintable(QString("the take's median pitch is %1 Hz; it " + "was sung at %2") + .arg(medianHz(pitchEvents(m_window->analyser2()))) + .arg(sungHz))); + sv::sv_frame_t loudFirst = -1, loudLast = -1; + soundInTake(0, P1 - margin, loudFirst, loudLast); + QVERIFY2(loudFirst < 0, "the take's audio is not silent before the " + "position it was recorded at"); + + // A second recording, into the gap after the first: the take so + // far is at the reference's rate, whatever the recording is at + sv::sv_frame_t secondFrames = -1; + recordAt(P2, 800, deviceRate, secondFrames); + if (QTest::currentTestFailed()) return; + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 2); + QCOMPARE(ranges[0], first); + + Coverage::Range second; + verifyRecordingPlaced(P2, secondFrames, deviceRate, second); + if (QTest::currentTestFailed()) return; + + wave = takeAudio(); + QVERIFY(wave); + QTRY_COMPARE(wave->getFrameCount(), second.end); + + soundInTake(first.end + margin, P2 - margin, loudFirst, loudLast); + QVERIFY2(loudFirst < 0, + "the gap between the two recordings is not silent"); + QVERIFY2(eventsBetween(pitchEvents(m_window->analyser2()), + first.end + margin, P2 - margin).empty(), + "the pitch track has something in the gap between the two " + "recordings"); + } + // Not a slot: QtTest would run it as a test void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -1586,6 +1823,193 @@ private slots: "recordings"); } + // A device at 48 kHz, as phones are, against a reference that is a + // 44.1 kHz model: the recording is made at the device's rate and the + // take is on the reference's timeline. Two recordings, the second in + // the gap after the first, must each land where and be as long as + // they were sung. The same at 44.1 kHz, where nothing is converted + void takes_placed_from_a_device_at_44100() { + verifyTakesPlacedFromDeviceAt(44100); + } + + void takes_placed_from_a_device_at_48000() { + verifyTakesPlacedFromDeviceAt(otherDeviceRate); + } + + // The singer of take_latency_removed_by_splice, exactly on time, on a + // device at 48 kHz: the round trip and the start gap are counted in + // the device's frames, the reference in its own, and the step the + // singer sings has to land where the reference steps all the same, + // in the live dots and in the pitch track + void latency_with_a_device_at_48000() { + const int K = 3 * 4096; + FakeAudioIO::Config config; + config.sampleRate = otherDeviceRate; + config.playbackLatency = 2 * 4096; + config.recordLatency = 4096; + config.input = twoTonesAt(lowHz, 0.75, highHz, 0.75, + otherDeviceRate); + config.inputDelay = K; + config.inputFollowsPlayback = true; + // The play source hands the device the reference in blocks counted + // at the reference's rate, and a block of the device's is 8% more + // of them at 48 kHz: big enough a block for that to show in the + // start gap + config.blockSize = 2048; + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + + // The reference steps 0.75 s after the take's position, as the + // singer does 0.75 s after hearing it there + const sv::sv_frame_t P = sv::sv_frame_t(1.5 * rate); + openReference(writeWav(twoNotes(1.5 + 0.75, 0.75))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(2200); + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + // The dots are on the reference's timeline, and at its rate + QCOMPARE(model->getSampleRate(), sv::sv_samplerate_t(rate)); + auto dots = model->getAllEvents(); + stopTake(); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, + "the reference was never played"); + + // The compensation is in the device's frames: the round trip it + // reports, and the start gap it knows the real length of. Not to + // within a few frames, as at 44.1 kHz: bqaudioio's ResamplerWrapper, + // which brings the reference to the device's rate, holds it back by + // some tens of frames of its own that nothing reports (about 25 by + // the first audible sample, half a millisecond) + sv::sv_frame_t gap = m_window->fake()->getFramesBeforePlayStart(); + sv::sv_frame_t assumedGap = m_window->recordingLatencyFrames() - K; + QVERIFY2(std::llabs(assumedGap - gap) <= 64, + qPrintable(QString("the application took the start gap to " + "be %1 frames; it was %2") + .arg(assumedGap).arg(gap))); + + sv::sv_frame_t refStep = stepFrame(pitchEvents(m_window->analyser())); + sv::sv_frame_t dotStep = stepFrame(dots); + sv::sv_frame_t sungStep = + stepFrame(pitchEvents(m_window->analyser2())); + QVERIFY(refStep > P); + QVERIFY2(dotStep > 0, "the live dots never reached the second note"); + QVERIFY2(sungStep > 0, "the take never reached the second note"); + + // The dots are coarser than pYIN: allow them a YIN window + QVERIFY2(std::llabs(dotStep - refStep) <= 2048, + qPrintable(QString("live step at %1, reference step at %2: " + "%3 frames apart") + .arg(dotStep).arg(refStep) + .arg(dotStep - refStep))); + QVERIFY2(std::llabs(sungStep - refStep) <= 2 * hop, + qPrintable(QString("sung step at %1, reference step at %2: " + "%3 frames (%4 ms) apart") + .arg(sungStep).arg(refStep) + .arg(sungStep - refStep) + .arg(1000.0 * double(sungStep - refStep) / rate, + 0, 'f', 1))); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0].start, P); + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + QVERIFY(events.front().getFrame() >= P - 4 * hop); + } + + // Record into Selection with a pre-roll, on a device at 48 kHz: the + // lead-in and the selection are on the reference's timeline and the + // frames the device delivers are at its own rate. The take has to + // wait for the whole of the selection before it stops itself, and + // keep exactly the selection + void preroll_and_punch_out_with_a_device_at_48000() { + const double leadIn = 0.5; + FakeAudioIO::Config config; + config.sampleRate = otherDeviceRate; + // High during the lead-in, low for a little longer than the + // selection, then high again: only the low note belongs in the take + config.input = twoTonesAt(highHz, leadIn, lowHz, 0.85, + otherDeviceRate); + auto after = toneAt(highHz, 0.8, otherDeviceRate); + config.input.insert(config.input.end(), after.begin(), after.end()); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(false); + m_window->setRecordIntoSelection(true); + setPreRollSeconds(leadIn); + m_window->setPreRoll(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(1.0 * rate); + const sv::sv_frame_t E = sv::sv_frame_t(1.8 * rate); + m_window->selectRange(P, E); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePosition(), P); + QCOMPARE(m_window->takeEnd(), E); + QCOMPARE(m_window->takePreRoll(), sv::sv_frame_t(leadIn * rate)); + QString path; + { + auto recording = sv::ModelById::getAs + (m_window->currentRecordingModelId()); + QVERIFY(recording); + path = recording->getLocation(); + } + + QTRY_VERIFY_WITH_TIMEOUT(!m_window->recordTarget()->isRecording(), 5000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + // It stopped once the lead-in, the selection and most of the + // margin had been recorded, counted at the device's rate + sv::WavFileReader reader { sv::FileSource(path) }; + QVERIFY(reader.isOK()); + sv::sv_frame_t needed = sv::sv_frame_t + ((leadIn + double(E - P) / rate + + 0.8 * TakeTiming::autoStopMarginSeconds()) * otherDeviceRate); + QVERIFY2(reader.getFrameCount() >= needed, + qPrintable(QString("the take stopped itself after %1 frames " + "at %2 Hz; the lead-in, the selection " + "and the margin are %3") + .arg(reader.getFrameCount()).arg(otherDeviceRate) + .arg(needed))); + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(int(ranges.size()), 1); + QCOMPARE(ranges[0], Coverage::Range(P, E)); + + auto wave = takeAudio(); + QVERIFY(wave); + QTRY_VERIFY(wave->isReady()); + QCOMPARE(wave->getFrameCount(), E); + sv::sv_frame_t first = -1, last = -1; + soundInTake(0, E + sv::sv_frame_t(0.1 * rate), first, last); + const sv::sv_frame_t close = sv::sv_frame_t(0.01 * rate); + QVERIFY2(std::llabs(first - P) <= close && + std::llabs(last - E) <= close, + qPrintable(QString("the take's audio is loud over [%1,%2]; " + "the selection is [%3,%4)") + .arg(first).arg(last).arg(P).arg(E))); + + // What it kept is the note sung inside the selection, not the one + // of the lead-in or the one after its end + auto events = pitchEvents(m_window->analyser2()); + QVERIFY(!events.empty()); + QVERIFY2(std::fabs(TestSignals::centsBetween + (medianHz(events), lowHz)) < 10.0, + qPrintable(QString("the take's median pitch is %1 Hz; the " + "selection was sung at %2, the lead-in " + "and what came after at %3") + .arg(medianHz(events)).arg(lowHz).arg(highHz))); + } + // Stop no longer analyses the whole of the take's audio: the new // audio goes under the pitch and notes layers that are there and only // the range the recording went into is analysed and merged into them. diff --git a/main/test/TestSingingTakes.h b/main/test/TestSingingTakes.h index a5f1afea..dd5363ad 100644 --- a/main/test/TestSingingTakes.h +++ b/main/test/TestSingingTakes.h @@ -20,6 +20,7 @@ // read back. #include "../SingingTakes.h" +#include "../TakeAudio.h" #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" @@ -48,10 +49,10 @@ class TestSingingTakes : public QObject int m_fileCounter = 0; // A recording of the given length, at a level that says which one it is - QString writeRecording(frame_t frames, float level) { + QString writeRecording(frame_t frames, float level, double rate = kRate) { QString path = m_dir.filePath (QString("recorded-%1.wav").arg(++m_fileCounter)); - sv::WavFileWriter writer(path, kRate, 1, + sv::WavFileWriter writer(path, rate, 1, sv::WavFileWriter::WriteToTarget); sv::floatvec_t data(frames, level); if (!writer.isOK() || !writer.putInterleavedFrames(data) || @@ -436,6 +437,60 @@ private slots: QVERIFY(!error.isEmpty()); QCOMPARE(takes.getAudioPath(), path); QVERIFY(takes.getCoverage() == before); + + // nor when it has to be converted first + error = takes.spliceRecording(m_dir.filePath("not-a-file.wav"), 0, 0, + -1, takeDirectory(), nullptr, 48000.0); + QVERIFY(!error.isEmpty()); + QCOMPARE(takes.getAudioPath(), path); + QVERIFY(takes.getCoverage() == before); + } + + // A recording from a device that does not run at the take's rate, the + // reference's, goes in at the take's rate: a second of it is a second + // of the take, and the offset, the position and the length are all + // frames at that rate. Twice, the second time into a take that is at + // the take's rate already + void splice_converts_a_recording_at_another_rate() { + SingingTakes takes; + QString recording = writeRecording(48000, 0.5f, 48000.0); + QString second = writeRecording(24000, 0.25f, 48000.0); + QVERIFY(!recording.isEmpty() && !second.isEmpty()); + + Coverage::Range placed; + QCOMPARE(takes.spliceRecording(recording, 4410, 2000, -1, + takeDirectory(), &placed, kRate), + QString()); + QCOMPARE(placed, Coverage::Range(2000, 2000 + 44100 - 4410)); + + QString path = takes.getAudioPath(); + QCOMPARE(TakeAudio::sampleRate(path), kRate); + QCOMPARE(framesIn(path), placed.end); + QVERIFY(std::fabs(sampleAt(path, 20000) - 0.5f) < 1e-3f); + + QCOMPARE(takes.spliceRecording(second, 0, 60000, 11025, + takeDirectory(), &placed, kRate), + QString()); + QCOMPARE(placed, Coverage::Range(60000, 71025)); + QCOMPARE(int(takes.getCoverage().getRanges().size()), 2); + + path = takes.getAudioPath(); + QCOMPARE(TakeAudio::sampleRate(path), kRate); + QCOMPARE(framesIn(path), frame_t(71025)); + QVERIFY(std::fabs(sampleAt(path, 20000) - 0.5f) < 1e-3f); + QVERIFY(std::fabs(sampleAt(path, 65000) - 0.25f) < 1e-3f); + + // The recordings are as they were, and nothing of the conversion + // is left beside the take's files + QCOMPARE(TakeAudio::sampleRate(recording), 48000.0); + QCOMPARE(framesIn(recording), frame_t(48000)); + QStringList left = QDir(takeDirectory()).entryList + (QDir::AllEntries | QDir::NoDotAndDotDot); + for (const QString &name : left) { + QVERIFY2(name.startsWith("take-") && name.endsWith(".wav"), + qPrintable(QString("\"%1\" was left in the takes folder") + .arg(name))); + } } // Erasing from the middle of a recording: the file is as long as it diff --git a/main/test/TestTakeAudio.h b/main/test/TestTakeAudio.h index cde5eed1..b79c020d 100644 --- a/main/test/TestTakeAudio.h +++ b/main/test/TestTakeAudio.h @@ -20,6 +20,8 @@ #include "../TakeAudio.h" +#include "TestSignals.h" + #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" @@ -452,6 +454,122 @@ private slots: QVERIFY(TakeAudio::erase(old, ranges, old) != ""); QVERIFY(allNearly(read(old), 0, 3000, 0.5f)); } + + // Converting a recording from a device that does not run at the + // reference's rate. As long as it was in seconds, to the frame, in + // as many channels, kept apart + void resample_keeps_the_length_and_the_channels() { + const frame_t n = 12345; + Signal stereo; + for (frame_t i = 0; i < n; ++i) { + stereo.push_back(0.25f); + stereo.push_back(-0.25f); + } + QString in = writeWav(stereo, 2, 48000.0); + QString out = newPath(); + QCOMPARE(TakeAudio::resample(in, 44100.0, out), QString()); + QCOMPARE(TakeAudio::sampleRate(out), 44100.0); + Audio a = read(out); + QVERIFY(a.ok); + QCOMPARE(a.channels, 2); + QCOMPARE(a.frames, frame_t(std::llround(n * 44100.0 / 48000.0))); + QVERIFY(allNearly(a, 1000, a.frames - 1000, 0.25f, 0)); + QVERIFY(allNearly(a, 1000, a.frames - 1000, -0.25f, 1)); + + // Upwards, and longer than the blocks the work is done in + const frame_t m = 100000; + in = writeWav(ramp(m), 1, 44100.0); + out = newPath(); + QCOMPARE(TakeAudio::resample(in, 48000.0, out), QString()); + QCOMPARE(TakeAudio::sampleRate(out), 48000.0); + a = read(out); + QCOMPARE(a.frames, frame_t(std::llround(m * 48000.0 / 44100.0))); + + // The original is as it was + QCOMPARE(TakeAudio::sampleRate(in), 44100.0); + QCOMPARE(read(in).frames, m); + } + + // A click is where it was in time, to within a frame: near the + // start, across the boundary of two of the blocks the work is done + // in, and near the end, which the resampler holds back until + // something follows it + void resample_keeps_clicks_where_they_were() { + const frame_t n = 50000; + const std::vector clicks { 1000, 16390, 33000, n - 200 }; + for (double from : { 48000.0, 44100.0 }) { + double to = (from == 48000.0 ? 44100.0 : 48000.0); + Signal s(n, 0.f); + for (frame_t c : clicks) s[c] = 0.9f; + QString in = writeWav(s, 1, from); + QString out = newPath(); + QCOMPARE(TakeAudio::resample(in, to, out), QString()); + Audio a = read(out); + QVERIFY(a.ok); + for (frame_t c : clicks) { + double want = double(c) * to / from; + frame_t peak = -1; + float loudest = 0.f; + for (frame_t i = frame_t(want) - 50; i <= frame_t(want) + 50; + ++i) { + if (i < 0 || i >= a.frames) continue; + if (std::fabs(a.at(i)) > loudest) { + loudest = std::fabs(a.at(i)); + peak = i; + } + } + QVERIFY2(loudest > 0.3f && std::fabs(double(peak) - want) <= 1.0, + qPrintable(QString("a click at frame %1 at %2 Hz " + "belongs at %3 at %4 Hz; the " + "loudest frame near it is %5, " + "at %6") + .arg(c).arg(from).arg(want).arg(to) + .arg(peak).arg(loudest))); + } + } + } + + // A tone keeps its pitch, its phase and its level: frame for frame, + // the same sine at the new rate + void resample_keeps_a_tone() { + const double hz = 1000.0, from = 48000.0, to = 44100.0; + QString in = writeWav(TestSignals::sine(hz, from, int(from)), 1, from); + QString out = newPath(); + QCOMPARE(TakeAudio::resample(in, to, out), QString()); + Audio a = read(out); + QCOMPARE(a.frames, frame_t(to)); + // Away from the ends, where the tone starts and stops abruptly + for (frame_t i = 1000; i < a.frames - 1000; ++i) { + float want = float(0.5 * std::sin(2.0 * TestSignals::kPi * hz * + double(i) / to)); + QVERIFY2(std::fabs(a.at(i) - want) < 0.005f, + qPrintable(QString("frame %1 is %2; a %3 Hz sine at " + "%4 Hz is %5 there") + .arg(i).arg(a.at(i)).arg(hz).arg(to) + .arg(want))); + } + } + + void resample_errors() { + QString in = writeWav(constant(1000, 0.5f), 1, 48000.0); + QString other = writeWav(constant(1000, 0.25f)); + + QString out = newPath(); + QVERIFY(TakeAudio::resample(m_dir.filePath("absent.wav"), 44100.0, + out) != ""); + QVERIFY(TakeAudio::resample(in, 0.0, out) != ""); + QVERIFY(TakeAudio::resample(in, 44100.0, "") != ""); + QVERIFY(!QFile::exists(out)); + + // never over a file that is there, the one read from least of all + QVERIFY(TakeAudio::resample(in, 44100.0, in) != ""); + QVERIFY(TakeAudio::resample(in, 44100.0, other) != ""); + QVERIFY(allNearly(read(in), 0, 1000, 0.5f)); + QVERIFY(allNearly(read(other), 0, 1000, 0.25f)); + + QCOMPARE(TakeAudio::sampleRate(in), 48000.0); + QCOMPARE(TakeAudio::sampleRate(m_dir.filePath("absent.wav")), 0.0); + } }; #endif diff --git a/main/test/TestTakeTiming.h b/main/test/TestTakeTiming.h index 25e7befb..a68813b0 100644 --- a/main/test/TestTakeTiming.h +++ b/main/test/TestTakeTiming.h @@ -166,6 +166,63 @@ private slots: QCOMPARE(plain.countdownText(0), QString()); } + // A device at 48 kHz, as phones are, against a reference at 44.1: + // P, E and R are the reference's frames, L and every count off the + // record target the recording's, and each answer is in the frames it + // is used in + void device_at_another_rate() { + const double deviceRate = 48000.0; + TakeTiming t = take(frame_t(10 * kRate), frame_t(0.5 * kRate), + frame_t(0.1 * deviceRate)); + t.recordRate = deviceRate; + + QCOMPARE(t.recordedToReference(frame_t(deviceRate)), frame_t(kRate)); + QCOMPARE(t.referenceToRecorded(frame_t(kRate)), frame_t(deviceRate)); + QCOMPARE(t.recordedToReference(12345), frame_t(11342)); + + // The splice reads the recording once it is at the reference's + // rate: the latency is converted, the lead-in is not + QCOMPARE(t.spliceOffset(), frame_t(0.1 * kRate) + frame_t(0.5 * kRate)); + QCOMPARE(t.playbackStart(), frame_t(9.5 * kRate)); + + // The record target counts the device's frames: the take waits + // for the latency, the lead-in, two seconds of selection and the + // margin, all at the device's rate + t.end = t.position + frame_t(2 * kRate); + QCOMPARE(t.spliceLength(), frame_t(2 * kRate)); + frame_t want = frame_t(0.1 * deviceRate) + frame_t(0.5 * deviceRate) + + frame_t(2 * deviceRate) + + frame_t(TakeTiming::autoStopMarginSeconds() * deviceRate); + QCOMPARE(t.autoStopFrames(), want); + QVERIFY(!t.shouldStopAt(want - 1)); + QVERIFY(t.shouldStopAt(want)); + + // A dot for what the device recorded one second after the lead-in + // is one second into the take, at the reference's rate + frame_t lead = frame_t(0.1 * deviceRate) + frame_t(0.5 * deviceRate); + QCOMPARE(t.liveFrameIntoTake(lead), frame_t(0)); + QCOMPARE(t.liveFrameIntoTake(lead + frame_t(deviceRate)), frame_t(kRate)); + QVERIFY(t.liveFrameIntoTake(lead - 100) < 0); + + // The countdown goes by the device's frames too + TakeTiming c = take(frame_t(10 * kRate), frame_t(3 * kRate)); + c.recordRate = deviceRate; + QCOMPARE(c.countdownSeconds(0), 3); + QCOMPARE(c.countdownSeconds(frame_t(1.5 * deviceRate)), 2); + QCOMPARE(c.countdownSeconds(frame_t(3 * deviceRate) - 100), 1); + QVERIFY(c.isInLeadIn(frame_t(3 * deviceRate) - 100)); + QVERIFY(!c.isInLeadIn(frame_t(3 * deviceRate))); + QCOMPARE(c.countdownSeconds(frame_t(3 * deviceRate)), 0); + + // At the reference's rate, or with none given, nothing changes + TakeTiming same = take(5000, 3000, 1500); + same.recordRate = kRate; + QCOMPARE(same.recordedToReference(12345), frame_t(12345)); + QCOMPARE(same.spliceOffset(), frame_t(4500)); + QCOMPARE(take(5000, 3000, 1500).recordedToReference(12345), + frame_t(12345)); + } + // Recording into a selection: the one the playhead is in, else the // first of them void selection_at_the_playhead_is_the_one_recorded_into() { From 688fd5067172b6a526fe52a0c60b7249fc189128 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:24:45 +0000 Subject: [PATCH 086/275] docs: phase a1 done, its log entry and the state after it Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 44 +++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 64043b1f..9a8fa7c2 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -121,12 +121,22 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: | No Qt Multimedia | Its input path is not low-latency and reports no latency (port-android.md, Audio) | | Features beyond the spec (latency calibration setting, session bundle) are not built unless a phone test shows they are needed | The spec says so | -## 4. State of the code (kept by the lead; as of 2026-09-25, after phase A0) +## 4. State of the code (kept by the lead; as of 2026-09-25, after phase A1) -- Nothing of the port exists yet. The branch holds `default`, the research docs, and the - container setup script `deploy/linux/container-setup.sh`. +- The branch holds `default`, the research docs, the container setup script + `deploy/linux/container-setup.sh`, and A1's fix. - Both suites pass on Linux (Qt 6.11.2 from conda-forge). `TestTakesFile`'s Windows path assertions run on Windows only. +- A device at another rate than the reference's works: the recording is resampled to the + reference's rate before the splice (`SingingTakes::spliceRecording(..., rate)`, + `TakeAudio::resample()`), and `TakeTiming` converts between device frames (latency, + frames received, the live tracker) and reference frames (`recordRate`, + `recordedToReference()`, `referenceToRecorded()`). An Oboe backend at 48 kHz needs + nothing more from Tony for placement. +- Known and not fixed: while recording at a device rate other than 44.1 kHz the play + cursor runs fast (svgui's `ViewManager::getPlaybackFrame()` adds device frames). A + `QEXPECT_FAIL` in `takes_placed_from_a_device_at_48000` marks it. It needs changes in + svcore, svgui and svapp; the lead has not made them. ## 5. Phases @@ -136,7 +146,7 @@ needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt. builds happen in the container.) - A0 — Desktop build and tests in the container. Done. -- A1 — Sample rate: a device that is not at 44.1 kHz. +- A1 — Sample rate: a device that is not at 44.1 kHz. Done. - A2 — Android toolchain and C libraries. - A3 — Tony as an APK (no audio): the test port. - A4 — Touch gestures on the panes. @@ -302,3 +312,29 @@ Left open: no model has: a warning in every recording test, on every platform. Not touched. - architecture.md "Signals" says such string connects never match: true for pointers (`Layer *`), but registered types match from Qt 6.5. For A8. + +### Phase A1 — 2026-09-25 +Built: the misplacement was real (`FakeAudioIO` at 48 kHz: take audio at 0.919 × its +place, coverage 8.8% too long). `TakeAudio::resample()` (streaming, bqresample's +libsamplerate medium sinc, as svcore uses) and `sampleRate()`; `spliceRecording(..., rate)` +converts a recording at another rate in a `QTemporaryDir` beside the take's files before +the splice, so take WAVs are always at the reference's rate. `TakeTiming::recordRate`: +L and every count off the record target are device frames; `spliceOffset()`, +`liveFrameIntoTake()` and the countdown answer in the reference's, `autoStopFrames()` in +the device's. `MainWindow` fills `recordRate` from the recording model, converts the play +source's output latency and block (`m_recordFramesPerPlayFrame`) to device frames, and +gives the live-dot model the main model's rate. +Choices / deviations: +- Sung test input at 48 kHz is sines at 220.5/294 Hz: pYIN needs whole-sample periods at + 44.1 kHz, and sawtooths whole at both rates went an octave low by alignment. +- `getTargetPlayLatency()` is taken as reference-rate frames: bqaudioio's ResamplerWrapper + converts it only if the device opened after a model was loaded (else ~1-3 ms off). +- The 48 kHz start gap is ~25 frames short of the first audible sample: the wrapper's + resampler delays playback and reports nothing. The test allows 64 frames. +The next phase must know: recordings stay at the device's rate; take audio is converted. +Left open: +- svgui fork: while recording, `ViewManager::getPlaybackFrame()` adds device frames, so the + cursor runs 8.8% fast at 48 kHz (QEXPECT_FAIL in `takes_placed_from_a_device_at_48000`). +- A loaded singing track not at 44.1 kHz: the splice refuses it and erase misplaces. +- For A8: takes.md "Known limitations" (last bullet), open-points.md weak spot, + recording.md "Latency" (units), testing.md (tones for another device rate). From 3e8c5d2d5aa85afdf4392d7b0710d254e9ff1d6c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:31:23 +0000 Subject: [PATCH 087/275] test: waitForRange() waits for the merge itself The ranged-analysis tests of TestSingingAnalysis took initialAnalysisCompleted as the sign that the range had been merged. The whole-file analysis each of them runs first can still be delivering its own completion when the range starts, and Analyser::layerCompletionChanged() emits the same signal for it, so now and then isAnalysingRange() was still true when it was read (ranged_range_shorter_than_the_margin, once in a full run on Linux). Wait for the flag, as docs/testing.md says to. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- main/test/TestSingingAnalysis.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 67b4a48d..02129077 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -217,12 +217,15 @@ class TestSingingAnalysis : public QObject QVERIFY(noteEvents(analyser).empty()); } - // Wait for a ranged analysis to be merged + // Wait for a ranged analysis to be merged. The completion signal + // alone does not say so: the whole-file analysis before it can still + // be delivering its own, late, when the range starts void waitForRange(Analyser &analyser, QSignalSpy &done) { QVERIFY2(done.count() > 0 || done.wait(30000), "the ranged analysis did not complete within 30 seconds"); - QVERIFY2(!analyser.isAnalysingRange(), - "the analyser still says a range is being analysed"); + QTRY_VERIFY2_WITH_TIMEOUT(!analyser.isAnalysingRange(), + "the analyser still says a range is being " + "analysed", 30000); } // Nothing of a ranged analysis may be left behind in the document From 3287396909729ae52bf67dfbbf62db3bc741e59f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:32:19 +0000 Subject: [PATCH 088/275] test: TestMainWindow in a header of its own The MainWindow subclass that TestRecordWorkflow drives is wanted by two more: a suite that works through the shown window, and a check with the real audio device. Moved as it was, word for word. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- main/test/TestMainWindow.h | 218 +++++++++++++++++++++++++++++++++ main/test/TestRecordWorkflow.h | 185 +--------------------------- 2 files changed, 219 insertions(+), 184 deletions(-) create mode 100644 main/test/TestMainWindow.h diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h new file mode 100644 index 00000000..e4d3ff68 --- /dev/null +++ b/main/test/TestMainWindow.h @@ -0,0 +1,218 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_MAIN_WINDOW_H +#define TEST_MAIN_WINDOW_H + +// The real MainWindow for the suites that drive it + +#include "FakeAudioIO.h" + +#include "../MainWindow.h" +#include "../Analyser.h" +#include "../CoverageStrip.h" +#include "../SingingTakes.h" + +#include "view/ViewManager.h" +#include "audio/AudioCallbackPlaySource.h" +#include "audio/AudioCallbackRecordTarget.h" + +#include +#include +#include +#include + +/** + * MainWindow with the fake device in place of a real one, and the + * protected state of the singing workflow opened up for inspection. + */ +class TestMainWindow : public MainWindow +{ +public: + TestMainWindow(FakeAudioIO::Config config, bool installDevice = true) : + MainWindow(AUDIO_PLAYBACK_AND_RECORD, true, false), + m_fakeConfig(config), + m_installDevice(installDevice) { } + + FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } + + void doRecord() { record(); } + void doPlay() { play(); } // and again to stop + void doAnalyseNow() { analyseNow(); } + void doLoadBackgroundMusic(QString path) { loadBackgroundMusic(path); } + + // Another audio file under the take's pitch and notes layers + QString doSwapSingingAudio(QString path) { + return swapSingingAudio(path); + } + + // Editing the singing of a take, as the two Edit menu actions do + void doEraseSingingInSelection() { eraseSingingInSelection(); } + void doSelectRecordingAtPlayhead() { selectRecordingAtPlayhead(); } + QAction *eraseSingingAction() { return m_eraseSingingAction; } + QAction *selectRecordingAction() { return m_selectRecordingAction; } + void doUpdateMenuStates() { updateMenuStates(); } + + // The takes of the session, as the Takes menu and the combo box do + bool doSwitchToTake(int index) { return switchToTake(index); } + void doChooseTakeInCombo(int index) { m_takeCombo->setCurrentIndex(index); } + void doNewEmptyTake() { newEmptyTake(); } + void doDuplicateTake() { duplicateTake(); } + void doRenameTake() { renameTake(); } + void doDeleteTake() { deleteTake(); } + bool doDeleteTakeAt(int index) { return deleteTakeAt(index); } + QComboBox *takeCombo() { return m_takeCombo; } + QAction *newTakeAction() { return m_newTakeAction; } + QAction *duplicateTakeAction() { return m_duplicateTakeAction; } + QAction *renameTakeAction() { return m_renameTakeAction; } + QAction *deleteTakeAction() { return m_deleteTakeAction; } + + // The two questions the take operations ask, answered from here: the + // suite cannot answer a dialog + void setDeleteTakeAnswer(bool yes) { m_deleteTakeAnswer = yes; } + int deleteTakeQuestions() const { return m_deleteTakeQuestions; } + void setTakeNameAnswer(QString name) { m_takeNameAnswer = name; } + + // True between the start of the analysis of a recorded range and the + // merge of its result into the take's pitch and notes + bool analysingRange() { + return m_analyser2 && m_analyser2->isAnalysingRange(); + } + sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } + sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } + + // Save As, with the file name given here instead of by a dialog: the + // session's own file is set, so that what is recorded next goes into + // its takes folder + bool doSaveSessionAs(QString path) { return saveSessionToPath(path); } + QString sessionFile() { return m_sessionFile; } + + // As answering "No" to "do you want to save?" + void discardModifications() { m_documentModified = false; } + bool isDocumentModified() { return m_documentModified; } + void doCloseSession() { discardModifications(); closeSession(); } + + void setPlayReferenceWhileRecording(bool on) { + m_playRefWhileRecording->setChecked(on); + } + void setPreRoll(bool on) { m_preRoll->setChecked(on); } + void setRecordIntoSelection(bool on) { + m_recordIntoSelection->setChecked(on); + } + QAction *playSingingAudioAction() { return m_playSingingAudio; } + + Analyser *analyser() { return m_analyser; } + Analyser *analyser2() { return m_analyser2; } + sv::Document *document() { return m_document; } + sv::PaneStack *paneStack() { return m_paneStack; } + sv::Layer *timeRuler() { return m_timeRulerLayer; } + sv::AudioCallbackRecordTarget *recordTarget() { return m_recordTarget; } + sv::AudioCallbackPlaySource *playSource() { return m_playSource; } + sv::ModelId mainModelId() { return getMainModelId(); } + + RealtimePitchTracker *realtimeTracker() { return m_realtimePitchTracker; } + sv::TimeValueLayer *realtimeLayer() { return m_realtimePitchLayer; } + sv::ModelId realtimeModelId() { return m_realtimePitchModelId; } + sv::ModelId currentRecordingModelId() { return m_currentRecordingModelId; } + sv::WaveformLayer *recordingLayer() { return m_recordingLayer; } + SingingTakes *takes() { return m_takes; } + sv::sv_frame_t takePosition() { return m_takePosition; } + sv::sv_frame_t takePreRoll() { return m_takePreRoll; } + sv::sv_frame_t takeEnd() { return m_takeEnd; } + bool takeTimerRunning() { return m_takeTimer && m_takeTimer->isActive(); } + + void seekTo(sv::sv_frame_t frame) { + m_viewManager->setPlaybackFrame(frame); + } + sv::sv_frame_t playbackFrame() { return m_viewManager->getPlaybackFrame(); } + + void selectRange(sv::sv_frame_t start, sv::sv_frame_t end) { + m_viewManager->addSelection(sv::Selection(start, end)); + } + void clearSelections() { m_viewManager->clearSelections(); } + sv::MultiSelection::SelectionList selections() { + return m_viewManager->getSelections(); + } + + // The question about recording over singing that is there is answered + // from here: the suite cannot answer a dialog + void setRecordOverAnswer(bool yes) { m_recordOverAnswer = yes; } + int recordOverQuestions() const { return m_recordOverQuestions; } + void clearRecordOverQuestions() { m_recordOverQuestions = 0; } + + sv::ModelId pendingSingingModelId() { return m_pendingSingingModelId; } + sv::ModelId backgroundMusicModelId() { return m_backgroundMusicModelId; } + sv::WaveformLayer *backgroundMusicLayer() { return m_backgroundMusicLayer; } + bool recordingInProgress() { return m_recordingInProgress; } + bool recordingAsSingingTrack() { return m_recordingAsSingingTrack; } + sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } + int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } + + CoverageStrip *coverageStrip() { return m_coverageStrip; } + + AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } + void doToggleAlternatePitch() { alternatePitchToggled(); } + void doStepAlternatePitch(bool up) { + if (up) alternatePitchUp(); else alternatePitchDown(); + } + QAction *alternatePitchAction() { return m_showAlternatePitch; } + QAction *alternatePitchUpAction() { return m_alternatePitchUpAction; } + QAction *alternatePitchDownAction() { return m_alternatePitchDownAction; } + + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { + onRealtimePitchDetected(frame, hz); + } + QString statusText() { return getStatusLabel()->text(); } + void setStatusText(QString text) { getStatusLabel()->setText(text); } + +protected: + void createAudioIO() override { + if (m_audioIO || m_playTarget) return; + if (!m_installDevice) return; + m_fakeConfig.inputIsKept = [this]() { + return m_recordTarget->isRecording(); + }; + m_audioIO = new FakeAudioIO + (m_recordTarget, m_playSource->getApplicationPlaybackSource(), + m_fakeConfig); + m_playSource->setSystemPlaybackTarget(m_audioIO); + } + + bool confirmRecordingOverTake() override { + ++m_recordOverQuestions; + return m_recordOverAnswer; + } + + bool confirmDeleteTake(QString) override { + ++m_deleteTakeQuestions; + return m_deleteTakeAnswer; + } + + QString askForTakeName(QString current) override { + return m_takeNameAnswer == "" ? current : m_takeNameAnswer; + } + + // The base class deleteAudioIO() deletes m_audioIO, which is right + // for the fake as well + +private: + FakeAudioIO::Config m_fakeConfig; + bool m_installDevice; + bool m_recordOverAnswer = true; + int m_recordOverQuestions = 0; + bool m_deleteTakeAnswer = true; + int m_deleteTakeQuestions = 0; + QString m_takeNameAnswer; +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d1614237..297959d7 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -23,7 +23,7 @@ // dialog and records it; each test then fails in cleanup(). #include "TestSignals.h" -#include "FakeAudioIO.h" +#include "TestMainWindow.h" #include "../MainWindow.h" #include "../Analyser.h" @@ -79,189 +79,6 @@ #include #include -/** - * MainWindow with the fake device in place of a real one, and the - * protected state of the singing workflow opened up for inspection. - */ -class TestMainWindow : public MainWindow -{ -public: - TestMainWindow(FakeAudioIO::Config config, bool installDevice = true) : - MainWindow(AUDIO_PLAYBACK_AND_RECORD, true, false), - m_fakeConfig(config), - m_installDevice(installDevice) { } - - FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } - - void doRecord() { record(); } - void doPlay() { play(); } // and again to stop - void doAnalyseNow() { analyseNow(); } - void doLoadBackgroundMusic(QString path) { loadBackgroundMusic(path); } - - // Another audio file under the take's pitch and notes layers - QString doSwapSingingAudio(QString path) { - return swapSingingAudio(path); - } - - // Editing the singing of a take, as the two Edit menu actions do - void doEraseSingingInSelection() { eraseSingingInSelection(); } - void doSelectRecordingAtPlayhead() { selectRecordingAtPlayhead(); } - QAction *eraseSingingAction() { return m_eraseSingingAction; } - QAction *selectRecordingAction() { return m_selectRecordingAction; } - void doUpdateMenuStates() { updateMenuStates(); } - - // The takes of the session, as the Takes menu and the combo box do - bool doSwitchToTake(int index) { return switchToTake(index); } - void doChooseTakeInCombo(int index) { m_takeCombo->setCurrentIndex(index); } - void doNewEmptyTake() { newEmptyTake(); } - void doDuplicateTake() { duplicateTake(); } - void doRenameTake() { renameTake(); } - void doDeleteTake() { deleteTake(); } - bool doDeleteTakeAt(int index) { return deleteTakeAt(index); } - QComboBox *takeCombo() { return m_takeCombo; } - QAction *newTakeAction() { return m_newTakeAction; } - QAction *duplicateTakeAction() { return m_duplicateTakeAction; } - QAction *renameTakeAction() { return m_renameTakeAction; } - QAction *deleteTakeAction() { return m_deleteTakeAction; } - - // The two questions the take operations ask, answered from here: the - // suite cannot answer a dialog - void setDeleteTakeAnswer(bool yes) { m_deleteTakeAnswer = yes; } - int deleteTakeQuestions() const { return m_deleteTakeQuestions; } - void setTakeNameAnswer(QString name) { m_takeNameAnswer = name; } - - // True between the start of the analysis of a recorded range and the - // merge of its result into the take's pitch and notes - bool analysingRange() { - return m_analyser2 && m_analyser2->isAnalysingRange(); - } - sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } - sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } - - // Save As, with the file name given here instead of by a dialog: the - // session's own file is set, so that what is recorded next goes into - // its takes folder - bool doSaveSessionAs(QString path) { return saveSessionToPath(path); } - QString sessionFile() { return m_sessionFile; } - - // As answering "No" to "do you want to save?" - void discardModifications() { m_documentModified = false; } - bool isDocumentModified() { return m_documentModified; } - void doCloseSession() { discardModifications(); closeSession(); } - - void setPlayReferenceWhileRecording(bool on) { - m_playRefWhileRecording->setChecked(on); - } - void setPreRoll(bool on) { m_preRoll->setChecked(on); } - void setRecordIntoSelection(bool on) { - m_recordIntoSelection->setChecked(on); - } - QAction *playSingingAudioAction() { return m_playSingingAudio; } - - Analyser *analyser() { return m_analyser; } - Analyser *analyser2() { return m_analyser2; } - sv::Document *document() { return m_document; } - sv::PaneStack *paneStack() { return m_paneStack; } - sv::Layer *timeRuler() { return m_timeRulerLayer; } - sv::AudioCallbackRecordTarget *recordTarget() { return m_recordTarget; } - sv::AudioCallbackPlaySource *playSource() { return m_playSource; } - sv::ModelId mainModelId() { return getMainModelId(); } - - RealtimePitchTracker *realtimeTracker() { return m_realtimePitchTracker; } - sv::TimeValueLayer *realtimeLayer() { return m_realtimePitchLayer; } - sv::ModelId realtimeModelId() { return m_realtimePitchModelId; } - sv::ModelId currentRecordingModelId() { return m_currentRecordingModelId; } - sv::WaveformLayer *recordingLayer() { return m_recordingLayer; } - SingingTakes *takes() { return m_takes; } - sv::sv_frame_t takePosition() { return m_takePosition; } - sv::sv_frame_t takePreRoll() { return m_takePreRoll; } - sv::sv_frame_t takeEnd() { return m_takeEnd; } - bool takeTimerRunning() { return m_takeTimer && m_takeTimer->isActive(); } - - void seekTo(sv::sv_frame_t frame) { - m_viewManager->setPlaybackFrame(frame); - } - sv::sv_frame_t playbackFrame() { return m_viewManager->getPlaybackFrame(); } - - void selectRange(sv::sv_frame_t start, sv::sv_frame_t end) { - m_viewManager->addSelection(sv::Selection(start, end)); - } - void clearSelections() { m_viewManager->clearSelections(); } - sv::MultiSelection::SelectionList selections() { - return m_viewManager->getSelections(); - } - - // The question about recording over singing that is there is answered - // from here: the suite cannot answer a dialog - void setRecordOverAnswer(bool yes) { m_recordOverAnswer = yes; } - int recordOverQuestions() const { return m_recordOverQuestions; } - void clearRecordOverQuestions() { m_recordOverQuestions = 0; } - - sv::ModelId pendingSingingModelId() { return m_pendingSingingModelId; } - sv::ModelId backgroundMusicModelId() { return m_backgroundMusicModelId; } - sv::WaveformLayer *backgroundMusicLayer() { return m_backgroundMusicLayer; } - bool recordingInProgress() { return m_recordingInProgress; } - bool recordingAsSingingTrack() { return m_recordingAsSingingTrack; } - sv::sv_frame_t recordingLatencyFrames() { return m_recordingLatencyFrames; } - int pendingExtraPaneCount() { return int(m_pendingExtraPanes.size()); } - - CoverageStrip *coverageStrip() { return m_coverageStrip; } - - AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } - void doToggleAlternatePitch() { alternatePitchToggled(); } - void doStepAlternatePitch(bool up) { - if (up) alternatePitchUp(); else alternatePitchDown(); - } - QAction *alternatePitchAction() { return m_showAlternatePitch; } - QAction *alternatePitchUpAction() { return m_alternatePitchUpAction; } - QAction *alternatePitchDownAction() { return m_alternatePitchDownAction; } - - void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { - onRealtimePitchDetected(frame, hz); - } - QString statusText() { return getStatusLabel()->text(); } - void setStatusText(QString text) { getStatusLabel()->setText(text); } - -protected: - void createAudioIO() override { - if (m_audioIO || m_playTarget) return; - if (!m_installDevice) return; - m_fakeConfig.inputIsKept = [this]() { - return m_recordTarget->isRecording(); - }; - m_audioIO = new FakeAudioIO - (m_recordTarget, m_playSource->getApplicationPlaybackSource(), - m_fakeConfig); - m_playSource->setSystemPlaybackTarget(m_audioIO); - } - - bool confirmRecordingOverTake() override { - ++m_recordOverQuestions; - return m_recordOverAnswer; - } - - bool confirmDeleteTake(QString) override { - ++m_deleteTakeQuestions; - return m_deleteTakeAnswer; - } - - QString askForTakeName(QString current) override { - return m_takeNameAnswer == "" ? current : m_takeNameAnswer; - } - - // The base class deleteAudioIO() deletes m_audioIO, which is right - // for the fake as well - -private: - FakeAudioIO::Config m_fakeConfig; - bool m_installDevice; - bool m_recordOverAnswer = true; - int m_recordOverQuestions = 0; - bool m_deleteTakeAnswer = true; - int m_deleteTakeQuestions = 0; - QString m_takeNameAnswer; -}; - class TestRecordWorkflow : public QObject { Q_OBJECT From a7e561c4515e5b2d6de0d9f63375dbc73e5e6351 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:33:05 +0000 Subject: [PATCH 089/275] test: the manual checklist, automated where it can be Of its 36 items, those about what is on the screen and what the window does with keys, mouse and dialogs become TestUiChecks, a suite of test-tony-app: the real window, shown offscreen and made active, driven with QTest key presses and mouse gestures on pane 0 and with the dialogs MainWindow shows, and judged by the pixels of the window's backing store. With TONY_TEST_SHOT_DIR set it saves what it judged, for a person to look over. The items about the audio device become test-tony-device, which is built with the rest but run only by hand: it records a four-minute click track through the air into one take and measures where the clicks landed, how long Stop took, which input carries the microphone, and whether the input comes back out. TONY_DEVICE_CHECK_FAKE runs it on the fake device fed back into itself, to check the check. FakeAudioIO can put its input on one channel, for a microphone on input 2. The checks found four defects, committed as expected failures: the live dots stop being drawn during a take (the dot model defers its change notices), the band of the coverage strip goes under the take's waveform from the second recording on, commitData() writes a .sv that Tony does not open as a session, and with playback constrained to the selection a pre-roll places the take a whole pre-roll early. They and the other findings are in docs/open-points.md. What is left of the checklist is the device check, six items by hand, and decisions. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- AGENTS.md | 2 +- docs/building.md | 2 +- docs/manual-checklist.md | 185 ++-- docs/open-points.md | 51 +- docs/testing.md | 55 +- main/test/FakeAudioIO.h | 11 + main/test/TestMainWindow.h | 29 +- main/test/TestRealDevice.h | 607 ++++++++++++ main/test/TestRecordWorkflow.h | 33 + main/test/TestUiChecks.h | 1561 +++++++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 7 + main/test/tony-device-check.cpp | 59 ++ meson.build | 32 + 13 files changed, 2511 insertions(+), 123 deletions(-) create mode 100644 main/test/TestRealDevice.h create mode 100644 main/test/TestUiChecks.h create mode 100644 main/test/tony-device-check.cpp diff --git a/AGENTS.md b/AGENTS.md index 44e4d87a..015351cd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,7 +27,7 @@ From **Git Bash** (the usual agent shell). `build.bat` does not run from sh. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-device.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log ``` diff --git a/docs/building.md b/docs/building.md index 8d77e1d7..3ee5107c 100644 --- a/docs/building.md +++ b/docs/building.md @@ -24,7 +24,7 @@ From PowerShell its output is safe to capture: `.\build.bat *> tmp\build.log`. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-device.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log tail -20 tmp/build.log ``` diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 9fa49878..81bb2d83 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -1,106 +1,83 @@ # Manual checklist: what no automated test can tell -The automated suites run against a fake audio device and an offscreen window. Everything -below needs a real device, real ears or real eyes. **As of 2026-09-20 none of it has been -tried by hand.** When an item has been checked, note the date and the result next to it; -when a change touches an area, the items of that area are what to ask the user to try. - -Launch with `.\build.bat run`. - -## Latency and live feedback - -1. **Latency on this machine.** Play Reference While Recording on, headphones, clap along - with a reference with a clear onset. Afterwards the take lines up with the reference by - eye and by ear; still does after save and reopen. -2. **Several phrases in one take.** Record two or three phrases at different positions: - every one sits in time, not just the first (each recording measures its start gap). -3. **Live dots** appear under the playback cursor, not behind it; stay after Stop until the - orange pitch track replaces them; the status bar stops changing when the take stops. - Recording over singing that is there: that take's own pitch track and notes are out of - sight for the take, so only the dots and the track being followed are on the pane, and - they are back when the take stops. -4. **Nothing of the take in the speakers while recording**: with speakers on, neither your - voice nor a synth tone comes back. Play Singing Audio keeps its state through the take. -5. **Stereo interface with the mic on input 2**: dots appear. -6. **No input device / device in use**: Record does nothing harmful, and the next file - opened is analysed as usual. - -## Recording from a position - -7. Seek into the song, Record, sing, Stop: the singing is where it was sung and the part - before it is untouched. Record again inside it: the overwrite question comes; No records - nothing. "Don't ask again" with Yes holds across sessions. -8. During a take at P > 0 the cursor starts at P, the pane follows it, and cursor, - reference and dots are in the same place. -9. **How long Stop takes on a 4-minute song**: a moment for the file copy, then new pitch - only where the singing was. A pause like a whole-song analysis means the ranged path - did not happen. -10. **The joins**: no click at the edges of a new range; the pitch track runs through the - join without a hole, a doubled dot or one note showing as two. Pitch and notes outside - the recorded range (± about 0.25 s) must not flicker or move at all. - -## Pre-roll and Record into Selection - -11. **Is 3 s right, is the countdown readable while singing?** (QSettings - `MainWindow/prerollseconds`; there is deliberately no UI yet.) -12. Sing through the lead-in: nothing of it is heard back, and nothing before P changed. -13. Pre-roll less than 3 s from the start of the song: shorter countdown, no attempt to - run from before frame 0. -14. Record into Selection stops by itself about 0.25 s after the end has been sung; what is - added is exactly the selection; no overwrite question. -15. **Both together — the practice loop this is all for**: select, Record, hear the - lead-in, sing, and be back with nothing to press; Play hears it. Is anything else - needed to make repeating that pleasant? -16. Constrain Playback to Selection together with a pre-roll: the lead-in is probably cut - short. Should the two be kept apart? - -## Coverage strip and erase - -17. The band is readable over waveform and dots at every zoom: height, colour, gaps. -18. It cannot be touched: clicking and dragging on it with either tool creates, moves or - selects nothing and does not change the pane's scale. -19. Select Recording at Playhead then Erase: audio silent, bar gone, pitch and notes gone, - no analysis afterwards. Erasing the middle of a long note leaves two. -20. Erase and Select Recording are greyed out with no take, no selection, while recording, - and for the second or two of analysis after Stop — and come back by themselves. - -## Undo - -21. Three recordings, Ctrl+Z three times: each takes back exactly one (audio, coverage, - band, pitch together). The menu says "Record Singing" / "Erase Singing", never anything - about a layer or pane. -22. Ctrl+Z immediately after Stop, before the pitch appears: nothing of that analysis - lands later; redo analyses again. -23. Undo of the very first recording leaves no singing track at all, and Record still - works. - -## Takes - -24. New Empty Take, record, switch back and forth: fast, no analysis, each take with its - own audio, pitch, notes and band. The inactive take is silent, and playback stops at - the end of the take on show even when another is longer. -25. Duplicate, record into the copy: the original is untouched. -26. Delete asks first and never deletes an audio file. Rename keeps the undo history — - every other take operation clears it without a prompt: acceptable in use? -27. Combo and Takes menu are greyed out during a take. -28. Analyse Now on a take re-analyses all of its coverage in place. - -## Sessions and files - -29. Before the first save, take files go to the record directory; after it, to - `.takes/`. Closing leaves each take's file, files any saved session named, - and nothing else Tony wrote. `recorded-*.wav` are never deleted. -30. Move `.ton` and folder together: everything plays. Move the `.ton` alone: exactly one - warning naming the folder, takes shown without sound, no "locate it?" question. -31. Save As copies the takes; the old `.ton` still opens and plays. -32. A `.ton` from before the takes work opens with the reference only and no dialog. -33. Open a `.ton`, Load Singing Track or Load Background Music, then Record: no crash, time - ruler still there. Closing afterwards asks whether to save. -34. Stop a take and close the window at once: no crash. -35. Log out with unsaved takes: what `commitData` writes into `~/.sv1` is playable. - -## Looks - -36. Alternate pitch track: faded brown is readable but secondary; dark brown during a take - is distinct from black; `8vb` / `8va` buttons look acceptable; the track stays in view - after an octave step. +The suites run against a fake audio device and an offscreen window. Most of what used to +be on this page is now checked automatically: + +- **`TestUiChecks`** (in `test-tony-app`) shows the real window, drives it with key + presses, mouse gestures and its own dialogs, and judges pane 0 by the pixels on the + screen. Each of its tests begins with a comment `// Checklist:` quoting the item it + replaced; `grep -n "Checklist:" main/test/*.h` lists them. With `TONY_TEST_SHOT_DIR` set + it saves what it looked at as PNG files. +- **`test-tony-device`** checks the real device: section 1. + +What is left needs a real device, real ears, or a decision. When an item has been checked, +note the date and the result next to it; when a change touches an area, the items of that +area are what to ask the user to try. Launch with `.\build.bat run`. + +## 1. The device check + +Once per machine, and again for each output device sung with (Bluetooth headphones have a +latency of their own). It covers: latency on this machine; several recordings in one take, +each in time; nothing of the take coming back out of the speakers; how long Stop takes on a +four-minute song; live dots from whichever input the microphone is on; and a device that +records nothing. + +It plays a four-minute reference of a tone and clicks, records it through the air at two +places into one take, and measures where the clicks landed in the take. + +1. In Tony, choose the devices under **Playback > Audio Output Device** and **Audio Input + Device** (or leave the system default). The check reads that choice, and nothing else, + from Tony's settings. +2. Speakers at a moderate volume and the microphone where it hears them; with headphones, + hold an ear cup against the microphone. A quiet room. It takes about a minute. +3. From Git Bash: + + ```sh + export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" + ninja -j 3 -C build_mingw test-tony-device.exe > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log + cd build_mingw && mkdir -p ../tmp/tl + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-device.exe > ../tmp/test.log 2>&1; echo "exit:$?" + grep -a "^FAIL\|^ Loc\|^QINFO\|^Totals" ../tmp/tl/TestRealDevice.txt + ``` + +4. Read the `QINFO` lines, one per recording: `clicks +x ms from the reference` (within + ±10 ms passes; the sign says late or early), `match` (below 0.1 the microphone did not + hear the speakers), `stop took`, `live dots`, and `channels (dB)`, which shows which + input the microphone is on. With a stereo interface, run it with the microphone on + input 2: the dots must still be there. + +`TONY_DEVICE_CHECK_FAKE=n` runs it with no hardware, on the fake device with its output fed +back into its input `n` frames later than it reports: for checking the check. + +Cloud run, 2026-09-25 (Linux, no sound card): with no device at all Record does no harm and +the next file is analysed (the "Couldn't open audio device" warning comes back once per file +opened); with a device that opens but delivers nothing the take is dropped quietly, no +harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, with +`n = 2205`. **Not yet run on real hardware.** + +## 2. Still by hand + +1. **A device in use**: another program holding the microphone exclusively. Record does + nothing harmful, and the next file opened is analysed as usual. +2. **The practice loop**, the one this is all for: select, Record, hear the lead-in, sing, + and be back with nothing to press; Play hears it, and it sits in time by ear. Is + anything else needed to make repeating that pleasant? +3. **Is a 3 s pre-roll right, and is the countdown readable while singing?** (QSettings + `MainWindow/prerollseconds`; there is deliberately no UI yet.) +4. **Looks**, from the screenshots (`TONY_TEST_SHOT_DIR=../tmp/shots` on a run of + `test-tony-app`) and in the app: the band along the bottom over waveform and dots; the + faded and dark brown of the alternate pitch track and the `8vb` / `8va` buttons (in the + `alternate_pitch_track_colours-window` shot). + Cloud review, 2026-09-25: the band is a clear orange strip over the grey waveform at + every zoom; faded brown reads as secondary to the black reference; during a take the + reference pitch is hidden, so dark brown only has to stand out from the dots, which it + does. Even 1920 px wide, the bottom toolbar overflows: what comes after Pre-roll, the + octave buttons included, is behind its » menu. Fonts there are Linux ones; how wide the + toolbars are on Windows is for the Windows screenshots. +5. **Take operations clear the undo history with no prompt** (all but Rename): acceptable + in use? +6. **Log out with unsaved takes** on Windows: its test does not run there, because + `commitData()` writes into the real profile. See open points for the file it writes. + +The defects the automated checks found, and the facts they established for the decisions +above, are in [open-points.md](open-points.md). diff --git a/docs/open-points.md b/docs/open-points.md index 7776f247..3449096f 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -10,10 +10,24 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. Is 3 s right, and should there be a control? - **No overwrite question when recording into a selection**: the selection is taken as the consent. Right in use? -- **Constrain Playback to Selection + pre-roll**: the play source constrains playback to - the selection, the lead-in is outside it, so it is cut short. Nothing keeps the two apart. +- **Constrain Playback to Selection + pre-roll**: worse than a short lead-in. The play + source starts playback at the selection, so none of the lead-in is played, and the splice, + which takes playback to have started a pre-roll earlier, places what was sung a whole + pre-roll too early (`preroll_with_playback_constrained_to_the_selection`, expected to + fail). Keep the two apart, or start playback at the lead-in regardless? - **Take operations clear the undo history with no prompt** (all but Rename). -- None of the [manual checklist](manual-checklist.md) has been run. +- **A selection is an entry of the undo history**: making one re-analyses the reference in + it (upstream Tony's pitch candidates) and pushes "Re-Analyse Selection". Selecting for + Record into Selection or for Erase therefore puts such entries between "Record Singing" + and "Erase Singing", and if the re-analysis finishes after an erase, Ctrl+Z takes it + back instead of the erase. +- **The Edit tool edits the take's note at the time it is used, wherever in the pane**, + the band of the coverage strip included. The strip itself takes no edits. Should the band + keep the tools off the notes? +- **The alternate pitch track at ±3 octaves** of a 220 Hz reference (28 Hz, 1.8 kHz) is + outside the range the pane shows, and nothing scrolls to it; ±2 is in view. +- Of the [manual checklist](manual-checklist.md), the device check has been run only in + the cloud (no sound card, and the fake device); nothing yet on real hardware. ## Not built @@ -25,6 +39,32 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Weak spots +Defects the checks of `TestUiChecks` found, each committed as a test expected to fail +(`QEXPECT_FAIL` names the cause): + +- **Live dots stop being drawn during a take.** The dot model is made with `notifyOnAdd` + false, so a dot added tells the pane nothing; dots are drawn only when one widens the + model's pitch range or the pane redraws for another reason (a page turn, a zoom). On a + steady note they stall within half a second (`live_dots_under_the_cursor`). Making the + model with `notifyOnAdd` true fixes it, as tried: the pane then repaints for every dot, + about 170 times a second, coalesced by Qt; whether that is cheap enough on the + development machine has not been measured. +- **The band of the coverage strip is hidden after the second recording.** The audio swap + makes the take's waveform layer again, on top, and `syncCoverageStrip()` raises nothing + once the strip is shown (`strip_on_top_after_another_recording`). +- **`commitData()` writes `~/.sv1/tmp-*.sv`**, Sonic Visualiser's extension; Tony opens + only `.ton` as a session, so what it saved at logout does not open from Recent Files + (`commit_data_writes_a_playable_session`; renamed to `.ton` it opens and plays). + +Seen and not pinned by a test: + +- After playback the pane's own cache of what it drew holds the translucent note boxes + painted twice over themselves, darker, until the next zoom or scroll. Seen with the + offscreen platform, through the window's backing store; whether it shows on a real screen + is not known. `TestUiChecks::grabPaneRedrawn()` works around it. +- With no audio device at all, "Couldn't open audio device" is shown again for every file + opened (`MainWindowBase::createAudioIO()` tries each time). + - **If pYIN fails part-way, the live dots wait for ever**: they are removed on `initialAnalysisCompleted`, which then never comes. - **`Analyser::newFileLoaded()` error path for the singing track** (pYIN plugin missing): @@ -40,5 +80,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. that printed the value otherwise. With `PlotStrip` it is no longer needed. - Untested by any suite: removal of dots placed before the latency was measured; the deferred and error paths of the dot teardown; `ContinuousSynth` deletion in the svapp - fork; the 30 s give-up of `waitForRangedAnalysis()`; `commitData()` relocating takes; - the two other ways `MainWindowBase::record()` can fail. + fork; the 30 s give-up of `waitForRangedAnalysis()`; `commitData()` relocating takes on + Windows (the test runs elsewhere only); the two other ways `MainWindowBase::record()` can + fail. diff --git a/docs/testing.md b/docs/testing.md index 0de32575..7496a3c2 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -6,9 +6,12 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | | `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow` | about 4.5 minutes (measured 2026-09-20), nearly all of it `TestRecordWorkflow`: takes are recorded in real time | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | +| `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | -`meson test` / `build.bat test` runs both plus four svcore suites. +`meson test` / `build.bat test` runs the first two plus four svcore suites. `test-tony-device` +is built with them and never run by `meson test`: it needs a microphone that hears the +speakers. - The `tony-app` meson test has `timeout: 900`; the suite took about 277 s unloaded when that was set. Every workflow test adds real time, so if the suite comes near it, raise it @@ -58,19 +61,23 @@ helpers must not be slots; connect to lambdas instead. For access to private sta whole file, not just around the range (`ranged_leaves_the_rest_alone`): that is what caught the merge damaging unchanged audio half a second away. -## What is there to reuse (`TestRecordWorkflow.h`) +## What is there to reuse (`TestRecordWorkflow.h`, `TestMainWindow.h`) - `FakeAudioIO` (`FakeAudioIO.h`): a duplex device with a worker thread that runs the callback in real time, input first and then output, as PortAudio and JACK do. `Config` sets rate, block size, reported latencies, a programmed mono input, its delay, and whether the input clock starts at the first audible output sample ("a singer exactly on - time"). It captures the output, so tests can assert what reached the speakers. -- `TestMainWindow`: subclass of `MainWindow` that exposes protected operations as - `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the - fake device through `createAudioIO()`, and **answers dialogs through virtual seams**: - `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, each with a - `set...Answer()` and a counter of questions asked. A test cannot answer a real dialog: - anything new that asks the user needs such a virtual. + time"), `loopback` (the output fed back into the input, as speakers into a microphone), + and `inputChannel` (the input on one channel only, as a microphone on input 2). It + captures the output, so tests can assert what reached the speakers. +- `TestMainWindow` (`TestMainWindow.h`, shared by the three suites that drive a window): + subclass of `MainWindow` that exposes protected operations as `doRecord()`, + `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the fake device + through `createAudioIO()` (or the real one, `setUseRealDevice()`), and **answers dialogs + through virtual seams**: `confirmRecordingOverTake()`, `confirmDeleteTake()`, + `askForTakeName()`, each with a `set...Answer()` and a counter of questions asked. + Anything new that asks the user needs such a virtual. `setRecordOverAskedInDialog()` + lets the real dialog through instead, for a test that presses its buttons. - Fixture helpers: `makeWindow(config)`, `writeWav()`, `openReference()`, `startTake()` / `stopTake()` / `take(ms)`, `verifyPlaySourceClean()`, `layersOnModel()`, `paneHasLayer()`, `documentHasLayer()`, `reopenAsSession()` / `reopenSession()`, @@ -139,7 +146,8 @@ it first, or break the code for a moment (mark the line `MUTATION`, and check purpose. "Fixing" one side makes the live dots and the pYIN track disagree. A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming -it; the marker goes in the commit that fixes it. There are none at present. +it; the marker goes in the commit that fixes it. There are four at present, all in +`TestUiChecks`, listed in [open-points.md](open-points.md). ## Timing and races @@ -168,6 +176,31 @@ it; the marker goes in the commit that fixes it. There are none at present. - After a `.ton` round trip compare frames exactly and values with a tolerance (six significant figures in the file). +## The window as it is seen (`TestUiChecks`) + +The window is shown (still offscreen), made active so that its shortcuts work, and driven +with `QTest` key presses, mouse gestures on pane 0 and the dialogs MainWindow shows. What +it draws is judged by pixels: + +- **Read the screen, not `QWidget::grab()`**: `grabPane()` copies pane 0 out of the + window's backing store, which holds what the pane's own paint events put there. `grab()` + has the pane paint itself once more and can show what the screen does not: it showed + live dots that the screen never got. +- After any playback the pane's cache holds the translucent note boxes painted twice. + Compare images only after `grabPaneRedrawn()`, which forces a full redraw (a zoom one + step away snaps back to the same level and redraws nothing; it doubles the level). +- The pane has its vertical scale at the left, about 30 px, over everything; the play + pointer is two dark lines around a light one (`pointerX()`), drawn over the band. +- The take's pitch track is under its notes' translucent purple, so its orange reads as + about (246, 126, 114) there: `isSinging()` takes both. Bright orange is the reference's + pitch candidates, which a selection makes. +- Record puts the view back on the take's position: work out x positions again after it. +- Timing checks in real time go through the pane's own timers (the pointer moves every + 20 ms): allow a tick. + +With `TONY_TEST_SHOT_DIR` set, the suite saves the images it judged, and some of the whole +window, as `-.png`, for the [manual checklist](manual-checklist.md)'s look. + ## What stays manual Anything about a real device, real timing by ear, or how something looks: diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h index d5f3a55e..cf91e70e 100644 --- a/main/test/FakeAudioIO.h +++ b/main/test/FakeAudioIO.h @@ -55,6 +55,11 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO // Frames of silence before the input int inputDelay = 0; + // The one channel the input arrives on, the others silent, as a + // microphone on input 2 of an interface is channel 1; -1 for + // every channel + int inputChannel = -1; + // Start the input clock at the first audible output sample // instead of at resume. With inputDelay equal to the reported // round trip, this is a singer who is exactly on time. @@ -229,7 +234,13 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO bool kept = !m_config.inputIsKept || m_config.inputIsKept(); long keptBefore = m_sinceResume; + std::vector silence(n, 0.f); std::vector inPtrs(ch, in.data()); + if (m_config.inputChannel >= 0) { + for (int c = 0; c < ch; ++c) { + if (c != m_config.inputChannel) inPtrs[c] = silence.data(); + } + } m_target->putSamples(inPtrs.data(), ch, n); if (kept) m_sinceResume += n; diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index e4d3ff68..e8084041 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -14,7 +14,8 @@ #ifndef TEST_MAIN_WINDOW_H #define TEST_MAIN_WINDOW_H -// The real MainWindow for the suites that drive it +// The real MainWindow for the suites that drive it: TestRecordWorkflow, +// TestUiChecks and the real-device check #include "FakeAudioIO.h" @@ -30,11 +31,13 @@ #include #include #include +#include #include /** * MainWindow with the fake device in place of a real one, and the * protected state of the singing workflow opened up for inspection. + * The real device can be asked for instead (setUseRealDevice()). */ class TestMainWindow : public MainWindow { @@ -46,6 +49,16 @@ class TestMainWindow : public MainWindow FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } + // The audio device Tony itself would open, from the settings, in + // place of the fake: for the checks that need real hardware. Set + // before the first file is opened, which is when the device is made + void setUseRealDevice(bool on) { m_useRealDevice = on; } + bool haveAudioDevice() { return m_audioIO || m_playTarget; } + + // A device that records as well: without one there is only a play + // target (MainWindowBase::createAudioIO()) + bool haveRecordingDevice() { return m_audioIO != nullptr; } + void doRecord() { record(); } void doPlay() { play(); } // and again to stop void doAnalyseNow() { analyseNow(); } @@ -76,6 +89,7 @@ class TestMainWindow : public MainWindow QAction *duplicateTakeAction() { return m_duplicateTakeAction; } QAction *renameTakeAction() { return m_renameTakeAction; } QAction *deleteTakeAction() { return m_deleteTakeAction; } + QMenu *takesMenu() { return m_takesMenu; } // The two questions the take operations ask, answered from here: the // suite cannot answer a dialog @@ -150,6 +164,10 @@ class TestMainWindow : public MainWindow int recordOverQuestions() const { return m_recordOverQuestions; } void clearRecordOverQuestions() { m_recordOverQuestions = 0; } + // ... or by MainWindow's own dialog, for a test that answers it by + // pressing its buttons + void setRecordOverAskedInDialog(bool on) { m_recordOverInDialog = on; } + sv::ModelId pendingSingingModelId() { return m_pendingSingingModelId; } sv::ModelId backgroundMusicModelId() { return m_backgroundMusicModelId; } sv::WaveformLayer *backgroundMusicLayer() { return m_backgroundMusicLayer; } @@ -178,6 +196,10 @@ class TestMainWindow : public MainWindow protected: void createAudioIO() override { if (m_audioIO || m_playTarget) return; + if (m_useRealDevice) { + MainWindow::createAudioIO(); + return; + } if (!m_installDevice) return; m_fakeConfig.inputIsKept = [this]() { return m_recordTarget->isRecording(); @@ -190,6 +212,9 @@ class TestMainWindow : public MainWindow bool confirmRecordingOverTake() override { ++m_recordOverQuestions; + if (m_recordOverInDialog) { + return MainWindow::confirmRecordingOverTake(); + } return m_recordOverAnswer; } @@ -208,7 +233,9 @@ class TestMainWindow : public MainWindow private: FakeAudioIO::Config m_fakeConfig; bool m_installDevice; + bool m_useRealDevice = false; bool m_recordOverAnswer = true; + bool m_recordOverInDialog = false; int m_recordOverQuestions = 0; bool m_deleteTakeAnswer = true; int m_deleteTakeQuestions = 0; diff --git a/main/test/TestRealDevice.h b/main/test/TestRealDevice.h new file mode 100644 index 00000000..cd8cdaec --- /dev/null +++ b/main/test/TestRealDevice.h @@ -0,0 +1,607 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_REAL_DEVICE_H +#define TEST_REAL_DEVICE_H + +// The checks that need the real audio device: the real MainWindow, +// opening the device the user chose in Tony, recording what its +// microphone hears of its own speakers. +// +// The reference is four minutes of a repeating two-second pattern: a +// second of a tone, for the live dots and pYIN, then a second holding +// three clicks at uneven spacing. Two recordings are made into one take +// with Play Reference While Recording on, and each recorded range of +// the take is lined up against the reference by its clicks. A take +// whose latency was compensated right has its clicks exactly where the +// reference has them. +// +// Not part of any automated run: see docs/manual-checklist.md for how +// to set it up. With no device to record from, or one that delivers +// nothing, the one check that runs is that Record does no harm. + +#include "TestSignals.h" +#include "TestMainWindow.h" + +#include "version.h" + +#include "layer/Layer.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" +#include "data/fileio/FileSource.h" +#include "base/RecordDirectory.h" +#include "transform/ModelTransformerFactory.h" +#include "widgets/InteractiveFileFinder.h" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +class TestRealDevice : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + static constexpr double songSeconds = 240.0; + static constexpr double patternSeconds = 2.0; + + // Where in each pattern the clicks are, in seconds: uneven, so that + // only the right lag lines all three up + static std::vector clickOffsets() { return { 1.10, 1.37, 1.71 }; } + + // How far a recorded range may sit from the reference and still be + // called in time: a few milliseconds of sound travelling from the + // speaker to the microphone, and the rest for the device + static constexpr double toleranceMs = 10.0; + + // The recordings: where, and for how long + static std::vector takePositions() { return { 60.0, 150.0 }; } + static constexpr int takeMs = 5000; + + struct Recorded { + Coverage::Range range; + double lagMs = 0.0; // later than the reference if positive + double peak = 0.0; // normalised correlation at that lag + double echoMs = 0.0; // strongest later peak, after the first + double echo = 0.0; // ... and its height + double stopSeconds = 0.0; // from Stop to the analysis merged + int dots = 0; // live dots drawn during it + std::vector channelLevels; // dB, of the raw recording + }; + + QTemporaryDir m_dir; + TestMainWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + std::vector m_reference; + QString m_referencePath; + double m_wholeSongSeconds = 0.0; + std::vector m_recorded; + bool m_deliveredNothing = false; + + static sv::sv_frame_t frames(double seconds) { + return sv::sv_frame_t(std::lround(seconds * rate)); + } + + // The tone half of each pattern, and the clicks: short bursts of + // noise, which line up by cross-correlation with no ambiguity of a + // period + std::vector makeReference(bool clicksOnly) { + std::vector data(size_t(frames(songSeconds)), 0.f); + const int clickFrames = int(0.003 * rate); + std::mt19937 random(42); + std::uniform_real_distribution noise(-1.f, 1.f); + std::vector click(static_cast(clickFrames), 0.f); + for (int i = 0; i < clickFrames; ++i) { + double window = 0.5 - 0.5 * std::cos(2.0 * TestSignals::kPi * i / + (clickFrames - 1)); + click[size_t(i)] = float(0.8 * window) * noise(random); + } + auto saw = TestSignals::sawtooth(220.5, rate, int(frames(1.0)), 0.25f); + + for (double t = 0.0; t + patternSeconds <= songSeconds; + t += patternSeconds) { + size_t at = size_t(frames(t)); + if (!clicksOnly) { + std::copy(saw.begin(), saw.end(), data.begin() + long(at)); + } + for (double offset : clickOffsets()) { + size_t c = size_t(frames(t + offset)); + std::copy(click.begin(), click.end(), data.begin() + long(c)); + } + } + return data; + } + + QString writeWav(const std::vector &data, QString name) { + QString path = m_dir.filePath(name); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + if (!writer.isOK() || + !writer.writeSamples(&ptr, sv::sv_frame_t(data.size())) || + !writer.close()) { + return {}; + } + return path; + } + + static std::vector readMono(QString path, sv::sv_frame_t from, + sv::sv_frame_t count) { + sv::WavFileReader reader { sv::FileSource(path) }; + if (!reader.isOK()) return {}; + int ch = reader.getChannelCount(); + auto data = reader.getInterleavedFrames(from, count); + std::vector mono(data.size() / size_t(ch), 0.f); + for (size_t i = 0; i < mono.size(); ++i) { + for (int c = 0; c < ch; ++c) mono[i] += data[i * size_t(ch) + size_t(c)]; + } + return mono; + } + + // The lag, in frames within +-maxLag, at which the recorded audio + // best matches the clicks of the reference, and how well: the peak of + // the normalised cross-correlation. The tone half of the reference + // is left out, so that only the clicks decide it + static std::pair bestLag(const std::vector &clicks, + const std::vector &recorded, + long maxLag, + std::vector *curve) { + // clicks runs from maxLag before the recorded range to maxLag + // after it + double er = 0.0; + for (float v : recorded) er += double(v) * v; + long best = 0; + double bestValue = -1.0; + for (long lag = -maxLag; lag <= maxLag; ++lag) { + double sum = 0.0, ec = 0.0; + for (size_t i = 0; i < recorded.size(); ++i) { + double c = clicks[size_t(long(i) + maxLag - lag)]; + sum += c * recorded[i]; + ec += c * c; + } + double value = (ec > 0.0 && er > 0.0) ? sum / std::sqrt(ec * er) + : 0.0; + if (curve) curve->push_back(value); + if (value > bestValue) { + bestValue = value; + best = lag; + } + } + return { best, bestValue }; + } + + static double levelDb(const std::vector &data) { + double sum = 0.0; + for (float v : data) sum += double(v) * v; + double rms = data.empty() ? 0.0 : std::sqrt(sum / double(data.size())); + return rms > 0.0 ? 20.0 * std::log10(rms) : -200.0; + } + + static bool analysed(Analyser *a) { + return a && a->getLayer(Analyser::PitchTrack) && + a->getLayer(Analyser::Notes) && + a->getInitialAnalysisCompletion() >= 100 && + !a->isAnalysingRange() && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + } + + // The newest raw recording: the file the device was recorded into, + // with every channel it had + QString newestRecording() { + QString newest; + QDateTime newestTime; + QDirIterator it(sv::RecordDirectory::getRecordContainerDirectory(), + { "recorded-*.wav" }, QDir::Files, + QDirIterator::Subdirectories); + while (it.hasNext()) { + QString path = it.next(); + QDateTime t = QFileInfo(path).lastModified(); + if (newest == "" || t > newestTime) { + newest = path; + newestTime = t; + } + } + return newest; + } + + // Not a slot: QtTest would run it as a test + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + QList buttons = box->buttons(); + m_dialogs.push_back(description); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) dialog->reject(); + else modal->close(); + } + + bool haveDevice() { return m_window && m_window->haveAudioDevice(); } + bool canRecord() { return m_window && m_window->haveRecordingDevice(); } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + QSettings().clear(); + + // The devices Tony uses, from Tony's own settings: whatever was + // chosen in its Playback menu is what is checked here. The rest of + // Tony's settings are not read, and nothing is written to them + { + QSettings tony("sonic-visualiser", "Tony"); + tony.beginGroup("Preferences"); + QSettings mine; + mine.beginGroup("Preferences"); + for (QString key : tony.childKeys()) { + if (key.startsWith("audio-")) { + mine.setValue(key, tony.value(key)); + qInfo("Tony's setting %s = \"%s\"", qPrintable(key), + qPrintable(tony.value(key).toString())); + } + } + mine.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + mine.endGroup(); + } + + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", true); + settings.setValue("preroll", false); + settings.setValue("recordintoselection", false); + settings.endGroup(); + SingingTakes::setOverwriteConfirmationWanted(true); + + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + sv::RecordDirectory::setRecordContainerDirectory + (m_dir.filePath("recorded")); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + + m_reference = makeReference(false); + m_referencePath = writeWav(m_reference, "reference.wav"); + QVERIFY(m_referencePath != ""); + + // TONY_DEVICE_CHECK_FAKE=n checks the check itself, with no + // hardware: the fake device, its output fed back into its input as + // speakers into a microphone, reporting a round trip of 2048 + // frames and taking 2048 + n. With n = 0 everything passes; the + // recordings come out n frames late otherwise + FakeAudioIO::Config fake; + bool useFake = qEnvironmentVariableIsSet("TONY_DEVICE_CHECK_FAKE"); + if (useFake) { + fake.playbackLatency = 1024; + fake.recordLatency = 1024; + fake.inputDelay = 2048 + + qEnvironmentVariableIntValue("TONY_DEVICE_CHECK_FAKE"); + fake.loopback = true; + qInfo("the fake device, %d frames from output to input", + fake.inputDelay); + } + m_window = new TestMainWindow(fake); + m_window->setUseRealDevice(!useFake); + m_window->setPlayReferenceWhileRecording(true); + m_window->resize(1200, 800); + m_window->show(); + + // The whole song's analysis, timed: what Stop must take much less + // time than + QElapsedTimer timer; + timer.start(); + QCOMPARE(m_window->openPath(m_referencePath, + MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 300000); + m_wholeSongSeconds = double(timer.elapsed()) / 1000.0; + qInfo("analysis of the whole %.0f s reference took %.1f s", + songSeconds, m_wholeSongSeconds); + } + + void cleanupTestCase() { + m_watchdog.stop(); + if (m_window) { + if (m_window->recordTarget()->isRecording()) m_window->doRecord(); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + sv::RecordDirectory::setRecordContainerDirectory(""); + } + + // Which device, and what it says of itself + void device() { + std::vector names = + breakfastquay::AudioFactory::getImplementationNames(); + QStringList implementations; + for (auto n : names) implementations << QString::fromStdString(n); + qInfo("audio drivers built in: %s", qPrintable(implementations.join(", "))); + + if (!haveDevice()) { + qInfo("no audio device could be opened: %s", + qPrintable(m_dialogs.join(" | "))); + m_dialogs.clear(); + QSKIP("no audio device: only no_input_does_no_harm applies"); + } + qInfo("playback latency reported: %lld frames (%.1f ms)", + (long long)m_window->playSource()->getTargetPlayLatency(), + double(m_window->playSource()->getTargetPlayLatency()) * 1000.0 / rate); + qInfo("record latency reported: %d frames (%.1f ms)", + m_window->recordTarget()->getSystemRecordLatency(), + m_window->recordTarget()->getSystemRecordLatency() * 1000.0 / rate); + QVERIFY2(canRecord(), "the device can play but not record: is a " + "microphone connected and allowed?"); + } + + // Two recordings into one take, at two places in the song, with the + // reference playing: the microphone hears it and the take records it + void record_the_reference_through_the_air() { + if (!canRecord()) QSKIP("no device to record from"); + + for (double at : takePositions()) { + Recorded r; + m_window->seekTo(frames(at)); + m_window->doRecord(); + QVERIFY2(m_window->recordTarget()->isRecording(), + qPrintable("Record did not start: " + m_dialogs.join(" | "))); + QTest::qWait(takeMs); + if (auto dots = sv::ModelById::getAs + (m_window->realtimeModelId())) { + r.dots = dots->getEventCount(); + } + sv::sv_frame_t received = + m_window->recordTarget()->getFramesReceived(); + QElapsedTimer timer; + timer.start(); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + if (received == 0) { + m_deliveredNothing = true; + QTRY_VERIFY(!m_window->recordingInProgress()); + QFAIL(qPrintable(QString("the device opened, but in %1 s it " + "delivered no input at all") + .arg(takeMs / 1000))); + } + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 60000); + r.stopSeconds = double(timer.elapsed()) / 1000.0; + + // Each channel of what the device delivered, to see which + // input the microphone is on + sv::WavFileReader reader { sv::FileSource(newestRecording()) }; + QVERIFY(reader.isOK()); + int channels = reader.getChannelCount(); + auto data = reader.getInterleavedFrames(0, reader.getFrameCount()); + for (int c = 0; c < channels; ++c) { + std::vector one; + for (size_t i = size_t(c); i < data.size(); i += size_t(channels)) { + one.push_back(data[i]); + } + r.channelLevels.push_back(levelDb(one)); + } + m_recorded.push_back(r); + } + + auto ranges = m_window->takes()->getCoverage().getRanges(); + QCOMPARE(ranges.size(), m_recorded.size()); + for (size_t i = 0; i < ranges.size(); ++i) { + m_recorded[i].range = ranges[i]; + } + QVERIFY(m_dialogs.isEmpty()); + } + + // Checklist: no input device, or one that records nothing: Record + // does nothing harmful, and the next file opened is analysed as usual + void no_input_does_no_harm() { + if (canRecord() && !m_deliveredNothing) { + QSKIP("the device records: this is for one that does not"); + } + m_dialogs.clear(); + m_window->seekTo(frames(30.0)); + m_window->doRecord(); + QTest::qWait(1000); + if (m_window->recordTarget()->isRecording()) m_window->doRecord(); + QTest::qWait(500); + qInfo("Record with no input: %s", + m_dialogs.isEmpty() ? "no dialog" + : qPrintable("dialog " + m_dialogs.join(" | "))); + m_dialogs.clear(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->recordingInProgress()); + QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY2(!m_window->takes()->haveTake(), + "a recording of nothing was kept as a take"); + + auto other = TestSignals::sawtooth(294.0, rate, int(frames(3.0)), 0.5f); + m_window->discardModifications(); + QCOMPARE(m_window->openPath(writeWav(other, "next.wav"), + MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + auto pitch = sv::ModelById::getAs + (m_window->analyser()->getLayer(Analyser::PitchTrack)->getModel()); + QVERIFY(pitch && pitch->getEventCount() > 10); + + // With no device at all, MainWindowBase tries again to open one + // for every file opened, and says so each time when it cannot. + // That warning is expected; anything else is not + QStringList others; + for (QString d : m_dialogs) { + if (!d.startsWith("Couldn't open audio device")) others << d; + } + qInfo("%d warnings about the device while opening the next file", + int(m_dialogs.size() - others.size())); + m_dialogs.clear(); + QVERIFY2(others.isEmpty(), qPrintable(others.join(" | "))); + } + + // Checklist: the take lines up with the reference (latency on this + // machine), and every recording in one take does (several phrases) + void takes_line_up_with_the_reference() { + if (m_recorded.empty()) QSKIP("nothing was recorded"); + const std::vector clicks = makeReference(true); + const long maxLag = long(frames(0.3)); + QString audio = m_window->takes()->getAudioPath(); + + for (Recorded &r : m_recorded) { + // Leave out the edges of the range: the start is where the + // reference only began to play + sv::sv_frame_t from = r.range.start + frames(0.5); + sv::sv_frame_t to = r.range.end - frames(0.2); + QVERIFY(to - from > frames(2.0)); + auto recorded = readMono(audio, from, to - from); + QCOMPARE(sv::sv_frame_t(recorded.size()), to - from); + + // Only the half of each pattern that holds the clicks: the + // tone of the other half is nothing to line up by, and would + // count against how well the clicks match + for (size_t i = 0; i < recorded.size(); ++i) { + double t = std::fmod(double(from + sv::sv_frame_t(i)) / rate, + patternSeconds); + if (t < 1.0) recorded[i] = 0.f; + } + std::vector ref(clicks.begin() + long(from) - maxLag, + clicks.begin() + long(to) + maxLag); + std::vector curve; + auto best = bestLag(ref, recorded, maxLag, &curve); + r.lagMs = double(best.first) * 1000.0 / rate; + r.peak = best.second; + + // The strongest match later than the first, beyond the length + // of a click and its ringing: an echo of the take played back + // out of the speakers would put the clicks there a second time + for (long lag = best.first + long(frames(0.015)); lag <= maxLag; + ++lag) { + double v = curve[size_t(lag + maxLag)]; + if (v > r.echo) { + r.echo = v; + r.echoMs = double(lag - best.first) * 1000.0 / rate; + } + } + + qInfo("recording at %.1f s: clicks %+.1f ms from the reference " + "(match %.2f); stop took %.1f s; %d live dots; channels " + "(dB) %s", + double(r.range.start) / rate, r.lagMs, r.peak, + r.stopSeconds, r.dots, + qPrintable([&]() { + QStringList l; + for (double d : r.channelLevels) { + l << QString::number(d, 'f', 1); + } + return l.join(" "); + }())); + } + + for (const Recorded &r : m_recorded) { + QVERIFY2(r.peak > 0.1, + qPrintable(QString("the clicks of the reference are " + "hardly in the recording at %1 s " + "(match %2): can the microphone hear " + "the speakers?") + .arg(double(r.range.start) / rate) + .arg(r.peak, 0, 'f', 2))); + QVERIFY2(std::fabs(r.lagMs) <= toleranceMs, + qPrintable(QString("the recording at %1 s is %2 ms %3 " + "the reference") + .arg(double(r.range.start) / rate) + .arg(std::fabs(r.lagMs), 0, 'f', 1) + .arg(r.lagMs > 0 ? "later than" : "earlier " + "than"))); + } + if (m_recorded.size() > 1) { + double spread = std::fabs(m_recorded[0].lagMs - + m_recorded[1].lagMs); + QVERIFY2(spread <= 5.0, + qPrintable(QString("the two recordings of one take are " + "%1 ms apart in their timing") + .arg(spread, 0, 'f', 1))); + } + } + + // Checklist: nothing of the take in the speakers while recording. + // If the input were played back out, the microphone would hear every + // click a second time, a round trip later + void nothing_of_the_take_comes_back_out() { + if (m_recorded.empty()) QSKIP("nothing was recorded"); + for (const Recorded &r : m_recorded) { + qInfo("recording at %.1f s: strongest later match %.2f at " + "+%.1f ms, against %.2f for the clicks themselves", + double(r.range.start) / rate, r.echo, r.echoMs, r.peak); + QVERIFY2(r.echo < 0.5 * r.peak, + qPrintable(QString("the clicks come again %1 ms later " + "at %2 of their strength: is the " + "input being played back out, by " + "Tony or by the system (\"Listen to " + "this device\")?") + .arg(r.echoMs, 0, 'f', 1) + .arg(r.echo / r.peak, 0, 'f', 2))); + } + } + + // Checklist: how long Stop takes on a four-minute song: new pitch only + // where the singing was, not a whole-song analysis + void stop_is_quicker_than_a_whole_song() { + if (m_recorded.empty()) QSKIP("nothing was recorded"); + for (const Recorded &r : m_recorded) { + QVERIFY2(r.stopSeconds < 0.5 * m_wholeSongSeconds, + qPrintable(QString("Stop took %1 s, the whole song's " + "analysis %2 s") + .arg(r.stopSeconds, 0, 'f', 1) + .arg(m_wholeSongSeconds, 0, 'f', 1))); + } + } + + // Checklist: live dots appear, whichever input the microphone is on + void live_dots_were_drawn() { + if (m_recorded.empty()) QSKIP("nothing was recorded"); + for (const Recorded &r : m_recorded) { + QVERIFY2(r.dots > 10, + qPrintable(QString("only %1 live dots during the " + "recording at %2 s") + .arg(r.dots) + .arg(double(r.range.start) / rate))); + } + } +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 297959d7..b565af3d 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -887,6 +887,39 @@ private slots: stopTake(); } + // A microphone on input 2 of an interface, nothing on input 1: the + // dots and the take's pitch come from the mixdown, so they are there + void live_dots_from_the_second_input() { + FakeAudioIO::Config config; + config.channels = 2; + config.inputChannel = 1; + config.input = tone(highHz, 3.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(1000); + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + auto events = model->getAllEvents(); + QVERIFY2(events.size() > 20, + qPrintable(QString("only %1 live dots after a second of " + "singing into input 2") + .arg(events.size()))); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(events), highHz)) < 10.0); + + stopTake(); + if (QTest::currentTestFailed()) return; + auto pitch = pitchEvents(m_window->analyser2()); + QVERIFY2(pitch.size() > 20, "the take of input 2 has no pitch track"); + QVERIFY(std::fabs(TestSignals::centsBetween + (medianHz(pitch), highHz)) < 10.0); + } + void live_dots_removed() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h new file mode 100644 index 00000000..e0af19b0 --- /dev/null +++ b/main/test/TestUiChecks.h @@ -0,0 +1,1561 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_UI_CHECKS_H +#define TEST_UI_CHECKS_H + +// Tier 6: the window as the user sees and handles it. The same +// MainWindow and fake device as TestRecordWorkflow, but shown, driven +// with key presses, mouse gestures and its own dialogs, and judged by +// the pixels of pane 0 as it is drawn -- the items of the manual +// checklist that are about what is on the screen. +// +// With TONY_TEST_SHOT_DIR set, the suite also saves what it looked at, +// as PNG files, for a person to look over. + +#include "TestSignals.h" +#include "TestMainWindow.h" + +#include "../AlternatePitchTrack.h" +#include "../TakeLayers.h" + +#include "version.h" + +#include "framework/Document.h" +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "layer/Layer.h" +#include "layer/ColourDatabase.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/model/NoteModel.h" +#include "data/model/RegionModel.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" +#include "data/fileio/FileSource.h" +#include "base/RecordDirectory.h" +#include "transform/ModelTransformerFactory.h" +#include "widgets/CommandHistory.h" +#include "widgets/InteractiveFileFinder.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +class TestUiChecks : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + + // Whole numbers of samples per period: see TestSingingAnalysis.h + static constexpr double lowHz = 220.5; + static constexpr double highHz = 294.0; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + TestMainWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + QString m_shotDir; + + // Given each modal dialog before the watchdog dismisses it: true if + // it answered the dialog itself + std::function m_answerDialog; + + static std::vector tone(double hz, double seconds) { + return TestSignals::sawtooth(hz, rate, int(seconds * rate), 0.5f); + } + + static sv::sv_frame_t frames(double seconds) { + return sv::sv_frame_t(seconds * rate); + } + + QString writeWav(const std::vector &data) { + QString path = m_dir.filePath + (QString("audio-%1.wav").arg(++m_fileCounter)); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + if (!writer.isOK() || + !writer.writeSamples(&ptr, sv::sv_frame_t(data.size())) || + !writer.close()) { + return {}; + } + return path; + } + + // Shown at a fixed size and active, as the user has it: key presses + // reach the window's shortcuts only when it is the active window + void makeWindow(FakeAudioIO::Config config) { + delete m_window; + m_window = new TestMainWindow(config); + m_window->resize(1200, 800); + m_window->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_window)); + m_window->activateWindow(); + QVERIFY(QTest::qWaitForWindowActive(m_window)); + } + + static bool analysed(Analyser *a) { + return a && a->getLayer(Analyser::PitchTrack) && + a->getLayer(Analyser::Notes) && + a->getInitialAnalysisCompletion() >= 100 && + !a->isAnalysingRange() && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + } + + void openReference(QString path) { + QVERIFY(!path.isEmpty()); + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + } + + void startTake() { + QVERIFY(!m_window->recordTarget()->isRecording()); + m_window->doRecord(); + QVERIFY(m_window->recordTarget()->isRecording()); + } + + void stopTake() { + QVERIFY(m_window->recordTarget()->isRecording()); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + } + + void take(int ms) { + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(ms); + stopTake(); + } + + sv::Pane *pane0() { return m_window->paneStack()->getPane(0); } + + // Whatever is seen of the page from start to end, and a little room + // either side + void showSeconds(double start, double end) { + sv::Pane *pane = pane0(); + int width = pane->width(); + int perPixel = int(std::ceil((end - start) * rate / (width * 0.8))); + pane->setZoomLevel(sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, + std::max(1, perPixel))); + pane->setCentreFrame(frames((start + end) / 2.0)); + } + + // Pane 0 as it is on the screen just now: the window's backing store, + // which holds what the pane's own paint events put there. Not + // QWidget::grab(), which has the pane paint itself once more for the + // occasion and so can show what the screen does not + QImage grabPane() { + QCoreApplication::processEvents(); + QPixmap window = m_window->screen()->grabWindow(m_window->winId()); + QRect rect(pane0()->mapTo(m_window, QPoint(0, 0)), pane0()->size()); + return window.copy(rect).toImage() + .convertToFormat(QImage::Format_RGB32); + } + + // ... and drawn afresh, all of it. After playback the pane's own + // cache of what it drew holds the translucent note boxes painted a + // second time over themselves, until the next zoom or scroll; that + // is how the view composes itself, not what a layer draws, and would + // otherwise be taken for a change to the layers + QImage grabPaneRedrawn() { + sv::Pane *pane = pane0(); + sv::ZoomLevel zoom = pane->getZoomLevel(); + sv::sv_frame_t centre = pane->getCentreFrame(); + // (a zoom level only one step away snaps back to this one) + pane->setZoomLevel(sv::ZoomLevel(zoom.zone, zoom.level * 2)); + grabPane(); + pane->setZoomLevel(zoom); + pane->setCentreFrame(centre); + return grabPane(); + } + + void saveShot(QString name, const QImage &image) { + if (m_shotDir == "") return; + QDir().mkpath(m_shotDir); + QString test = QTest::currentTestFunction(); + image.save(QDir(m_shotDir).filePath(test + "-" + name + ".png")); + } + + // The whole window, as grabPane() takes the pane + void saveWindowShot(QString name) { + if (m_shotDir == "") return; + QCoreApplication::processEvents(); + saveShot(name, m_window->screen()->grabWindow(m_window->winId()) + .toImage()); + } + + static bool closeTo(QRgb pixel, QColor colour, int tolerance) { + return std::abs(qRed(pixel) - colour.red()) <= tolerance && + std::abs(qGreen(pixel) - colour.green()) <= tolerance && + std::abs(qBlue(pixel) - colour.blue()) <= tolerance; + } + + static QColor named(QString name) { + auto cdb = sv::ColourDatabase::getInstance(); + return cdb->getColour(cdb->getColourIndex(name)); + } + + // The orange of the live dots, the take's pitch track and the + // coverage strip; the purple of the take's notes + static bool isOrange(QRgb pixel) { + return closeTo(pixel, named("Orange"), 30); + } + static bool isPurple(QRgb pixel) { + return closeTo(pixel, named("Bright Purple"), 30); + } + + // Orange, or orange seen through the translucent fill of the take's + // notes, which are drawn over its pitch track: what is sung, as it is + // drawn. Not the brighter orange of the pitch candidates + static bool isSinging(QRgb pixel) { + return qRed(pixel) >= 200 && qGreen(pixel) >= 90 && + qGreen(pixel) <= 170 && qBlue(pixel) <= 140; + } + + // Pixels of a colour in the columns [x0, x1), above the band of the + // coverage strip at the bottom of the pane + static int countAbove(const QImage &image, int x0, int x1, + std::function is) { + int n = 0; + x0 = std::max(0, x0); + x1 = std::min(image.width(), x1); + for (int x = x0; x < x1; ++x) { + for (int y = 0; y < image.height() - 10; ++y) { + if (is(image.pixel(x, y))) ++n; + } + } + return n; + } + + // Leftmost and rightmost column holding such a pixel, or -1 + static std::pair extentAbove(const QImage &image, + std::function is) { + int left = -1, right = -1; + for (int x = 0; x < image.width(); ++x) { + for (int y = 0; y < image.height() - 10; ++y) { + if (is(image.pixel(x, y))) { + if (left < 0) left = x; + right = x; + break; + } + } + } + return { left, right }; + } + + // The play pointer as View::drawPlayPointer() draws it: a line of the + // background colour between two of the foreground, the whole height + // of the pane. -1 if there is none + static int pointerX(const QImage &image) { + auto dark = [](QRgb p) { return qGray(p) < 80; }; + auto light = [](QRgb p) { return qGray(p) > 200; }; + int h = image.height(); + for (int x = 1; x + 1 < image.width(); ++x) { + int n = 0; + for (int y = 1; y + 1 < h; ++y) { + if (dark(image.pixel(x - 1, y)) && light(image.pixel(x, y)) && + dark(image.pixel(x + 1, y))) ++n; + } + if (n > (h * 9) / 10) return x; + } + return -1; + } + + QMenu *menuTitled(QString title) { + for (QAction *a : m_window->menuBar()->actions()) { + if (a->menu() && a->text() == title) return a->menu(); + } + return nullptr; + } + + // The Undo item of the Edit menu, as the user reads it + QString undoText() { + QMenu *edit = menuTitled(tr("&Edit")); + if (!edit) return "(no Edit menu)"; + for (QAction *a : edit->actions()) { + if (a->shortcut() == QKeySequence(tr("Ctrl+Z"))) return a->text(); + } + return "(no Undo item)"; + } + + void press(QKeySequence keys) { + QVERIFY(QTest::qWaitForWindowActive(m_window)); + QTest::keySequence(m_window, keys); + QCoreApplication::processEvents(); + } + + Coverage::Ranges coverage() { + return m_window->takes()->getCoverage().getRanges(); + } + + // Everything in pane 0 a gesture could change: the events of every + // layer's model, how each layer scales itself, the zoom, the + // selection and the undo history. The centre frame is left out: + // dragging with the navigate tool is meant to scroll + struct PaneState { + std::map events; + std::map> extents; + std::vector layers; + sv::ZoomLevel zoom; + sv::MultiSelection::SelectionList selections; + QString undo; + + bool operator==(const PaneState &s) const { + return events == s.events && extents == s.extents && + layers == s.layers && zoom == s.zoom && + selections == s.selections && undo == s.undo; + } + }; + + PaneState paneState() { + PaneState s; + sv::Pane *pane = pane0(); + for (int i = 0; i < pane->getLayerCount(); ++i) { + sv::Layer *layer = pane->getLayer(i); + QString key = QString("%1 %2").arg(i).arg(layer->objectName()); + s.layers.push_back(key); + sv::ModelId id = layer->getModel(); + if (auto m = sv::ModelById::getAs(id)) { + s.events[key] = m->getAllEvents(); + } else if (auto m = sv::ModelById::getAs(id)) { + s.events[key] = m->getAllEvents(); + } else if (auto m = sv::ModelById::getAs(id)) { + s.events[key] = m->getAllEvents(); + } + double min = 0.0, max = 0.0; + if (layer->getDisplayExtents(min, max)) { + s.extents[key] = { min, max }; + } + } + s.zoom = pane->getZoomLevel(); + s.selections = m_window->selections(); + s.undo = undoText(); + return s; + } + + static QString describe(const PaneState &a, const PaneState &b) { + QStringList what; + if (a.layers != b.layers) what << "the layers of the pane"; + for (const auto &e : a.events) { + auto i = b.events.find(e.first); + if (i == b.events.end() || i->second != e.second) { + what << QString("the events of \"%1\"").arg(e.first); + } + } + for (const auto &e : a.extents) { + auto i = b.extents.find(e.first); + if (i == b.extents.end() || i->second != e.second) { + what << QString("the scale of \"%1\"").arg(e.first); + } + } + if (!(a.zoom == b.zoom)) what << "the zoom"; + if (a.selections != b.selections) what << "the selection"; + if (a.undo != b.undo) what << "the undo history (" + b.undo + ")"; + return what.join(", "); + } + + // Not a slot: QtTest would run it as a test + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + if (m_answerDialog && m_answerDialog(modal)) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + m_dialogs.push_back(description); + // The last button is Cancel (or No) in every question Tony + // asks; see TestRecordWorkflow::dismissDialog() + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + + // Press the button of this role in a message box, ticking its check + // box first if asked to; the text of the box goes into asked + static std::function + answerWith(QMessageBox::StandardButton button, bool tick, + QStringList *asked) { + return [=](QWidget *modal) { + auto box = qobject_cast(modal); + if (!box || !box->button(button)) return false; + if (asked) asked->push_back(box->windowTitle()); + if (tick && box->checkBox()) box->checkBox()->setChecked(true); + box->button(button)->click(); + return true; + }; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + m_shotDir = qEnvironmentVariable("TONY_TEST_SHOT_DIR"); + + QSettings().clear(); + + // As TestRecordWorkflow: no network question, .ton is a session, + // takes go into the temporary directory + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + + sv::RecordDirectory::setRecordContainerDirectory + (m_dir.filePath("recorded")); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + m_answerDialog = nullptr; + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", false); + settings.setValue("preroll", false); + settings.setValue("recordintoselection", false); + settings.remove("prerollseconds"); + settings.endGroup(); + settings.beginGroup("Analyser"); + settings.remove(""); + settings.endGroup(); + SingingTakes::setOverwriteConfirmationWanted(true); + } + + void cleanup() { + m_answerDialog = nullptr; + if (m_window) { + if (m_window->recordTarget()->isRecording()) { + m_window->doRecord(); + } + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + sv::RecordDirectory::setRecordContainerDirectory(""); + } + + // Checklist: live dots appear under the playback cursor, not behind + // it; during a take at P > 0 the cursor starts at P, the pane follows + // it, and cursor and dots are in the same place. Singing exactly in + // time with the reference, through a device with latency + void live_dots_under_the_cursor() { + FakeAudioIO::Config config; + config.recordLatency = 512; + config.playbackLatency = 1024; + config.inputDelay = 1536; + config.inputFollowsPlayback = true; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 12.0))); + if (QTest::currentTestFailed()) return; + + // A page of about two seconds, so that the take runs off it + const sv::sv_frame_t P = frames(4.0); + showSeconds(3.5, 5.5); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + + // The most the newest dot may trail the cursor by: the latency of + // the device, YIN's window and the delivery of the dots, with room + // to spare. A dot a whole latency out, or one at the take's own + // frame rather than the song's, is far more than this + const sv::sv_frame_t lag = frames(0.3); + sv::Pane *pane = pane0(); + QElapsedTimer timer; + timer.start(); + int pages = 0, looked = 0; + int firstPageStart = -1; + bool sawFirstDots = false; + sv::sv_frame_t worstLag = 0; + QString worst; + + while (timer.elapsed() < 3000) { + QTest::qWait(150); + sv::sv_frame_t cursorBefore = m_window->playbackFrame(); + QImage image = grabPane(); + sv::sv_frame_t cursor = m_window->playbackFrame(); + int x = pointerX(image); + QVERIFY2(x >= 0, + qPrintable(QString("%1 ms into the take there is no play " + "pointer on the pane: it has not " + "followed the cursor (frame %2)") + .arg(timer.elapsed()).arg(cursor))); + // The view moves its pointer on a timer of its own (20 ms in + // the svgui fork), so it may be one tick behind + int earliest = pane->getXForFrame(cursorBefore - frames(0.05)); + int latest = pane->getXForFrame(cursor); + QVERIFY2(x >= earliest - 2 && x <= latest + 2, + qPrintable(QString("the pointer is drawn at x = %1, the " + "playback frame is between x = %2 " + "and %3") + .arg(x).arg(earliest).arg(latest))); + + int start = int(pane->getStartFrame()); + if (firstPageStart < 0) firstPageStart = start; + else if (start != firstPageStart) ++pages; + + auto dots = extentAbove(image, isSinging); + if (dots.second < 0) continue; + ++looked; + + // Never ahead of the cursor; how far behind it, and the + // newest dot the model has, for the worst moment + sv::sv_frame_t newest = pane->getFrameForX(dots.second); + QVERIFY2(newest <= cursor + pane->getZoomLevel().level * 3, + qPrintable(QString("%1 ms into the take a dot is drawn at " + "frame %2, ahead of the cursor at %3") + .arg(timer.elapsed()).arg(newest) + .arg(cursor))); + if (cursor - newest > worstLag) { + worstLag = cursor - newest; + sv::sv_frame_t inModel = -1; + if (auto model = sv::ModelById::getAs + (m_window->realtimeModelId())) { + auto all = model->getAllEvents(); + if (!all.empty()) inModel = all.back().getFrame(); + } + worst = QString("%1 ms into the take the newest dot drawn is " + "%2 ms behind the cursor; the newest in the " + "model is %3 ms behind it") + .arg(timer.elapsed()) + .arg(double(cursor - newest) * 1000.0 / rate, 0, 'f', 0) + .arg(double(cursor - inModel) * 1000.0 / rate, 0, 'f', 0); + } + + // The first dots of the take are at P, where the singing began + if (!sawFirstDots && pane->getStartFrame() < P) { + sawFirstDots = true; + sv::sv_frame_t oldest = pane->getFrameForX(dots.first); + QVERIFY2(std::llabs(oldest - P) < frames(0.1), + qPrintable(QString("the first dot of a take at frame " + "%1 is at frame %2") + .arg(P).arg(oldest))); + saveShot("first-dots", image); + qInfo("newest dot %.0f ms behind the cursor", + double(cursor - newest) * 1000.0 / rate); + } + } + QVERIFY2(looked >= 5, "hardly any live dots were drawn"); + QVERIFY(sawFirstDots); + QVERIFY2(pages >= 1, "the take never ran off the first page, so " + "whether the pane follows it was not seen"); + saveWindowShot("during"); + qInfo("%s", qPrintable(worst)); + QEXPECT_FAIL("", "the live dot model is made with notifyOnAdd false, " + "so a dot added to it tells the pane nothing: dots are " + "drawn only when one widens the model's range or the " + "pane redraws for another reason", Continue); + QVERIFY2(worstLag <= lag, qPrintable(worst)); + + // The dots stay until the take's pitch track is there, which is + // drawn over the same place in the same orange: at no moment is + // what was sung missing from the pane + const sv::sv_frame_t end = m_window->playbackFrame(); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + showSeconds(4.0, 4.0 + double(end - P) / rate); + int sungFrom = pane->getXForFrame(P + frames(0.2)); + int sungTo = pane->getXForFrame(end - frames(0.5)); + QElapsedTimer sinceStop; + sinceStop.start(); + while (!analysed(m_window->analyser2())) { + QVERIFY2(countAbove(grabPane(), sungFrom, sungTo, isSinging) > 0, + qPrintable(QString("%1 ms after Stop nothing of what was " + "sung is drawn") + .arg(sinceStop.elapsed()))); + QVERIFY(sinceStop.elapsed() < 30000); + QTest::qWait(20); + } + QImage after = grabPane(); + QVERIFY(countAbove(after, sungFrom, sungTo, isSinging) > 0); + saveShot("analysed", after); + + // The status bar is still once nothing is recorded or played + QTest::qWait(100); + QString status = m_window->statusText(); + QTest::qWait(400); + QCOMPARE(m_window->statusText(), status); + } + + // Checklist: recording over singing that is there, that take's own + // pitch track and notes are out of sight for the take, so only the + // dots are drawn, and they are back when the take stops + void own_pitch_out_of_sight_while_recording_over_it() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(2200); + if (QTest::currentTestFailed()) return; + showSeconds(0.0, 3.0); + sv::Pane *pane = pane0(); + + const sv::sv_frame_t P = frames(0.4); + int ahead0 = pane->getXForFrame(frames(1.3)); + int ahead1 = pane->getXForFrame(frames(1.9)); + QImage before = grabPane(); + QVERIFY2(countAbove(before, ahead0, ahead1, isSinging) > 0, + "the first take's pitch track is not drawn"); + QVERIFY2(countAbove(before, 0, before.width(), isPurple) > 0, + "the first take's notes are not drawn"); + saveShot("before", before); + + m_window->seekTo(P); + m_window->setRecordOverAnswer(true); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(500); + + // Ahead of the cursor, where the old singing is and nothing new + // has been sung yet, nothing of the take is drawn. (Record puts + // the view back on P, so where things are is asked again) + QImage during = grabPane(); + ahead0 = pane->getXForFrame(frames(1.3)); + ahead1 = pane->getXForFrame(frames(1.9)); + int x = pointerX(during); + QVERIFY2(x >= 0 && x < ahead0, + qPrintable(QString("the pointer is at x = %1, the old " + "singing to look at from x = %2") + .arg(x).arg(ahead0))); + QVERIFY2(countAbove(during, ahead0, ahead1, isSinging) == 0, + "the take's own pitch track is drawn during a take over it"); + QVERIFY2(countAbove(during, 0, during.width(), isPurple) == 0, + "the take's own notes are drawn during a take over it"); + QVERIFY2(countAbove(during, pane->getXForFrame(P), x, isSinging) > 0, + "no live dots behind the cursor"); + saveShot("during", during); + + stopTake(); + if (QTest::currentTestFailed()) return; + QImage after = grabPane(); + ahead0 = pane->getXForFrame(frames(1.3)); + ahead1 = pane->getXForFrame(frames(1.9)); + QVERIFY2(countAbove(after, ahead0, ahead1, isSinging) > 0, + "the rest of the old pitch track did not come back"); + QVERIFY2(countAbove(after, 0, after.width(), isPurple) > 0, + "the notes did not come back"); + saveShot("after", after); + } + + // Checklist: the coverage strip cannot be touched. Clicking, double- + // clicking and dragging on it with either tool creates, moves, selects + // and edits nothing of its own and does not change the pane's scale. + // The Edit tool acts on the take's note at the time it is used, + // wherever in the pane that is -- the band too -- so an edit of that + // note is what it may do, and one undo takes that back exactly + void strip_ignores_the_mouse() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + // One recording: see strip_on_top_after_another_recording + m_window->seekTo(frames(0.5)); + take(1500); + if (QTest::currentTestFailed()) return; + m_window->clearSelections(); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + showSeconds(0.0, 3.0); + + sv::Pane *pane = pane0(); + auto range = coverage()[0]; + QImage image = grabPaneRedrawn(); + const int y = image.height() - 3; + const int inBand = pane->getXForFrame(range.start + frames(0.3)); + const int atEnd = pane->getXForFrame(range.end) - 1; + const int beyond = pane->getXForFrame(range.end + frames(0.4)); + QVERIFY2(isOrange(image.pixel(inBand, y)) && + isOrange(image.pixel(atEnd - 2, y)) && + !isOrange(image.pixel(beyond, y)), + "the strip is not where the gestures are going to be made"); + saveShot("strip", image); + + auto drag = [&](int from, int to) { + QTest::mousePress(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(from, y)); + for (int i = 1; i <= 10; ++i) { + QTest::mouseMove(pane, QPoint(from + (to - from) * i / 10, y)); + QTest::qWait(10); + } + QTest::mouseRelease(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(to, y)); + }; + + struct Gesture { QString name; std::function act; }; + std::vector gestures { + { "a click in the band", [&]() { + QTest::mouseClick(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(inBand, y)); } }, + { "a double-click in the band", [&]() { + QTest::mouseDClick(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(inBand, y)); } }, + { "a click at its end", [&]() { + QTest::mouseClick(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(atEnd, y)); } }, + { "a drag along it", [&]() { drag(inBand, atEnd); } }, + { "a drag off its end", [&]() { drag(atEnd, beyond); } }, + { "a drag onto it", [&]() { drag(beyond, inBand); } }, + }; + + // Everything but the take's notes and the undo history + auto apartFromNotes = [&](PaneState s) { + for (auto i = s.events.begin(); i != s.events.end(); ) { + if (i->first.endsWith(TakeLayers::nameFor + (m_window->takes()->getActiveName(), + TakeLayers::Notes))) { + i = s.events.erase(i); + } else { + ++i; + } + } + s.undo = ""; + return s; + }; + + int noteEdits = 0; + for (QString tool : { QString("navigate"), QString("edit") }) { + press(QKeySequence(tool == "navigate" ? "1" : "2")); + for (const Gesture &g : gestures) { + sv::sv_frame_t centre = pane->getCentreFrame(); + const PaneState before = paneState(); + g.act(); + QTest::qWait(300); // outlast the double-click interval + // The navigate tool scrolls when dragged, as anywhere + pane->setCentreFrame(centre); + PaneState after = paneState(); + if (tool == "navigate") { + QVERIFY2(after == before, + qPrintable(QString("with the navigate tool, %1 " + "changed %2") + .arg(g.name) + .arg(describe(before, after)))); + continue; + } + QVERIFY2(apartFromNotes(after) == apartFromNotes(before), + qPrintable(QString("with the edit tool, %1 changed " + "%2") + .arg(g.name) + .arg(describe(before, after)))); + if (after.undo != before.undo) { + ++noteEdits; + press(QKeySequence(tr("Ctrl+Z"))); + PaneState undone = paneState(); + QVERIFY2(undone == before, + qPrintable(QString("undoing what %1 did with " + "the edit tool left %2") + .arg(g.name) + .arg(describe(before, undone)))); + } + } + } + qInfo("%d of the gestures with the edit tool edited the take's note", + noteEdits); + press(QKeySequence("1")); + } + + // Checklist: the band is readable over waveform and dots at every + // zoom: where there is singing the band is drawn over whatever else + // is there, band high, and nowhere else + void strip_band_at_every_zoom() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 6.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(frames(1.0)); + take(1500); + if (QTest::currentTestFailed()) return; + auto ranges = coverage(); + QCOMPARE(int(ranges.size()), 1); + const auto range = ranges[0]; + + sv::Pane *pane = pane0(); + for (int perPixel : { 16, 128, 1024, 8192 }) { + // Centred on the end of the singing: some of the band, and + // some of the pane with nothing recorded, at every zoom + pane->setZoomLevel(sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, + perPixel)); + pane->setCentreFrame(range.end); + QImage image = grabPaneRedrawn(); + saveShot(QString("zoom-%1").arg(perPixel), image); + const int h = image.height(); + + // Not under the vertical scale at the left, nor where the play + // pointer is drawn over everything + int pointer = pointerX(image); + int inRange = 0, outside = 0; + for (int x = pane->getVerticalScaleWidth() + 1; + x < image.width(); ++x) { + if (pointer >= 0 && std::abs(x - pointer) <= 2) continue; + sv::sv_frame_t f0 = pane->getFrameForX(x); + sv::sv_frame_t f1 = pane->getFrameForX(x + 1); + if (f0 < 0) continue; + if (f0 >= range.start && f1 <= range.end) { + ++inRange; + for (int yy = h - 6; yy < h; ++yy) { + QVERIFY2(isOrange(image.pixel(x, yy)), + qPrintable(QString("at %1 frames per pixel " + "the band has a hole at " + "x = %2, y = %3") + .arg(perPixel).arg(x).arg(yy))); + } + QVERIFY2(!isOrange(image.pixel(x, h - 9)), + qPrintable(QString("at %1 frames per pixel the " + "band is more than a band " + "high at x = %2") + .arg(perPixel).arg(x))); + } else if (f1 <= range.start || f0 >= range.end) { + ++outside; + QVERIFY2(!isOrange(image.pixel(x, h - 3)), + qPrintable(QString("at %1 frames per pixel the " + "band is drawn at x = %2, " + "where nothing was recorded") + .arg(perPixel).arg(x))); + } + } + QVERIFY2(inRange >= 3 && outside >= 10, + qPrintable(QString("at %1 frames per pixel %2 columns " + "of the band and %3 without it are in " + "view") + .arg(perPixel).arg(inRange).arg(outside))); + } + saveWindowShot("window"); + } + + // The band after a second recording, in a gap of the first: the take's + // audio is swapped for a file holding both, and the band has to stay + // on top of the waveform of the new file + void strip_on_top_after_another_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(1000); + if (QTest::currentTestFailed()) return; + showSeconds(0.0, 3.0); + sv::Pane *pane = pane0(); + const sv::sv_frame_t inFirst = frames(0.4); + QImage image = grabPaneRedrawn(); + QVERIFY(isOrange(image.pixel(pane->getXForFrame(inFirst), + image.height() - 3))); + + m_window->seekTo(frames(2.0)); + take(700); + if (QTest::currentTestFailed()) return; + showSeconds(0.0, 3.0); + image = grabPaneRedrawn(); + saveShot("after-second", image); + int y = image.height() - 3; + QEXPECT_FAIL("", "the take's waveform layer made by the audio swap " + "is attached above the coverage strip and covers the " + "band (syncCoverageStrip() raises nothing once the " + "strip is shown)", Continue); + QVERIFY2(isOrange(image.pixel(pane->getXForFrame(inFirst), y)) && + isOrange(image.pixel(pane->getXForFrame(frames(2.2)), y)), + "after a second recording the band is not drawn"); + } + + // Checklist: Erase and Select Recording are greyed out with no take, + // no selection, while recording and during the analysis after Stop; + // the Takes menu and combo during a take -- and all of them come + // back by themselves, with nothing but the take's own events to + // update them + void menus_follow_the_take_by_themselves() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + QAction *erase = m_window->eraseSingingAction(); + QAction *select = m_window->selectRecordingAction(); + QMenu *takes = m_window->takesMenu(); + QVERIFY(takes); + QVERIFY(menuTitled(tr("&Edit"))->actions().contains(erase)); + QVERIFY(menuTitled(tr("&Edit"))->actions().contains(select)); + + auto takeItemsEnabled = [&]() { + int n = 0; + for (QAction *a : takes->actions()) { + if (!a->isSeparator() && a->isEnabled()) ++n; + } + return n; + }; + + // No take + m_window->selectRange(0, frames(0.5)); + QVERIFY(!erase->isEnabled()); + QVERIFY(!select->isEnabled()); + m_window->clearSelections(); + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(200); + QVERIFY2(!m_window->takeCombo()->isEnabled(), + "the take combo can be used during a take"); + QVERIFY2(takeItemsEnabled() == 0, + "items of the Takes menu can be used during a take"); + m_window->selectRange(0, frames(0.3)); + QVERIFY(!erase->isEnabled()); + QVERIFY(!select->isEnabled()); + QTest::qWait(600); + + // Stop, and the moment after it: the recorded range is being + // analysed, and erasing would throw that away + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(m_window->analysingRange()); + QVERIFY2(!erase->isEnabled(), + "Erase can be used while the take is being analysed"); + + // ... and everything back, by itself + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QTRY_VERIFY2(erase->isEnabled(), + "Erase did not come back after the analysis"); + QVERIFY(select->isEnabled()); + QVERIFY(m_window->takeCombo()->isEnabled()); + QVERIFY(takeItemsEnabled() > 0); + + // No selection, nothing to erase + m_window->clearSelections(); + QVERIFY(!erase->isEnabled()); + QVERIFY(select->isEnabled()); + } + + // Checklist: three recordings, Ctrl+Z three times, each takes back + // exactly one; the menu says "Record Singing" / "Erase Singing" and + // never anything about a layer or pane. With the keys, as the user + // does it + void ctrl_z_takes_back_one_recording_at_a_time() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + QCOMPARE(undoText(), tr("Nothing to undo")); + + for (double at : { 0.0, 1.5, 3.0 }) { + m_window->seekTo(frames(at)); + take(600); + if (QTest::currentTestFailed()) return; + } + QCOMPARE(int(coverage().size()), 3); + + // Every name the user can see in the Edit menu and in the undo + // and redo menus of the toolbar + auto verifyNames = [this]() { + QStringList seen; + for (QAction *a : menuTitled(tr("&Edit"))->actions()) { + seen << a->text(); + } + for (QToolBar *bar : m_window->findChildren()) { + for (QAction *a : bar->actions()) { + if (!a->menu()) continue; + for (QAction *b : a->menu()->actions()) seen << b->text(); + } + } + for (QString text : seen) { + QVERIFY2(!text.contains("Layer", Qt::CaseInsensitive) && + !text.contains("Pane", Qt::CaseInsensitive), + qPrintable("the undo history shows \"" + text + "\"")); + } + }; + + for (int left : { 2, 1, 0 }) { + QCOMPARE(undoText(), tr("&Undo %1").arg(tr("Record Singing"))); + verifyNames(); + if (QTest::currentTestFailed()) return; + press(QKeySequence(tr("Ctrl+Z"))); + QTRY_VERIFY(m_window->takes()->haveTake() ? + int(coverage().size()) == left : left == 0); + } + QCOMPARE(undoText(), tr("Nothing to undo")); + + // ... and all three back + for (int back : { 1, 2, 3 }) { + press(QKeySequence(tr("Ctrl+Shift+Z"))); + QTRY_VERIFY(m_window->takes()->haveTake() && + int(coverage().size()) == back); + } + QVERIFY(!m_window->analysingRange()); + QCOMPARE(undoText(), tr("&Undo %1").arg(tr("Record Singing"))); + + // Erase the middle of the second, with its shortcut. A selection + // has the reference's pitch analysed again in it, and that is an + // entry of the undo history of its own; it is waited for here, so + // that it is not what the next Ctrl+Z takes back + auto second = coverage()[1]; + m_window->selectRange(second.start + frames(0.1), + second.start + frames(0.3)); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + QTest::qWait(200); + qInfo("after making a selection the Edit menu says \"%s\"", + qPrintable(undoText())); + QTRY_VERIFY(m_window->eraseSingingAction()->isEnabled()); + press(QKeySequence(tr("Ctrl+D"))); + QTRY_COMPARE(int(coverage().size()), 4); + QCOMPARE(undoText(), tr("&Undo %1").arg(tr("Erase Singing"))); + verifyNames(); + if (QTest::currentTestFailed()) return; + + press(QKeySequence(tr("Ctrl+Z"))); + QTRY_COMPARE(int(coverage().size()), 3); + QCOMPARE(coverage()[1], second); + } + + // Checklist: recording again inside singing that is there asks first; + // No records nothing; "Don't ask again" with Yes holds across + // sessions. The dialog MainWindow shows, answered with its buttons + void record_over_question_in_its_dialog() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + m_window->setRecordOverAskedInDialog(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + take(1500); + if (QTest::currentTestFailed()) return; + const Coverage::Ranges first = coverage(); + const QString firstPath = m_window->takes()->getAudioPath(); + + // No: nothing recorded, and nothing left running + QStringList asked; + m_answerDialog = answerWith(QMessageBox::No, false, &asked); + m_window->seekTo(frames(0.5)); + m_window->doRecord(); + QCOMPARE(asked.size(), 1); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->recordingInProgress()); + QTest::qWait(300); + QCOMPARE(coverage(), first); + QCOMPARE(m_window->takes()->getAudioPath(), firstPath); + + // Yes, and don't ask again + m_answerDialog = answerWith(QMessageBox::Yes, true, &asked); + m_window->seekTo(frames(0.5)); + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(asked.size(), 2); + QTest::qWait(500); + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->takes()->getAudioPath() != firstPath); + + // A session of its own, in a window of its own: not asked + m_answerDialog = answerWith(QMessageBox::Yes, false, &asked); + m_window->doCloseSession(); + makeWindow(config); + if (QTest::currentTestFailed()) return; + m_window->setRecordOverAskedInDialog(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + take(1200); + if (QTest::currentTestFailed()) return; + m_window->seekTo(frames(0.3)); + take(400); + if (QTest::currentTestFailed()) return; + QCOMPARE(asked.size(), 2); + QVERIFY(!SingingTakes::isOverwriteConfirmationWanted()); + } + + // Checklist: pitch and notes outside the recorded range (± about + // 0.25 s) must not flicker or move at all. The pane before and after a + // recording into the middle of a take, compared pixel for pixel + // outside it, and the take's layers the same objects throughout + void nothing_outside_the_range_moves() { + FakeAudioIO::Config config; + config.input = tone(highHz, 8.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + take(3500); + if (QTest::currentTestFailed()) return; + showSeconds(0.0, 3.8); + const sv::sv_frame_t P = frames(1.4); + m_window->seekTo(P); + QImage before = grabPaneRedrawn(); + saveShot("before", before); + sv::Layer *pitch = m_window->analyser2()->getLayer(Analyser::PitchTrack); + sv::Layer *notes = m_window->analyser2()->getLayer(Analyser::Notes); + + QSignalSpy relayered(m_window->analyser2(), SIGNAL(layersChanged())); + m_window->setRecordOverAnswer(true); + take(700); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->analyser2()->getLayer(Analyser::PitchTrack), pitch); + QCOMPARE(m_window->analyser2()->getLayer(Analyser::Notes), notes); + QVERIFY2(relayered.isEmpty(), + "the take's layers were replaced: they would have vanished " + "from the pane for a moment"); + + m_window->seekTo(P); + showSeconds(0.0, 3.8); + QImage after = grabPaneRedrawn(); + saveShot("after", after); + + // What was recorded is at most as long as the wait for it; the + // analysis of it reaches a quarter of a second either side + sv::Pane *pane = pane0(); + int left = pane->getXForFrame(P - frames(0.3)); + int right = pane->getXForFrame(P + frames(0.8) + frames(0.3)); + int pointer = pane->getXForFrame(P); + QVERIFY(left > 20 && right < before.width() - 20); + + // The band of the coverage strip along the bottom is left to + // strip_on_top_after_another_recording + int differing = 0; + QString first; + for (int x = 0; x < before.width(); ++x) { + if (x >= left && x <= right) continue; + if (std::abs(x - pointer) <= 3) continue; + for (int y = 0; y < before.height() - 10; ++y) { + if (before.pixel(x, y) != after.pixel(x, y)) { + if (first == "") { + first = QString("x = %1, y = %2").arg(x).arg(y); + } + ++differing; + } + } + } + QVERIFY2(differing == 0, + qPrintable(QString("%1 pixels outside the recorded range " + "and its margin changed, the first at %2 " + "(range drawn from x = %3 to %4)") + .arg(differing).arg(first).arg(left).arg(right))); + } + + // Checklist: open a .ton, Load Singing Track, then Record: closing + // afterwards asks whether to save. The window's own close, as the + // title bar's button does it + void closing_after_a_take_asks_to_save() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + QString session = m_dir.filePath + (QString("session-%1.ton").arg(++m_fileCounter)); + QVERIFY(m_window->saveSessionFile(session)); + openReference(session); + if (QTest::currentTestFailed()) return; + + m_window->loadSingingTrack(writeWav(tone(highHz, 1.0))); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + m_window->seekTo(frames(1.2)); + take(500); + if (QTest::currentTestFailed()) return; + + QStringList asked; + m_answerDialog = answerWith(QMessageBox::Cancel, false, &asked); + QVERIFY(!m_window->close()); + QCOMPARE(asked, QStringList({ tr("Session modified") })); + QVERIFY2(m_window->isVisible(), "Cancel did not keep the window open"); + QVERIFY(m_window->takes()->haveTake()); + } + + // Checklist: stop a take and close the window at once: no crash. + // Closed and deleted while the take is still being analysed + void stop_then_close_the_window_at_once() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(1000); + m_window->doRecord(); + QVERIFY(m_window->analysingRange()); + + QStringList asked; + m_answerDialog = answerWith(QMessageBox::No, false, &asked); + QVERIFY(m_window->close()); + QCOMPARE(asked, QStringList({ tr("Session modified") })); + delete m_window; + m_window = nullptr; + + // Whatever was still running finishes with no window to go to + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + QTest::qWait(500); + } + + // Checklist: log out with unsaved takes: what commitData writes into + // ~/.sv1 is playable. Only where the home directory can be moved + // for the test: on Windows it is the profile's, whatever HOME says + void commit_data_writes_a_playable_session() { +#ifdef Q_OS_WIN + QSKIP("commitData() writes into the real profile's .sv1 on Windows"); +#else + QByteArray home = qgetenv("HOME"); + QString fakeHome = m_dir.filePath("home"); + QVERIFY(QDir().mkpath(fakeHome)); + qputenv("HOME", fakeHome.toLocal8Bit()); + QCOMPARE(QDir::homePath(), fakeHome); + + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + take(1000); + if (QTest::currentTestFailed()) return; + const Coverage::Ranges recorded = coverage(); + + bool committed = m_window->commitData(false); + qputenv("HOME", home); + QVERIFY(committed); + + QStringList written = QDir(fakeHome + "/.sv1") + .entryList({ "tmp-*" }, QDir::Files); + QCOMPARE(written.size(), 1); + QString path = fakeHome + "/.sv1/" + written[0]; + + // It goes on the Recent Files list, which opens it with openPath() + m_window->doCloseSession(); + MainWindow::FileOpenStatus opened = + m_window->openPath(path, MainWindow::ReplaceSession); + QEXPECT_FAIL("", "commitData() names the file tmp-*.sv, Sonic " + "Visualiser's session extension; Tony opens only .ton " + "as a session", Continue); + QCOMPARE(opened, MainWindow::FileOpenSucceeded); + + // What is in it, under the name Tony would open + m_window->doCloseSession(); + QString ton = QFileInfo(path).path() + "/" + + QFileInfo(path).completeBaseName() + ".ton"; + QVERIFY(QFile::rename(path, ton)); + QCOMPARE(m_window->openPath(ton, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QVERIFY2(m_window->takes()->haveTake(), + "the session written at logout has no take in it"); + QCOMPARE(coverage(), recorded); + + QString audio = m_window->takes()->getAudioPath(); + QVERIFY2(QFileInfo(audio).exists(), + qPrintable("the take's audio is not there: " + audio)); + sv::WavFileReader reader { sv::FileSource(audio) }; + QVERIFY(reader.isOK()); + auto data = reader.getInterleavedFrames + (recorded[0].start + 2000, recorded[0].end - recorded[0].start - 4000); + double sum = 0.0; + for (float v : data) sum += double(v) * double(v); + QVERIFY2(!data.empty() && std::sqrt(sum / double(data.size())) > 0.01, + "the take in the session written at logout is silent"); +#endif + } + + // Checklist: the alternate pitch track is faded brown, dark brown + // during a take, and stays in view after an octave step + void alternate_pitch_track_colours() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + showSeconds(0.0, 3.0); + + QColor faded(164, 146, 136), dark(74, 37, 17); + auto isFaded = [&](QRgb p) { return closeTo(p, faded, 12); }; + auto isDark = [&](QRgb p) { return closeTo(p, dark, 12); }; + + m_window->doToggleAlternatePitch(); + QTRY_VERIFY(m_window->alternatePitch()->getLayer()); + QTest::qWait(200); + QImage image = grabPane(); + QVERIFY2(countAbove(image, 0, image.width(), isFaded) > 0, + "no faded brown track is drawn"); + saveShot("faded", image); + + // Wide enough for the whole of every toolbar, 8vb and 8va included + QSize size = m_window->size(); + m_window->resize(1920, 1000); + saveWindowShot("window"); + m_window->resize(size); + showSeconds(0.0, 3.0); + + // Two octaves either way stay in view of a 220 Hz reference. The + // third is only reported: 28 Hz and 1.8 kHz are past the range the + // pane shows, and nothing scrolls to them + auto inView = [&]() { + QTest::qWait(300); + image = grabPane(); + int octaves = m_window->alternatePitch()->getOctaves(); + saveShot(QString("octaves%1").arg(octaves), image); + return countAbove(image, 0, image.width(), isFaded) > 0; + }; + QCOMPARE(m_window->alternatePitch()->getOctaves(), -1); + for (bool up : { false, true, true, true }) { + m_window->doStepAlternatePitch(up); + QVERIFY2(inView(), + qPrintable(QString("the track is out of view at %1 " + "octaves") + .arg(m_window->alternatePitch()->getOctaves()))); + } + QCOMPARE(m_window->alternatePitch()->getOctaves(), 2); + m_window->doStepAlternatePitch(true); + qInfo("at +3 octaves the track is %s", inView() ? "in view" : "out of view"); + for (int i = 0; i < 5; ++i) m_window->doStepAlternatePitch(false); + QCOMPARE(m_window->alternatePitch()->getOctaves(), -3); + qInfo("at -3 octaves the track is %s", inView() ? "in view" : "out of view"); + m_window->doStepAlternatePitch(true); + m_window->doStepAlternatePitch(true); + QCOMPARE(m_window->alternatePitch()->getOctaves(), -1); + + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(400); + image = grabPane(); + QVERIFY2(countAbove(image, 0, image.width(), isDark) > 0, + "the track followed during a take is not dark brown"); + QVERIFY2(countAbove(image, 0, image.width(), isFaded) == 0, + "the track is still faded during a take"); + saveShot("dark", image); + stopTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(countAbove(grabPane(), 0, image.width(), isFaded) > 0); + } + + // Checklist: pre-roll less than 3 s from the start of the song: a + // shorter countdown, and no attempt to run from before frame 0 + void preroll_near_the_start_of_the_song() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + m_window->setPlayReferenceWhileRecording(true); + m_window->setPreRoll(true); + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(frames(1.0)); + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePreRoll(), frames(1.0)); + QStringList counted; + QElapsedTimer timer; + timer.start(); + while (timer.elapsed() < 1500) { + QVERIFY2(m_window->playbackFrame() >= 0, + qPrintable(QString("the cursor is at frame %1") + .arg(m_window->playbackFrame()))); + QString text = m_window->statusText(); + if (text.startsWith(tr("Recording in ")) && + !counted.contains(text)) { + counted << text; + } + QTest::qWait(20); + } + QCOMPARE(counted, QStringList({ tr("Recording in %1…").arg(1) })); + stopTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(coverage()[0].start, frames(1.0)); + } + + // Checklist: Constrain Playback to Selection together with a pre-roll. + // The device hears its own output here, so the "singer" sings exactly + // what is played: a take that is in time holds the reference's high + // note where the reference has it + void preroll_with_playback_constrained_to_the_selection() { + for (bool constrained : { false, true }) { + FakeAudioIO::Config config; + config.loopback = true; + config.recordLatency = 512; + config.playbackLatency = 512; + config.inputDelay = 1024; + makeWindow(config); + if (QTest::currentTestFailed()) return; + m_window->setPlayReferenceWhileRecording(true); + m_window->setPreRoll(true); + m_window->setRecordIntoSelection(true); + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("prerollseconds", 1.0); + settings.endGroup(); + + // Low, then high from 2 to 2.75 s, then low again + auto reference = tone(lowHz, 2.0); + auto high = tone(highHz, 0.75); + auto low = tone(lowHz, 1.25); + reference.insert(reference.end(), high.begin(), high.end()); + reference.insert(reference.end(), low.begin(), low.end()); + openReference(writeWav(reference)); + if (QTest::currentTestFailed()) return; + + m_window->selectRange(frames(2.0), frames(3.5)); + QAction *constrain = nullptr; + for (QAction *a : m_window->findChildren()) { + if (a->text() == tr("Constrain Playback to Selection")) { + constrain = a; + } + } + QVERIFY(constrain); + if (constrain->isChecked() != constrained) constrain->trigger(); + QCOMPARE(constrain->isChecked(), constrained); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT + (!m_window->recordTarget()->isRecording(), 8000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + + double from = -1.0, to = -1.0; + for (const auto &e : sv::ModelById::getAs + (m_window->analyser2()->getLayer(Analyser::PitchTrack) + ->getModel())->getAllEvents()) { + if (e.getValue() < std::sqrt(lowHz * highHz)) continue; + if (from < 0.0) from = double(e.getFrame()) / rate; + to = double(e.getFrame()) / rate; + } + qInfo("playback %s: the high note is in the take from %.3f to " + "%.3f s", constrained ? "constrained" : "not constrained", + from, to); + if (constrained) { + QEXPECT_FAIL("", "with playback constrained to the " + "selection the lead-in is not played: playback " + "starts at the selection, and what is sung to it " + "is placed a whole pre-roll too early", Continue); + } + QVERIFY2(std::fabs(from - 2.0) < 0.05 && std::fabs(to - 2.75) < 0.05, + qPrintable(QString("the high note of the reference, " + "2.000 to 2.750 s, is in the take " + "from %1 to %2 s") + .arg(from, 0, 'f', 3).arg(to, 0, 'f', 3))); + + if (constrain->isChecked()) constrain->trigger(); + m_window->clearSelections(); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + } + } + + // Checklist: switching between takes is fast and analyses nothing + void switching_takes_is_quick() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 30.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + m_window->doNewEmptyTake(); + m_window->seekTo(frames(2.0)); + take(800); + if (QTest::currentTestFailed()) return; + + for (int to : { 0, 1, 0, 1 }) { + QElapsedTimer timer; + timer.start(); + m_window->doChooseTakeInCombo(to); + QCoreApplication::processEvents(); + grabPane(); + qint64 ms = timer.elapsed(); + qInfo("switch to take %d took %lld ms", to + 1, ms); + QVERIFY2(ms < 1000, + qPrintable(QString("switching takes took %1 ms").arg(ms))); + QCOMPARE(m_window->takes()->getActiveIndex(), to); + QVERIFY(!m_window->analysingRange()); + QVERIFY(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers()); + } + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 6ad3eaf0..08fbec57 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -14,6 +14,7 @@ #include "TestSingingDocument.h" #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" +#include "TestUiChecks.h" #include "RunSuite.h" @@ -70,6 +71,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestUiChecks t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/main/test/tony-device-check.cpp b/main/test/tony-device-check.cpp new file mode 100644 index 00000000..54028756 --- /dev/null +++ b/main/test/tony-device-check.cpp @@ -0,0 +1,59 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +// The checks that need a real audio device and a microphone that can +// hear the speakers. Run by hand, never by meson test: see +// docs/manual-checklist.md + +#include "TestRealDevice.h" + +#include "RunSuite.h" + +#include "system/Init.h" + +#include +#include +#include + +#include + +using namespace std; +using namespace sv; + +int main(int argc, char *argv[]) +{ + svSystemSpecificInitialisation(); + + // Not offscreen by default, unlike the other suites: the window is + // there to be watched while it records + + // Names distinct from the application's: Tony's own settings are only + // read, for the choice of device + QApplication app(argc, argv); + app.setOrganizationName("tony-tests"); + app.setApplicationName("test-tony-device"); + + qputenv("VAMP_PATH", + QDir::toNativeSeparators(app.applicationDirPath()).toLocal8Bit()); + + TestRealDevice t; + bool ok = runSuite(&t, argc, argv); + + if (!ok) { + SVCERR << "\n********* the device check failed\n" << endl; + return 1; + } else { + SVCERR << "The device check passed" << endl; + return 0; + } +} diff --git a/meson.build b/meson.build index 8f8b0e6e..b76bd646 100644 --- a/meson.build +++ b/meson.build @@ -1388,6 +1388,7 @@ tony_app_test_moc_files = qt.preprocess( 'main/test/TestSingingDocument.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', + 'main/test/TestUiChecks.h', ]) tony_app_test_exe = executable( @@ -1414,6 +1415,37 @@ tony_app_test_exe = executable( win_subsystem: 'console' ) +# The checks that need a real audio device, with its microphone hearing +# its speakers. Built with the rest, run only by hand: not a meson test +tony_device_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestRealDevice.h', +]) + +tony_device_test_exe = executable( + 'test-tony-device', + tony_device_test_moc_files, + 'main/test/tony-device-check.cpp', + dependencies: [ + tony_app_dep, + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' +) + test('tony-core', tony_core_test_exe) test('tony-app', tony_app_test_exe, depends: pyin_plugin, From 58e150dca6bca846ad4a1d37b4dce14a3eedd948 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:33:14 +0000 Subject: [PATCH 090/275] docs: building and testing on Linux in a cloud session How Tony was built and both suites run in a cloud container: Ubuntu's packages, but Qt 6.11 from conda-forge, because under Ubuntu's Qt 6.4 the string-based connects of Analyser with sv:: types do not resolve and no analysis ever completes; the libraries cloned at their pins, the hg ones from their GitHub mirrors; and the four TestTakesFile tests that are about Windows paths and fail anywhere else. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/building.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/building.md b/docs/building.md index 3ee5107c..b0e0de1e 100644 --- a/docs/building.md +++ b/docs/building.md @@ -66,6 +66,31 @@ meson setup --wipe build_mingw > tmp/build.log 2>&1 && ninja -j 3 -C build_mingw echo "exit:$?" >> tmp/build.log ``` +## On Linux (a cloud session) + +Not how the project is developed, but it builds and both suites run; this is how it was done +on 2026-09-25 (Ubuntu 24.04, no sound card): + +- Packages: the `apt-get install` list of `.github/workflows/linux.yml` (`smlnj` and + `mercurial` are not needed, and `libboost-dev` does for `libboost-all-dev`), plus + `librubberband-dev`, `libjack-jackd2-dev`, `libasound2-dev`, `libopusenc-dev`, `meson`. +- **Qt 6.11 from conda-forge, not Ubuntu's 6.4.** Under 6.4 the string-based connects of + `Analyser` with `sv::` types do not resolve ("No such slot + Analyser::layerCompletionChanged(ModelId)"), so pYIN's completion never arrives and every + analysing test times out. download.qt.io's mirrors are blocked by the session's proxy; + conda-forge is not: + `micromamba create -p /opt/qt611 -c conda-forge qt6-main=6.11.1`, then a directory with + links to only its `Qt6*.pc` files, so that nothing else of conda's is picked up: + `PKG_CONFIG_PATH= meson setup build_qt611`, and + `LD_LIBRARY_PATH=/opt/qt611/lib` to run. +- The libraries by `git clone` at the pins of `repoint-lock.json`. sourcehut (the `hg` + ones) was unreachable; their GitHub mirrors (`github.com/breakfastquay/...`) are at the + same tips. +- `-j 4` on four cores; the whole build takes about 20 minutes. Run the app suite with + nothing else building: it records in real time. +- Four tests of `TestTakesFile` fail on Linux and nowhere else: they are about Windows + paths (backslashes, drive letters, case). + ## What is particular about this `meson.build` - A MinGW/GCC win64 branch that upstream does not have: msys2 system libraries, From a03b7ecd6ef563cdfb15aa8ac23ab4e6dd79c4c2 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:52:22 +0000 Subject: [PATCH 091/275] feat: calibrate audio verdicts and the round trip they correct judgeTake() runs the sweep finder over a take's punch-ins and says what the audio path did: the median offset across punch-ins, how far they disagree, whether the offset grows with position, whether the input clipped or the sweeps faded, and whether each sweep came back a second time, which is a monitored input. calibratedRoundTrip() adds the median offset to the round trip the take was placed with; a take that lands late was spliced from too early a frame. The sign, the position rule and the echo's minimum delay were each seen to fail their test when broken. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 14 + docs/calibrate-audio.md | 1 + main/LatencyCheck.cpp | 260 ++++++++++++++++- main/LatencyCheck.h | 240 +++++++++++++++- main/test/TestLatencyCheck.h | 424 ++++++++++++++++++++++++++++ 5 files changed, 935 insertions(+), 4 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index bf72950c..37825a9e 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -281,3 +281,17 @@ The next phase must know: - Measured: noise alone 6–12 dB over the median (threshold 15); SNR 0 / −10 dB: 38 / 28 dB over the median, 26 / 16 dB over the second. In digital silence the median is ~0, so over-the-median reads up to the 200 dB clamp. - About 20 ms per call. The second peak's position is computed but not returned; second arrivals (A2) need it. Left open: every threshold untuned; nothing reads `inputPeak` yet. + +### Phase A2 — 2026-09-25 +Built: in `main/LatencyCheck.{h,cpp}`: `Arrival::secondDelaySeconds`, `secondLevelDb`; `judgeTake(layout, take, count, rate, punchIns)` → `TakeSummary {verdict, flags, punchIns[], events[], judged, found, medianOffset, spread, slope, slopeResidual, inputPeak, fadingDb, echo}`; `PunchIn {start, end}` in timeline seconds; `Verdict`, declared in precedence order (NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok); `verdictName()`; `calibratedRoundTrip(used, offset)` = used + offset. 9 tests in `TestLatencyCheck` itself (reusing its helpers); the class now takes 2.9 s. +Choices / deviations: +- Judged: the finder's window, plus a sweep's length past it, inside the range with 50 ms to spare (the splice crossfades 5 ms). An event under a later, overlapping punch-in is judged in that one only. Ranges stop at the take's end. +- Across = median of the punch-ins' medians (each stream start counts once); spread = their max − min. Unsteady/Scattered take the larger of that and any spread within one punch-in. PositionDependent: 3 punch-ins at least (a line through two always fits), |slope| > 0.5 %, and what the least-squares line leaves ≤ 5 ms. NoSignal also when nothing is found, or nothing judged. +- Fading reads `levelDb`, not a confidence: over the median of near silence a confidence runs up to the 200 dB clamp. Median of the first half of the judged events (as recorded) minus that of the second ≥ 10 dB; needs 6 events. +- Echo is judged over events *heard* (≥ 15 dB over the median), not found: an echo within 6 dB leaves nothing found. Added `kEchoMinDelaySeconds` = 20 ms: a reflection 9 ms after the direct sound and 5.5 dB stronger leaves the tail of its peak at 10.1 ms, 9–23 dB down, after every sweep, and was reported as an echo. Also ≤ 30 dB down, within 3 ms of the median delay, in more than half of the heard events and 3 at least. +- Clipped: the largest sample inside the ranges ≥ −0.2 dBFS. +The next phase must know: +- **At 48 kHz, §2's punch-ins at 14 and 20 s land 1.14 and 1.63 s early, beyond the finder's 0.8 s reach.** Measured: the finder takes the neighbouring sweeps, fully confident (+662, +575 ms), and the verdict is Scattered. PositionDependent needs punch-ins that start before about 9.8 s, so §6's "a 48 kHz fake reports PositionDependent" fails with §2's punch-ins. The rates themselves can name the rate. +- §2's punch-ins as [2,7], [8,13], [14,19], [20,25] s judge 7 events (2, 2, 2, 1). A punch-in shorter than 1.95 s judges none. +- An echo under 20 ms (an interface's direct monitor) is not seen. +Left open: every threshold untuned. The verdict thresholds came from the lead's brief; spec §5 has none of them. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 47c891eb..51f9057f 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -273,6 +273,7 @@ marked "Done" when it is committed. - **A1** Test reference and sweep finder: `LatencyCheck` generator and per-event analysis. Done. - **A2** Verdicts and calibration arithmetic: aggregation over events and punch-ins. + Done. 2. **Runner, dialog and calibration page** (every build), with app tests. - **B1** The alignment check runner and its app tests. - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp index 308849fc..f07e3289 100644 --- a/main/LatencyCheck.cpp +++ b/main/LatencyCheck.cpp @@ -312,9 +312,11 @@ LatencyCheck::findSweep(const float *take, sv_frame_t count, // another candidate for where the sweep is const int closeBy = int(framesAt(kSecondPeakSeconds, rate)); double second = 0.0; + int secondAt = chosen; for (int j = 0; j < lags; ++j) { - if (std::abs(j - chosen) > closeBy) { - second = std::max(second, envelope[j]); + if (std::abs(j - chosen) > closeBy && envelope[j] > second) { + second = envelope[j]; + secondAt = j; } } @@ -327,9 +329,263 @@ LatencyCheck::findSweep(const float *take, sv_frame_t count, arrival.peakOverMedianDb = decibels(envelope[chosen], median); arrival.peakOverSecondDb = decibels(envelope[chosen], second); arrival.levelDb = decibels(envelope[chosen], energy); + arrival.secondDelaySeconds = double(secondAt - chosen) / rate; + arrival.secondLevelDb = decibels(second, envelope[chosen]); arrival.found = arrival.peakOverMedianDb >= kMinPeakOverMedianDb && arrival.peakOverSecondDb >= kMinPeakOverSecondDb; return arrival; } + +namespace LatencyCheck { +namespace { + +double +median(vector v) +{ + if (v.empty()) return 0.0; + std::sort(v.begin(), v.end()); + const size_t n = v.size(); + return (n % 2) ? v[n/2] : 0.5 * (v[n/2 - 1] + v[n/2]); +} + +double +spreadOf(const vector &v) +{ + if (v.empty()) return 0.0; + auto range = std::minmax_element(v.begin(), v.end()); + return *range.second - *range.first; +} + +// The echo, if more than half of the heard sweeps have a second peak +// after them at one delay. When those that agree are more than half +// of the candidates, the median of the candidates' delays is among +// theirs, so looking around it finds them +Echo +echoOf(const vector &events) +{ + Echo echo; + + int heard = 0; + vector candidates; + for (const EventResult &e : events) { + const Arrival &a = e.arrival; + if (a.peakOverMedianDb < kMinPeakOverMedianDb) continue; + ++heard; + if (a.secondDelaySeconds >= kEchoMinDelaySeconds && + a.secondLevelDb >= -kEchoMaxBelowDb) { + candidates.push_back(&a); + } + } + if (candidates.empty()) return echo; + + vector delays; + for (const Arrival *a : candidates) delays.push_back(a->secondDelaySeconds); + const double centre = median(delays); + + vector agreeing, levels; + for (const Arrival *a : candidates) { + if (std::fabs(a->secondDelaySeconds - centre) <= kEchoToleranceSeconds) { + agreeing.push_back(a->secondDelaySeconds); + levels.push_back(a->secondLevelDb); + } + } + + const int count = int(agreeing.size()); + if (count >= kMinEchoEvents && 2 * count > heard) { + echo.heard = true; + echo.delaySeconds = median(agreeing); + echo.levelDb = median(levels); + echo.events = count; + } + return echo; +} + +} // namespace +} // namespace LatencyCheck + +const char * +LatencyCheck::verdictName(Verdict verdict) +{ + switch (verdict) { + case Verdict::NoSignal: return "NoSignal"; + case Verdict::Clipped: return "Clipped"; + case Verdict::Fading: return "Fading"; + case Verdict::PositionDependent: return "PositionDependent"; + case Verdict::Scattered: return "Scattered"; + case Verdict::Unsteady: return "Unsteady"; + case Verdict::Ok: return "Ok"; + } + return "Ok"; +} + +bool +LatencyCheck::TakeSummary::flagged(Verdict v) const +{ + return std::find(flags.begin(), flags.end(), v) != flags.end(); +} + +LatencyCheck::TakeSummary +LatencyCheck::judgeTake(const Layout &layout, + const float *take, sv_frame_t count, + sv_samplerate_t rate, + const vector &punchIns) +{ + TakeSummary summary; + + const double takeSeconds = + (take && count > 0 && rate > 0) ? double(count) / rate : 0.0; + + // All that the finder reads for an event, around its expected time + const double before = kSearchSeconds + kJudgeMarginSeconds; + const double after = kSearchSeconds + kSweepSeconds + kJudgeMarginSeconds; + + for (int p = 0; p < int(punchIns.size()); ++p) { + + PunchInResult result; + result.range = punchIns[p]; + const double start = punchIns[p].start; + const double end = std::min(punchIns[p].end, takeSeconds); + + const sv_frame_t from = std::max(sv_frame_t(0), framesAt(start, rate)); + const sv_frame_t to = std::min(count, framesAt(end, rate)); + for (sv_frame_t f = from; f < to; ++f) { + summary.inputPeak = std::max(summary.inputPeak, + double(std::fabs(take[f]))); + } + + vector offsets; + for (int i = 0; i < int(layout.events.size()); ++i) { + + const double expected = + double(layout.events[i].sweepStart) / layout.rate; + const double first = expected - before; + const double last = expected + after; + if (first < start || last > end) continue; + + // A later punch-in over any of it replaced what this one + // recorded there + bool replaced = false; + for (int q = p + 1; q < int(punchIns.size()); ++q) { + if (first < punchIns[q].end && last > punchIns[q].start) { + replaced = true; + } + } + if (replaced) continue; + + EventResult e; + e.event = i; + e.punchIn = p; + e.expectedSeconds = expected; + e.arrival = findSweep(take, count, rate, expected); + + ++result.judged; + if (e.arrival.found) { + ++result.found; + offsets.push_back(e.arrival.errorSeconds); + } + summary.events.push_back(e); + } + + result.medianOffset = median(offsets); + result.spread = spreadOf(offsets); + summary.judged += result.judged; + summary.found += result.found; + summary.punchIns.push_back(result); + } + + // Across punch-ins: each one that found anything counts once, at + // its median, since what moves from one to the next is the stream + // start, which every punch-in makes once + vector positions, medians; + double within = 0.0; + for (const PunchInResult &r : summary.punchIns) { + if (r.found == 0) continue; + positions.push_back(r.range.start); + medians.push_back(r.medianOffset); + within = std::max(within, r.spread); + } + summary.medianOffset = median(medians); + summary.spread = spreadOf(medians); + + // Least squares. A device at another rate, placed frame for frame, + // misplaces a punch-in in proportion to where it starts + summary.slopeResidual = summary.spread; + if (positions.size() >= 2) { + double mx = 0.0, my = 0.0; + for (size_t k = 0; k < positions.size(); ++k) { + mx += positions[k]; + my += medians[k]; + } + mx /= double(positions.size()); + my /= double(positions.size()); + double sxx = 0.0, sxy = 0.0; + for (size_t k = 0; k < positions.size(); ++k) { + sxx += (positions[k] - mx) * (positions[k] - mx); + sxy += (positions[k] - mx) * (medians[k] - my); + } + if (sxx > 0.0) { + summary.slope = sxy / sxx; + vector residuals; + for (size_t k = 0; k < positions.size(); ++k) { + residuals.push_back(medians[k] - + (my + summary.slope * (positions[k] - mx))); + } + summary.slopeResidual = spreadOf(residuals); + } + } + + // The level rather than a confidence: over the median of a window + // of near silence a confidence is anything up to the 200 dB clamp, + // so its trend in a clean take is noise, while the level is what + // an echo canceller, noise suppressor or gain control changes. + // Events not found count too, at the level of whatever the finder + // took instead, which is lower when the sweep has gone + const int n = int(summary.events.size()); + if (n >= kMinFadingEvents) { + vector early, late; + for (int k = 0; k < n / 2; ++k) { + early.push_back(summary.events[k].arrival.levelDb); + } + for (int k = n - n / 2; k < n; ++k) { + late.push_back(summary.events[k].arrival.levelDb); + } + summary.fadingDb = median(early) - median(late); + } + + summary.echo = echoOf(summary.events); + + const double spread = std::max(summary.spread, within); + + if (summary.found == 0 || + double(summary.found) / double(summary.judged) < kMinFoundShare) { + summary.flags.push_back(Verdict::NoSignal); + } + if (summary.inputPeak >= ratioOf(kClippedDbfs)) { + summary.flags.push_back(Verdict::Clipped); + } + if (summary.fadingDb >= kFadingDb) { + summary.flags.push_back(Verdict::Fading); + } + if (int(positions.size()) >= kMinPunchInsForSlope && + std::fabs(summary.slope) > kPositionSlope && + std::max(summary.slopeResidual, within) <= kSteadySeconds) { + summary.flags.push_back(Verdict::PositionDependent); + } + if (spread > kScatteredSeconds) { + summary.flags.push_back(Verdict::Scattered); + } else if (spread > kSteadySeconds) { + summary.flags.push_back(Verdict::Unsteady); + } + + summary.verdict = summary.flags.empty() ? Verdict::Ok : summary.flags.front(); + return summary; +} + +double +LatencyCheck::calibratedRoundTrip(double usedRoundTripSeconds, + double medianOffsetSeconds) +{ + return usedRoundTripSeconds + medianOffsetSeconds; +} diff --git a/main/LatencyCheck.h b/main/LatencyCheck.h index 3b1cc263..7ca43790 100644 --- a/main/LatencyCheck.h +++ b/main/LatencyCheck.h @@ -32,7 +32,9 @@ * * Pure functions over sample buffers: no model, no window and no * device, so all of it is tested without them (TestLatencyCheck). - * Judging a take from its events is a later step. + * judgeTake() gathers the finder's results over a take's punch-ins + * into a verdict, and calibratedRoundTrip() turns the offset it + * measured into the round trip to use instead. */ namespace LatencyCheck { @@ -187,9 +189,19 @@ namespace LatencyCheck /// full scale being 1 double inputPeak; + /// The second peak, which peakOverSecondDb measures against: + /// how long after the chosen peak it comes (negative if + /// before), and its level against the chosen peak's, in dB. + /// A monitoring echo is a second peak at the same delay after + /// every sweep. With nothing more than kSecondPeakSeconds from + /// the chosen peak, the delay is 0 + double secondDelaySeconds; + double secondLevelDb; + Arrival() : found(false), errorFrames(0), errorSeconds(0), peakOverMedianDb(0), peakOverSecondDb(0), - levelDb(0), inputPeak(0) { } + levelDb(0), inputPeak(0), + secondDelaySeconds(0), secondLevelDb(0) { } }; /** @@ -201,6 +213,230 @@ namespace LatencyCheck */ Arrival findSweep(const float *take, sv::sv_frame_t count, sv::sv_samplerate_t rate, double expectedSeconds); + + /** + * Judging a take. An offset is where a sweep was found minus + * where the reference has it (Arrival::errorSeconds): positive is + * late. Starting values like the finder's, to be tuned from the + * reports of real runs. + */ + + /// An event is judged in a punch-in only when all that the finder + /// reads for it (its window, and a sweep's length past the window's + /// end) lies inside the punch-in's range and this far from its + /// ends. The splice cuts the punch-in's audio at the range ends and + /// crossfades it over 5 ms into what was there, so a sweep near an + /// end may be cut, which says nothing about the audio path + constexpr double kJudgeMarginSeconds = 0.05; + + /// NoSignal: fewer than this share of the judged events were found + constexpr double kMinFoundShare = 2.0 / 3.0; + + /// Clipped: the input reached this close to full scale somewhere + /// in the punch-ins. Not at full scale exactly: a device's integer + /// samples, converted, stop a little short of it + constexpr double kClippedDbfs = -0.2; + + /// Unsteady: the offsets found disagree by more than this, across + /// punch-ins or within one ... + constexpr double kSteadySeconds = 0.005; + + /// ... Scattered: by more than this + constexpr double kScatteredSeconds = 0.015; + + /// PositionDependent: the punch-ins' offsets lie on a line over + /// their positions, steeper than this (0.5 %: a 44.1 kHz reference + /// recorded at 48 kHz and placed frame for frame is 8 %), and + /// leave less than kSteadySeconds about it ... + constexpr double kPositionSlope = 0.005; + + /// ... fitted over this many punch-ins at least, since a line + /// through two always fits + constexpr int kMinPunchInsForSlope = 3; + + /// Fading: the sweeps' level (Arrival::levelDb) over the judged + /// events, in the order they were recorded, fell by this much from + /// the first half to the second, each half taken by its median. + /// Through a steady path every sweep arrives at the same level, + /// within 0.5 dB even in noise at -10 dB SNR. An echo canceller + /// or noise suppressor taking the sweeps out brings them down + /// toward the noise, and gain control by whatever it corrects; 10 + /// dB leaves room for an earcup that shifts a little + constexpr double kFadingDb = 10.0; + + /// ... judged only from this many events on, so that each half has + /// three and one missing event cannot move its median + constexpr int kMinFadingEvents = 6; + + /** + * A monitoring echo: the take's own input played back out and heard + * again. It is the second peak (Arrival::secondDelaySeconds), at + * the same delay within kEchoToleranceSeconds, in more than half of + * the events whose sweep was heard, and at least kMinEchoEvents of + * them. Heard, not found: an echo within kMinPeakOverSecondDb of + * the direct sound leaves no sweep found, and should still be named. + * + * Other things come at a fixed delay after every sweep too, so an + * echo has to be at least kEchoMinDelaySeconds late and at most + * kEchoMaxBelowDb down. A reflection a little closer than + * kSecondPeakSeconds leaves the tail of its peak just past that, 9 + * to 23 dB down (measured for one 5.5 dB stronger than the direct + * sound, 9 ms after it, from a 1-2 kHz path to the full band), + * which is why the delay. The matched filter's own tail is over + * 80 dB down, and noise peaks come as loud as 26 dB down at 0 dB + * SNR but never agree on a delay. + * + * The Windows audio engine works in 10 ms periods, so what it + * monitors comes back two periods late at least: one to capture + * it and one to play it. An echo closer than that, such as an + * interface's direct monitor, is not seen, nor one quieter than + * the tail of a close reflection. + */ + constexpr double kEchoMinDelaySeconds = 0.02; + constexpr double kEchoMaxBelowDb = 30.0; + constexpr double kEchoToleranceSeconds = 0.003; + constexpr int kMinEchoEvents = 3; + + /** + * What a take says about the audio path. In this order of + * precedence: when several apply, the first is the verdict, and + * names what to fix first. Without the sweeps nothing else can be + * judged; a clipped or processed input can move where they are + * found; and a rate that misplaces punch-ins also scatters them. + */ + enum class Verdict { + NoSignal, ///< too few sweeps found + Clipped, ///< the input reached full scale + Fading, ///< the sweeps' level fell over the run + PositionDependent, ///< offset grows with position: a rate + ///< other than the reference's + Scattered, ///< offsets disagree by more than 15 ms + Unsteady, ///< offsets disagree by 5 to 15 ms + Ok + }; + + /// The verdict's name as the enum spells it, for logs + const char *verdictName(Verdict verdict); + + /// A range of the timeline that one punch-in recorded, in seconds + struct PunchIn { + double start; + double end; + + PunchIn() : start(0), end(0) { } + PunchIn(double s, double e) : start(s), end(e) { } + }; + + /// One judged event: which of the layout's, in which punch-in, and + /// what the finder made of it + struct EventResult { + int event; + int punchIn; + double expectedSeconds; + Arrival arrival; + + EventResult() : event(0), punchIn(0), expectedSeconds(0) { } + }; + + /// One punch-in's figures. The offsets are those of its found + /// events, in seconds; with none found, both are 0 + struct PunchInResult { + PunchIn range; + int judged; + int found; + double medianOffset; + double spread; ///< largest offset minus smallest + + PunchInResult() : judged(0), found(0), medianOffset(0), spread(0) { } + }; + + /// A monitoring echo, if one was heard; see kEchoMaxBelowDb + struct Echo { + bool heard; + double delaySeconds; ///< after the direct sound, median + double levelDb; ///< against the direct sound, median + int events; ///< how many events it was heard in + + Echo() : heard(false), delaySeconds(0), levelDb(0), events(0) { } + }; + + /// What judgeTake() made of a take. Offsets in seconds + struct TakeSummary { + Verdict verdict; + + /// Every verdict that applied, in order of precedence; empty + /// for Ok + std::vector flags; + + std::vector punchIns; ///< as given + std::vector events; ///< judged, as recorded + + int judged; + int found; + + /// Across punch-ins, over those with an event found: the + /// median of their median offsets, which weighs every stream + /// start alike, and the largest minus the smallest of them + double medianOffset; + double spread; + + /// The line fitted to those medians over the punch-ins' start + /// times: seconds of offset per second of position, and what + /// it leaves, as largest minus smallest. 0 and the spread + /// itself below two punch-ins + double slope; + double slopeResidual; + + /// The largest sample in the punch-ins' ranges, full scale 1 + double inputPeak; + + /// How far the sweeps' level fell, first half to second, in + /// dB; 0 below kMinFadingEvents judged events + double fadingDb; + + Echo echo; + + bool flagged(Verdict v) const; + + TakeSummary() : verdict(Verdict::NoSignal), judged(0), found(0), + medianOffset(0), spread(0), slope(0), + slopeResidual(0), inputPeak(0), fadingDb(0) { } + }; + + /** + * Judge a take recorded against the reference made from this + * layout. + * + * The take is one channel of samples, as its model gives them, at + * the take's own rate, which need not be the layout's. The punch- + * ins are the timeline ranges they recorded, in seconds, in the + * order they were recorded; where a later one overlaps an earlier + * one, the take holds the later one's audio there. So an event is + * judged in a punch-in that holds all the finder reads for it + * (kJudgeMarginSeconds) and where no later one overlaps that, and + * in no other. Ranges are cut short at the end of the take. + * + * NoSignal applies when fewer than kMinFoundShare of the judged + * events were found, and also when none was, judged or not: there + * is then nothing to measure. Unsteady and Scattered look at the + * spread across punch-ins and within each, whichever is larger. + */ + TakeSummary judgeTake(const Layout &layout, + const float *take, sv::sv_frame_t count, + sv::sv_samplerate_t rate, + const std::vector &punchIns); + + /** + * The round trip that would have placed the take right, in seconds: + * the one it was placed with plus the median offset measured. + * + * The offset is found minus expected, so positive means the take + * landed late: the latency used was too small, the take was spliced + * from too early a frame of the recording, and the round trip has + * to grow. + */ + double calibratedRoundTrip(double usedRoundTripSeconds, + double medianOffsetSeconds); } #endif diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h index b26bdd98..583208f0 100644 --- a/main/test/TestLatencyCheck.h +++ b/main/test/TestLatencyCheck.h @@ -143,6 +143,83 @@ class TestLatencyCheck : public QObject .toUtf8(); } + typedef std::vector punchins_t; + + // The take as the splice would make it: inside each punch-in's + // range, the reference as it arrived that time, this many frames + // late (early, if negative); silence outside them. A later punch-in + // replaces an earlier one where they overlap + static samples_t spliced(const samples_t &reference, double rate, + const punchins_t &punchIns, + const std::vector &shifts) { + const frame_t n = frame_t(reference.size()); + samples_t take(reference.size(), 0.f); + for (size_t p = 0; p < punchIns.size(); ++p) { + const frame_t from = std::max(frame_t(0), + framesOf(punchIns[p].start, rate)); + const frame_t to = std::min(n, framesOf(punchIns[p].end, rate)); + for (frame_t f = from; f < to; ++f) { + const frame_t g = f - shifts[p]; + take[f] = (g >= 0 && g < n) ? reference[g] : 0.f; + } + } + return take; + } + + // Four punch-ins at about 2, 8, 14 and 20 s, as a calibration makes + // them. In the calibration layout they judge two events each, and + // one in the last: 3.1, 4.7 / 9.1, 11.4 / 15.7, 17.7 / 21.9 s + static punchins_t fourPunchIns() { + return { { 2.0, 7.0 }, { 8.0, 13.0 }, { 14.0, 19.0 }, { 20.0, 25.0 } }; + } + + // Room noise at about -65 dBFS RMS, far under the sweeps + static samples_t withRoomNoise(const samples_t &x) { + return mixed(x, TestSignals::whiteNoise(int(x.size()), 4711, 0.001), + 1.0); + } + + // Scale one event's sweep, as it lies in a take placed this late + static void scaleSweep(samples_t &take, const LatencyCheck::Layout &layout, + int event, frame_t shift, double gain) { + const frame_t from = layout.events[event].sweepStart + shift; + const frame_t length = framesOf(LatencyCheck::kSweepSeconds, layout.rate); + for (frame_t f = from; f < from + length; ++f) { + take[f] = float(take[f] * gain); + } + } + + static LatencyCheck::TakeSummary judge(const LatencyCheck::Layout &layout, + const samples_t &take, double rate, + const punchins_t &punchIns) { + return LatencyCheck::judgeTake(layout, take.data(), + frame_t(take.size()), rate, punchIns); + } + + static QByteArray describe(const LatencyCheck::TakeSummary &s) { + QStringList flags; + for (LatencyCheck::Verdict v : s.flags) { + flags << LatencyCheck::verdictName(v); + } + return QString("%1 [%2], found %3 of %4, median %5 ms, spread %6 " + "ms, slope %7 %, residual %8 ms, input peak %9, " + "fading %10 dB, echo %11 at %12 ms %13 dB") + .arg(LatencyCheck::verdictName(s.verdict)) + .arg(flags.join(" ")) + .arg(s.found) + .arg(s.judged) + .arg(s.medianOffset * 1000.0) + .arg(s.spread * 1000.0) + .arg(s.slope * 100.0) + .arg(s.slopeResidual * 1000.0) + .arg(s.inputPeak) + .arg(s.fadingDb) + .arg(s.echo.heard ? "heard" : "none") + .arg(s.echo.delaySeconds * 1000.0) + .arg(s.echo.levelDb) + .toUtf8(); + } + private slots: // The same layout and the same samples every time: nothing in the // reference depends on a clock or a random source @@ -559,6 +636,353 @@ private slots: QVERIFY(!find(take, kRate, -1.0).found); QVERIFY(!find(samples_t(), kRate, 0.1).found); } + + // Judging a take. Four punch-ins all placed the same, 12345 frames + // late: every judged event found at that offset, in the order the + // punch-ins were recorded, and nothing else to say + void judge_a_steady_take() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const frame_t shift = 12345; + const punchins_t punchIns = fourPunchIns(); + const samples_t take = spliced(LatencyCheck::generate(layout), kRate, + punchIns, { shift, shift, shift, shift }); + + const LatencyCheck::TakeSummary s = judge(layout, take, kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, describe(s).constData()); + QVERIFY2(s.flags.empty(), describe(s).constData()); + QCOMPARE(s.judged, 7); + QCOMPARE(s.found, 7); + QVERIFY2(std::fabs(s.medianOffset - shift / kRate) < 1e-9, + describe(s).constData()); + QVERIFY2(s.spread < 1e-9 && std::fabs(s.slope) < 1e-9 && + s.slopeResidual < 1e-9, describe(s).constData()); + QVERIFY2(std::fabs(s.inputPeak - ratioOf(LatencyCheck::kPeakDbfs)) < 1e-5, + describe(s).constData()); + QVERIFY2(std::fabs(s.fadingDb) < 0.01, describe(s).constData()); + QVERIFY2(!s.echo.heard, describe(s).constData()); + + QCOMPARE(int(s.punchIns.size()), 4); + const int judgedIn[] = { 2, 2, 2, 1 }; + for (int p = 0; p < 4; ++p) { + QCOMPARE(s.punchIns[p].range.start, punchIns[p].start); + QCOMPARE(s.punchIns[p].judged, judgedIn[p]); + QCOMPARE(s.punchIns[p].found, judgedIn[p]); + QVERIFY(std::fabs(s.punchIns[p].medianOffset - shift / kRate) < 1e-9); + QVERIFY(s.punchIns[p].spread < 1e-9); + } + + const int events[] = { 1, 2, 4, 5, 7, 8, 10 }; + const int in[] = { 0, 0, 1, 1, 2, 2, 3 }; + QCOMPARE(int(s.events.size()), 7); + for (int k = 0; k < 7; ++k) { + QCOMPARE(s.events[k].event, events[k]); + QCOMPARE(s.events[k].punchIn, in[k]); + QCOMPARE(s.events[k].expectedSeconds, expectedAt(layout, events[k])); + QCOMPARE(s.events[k].arrival.errorFrames, shift); + } + } + + // A sweep lost, in room noise. Found out of judged is what counts: + // one or two of six lost still measures, three is too many, and + // silence is nothing at all. The first sweep lost is in the second + // half of the run, and one of three there does not make it Fading + void judge_a_take_with_events_missing() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const frame_t shift = 12345; + const punchins_t punchIns = { { 2.0, 7.0 }, { 8.0, 13.0 }, { 14.0, 19.0 } }; + samples_t take = spliced(LatencyCheck::generate(layout), kRate, + punchIns, { shift, shift, shift }); + + // Events 1, 2 / 4, 5 / 7, 8 judged + scaleSweep(take, layout, 5, shift, 0.0); + LatencyCheck::TakeSummary s = + judge(layout, withRoomNoise(take), kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, describe(s).constData()); + QCOMPARE(s.judged, 6); + QCOMPARE(s.found, 5); + QCOMPARE(s.punchIns[1].found, 1); + QVERIFY(!s.events[3].arrival.found); + QCOMPARE(s.events[3].event, 5); + QVERIFY2(std::abs(s.medianOffset * kRate - shift) < 1.0, + describe(s).constData()); + + // Exactly two thirds found + scaleSweep(take, layout, 1, shift, 0.0); + s = judge(layout, withRoomNoise(take), kRate, punchIns); + QCOMPARE(s.found, 4); + QVERIFY2(!s.flagged(LatencyCheck::Verdict::NoSignal), describe(s).constData()); + + scaleSweep(take, layout, 7, shift, 0.0); + s = judge(layout, withRoomNoise(take), kRate, punchIns); + QCOMPARE(s.found, 3); + QVERIFY2(s.verdict == LatencyCheck::Verdict::NoSignal, describe(s).constData()); + + s = judge(layout, withRoomNoise(samples_t(take.size(), 0.f)), kRate, + punchIns); + QCOMPARE(s.judged, 6); + QCOMPARE(s.found, 0); + QVERIFY2(s.verdict == LatencyCheck::Verdict::NoSignal, describe(s).constData()); + QCOMPARE(s.medianOffset, 0.0); + } + + // Something in the input path taking the sweeps out as the run goes + // on: 5 dB quieter at every event, down to -30 dB, all still found + void judge_a_fading_take() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const frame_t shift = 12345; + const punchins_t punchIns = fourPunchIns(); + samples_t take = spliced(LatencyCheck::generate(layout), kRate, + punchIns, { shift, shift, shift, shift }); + const int events[] = { 1, 2, 4, 5, 7, 8, 10 }; + for (int k = 0; k < 7; ++k) { + scaleSweep(take, layout, events[k], shift, ratioOf(-5.0 * k)); + } + + const LatencyCheck::TakeSummary s = judge(layout, take, kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Fading, describe(s).constData()); + QCOMPARE(int(s.flags.size()), 1); + QCOMPARE(s.found, 7); + // -5 dB against -25, the halves' medians + QVERIFY2(std::fabs(s.fadingDb - 20.0) < 0.1, describe(s).constData()); + QVERIFY2(std::fabs(s.medianOffset - shift / kRate) < 1e-9, + describe(s).constData()); + } + + // Too loud for the input: clipped at full scale. Found where they + // are all the same, but the verdict is the level. Just under it + // (-1.1 dBFS) is fine + void judge_a_clipped_take() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const frame_t shift = 12345; + const punchins_t punchIns = fourPunchIns(); + const samples_t take = spliced(LatencyCheck::generate(layout), kRate, + punchIns, { shift, shift, shift, shift }); + + samples_t loud(take), clipped(take); + for (float &v : loud) v *= 3.5f; + for (float &v : clipped) v = std::max(-1.f, std::min(1.f, v * 8.f)); + + LatencyCheck::TakeSummary s = judge(layout, clipped, kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Clipped, describe(s).constData()); + QCOMPARE(s.inputPeak, 1.0); + QCOMPARE(s.found, 7); + QVERIFY2(std::abs(s.medianOffset * kRate - shift) < 1.0, + describe(s).constData()); + + s = judge(layout, loud, kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, describe(s).constData()); + QVERIFY2(s.inputPeak < ratioOf(-1.0), describe(s).constData()); + } + + // A device at 48 kHz, its take placed frame for frame on the 44.1 + // kHz timeline: each punch-in lands early by P (1 - 44100/48000), + // P being where it starts, and the take is read at 48 kHz. On a + // line, so PositionDependent (and Scattered, since it is). The same + // punch-ins at offsets whose line is as steep but leaves 30 ms + // about it: Scattered. The punch-ins stay under 10 s, where the + // misplacement is still within the finder's reach + void judge_position_dependent_against_scattered() { + const double deviceRate = 48000.0; + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t recorded = + LatencyCheck::generate(LatencyCheck::calibrationLayout(deviceRate)); + + // Events 0, 1 and 3, one each + const punchins_t punchIns = { { 0.1, 2.1 }, { 2.2, 4.2 }, { 6.3, 8.3 } }; + std::vector shifts; + for (const LatencyCheck::PunchIn &p : punchIns) { + shifts.push_back(-framesOf(p.start * (1.0 - kRate / deviceRate), + deviceRate)); + } + + LatencyCheck::TakeSummary s = + judge(layout, spliced(recorded, deviceRate, punchIns, shifts), + deviceRate, punchIns); + QCOMPARE(s.found, 3); + QVERIFY2(s.verdict == LatencyCheck::Verdict::PositionDependent, + describe(s).constData()); + QVERIFY2(s.flagged(LatencyCheck::Verdict::Scattered), describe(s).constData()); + QVERIFY2(std::fabs(s.slope + (1.0 - kRate / deviceRate)) < 1e-4, + describe(s).constData()); + QVERIFY2(s.slopeResidual < 0.0001, describe(s).constData()); + + // Slope about -0.7 %, residuals +12, -18 and +6 ms + const double offsets[] = { 0.0, -0.045, -0.050 }; + for (int p = 0; p < 3; ++p) shifts[p] = framesOf(offsets[p], deviceRate); + s = judge(layout, spliced(recorded, deviceRate, punchIns, shifts), + deviceRate, punchIns); + QCOMPARE(s.found, 3); + QVERIFY2(std::fabs(s.slope) > LatencyCheck::kPositionSlope, + describe(s).constData()); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Scattered, + describe(s).constData()); + QVERIFY2(!s.flagged(LatencyCheck::Verdict::PositionDependent), + describe(s).constData()); + } + + // Two stream starts that placed their punch-ins differently. Two + // punch-ins always lie on a line, so however steep (here 20 ms over + // 2.1 s), it is never PositionDependent + void judge_two_punch_ins_apart() { + const LatencyCheck::Layout layout = firstEvents(3); + const samples_t reference = LatencyCheck::generate(layout); + const punchins_t punchIns = { { 0.1, 2.1 }, { 2.2, 4.2 } }; + const frame_t shift = framesOf(0.1); + + struct { double apart; LatencyCheck::Verdict verdict; } cases[] = { + { 0.020, LatencyCheck::Verdict::Scattered }, + { 0.010, LatencyCheck::Verdict::Unsteady }, + { 0.003, LatencyCheck::Verdict::Ok }, + }; + for (const auto &c : cases) { + const frame_t other = shift + framesOf(c.apart); + const LatencyCheck::TakeSummary s = + judge(layout, spliced(reference, kRate, punchIns, { shift, other }), + kRate, punchIns); + QCOMPARE(s.found, 2); + QVERIFY2(s.verdict == c.verdict, describe(s).constData()); + QCOMPARE(int(s.flags.size()), c.verdict == LatencyCheck::Verdict::Ok ? 0 : 1); + QVERIFY2(std::fabs(s.spread - c.apart) < 2.0 / kRate, + describe(s).constData()); + QVERIFY2(std::fabs(s.medianOffset - (shift + other) / 2.0 / kRate) + < 1e-9, describe(s).constData()); + } + } + + // The take's own input played back out and heard again, 40 ms + // later and 12 dB down: named, and the take still measures. At 3 + // dB down it hides the sweeps from the finder, and is still named. + // Noise at 0 dB SNR has second peaks as loud, at no one delay. A + // reflection 9 ms late and 5.5 dB stronger leaves a second peak + // at 10.1 ms, 23 dB down, after every sweep: too early for an echo + void judge_hears_a_monitoring_echo() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const frame_t shift = 12345; + const punchins_t punchIns = fourPunchIns(); + const samples_t take = spliced(LatencyCheck::generate(layout), kRate, + punchIns, { shift, shift, shift, shift }); + const frame_t delay = framesOf(0.040); + + LatencyCheck::TakeSummary s = + judge(layout, mixed(take, shifted(take, delay), ratioOf(-12.0)), + kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, describe(s).constData()); + QVERIFY2(s.echo.heard, describe(s).constData()); + QCOMPARE(s.echo.events, 7); + QVERIFY2(std::fabs(s.echo.delaySeconds - delay / kRate) < 1e-9, + describe(s).constData()); + QVERIFY2(std::fabs(s.echo.levelDb + 12.0) < 0.5, describe(s).constData()); + + s = judge(layout, mixed(take, shifted(take, delay), ratioOf(-3.0)), + kRate, punchIns); + QCOMPARE(s.found, 0); + QVERIFY2(s.verdict == LatencyCheck::Verdict::NoSignal, describe(s).constData()); + QVERIFY2(s.echo.heard, describe(s).constData()); + QVERIFY2(std::fabs(s.echo.levelDb + 3.0) < 0.5, describe(s).constData()); + + const double sweepRms = rms(LatencyCheck::sweep(kRate)); + const samples_t noisy = + mixed(take, TestSignals::whiteNoise(int(take.size()), 20260925, + sweepRms * std::sqrt(3.0)), 1.0); + s = judge(layout, noisy, kRate, punchIns); + QCOMPARE(s.found, 7); + for (const LatencyCheck::EventResult &e : s.events) { + QVERIFY(e.arrival.secondLevelDb > -LatencyCheck::kEchoMaxBelowDb); + } + QVERIFY2(!s.echo.heard, describe(s).constData()); + + s = judge(layout, mixed(take, shifted(take, framesOf(0.009)), + ratioOf(5.5)), + kRate, punchIns); + QCOMPARE(s.found, 7); + QVERIFY2(std::fabs(s.medianOffset - shift / kRate) < 1e-9, + describe(s).constData()); + for (const LatencyCheck::EventResult &e : s.events) { + QVERIFY(e.arrival.secondLevelDb > -LatencyCheck::kEchoMaxBelowDb); + } + QVERIFY2(!s.echo.heard, describe(s).constData()); + } + + // What the finder reads for an event has to lie inside the punch-in, + // kJudgeMarginSeconds from its ends; otherwise the event is not + // judged, since a sweep cut by the splice is no fault of the path. + // Where a later punch-in overlaps, the event is judged in that one + void judge_only_events_inside_a_punch_in() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t reference = LatencyCheck::generate(layout); + const frame_t shift = framesOf(0.1); + const double before = + LatencyCheck::kSearchSeconds + LatencyCheck::kJudgeMarginSeconds; + const double after = LatencyCheck::kSearchSeconds + + LatencyCheck::kSweepSeconds + LatencyCheck::kJudgeMarginSeconds; + + // The sweep at 3.1 s arrives at 3.2 s, where the range ends + punchins_t punchIns = { { 2.0, 3.2 } }; + LatencyCheck::TakeSummary s = + judge(layout, spliced(reference, kRate, punchIns, { shift }), + kRate, punchIns); + QCOMPARE(s.judged, 0); + QVERIFY2(s.verdict == LatencyCheck::Verdict::NoSignal, describe(s).constData()); + + // Just inside the margin at either end, then just outside it + const double t = expectedAt(layout, 4); + for (double by : { 0.001, -0.001 }) { + const bool inside = by > 0.0; + for (const LatencyCheck::PunchIn &p : + { LatencyCheck::PunchIn(t - before - by, t + after + 0.1), + LatencyCheck::PunchIn(t - before - 0.1, t + after + by) }) { + punchIns = { p }; + s = judge(layout, spliced(reference, kRate, punchIns, { shift }), + kRate, punchIns); + QCOMPARE(s.judged, inside ? 1 : 0); + QCOMPARE(s.found, inside ? 1 : 0); + } + } + + // 9.1 s in the first punch-in only; 11.4 s in both, so in the + // second, which recorded over the first + const frame_t later = framesOf(0.2); + punchIns = { { 8.0, 13.0 }, { 10.5, 13.0 } }; + s = judge(layout, spliced(reference, kRate, punchIns, { shift, later }), + kRate, punchIns); + QCOMPARE(s.judged, 2); + QCOMPARE(s.punchIns[0].judged, 1); + QCOMPARE(s.punchIns[1].judged, 1); + QCOMPARE(s.events[0].event, 4); + QCOMPARE(s.events[1].event, 5); + QCOMPARE(s.events[1].punchIn, 1); + QCOMPARE(s.events[1].arrival.errorFrames, later); + } + + // The calibration's arithmetic. A take that landed late was placed + // with too small a round trip, and the new one is larger by the + // offset; early, smaller. From takes placed with 0.2 and 0.31 s + // through a path whose round trip is 0.25 s + void calibration_adds_the_offset() { + QVERIFY(std::fabs(LatencyCheck::calibratedRoundTrip(0.2, 0.03) - 0.23) + < 1e-12); + QVERIFY(std::fabs(LatencyCheck::calibratedRoundTrip(0.2, -0.05) - 0.15) + < 1e-12); + + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t reference = LatencyCheck::generate(layout); + const punchins_t punchIns = { { 2.0, 7.0 } }; + const double roundTrip = 0.25; + + for (double used : { 0.2, 0.31 }) { + const frame_t late = framesOf(roundTrip - used); + const LatencyCheck::TakeSummary s = + judge(layout, spliced(reference, kRate, punchIns, { late }), + kRate, punchIns); + QCOMPARE(s.found, 2); + QVERIFY2((s.medianOffset > 0.0) == (used < roundTrip), + describe(s).constData()); + const double measured = + LatencyCheck::calibratedRoundTrip(used, s.medianOffset); + QVERIFY2(std::fabs(measured - roundTrip) < 1e-9, + describe(s).constData()); + } + } }; #endif From b73adaa39b6627149168936757c02a9718360456 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:53:48 +0000 Subject: [PATCH 092/275] docs: calibrate audio, B1 refined; punch-ins from the layout, rate named directly A2 found that a 48 kHz take runs off the finder's reach from about 10 s in, and that 5 s punch-ins judge one or two sweeps. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 63 ++++++++++++++++++++++++++++- docs/calibrate-audio.md | 9 ++++- 2 files changed, 68 insertions(+), 4 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 37825a9e..14137634 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`). +Done: A1 (`944df7c`), A2 (`a03b7ec`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -212,7 +212,66 @@ Done: A1 (`944df7c`). ### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) -To be refined by the lead after A2. +Read also: `docs/recording.md` whole (154 lines), `docs/architecture.md` sections on +layers, models and commands (search the headings), `docs/testing.md` "What there is to +reuse". In `MainWindow.cpp`, read by range: `record()`, the deferred lambda in +`recordingStarted()`, `pollTakeProgress()`, `finishSingingTake()` and +`wantedPreRollFrames()`. + +- **Pure helper first**, in `LatencyCheck`, with a core test: + - `punchInsFor(layout, count, eventsEach)` returns `count` consecutive punch-in ranges + in seconds. Each holds `eventsEach` events that `judgeTake()` will judge, using its + margins. + - The calibration uses 4 × 3 on the calibration layout. Tests use a short layout of + 2 × 2 (≈ 8 s) to keep real time down. +- **New `main/AudioCheckRunner.{h,cpp}`** in `tony_app_files`. A QObject owned and wired + by `MainWindow`. It is driven by signals and a polling timer, like + `pollTakeProgress()`, **not by nested event loops**: it runs in every build, and the + user can close the window at any moment. +- **Steps:** + 1. Write the reference WAV to the app data directory, overwriting the old one. Mono, + 44.1 kHz. + 2. `checkSaveModified()`, then `openPath(path, ReplaceSession)`. Wait for the + reference's analysis, as `openReference()` in the tests does. + 3. For each punch-in: select its range and call `record()`; the take stops itself. + Wait for the take's analysis before the next punch-in. It is not strictly needed, + but it keeps pYIN's CPU load out of the next take's timing. + 4. Read the take's audio: the model `analyser2()->getMainModelId()`, mixed to mono, + at its own rate. Call `judgeTake()`. +- **Record into Selection, Play Reference While Recording and a 1 s pre-roll** apply to + the check's own takes through an override in `MainWindow`. `record()`, + `recordingStarted()` and `wantedPreRollFrames()` consult it. **Never through + `setChecked()`**: those actions write QSettings. +- **What each take used.** Keep, per take, the round trip it used: today + `computeRecordingLatency(out, in)`, which B2 will change. Also keep the reported + output and input latency, and the **recording's sample rate**. +- **The result** carries: + - the `TakeSummary`; + - the round trip used, and the reported pair; + - the recording's rate and the reference's; + - a **rate-mismatch flag**, set when the two rates differ, whatever the sweeps say + (A2's finding: a 48 kHz take runs off the finder's reach); + - the calibrated round trip from `calibratedRoundTrip()`, meaningful only on Ok or + Unsteady. + + Emitted as a signal when done. Failures (no device, recording refused, the session + closed mid-run) end the run with a reason. +- **Cancel** stops a take in progress through the normal Stop path and clears the + override. `closeSession()` and `~MainWindow()` cancel a running check. +- **Not in this phase:** storing or using the result (B2), any dialog or menu entry (B3). +- **App tests.** Choose between a new class and adding to `TestRecordWorkflow`; say + which. `TestMainWindow` and the fixtures live in `TestRecordWorkflow.h`. Use + `FakeAudioIO` `loopback = true` and the short layout. + - Wrong reported latencies (e.g. 2·4096 and 4096), `inputDelay` = 3·4096 + 123. The + median offset equals the difference, in seconds at 44.1 kHz, to within a few + frames. The verdict is Ok. The calibrated round trip equals `inputDelay`. + - Fake at 48 kHz: the rate mismatch is flagged and both rates are given. Assert only + what the runner reports, and that nothing crashes. What Tony does with 48 kHz takes + is a separate, known bug. + - After a check, the three toggles and their QSettings values are as before. + - Cancel during a take leaves no recording in progress and the override cleared. + - **Show failure** for the first test with the override's Play Reference half + removed: nothing is heard, so NoSignal. ### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 51f9057f..26d440ff 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -40,7 +40,9 @@ with less noise. 2. **Test session.** Tony writes a generated test reference and opens it the way File ▸ Open does. You are asked to save your work first. The run never touches your song's takes or undo history. -3. **Calibration, about 30 s.** Four punch-ins at about 2, 8, 14 and 20 s, using the +3. **Calibration, about 30 s.** Four punch-ins, each spanning three of the reference's + sweeps with the room the finder needs around them. The ranges come from the layout, + not from fixed times: a 5 s punch-in judges only one or two sweeps. They use the ordinary take path with Play Reference While Recording on. Every punch-in restarts the stream, as a real take does. 4. **Result page:** @@ -247,7 +249,10 @@ setting changes only through Use this latency. - **Calibration.** Wrong reported latencies are measured right. With the stored figure, `latency_end_to_end`'s recipe lines up; without it, the same test fails. A stale key falls back to the reported sum. The user's three toggles and their - settings are untouched. A 48 kHz fake reports PositionDependent and names the rate. + settings are untouched. A 48 kHz fake is reported as a rate mismatch, naming + both rates. This comes from comparing the recording's rate with the reference's, + not from the sweeps: from about 10 s into the reference, a 48 kHz take lands further + off than the finder searches, and the sweeps then read as Scattered (found in A2). - **Every dev check at least twice.** Once passing on a calibrated fake, and once **failing** under an injected fault: - an uncalibrated offset, for items 1, 2, 7 and 13; From 7ece0721fce0fefbe4144986c264642047b67ff7 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 23:57:49 +0000 Subject: [PATCH 093/275] feat: highlight the word being sung, fade the waveform under the lyrics The word at the playback position is drawn in amber: MainWindow passes every frame the view manager reports to LyricsTrack, and so to the layer, while playing, while recording and on a seek with playback stopped. It does so before the pre-roll countdown can return, as the reference plays during a lead-in; an import and a session load light the word at the cursor at once. The lyrics are drawn over the waveforms at the bottom of the pane, so while they are on show both analysers draw theirs in a new Pale Grey instead of Grey. Analyser::setWaveformFaded() sets the colour on the layer and remembers it for a waveform made or taken over later; it writes no setting, unlike setVisible(), and marks nothing modified. MainWindow applies it after anything that shows or hides the lyrics, after every new singing analyser, after a session load whether or not it had lyrics (the colour is saved with the layer), and on close. The docs describe the lyrics as they are now drawn, and one more test that races the analysis on Linux joins the list of those expected to fail there. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- README.md | 14 +- docs/architecture.md | 35 ++++- docs/forks.md | 32 ++-- docs/manual-checklist.md | 51 ++++-- docs/open-points.md | 20 ++- docs/testing.md | 7 +- main/Analyser.cpp | 35 ++++- main/Analyser.h | 19 +++ main/LyricsTrack.cpp | 16 +- main/LyricsTrack.h | 13 +- main/MainWindow.cpp | 43 ++++++ main/MainWindow.h | 13 +- main/test/TestRecordWorkflow.h | 275 ++++++++++++++++++++++++++++++++- 13 files changed, 513 insertions(+), 60 deletions(-) diff --git a/README.md b/README.md index a7792582..8768e0c7 100644 --- a/README.md +++ b/README.md @@ -48,12 +48,14 @@ orange pitch track to compare with it. between them needs no re-analysis. The audio of a session's takes is kept in a folder named after the session beside it, so the two can be moved together * timed lyrics: File -> Import Lyrics... reads an LRC file, timed by line or - by word, and shows the words along the top of the pane, each over a bar for - as long as it is sung. They are saved with the session; View -> Show Lyrics - hides them and File -> Remove Lyrics takes them out. The reference must be - the recording the lyrics were timed to (a Moises stem and its original mix - share a timeline); to change a word or its time, edit the file and import it - again + by word, and shows the words in boxes along the bottom of the pane, at the + time and for as long as each is sung. The word at the playback position is + highlighted, while playing, while recording and wherever you click, and the + waveform is faded while the words are on show so that they can be read over + it. They are saved with the session; View -> Show Lyrics hides them and + File -> Remove Lyrics takes them out. The reference must be the recording + the lyrics were timed to (a Moises stem and its original mix share a + timeline); to change a word or its time, edit the file and import it again * LRC files can be exported from Moises with the Moises-Lyric-Exporter browser extension. Set its offset to 0 (otherwise every line is 0.2 s early, and Tony cannot tell) and its gap threshold as low as it goes (so that it marks diff --git a/docs/architecture.md b/docs/architecture.md index a9b6c6e7..7e0820ec 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -15,8 +15,8 @@ Upstream Tony analyses the pitch of one recording. This fork makes it a singing the rest, erase, undo, and keep several takes. See [takes.md](takes.md). 5. Around that: play the reference while recording, latency compensation, pre-roll, record into selection, an octave-shifted "alternate" pitch track to follow, timed - lyrics along the top of the pane, and a background music track that is played but - never analysed. + lyrics along the bottom of the pane with the word being sung highlighted, and a + background music track that is played but never analysed. The user-facing description is in the [README](../README.md). @@ -67,7 +67,8 @@ The reference is the pane's **work model** (`Pane::setWorkModel()`, svgui fork, take's audio file, or of the recording being written. Colours: reference pitch black / notes bright blue; singing and live dots orange / notes -bright purple; alternate pitch faded brown, dark brown while a take is recorded. +bright purple; alternate pitch faded brown, dark brown while a take is recorded; both +waveforms grey, pale grey while lyrics are on show over them. ## Rules of the SV libraries @@ -208,8 +209,9 @@ is the only reference to the model until `m_analyser2` has a layer of its own. I in pane 0 on the reference's timeline, a region per word (or per line, for a line with no word times): frame = start, duration = end - start (at least one frame), label = the word, value = the line's index, which is what the bold line starts go by. It is drawn by the -svgui fork's `PlotLyrics` style ([forks.md](forks.md)), because a session restores only -layers `LayerFactory` can make. Found again after a session load by its untranslated object +svgui fork's `PlotLyrics` style ([forks.md](forks.md)), in boxes along the bottom of the +pane just above the coverage strip, because a session restores only layers +`LayerFactory` can make. Found again after a session load by its untranslated object name `"Lyrics"`, in `analyseNewMainModel()` after the alternate pitch track. Its model is taken out of the play source after an import and again after a load: a word past the end of the reference would hold playback open. It is **never the pane's top layer**, because @@ -225,3 +227,26 @@ Music: the simpler option, and the file is still there to import again. Show Lyr visibility, which the session saves, not a QSettings key. The parser strips control characters (and U+FFFE, U+FFFF, which the UTF-8 decoder lets through) from every label: XML 1.0 cannot hold them, and one in a label would make the `.ton` unreadable. + +The word being sung is highlighted. `MainWindow::playbackFrameChanged()` passes every +frame the view manager reports to `LyricsTrack::setPlaybackFrame()`, which passes it to +the layer's `setHighlightFrame()`: while playing, while recording (when the frame runs with +the reference from where the take starts), and on a seek with playback stopped, since +`ViewManager::setPlaybackFrame()` emits whenever the frame changes. It does so **before** +`showTakeCountdown()` can return, or the highlight would stand still through a pre-roll's +lead-in while the reference plays. An import and an adopt pass the current frame at once. +The layer repaints only when the word changes; the highlight is not saved and makes no +command. + +While the lyrics are shown and visible, both analysers' waveforms are drawn "Pale Grey" +instead of "Grey" so that the words can be read over them: `Analyser::setWaveformFaded()`, +applied to both by `MainWindow::updateWaveformFade()`. The colour is set straight on the +layer, **never through `setVisible()` or anything else that writes QSettings**, and marks +nothing modified. The analyser remembers the fade for a waveform it makes or takes over, +but a new analyser starts without it, and a session saves the colour with the reference's +waveform layer. So `updateWaveformFade()` runs after an import, a remove and Show Lyrics; +after `setupSingingTrackAnalyser()`, which every recording, take switch, Load Singing +Track and session load goes through with a new waveform; in `analyseNewMainModel()` +whether or not lyrics were adopted, because the analyser took the saved layer over before +the lyrics were looked for; and in `closeSession()`, because `m_analyser` lives on for the +next file. diff --git a/docs/forks.md b/docs/forks.md index 0b99deed..09bff04e 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -73,16 +73,28 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` - `RegionLayer::PlotStrip` plot style: the coverage strip. Saved through the existing `plotStyle` attribute. - `RegionLayer::PlotLyrics` plot style, after `PlotStrip` so saved numbers keep their - meaning: the lyrics. Each region's label along the top of the view over a bar as long as - the region, in two rows, the first word of a line (where the value changes) in bold; no - vertical scale, no feature description, not editable. The static, pure - `assignLabelRows()` places the labels (Tony's app suite tests it): a label that fits in - no row is left out. Where a label goes depends on the labels before it, so the layout is - made for the **whole model** at once and cached per zoom level and font; a strip newly - scrolled into sight then agrees with what is already on show. That is what lets the - layer stay **scrollable**: `View::getNonScrollableFrontLayers()` treats every layer in - front of a non-scrollable one as non-scrollable too, so the pitch tracks above the - lyrics would repaint on every cursor update. + meaning: the lyrics. Each region's label is centred in a light box, dark text whatever + the view's colours, in two rows along the bottom of the view just above `PlotStrip`'s + 8 px (row 0 lowest), with a bar in the base colour under each region. A box spans the + region, or the label centred on it where the label is longer (`getLyricsBoxSpan()`). The + first word of a line (where the value changes) is bold. The font + (`getLyricsFontPixelSize()`) is twice the view's at the least, grows with the zoom up to + four times, and is never more than an eighth of the view's height. No vertical scale, no + feature description, not editable. The static, pure `assignLabelRows()` places the boxes + (Tony's app suite tests it): one that fits in no row is left out. + `setHighlightFrame()` draws the region at that frame in amber (the latest to start, where + regions overlap) and emits `layerParametersChanged()` only when that region changes: the + highlight is painted into the view's cache, so each new word repaints the view, a few + times a second at most, and only views listen to that signal, so nothing is marked + modified. `getHighlightedEvent()` says which region it is. A highlighted word that was + left out is drawn in row 0 over the others for as long as it is highlighted: at the + usual zoom the larger font leaves many words out, and the one being sung is the one the + singer must be able to read. Where a label goes depends on the labels before it, so the + layout is made for the **whole model** at once and cached per zoom level and font; a + strip newly scrolled into sight then agrees with what is already on show. That is what + lets the layer stay **scrollable**: `View::getNonScrollableFrontLayers()` treats every + layer in front of a non-scrollable one as non-scrollable too, so the pitch tracks above + the lyrics would repaint on every cursor update. - `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the notes of a take that is put away. - `Pane::setWorkModel()` / `getWorkModel()`: which model's extents are blocked off at the diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 35b7ee7d..20e75837 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -107,33 +107,52 @@ Launch with `.\build.bat run`. ## Lyrics -37. **Legibility**: the words are readable over the reference and singing pitch tracks, - the alternate pitch track and the live dots, and the pitch shows through between them. - Grey bars: right colour? -38. **Band height and font** at the zoom used while singing: can the words be read while - singing, and does the band hide too much of the top of the pitch range? +37. **Legibility**: the words, dark on light boxes along the bottom of the pane, are + readable over the waveform, the pitch tracks, the alternate pitch track and the live + dots low in the range. The waveform is pale grey (225, 225, 225) while the lyrics are + on show: faint enough for the words, still enough to see where the singing is? Grey + bars under the boxes: right colour? +38. **Rows and font** at the zoom used while singing: can the words be read while + singing, and do the two rows hide too much of the bottom of the pitch range? The font + grows as you zoom in (twice the usual size up to four times, never more than an eighth + of the pane's height): right size at each zoom, words centred in their boxes? 39. **Density**: zoom out until words drop out (they are left out, never drawn over each - other) and back in (they return). At the usual zoom on a fast song, how many drop out: - are two rows enough? + other, except the highlighted one) and back in (they return). At the usual zoom the + larger font leaves many words out: how many on a fast song, and are two rows enough? 40. **Bold line starts**: do they read as the start of a phrase, or as noise? -41. **The left edge**: a word in the first ~30 px of the view (at 0 s, with the view at the - start) is under the pane's vertical scale. How much does that matter in use? +41. **The left edge**: a word in the first ~30 px of the view is under the pane's vertical + scale, at the bottom left (scroll so that a word is at the left edge). How much does + that matter in use? 42. **Inferred ends**: the exporter writes no end times. With word timing the last word of a line ends at the next line but at most 2 s after it starts, unless a `♪` line marks the end; with line timing a line lasts until the next one, and the last line 5 s. Do - those bars mislead? The start times are exact. + those boxes and bars mislead, and does the last word of a line stay highlighted too + long? The start times are exact. 43. **Hover readout**: with the lyrics shown, hovering over the pitch tracks gives the same readout and the same vertical scale as without them, also after turning the alternate pitch track off and after deleting a take. 44. **A real Moises export** of one of your songs (exporter offset 0, gap threshold low), imported onto the Moises stem or the original mix: the words line up with the vocal, by - eye and while playing. All early or late by the same amount means the reference is not - the recording Moises timed; an `[offset:]` line in the file moves them. + eye and while playing, and the highlight moves with the voice. All early or late by the + same amount means the reference is not the recording Moises timed; an `[offset:]` line + in the file moves them. 45. **Finnish text**: ä and ö come out right in the pane, and again after save and reopen. 46. **Show Lyrics and Remove Lyrics**: hiding keeps the words for later, Remove takes them out, a second import replaces the first; each makes Close ask whether to save. The - status bar after an import counts words and lines and names anything skipped. -47. **Session**: save and reopen: the same words, hidden or shown as saved; playback still - ends at the end of the song, even with words past it. + waveform is pale while the words are on show and grey again when they are hidden or + removed. The status bar after an import counts words and lines and names anything + skipped. +47. **Session**: save and reopen: the same words, hidden or shown as saved, and the + waveform pale or grey to match; playback still ends at the end of the song, even with + words past it. 48. **During a take**: the words stay on show and readable while recording, with the - countdown and the live dots; Import Lyrics is greyed out while recording. + countdown and the live dots, and the take's waveform is pale as well; Import Lyrics is + greyed out while recording. +49. **The highlight while playing**: the word being sung turns amber as the reference + reaches it and light again when it ends, in step with the voice, without flicker or + visible lag. With playback stopped, a click or seek into a word highlights it at once, + and one into a gap highlights nothing. Is amber readable, and distinct enough? +50. **The highlight while recording**: with a pre-roll, the highlight runs with the + reference through the lead-in, while the countdown is on the status bar, and on + through the take from its position; after Stop it is back on the word at the take's + position. The same with Play Reference While Recording off. diff --git a/docs/open-points.md b/docs/open-points.md index 39befd18..04820e4a 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -29,9 +29,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. all off by the same amount because the reference is not the recording they were timed to. Import and Remove are not undoable, and a new import replaces the lyrics without asking. -- **No highlight of the word being sung**: the playback cursor crosses the words. A - highlight would tie the lyrics layer's painting to the play position, which defeats the - view's paint cache. - **LRC only**: no SRT, TTML or Moises JSON. The exporter's TTML carries real word (and syllable) end times where its LRC has none, so it is the natural second format if the inferred ends turn out misleading; another format is another function beside @@ -53,11 +50,20 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - `Coverage::regionLabel()` gives every region a blank label, for a stock `RegionLayer` that printed the value otherwise. With `PlotStrip` it is no longer needed. - **A word in the first ~30 px of the view is hidden** under the pane's vertical scale, - which the pane draws over the left edge for its top layer. Seen with a word at 0 s and - the view at the start. + which the pane draws over the left edge for its top layer: at the bottom left now, as it + was at the top. Seen by rendering the window with the view scrolled so that a word sat + at the left edge: only its last letters showed. With the view at the start, that zoom + (about 86 px/s) showed 1.6 s before 0 s, so a word at 0 s was clear of the scale, but + the half of its box before 0 s was under the pale wash the pane draws before the start + of the reference. - **The lyrics' layout is a guess at what reads well**: two rows (a word with no room is - left out), the bold line starts, and the 2 s / 5 s caps on inferred ends are all - constants to be judged by eye ([manual checklist](manual-checklist.md)). + left out), the bold line starts, the font size, and the 2 s / 5 s caps on inferred ends + are all constants to be judged by eye ([manual checklist](manual-checklist.md)). +- **At the usual zoom many words have no room**: with the font twice the view's at the + least, most labels are wider than the time their word takes on screen, so the boxes + overlap their neighbours and two rows do not hold them all (seen at 100 px/s). Only the + word being sung is always drawn, over the others if need be. Zooming in, a smaller font + or a third row would each help. - **Right after `closeSession()`, Show Lyrics and the alternate pitch actions keep their enabled and checked states** until the next reference or session opens: nothing there calls `updateLayerStatuses()`. Show Lyrics then does nothing when chosen. diff --git a/docs/testing.md b/docs/testing.md index 2b81e9b9..01a31aa3 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -52,9 +52,10 @@ Windows path would start an escape in the C string. `in_folder`, which test Windows paths (`C:\...`, case-insensitive). App, all `TestRecordWorkflow`: `stale_pitch_event_ignored`, whose string-based `invokeMethod` with an `sv::` type Qt 6.4 cannot match; and `take_analysis_covers_the_range_it_lost`, - `range_analysis_torn_down_while_running`, `save_during_ranged_analysis` and - `undo_during_analysis_then_redo`, where the analysis finishes before the race they need - can be set up. Which of those four fail changes from run to run. + `range_analysis_torn_down_while_running`, `save_during_ranged_analysis`, + `undo_during_analysis_then_redo` and `analyse_now_reanalyses_the_take`, where the + analysis finishes before the race they need can be set up. Which of those five fail + changes from run to run. ## Design principles diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 9b8cd77e..3b60b609 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -68,7 +68,8 @@ Analyser::Analyser(ColorScheme colorScheme) : m_rangedEnd(0), m_rangedMergeStart(0), m_rangedMergeEnd(0), - m_rangedClippedEnd(false) + m_rangedClippedEnd(false), + m_waveformFaded(false) { QSettings settings; settings.beginGroup("LayerDefaults"); @@ -490,6 +491,9 @@ Analyser::addWaveform() if (existing && existing->getModel() == m_fileModel) { cerr << "recording existing waveform layer (matching our file model)" << endl; m_layers[Audio] = existing; + // A session saves the colour with the layer, faded or not + // as it was then; it is to be what it is now + existing->setBaseColour(getWaveformColour()); return ""; } } @@ -518,8 +522,7 @@ Analyser::addWaveform() waveform->setMiddleLineHeight(0.9); waveform->setShowMeans(false); // too small & pale for this - waveform->setBaseColour - (ColourDatabase::getInstance()->getColourIndex(tr("Grey"))); + waveform->setBaseColour(getWaveformColour()); auto params = waveform->getPlayParameters(); if (params) { params->setPlayPan(-1); @@ -532,6 +535,32 @@ Analyser::addWaveform() return ""; } +int +Analyser::getWaveformColour() const +{ + ColourDatabase *cdb = ColourDatabase::getInstance(); + int colour = -1; + if (m_waveformFaded) colour = cdb->getColourIndex(tr("Pale Grey")); + // MainWindow names the colours; without that one, grey will do + if (colour < 0) colour = cdb->getColourIndex(tr("Grey")); + return colour; +} + +void +Analyser::setWaveformFaded(bool faded) +{ + m_waveformFaded = faded; + + // Straight on the layer, which only repaints: no command, no + // modified flag, and not saveState(), which is for the user's own + // choices and would write this to the settings both analysers share + if (Layer *audio = m_layers[Audio]) { + if (auto waveform = qobject_cast(audio)) { + waveform->setBaseColour(getWaveformColour()); + } + } +} + QString Analyser::buildAnalysisTransforms(Transforms &transforms) { diff --git a/main/Analyser.h b/main/Analyser.h index 2f90469c..2e1fbd28 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -122,6 +122,18 @@ class Analyser : public QObject, void setAudible(Component c, bool v); void toggleAudible(Component c) { setAudible(c, !isAudible(c)); } + /** + * Draw the waveform paler than usual, for while something is drawn + * over it (the lyrics), or back in its usual grey. Remembered, so + * that a waveform this analyser makes or takes over later is drawn + * the same way. Unlike setVisible() and setAudible() this is not a + * setting: nothing is written to QSettings, and nothing is marked + * modified. A session saves the colour with the layer, so whoever + * calls this has to call it again after a load. + */ + void setWaveformFaded(bool faded); + bool isWaveformFaded() const { return m_waveformFaded; } + void cycleStatus(Component c) { if (isVisible(c)) { if (isAudible(c)) { @@ -405,12 +417,19 @@ protected slots: TakeEvents::Change m_rangedPitchChange; TakeEvents::Change m_rangedNotesChange; + // See setWaveformFaded() + bool m_waveformFaded; + QString doAllAnalyses(bool withPitchTrack); QString addVisualisations(); QString addWaveform(); QString addAnalyses(); + // The colour the waveform is to be drawn in now: see + // setWaveformFaded() + int getWaveformColour() const; + // The colours and play parameters of the pitch and notes layers, // whichever way they were made void configureAnalysisLayers(); diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index 5ff862b5..3f84e16d 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -156,15 +156,15 @@ LyricsTrack::configureLayer() m_layer->setObjectName(layerName()); - // Words along the top of the pane: a plot style the svgui fork has - // for this. It has no vertical scale and takes no edits. + // Words along the bottom of the pane: a plot style the svgui fork + // has for this. It has no vertical scale and takes no edits. // EqualSpaced as well, so that nothing in the pane can align its // scale to this layer m_layer->setVerticalScale(RegionLayer::EqualSpaced); m_layer->setPlotStyle(RegionLayer::PlotLyrics); - // The bar under each word; the words themselves are in the view's - // own colours. The layer's default would be black + // The bar under each word; the words themselves are dark on light + // boxes whatever the colours. The layer's default would be black m_layer->setBaseColour (ColourDatabase::getInstance()->getColourIndex(tr("Grey"))); @@ -208,6 +208,14 @@ LyricsTrack::isVisible() const return m_layer && m_pane && !m_layer->isLayerDormant(m_pane); } +void +LyricsTrack::setPlaybackFrame(sv_frame_t frame) +{ + // Hidden or not: shown again, the lyrics have the right word lit + // already, even with playback stopped + if (m_layer) m_layer->setHighlightFrame(frame); +} + void LyricsTrack::keepUnderTop() { diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index 9d73c077..e39ff4fa 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -31,8 +31,9 @@ class RegionLayer; /** * The timed lyrics of the session: a RegionLayer in pane 0 with one - * region per word (lyricsToEvents()), drawn as words along the top of - * the pane by the lyrics plot style of the svgui fork. + * region per word (lyricsToEvents()), drawn as words in boxes along the + * bottom of the pane by the lyrics plot style of the svgui fork, with + * the word at the playback position highlighted. * * The lyrics belong to the song, not to a take: one set per session, * which only an import replaces. The layer and its model are ordinary @@ -86,6 +87,14 @@ class LyricsTrack : public QObject void setVisible(bool visible); bool isVisible() const; + /** + * The playback position, or the recording position during a take: + * the word there is the one highlighted. The layer repaints only + * when that is another word, so this can be called for every frame + * the view manager reports. Nothing is saved or marked modified. + */ + void setPlaybackFrame(sv::sv_frame_t frame); + sv::RegionLayer *getLayer() const { return m_layer; } /// The model the words are in diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 795d2f11..1cd4f01b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -247,6 +247,9 @@ MainWindow::MainWindow(AudioMode audioMode, cdb->setUseDarkBackground(cdb->addColour(Qt::green, tr("Bright Green")), true); cdb->setUseDarkBackground(cdb->addColour(QColor(225, 74, 255), tr("Bright Purple")), true); cdb->setUseDarkBackground(cdb->addColour(QColor(255, 188, 80), tr("Bright Orange")), true); + // The waveforms under the lyrics (Analyser::setWaveformFaded()). + // Last, so that the colours before it keep their indices + cdb->addColour(QColor(225, 225, 225), tr("Pale Grey")); Preferences::getInstance()->setResampleOnLoad(true); Preferences::getInstance()->setFixedSampleRate(44100); @@ -2591,6 +2594,9 @@ MainWindow::closeSession() m_alternatePitch->hide(); m_coverageStrip->hide(); m_lyrics->hide(); + // The fade goes with the lyrics. m_analyser stays for the next file, + // and would make that one's waveform faded as well + updateWaveformFade(); m_referencePitchHiddenForTake = false; m_singingPitchHiddenForTake = false; m_singingNotesHiddenForTake = false; @@ -3454,6 +3460,11 @@ MainWindow::importLyricsFrom(QString path) m_playSource->removeModel(m_lyrics->getModelId()); } + // The word at the cursor, without waiting for playback to move it; + // and the waveforms fade under the words + m_lyrics->setPlaybackFrame(m_viewManager->getPlaybackFrame()); + updateWaveformFade(); + // The layer arrived without a command, as it must, and an import is // not undoable; but the session has changed documentModified(); @@ -3486,6 +3497,7 @@ MainWindow::removeLyrics() { if (!m_lyrics->isShown()) return; m_lyrics->hide(); + updateWaveformFade(); // As for the import: no command, but the session has changed documentModified(); updateMenuStates(); @@ -3502,9 +3514,22 @@ MainWindow::showLyricsToggled() m_lyrics->setVisible(!m_lyrics->isVisible()); documentModified(); } + updateWaveformFade(); updateLayerStatuses(); } +void +MainWindow::updateWaveformFade() +{ + // The words are drawn over the bottom of the pane, where the + // waveforms are, and have to be read over them. Not with + // Analyser::setVisible() or anything else that writes a setting: + // this is the state of the session, which saves the colour + bool faded = m_lyrics && m_lyrics->isShown() && m_lyrics->isVisible(); + if (m_analyser) m_analyser->setWaveformFaded(faded); + if (m_analyser2) m_analyser2->setWaveformFaded(faded); +} + void MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnalysis) { @@ -3567,6 +3592,10 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal audio->setSavedInSession(false); } + // A new analyser, with a new waveform, under the lyrics as much as + // the one it replaces: a take switch and every recording come here + updateWaveformFade(); + // m_analyser2->newFileLoaded() has now created its own WaveformLayer // referencing singingModelId. This means it is safe to delete the orphan // WaveformLayer that MainWindowBase::record() put in the extra pane: @@ -4302,6 +4331,11 @@ MainWindow::recordDurationChanged(sv_frame_t frame, sv_samplerate_t rate) void MainWindow::playbackFrameChanged(sv_frame_t frame) { + // The word being sung, before the countdown can return: the reference + // plays during a lead-in, and the words go with it. This comes while + // playing, while recording, and for a seek with playback stopped + if (m_lyrics) m_lyrics->setPlaybackFrame(frame); + if (showTakeCountdown()) return; MainWindowBase::playbackFrameChanged(frame); } @@ -7600,10 +7634,19 @@ MainWindow::analyseNewMainModel() if (m_playSource && !m_lyrics->getModelId().isNone()) { m_playSource->removeModel(m_lyrics->getModelId()); } + // The word at the cursor, without waiting for playback to move it + m_lyrics->setPlaybackFrame(m_viewManager->getPlaybackFrame()); // Remove Lyrics; Show Lyrics is set by updateLayerStatuses() below updateMenuStates(); } + // Whether there were lyrics or not. The waveform's colour is saved + // with its layer, and the analyser, which took the layer over before + // the lyrics were looked for, gave it the fade it knew of then: + // faded if lyrics were found on show, and otherwise grey, even in a + // session saved faded whose lyrics have gone since + updateWaveformFade(); + if (!m_withSpectrogram) { m_analyser->setVisible(Analyser::Spectrogram, false); } diff --git a/main/MainWindow.h b/main/MainWindow.h index a5ad5368..8f5d07c9 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -364,14 +364,21 @@ protected slots: // is a take and none yet, take it away when the take goes void syncCoverageStrip(); - // The timed lyrics of the session, drawn along the top of pane 0 and - // stored in the session with the layer that draws them. They belong - // to the song, not to a take. Display only + // The timed lyrics of the session, drawn along the bottom of pane 0 + // and stored in the session with the layer that draws them. They + // belong to the song, not to a take. Display only LyricsTrack *m_lyrics; QAction *m_importLyricsAction; QAction *m_removeLyricsAction; QAction *m_showLyrics; + // Fade the waveforms of both analysers while the lyrics are on show + // over them, and not otherwise. Called after anything that shows or + // hides the lyrics, and after anything that makes an analyser: a new + // one starts unfaded, and a session's lyrics are found only after + // the reference's analyser has taken its waveform over + void updateWaveformFade(); + // Put the lyrics of this LRC file on the reference's timeline, in // place of any there are. Not undoable, as loading background music // is not, and nothing goes onto the undo stack. False if the file diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index f5dd9a38..da8b5538 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -944,6 +944,61 @@ class TestRecordWorkflow : public QObject qPrintable(when + ": the lyrics are the pane's top layer")); QVERIFY2(m_window->playSource()->getModels().count(model) == 0, qPrintable(when + ": the lyrics are in the play source")); + + // and the waveforms under them faded, the take's as well: a new + // take, a switch and a session load each make the take's analyser + // and its waveform again + QString colour = waveformColour(m_window->analyser()); + QVERIFY2(colour == "Pale Grey", + qPrintable(when + ": the reference's waveform is " + colour)); + Analyser *a2 = m_window->analyser2(); + if (a2 && a2->getLayer(Analyser::Audio)) { + colour = waveformColour(a2); + QVERIFY2(colour == "Pale Grey", + qPrintable(when + ": the take's waveform is " + colour)); + } + } + + // The name of the colour of an analyser's waveform, "" if it has none + static QString waveformColour(Analyser *a) { + int colour = colourOf(a ? a->getLayer(Analyser::Audio) : nullptr); + if (colour < 0) return {}; + return sv::ColourDatabase::getInstance()->getColourName(colour); + } + + // The word the lyrics layer has highlighted, "" for none + QString highlightedWord() { + sv::RegionLayer *layer = m_window->lyrics()->getLayer(); + sv::Event e(0); + if (!layer || !layer->getHighlightedEvent(e)) return {}; + return e.getLabel(); + } + + // Every key of the settings, with its value, in the form the test + // messages show + static QMap allSettings() { + QSettings settings; + QMap all; + for (const QString &key : settings.allKeys()) { + all[key] = settings.value(key).toString(); + } + return all; + } + + static QStringList settingsChanged(const QMap &before, + const QMap &after) { + QStringList changed; + for (auto i = after.begin(); i != after.end(); ++i) { + if (!before.contains(i.key())) { + changed << i.key() + " added: " + i.value(); + } else if (before[i.key()] != i.value()) { + changed << i.key() + ": " + before[i.key()] + " -> " + i.value(); + } + } + for (auto i = before.begin(); i != before.end(); ++i) { + if (!after.contains(i.key())) changed << i.key() + " removed"; + } + return changed; } static QString lyricsFixture(const char *name) { @@ -975,6 +1030,12 @@ class TestRecordWorkflow : public QObject return path; } + // Two words and a gap, then one more: every word with its end given + static QByteArray gappedLyrics() { + return "[00:00.20]<00:00.20>Yksi <00:00.60>kaksi <00:00.90>\n" + "[00:01.40]<00:01.40>kolme <00:01.80>\n"; + } + // The pre-roll's length has no UI: it is read from the settings when // Record is pressed. These are the test suite's own settings // (tony-app-test), not the user's @@ -5545,7 +5606,7 @@ private slots: } // Timed lyrics: an LRC file imported onto the reference's timeline, - // drawn along the top of pane 0 by LyricsTrack's layer + // drawn along the bottom of pane 0 by LyricsTrack's layer void lyrics_import_shows_words() { makeWindow(FakeAudioIO::Config()); @@ -6174,6 +6235,218 @@ private slots: noLyrics("a session without lyrics was given the last one's"); } + // The word at the playback position is highlighted with playback + // stopped as well: at once when the lyrics are imported with the + // cursor in a word, and wherever a seek puts the cursor + void lyrics_highlight_follows_seek() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(sv::sv_frame_t(0.7 * rate)); + QVERIFY(m_window->doImportLyricsFrom(writeLrc(gappedLyrics()))); + QCOMPARE(highlightedWord(), QString("kaksi")); + + // Before the first word, in each word, and in the gaps after them + const struct { double seconds; const char *word; } seeks[] = { + { 0.30, "Yksi" }, { 1.10, "" }, { 1.50, "kolme" }, + { 0.10, "" }, { 0.80, "kaksi" }, { 1.90, "" }, + }; + for (const auto &s : seeks) { + m_window->seekTo(sv::sv_frame_t(s.seconds * rate)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(s.word), 1000); + } + + // A highlight is not a change to the session + m_window->discardModifications(); + m_window->seekTo(sv::sv_frame_t(0.3 * rate)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("Yksi"), 1000); + QVERIFY(!m_window->isDocumentModified()); + } + + // While the reference plays, the highlight moves on from word to word + void lyrics_highlight_follows_playback() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->doImportLyricsFrom + (writeLrc("[00:00.00]<00:00.00>Yksi <00:00.25>kaksi " + "<00:00.50>kolme <00:00.75>neljä <00:01.00>viisi " + "<00:01.25>kuusi <00:01.50>\n"))); + QCOMPARE(m_window->playbackFrame(), sv::sv_frame_t(0)); + QCOMPARE(highlightedWord(), QString("Yksi")); + + m_window->doPlay(); + QTRY_VERIFY_WITH_TIMEOUT(highlightedWord() == "neljä", 2000); + m_window->doPlay(); + QVERIFY2(m_window->fake()->getPlayStartFrame() >= 0, + "nothing was played"); + } + + // During a take the highlight follows the cursor, which runs with the + // reference: through the lead-in of a pre-roll, while the status bar + // counts down, and from the take's position on the reference's + // timeline, not the recording's own. The take's waveform is faded + // under the lyrics like the reference's + void lyrics_highlight_and_fade_in_a_take() { + const double leadIn = 0.6; + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + makeWindow(config); + setPreRollSeconds(leadIn); + m_window->setPreRoll(true); + openReference(writeWav(tone(lowHz, 3.0))); + if (QTest::currentTestFailed()) return; + + // A word in the lead-in, a gap at the take's position, and two + // words after it + QVERIFY(m_window->doImportLyricsFrom + (writeLrc("[00:01.50]<00:01.50>Alku <00:01.90>\n" + "[00:02.10]<00:02.10>kaksi <00:02.40>kolme " + "<00:03.00>\n"))); + const sv::sv_frame_t P = sv::sv_frame_t(2.0 * rate); + m_window->seekTo(P); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + startTake(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->takePreRoll(), sv::sv_frame_t(leadIn * rate)); + QTRY_VERIFY_WITH_TIMEOUT(highlightedWord() == "Alku", 1000); + QVERIFY2(m_window->statusText().startsWith("Recording in "), + qPrintable(QString("the word in the lead-in is lit, but the " + "status bar says \"%1\" rather than " + "counting down") + .arg(m_window->statusText()))); + QTRY_VERIFY_WITH_TIMEOUT(highlightedWord() == "kolme", 1500); + stopTake(); + if (QTest::currentTestFailed()) return; + + // Back at the take's position, in the gap + QCOMPARE(m_window->playbackFrame(), P); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + // The take's analyser was made when the take stopped, with a + // waveform of its own, and that is faded too + Analyser *a2 = m_window->analyser2(); + QVERIFY(a2 && a2->getLayer(Analyser::Audio)); + QCOMPARE(waveformColour(a2), QString("Pale Grey")); + QCOMPARE(waveformColour(m_window->analyser()), QString("Pale Grey")); + } + + // The waveform is faded while the lyrics are on show over it, and + // only then. It is no setting of the user's: nothing goes into the + // settings, as Analyser::setVisible() and setAudible() would put it, + // no command is made, and the fade marks nothing modified + void lyrics_fade_the_waveform() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + Analyser *a = m_window->analyser(); + QCOMPARE(waveformColour(a), QString("Grey")); + + // Values the waveform does not have, which a write of its state to + // the settings would put right + QSettings settings; + settings.beginGroup("Analyser"); + settings.setValue(QString("visible-%1").arg(int(Analyser::Audio)), + !a->isVisible(Analyser::Audio)); + settings.setValue(QString("audible-%1").arg(int(Analyser::Audio)), + !a->isAudible(Analyser::Audio)); + settings.endGroup(); + settings.sync(); + auto before = allSettings(); + auto *history = sv::CommandHistory::getInstance(); + QSignalSpy commands(history, qOverload<> + (&sv::CommandHistory::commandExecuted)); + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + QCOMPARE(waveformColour(a), QString("Pale Grey")); + + QAction *show = m_window->showLyricsAction(); + show->trigger(); + QVERIFY(!m_window->lyrics()->isVisible()); + QCOMPARE(waveformColour(a), QString("Grey")); + show->trigger(); + QVERIFY(m_window->lyrics()->isVisible()); + QCOMPARE(waveformColour(a), QString("Pale Grey")); + + m_window->removeLyricsAction()->trigger(); + QVERIFY(!m_window->lyrics()->isShown()); + QCOMPARE(waveformColour(a), QString("Grey")); + + QCOMPARE(int(commands.count()), 0); + QStringList changed = settingsChanged(before, allSettings()); + QVERIFY2(changed.isEmpty(), + qPrintable("changed in the settings: " + changed.join(", "))); + + // The import, Show Lyrics and Remove Lyrics change the session; + // the fade on its own does not + m_window->discardModifications(); + a->setWaveformFaded(true); + QCOMPARE(waveformColour(a), QString("Pale Grey")); + a->setWaveformFaded(false); + QCOMPARE(waveformColour(a), QString("Grey")); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(int(commands.count()), 0); + } + + // The session saves the waveform's colour with its layer, faded or + // not. Opened, it is faded under lyrics on show and grey otherwise, + // whatever it was saved as, and the word at the cursor is lit at once + void lyrics_fade_survives_a_session() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + // Saved faded with no lyrics, as a session whose lyrics have gone + // since would be + m_window->analyser()->setWaveformFaded(true); + QString fadedWithout = m_dir.filePath("faded-without-lyrics.ton"); + QVERIFY(m_window->saveSessionFile(fadedWithout)); + m_window->analyser()->setWaveformFaded(false); + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + m_window->seekTo(sv::sv_frame_t(0.5 * rate)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("nollaa"), 1000); + QString shown = m_dir.filePath("lyrics-shown-faded.ton"); + QVERIFY(m_window->saveSessionFile(shown)); + m_window->showLyricsAction()->trigger(); + QCOMPARE(waveformColour(m_window->analyser()), QString("Grey")); + QString hidden = m_dir.filePath("lyrics-hidden-grey.ton"); + QVERIFY(m_window->saveSessionFile(hidden)); + + reopenSession(shown); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->lyrics()->isVisible()); + QCOMPARE(waveformColour(m_window->analyser()), QString("Pale Grey")); + sv::RegionLayer *layer = m_window->lyrics()->getLayer(); + QCOMPARE(layer->getHighlightFrame(), m_window->playbackFrame()); + + reopenSession(hidden); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->lyrics()->isShown()); + QVERIFY(!m_window->lyrics()->isVisible()); + QCOMPARE(waveformColour(m_window->analyser()), QString("Grey")); + + // Opened after one with lyrics on show, and saved faded itself. + // The close in between takes the fade away: the analyser of the + // reference stays for the next file + reopenSession(shown); + if (QTest::currentTestFailed()) return; + QCOMPARE(waveformColour(m_window->analyser()), QString("Pale Grey")); + QVERIFY(m_window->analyser()->isWaveformFaded()); + m_window->doCloseSession(); + QVERIFY2(!m_window->analyser()->isWaveformFaded(), + "the fade outlived the session"); + openReference(fadedWithout); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->lyrics()->isShown()); + QCOMPARE(waveformColour(m_window->analyser()), QString("Grey")); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform From 2a20de5417c682f212c3252a5062938eafb587d7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:14:57 +0000 Subject: [PATCH 094/275] fix: live dots are drawn as they come during a take The live dot model is made with notifyOnAdd false, so a dot added to it told the pane nothing, and the pane drew the dots only when one widened the model's pitch range or it redrew for some other reason: on a steady note they stalled within half a second of the take's start while the cursor ran on. Notifying for each dot instead has the pane draw every layer again for each of ~170 dots a second, which in a test took the GUI thread from 29 % to nearly 80 % of a core. ModelChangeThrottle (tony_core) tells the pane of the dots added at once and then at most every 40 ms, for about 54 % in the same test. The newest dot drawn now trails the cursor by about 120 ms, against 80-110 ms for the newest in the model. TestUiChecks::grabPane() has the pane paint itself before it is read: a pane that had just turned a page was still the old page in the backing store, which failed the pointer check now and then. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/architecture.md | 2 +- docs/forks.md | 3 + docs/manual-checklist.md | 3 + docs/open-points.md | 12 +- docs/recording.md | 9 ++ docs/testing.md | 12 +- main/MainWindow.cpp | 11 ++ main/MainWindow.h | 5 + main/ModelChangeThrottle.cpp | 86 ++++++++++++++ main/ModelChangeThrottle.h | 66 +++++++++++ main/test/TestModelChangeThrottle.h | 166 ++++++++++++++++++++++++++++ main/test/TestUiChecks.h | 10 +- main/test/tony-core-test.cpp | 7 ++ meson.build | 2 + 14 files changed, 375 insertions(+), 19 deletions(-) create mode 100644 main/ModelChangeThrottle.cpp create mode 100644 main/ModelChangeThrottle.h create mode 100644 main/test/TestModelChangeThrottle.h diff --git a/docs/architecture.md b/docs/architecture.md index fa0ba02c..a9d9d9d0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -30,7 +30,7 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h` | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `ModelChangeThrottle`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h` | | `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `TakeCommands`, `TakeLayers`, `PaneUtils` | When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` diff --git a/docs/forks.md b/docs/forks.md index cc25196c..42651b52 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -95,6 +95,9 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` ## Changes that would tidy Tony up but were not made +- A way to keep a layer out of `View`'s cache (a view told of a change to a cached layer's + model draws every cached layer again) would let the live dots be drawn as they come, + where `ModelChangeThrottle` now has the pane redrawn in full 25 times a second. - `Document::setModelSource()` (or any way to set or clear a derivation record) would replace `MainWindow::adoptTakeLayers()` setting source models by hand. - A hook in `MainWindowBase::toXml()` would save `MainWindow::toXml()` buffering the whole diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 81bb2d83..77d3fc84 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -78,6 +78,9 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w in use? 6. **Log out with unsaved takes** on Windows: its test does not run there, because `commitData()` writes into the real profile. See open points for the file it writes. +7. **Live dots on this machine**: during a take the dots keep up with the cursor and grow + smoothly, and neither they nor the cursor stutter, in a maximised window. Each batch of + dots, 25 a second, has the whole pane drawn again; see open points. The defects the automated checks found, and the facts they established for the decisions above, are in [open-points.md](open-points.md). diff --git a/docs/open-points.md b/docs/open-points.md index 3449096f..4ab1f02d 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -39,16 +39,14 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Weak spots +- **The live dots cost a redraw of the whole pane 25 times a second** during a take + (`ModelChangeThrottle`): on the cloud machine, a 1920 px window, the GUI thread went from + 29 % to about 54 % of a core. A HiDPI screen makes each redraw dearer. A way in svgui to + keep a layer out of a view's cache would make it nearly free ([forks.md](forks.md)). + Defects the checks of `TestUiChecks` found, each committed as a test expected to fail (`QEXPECT_FAIL` names the cause): -- **Live dots stop being drawn during a take.** The dot model is made with `notifyOnAdd` - false, so a dot added tells the pane nothing; dots are drawn only when one widens the - model's pitch range or the pane redraws for another reason (a page turn, a zoom). On a - steady note they stall within half a second (`live_dots_under_the_cursor`). Making the - model with `notifyOnAdd` true fixes it, as tried: the pane then repaints for every dot, - about 170 times a second, coalesced by Qt; whether that is cheap enough on the - development machine has not been measured. - **The band of the coverage strip is hidden after the second recording.** The audio swap makes the take's waveform layer again, on top, and `syncCoverageStrip()` raises nothing once the strip is shown (`strip_on_top_after_another_recording`). diff --git a/docs/recording.md b/docs/recording.md index 653f2bb4..a2aa304f 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -147,6 +147,15 @@ dot model outlives the take. there is not a full window yet. `kHopSize` is also the resolution of the dot model, whose unit must be `"Hz"` for the layer to align to the pane's log-frequency scale. +**Telling the pane of the dots.** A pane told of a change to one of its layers' models +draws every layer again. A notice for each dot, about 170 a second, took the GUI thread +from 29 % to nearly 80 % of a core (cloud machine, 1920 px window). So the dot model is +made with `notifyOnAdd` false, and then it tells nobody of a dot at all: the dots were +drawn only when one widened the model's pitch range, and stalled within half a second on a +steady note. `onRealtimePitchDetected()` hands each dot's frames to +`m_realtimeDotsNotifier` (`ModelChangeThrottle`, `tony_core`), which tells the pane at +once and then at most every 40 ms (about 54 % of a core in the same test). + Correct as they are, though they look wrong: - The `1/frameSize` scale in the FFT difference function: bqfft's inverse is unscaled. diff --git a/docs/testing.md b/docs/testing.md index 7496a3c2..cf5e8bd2 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,7 +5,7 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming` | seconds | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | | `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | @@ -146,7 +146,7 @@ it first, or break the code for a moment (mark the line `MUTATION`, and check purpose. "Fixing" one side makes the live dots and the pYIN track disagree. A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming -it; the marker goes in the commit that fixes it. There are four at present, all in +it; the marker goes in the commit that fixes it. There are three at present, all in `TestUiChecks`, listed in [open-points.md](open-points.md). ## Timing and races @@ -182,10 +182,10 @@ The window is shown (still offscreen), made active so that its shortcuts work, a with `QTest` key presses, mouse gestures on pane 0 and the dialogs MainWindow shows. What it draws is judged by pixels: -- **Read the screen, not `QWidget::grab()`**: `grabPane()` copies pane 0 out of the - window's backing store, which holds what the pane's own paint events put there. `grab()` - has the pane paint itself once more and can show what the screen does not: it showed - live dots that the screen never got. +- **Read the screen**: `grabPane()` has pane 0 paint itself, as its next update would, and + copies it out of the window's backing store. Without the paint, a pane that has just + turned a page is still the old page in the backing store while every position asked of + it is on the new one: a play pointer 500 px from where it was looked for. - After any playback the pane's cache holds the translucent note boxes painted twice. Compare images only after `grabPaneRedrawn()`, which forces a full redraw (a zoom one step away snaps back to the same level and redraws nothing; it doubles the level). diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 534cf8a7..7474ffa4 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -132,6 +132,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_analyser2(nullptr), m_realtimePitchTracker(nullptr), m_realtimePitchLayer(nullptr), + m_realtimeDotsNotifier(40), m_overview(0), m_showSingingPitch(nullptr), m_showSingingNotes(nullptr), @@ -3616,6 +3617,12 @@ MainWindow::setupRealtimePitchLayer() // Its resolution is the YIN hop size: one estimate per hop. // Unit "Hz" is required so TimeValueLayer::shouldAutoAlign() defers to // the pane's log-frequency coordinate system (same as the pYIN pitch track). + // + // notifyOnAdd false: a pane told of a change to one of its layers' + // models draws all of them again, and a notice for each of the ~170 + // dots a second kept the GUI thread busy most of the time. The model + // then tells nobody of a dot, though, so m_realtimeDotsNotifier tells + // the pane of what was added, 25 times a second auto pitchModel = std::make_shared (sr, RealtimePitchTracker::kHopSize, false); pitchModel->setObjectName(tr("Realtime Pitch (Live)")); @@ -3642,6 +3649,7 @@ MainWindow::setupRealtimePitchLayer() // Associate our pre-filled SparseTimeValueModel with the layer. // The model was already registered via addNonDerivedModel above. m_document->setModel(m_realtimePitchLayer, m_realtimePitchModelId); + m_realtimeDotsNotifier.setModel(m_realtimePitchModelId); m_realtimePitchLayer->setVerticalScale(TimeValueLayer::AutoAlignScale); m_realtimePitchLayer->setPlotStyle(TimeValueLayer::PlotPoints); @@ -3680,6 +3688,7 @@ void MainWindow::teardownRealtimePitchLayer() { stopRealtimePitchTracker(); + m_realtimeDotsNotifier.setModel({}); if (m_realtimeLayerTeardownConnection) { disconnect(m_realtimeLayerTeardownConnection); @@ -4280,6 +4289,8 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) if (m) { m->add(Event(dotFrame, float(hz), tr(""))); + m_realtimeDotsNotifier.changed + (dotFrame, dotFrame + RealtimePitchTracker::kHopSize); } // Convert Hz to MIDI note number and cents deviation. diff --git a/main/MainWindow.h b/main/MainWindow.h index d27a635d..d03ab0f0 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -24,6 +24,7 @@ #include "SingingTakes.h" #include "TakeCommands.h" #include "TakeTiming.h" +#include "ModelChangeThrottle.h" #include #include @@ -311,6 +312,10 @@ protected slots: // Model backing the realtime layer (owned by the document). sv::ModelId m_realtimePitchModelId; + // Tells the pane of the dots added to that model, which tells nobody + // itself (see setupRealtimePitchLayer()) + ModelChangeThrottle m_realtimeDotsNotifier; + sv::Overview *m_overview; // Actions/toolbar items for the singing track diff --git a/main/ModelChangeThrottle.cpp b/main/ModelChangeThrottle.cpp new file mode 100644 index 00000000..6b362cfa --- /dev/null +++ b/main/ModelChangeThrottle.cpp @@ -0,0 +1,86 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "ModelChangeThrottle.h" + +#include + +using namespace sv; + +ModelChangeThrottle::ModelChangeThrottle(int intervalMs) : + m_pending(false), + m_from(0), + m_to(0) +{ + m_timer.setInterval(intervalMs); + QObject::connect(&m_timer, &QTimer::timeout, + &m_timer, [this]() { intervalUp(); }); +} + +ModelChangeThrottle::~ModelChangeThrottle() +{ +} + +void +ModelChangeThrottle::setModel(ModelId model) +{ + m_timer.stop(); + m_pending = false; + m_model = model; +} + +void +ModelChangeThrottle::changed(sv_frame_t from, sv_frame_t to) +{ + if (m_model.isNone()) return; + + if (m_pending) { + m_from = std::min(m_from, from); + m_to = std::max(m_to, to); + } else { + m_from = from; + m_to = to; + m_pending = true; + } + + // Quiet until now: tell at once, and hold back what comes next for an + // interval + if (!m_timer.isActive()) { + tell(); + m_timer.start(); + } +} + +void +ModelChangeThrottle::intervalUp() +{ + // Nothing came in the interval: quiet again + if (!m_pending) { + m_timer.stop(); + return; + } + tell(); +} + +void +ModelChangeThrottle::tell() +{ + m_pending = false; + auto model = ModelById::get(m_model); + if (!model) return; + + // The model's own notice, as it would have sent it itself for each + // change: a view redraws only what it is told of + emit model->modelChangedWithin(m_model, m_from, m_to); +} diff --git a/main/ModelChangeThrottle.h b/main/ModelChangeThrottle.h new file mode 100644 index 00000000..59f99094 --- /dev/null +++ b/main/ModelChangeThrottle.h @@ -0,0 +1,66 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_MODEL_CHANGE_THROTTLE_H +#define TONY_MODEL_CHANGE_THROTTLE_H + +#include "base/BaseTypes.h" +#include "data/model/Model.h" + +#include + +/** + * Tells the views of a model what has changed in it, at most once an + * interval: for a model written to faster than it is worth drawing, and + * made to hold back its own change notices (notifyOnAdd false), such as + * the live pitch dots of a take. + * + * A pane that is told of a change to one of its layers' models draws + * every layer again, so a notice for each of the ~170 dots a second + * kept the GUI thread busy for three quarters of its time. Told + * nothing, a pane never draws the dots at all. + * + * The first change after a quiet interval is told at once; changes + * that follow within the interval are told together at its end. Lives + * on the GUI thread, as the model's writer and its views do. + */ +class ModelChangeThrottle +{ +public: + explicit ModelChangeThrottle(int intervalMs); + ~ModelChangeThrottle(); + + ModelChangeThrottle(const ModelChangeThrottle &) = delete; + ModelChangeThrottle &operator=(const ModelChangeThrottle &) = delete; + + /// The model whose views are told. A model of none (the default) + /// stops, and a change not yet told is forgotten + void setModel(sv::ModelId model); + sv::ModelId getModel() const { return m_model; } + + /// Frames [from, to) of the model have changed + void changed(sv::sv_frame_t from, sv::sv_frame_t to); + +private: + void intervalUp(); + void tell(); + + sv::ModelId m_model; + QTimer m_timer; + bool m_pending; + sv::sv_frame_t m_from; + sv::sv_frame_t m_to; +}; + +#endif diff --git a/main/test/TestModelChangeThrottle.h b/main/test/TestModelChangeThrottle.h new file mode 100644 index 00000000..355bad6f --- /dev/null +++ b/main/test/TestModelChangeThrottle.h @@ -0,0 +1,166 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_MODEL_CHANGE_THROTTLE_H +#define TEST_MODEL_CHANGE_THROTTLE_H + +// Tier 2: telling a model's views of its changes, at most once an +// interval. What a pane does when told is TestUiChecks' business +// (live_dots_under_the_cursor). + +#include "../ModelChangeThrottle.h" + +#include "data/model/SparseTimeValueModel.h" + +#include +#include +#include + +#include +#include +#include + +class TestModelChangeThrottle : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + + static constexpr int kInterval = 50; + + // A model like the live dots': changes held back by the model itself + std::shared_ptr m_model; + sv::ModelId m_id; + std::vector> m_told; + +private slots: + void init() { + m_model = std::make_shared(44100, 256, false); + m_id = sv::ModelById::add(m_model); + m_told.clear(); + connect(m_model.get(), &sv::Model::modelChangedWithin, + this, [this](sv::ModelId, frame_t from, frame_t to) { + m_told.push_back({ from, to }); + }); + } + + void cleanup() { + m_model.reset(); + sv::ModelById::release(m_id); + } + + // Why there is a throttle at all: a dot added to such a model tells + // nobody, so nothing would ever draw it + void the_model_tells_nothing_itself() { + m_model->add(sv::Event(1000, 220.f, "")); + m_model->add(sv::Event(1256, 221.f, "")); + QCoreApplication::processEvents(); + QCOMPARE(int(m_told.size()), 0); + } + + void first_change_is_told_at_once() { + ModelChangeThrottle throttle(kInterval); + throttle.setModel(m_id); + throttle.changed(1000, 1256); + QCOMPARE(int(m_told.size()), 1); + QCOMPARE(m_told[0], std::make_pair(frame_t(1000), frame_t(1256))); + } + + // ... and what follows within the interval is told together, once, + // when it is up + void changes_within_an_interval_are_told_together() { + ModelChangeThrottle throttle(kInterval); + throttle.setModel(m_id); + throttle.changed(1000, 1256); + throttle.changed(2000, 2256); + throttle.changed(1500, 1756); + QCOMPARE(int(m_told.size()), 1); + + QTRY_COMPARE_WITH_TIMEOUT(int(m_told.size()), 2, kInterval * 10); + QCOMPARE(m_told[1], std::make_pair(frame_t(1500), frame_t(2256))); + + // Nothing more to tell + QTest::qWait(kInterval * 3); + QCOMPARE(int(m_told.size()), 2); + } + + // A steady stream is told once an interval, however many changes + void a_stream_is_told_once_an_interval() { + ModelChangeThrottle throttle(kInterval); + throttle.setModel(m_id); + QElapsedTimer timer; + timer.start(); + int changes = 0; + while (timer.elapsed() < kInterval * 10) { + throttle.changed(changes * 256, changes * 256 + 256); + ++changes; + QTest::qWait(2); + } + QVERIFY(changes > 100); + QVERIFY2(m_told.size() >= 5 && m_told.size() <= 13, + qPrintable(QString("%1 changes over ten intervals were told " + "%2 times") + .arg(changes).arg(m_told.size()))); + // Between them the notices cover every change + QTest::qWait(kInterval * 2); + QCOMPARE(m_told.front().first, frame_t(0)); + QCOMPARE(m_told.back().second, frame_t(changes * 256)); + for (size_t i = 1; i < m_told.size(); ++i) { + QVERIFY(m_told[i].first <= m_told[i-1].second); + } + } + + // After a quiet interval the next change is told at once again + void quiet_then_told_at_once() { + ModelChangeThrottle throttle(kInterval); + throttle.setModel(m_id); + throttle.changed(1000, 1256); + QTest::qWait(kInterval * 3); + QCOMPARE(int(m_told.size()), 1); + throttle.changed(5000, 5256); + QCOMPARE(int(m_told.size()), 2); + } + + // A change not yet told goes with the model, and with no model + // nothing is told at all + void no_model_tells_nothing() { + ModelChangeThrottle throttle(kInterval); + throttle.changed(1000, 1256); + QCOMPARE(int(m_told.size()), 0); + + throttle.setModel(m_id); + throttle.changed(1000, 1256); + throttle.changed(2000, 2256); + QCOMPARE(int(m_told.size()), 1); + throttle.setModel({}); + QTest::qWait(kInterval * 3); + QCOMPARE(int(m_told.size()), 1); + } + + // The model released under it: nothing to tell, and no harm + void model_gone() { + ModelChangeThrottle throttle(kInterval); + sv::ModelId id = m_id; + throttle.setModel(id); + throttle.changed(1000, 1256); + throttle.changed(2000, 2256); + m_model.reset(); + sv::ModelById::release(id); + m_id = {}; + QTest::qWait(kInterval * 3); + throttle.changed(3000, 3256); + QCOMPARE(int(m_told.size()), 1); + } +}; + +#endif diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index e0af19b0..2686602c 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -178,9 +178,13 @@ class TestUiChecks : public QObject // Pane 0 as it is on the screen just now: the window's backing store, // which holds what the pane's own paint events put there. Not // QWidget::grab(), which has the pane paint itself once more for the - // occasion and so can show what the screen does not + // occasion and so can show what the screen does not. The pane is + // painted first, as it would be at the next update: a pane that has + // just turned a page is still the old page on the screen until then, + // while every position asked of it is on the new one QImage grabPane() { QCoreApplication::processEvents(); + pane0()->repaint(); QPixmap window = m_window->screen()->grabWindow(m_window->winId()); QRect rect(pane0()->mapTo(m_window, QPoint(0, 0)), pane0()->size()); return window.copy(rect).toImage() @@ -604,10 +608,6 @@ private slots: "whether the pane follows it was not seen"); saveWindowShot("during"); qInfo("%s", qPrintable(worst)); - QEXPECT_FAIL("", "the live dot model is made with notifyOnAdd false, " - "so a dot added to it tells the pane nothing: dots are " - "drawn only when one widens the model's range or the " - "pane redraws for another reason", Continue); QVERIFY2(worstLag <= lag, qPrintable(worst)); // The dots stay until the take's pitch track is there, which is diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 4a7ed2b5..5beddc6d 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -20,6 +20,7 @@ #include "TestSingingTakes.h" #include "TestTakesFile.h" #include "TestTakeTiming.h" +#include "TestModelChangeThrottle.h" #include "RunSuite.h" @@ -98,6 +99,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestModelChangeThrottle t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index b76bd646..027f2826 100644 --- a/meson.build +++ b/meson.build @@ -1095,6 +1095,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/Coverage.cpp', + 'main/ModelChangeThrottle.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', @@ -1356,6 +1357,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestSingingTakes.h', 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', + 'main/test/TestModelChangeThrottle.h', ]) tony_core_test_exe = executable( From 329d6ba8338b55e15ea5a0515020696e72cea60b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:15:20 +0000 Subject: [PATCH 095/275] fix: the coverage band stays on top of the take's waveform From the second recording of a take on, the band along the bottom of the pane was drawn under the take's waveform, and only specks of it showed: the audio swap makes the waveform layer again, on top of everything, and syncCoverageStrip() restacked nothing once the strip was shown. It now raises the strip whenever it is not the topmost layer, which also covers undo, redo and a session opened again. The raise is the view's own bookkeeping, so the strip's model stays out of the play source. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/open-points.md | 3 -- docs/takes.md | 5 ++- docs/testing.md | 2 +- main/MainWindow.cpp | 11 ++++++ main/test/TestUiChecks.h | 79 +++++++++++++++++++++++++++++----------- 5 files changed, 72 insertions(+), 28 deletions(-) diff --git a/docs/open-points.md b/docs/open-points.md index 4ab1f02d..68b4b7fb 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -47,9 +47,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. Defects the checks of `TestUiChecks` found, each committed as a test expected to fail (`QEXPECT_FAIL` names the cause): -- **The band of the coverage strip is hidden after the second recording.** The audio swap - makes the take's waveform layer again, on top, and `syncCoverageStrip()` raises nothing - once the strip is shown (`strip_on_top_after_another_recording`). - **`commitData()` writes `~/.sv1/tmp-*.sv`**, Sonic Visualiser's extension; Tony opens only `.ton` as a session, so what it saved at logout does not open from Recent Files (`commit_data_writes_a_playable_session`; renamed to `.ton` it opens and plays). diff --git a/docs/takes.md b/docs/takes.md index f83af74e..01bac609 100644 --- a/docs/takes.md +++ b/docs/takes.md @@ -180,8 +180,9 @@ display-only, `EqualSpaced` scale so the pane's scale is untouched) per take, in take's coverage is read back out of its strip. `MainWindow::syncCoverageStrip()` is the one place it is kept in step. Call it **after** a -swap, never before: the swap restores the pane's state as it found it. It also removes the -strip's model from the play source. +swap, never before: the swap restores the pane's state as it found it, and makes the take's +waveform layer again, on top, where it covers the band; `syncCoverageStrip()` raises the +strip above it every time. It also removes the strip's model from the play source. ## Files on disk diff --git a/docs/testing.md b/docs/testing.md index cf5e8bd2..01112970 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -146,7 +146,7 @@ it first, or break the code for a moment (mark the line `MUTATION`, and check purpose. "Fixing" one side makes the live dots and the pYIN track disagree. A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming -it; the marker goes in the commit that fixes it. There are three at present, all in +it; the marker goes in the commit that fixes it. There are two at present, all in `TestUiChecks`, listed in [open-points.md](open-points.md). ## Timing and races diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 7474ffa4..29ffafc3 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3284,6 +3284,17 @@ MainWindow::syncCoverageStrip() m_coverageStrip->setCoverage(m_takes->getCoverage()); + // The band runs along the bottom of the pane, over the take's + // waveform, so it has to be above that layer. The swap makes the + // waveform layer again, on top of everything (as does activating a + // take), so this is looked at on every call and not only when the + // strip is first shown + Layer *strip = m_coverageStrip->getLayer(); + int layers = pane->getLayerCount(); + if (strip && layers > 0 && pane->getLayer(layers - 1) != strip) { + TakeLayers::raise(pane, strip); + } + if (!wasShown) { // The new layer is on top, where a tool would look for the layer // to act on, so the tracks that can be edited go back there, as diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 2686602c..993ec35f 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -717,7 +717,7 @@ private slots: openReference(writeWav(tone(lowHz, 4.0))); if (QTest::currentTestFailed()) return; - // One recording: see strip_on_top_after_another_recording + // One recording, so that the band has an end to drag off m_window->seekTo(frames(0.5)); take(1500); if (QTest::currentTestFailed()) return; @@ -895,8 +895,10 @@ private slots: } // The band after a second recording, in a gap of the first: the take's - // audio is swapped for a file holding both, and the band has to stay - // on top of the waveform of the new file + // audio is swapped for a file holding both, and the take's waveform + // layer is made again. The band has to stay on top of it, then and + // after everything else that swaps the audio or makes the layer again: + // undo, redo, and opening the session void strip_on_top_after_another_recording() { FakeAudioIO::Config config; config.input = tone(highHz, 6.0); @@ -905,29 +907,64 @@ private slots: openReference(writeWav(tone(lowHz, 4.0))); if (QTest::currentTestFailed()) return; + const sv::sv_frame_t inFirst = frames(0.4); + const sv::sv_frame_t inSecond = frames(2.2); + + // Whether the band is drawn at a frame: a few columns either side + // of it along the bottom row, all orange + auto band = [&](sv::sv_frame_t frame, QString shot) { + showSeconds(0.0, 3.0); + QImage image = grabPaneRedrawn(); + saveShot(shot, image); + int x = pane0()->getXForFrame(frame); + for (int dx = -3; dx <= 3; ++dx) { + if (!isOrange(image.pixel(x + dx, image.height() - 3))) { + return false; + } + } + return true; + }; + take(1000); if (QTest::currentTestFailed()) return; - showSeconds(0.0, 3.0); - sv::Pane *pane = pane0(); - const sv::sv_frame_t inFirst = frames(0.4); - QImage image = grabPaneRedrawn(); - QVERIFY(isOrange(image.pixel(pane->getXForFrame(inFirst), - image.height() - 3))); + QVERIFY(band(inFirst, "first")); m_window->seekTo(frames(2.0)); take(700); if (QTest::currentTestFailed()) return; - showSeconds(0.0, 3.0); - image = grabPaneRedrawn(); - saveShot("after-second", image); - int y = image.height() - 3; - QEXPECT_FAIL("", "the take's waveform layer made by the audio swap " - "is attached above the coverage strip and covers the " - "band (syncCoverageStrip() raises nothing once the " - "strip is shown)", Continue); - QVERIFY2(isOrange(image.pixel(pane->getXForFrame(inFirst), y)) && - isOrange(image.pixel(pane->getXForFrame(frames(2.2)), y)), + QVERIFY2(band(inFirst, "second") && band(inSecond, "second"), "after a second recording the band is not drawn"); + + // Raised, and still a picture, not sound: out of the play source + auto stripIsSilent = [this]() { + return !m_window->playSource()->getModels().count + (m_window->coverageStrip()->getModelId()); + }; + QVERIFY2(stripIsSilent(), "the strip's model is in the play source"); + + QCOMPARE(undoText(), tr("&Undo %1").arg(tr("Record Singing"))); + press(QKeySequence(tr("Ctrl+Z"))); + QTRY_COMPARE(int(coverage().size()), 1); + QVERIFY2(band(inFirst, "undone") && !band(inSecond, "undone"), + "after an undo the band is not what is left of the take"); + + press(QKeySequence(tr("Ctrl+Shift+Z"))); + QTRY_COMPARE(int(coverage().size()), 2); + QVERIFY2(band(inFirst, "redone") && band(inSecond, "redone"), + "after a redo the band is not drawn"); + + QString session = m_dir.filePath + (QString("session-%1.ton").arg(++m_fileCounter)); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); + QTRY_VERIFY(m_window->takes()->haveTake() && + int(coverage().size()) == 2); + QVERIFY2(band(inFirst, "reopened") && band(inSecond, "reopened"), + "in the session opened again the band is not drawn"); + QVERIFY2(stripIsSilent(), "the strip's model is in the play source"); } // Checklist: Erase and Select Recording are greyed out with no take, @@ -1184,14 +1221,12 @@ private slots: int pointer = pane->getXForFrame(P); QVERIFY(left > 20 && right < before.width() - 20); - // The band of the coverage strip along the bottom is left to - // strip_on_top_after_another_recording int differing = 0; QString first; for (int x = 0; x < before.width(); ++x) { if (x >= left && x <= right) continue; if (std::abs(x - pointer) <= 3) continue; - for (int y = 0; y < before.height() - 10; ++y) { + for (int y = 0; y < before.height(); ++y) { if (before.pixel(x, y) != after.pixel(x, y)) { if (first == "") { first = QString("x = %1, y = %2").arg(x).arg(y); From c733e3b8d98fd448c9813d4d45dd23e41f9a909d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:15:32 +0000 Subject: [PATCH 096/275] fix: what commitData() saves at logout is a .ton commitData(), the save made when the system logs the user out with unsaved work, named its file ~/.sv1/tmp-*.sv, Sonic Visualiser's session extension. Tony opens only .ton as a session and tried the file as audio, so the takes it saved could not be opened from Recent Files, where it is put. It now uses the application's own session extension. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/manual-checklist.md | 3 ++- docs/open-points.md | 9 --------- docs/testing.md | 2 +- main/MainWindow.cpp | 17 ++++++++++++----- main/test/TestUiChecks.h | 16 +++------------- 5 files changed, 18 insertions(+), 29 deletions(-) diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 77d3fc84..6f866aaf 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -77,7 +77,8 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w 5. **Take operations clear the undo history with no prompt** (all but Rename): acceptable in use? 6. **Log out with unsaved takes** on Windows: its test does not run there, because - `commitData()` writes into the real profile. See open points for the file it writes. + `commitData()` writes into the real profile. Afterwards `~/.sv1/tmp-*.ton` is on the + Recent Files list, opens, and its takes play. 7. **Live dots on this machine**: during a take the dots keep up with the cursor and grow smoothly, and neither they nor the cursor stutter, in a maximised window. Each batch of dots, 25 a second, has the whole pane drawn again; see open points. diff --git a/docs/open-points.md b/docs/open-points.md index 68b4b7fb..3f657a27 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -44,15 +44,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. 29 % to about 54 % of a core. A HiDPI screen makes each redraw dearer. A way in svgui to keep a layer out of a view's cache would make it nearly free ([forks.md](forks.md)). -Defects the checks of `TestUiChecks` found, each committed as a test expected to fail -(`QEXPECT_FAIL` names the cause): - -- **`commitData()` writes `~/.sv1/tmp-*.sv`**, Sonic Visualiser's extension; Tony opens - only `.ton` as a session, so what it saved at logout does not open from Recent Files - (`commit_data_writes_a_playable_session`; renamed to `.ton` it opens and plays). - -Seen and not pinned by a test: - - After playback the pane's own cache of what it drew holds the translucent note boxes painted twice over themselves, darker, until the next zoom or scroll. Seen with the offscreen platform, through the window's backing store; whether it shows on a real screen diff --git a/docs/testing.md b/docs/testing.md index 01112970..15551826 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -146,7 +146,7 @@ it first, or break the code for a moment (mark the line `MUTATION`, and check purpose. "Fixing" one side makes the live dots and the pYIN track disagree. A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming -it; the marker goes in the commit that fixes it. There are two at present, all in +it; the marker goes in the commit that fixes it. There is one at present, in `TestUiChecks`, listed in [open-points.md](open-points.md). ## Timing and races diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 29ffafc3..0b2540bc 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -72,6 +72,7 @@ #include "widgets/RangeInputDialog.h" #include "widgets/ActivityLog.h" +#include "widgets/InteractiveFileFinder.h" // For version information #include "vamp/vamp.h" @@ -6083,14 +6084,20 @@ MainWindow::commitData(bool mayAskUser) if (!QFileInfo(svDir).isDir()) return false; } - // This name doesn't have to be unguessable + // This name doesn't have to be unguessable. Its extension is the + // one this application opens as a session -- .ton, not Sonic + // Visualiser's .sv, which Tony would try to open as audio + QString extension = InteractiveFileFinder::getInstance() + ->getApplicationSessionExtension(); #ifndef _WIN32 - QString fname = QString("tmp-%1-%2.sv") + QString fname = QString("tmp-%1-%2.%3") .arg(QDateTime::currentDateTime().toString("yyyyMMddhhmmsszzz")) - .arg(QProcess().processId()); + .arg(QProcess().processId()) + .arg(extension); #else - QString fname = QString("tmp-%1.sv") - .arg(QDateTime::currentDateTime().toString("yyyyMMddhhmmsszzz")); + QString fname = QString("tmp-%1.%2") + .arg(QDateTime::currentDateTime().toString("yyyyMMddhhmmsszzz")) + .arg(extension); #endif QString fpath = QDir(svDir).filePath(fname); if (saveSessionFile(fpath)) { diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 993ec35f..bc8f58d7 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -1333,22 +1333,12 @@ private slots: .entryList({ "tmp-*" }, QDir::Files); QCOMPARE(written.size(), 1); QString path = fakeHome + "/.sv1/" + written[0]; + QVERIFY2(path.endsWith(".ton"), + qPrintable("the session is written as " + path)); // It goes on the Recent Files list, which opens it with openPath() m_window->doCloseSession(); - MainWindow::FileOpenStatus opened = - m_window->openPath(path, MainWindow::ReplaceSession); - QEXPECT_FAIL("", "commitData() names the file tmp-*.sv, Sonic " - "Visualiser's session extension; Tony opens only .ton " - "as a session", Continue); - QCOMPARE(opened, MainWindow::FileOpenSucceeded); - - // What is in it, under the name Tony would open - m_window->doCloseSession(); - QString ton = QFileInfo(path).path() + "/" + - QFileInfo(path).completeBaseName() + ".ton"; - QVERIFY(QFile::rename(path, ton)); - QCOMPARE(m_window->openPath(ton, MainWindow::ReplaceSession), + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), MainWindow::FileOpenSucceeded); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); QVERIFY2(m_window->takes()->haveTake(), From a30fa8f868cc10bb140a3bd8e55116ab569de903 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:15:50 +0000 Subject: [PATCH 097/275] fix: a take is in time with playback constrained to the selection With Constrain Playback to Selection on, the play source started the reference in the selection instead of at the lead-in of a pre-roll (or at a playhead outside the selection), and stopped or looped it at its end, while the take counts the reference as playing on from where playback was asked to start. With a pre-roll, what was sung was placed a whole pre-roll too early. The constraint is now lifted just before the reference starts for a take and put back when the take is over or the session closes. It goes through the view manager, which writes no settings; the toolbar button follows, and is greyed out during the take so that it cannot be put back meanwhile. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/manual-checklist.md | 4 ++-- docs/open-points.md | 9 +++------ docs/recording.md | 6 ++++++ docs/testing.md | 3 +-- main/MainWindow.cpp | 31 +++++++++++++++++++++++++++++++ main/MainWindow.h | 7 +++++++ main/test/TestUiChecks.h | 19 +++++++++++++------ 7 files changed, 63 insertions(+), 16 deletions(-) diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 6f866aaf..12457761 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -83,5 +83,5 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w smoothly, and neither they nor the cursor stutter, in a maximised window. Each batch of dots, 25 a second, has the whole pane drawn again; see open points. -The defects the automated checks found, and the facts they established for the decisions -above, are in [open-points.md](open-points.md). +The questions the automated checks raised, and the facts they established for the +decisions above, are in [open-points.md](open-points.md). diff --git a/docs/open-points.md b/docs/open-points.md index 3f657a27..d00fdd7b 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -10,11 +10,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. Is 3 s right, and should there be a control? - **No overwrite question when recording into a selection**: the selection is taken as the consent. Right in use? -- **Constrain Playback to Selection + pre-roll**: worse than a short lead-in. The play - source starts playback at the selection, so none of the lead-in is played, and the splice, - which takes playback to have started a pre-roll earlier, places what was sung a whole - pre-roll too early (`preroll_with_playback_constrained_to_the_selection`, expected to - fail). Keep the two apart, or start playback at the lead-in regardless? - **Take operations clear the undo history with no prompt** (all but Rename). - **A selection is an entry of the undo history**: making one re-analyses the reference in it (upstream Tony's pitch candidates) and pushes "Re-Analyse Selection". Selecting for @@ -43,7 +38,9 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. (`ModelChangeThrottle`): on the cloud machine, a 1920 px window, the GUI thread went from 29 % to about 54 % of a core. A HiDPI screen makes each redraw dearer. A way in svgui to keep a layer out of a view's cache would make it nearly free ([forks.md](forks.md)). - +- **Loop Playback is left on during a take**, unlike Constrain Playback to Selection: a + take that runs past the end of the reference would hear it start again while the take + places what is sung after the end. Not tried. - After playback the pane's own cache of what it drew holds the translucent note boxes painted twice over themselves, darker, until the next zoom or scroll. Seen with the offscreen platform, through the window's backing store; whether it shows on a real screen diff --git a/docs/recording.md b/docs/recording.md index a2aa304f..7b22975b 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -127,6 +127,12 @@ dot model outlives the take. - **Pre-roll**: R = `MainWindow/prerollseconds` (3 s, no UI on purpose) clipped to the start of the song. The device records from the press of Record as always; the splice simply starts R frames later. With Play Reference off it is just a pause. +- **Constrain Playback to Selection is lifted for a take** (`liftPlaySelectionForTake()`, + just before `play(S)`) and put back in `recordingFinishedFull()` and `closeSession()`. + Constrained, the play source starts in the selection rather than at S and stops or loops + at its end, while the take counts the reference as playing on from S without a break: + with a pre-roll, what was sung landed a whole pre-roll early. Done through `ViewManager`, + which writes no settings; the button follows, and is greyed out during the take. - **Record into Selection**: the take stops itself when `getFramesReceived()` reaches `L + R + (E − P) + 0.25 s` (`autoStopFrames()`). `pollTakeProgress()` calls `record()` — the same path as the Stop button, so everything that ends a take is in one place. The diff --git a/docs/testing.md b/docs/testing.md index 15551826..6d4e0d9f 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -146,8 +146,7 @@ it first, or break the code for a moment (mark the line `MUTATION`, and check purpose. "Fixing" one side makes the live dots and the pYIN track disagree. A bug that is known and not yet fixed is committed as a test with `QEXPECT_FAIL` naming -it; the marker goes in the commit that fixes it. There is one at present, in -`TestUiChecks`, listed in [open-points.md](open-points.md). +it; the marker goes in the commit that fixes it. There are none at present. ## Timing and races diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0b2540bc..5f548aab 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -194,6 +194,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_recordingAsSingingTrack(false), m_singingAudioMutedForTake(false), m_singingAudioAfterTake(true), + m_playSelectionLiftedForTake(false), m_paneCountBeforeRecording(0), m_currentRecordingModelId(), m_recordingLayer(nullptr), @@ -2200,6 +2201,10 @@ MainWindow::updateMenuStates() emit canEraseSinging(haveCoverage && !inTake && haveSelection && !analysingRange); + // Nor can playback be constrained to the selection during a take: + // see liftPlaySelectionForTake() + if (inTake) emit canPlaySelection(false); + // The takes of the session: switching and making one need a session // and nothing running, and the rest need a take to act on as well bool canChange = takeOperationsAllowed(); @@ -2551,6 +2556,7 @@ MainWindow::closeSession() m_currentRecordingModelId = {}; m_recordingAsSingingTrack = false; m_singingAudioMutedForTake = false; + restorePlaySelectionAfterTake(); m_analysedMainModelId = {}; // Nothing is left waiting for a merge, and the history that holds the @@ -3497,6 +3503,29 @@ MainWindow::restoreSingingAudioAfterTake() updateLayerStatuses(); } +void +MainWindow::liftPlaySelectionForTake() +{ + // Playback constrained to the selection starts in the selection, not + // at the lead-in of a pre-roll or at a playhead outside it, and stops + // or loops at its end while the recording runs on. What was sung would + // then not be where the take puts it: the take counts the reference + // as playing on from playbackStart() without a break. Through the + // view manager, which writes no settings; the button follows it, and + // is greyed out meanwhile (updateMenuStates()) + if (!m_viewManager->getPlaySelectionMode()) return; + m_viewManager->setPlaySelectionMode(false); + m_playSelectionLiftedForTake = true; +} + +void +MainWindow::restorePlaySelectionAfterTake() +{ + if (!m_playSelectionLiftedForTake) return; + m_playSelectionLiftedForTake = false; + m_viewManager->setPlaySelectionMode(true); +} + void MainWindow::teardownSingingTrackAnalyser() { @@ -4205,6 +4234,7 @@ MainWindow::recordingStarted() m_recordingStartGapMeasured = -1; m_awaitingReferenceStart = true; + liftPlaySelectionForTake(); m_viewManager->setPlaybackFrame(playbackStart); m_playSource->play(playbackStart); } @@ -4376,6 +4406,7 @@ MainWindow::recordingFinishedFull(Analyser *analysing) if (m_audioIO) m_audioIO->suspend(); else if (m_playTarget) m_playTarget->suspend(); } + restorePlaySelectionAfterTake(); updateLayerStatuses(); updateMenuStates(); diff --git a/main/MainWindow.h b/main/MainWindow.h index d03ab0f0..94edfa79 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -775,6 +775,13 @@ protected slots: bool m_singingAudioAfterTake; void restoreSingingAudioAfterTake(); + // Playback constrained to the selection is lifted while the reference + // plays for a take, and put back when the take is over: true while it + // is lifted + bool m_playSelectionLiftedForTake; + void liftPlaySelectionForTake(); + void restorePlaySelectionAfterTake(); + // The main model last handed to m_analyser by analyseNewMainModel(). // audioFileLoaded() is emitted for additional models too (a singing // track, background music), and the reference must not be set up again diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index bc8f58d7..47475bac 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -1513,10 +1513,23 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; + + // Lifted for the take, and not to be put back during it + QTest::qWait(300); + QVERIFY2(!constrain->isChecked(), + "playback is constrained to the selection during a take"); + QVERIFY2(!constrain->isEnabled(), + "playback can be constrained to the selection during a " + "take"); + QTRY_VERIFY_WITH_TIMEOUT (!m_window->recordTarget()->isRecording(), 8000); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + // ... and as it was afterwards + QCOMPARE(constrain->isChecked(), constrained); + QVERIFY(constrain->isEnabled()); + double from = -1.0, to = -1.0; for (const auto &e : sv::ModelById::getAs (m_window->analyser2()->getLayer(Analyser::PitchTrack) @@ -1528,12 +1541,6 @@ private slots: qInfo("playback %s: the high note is in the take from %.3f to " "%.3f s", constrained ? "constrained" : "not constrained", from, to); - if (constrained) { - QEXPECT_FAIL("", "with playback constrained to the " - "selection the lead-in is not played: playback " - "starts at the selection, and what is sung to it " - "is placed a whole pre-roll too early", Continue); - } QVERIFY2(std::fabs(from - 2.0) < 0.05 && std::fabs(to - 2.75) < 0.05, qPrintable(QString("the high note of the reference, " "2.000 to 2.750 s, is in the take " From f1de6a2828a9af221199d6b4f18868778680a829 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:28:29 +0000 Subject: [PATCH 098/275] build: android toolchain, qt for android and the c libraries in the container setup-toolchain.sh installs the sdk, ndk r27c and jdk 21; build-qt.sh builds the host qt and qt for android 6.11.2 (qtbase, qtsvg) from source, as download.qt.io only redirects qt's binaries to mirrors the container cannot reach; build-deps.sh cross-compiles tony's libraries as static pic libraries into /opt/android/deps-arm64-v8a, writes the meson cross file, and links them all into an arm64 shared library to check. work orders: a2 done, its log entry, fftw is double precision. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/android/build-deps.sh | 511 ++++++++++++++++++++++++++++++ deploy/android/build-qt.sh | 241 ++++++++++++++ deploy/android/setup-toolchain.sh | 171 ++++++++++ docs/android-work-orders.md | 33 +- 4 files changed, 953 insertions(+), 3 deletions(-) create mode 100755 deploy/android/build-deps.sh create mode 100755 deploy/android/build-qt.sh create mode 100755 deploy/android/setup-toolchain.sh diff --git a/deploy/android/build-deps.sh b/deploy/android/build-deps.sh new file mode 100755 index 00000000..7716c8ec --- /dev/null +++ b/deploy/android/build-deps.sh @@ -0,0 +1,511 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Cross-compiles the libraries Tony links for Android (arm64-v8a, API +# 28) with NDK r27c, in the Ubuntu 24.04 cloud container: +# +# /opt/android/deps-arm64-v8a the libraries: include/, lib/ and +# lib/pkgconfig/ +# /opt/android/cross-arm64-v8a.ini the meson cross file for the NDK, +# used here and for Tony itself +# +# Run deploy/android/setup-toolchain.sh first, for the NDK and the +# build tools. +# +# All are static libraries of position-independent code, so that they +# end up inside Tony's own shared library and add no .so files to the +# APK. When Tony links them through pkg-config it needs their private +# dependencies too: meson's dependency(..., static: true) asks for them. +# +# What Tony needs, from meson.build's Linux branch and the library +# directories: +# libsndfile WAV, for reading and for writing takes; without its +# codec libraries (FLAC, Vorbis, Opus, MPEG) +# libsamplerate bqresample: playback and takes at another rate +# fftw3 bqfft; double precision only, as meson.build defines +# FFTW_DOUBLE_ONLY on every platform +# Rubber Band 3 playback at another speed (svapp), with its built-in +# FFT and resampler +# serd, sord dataquay's RDF store, with zix, which sord needs +# libmad MP3 reading; with libid3tag and the NDK's zlib +# libid3tag MP3 tags +# libogg, opus, Opus reading, read-only (HAVE_OPUS_READ_ONLY; no +# opusfile libopusenc); opusfile without its HTTP support +# bzip2 svcore's BZipFileDevice, which includes bzlib.h +# whatever the defines say; the NDK has no bzip2 +# Boost headers only: pYIN's boost/math +# Left out, as on a phone they have no use: oggz and fishsound (Ogg +# Vorbis), JACK, PulseAudio, ALSA and PortAudio, liblo (never enabled). +# +# Sources: release tarballs from GitHub or from Ubuntu's archive, whose +# pool keeps a file unchanged for good, each checked against the SHA-256 +# recorded below; Rubber Band is its Git tag, checked against the +# commit. (xiph.org, fftw.org, codeberg.org, download.drobilla.net and +# breakfastquay.com, the upstream sites, cannot be reached from the +# container.) +# +# Safe to run again. A library whose stamp in /share/tony-deps +# names the version below is left alone; delete the stamp to build it +# again. The cross file is written every time. It ends by building a +# small shared library that uses every library, found through +# pkg-config with the cross file as Tony's build will, and checking +# that it is an arm64 shared object with none of them left to load. +# +# Usage, from anywhere: +# deploy/android/build-deps.sh + +set -eu -o pipefail + +if [ "$#" -ne 0 ]; then + echo "Usage: $0" 1>&2 + exit 2 +fi + +android=/opt/android +ndk=$android/sdk/ndk/27.2.12479018 +logs=$android/logs + +abi=arm64-v8a +api=28 +triple=aarch64-linux-android +prefix=$android/deps-$abi +cross=$android/cross-$abi.ini + +toolchain=$ndk/toolchains/llvm/prebuilt/linux-x86_64 +cc=$toolchain/bin/$triple$api-clang +cxx=$toolchain/bin/$triple$api-clang++ + +jobs=$(nproc) + +ubuntu=https://archive.ubuntu.com/ubuntu/pool +github=https://github.com + +if [ ! -x "$cc" ]; then + echo "ERROR: no NDK at $ndk: run deploy/android/setup-toolchain.sh first" 1>&2 + exit 1 +fi + +mkdir -p "$logs" "$prefix/share/tony-deps" +work=$(mktemp -d "$android/tmp.XXXXXX") +trap 'rm -rf "$work"' EXIT + +# 1. The cross file. c_args and link args apply to what is built for +# Android. The page size is for Android 15's 16 KB pages, which NDK r27 +# does not set by itself. There is no sys_root: meson would prefix it +# to every path in the prefix's .pc files. boost_root keeps meson's +# Boost lookup out of the build machine's /usr. + +cat > "$cross" < "$prefix/lib/pkgconfig/bzip2.pc" <&2 + exit 1 + fi + meson_build "$work/rubberband" -Dfft=builtin -Dresampler=builtin \ + -Djni=disabled -Dladspa=disabled -Dlv2=disabled -Dvamp=disabled \ + -Dcmdline=disabled -Dtests=disabled +} + +build_boost() { + # Ubuntu 24.04's Boost, as the desktop build. The tarball is Boost's + # modular layout, each library's headers under libs//include + # (numeric/ for four): merged here as Boost's "b2 headers" + # would. Tony uses only headers. + local archive=$work/boost.tar.xz + curl -sSfL --retry 5 --retry-all-errors -o "$archive" \ + "$ubuntu/main/b/boost1.83/boost1.83_1.83.0.orig.tar.xz" + echo "404df4b4072fc7f2d4483d4fc2d61ff6f554dd80c9a812652684d5952e881c91 $archive" | + sha256sum -c --quiet + tar xf "$archive" -C "$work" --wildcards 'boost/libs/*/include/boost/*' + rm "$archive" + rm -rf "$prefix/include/boost" + mkdir -p "$prefix/include/boost" + local include + for include in "$work"/boost/libs/*/include "$work"/boost/libs/numeric/*/include; do + cp -R "$include/boost/." "$prefix/include/boost/" + done + grep -q '^#define BOOST_LIB_VERSION "1_83"' "$prefix/include/boost/version.hpp" +} + +install_lib() { + local name="$1" version="$2" + local stamp=$prefix/share/tony-deps/$name + local log=$logs/deps-$name.log + if [ -f "$stamp" ] && [ "$(cat "$stamp")" = "$version" ]; then + echo " $name $version: installed already" + return + fi + echo " $name $version: building (log: $log)" + rm -f "$stamp" + # Not in a condition, or -e would not hold inside the subshell + set +e + ( set -e; "build_$name" ) > "$log" 2>&1 + local status=$? + set -e + if [ "$status" -ne 0 ]; then + tail -30 "$log" 1>&2 + echo "ERROR: $name failed; the whole log is $log" 1>&2 + exit 1 + fi + rm -rf "${work:?}"/* + echo "$version" > "$stamp" +} + +echo +echo "Libraries in $prefix:" +install_lib bzip2 1.0.8 +install_lib ogg 1.3.5 +install_lib opus 1.5.2 +install_lib opusfile 0.12 +install_lib sndfile 1.2.2 +install_lib samplerate 0.2.2 +install_lib fftw3 3.3.10 +install_lib mad 0.16.4 +install_lib id3tag 0.16.3 +install_lib zix 0.4.2 +install_lib serd 0.32.2 +install_lib sord 0.16.16 +install_lib rubberband 3.3.0 +install_lib boost 1.83.0 + +shared=$(find "$prefix/lib" -name "*.so*") +if [ -n "$shared" ]; then + echo "ERROR: shared libraries in $prefix/lib, where there should be none:" 1>&2 + echo "$shared" 1>&2 + exit 1 +fi + +# 4. Check: a shared library using every library, found as Tony's +# build will find them. meson links shared libraries with +# --no-undefined, so a missing library fails the link. + +echo +echo "Checking" +check=$work/check +mkdir -p "$check" +cat > "$check/meson.build" <<'EOF' +project('tony-deps-check', 'cpp', default_options: ['cpp_std=c++17']) +deps = [dependency('boost')] +foreach name : ['sndfile', 'samplerate', 'fftw3', 'rubberband', 'sord-0', + 'serd-0', 'mad', 'id3tag', 'opusfile', 'bzip2'] + deps += dependency(name, static: true) +endforeach +shared_library('tonydepscheck', 'check.cpp', dependencies: deps) +EOF +cat > "$check/check.cpp" <<'EOF' +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +// Something from each library, so that the static linker has to find it +extern "C" int tony_deps_check(const unsigned char *data, int size) +{ + int n = sf_version_string()[0]; + n += src_is_valid_ratio(2.0); + + double *in = fftw_alloc_real(16); + fftw_complex *out = fftw_alloc_complex(9); + fftw_plan plan = fftw_plan_dft_r2c_1d(16, in, out, FFTW_ESTIMATE); + fftw_execute(plan); + fftw_destroy_plan(plan); + fftw_free(out); + fftw_free(in); + + RubberBand::RubberBandStretcher stretcher + (44100, 1, RubberBand::RubberBandStretcher::OptionEngineFiner); + n += int(stretcher.getStartDelay()); + + SordWorld *world = sord_world_new(); + n += int(sord_num_nodes(world)); + sord_world_free(world); + n += serd_strerror(SERD_SUCCESS)[0]; + + struct mad_stream stream; + struct mad_frame frame; + mad_stream_init(&stream); + mad_frame_init(&frame); + mad_stream_buffer(&stream, data, size); + n += mad_frame_decode(&frame, &stream); + mad_frame_finish(&frame); + mad_stream_finish(&stream); + + struct id3_tag *tag = id3_tag_parse(data, size); + if (tag) id3_tag_delete(tag); + + int error = 0; + OggOpusFile *opus = op_open_memory(data, size, &error); + if (opus) op_free(opus); + n += error; + + n += BZ2_bzlibVersion()[0]; + + boost::math::normal_distribution normal(0.0, 1.0); + n += int(boost::math::cdf(normal, double(size)) * 10); + + return n; +} +EOF +log=$logs/deps-check.log +set +e +( set -e + meson setup "$check/build" "$check" --cross-file "$cross" --buildtype release + ninja -C "$check/build" ) > "$log" 2>&1 +status=$? +set -e +if [ "$status" -ne 0 ]; then + tail -30 "$log" 1>&2 + echo "ERROR: the check did not build; the whole log is $log" 1>&2 + exit 1 +fi + +so=$check/build/libtonydepscheck.so +kind=$(file -b "$so") +needed=$("$toolchain/bin/llvm-readelf" -d "$so" | sed -n 's/.*(NEEDED).*\[\(.*\)\]/\1/p' | tr '\n' ' ') +align=$("$toolchain/bin/llvm-readelf" -lW "$so" | awk '$1 == "LOAD" { print $NF }' | sort -u | tr '\n' ' ') +echo " $(basename "$so"): $(echo "$kind" | cut -d, -f1-2)" +echo " it loads: $needed" +echo " segment alignment: $align" +case "$kind" in + "ELF 64-bit LSB shared object, ARM aarch64"*) ;; + *) echo "ERROR: not an arm64 shared object" 1>&2; exit 1 ;; +esac +for lib in $needed; do + case "$lib" in + libc.so|libm.so|libdl.so|libz.so|libc++_shared.so) ;; + *) echo "ERROR: it loads $lib, which is not part of Android or the NDK" 1>&2; exit 1 ;; + esac +done +if [ "$align" != "0x4000 " ]; then + echo "ERROR: its segments are not aligned for 16 KB pages" 1>&2 + exit 1 +fi + +cat <&2 + exit 2 +fi + +android=/opt/android +sdk=$android/sdk +ndk=$sdk/ndk/27.2.12479018 +logs=$android/logs + +qt_version=6.11.2 +qtbase_commit=ef55f427f2c8b410d34f8a7681020a3000cf6866 +qtsvg_commit=17ca512f903f935282ebeca496aac5d11ba4199a + +qt=$android/qt/$qt_version +qt_host=$qt/gcc_64 +qt_android=$qt/android_arm64_v8a + +jobs=$(nproc) + +export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-$(dpkg --print-architecture) + +# Qt's tools warn at every start in the container's C locale +export LC_ALL=C.UTF-8 + +if [ ! -x "$JAVA_HOME/bin/javac" ] || [ ! -f "$ndk/source.properties" ] || + [ ! -d "$sdk/platforms/android-36" ]; then + echo "ERROR: no JDK 21, NDK or SDK platform: run deploy/android/setup-toolchain.sh first" 1>&2 + exit 1 +fi + +mkdir -p "$logs" +work=$(mktemp -d "$android/tmp.XXXXXX") +trap 'rm -rf "$work"' EXIT + +# Runs a command with its output in a log file, and shows the end of +# the log if it fails +logged() { + local log="$1" + shift + if ! "$@" >> "$log" 2>&1; then + tail -30 "$log" 1>&2 + echo "ERROR: failed; the whole log is $log" 1>&2 + exit 1 + fi +} + +fetch() { + local name="$1" commit="$2" + if [ ! -d "$work/$name" ]; then + echo " fetching $name $qt_version" + git clone -q -c advice.detachedHead=false --depth 1 \ + --branch "v$qt_version" "https://github.com/qt/$name.git" "$work/$name" + if [ "$(git -C "$work/$name" rev-parse HEAD)" != "$commit" ]; then + echo "ERROR: tag v$qt_version of $name is not at $commit" 1>&2 + exit 1 + fi + fi +} + +qt_at() { + if [ -x "$1/bin/qmake" ]; then + "$1/bin/qmake" -query QT_VERSION 2>/dev/null || true + fi +} + +# Configures, builds and installs one Qt module from source directory +# $3, in build directory $2, logging to $1; the rest are the +# configure arguments +build_module() { + local log="$1" build="$2" configure="$3" + shift 3 + rm -f "$log" + mkdir -p "$build" + (cd "$build" && logged "$log" "$configure" "$@") + logged "$log" cmake --build "$build" --parallel "$jobs" + logged "$log" cmake --install "$build" + rm -rf "$build" +} + +# 1. The host Qt + +echo +if [ "$(qt_at "$qt_host")" = "$qt_version" ]; then + echo "Host Qt $qt_version: in $qt_host already" +else + echo "Building host Qt $qt_version in $qt_host (log: $logs/qt-host.log)" + start=$SECONDS + fetch qtbase "$qtbase_commit" + rm -rf "$qt_host" + build_module "$logs/qt-host.log" "$work/build-host" "$work/qtbase/configure" \ + -prefix "$qt_host" -release -nomake tests -nomake examples \ + -qt-zlib -qt-pcre -qt-doubleconversion -qt-freetype -qt-harfbuzz \ + -qt-libpng -qt-libjpeg -no-icu -no-glib -no-dbus -no-opengl \ + -no-xcb -no-gtk -no-feature-sql -no-feature-printsupport + echo " done in $(( (SECONDS - start) / 60 )) min" +fi + +# 2. Qt for Android: qtbase, then qtsvg on top of it + +echo +if [ "$(qt_at "$qt_android")" = "$qt_version" ] && + [ -f "$qt_android/lib/libQt6Core_arm64-v8a.so" ]; then + echo "Qt $qt_version for Android: in $qt_android already" +else + echo "Building Qt $qt_version for Android in $qt_android (log: $logs/qt-android.log)" + start=$SECONDS + fetch qtbase "$qtbase_commit" + rm -rf "$qt_android" + build_module "$logs/qt-android.log" "$work/build-android" "$work/qtbase/configure" \ + -platform android-clang -prefix "$qt_android" \ + -android-sdk "$sdk" -android-ndk "$ndk" -android-abis arm64-v8a \ + -qt-host-path "$qt_host" -release -nomake tests -nomake examples \ + -no-feature-sql -no-feature-printsupport + echo " done in $(( (SECONDS - start) / 60 )) min" +fi + +echo +if [ -f "$qt_android/lib/libQt6Svg_arm64-v8a.so" ]; then + echo "Qt Svg for Android: in $qt_android already" +else + echo "Building Qt Svg $qt_version for Android (log: $logs/qtsvg-android.log)" + fetch qtsvg "$qtsvg_commit" + build_module "$logs/qtsvg-android.log" "$work/build-svg" \ + "$qt_android/bin/qt-configure-module" "$work/qtsvg" +fi + +# 3. Check. qt-cmake configures and builds a small program using the +# modules Tony uses, as the Android build will: this needs Qt for +# Android, the host Qt, the NDK and the SDK to agree. + +echo +echo "Checking" +echo " host Qt: $("$qt_host/bin/qmake" -query QT_VERSION) at $("$qt_host/bin/qmake" -query QT_INSTALL_PREFIX)" +echo " Android Qt: $("$qt_android/bin/qmake" -query QT_VERSION) at $("$qt_android/bin/qmake" -query QT_INSTALL_PREFIX), host prefix $("$qt_android/bin/qmake" -query QT_HOST_PREFIX)" + +check=$work/check +mkdir -p "$check/src" +cat > "$check/src/CMakeLists.txt" <<'EOF' +cmake_minimum_required(VERSION 3.22) +project(check LANGUAGES CXX) +find_package(Qt6 REQUIRED COMPONENTS Core Gui Widgets Xml Network Svg Test) +qt_add_executable(check check.cpp) +target_link_libraries(check PRIVATE Qt6::Widgets Qt6::Xml Qt6::Network Qt6::Svg) +EOF +cat > "$check/src/check.cpp" <<'EOF' +#include +#include +#include +#include +#include +int main(int argc, char **argv) +{ + QApplication app(argc, argv); + QDomDocument doc; + QNetworkAccessManager network; + QSvgRenderer svg; + QLabel label("Tony"); + label.show(); + return app.exec(); +} +EOF +log=$logs/qt-check.log +rm -f "$log" +logged "$log" "$qt_android/bin/qt-cmake" -S "$check/src" -B "$check/build" -G Ninja \ + -DCMAKE_BUILD_TYPE=Release -DANDROID_SDK_ROOT="$sdk" -DANDROID_NDK_ROOT="$ndk" +# The library only: the default target goes on to run androiddeployqt +# and Gradle, which is the APK build's business +logged "$log" cmake --build "$check/build" --target check +echo " qt-cmake: $(file -b "$check/build/libcheck_arm64-v8a.so" | cut -d, -f1-2)" +# The deployment settings Qt's CMake support wrote, kept as a model for +# the file Tony's build has to write for androiddeployqt +cp "$check/build/android-check-deployment-settings.json" "$logs/" + +# It prints its usage and exits with 1 +log=$logs/androiddeployqt-help.log +"$qt_host/bin/androiddeployqt" --help > "$log" 2>&1 || true +if ! grep -q -- "--output " "$log"; then + cat "$log" 1>&2 + echo "ERROR: androiddeployqt does not run" 1>&2 + exit 1 +fi +echo " androiddeployqt: runs" + +cat <&2 + exit 2 +fi + +sudo="" +if [ "$(id -u)" -ne 0 ]; then + sudo=sudo +fi + +android=/opt/android +sdk=$android/sdk + +# The command-line tools 22.0: the last release whose sdkmanager is +# the Java one. In 23.0 sdkmanager became a wrapper around a new +# "android" tool, with other package names. The SHA-1 is the one +# Google's repository2-3.xml gives; the SHA-256 was recorded here. +cmdline_tools_build=15859902 +cmdline_tools_sha1=040d3996a65543d22ec4bf73e4c37aa37a8d4af4 +cmdline_tools_sha256=4e4c464f145a7512b57d088ac6c278c03c9eea610886b35a5e0804e74eedf583 + +ndk_version=27.2.12479018 +ndk=$sdk/ndk/$ndk_version + +# sdkmanager takes the latest revision of each of these, and checks +# the SHA-1 of what it downloads. The NDK's path names its exact version. +sdk_packages="platforms;android-36 build-tools;36.0.0 platform-tools ndk;$ndk_version" + +# 1. Packages: the JDK, the host compiler and build tools for Qt and +# the libraries, and "file" for the checks. The JDK is found by its +# path, not by /usr/bin/java, which another JDK may own. + +jdk_package=openjdk-21-jdk-headless +java_home=/usr/lib/jvm/java-21-openjdk-$(dpkg --print-architecture) + +packages=" +$jdk_package curl ca-certificates unzip xz-utils git patch file +build-essential pkg-config cmake ninja-build meson +" + +missing="" +for p in $packages; do + if ! dpkg-query -W -f='${Status}' "$p" 2>/dev/null | grep -q "install ok installed"; then + missing="$missing $p" + fi +done + +if [ -n "$missing" ]; then + echo "Installing packages:$missing" + # Some of the container's own apt sources (PPAs) are blocked; + # apt-get update warns about them and carries on. + $sudo apt-get update -q + $sudo env DEBIAN_FRONTEND=noninteractive \ + apt-get install -y -q --no-install-recommends $missing +else + echo "Packages: all installed" +fi + +if [ ! -x "$java_home/bin/javac" ]; then + echo "ERROR: no JDK 21 at $java_home" 1>&2 + exit 1 +fi +export JAVA_HOME=$java_home + +# Everything below runs as the user who owns /opt/android +if [ ! -d "$android" ]; then + $sudo mkdir -p "$android" + $sudo chown "$(id -u):$(id -g)" "$android" +fi +work=$(mktemp -d "$android/tmp.XXXXXX") +trap 'rm -rf "$work"' EXIT + +# 2. The SDK's command-line tools, in the layout sdkmanager expects: +# sdk/cmdline-tools/latest. + +sdkmanager=$sdk/cmdline-tools/latest/bin/sdkmanager + +echo +if [ -x "$sdkmanager" ]; then + echo "SDK command-line tools: installed already" +else + echo "Installing the SDK command-line tools ($cmdline_tools_build) in $sdk" + zip=$work/cmdline-tools.zip + # The container's proxy now and then answers 502 for a moment + curl -sSfL --retry 5 --retry-all-errors -o "$zip" \ + "https://dl.google.com/android/repository/commandlinetools-linux-${cmdline_tools_build}_latest.zip" + echo "$cmdline_tools_sha1 $zip" | sha1sum -c --quiet + echo "$cmdline_tools_sha256 $zip" | sha256sum -c --quiet + unzip -q "$zip" -d "$work" + rm "$zip" + mkdir -p "$sdk/cmdline-tools" + rm -rf "$sdk/cmdline-tools/latest" + mv "$work/cmdline-tools" "$sdk/cmdline-tools/latest" +fi + +# 3. SDK packages and the NDK. Installing them means accepting the +# SDK's licences, as every unattended Android build does. + +missing="" +for p in $sdk_packages; do + if [ ! -f "$sdk/${p//;//}/source.properties" ]; then + missing="$missing $p" + fi +done + +echo +if [ -n "$missing" ]; then + echo "Installing SDK packages:$missing" + log=$work/sdkmanager.log + { yes || true; } | "$sdkmanager" --sdk_root="$sdk" --licenses > "$log" 2>&1 || + { tail -20 "$log" 1>&2; exit 1; } + "$sdkmanager" --sdk_root="$sdk" --install $missing >> "$log" 2>&1 || + { tail -20 "$log" 1>&2; exit 1; } + for p in $missing; do + if [ ! -f "$sdk/${p//;//}/source.properties" ]; then + tail -20 "$log" 1>&2 + echo "ERROR: sdkmanager did not install $p" 1>&2 + exit 1 + fi + done +else + echo "SDK packages: all installed" +fi + +if ! grep -q "^Pkg.Revision = $ndk_version\$" "$ndk/source.properties"; then + echo "ERROR: $ndk is not NDK $ndk_version" 1>&2 + exit 1 +fi + +cat < Date: Sat, 26 Sep 2026 00:29:31 +0000 Subject: [PATCH 099/275] docs: a3 split into building tony for android and packaging the apk Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 42 +++++++++++++++++++++++++++++-------- 1 file changed, 33 insertions(+), 9 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 58f25440..287a1b12 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -142,7 +142,7 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: ## 5. Phases -Order: A0, A1, A2, A3, then A4 and A5 while the user tries the APK on the phone. A6 +Order: A0, A1, A2, A3a, A3b, then A4 and A5 while the user tries the APK on the phone. A6 needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt.io` and `dl.google.com` are reachable from the container. GitHub workflows are turned off: all builds happen in the container.) @@ -150,7 +150,8 @@ builds happen in the container.) - A0 — Desktop build and tests in the container. Done. - A1 — Sample rate: a device that is not at 44.1 kHz. Done. - A2 — Android toolchain and C libraries. Done. -- A3 — Tony as an APK (no audio): the test port. +- A3a — Tony builds for Android. +- A3b — Tony as an APK (no audio): the test port. - A4 — Touch gestures on the panes. - A5 — Compact touch mode. - A6 — Oboe audio backend. @@ -219,17 +220,40 @@ tests". - The toolchain is installed into the container (outside the repo, e.g. under `/opt`), so that A3 can iterate locally. -### A3 — Tony as an APK, without audio: the test port +### A3a — Tony builds for Android + +Read: [port-android.md](port-android.md) "Qt for Android", "Build and packaging"; +[mobile-port.md](mobile-port.md) "Build and tests"; [forks.md](forks.md) "Changing a fork"; +the A2 log entry. + +- Route (a) from section 3: an `android` branch in `meson.build` (today an unknown system + is an error), the cross file from A2 plus whatever Tony needs on top (Qt's tools), and + Tony built as the shared library `libTony_arm64-v8a.so` that Qt for Android loads, with + `main` exported. The test executables are not built for Android. +- A script, `deploy/android/build-tony.sh`, that configures and builds it into its own + build directory (not `build/`). +- The pYIN and CHP plugins cross-compiled as well; how they are packaged is A3b's. +- Fixes needed for Android in the fork directories (`svcore/`, `svgui/`, `svapp/`, + `bqaudiostream/`) may be made there this once, smallest possible, guarded for Android + where they would change anything elsewhere, uncommitted, and listed in the report; the + lead commits and pushes them. The upstream libraries (`bqaudioio/`, `bqvec/`, + `dataquay/`, `vamp-plugin-sdk/`, `checker/`, `pyin/`, ...) stay untouched: work around + in `meson.build`, or report. +- The desktop build and suites unchanged and green. +- Result: the library links; `readelf` shows it exports `main` and needs nothing outside + the NDK's system libraries and Qt's. + +### A3b — Tony as an APK, without audio: the test port Read: [port-android.md](port-android.md) all of "Platform facts" and "Test port"; -[mobile-port.md](mobile-port.md) "The pYIN plugin", "The window", "Build and tests". +[mobile-port.md](mobile-port.md) "The pYIN plugin", "The window". -- Route (a) from section 3: an `android` branch in `meson.build`, an NDK cross file, Tony - built as `libTony_arm64-v8a.so`, a script that writes the deployment JSON and runs - androiddeployqt, a custom `AndroidManifest.xml` (with `RECORD_AUDIO` for later). +- A script that writes the deployment JSON (A2 left a model written by Qt's CMake in + `/opt/android/logs/android-check-deployment-settings.json`) and runs androiddeployqt; a + custom `AndroidManifest.xml` (with `RECORD_AUDIO` for later). - Audio: `AUDIO_NONE` under `Q_OS_ANDROID` for now. - The menu bar: exclude `Q_OS_ANDROID` from the `Q_OS_LINUX` `setNativeMenuBar(false)` - only if the ⋮ options menu works better on a phone; otherwise keep the in-window bar and + only if the options menu works better on a phone; otherwise keep the in-window bar and say why. - pYIN found on the phone: `libpyin.so` naming plus legacy packaging, or linking it in. The log must show "Setting VAMP_PATH to ...". @@ -264,7 +288,7 @@ Read: [mobile-port.md](mobile-port.md) "The window", "Work common to both ports" ### A6 — Oboe audio backend -After the user's phone test of A3. Detailed when it starts. +After the user's phone test of A3b. Detailed when it starts. ### A7 — Android files, permission and lifecycle From 58de0748c45c5c536a9017885d72de0dfb3f8590 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:44:58 +0000 Subject: [PATCH 100/275] feat: an audio check that records ordinary takes against a test reference AudioCheckRunner writes the calibration reference, opens it as the session, records punch-ins into selections derived from its layout with the reference playing, and judges the take's own file with judgeTake(): the file, because the model is normalised to full scale and resampled as it is read. It reports the round trip each take used, the reported latencies, and the recording's rate, flagging a rate that differs from the reference's whatever the sweeps say. The check's takes record into the selection, play the reference and have a 1 s lead-in through an override in MainWindow, never through the toggles, which write the user's settings. It is driven by a polling timer, not nested event loops; closing the session or the window ends it. TestAudioCheck runs it on the loopback fake; removing each half of the override, or the close hook, was seen to fail a test. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green (TestAudioCheck 8, TestRecordWorkflow 92, TestSingingAnalysis 20, TestSingingDocument 12). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 19 ++ docs/calibrate-audio.md | 30 +- main/AudioCheckRunner.cpp | 445 ++++++++++++++++++++++++++++ main/AudioCheckRunner.h | 214 +++++++++++++ main/LatencyCheck.cpp | 34 +++ main/LatencyCheck.h | 20 ++ main/LatencyUtils.h | 30 ++ main/MainWindow.cpp | 60 +++- main/MainWindow.h | 24 +- main/test/TestAudioCheck.h | 398 +++++++++++++++++++++++++ main/test/TestLatencyCheck.h | 67 +++++ main/test/TestRecordWorkflow.h | 11 + main/test/tony-app-test.cpp | 7 + meson.build | 3 + 14 files changed, 1339 insertions(+), 23 deletions(-) create mode 100644 main/AudioCheckRunner.cpp create mode 100644 main/AudioCheckRunner.h create mode 100644 main/test/TestAudioCheck.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 14137634..d659c9b2 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -354,3 +354,22 @@ The next phase must know: - §2's punch-ins as [2,7], [8,13], [14,19], [20,25] s judge 7 events (2, 2, 2, 1). A punch-in shorter than 1.95 s judges none. - An echo under 20 ms (an interface's direct monitor) is not seen. Left open: every threshold untuned. The verdict thresholds came from the lead's brief; spec §5 has none of them. + +### Phase B1 — 2026-09-26 +Built: `LatencyCheck::punchInsFor()` (+ `kPunchInSlackSeconds`, 10 ms) and core test `punch_ins_hold_the_events_asked_for`. `TakeLatency` in `LatencyUtils.h`. `main/AudioCheckRunner.{h,cpp}` (`tony_app`): `Plan`, `start()`, `cancel()`, `sessionClosing()`, `finished(AudioCheckResult)`. `MainWindow`: `friend class AudioCheckRunner`, the override `m_audioCheckTakes` (read by `record()`, the `recordingStarted()` lambda, `wantedPreRollFrames()`), `m_takeLatency`, the runner made in the constructor, deleted first in `~MainWindow`, told by `closeSession()`. New app class `main/test/TestAudioCheck.h`: 6 tests, 32 s. +Choices / deviations: +- **4 × 3 does not fit the calibration layout** (spec §2 now says why). `punchInsFor()` returns nothing then; 4 × 2 and 3 × 3 fit. B3 needs the lead's choice. +- The take is read from its **file**, not its model: the model is peak-normalised as read (measured: every take Clipped) and resampled to 44.1 kHz. +- `friend` over accessors: the runner needs about nine internals. A new test class, not `TestRecordWorkflow` (5475 lines); its watchdog and init/cleanup are copied. +- Selection: `clearSelections()` + `addSelectionQuietly()`, so the reference is not re-analysed during a take; each punch-in adds one or two "Select" undo steps, as a user's selection does. +- Waits: poll every 50 ms for "nothing being analysed", not "analysed": with auto-analysis off the reference never gets layers (test `check_runs_without_automatic_analysis`). Limits 60 s reference, 30 s a take's analysis, take length + 10 s to stop (then the Stop path); each ends the run with a reason. +- `~MainWindow` deletes the runner: the run ends silently and a take in progress is left to the destructor (the Stop path would splice and start pYIN mid-teardown). `closeSession()` → Stop path + `finished()`. +- Reference: `AppDataLocation/calibrate-audio-reference.wav` unless the plan names a path (tests: their temp dir). Save question first, then write, then open. +- `calibrationUsable()`: Ok or Unsteady, and no rate mismatch. +- No loopback gain was needed (see below). +The next phase must know: +- `Analyser` pans the reference hard left, sonification hard right: only the **left earcup** carries sweeps. Normalised, the reference plays at 0 dBFS, not −12. The fake averages channels: sweeps loop back at half level (peak 0.905 with the synth). +- `getTargetPlayLatency()` counts session frames, `getSystemRecordLatency()` device frames; the result converts each at its own rate. At 48 kHz the round trip used was 242 ms, not 256. +- 48 kHz fake: Scattered, 3 of 3 found, offsets −200 ms median, as A2 foresaw. +- No progress signal yet; B3's dialog may want one (`m_punchIn`). +Left open: no test deletes the window mid-check. Seen while proving the session-close hook: closing a session during an **ordinary** take, then pressing Stop, hangs (pre-existing). diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 26d440ff..fd593e19 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -45,6 +45,12 @@ with less noise. not from fixed times: a 5 s punch-in judges only one or two sweeps. They use the ordinary take path with Play Reference While Recording on. Every punch-in restarts the stream, as a real take does. + *Found in B1:* the calibration layout cannot hold four times three. Where one + punch-in ends and the next begins, two sweeps must be 1.9 s apart (what the finder + reads around each), and the layout's boundaries after its 3rd, 6th and 9th sweeps + are 2.5, 1.7 and 1.8 s. `punchInsFor()` returns nothing for 4 × 3; 4 × 2 and 3 × 3 + fit. Swapping two pairs of spacings (`{21,16,25,19,17,23,26,20,24,18,22}`) would + make 4 × 3 fit, ending at 25.2 s. 4. **Result page:** - **Round trip:** measured, next to the driver's figure. - **Spread between punch-ins:** how much the driver's timing moves from one stream @@ -280,7 +286,7 @@ marked "Done" when it is committed. - **A2** Verdicts and calibration arithmetic: aggregation over events and punch-ins. Done. 2. **Runner, dialog and calibration page** (every build), with app tests. - - **B1** The alignment check runner and its app tests. + - **B1** The alignment check runner and its app tests. Done. - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, `recordingStarted()`, staleness). This was step 3 below; it moved up because the dialog needs it. @@ -325,7 +331,9 @@ could convert. The button then shows the fix working on each device. and Cancel must always leave a clean state. `closeSession()` already stops take polling. - **Loudness.** The sweeps are −12 dBFS with earcups off the ears; the dialog says so - before starting. + before starting. *Found in B1:* not as played. Tony normalises every audio file to + full scale as it reads it (`Preferences::setNormaliseAudio(true)`), so the reference + plays at 0 dBFS, in the left channel only (§11). - **Cursor versus dots.** The cursor subtracts the *reported* output latency. With a measured round trip the dots move to the right place and may sit off the cursor. Item 8's number will show how much. Fixing it needs the round trip split between output @@ -363,7 +371,13 @@ Checked on 2026-09-25, so that phases do not re-derive them. start gap comes from the play-start callback set in `MainWindow`'s constructor: `getFramesReceived() − blockFrames` on the first output block with audio. `refineRecordingLatency()` and `currentRecordingLatency()` swap the estimate for the - measurement. + measurement. *Found in B1:* `getTargetPlayLatency()` counts frames of the session's + rate (bqaudioio's `ResamplerWrapper` converts it), `getSystemRecordLatency()` the + device's, and L is taken off the recording, in the device's. They differ only when + the device is not at 44.1 kHz. +- **Where the reference is heard** (found in B1). `Analyser` pans the reference hard + left and its pitch and notes sonification hard right, so only the left earcup + carries the sweeps; the right one carries the synth. - **bqaudioio `PortAudioIO`** (upstream, not a fork): - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; - input goes to the record target **before** output is asked for, in the same @@ -395,14 +409,18 @@ Checked on 2026-09-25, so that phases do not re-derive them. the layers exist. See `analysed()` in `TestRecordWorkflow.h`. - **The take after Stop.** - Its audio is the model `analyser2()->getMainModelId()`, and its file is - `m_takes->getAudioPath()`. + `m_takes->getAudioPath()`. *Found in B1:* the model is normalised to full scale + and resampled to 44.1 kHz as it is read, so every take read from it is Clipped; + the check reads the file. - Coverage is `m_takes->getCoverage().getRanges()`. - The take is analysed when `analysed(analyser2())` holds. - **Levels.** `getOutputLevels()` and `getInputLevels()`, on the play source and record target, return per-channel peaks since the last call. - **Fake device.** `FakeAudioIO::Config::loopback` adds the output to the input - `inputDelay` frames late; no test uses it yet. The reported latencies are independent - of the real delay. `TestMainWindow::createAudioIO()` installs the fake. + `inputDelay` frames late; `TestAudioCheck` uses it. The reported latencies are + independent of the real delay. `TestMainWindow::createAudioIO()` installs the fake. + Its output is the mean of the channels, so the hard-left reference loops back at + half level. - **Menus.** The Playback menu is built in `MainWindow::setupToolbars()` (`m_playbackMenu`). The audio device submenus are there too. - **Build types.** `build.bat` uses `debugoptimized`; `meson.build` defaults to diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp new file mode 100644 index 00000000..1fc9aa67 --- /dev/null +++ b/main/AudioCheckRunner.cpp @@ -0,0 +1,445 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AudioCheckRunner.h" + +#include "MainWindow.h" +#include "Analyser.h" +#include "SingingTakes.h" + +#include "audio/AudioCallbackRecordTarget.h" +#include "base/Selection.h" +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" +#include "data/model/WaveFileModel.h" +#include "transform/ModelTransformerFactory.h" +#include "view/ViewManager.h" + +#include +#include +#include +#include + +#include +#include + +using std::cerr; +using std::endl; +using std::vector; + +using namespace sv; + +bool +AudioCheckResult::calibrationUsable() const +{ + // A take recorded at another rate is misplaced by an amount that + // grows with its position, so no one round trip places it right + if (failure != "" || rateMismatch) return false; + return summary.verdict == LatencyCheck::Verdict::Ok || + summary.verdict == LatencyCheck::Verdict::Unsteady; +} + +AudioCheckRunner::AudioCheckRunner(MainWindow *window) : + QObject(window), + m_window(window), + m_timer(new QTimer(this)), + m_step(Step::Idle), + m_punchIn(0), + m_openingReference(false), + m_inPoll(false), + m_stepLimitMs(0) +{ + m_timer->setInterval(kPollMs); + connect(m_timer, &QTimer::timeout, this, &AudioCheckRunner::poll); +} + +AudioCheckRunner::~AudioCheckRunner() +{ + // Deleted by the window first thing in its destructor, so the window + // is whole here. A run ends without a word: whoever was waiting for + // it goes with the window. A take in progress is left to the window, + // as any take is when it is deleted: stopping it here would splice + // it and start its analysis in the middle of the window's teardown + m_timer->stop(); + if (m_step != Step::Idle) { + cerr << "AudioCheckRunner: the window is going; the run ends" << endl; + m_window->m_audioCheckTakes = false; + m_step = Step::Idle; + } +} + +QString +AudioCheckRunner::defaultReferencePath() +{ + QString dir = + QStandardPaths::writableLocation(QStandardPaths::AppDataLocation); + return QDir(dir).filePath("calibrate-audio-reference.wav"); +} + +bool +AudioCheckRunner::start(const Plan &plan) +{ + if (m_step != Step::Idle) return false; + + if (m_window->m_recordTarget && m_window->m_recordTarget->isRecording()) { + cerr << "AudioCheckRunner::start: a take is being recorded" << endl; + return false; + } + + vector punchIns = LatencyCheck::punchInsFor + (plan.layout, plan.punchIns, plan.eventsEach); + if (punchIns.empty()) { + cerr << "AudioCheckRunner::start: the layout has no room for " + << plan.punchIns << " punch-ins of " << plan.eventsEach + << " events" << endl; + return false; + } + + m_plan = plan; + if (m_plan.referencePath == "") { + m_plan.referencePath = defaultReferencePath(); + } + m_result = AudioCheckResult(); + m_punchIns = punchIns; + m_starts.clear(); + m_ends.clear(); + m_punchIn = 0; + + cerr << "AudioCheckRunner::start: " << m_punchIns.size() + << " punch-ins against " << m_plan.referencePath << endl; + + // Nothing is done before the first poll, so that however the run + // ends, the caller hears of it through finished() and never from + // inside this call + setStep(Step::OpeningReference, 0); + m_timer->start(); + return true; +} + +void +AudioCheckRunner::cancel() +{ + if (m_step == Step::Idle) return; + stopTake(); + end(tr("The check was cancelled.")); +} + +bool +AudioCheckRunner::isRunning() const +{ + return m_step != Step::Idle; +} + +void +AudioCheckRunner::sessionClosing() +{ + // Opening the reference closes the session it replaces + if (m_step == Step::Idle || m_openingReference) return; + stopTake(); + end(tr("The session was closed during the check.")); +} + +void +AudioCheckRunner::poll() +{ + if (m_inPoll) return; + m_inPoll = true; + + switch (m_step) { + + case Step::Idle: + m_timer->stop(); + break; + + case Step::OpeningReference: + openReference(); + break; + + case Step::AnalysingReference: + if (!analysing(m_window->m_analyser)) { + startPunchIn(); + } else if (stepTimedOut()) { + end(tr("The test reference was not analysed in time.")); + } + break; + + case Step::Recording: + if (!m_window->m_recordTarget || + !m_window->m_recordTarget->isRecording()) { + takeStopped(); + } else if (stepTimedOut()) { + stopTake(); + end(tr("A take did not stop at the end of its range.")); + } + break; + + case Step::AnalysingTake: + // Not needed for the judgement, which reads the take's audio: it + // keeps pYIN's load out of the timing of the next take + if (!analysing(m_window->m_analyser2)) { + if (++m_punchIn < int(m_starts.size())) { + startPunchIn(); + } else { + judge(); + } + } else if (stepTimedOut()) { + end(tr("A take was not analysed in time.")); + } + break; + } + + m_inPoll = false; +} + +void +AudioCheckRunner::openReference() +{ + // Each call below may show a dialog, and while one is up the run may + // end: the session closed, or the check cancelled + if (!m_window->checkSaveModified()) { + if (m_step == Step::OpeningReference) { + end(tr("Your session was kept open, so the check did not " + "replace it.")); + } + return; + } + if (m_step != Step::OpeningReference) return; + + // Written afresh every time, over the one a check before wrote: it + // is made from the plan, and a session opened from it has read it + // whole already + QString path = m_plan.referencePath; + QDir().mkpath(QFileInfo(path).absolutePath()); + vector samples = LatencyCheck::generate(m_plan.layout); + WavFileWriter writer(path, m_plan.layout.rate, 1, + WavFileWriter::WriteToTarget); + const float *data = samples.data(); + if (!writer.isOK() || + !writer.writeSamples(&data, sv_frame_t(samples.size())) || + !writer.close()) { + end(tr("The test reference could not be written to \"%1\".") + .arg(path)); + return; + } + + m_openingReference = true; + MainWindowBase::FileOpenStatus status = + m_window->openPath(path, MainWindowBase::ReplaceSession); + m_openingReference = false; + if (m_step != Step::OpeningReference) return; + + auto model = m_window->getMainModel(); + if (status != MainWindowBase::FileOpenSucceeded || !model) { + end(tr("The test reference \"%1\" could not be opened.").arg(path)); + return; + } + + // The punch-ins from here on are as the takes record them: in whole + // frames of the session, which is what the selections are made of + const sv_samplerate_t rate = model->getSampleRate(); + m_result.referenceRate = rate; + for (LatencyCheck::PunchIn &p : m_punchIns) { + m_starts.push_back(sv_frame_t(std::llround(p.start * rate))); + m_ends.push_back(sv_frame_t(std::llround(p.end * rate))); + p = LatencyCheck::PunchIn(double(m_starts.back()) / rate, + double(m_ends.back()) / rate); + } + + setStep(Step::AnalysingReference, kReferenceTimeoutMs); +} + +void +AudioCheckRunner::startPunchIn() +{ + // A take of the user's own is never stopped by this record(), which + // would be a Stop + if (m_window->m_recordTarget && m_window->m_recordTarget->isRecording()) { + end(tr("Another take was being recorded.")); + return; + } + + const sv_frame_t from = m_starts[m_punchIn]; + const sv_frame_t to = m_ends[m_punchIn]; + const Step step = m_step; + + cerr << "AudioCheckRunner: punch-in " << (m_punchIn + 1) << " of " + << m_starts.size() << ", [" << from << "," << to << ")" << endl; + + // The selection is what Record into Selection records. Made quietly: + // a selection the user makes has the reference's pitch analysed + // again inside it, which would load the machine during the take + ViewManager *viewManager = m_window->m_viewManager; + viewManager->clearSelections(); + viewManager->addSelectionQuietly(Selection(from, to)); + + m_window->m_audioCheckTakes = true; + m_window->record(); + if (m_step != step) return; + + if (!m_window->m_recordTarget || + !m_window->m_recordTarget->isRecording()) { + end(tr("The take did not start: the audio device could not be " + "opened, or it refused to record.")); + return; + } + + // The lead-in and the range, then time for the latency and for the + // take to see that it has reached the end + const double seconds = + double(m_window->m_takePreRoll + (to - from)) / m_result.referenceRate; + setStep(Step::Recording, qint64(seconds * 1000.0) + kTakeStopTimeoutMs); +} + +void +AudioCheckRunner::takeStopped() +{ + m_window->m_audioCheckTakes = false; + m_result.takes.push_back(m_window->m_takeLatency); + + // The splice went wrong (the window has said so), or the recording + // was too short to hold anything + const SingingTakes *takes = m_window->m_takes; + if (!takes->haveTake() || + !takes->getCoverage().contains(m_starts[m_punchIn])) { + end(tr("A take was not added to the singing track.")); + return; + } + + setStep(Step::AnalysingTake, kTakeAnalysisTimeoutMs); +} + +void +AudioCheckRunner::judge() +{ + // The take's own file, and not its model: the model is normalised to + // full scale as it is read (the "normalise audio" preference), which + // would have every take clipped, and resampled to the session's rate. + // The file holds what was recorded, at the rate it was recorded at + QString path = m_window->m_takes->getAudioPath(); + FileSource source(path); + WavFileReader reader(source); + if (!reader.isOK() || reader.getChannelCount() < 1) { + end(tr("The take's audio file \"%1\" could not be read: %2") + .arg(path).arg(reader.getError())); + return; + } + + const int channels = reader.getChannelCount(); + const floatvec_t data = reader.getInterleavedFrames + (0, reader.getFrameCount()); + const sv_frame_t count = sv_frame_t(data.size()) / channels; + vector mono(count, 0.f); + for (sv_frame_t i = 0; i < count; ++i) { + float sum = 0.f; + for (int c = 0; c < channels; ++c) sum += data[i * channels + c]; + mono[i] = sum / float(channels); + } + + m_result.summary = LatencyCheck::judgeTake + (m_plan.layout, mono.data(), count, reader.getSampleRate(), m_punchIns); + + end(""); +} + +void +AudioCheckRunner::setStep(Step step, qint64 limitMs) +{ + m_step = step; + m_stepLimitMs = limitMs; + m_stepClock.start(); +} + +bool +AudioCheckRunner::stepTimedOut() const +{ + return m_stepLimitMs > 0 && m_stepClock.elapsed() > m_stepLimitMs; +} + +void +AudioCheckRunner::stopTake() +{ + if (m_step != Step::Recording) return; + if (m_window->m_recordTarget && m_window->m_recordTarget->isRecording()) { + // The Stop button's path, which is also how a take ends itself + m_window->record(); + } +} + +void +AudioCheckRunner::end(QString failure) +{ + if (m_step == Step::Idle) return; + + m_timer->stop(); + m_step = Step::Idle; + m_window->m_audioCheckTakes = false; + + m_result.failure = failure; + if (!m_result.takes.empty()) { + // Each at the rate it counts in (see TakeLatency) + const TakeLatency &first = m_result.takes.front(); + m_result.usedRoundTrip = first.recordingSeconds(first.roundTrip); + m_result.reportedInputLatency = + first.recordingSeconds(first.reportedInput); + if (m_result.referenceRate > 0) { + m_result.reportedOutputLatency = + double(first.reportedOutput) / m_result.referenceRate; + } + m_result.recordingRate = first.recordingRate; + m_result.rateMismatch = m_result.recordingRate > 0 && + m_result.referenceRate > 0 && + m_result.recordingRate != m_result.referenceRate; + } + if (failure == "") { + m_result.calibratedRoundTrip = LatencyCheck::calibratedRoundTrip + (m_result.usedRoundTrip, m_result.summary.medianOffset); + } + + if (failure != "") { + cerr << "AudioCheckRunner: the check ended: " << failure << endl; + } else { + const LatencyCheck::TakeSummary &s = m_result.summary; + cerr << "AudioCheckRunner: " << LatencyCheck::verdictName(s.verdict) + << ", " << s.found << " of " << s.judged << " sweeps found, " + << "median offset " << s.medianOffset * 1000.0 << " ms, spread " + << s.spread * 1000.0 << " ms, input peak " << s.inputPeak + << "; round trip used " << m_result.usedRoundTrip * 1000.0 + << " ms, measured " << m_result.calibratedRoundTrip * 1000.0 + << " ms; recorded at " << m_result.recordingRate + << " Hz, reference at " << m_result.referenceRate << " Hz" + << endl; + } + + // A copy: whoever hears of it may start another run + AudioCheckResult result = m_result; + emit finished(result); +} + +bool +AudioCheckRunner::analysing(Analyser *a) +{ + // What the app suite waits for: the first analysis complete, that of + // a recorded range merged, and the transform threads gone as well. + // Except that an analyser with no pitch track has nothing to wait + // for: with automatic analysis switched off, the reference is never + // analysed, and the check needs no analysis of its own + if (ModelTransformerFactory::getInstance()->haveRunningTransformers()) { + return true; + } + if (!a) return false; + if (a->isAnalysingRange()) return true; + return a->getLayer(Analyser::PitchTrack) && + a->getInitialAnalysisCompletion() < 100; +} diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h new file mode 100644 index 00000000..6d0cbc80 --- /dev/null +++ b/main/AudioCheckRunner.h @@ -0,0 +1,214 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_AUDIO_CHECK_RUNNER_H +#define TONY_AUDIO_CHECK_RUNNER_H + +#include "LatencyCheck.h" +#include "LatencyUtils.h" + +#include "base/BaseTypes.h" + +#include +#include +#include + +#include + +class MainWindow; +class Analyser; +class QTimer; + +/** + * What an audio check found. Times are in seconds. + */ +struct AudioCheckResult +{ + /// Why the run ended before it judged the take; empty if it did. + /// The rest is filled in only as far as the run got + QString failure; + + LatencyCheck::TakeSummary summary; + + /// What each punch-in was placed with, in the order recorded + std::vector takes; + + /// The round trip the first punch-in was placed with, as frames of + /// the recording, and the two latencies the device reported, each at + /// the rate it counts in (see TakeLatency) + double usedRoundTrip; + double reportedOutputLatency; + double reportedInputLatency; + + /// The rate the device recorded at, and the session's, which the + /// reference was made at + sv::sv_samplerate_t recordingRate; + sv::sv_samplerate_t referenceRate; + + /// The two rates differ. Set from the rates, whatever the sweeps + /// say: a take recorded at another rate is placed frame for frame + /// (a known bug), so it lands further off the further into the + /// reference it is, soon further than the finder looks + bool rateMismatch; + + /// The round trip that would have placed the takes right + /// (LatencyCheck::calibratedRoundTrip()); see calibrationUsable() + double calibratedRoundTrip; + + /// Whether calibratedRoundTrip means anything: the run was judged + /// Ok or Unsteady, at the reference's rate + bool calibrationUsable() const; + + AudioCheckResult() : usedRoundTrip(0), reportedOutputLatency(0), + reportedInputLatency(0), recordingRate(0), + referenceRate(0), rateMismatch(false), + calibratedRoundTrip(0) { } +}; + +/** + * The audio check: a test reference, written out and opened as a + * session of its own, then punch-ins recorded against it through the + * ordinary take path, and the take that comes out of them judged by + * LatencyCheck::judgeTake(). Every punch-in restarts the stream, as a + * real take does, and is placed with the latency the window uses for + * any take, which is what is being measured. + * + * The takes are recorded with Record into Selection, Play Reference + * While Recording and a pre-roll of kPreRollSeconds, whatever the + * toolbar says: MainWindow::record() and the rest consult an override + * the runner sets for each of its takes, since the toolbar's toggles + * write the user's settings. A punch-in's range is made the selection + * (the previous one cleared, then this one selected: "Select" steps in + * the history, as when the user selects), and record() is called; the + * take stops itself at the end of the selection, through the same path + * as the Stop button. + * + * Driven by a polling timer, like MainWindow's own take polling, and + * never by a nested event loop: this runs in every build, and the + * window can be closed at any moment. MainWindow owns it, deletes it + * first thing in its destructor, and tells it when the session closes. + * A friend of MainWindow: it drives the window's take path, and reads + * what the take was placed with, but changes nothing else there. + */ +class AudioCheckRunner : public QObject +{ + Q_OBJECT + +public: + /// The lead-in of the check's takes + static constexpr double kPreRollSeconds = 1.0; + + /// How often the runner looks at how a step is going + static constexpr int kPollMs = 50; + + /// How long a step may take before the run gives up on it: the + /// first analysis of the reference, a take's analysis, and a take + /// beyond its own length (lead-in and range) to stop itself + static constexpr int kReferenceTimeoutMs = 60000; + static constexpr int kTakeAnalysisTimeoutMs = 30000; + static constexpr int kTakeStopTimeoutMs = 10000; + + /// What a run records + struct Plan { + LatencyCheck::Layout layout; + int punchIns; + int eventsEach; + + /// Where the reference is written, over whatever is there; "" + /// for defaultReferencePath() + QString referencePath; + + Plan() : punchIns(0), eventsEach(0) { } + }; + + explicit AudioCheckRunner(MainWindow *window); + virtual ~AudioCheckRunner(); + + /// A file in the application's data directory + static QString defaultReferencePath(); + + /** + * Begin a run. False, with nothing started, if one is running + * already, if a take is being recorded, or if the plan's punch-ins + * do not fit its layout (LatencyCheck::punchInsFor()). Otherwise + * finished() comes once, at the end, however the run ends. + */ + bool start(const Plan &plan); + + /// End the run: a take in progress is stopped through the Stop + /// path, and finished() says the run was cancelled + void cancel(); + + bool isRunning() const; + + /// Called by MainWindow::closeSession() once the session is sure to + /// close: a run ends then, unless it is the run replacing it + void sessionClosing(); + +signals: + void finished(const AudioCheckResult &result); + +private: + enum class Step { + Idle, + OpeningReference, + AnalysingReference, + Recording, + AnalysingTake + }; + + MainWindow *m_window; + QTimer *m_timer; + Step m_step; + Plan m_plan; + AudioCheckResult m_result; + + /// The punch-ins in seconds, from the plan, until the reference is + /// open; from then on as the takes record them, in whole frames of + /// the session + std::vector m_punchIns; + std::vector m_starts; + std::vector m_ends; + int m_punchIn; + + /// Set while the runner replaces the session itself + bool m_openingReference; + + /// Set while poll() runs: a dialog shown from inside a step runs + /// an event loop of its own, in which the timer goes on firing + bool m_inPoll; + + QElapsedTimer m_stepClock; + qint64 m_stepLimitMs; + + void poll(); + void openReference(); + void startPunchIn(); + void takeStopped(); + void judge(); + + void setStep(Step step, qint64 limitMs); + bool stepTimedOut() const; + + /// Stop a take the check is recording, through the Stop path + void stopTake(); + + /// End the run and say so; failure is empty for a run judged + void end(QString failure); + + /// Whether the analyser, or any transform, is still at work + static bool analysing(Analyser *analyser); +}; + +#endif diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp index f07e3289..d19da71c 100644 --- a/main/LatencyCheck.cpp +++ b/main/LatencyCheck.cpp @@ -583,6 +583,40 @@ LatencyCheck::judgeTake(const Layout &layout, return summary; } +vector +LatencyCheck::punchInsFor(const Layout &layout, int count, int eventsEach) +{ + if (count <= 0 || eventsEach <= 0 || layout.rate <= 0) return {}; + + // All that judgeTake() reads for an event, as it works it out, and a + // little more + const double before = + kSearchSeconds + kJudgeMarginSeconds + kPunchInSlackSeconds; + const double after = kSearchSeconds + kSweepSeconds + + kJudgeMarginSeconds + kPunchInSlackSeconds; + const double length = double(layout.length) / layout.rate; + const int n = int(layout.events.size()); + + auto at = [&](int i) { + return double(layout.events[i].sweepStart) / layout.rate; + }; + + vector punchIns; + double free = 0.0; + int i = 0; + while (int(punchIns.size()) < count) { + while (i < n && at(i) - before < free) ++i; + const int last = i + eventsEach - 1; + if (last >= n) return {}; + const PunchIn p(at(i) - before, at(last) + after); + if (p.end > length) return {}; + punchIns.push_back(p); + free = p.end; + i = last + 1; + } + return punchIns; +} + double LatencyCheck::calibratedRoundTrip(double usedRoundTripSeconds, double medianOffsetSeconds) diff --git a/main/LatencyCheck.h b/main/LatencyCheck.h index 7ca43790..c3e81712 100644 --- a/main/LatencyCheck.h +++ b/main/LatencyCheck.h @@ -426,6 +426,26 @@ namespace LatencyCheck sv::sv_samplerate_t rate, const std::vector &punchIns); + /// punchInsFor() leaves this much more room around what the finder + /// reads than judgeTake() asks for, so that a range rounded to whole + /// frames, at any rate, still holds all of it + constexpr double kPunchInSlackSeconds = 0.01; + + /** + * The ranges of a run of punch-ins against the reference made from + * this layout: count of them, one after another along the timeline + * and not overlapping, each holding eventsEach consecutive events + * that judgeTake() judges in it, and no other. + * + * An event is judged only where all that the finder reads for it + * lies inside one range, so where one range ends and the next + * begins, that much of the spacing between two events is lost (1.9 + * s): an event whose reading would begin before the previous range + * ends is left out. Empty if the layout has no room for them all. + */ + std::vector punchInsFor(const Layout &layout, + int count, int eventsEach); + /** * The round trip that would have placed the take right, in seconds: * the one it was placed with plus the median offset measured. diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index fba94a51..d5dfdf94 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -33,6 +33,36 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, return outputLatency + inputLatency; } +/** + * What a take was placed with: the round trip taken off the front of + * its recording (besides the start gap, which is measured), the output + * and input latencies the device reported, and the rate the device + * recorded at. All 0 until known; the round trip stays 0 for a take + * made without the reference playing, which is placed with none. + * + * In frames as the window has them. The round trip is taken off the + * recording, and the input latency is the device's, so both count + * frames of the recording; but the play source reports the output + * latency in frames of the session, converted when it resamples to the + * device. The two kinds differ only when the device's rate is not the + * session's, and the round trip then adds the one to the other. + */ +struct TakeLatency +{ + sv::sv_frame_t roundTrip; + sv::sv_frame_t reportedOutput; + sv::sv_frame_t reportedInput; + sv::sv_samplerate_t recordingRate; + + TakeLatency() : roundTrip(0), reportedOutput(0), reportedInput(0), + recordingRate(0) { } + + /// Frames of the recording in seconds; 0 while its rate is unknown + double recordingSeconds(sv::sv_frame_t frames) const { + return recordingRate > 0 ? double(frames) / recordingRate : 0.0; + } +}; + /** * Where to draw a live pitch dot for sound found at the given frame * of a take that is going to be shifted earlier by the given latency diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index b1e52653..fd2a35d6 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -18,6 +18,7 @@ #include "MainWindow.h" #include "NetworkPermissionTester.h" #include "Analyser.h" +#include "AudioCheckRunner.h" #include "LatencyUtils.h" #include "PaneUtils.h" #include "TakeEvents.h" @@ -199,7 +200,10 @@ MainWindow::MainWindow(AudioMode audioMode, m_recordingLatencyFrames(0), m_recordingStartGapEstimate(0), m_recordingStartGapMeasured(-1), - m_awaitingReferenceStart(false) + m_awaitingReferenceStart(false), + m_takeLatency(), + m_audioCheck(nullptr), + m_audioCheckTakes(false) { setWindowTitle(QApplication::applicationName()); @@ -393,6 +397,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_takes = new SingingTakes(this); m_coverageStrip = new CoverageStrip(this); + m_audioCheck = new AudioCheckRunner(this); // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end @@ -450,6 +455,10 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindow::~MainWindow() { + // A check still running ends here, before anything it reads goes + delete m_audioCheck; + m_audioCheck = nullptr; + // Nothing must poll a take while the window is coming down stopTakePolling(); @@ -2533,6 +2542,10 @@ MainWindow::closeSession() { if (!checkSaveModified()) return; + // A check has nothing left to record into; a take of its own that is + // running is stopped through the Stop path, as the check's Cancel does + if (m_audioCheck) m_audioCheck->sessionClosing(); + // Nothing of a take that is still running outlives its session stopTakePolling(); @@ -3836,6 +3849,7 @@ MainWindow::record() m_recordingStartGapEstimate = 0; m_awaitingReferenceStart = false; m_recordingStartGapMeasured = -1; + m_takeLatency = TakeLatency(); if (haveReference) { @@ -3849,10 +3863,12 @@ MainWindow::record() // Record into Selection: the selection is what is recorded, so it // says where the take starts and where it stops, and the playhead // only picks which selection that is. With none selected this is - // an ordinary recording from the playhead. + // an ordinary recording from the playhead. The audio check's + // takes always record into the selection it makes. sv_frame_t end = -1; - if (m_recordIntoSelection && m_recordIntoSelection->isChecked() && - m_viewManager) { + bool intoSelection = m_audioCheckTakes || + (m_recordIntoSelection && m_recordIntoSelection->isChecked()); + if (intoSelection && m_viewManager) { Coverage::Ranges selected; for (const Selection &s : m_viewManager->getSelections()) { if (!s.isEmpty()) { @@ -4143,9 +4159,11 @@ MainWindow::recordingStarted() // hears the reference from there. The audio IO was already // resumed by record() so m_playSource can be started directly // without calling MainWindowBase::play() (which would stop - // recording if isRecording() is true). - if (m_recordingAsSingingTrack && - m_playRefWhileRecording && m_playRefWhileRecording->isChecked() && + // recording if isRecording() is true). The audio check's takes + // always play it: they measure where it arrives. + bool playReference = m_audioCheckTakes || + (m_playRefWhileRecording && m_playRefWhileRecording->isChecked()); + if (m_recordingAsSingingTrack && playReference && m_playSource && !m_playSource->isPlaying()) { cerr << "MainWindow::recordingStarted: starting reference playback" << endl; @@ -4175,9 +4193,13 @@ MainWindow::recordingStarted() // figure, and refineRecordingLatency() picks it up. m_recordingStartGapEstimate = m_recordTarget ? m_recordTarget->getFramesReceived() : 0; - m_recordingLatencyFrames = - computeRecordingLatency(outputLatency, inputLatency) + - m_recordingStartGapEstimate; + sv_frame_t roundTrip = + computeRecordingLatency(outputLatency, inputLatency); + m_recordingLatencyFrames = roundTrip + m_recordingStartGapEstimate; + + m_takeLatency.roundTrip = roundTrip; + m_takeLatency.reportedOutput = outputLatency; + m_takeLatency.reportedInput = inputLatency; cerr << "MainWindow::recordingStarted: output latency=" << outputLatency << " input latency=" << inputLatency << " estimated start gap=" << m_recordingStartGapEstimate @@ -4234,12 +4256,17 @@ MainWindow::currentTakeTiming() const sv_frame_t MainWindow::wantedPreRollFrames() const { - if (!m_preRoll || !m_preRoll->isChecked()) return 0; - - QSettings settings; - settings.beginGroup("MainWindow"); - double seconds = settings.value("prerollseconds", 3.0).toDouble(); - settings.endGroup(); + double seconds = 0.0; + if (m_audioCheckTakes) { + // The audio check's takes have a lead-in of their own + seconds = AudioCheckRunner::kPreRollSeconds; + } else { + if (!m_preRoll || !m_preRoll->isChecked()) return 0; + QSettings settings; + settings.beginGroup("MainWindow"); + seconds = settings.value("prerollseconds", 3.0).toDouble(); + settings.endGroup(); + } if (seconds <= 0.0) return 0; auto model = getMainModel(); @@ -4385,6 +4412,7 @@ MainWindow::finishSingingTake() (m_currentRecordingModelId)) { recordingPath = wfm->getLocation(); recorded = wfm->getFrameCount(); + m_takeLatency.recordingRate = wfm->getSampleRate(); } // The take is over: the tracker goes first, so that the recording's diff --git a/main/MainWindow.h b/main/MainWindow.h index d27a635d..fd03fd34 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -24,6 +24,7 @@ #include "SingingTakes.h" #include "TakeCommands.h" #include "TakeTiming.h" +#include "LatencyUtils.h" #include #include @@ -35,6 +36,8 @@ class QTimer; class QComboBox; class QActionGroup; +class AudioCheckRunner; + namespace sv { class VersionTester; class ActivityLog; @@ -46,6 +49,10 @@ class MainWindow : public sv::MainWindowBase { Q_OBJECT + // The audio check drives the take path of the window, and reads what + // each take was placed with; see AudioCheckRunner + friend class AudioCheckRunner; + public: MainWindow(AudioMode audioMode, bool withSonification = true, @@ -566,7 +573,8 @@ protected slots: TakeTiming currentTakeTiming() const; // The pre-roll asked for, in frames of the reference: the QSettings - // value MainWindow/prerollseconds (3 s), or 0 with the toggle off + // value MainWindow/prerollseconds (3 s), or 0 with the toggle off; + // for a take of the audio check, the check's own sv::sv_frame_t wantedPreRollFrames() const; // Put the countdown of a pre-roll's lead-in in the status bar, and @@ -843,6 +851,20 @@ protected slots: std::atomic m_recordingStartGapMeasured; std::atomic m_awaitingReferenceStart; + // What the take being recorded, or the last one, was placed with: + // cleared when a take starts, the round trip and the latencies the + // device reported filled in when the reference starts to play, and + // the recording's rate when the take is spliced in + TakeLatency m_takeLatency; + + // The audio check, and the override it sets for each take of its + // own: Record into Selection, Play Reference While Recording and a + // pre-roll of AudioCheckRunner::kPreRollSeconds, whatever the toolbar + // says. Not by setting the toggles, which write the user's settings. + // record(), recordingStarted() and wantedPreRollFrames() consult it + AudioCheckRunner *m_audioCheck; + bool m_audioCheckTakes; + void refineRecordingLatency(); // The best figure for the latency as things stand, measurement diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h new file mode 100644 index 00000000..18124afc --- /dev/null +++ b/main/test/TestAudioCheck.h @@ -0,0 +1,398 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_AUDIO_CHECK_H +#define TEST_AUDIO_CHECK_H + +// Tier 5, as TestRecordWorkflow: the audio check (AudioCheckRunner) on +// the real MainWindow, recording from the fake device with its output +// looped back into its input, as an earcup held to the mic. A run is +// two punch-ins of two sweeps each on the first 11 s of the calibration +// reference, about 13 s of real time. +// +// The same dialog watchdog as TestRecordWorkflow's: a dialog would +// block the test for ever, so it is dismissed, and the test fails in +// cleanup(). + +#include "TestRecordWorkflow.h" + +#include "../AudioCheckRunner.h" +#include "../LatencyCheck.h" + +class TestAudioCheck : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + + // What the device reports, and what the round trip really is + static constexpr int reportedOut = 2 * 4096; + static constexpr int reportedIn = 4096; + static constexpr int roundTrip = 3 * 4096 + 123; + + QTemporaryDir m_dir; + TestMainWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + + // What the runner said when the run ended, and how often it said it + AudioCheckResult m_result; + int m_finished = 0; + + void makeWindow(FakeAudioIO::Config config, bool installDevice = true) { + delete m_window; + m_window = new TestMainWindow(config, installDevice); + m_result = AudioCheckResult(); + m_finished = 0; + connect(m_window->audioCheck(), &AudioCheckRunner::finished, + this, [this](const AudioCheckResult &result) { + m_result = result; + ++m_finished; + }); + } + + // Speakers into the microphone: the output comes back as input, + // roundTrip frames late, while the device reports latencies that + // add up to less + static FakeAudioIO::Config loopback() { + FakeAudioIO::Config config; + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.inputDelay = roundTrip; + config.loopback = true; + return config; + } + + // Two punch-ins of two events each, on the calibration reference cut + // short after its fifth event. The event at 4.7 s is left out: it is + // too close to the one before for a punch-in to end between them + AudioCheckRunner::Plan shortPlan() { + AudioCheckRunner::Plan plan; + plan.layout = LatencyCheck::calibrationLayout(); + plan.layout.length = plan.layout.events[5].sweepStart; + plan.layout.events.resize(5); + plan.punchIns = 2; + plan.eventsEach = 2; + plan.referencePath = m_dir.filePath("reference.wav"); + return plan; + } + + void startCheck() { + // As answering "No" to "do you want to save?", which the check + // asks before it replaces the session + m_window->discardModifications(); + QVERIFY(m_window->audioCheck()->start(shortPlan())); + QVERIFY(m_window->audioCheck()->isRunning()); + } + + void runCheck() { + startCheck(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0, 60000); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->audioCheck()->isRunning()); + } + + // The three toggles of the take path, and the settings they and the + // pre-roll's length are kept in + QStringList toggles() { + QSettings settings; + settings.beginGroup("MainWindow"); + QStringList state; + state << QString("play reference %1, setting %2") + .arg(m_window->playReferenceWhileRecordingAction()->isChecked()) + .arg(settings.value("playrefwhilerecording").toString()); + state << QString("pre-roll %1, setting %2, %3 s") + .arg(m_window->preRollAction()->isChecked()) + .arg(settings.value("preroll").toString()) + .arg(settings.value("prerollseconds").toString()); + state << QString("record into selection %1, setting %2") + .arg(m_window->recordIntoSelectionAction()->isChecked()) + .arg(settings.value("recordintoselection").toString()); + settings.endGroup(); + return state; + } + + static QByteArray describe(const AudioCheckResult &r) { + const LatencyCheck::TakeSummary &s = r.summary; + QStringList flags; + for (LatencyCheck::Verdict v : s.flags) { + flags << LatencyCheck::verdictName(v); + } + return QString("%1 %2 [%3], found %4 of %5, median offset %6 " + "frames, spread %7 ms, input peak %8; %9 takes, " + "round trip used %10 frames, measured %11 frames; " + "recorded at %12 Hz, reference at %13 Hz") + .arg(r.failure == "" ? "judged:" : "failed: " + r.failure) + .arg(LatencyCheck::verdictName(s.verdict)) + .arg(flags.join(" ")) + .arg(s.found) + .arg(s.judged) + .arg(s.medianOffset * rate) + .arg(s.spread * 1000.0) + .arg(s.inputPeak) + .arg(r.takes.size()) + .arg(r.usedRoundTrip * rate) + .arg(r.calibratedRoundTrip * rate) + .arg(r.recordingRate) + .arg(r.referenceRate) + .toUtf8(); + } + + // Not a slot: QtTest would run it as a test. As TestRecordWorkflow's + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + m_dialogs.push_back(description); + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + QSettings().clear(); + + // Otherwise the MainWindow constructor asks, in a dialog + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + + // Recordings and takes go here and not among the user's + sv::RecordDirectory::setRecordContainerDirectory + (m_dir.filePath("recorded")); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", false); + settings.setValue("preroll", false); + settings.setValue("recordintoselection", false); + settings.remove("prerollseconds"); + settings.endGroup(); + settings.beginGroup("Analyser"); + settings.remove(""); + settings.endGroup(); + SingingTakes::setOverwriteConfirmationWanted(true); + } + + void cleanup() { + if (m_window) { + if (m_window->recordTarget()->isRecording()) { + m_window->doRecord(); + } + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + sv::RecordDirectory::setRecordContainerDirectory(""); + } + + // The device reports 2 x 4096 frames out and 4096 in, and the round + // trip is 123 frames longer. Every take lands 123 frames late, the + // check finds them there, and the round trip it works out is the + // real one. Its takes play the reference, record into their ranges + // and have its own lead-in, and the user's three toggles, set + // otherwise, are as they were, and so are their settings + void check_measures_the_round_trip() { + makeWindow(loopback()); + { + QSettings settings; + settings.setValue("MainWindow/prerollseconds", 3.0); + } + m_window->setPlayReferenceWhileRecording(false); + m_window->setPreRoll(true); + m_window->setRecordIntoSelection(false); + const QStringList before = toggles(); + + runCheck(); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QCOMPARE(r.summary.judged, 4); + QCOMPARE(r.summary.found, 4); + QVERIFY2(std::fabs(r.summary.medianOffset * rate - + (roundTrip - reportedOut - reportedIn)) <= 4.0, + describe(r).constData()); + + QCOMPARE(int(r.takes.size()), 2); + for (const TakeLatency &t : r.takes) { + QCOMPARE(t.roundTrip, sv::sv_frame_t(reportedOut + reportedIn)); + QCOMPARE(t.reportedOutput, sv::sv_frame_t(reportedOut)); + QCOMPARE(t.reportedInput, sv::sv_frame_t(reportedIn)); + QCOMPARE(t.recordingRate, rate); + } + QCOMPARE(r.usedRoundTrip, (reportedOut + reportedIn) / rate); + QCOMPARE(r.reportedOutputLatency, reportedOut / rate); + QCOMPARE(r.reportedInputLatency, reportedIn / rate); + QCOMPARE(r.recordingRate, rate); + QCOMPARE(r.referenceRate, rate); + QVERIFY(!r.rateMismatch); + QVERIFY(r.calibrationUsable()); + QVERIFY2(std::fabs(r.calibratedRoundTrip * rate - roundTrip) <= 4.0, + describe(r).constData()); + + // The second punch-in starts 6.3 s in, with room for all of the + // check's lead-in + QCOMPARE(m_window->takePreRoll(), + sv::sv_frame_t(AudioCheckRunner::kPreRollSeconds * rate)); + + QCOMPARE(toggles(), before); + QVERIFY(!m_window->audioCheckTakes()); + } + + // A device at 48 kHz: the takes are recorded at a rate other than the + // reference's, which is reported with both rates, whatever the sweeps + // say. What Tony does with such takes is a known bug of its own, so + // only what the check reports is asserted, and that it ends + void check_flags_a_rate_mismatch() { + FakeAudioIO::Config config = loopback(); + config.sampleRate = 48000; + makeWindow(config); + + runCheck(); + if (QTest::currentTestFailed()) return; + + qDebug() << "48 kHz:" << describe(m_result).constData(); + QVERIFY2(m_result.rateMismatch, describe(m_result).constData()); + QCOMPARE(m_result.recordingRate, 48000.0); + QCOMPARE(m_result.referenceRate, rate); + QVERIFY(!m_result.calibrationUsable()); + QVERIFY(!m_window->audioCheckTakes()); + } + + // Cancel during a take stops it through the Stop path, clears the + // override, ends the run once and starts nothing more + void check_cancelled_during_a_take() { + makeWindow(loopback()); + const QStringList before = toggles(); + + startCheck(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY(m_window->audioCheckTakes()); + QTest::qWait(1000); + + m_window->audioCheck()->cancel(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY(!m_window->audioCheckTakes()); + QVERIFY(!m_window->audioCheck()->isRunning()); + QCOMPARE(m_finished, 1); + QVERIFY(m_result.failure != ""); + + QTest::qWait(500); + QVERIFY(!m_window->recordTarget()->isRecording()); + QCOMPARE(m_finished, 1); + QCOMPARE(toggles(), before); + } + + // No device to record from: the take does not start, and the run + // ends there and says why + void check_fails_without_a_device() { + makeWindow(loopback(), false); + + runCheck(); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_result.failure != "", describe(m_result).constData()); + QVERIFY(m_result.takes.empty()); + QVERIFY(!m_result.calibrationUsable()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY(!m_window->audioCheckTakes()); + } + + // With automatic analysis switched off the reference is never + // analysed, and the check does not wait for it: it needs none + void check_runs_without_automatic_analysis() { + { + QSettings settings; + settings.setValue("Analyser/auto-analysis", false); + } + makeWindow(loopback()); + + startCheck(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 10000); + QVERIFY(!m_window->analyser()->getLayer(Analyser::PitchTrack)); + + m_window->audioCheck()->cancel(); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->recordTarget()->isRecording()); + } + + // Closing the session during a take ends the run the same way + void check_ends_when_the_session_closes() { + makeWindow(loopback()); + + startCheck(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QTest::qWait(1000); + + m_window->doCloseSession(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->audioCheckTakes()); + QVERIFY(!m_window->audioCheck()->isRunning()); + QCOMPARE(m_finished, 1); + QVERIFY(m_result.failure != ""); + + QTest::qWait(500); + QVERIFY(!m_window->recordTarget()->isRecording()); + QCOMPARE(m_finished, 1); + } +}; + +#endif diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h index 583208f0..758d08e5 100644 --- a/main/test/TestLatencyCheck.h +++ b/main/test/TestLatencyCheck.h @@ -954,6 +954,73 @@ private slots: QCOMPARE(s.events[1].arrival.errorFrames, later); } + // The punch-ins of a run of takes. Each holds the events asked for, + // and judgeTake() judges those in it and no other: the reference + // itself, as a take placed right, finds every one where it is. Also + // with the ranges rounded to whole frames, as the takes record them. + // Between two ranges an event may have to be left out: in the + // calibration layout the sweep at 4.7 s is only 1.6 s after the one + // at 3.1 s, and the finder reads 1.9 s around the two of them + void punch_ins_hold_the_events_asked_for() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t reference = LatencyCheck::generate(layout); + const double length = double(layout.length) / layout.rate; + + struct Case { int count; int each; }; + for (Case c : { Case { 2, 2 }, Case { 4, 2 }, Case { 3, 3 }, + Case { 1, 12 } }) { + + const punchins_t exact = + LatencyCheck::punchInsFor(layout, c.count, c.each); + QCOMPARE(int(exact.size()), c.count); + + punchins_t rounded; + for (const LatencyCheck::PunchIn &p : exact) { + rounded.push_back(LatencyCheck::PunchIn + (framesOf(p.start) / kRate, + framesOf(p.end) / kRate)); + } + + for (const punchins_t &punchIns : { exact, rounded }) { + for (int p = 0; p < c.count; ++p) { + QVERIFY(punchIns[p].start >= 0.0); + QVERIFY(punchIns[p].end <= length); + QVERIFY(punchIns[p].end > punchIns[p].start); + if (p > 0) QVERIFY(punchIns[p].start >= punchIns[p-1].end); + } + + const LatencyCheck::TakeSummary s = + judge(layout, reference, kRate, punchIns); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, + describe(s).constData()); + QCOMPARE(s.judged, c.count * c.each); + QCOMPARE(s.found, c.count * c.each); + for (int p = 0; p < c.count; ++p) { + QCOMPARE(s.punchIns[p].judged, c.each); + } + // Consecutive events within each punch-in + for (int k = 1; k < s.judged; ++k) { + if (s.events[k].punchIn == s.events[k-1].punchIn) { + QCOMPARE(s.events[k].event, s.events[k-1].event + 1); + } + } + } + } + + // The two punch-ins of the app suite's check: the event at 4.7 s + // is left out + const punchins_t two = LatencyCheck::punchInsFor(layout, 2, 2); + const LatencyCheck::TakeSummary s = judge(layout, reference, kRate, two); + const int events[] = { 0, 1, 3, 4 }; + QCOMPARE(int(s.events.size()), 4); + for (int k = 0; k < 4; ++k) QCOMPARE(s.events[k].event, events[k]); + + // More than the layout holds, and nothing asked for + QVERIFY(LatencyCheck::punchInsFor(layout, 7, 2).empty()); + QVERIFY(LatencyCheck::punchInsFor(layout, 1, 13).empty()); + QVERIFY(LatencyCheck::punchInsFor(layout, 0, 2).empty()); + } + // The calibration's arithmetic. A take that landed late was placed // with too small a round trip, and the new one is larger by the // offset; early, smaller. From takes placed with 0.2 and 0.31 s diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 9e6b68ce..b3f28c78 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -156,6 +156,17 @@ class TestMainWindow : public MainWindow void setRecordIntoSelection(bool on) { m_recordIntoSelection->setChecked(on); } + QAction *playReferenceWhileRecordingAction() { + return m_playRefWhileRecording; + } + QAction *preRollAction() { return m_preRoll; } + QAction *recordIntoSelectionAction() { return m_recordIntoSelection; } + + // The audio check, the override it sets for its own takes, and what + // the last take was placed with + AudioCheckRunner *audioCheck() { return m_audioCheck; } + bool audioCheckTakes() { return m_audioCheckTakes; } + TakeLatency takeLatency() { return m_takeLatency; } QAction *playSingingAudioAction() { return m_playSingingAudio; } Analyser *analyser() { return m_analyser; } diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 6ad3eaf0..e8f310ef 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -14,6 +14,7 @@ #include "TestSingingDocument.h" #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" +#include "TestAudioCheck.h" #include "RunSuite.h" @@ -70,6 +71,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestAudioCheck t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 88d76863..acbf5ba4 100644 --- a/meson.build +++ b/meson.build @@ -1106,6 +1106,7 @@ tony_core_files = [ tony_app_files = [ 'main/AlternatePitchTrack.cpp', + 'main/AudioCheckRunner.cpp', 'main/CoverageStrip.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', @@ -1126,6 +1127,7 @@ tony_app_moc_files = qt.preprocess( 'main/MainWindow.h', 'main/Analyser.h', 'main/AlternatePitchTrack.h', + 'main/AudioCheckRunner.h', 'main/CoverageStrip.h', ]) @@ -1390,6 +1392,7 @@ tony_app_test_moc_files = qt.preprocess( 'main/test/TestSingingDocument.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', + 'main/test/TestAudioCheck.h', ]) tony_app_test_exe = executable( From 7fb41743a490faf6bc5cb75e892f56c9b1ad2550 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:48:38 +0000 Subject: [PATCH 101/275] fix: the live dots are drawn without drawing the whole pane again The live dots layer is kept out of the pane's cache (svgui fork, Layer::setCachedInView): told of a change to the model of a layer in its cache, a pane draws every layer in it again, so each of the 25 notices a second drew the reference's waveform, pitch track and notes. That alone saved nothing, because on a cache hit View drew every cached layer into its buffer anyway and then covered it with the cache; the fork no longer does. With both, the GUI thread used about 22 % of a core during a take in a 1920 px window on the cloud machine, against 51-54 % before; the newest dot drawn trails the cursor by at most 90 ms there. TestViewCache checks which layers a pane has draw, with a layer in and out of its cache; live_dots_appear that the dots layer is kept out. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019UV9LcYjNdAyq7Edp5Jygt --- docs/forks.md | 9 +- docs/manual-checklist.md | 3 +- docs/open-points.md | 4 - docs/recording.md | 19 +++-- docs/testing.md | 2 +- main/MainWindow.cpp | 14 +-- main/ModelChangeThrottle.h | 7 +- main/test/TestRecordWorkflow.h | 2 + main/test/TestViewCache.h | 152 +++++++++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 7 ++ meson.build | 1 + repoint-lock.json | 2 +- 12 files changed, 196 insertions(+), 26 deletions(-) create mode 100644 main/test/TestViewCache.h diff --git a/docs/forks.md b/docs/forks.md index 42651b52..87189c2c 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -82,6 +82,12 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` progress, whose end crawls along behind the playback cursor. `MainWindow` names the reference instead. - `Layer::setSavedInSession(false)`: `View::toXml()` leaves the layer out. +- `Layer::setCachedInView(false)`: `View` draws the layer, and every layer in front of it, + at every paint instead of keeping it in its cache, and a change to its model leaves the + cache alone. For the live dots ([recording.md](recording.md)). +- `View::paintEvent()` on a cache hit no longer has the cached layers draw into its buffer, + where the cache then covered them. Upstream has done that since 2018, so the cache saved + nothing and every paint, down to the play pointer's few pixels, drew every layer. ## Known defects in the forks, not fixed @@ -95,9 +101,6 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` ## Changes that would tidy Tony up but were not made -- A way to keep a layer out of `View`'s cache (a view told of a change to a cached layer's - model draws every cached layer again) would let the live dots be drawn as they come, - where `ModelChangeThrottle` now has the pane redrawn in full 25 times a second. - `Document::setModelSource()` (or any way to set or clear a derivation record) would replace `MainWindow::adoptTakeLayers()` setting source models by hand. - A hook in `MainWindowBase::toXml()` would save `MainWindow::toXml()` buffering the whole diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 12457761..385338d9 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -80,8 +80,7 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w `commitData()` writes into the real profile. Afterwards `~/.sv1/tmp-*.ton` is on the Recent Files list, opens, and its takes play. 7. **Live dots on this machine**: during a take the dots keep up with the cursor and grow - smoothly, and neither they nor the cursor stutter, in a maximised window. Each batch of - dots, 25 a second, has the whole pane drawn again; see open points. + smoothly, and neither they nor the cursor stutter, in a maximised window. The questions the automated checks raised, and the facts they established for the decisions above, are in [open-points.md](open-points.md). diff --git a/docs/open-points.md b/docs/open-points.md index d00fdd7b..4811901b 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -34,10 +34,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Weak spots -- **The live dots cost a redraw of the whole pane 25 times a second** during a take - (`ModelChangeThrottle`): on the cloud machine, a 1920 px window, the GUI thread went from - 29 % to about 54 % of a core. A HiDPI screen makes each redraw dearer. A way in svgui to - keep a layer out of a view's cache would make it nearly free ([forks.md](forks.md)). - **Loop Playback is left on during a take**, unlike Constrain Playback to Selection: a take that runs past the end of the reference would hear it start again while the take places what is sung after the end. Not tried. diff --git a/docs/recording.md b/docs/recording.md index 7b22975b..3af9cf0f 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -153,14 +153,21 @@ dot model outlives the take. there is not a full window yet. `kHopSize` is also the resolution of the dot model, whose unit must be `"Hz"` for the layer to align to the pane's log-frequency scale. -**Telling the pane of the dots.** A pane told of a change to one of its layers' models -draws every layer again. A notice for each dot, about 170 a second, took the GUI thread -from 29 % to nearly 80 % of a core (cloud machine, 1920 px window). So the dot model is -made with `notifyOnAdd` false, and then it tells nobody of a dot at all: the dots were -drawn only when one widened the model's pitch range, and stalled within half a second on a +**Telling the pane of the dots.** Every notice of a change to the dot model is a redraw of +the pane, and a notice for each dot would be about 170 a second. So the dot model is made +with `notifyOnAdd` false, and then it tells nobody of a dot at all: the dots were drawn +only when one widened the model's pitch range, and stalled within half a second on a steady note. `onRealtimePitchDetected()` hands each dot's frames to `m_realtimeDotsNotifier` (`ModelChangeThrottle`, `tony_core`), which tells the pane at -once and then at most every 40 ms (about 54 % of a core in the same test). +once and then at most every 40 ms. + +**The dots are kept out of the pane's cache** (`Layer::setCachedInView(false)`, svgui +fork). Told of a change to the model of a layer in its cache, a pane draws every layer in +the cache again: the reference's waveform, pitch track and notes, 25 times a second. Kept +out, each notice costs a copy of the cache and the dots. During a take in a 1920 px window +on the cloud machine the GUI thread used about 22 % of a core, against 35 % with the dots +in the cache. Every layer in front of the dots is drawn at every paint as well: only cheap +ones, such as the coverage strip, may be raised above them. Correct as they are, though they look wrong: diff --git a/docs/testing.md b/docs/testing.md index 6d4e0d9f..04e1ffbf 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -6,7 +6,7 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | | `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | | `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | `meson test` / `build.bat test` runs the first two plus four svcore suites. `test-tony-device` diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 5f548aab..b0c9af7c 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3659,11 +3659,10 @@ MainWindow::setupRealtimePitchLayer() // Unit "Hz" is required so TimeValueLayer::shouldAutoAlign() defers to // the pane's log-frequency coordinate system (same as the pYIN pitch track). // - // notifyOnAdd false: a pane told of a change to one of its layers' - // models draws all of them again, and a notice for each of the ~170 - // dots a second kept the GUI thread busy most of the time. The model - // then tells nobody of a dot, though, so m_realtimeDotsNotifier tells - // the pane of what was added, 25 times a second + // notifyOnAdd false: a notice for each of the ~170 dots a second + // would be a redraw for each. The model then tells nobody of a dot, + // though, so m_realtimeDotsNotifier tells the pane of what was added, + // 25 times a second auto pitchModel = std::make_shared (sr, RealtimePitchTracker::kHopSize, false); pitchModel->setObjectName(tr("Realtime Pitch (Live)")); @@ -3694,6 +3693,11 @@ MainWindow::setupRealtimePitchLayer() m_realtimePitchLayer->setVerticalScale(TimeValueLayer::AutoAlignScale); m_realtimePitchLayer->setPlotStyle(TimeValueLayer::PlotPoints); + // Out of the pane's cache: told of a change to the model of a layer + // in it, the pane draws every layer in it again -- the reference's + // pitch track, notes and waveform, 25 times a second + m_realtimePitchLayer->setCachedInView(false); + // Singing/recording track uses the "Orange" colour so it is visually // distinct from the reference track (black) and notes (blue). ColourDatabase *cdb = ColourDatabase::getInstance(); diff --git a/main/ModelChangeThrottle.h b/main/ModelChangeThrottle.h index 59f99094..3c6dbc9d 100644 --- a/main/ModelChangeThrottle.h +++ b/main/ModelChangeThrottle.h @@ -26,10 +26,9 @@ * made to hold back its own change notices (notifyOnAdd false), such as * the live pitch dots of a take. * - * A pane that is told of a change to one of its layers' models draws - * every layer again, so a notice for each of the ~170 dots a second - * kept the GUI thread busy for three quarters of its time. Told - * nothing, a pane never draws the dots at all. + * A pane draws itself again for every notice it is told, and the dots + * come about 170 a second. Told nothing, a pane never draws the dots + * at all. * * The first change after a quiet interval is told at once; changes * that follow within the interval are told together at its end. Lives diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index b565af3d..553cedc2 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -871,6 +871,8 @@ private slots: QVERIFY(m_window->realtimeTracker()); QVERIFY(m_window->realtimeLayer()); QVERIFY(paneHasLayer(0, m_window->realtimeLayer())); + // Drawn by itself as dots come, not with every layer of the pane + QVERIFY(!m_window->realtimeLayer()->isCachedInView()); auto model = sv::ModelById::getAs (m_window->realtimeModelId()); QVERIFY(model); diff --git a/main/test/TestViewCache.h b/main/test/TestViewCache.h new file mode 100644 index 00000000..d25f62c2 --- /dev/null +++ b/main/test/TestViewCache.h @@ -0,0 +1,152 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_VIEW_CACHE_H +#define TEST_VIEW_CACHE_H + +// Tier 3: what a pane draws again from its layers, and what it takes +// from its cache of what they drew (the svgui fork). No MainWindow. +// That the live dots of a take are kept out of the cache is +// TestRecordWorkflow's business (live_dots_appear). + +#include "view/Pane.h" +#include "view/ViewManager.h" +#include "layer/TimeValueLayer.h" +#include "data/model/SparseTimeValueModel.h" + +#include +#include +#include + +#include +#include + +class TestViewCache : public QObject +{ + Q_OBJECT + + // A layer that counts the times the pane has it draw + class CountedLayer : public sv::TimeValueLayer + { + public: + mutable int drawn = 0; + void paint(sv::LayerGeometryProvider *v, QPainter &paint, + QRect rect) const override { + ++drawn; + sv::TimeValueLayer::paint(v, paint, rect); + } + }; + + sv::ViewManager *m_viewManager = nullptr; + sv::Pane *m_pane = nullptr; + + // Back to front + std::vector m_layers; + std::vector m_models; + + void changeModelOf(int layer) { + auto model = sv::ModelById::get(m_models[layer]); + emit model->modelChangedWithin(m_models[layer], 0, 44100); + } + + // Painted until the pane has nothing left to settle, then counted + // from nothing + void settle() { + for (int i = 0; i < 3; ++i) { + QCoreApplication::processEvents(); + m_pane->repaint(); + } + for (auto layer : m_layers) layer->drawn = 0; + } + + std::vector drawn() { + std::vector counts; + for (auto layer : m_layers) counts.push_back(layer->drawn); + return counts; + } + +private slots: + void init() { + m_viewManager = new sv::ViewManager(); + m_pane = new sv::Pane(); + m_pane->setViewManager(m_viewManager); + m_pane->resize(400, 200); + + for (int i = 0; i < 3; ++i) { + auto model = std::make_shared + (44100, 256, false); + for (int j = 0; j < 20; ++j) { + model->add(sv::Event(j * 2048, 220.f + float(i * 20 + j), "")); + } + m_models.push_back(sv::ModelById::add(model)); + auto layer = new CountedLayer(); + layer->setModel(m_models.back()); + m_layers.push_back(layer); + m_pane->addLayer(layer); + } + + m_pane->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_pane)); + } + + void cleanup() { + delete m_pane; + m_pane = nullptr; + for (auto layer : m_layers) delete layer; + m_layers.clear(); + for (auto id : m_models) sv::ModelById::release(id); + m_models.clear(); + delete m_viewManager; + m_viewManager = nullptr; + } + + // Painted again with nothing changed, the pane takes what it shows + // from its cache and has none of its layers draw + void nothing_changed_draws_nothing() { + settle(); + m_pane->repaint(); + m_pane->repaint(QRect(100, 0, 20, m_pane->height())); + QCOMPARE(drawn(), std::vector({ 0, 0, 0 })); + } + + // A change to the model of any layer in the cache has every layer + // in it drawn again + void a_change_draws_the_cache_again() { + settle(); + changeModelOf(1); + m_pane->repaint(); + QCOMPARE(drawn(), std::vector({ 1, 1, 1 })); + } + + // Kept out of the cache, a layer is drawn every time the pane is + // painted, and so is every layer in front of it; a change to its + // model leaves the layers behind it in the cache + void a_layer_kept_out_is_drawn_by_itself() { + m_layers[1]->setCachedInView(false); + settle(); + + changeModelOf(1); + m_pane->repaint(); + QCOMPARE(drawn(), std::vector({ 0, 1, 1 })); + + m_pane->repaint(); + QCOMPARE(drawn(), std::vector({ 0, 2, 2 })); + + // What is still in the cache is drawn again as ever + changeModelOf(0); + m_pane->repaint(); + QCOMPARE(drawn(), std::vector({ 1, 3, 3 })); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 08fbec57..f085745e 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -12,6 +12,7 @@ */ #include "TestSingingDocument.h" +#include "TestViewCache.h" #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" #include "TestUiChecks.h" @@ -59,6 +60,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestViewCache t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestSingingAnalysis t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 027f2826..611ffb73 100644 --- a/meson.build +++ b/meson.build @@ -1388,6 +1388,7 @@ tony_core_test_exe = executable( tony_app_test_moc_files = qt.preprocess( moc_headers: [ 'main/test/TestSingingDocument.h', + 'main/test/TestViewCache.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', 'main/test/TestUiChecks.h', diff --git a/repoint-lock.json b/repoint-lock.json index e495809b..6798fdb4 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "bdac54f488f4d0346eaf692ec4a46d436db0ca87" + "pin": "15a0bd24fb69d9a03fef6ea8ef014db27d383fc0" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From aabc98923b2446318c522172d287f0c9a8a139af Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:51:42 +0000 Subject: [PATCH 102/275] fix: the calibration layout has room for four punch-ins of three sweeps Where two punch-ins meet, the finder needs 1.9 s between the last sweep of one and the first of the next; the old order had 2.5, 1.7 and 1.8 s there, so no grouping of four punch-ins of three events fitted. Two pairs of spacings are swapped. punch_ins_hold_the_events_asked_for now asks for 4 x 3 and failed on the old order; the punch-in judging test takes its events from the layout rather than from fixed times. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- main/LatencyCheck.cpp | 7 +++++-- main/test/TestLatencyCheck.h | 20 ++++++++++++++------ 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp index d19da71c..779752df 100644 --- a/main/LatencyCheck.cpp +++ b/main/LatencyCheck.cpp @@ -28,8 +28,11 @@ namespace { const double pi = 3.14159265358979323846; // Sweep to sweep, in tenths of a second so that the times are the same -// at every rate: each value from 1.6 to 2.6 s once, in an irregular order -const int calibrationSpacings[] = { 21, 16, 25, 19, 23, 17, 26, 20, 18, 24, 22 }; +// at every rate: each value from 1.6 to 2.6 s once, in an irregular order. +// The 4th, 7th and 10th are 1.9 s or more: a calibration's four punch-ins +// of three events each meet there, and the finder needs 1.9 s between +// the last sweep of one punch-in and the first of the next +const int calibrationSpacings[] = { 21, 16, 25, 19, 17, 23, 26, 20, 24, 18, 22 }; const int calibrationSpacingCount = int(sizeof(calibrationSpacings) / sizeof(calibrationSpacings[0])); diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h index 758d08e5..87acf0a5 100644 --- a/main/test/TestLatencyCheck.h +++ b/main/test/TestLatencyCheck.h @@ -939,17 +939,23 @@ private slots: } } - // 9.1 s in the first punch-in only; 11.4 s in both, so in the - // second, which recorded over the first + // Event 5 in the first punch-in only; event 6 in both, so in the + // second, which recorded over the first. They are far enough + // apart for the second to start after all the finder reads for + // event 5 + const double t5 = expectedAt(layout, 5); + const double t6 = expectedAt(layout, 6); + QVERIFY(t6 - t5 > before + after + 0.2); const frame_t later = framesOf(0.2); - punchIns = { { 8.0, 13.0 }, { 10.5, 13.0 } }; + punchIns = { { t5 - before - 0.1, t6 + after + 0.1 }, + { t6 - before - 0.1, t6 + after + 0.1 } }; s = judge(layout, spliced(reference, kRate, punchIns, { shift, later }), kRate, punchIns); QCOMPARE(s.judged, 2); QCOMPARE(s.punchIns[0].judged, 1); QCOMPARE(s.punchIns[1].judged, 1); - QCOMPARE(s.events[0].event, 4); - QCOMPARE(s.events[1].event, 5); + QCOMPARE(s.events[0].event, 5); + QCOMPARE(s.events[1].event, 6); QCOMPARE(s.events[1].punchIn, 1); QCOMPARE(s.events[1].arrival.errorFrames, later); } @@ -967,8 +973,10 @@ private slots: const double length = double(layout.length) / layout.rate; struct Case { int count; int each; }; + // 4 x 3 is what a calibration records (spec 2), and fits only + // because the spacings where the punch-ins meet are wide enough for (Case c : { Case { 2, 2 }, Case { 4, 2 }, Case { 3, 3 }, - Case { 1, 12 } }) { + Case { 4, 3 }, Case { 1, 12 } }) { const punchins_t exact = LatencyCheck::punchInsFor(layout, c.count, c.each); From 46425d3bdbbd904938fc473ec43948ed88549a01 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:51:42 +0000 Subject: [PATCH 103/275] docs: calibrate audio, B2 to B4 refined after B1 B3 is split into the check's playback and progress (B3) and the dialog and menu (B4). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 120 ++++++++++++++++++++++++++-- docs/calibrate-audio.md | 6 +- 2 files changed, 118 insertions(+), 8 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index d659c9b2..8ae20bc0 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -275,11 +275,114 @@ reuse". In `MainWindow.cpp`, read by range: `record()`, the deferred lambda in ### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") -To be refined by the lead after B1. - -### B3 — Calibrate Audio dialog and menu (spec §2) - -To be refined by the lead after B2. +Read also: `docs/recording.md` "Latency"; `main/LatencyUtils.h` whole; in +`MainWindow.cpp`, the deferred lambda in `recordingStarted()` (search +`m_takeLatency.roundTrip`) and `refineRecordingLatency()`; `AudioCheckRunner.h` +(`AudioCheckResult`, `calibrationUsable()`). + +- **New `main/LatencyCalibration.{h,cpp}`** in `tony_core`: + - **Key.** The three Preferences values `createAudioIO()` reads (`audio-target`, + then `audio-playback-device` and `audio-record-device`, each suffixed with the + implementation when one is pinned; see `audioDeviceSettingKey()` in + `MainWindow.cpp`) and the recording's rate. Device names can hold `/` and non-ASCII + characters: encode them, so that QSettings does not make subgroups. + - **Stored:** round trip and spread in seconds, the date, and the reported output + and input latency, each **in seconds**. The reported pair is the staleness + fingerprint: a stored figure is stale when either differs by more than a named + tolerance. + - `store`, `load`, `forget`, and a staleness test. QSettings group + `LatencyCalibration`. +- **Use in `recordingStarted()`.** The round trip is the stored one when there is one + for this key and it is not stale; otherwise the reported sum. Either way it is + **converted to frames at the recording's rate**, from seconds. + - Fix the reported sum's units while you are there. `getTargetPlayLatency()` counts + frames at the session's rate (`ResamplerWrapper` converts it), and + `getSystemRecordLatency()` at the device's. Today the two are added as they come; + B1 measured 242 ms instead of 256 at 48 kHz. + - The recording's model (`m_currentRecordingModelId`) exists by the time the lambda + runs, and gives the rate. + - Keep in `m_takeLatency` which source was used (reported or measured), and log it. + - The start gap and everything downstream stay as they are. +- **MainWindow API for B4.** Store a check's result, forget the stored figure, and + describe the figure in use (source, milliseconds, date). +- **Not in this phase:** any dialog, menu entry or playback change. +- **Tests:** + - **Core.** Store, load and forget round trip, with a device name holding `/` and + `ä`; the staleness tolerance on both sides; different keys stay apart. Use a + QSettings scope the tests own and clear. + - **App.** + - `latency_end_to_end`'s recipe with wrong reported latencies and a stored figure + equal to `inputDelay`: the sung step lands on the reference's. **The same test + without the stored figure must fail**; show it. + - A stale stored figure (fingerprint differs): the reported sum is used. + - 48 kHz fake: the reported sum, in recording frames, equals + `playbackLatency + recordLatency`, both device frames, to within a frame or two. + - Check, store, check again (short plan): the second check's median offset is + within a few frames of 0. This is the strongest test in the feature, and it + takes about 25 s. + +### B3 — The check's playback, and progress (spec §2, §8 "Loudness") + +Read also: `docs/architecture.md` on play parameters and on `Analyser::setAudible()` +(search "audible"); in `Analyser.cpp`, where the reference's pan and the +sonification's audibility are set (search `setPlayPan`, `PlayParameters`). + +- For the check's session only, the reference plays **centred** and at **−12 dBFS + peak** after normalisation: the model is normalised to full scale when it is read, so + the gain has to come off at playback. The pitch-track sonification is **silent**. + - Do it through the play parameters of the check session's models and layers, never + `Analyser::setAudible()`, which writes the shared settings (see `AGENTS.md`). + - Make sure nothing puts them back later in the run, for example the analysis + finishing, or the next take. + - Nothing of the user's own sessions changes. +- **A progress signal** on the runner: step, punch-in *k* of *n*, and the seconds left + where known, for B4's dialog. +- **Tests:** + - During a check the reference's play parameters are centred at the planned gain, + and the sonification is not audible. + - Afterwards, a newly opened ordinary file plays as before: reference pan and + sonification as the settings say. + - The sweeps reach `FakeAudioIO`'s captured output at about −12 dBFS on the mixed + channel. + - Progress reports each punch-in in order. + +### B4 — Calibrate Audio dialog and menu (spec §2) + +Read also: how an existing Tony dialog is built and tested (search +`confirmRecordingOverTake` and `askForTakeName` in `MainWindow.cpp` and +`TestRecordWorkflow.h`). + +- **`main/CalibrateAudioDialog.{h,cpp}`**, non-modal and thin. Its pages: + 1. **Instructions:** the output and input device names, the latency in use with its + source, "hold one earcup against the mic, off your ears", moderate volume. + 2. **Progress,** from B3's signal, with Cancel. + 3. **Result:** + - the verdict in plain words, with the fix for each failure (spec §2 and §8); + - the measured round trip against the driver's figure; + - the spread; + - both rates, with a plain sentence when they differ; + - the input peak; + - the echo, if one was heard. + + **Use this latency** (only when `calibrationUsable()`), **Check Again** and + **Close**. +- **Menu, Playback:** + - **Calibrate Audio…**, disabled while recording; + - a disabled line saying the latency in use ("Latency: measured 187 ms, 25 Sep" or + "Latency: driver's figure, 400 ms"); + - **Forget Measured Latency**. +- The calibration plan is 4 punch-ins × 3 events on the calibration layout; it fits + since the lead's spacing change. +- Anything that asks the user goes through a virtual seam, as + `confirmRecordingOverTake()` does. The app tests drive the dialog's slots directly; + the dialog watchdog fails a test on any unexpected modal dialog. +- **Tests:** + - the menu starts a check; + - Use this latency stores (B2's API), and the menu line changes; + - Forget clears it; + - the dialog's result words for NoSignal and for a rate mismatch; + - Calibrate Audio is disabled during an ordinary take. +- **After B4 the user runs it on Windows** (the checkpoint in spec §7). ### C0 — TakeDiff (spec §5 "tony_core") @@ -373,3 +476,8 @@ The next phase must know: - 48 kHz fake: Scattered, 3 of 3 found, offsets −200 ms median, as A2 foresaw. - No progress signal yet; B3's dialog may want one (`m_punchIn`). Left open: no test deletes the window mid-check. Seen while proving the session-close hook: closing a session during an **ordinary** take, then pressing Stop, hangs (pre-existing). + +### Lead — 2026-09-26, after B1 +- Reordered the calibration spacings to `{21,16,25,19,17,23,26,20,24,18,22}` (B1's suggestion) so that 4 × 3 punch-ins fit; `punch_ins_hold_the_events_asked_for` now asks for 4 × 3 and failed on the old order. `judge_only_events_inside_a_punch_in` names its events from the layout instead of 9.1 and 11.4 s. +- Split B3 into B3 (the check's playback and progress) and B4 (dialog and menu), after B1 needed 370k tokens. +- Calibration sweeps now at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, 17.7, 20.1, 21.9, 24.1 s. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index fd593e19..94a8c2e5 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -290,13 +290,15 @@ marked "Done" when it is committed. - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, `recordingStarted()`, staleness). This was step 3 below; it moved up because the dialog needs it. - - **B3** The Calibrate Audio dialog and menu entry. + - **B3** The check's playback: the reference centred at −12 dBFS, the sonification + silent, and a progress signal. + - **B4** The Calibrate Audio dialog and menu entry. **Then you run it on your PC.** Its numbers settle three things: how wrong the driver's figure is, whether the offset holds across stream restarts on MME, and whether your device's rate hits the takes. Work goes on meanwhile: only the thresholds and the restart-jitter remedy wait on those numbers. -3. **Calibration in use:** built in B2 (Use this latency and Forget in B3). +3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). 4. **Dev-check framework:** - **C0** `TakeDiff`, pure. - **C1** Build flag, `DevChecks`, `TakeObserver`, report, friend access. First From f996f4eeafe916ea527c2ecc4a818c5c47b7332d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:53:29 +0000 Subject: [PATCH 104/275] build: build tony for android as the shared library qt loads meson.build gets an android branch: the linux branch's libraries and defines less jack, pulseaudio, alsa, portaudio, oggz and fishsound, opus read-only, every library static (only then does pkg-config name their private libraries). there tony is libTony_arm64-v8a.so, which qt for android's launcher loads to call main(), with the static libraries' symbols kept out of its dynamic table; the tests are left out. deploy/android/qt-arm64-v8a.ini names qt for android's qmake for meson; deploy/android/build-tony.sh configures, builds into build-android/ and checks the libraries with readelf. the desktop build.ninja is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- .gitignore | 1 + deploy/android/build-tony.sh | 164 +++++++++++ deploy/android/qt-arm64-v8a.ini | 17 ++ docs/android-work-orders.md | 25 +- meson.build | 493 +++++++++++++++++++------------- 5 files changed, 507 insertions(+), 193 deletions(-) create mode 100755 deploy/android/build-tony.sh create mode 100644 deploy/android/qt-arm64-v8a.ini diff --git a/.gitignore b/.gitignore index 96440733..75ea997c 100644 --- a/.gitignore +++ b/.gitignore @@ -34,6 +34,7 @@ chp pyin glob:.repoint* build +build-android build_win32 build_win64 build_win64_debug diff --git a/deploy/android/build-tony.sh b/deploy/android/build-tony.sh new file mode 100755 index 00000000..d152d2a9 --- /dev/null +++ b/deploy/android/build-tony.sh @@ -0,0 +1,164 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Builds Tony for Android (arm64-v8a, API 28) in the Ubuntu 24.04 cloud +# container, into build-android/ in the repository: +# +# build-android/libTony_arm64-v8a.so the application: a shared +# library, which Qt for Android's +# launcher loads to call its main() +# build-android/pyin.so, chp.so the Vamp plugins +# +# Run deploy/android/setup-toolchain.sh, build-qt.sh and build-deps.sh +# first. meson gets two cross files: the one build-deps.sh writes, for +# the NDK and the libraries, and deploy/android/qt-arm64-v8a.ini, for +# Qt. The build type is the desktop builds' (build.bat, container-setup.sh): +# optimised, with asserts and debug information. The libraries here keep +# that information, for symbolising a crash from the phone; the copies +# in an APK are to be stripped. +# +# The build directory is configured once; after that ninja configures it +# again by itself when meson.build or a cross file changes. --wipe +# configures it from scratch, for a new NDK or Qt. The logs are +# /opt/android/logs/tony-setup.log and tony-build.log. It ends by checking +# the application library: an arm64 shared object, aligned for 16 KB +# pages, exporting main(), and loading nothing but Android's system +# libraries, the NDK's C++ library and Qt's; and the plugins likewise, +# exporting their Vamp entry point. +# +# Usage, from anywhere: +# deploy/android/build-tony.sh [--wipe] + +set -eu -o pipefail + +wipe=no +if [ "$#" -eq 1 ] && [ "$1" = "--wipe" ]; then + wipe=yes +elif [ "$#" -ne 0 ]; then + echo "Usage: $0 [--wipe]" 1>&2 + exit 2 +fi + +android=/opt/android +ndk=$android/sdk/ndk/27.2.12479018 +logs=$android/logs +cross=$android/cross-arm64-v8a.ini + +repo=$(cd "$(dirname "$0")/../.." && pwd) +qt_cross=$repo/deploy/android/qt-arm64-v8a.ini +build=$repo/build-android + +readelf=$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf + +jobs=$(nproc) + +# Qt's qmake, which meson runs, warns at every start in the container's +# C locale +export LC_ALL=C.UTF-8 + +if [ ! -f "$cross" ]; then + echo "ERROR: no $cross: run deploy/android/build-deps.sh first" 1>&2 + exit 1 +fi +qmake=$(sed -n "s/^qmake6 = '\(.*\)'$/\1/p" "$qt_cross") +if [ ! -x "$qmake" ]; then + echo "ERROR: no Qt for Android at $qmake: run deploy/android/build-qt.sh first" 1>&2 + exit 1 +fi + +mkdir -p "$logs" + +# 1. Configure + +setup_args=(--buildtype=debugoptimized --cross-file "$cross" --cross-file "$qt_cross") + +log=$logs/tony-setup.log +if [ -f "$build/build.ninja" ] && [ "$wipe" = "no" ]; then + echo "build-android/: configured already" +else + if [ -f "$build/build.ninja" ]; then + echo "Configuring build-android/ from scratch" + setup_args+=(--wipe) + else + echo "Configuring build-android/" + fi + if ! meson setup "$build" "$repo" "${setup_args[@]}" > "$log" 2>&1; then + tail -30 "$log" 1>&2 + echo "ERROR: meson setup failed; the whole log is $log" 1>&2 + exit 1 + fi +fi + +# 2. Build: the application library and the plugins, which on Android +# are all there is (meson.build leaves the tests out) + +echo "Building" +log=$logs/tony-build.log +if ! ninja -j "$jobs" -C "$build" > "$log" 2>&1; then + if grep -q "error:" "$log"; then + grep "error:" "$log" | head -30 1>&2 + else + tail -30 "$log" 1>&2 + fi + echo "ERROR: the build failed; the whole log is $log" 1>&2 + exit 1 +fi + +# 3. Check + +echo +echo "Checking" + +# check +check() { + local so="$build/$1" symbol="$2" allowed="$3" + local kind needed align + kind=$(file -b "$so") + needed=$("$readelf" -d "$so" | sed -n 's/.*(NEEDED).*\[\(.*\)\]/\1/p' | tr '\n' ' ') + align=$("$readelf" -lW "$so" | awk '$1 == "LOAD" { print $NF }' | sort -u | tr '\n' ' ') + echo " $1: $(echo "$kind" | cut -d, -f1-2)" + echo " it loads: $needed" + case "$kind" in + "ELF 64-bit LSB shared object, ARM aarch64"*) ;; + *) echo "ERROR: not an arm64 shared object" 1>&2; exit 1 ;; + esac + for lib in $needed; do + if ! echo "$lib" | grep -qxE "$allowed"; then + echo "ERROR: it loads $lib, which is not among the libraries it may load" 1>&2 + exit 1 + fi + done + if [ "$align" != "0x4000 " ]; then + echo "ERROR: its segments are not aligned for 16 KB pages" 1>&2 + exit 1 + fi + if ! "$readelf" --dyn-syms -W "$so" | + awk -v s="$symbol" '$8 == s && $5 == "GLOBAL" && $6 == "DEFAULT" && $7 != "UND" { found = 1 } + END { exit !found }'; then + echo "ERROR: it does not export $symbol" 1>&2 + exit 1 + fi + echo " it exports $symbol" +} + +system_libraries='lib(c|m|dl|log|z|android|c\+\+_shared)\.so' + +check libTony_arm64-v8a.so main "$system_libraries|libQt6[A-Za-z]+_arm64-v8a\.so" +check pyin.so vampGetPluginDescriptor "$system_libraries" +check chp.so vampGetPluginDescriptor "$system_libraries" + +cat <= 3.0.0', static: true) + sndfile_dep = dependency('sndfile', version: '>= 1.0.16', static: true) + samplerate_dep = dependency('samplerate', version: '>= 0.1.2', static: true) + rubberband_dep = dependency('rubberband', version: '>= 3.0.0', static: true) + sord_dep = dependency('sord-0', version: '>= 0.5', static: true) + serd_dep = dependency('serd-0', version: '>= 0.5', static: true) + mad_dep = dependency('mad', version: '>= 0.15.0', static: true) + id3tag_dep = dependency('id3tag', version: '>= 0.15.0', static: true) + opus_dep = dependency('opusfile', static: true) + + feature_dependencies = [ + vamphostsdk_dep, + bzip2_dep, + fftw3_dep, + sndfile_dep, + samplerate_dep, + rubberband_dep, + sord_dep, + serd_dep, + mad_dep, + id3tag_dep, + opus_dep, + ] + + feature_defines = [ + '-DHAVE_BZ2', + '-DHAVE_FFTW3', + '-DFFTW_DOUBLE_ONLY', + '-DHAVE_SNDFILE', + '-DHAVE_LIBSAMPLERATE', + '-DHAVE_RUBBERBAND', + '-DHAVE_SORD', '-DUSE_SORD', + '-DHAVE_SERD', + '-DHAVE_MAD', + '-DHAVE_ID3TAG', + '-DHAVE_OPUS', + '-DHAVE_OPUS_READ_ONLY', + ] + + svcore_moc_args = [ + '-DHAVE_MAD' + ] + + vamp_symbol_args += [ + '-Wl,--version-script=' + meson.current_source_dir() / 'vamp-plugin-sdk/vamp-plugin.map' + ] + else error('This operating system ("' + system + '") is not supported by this build file') endif # system @@ -1166,6 +1232,10 @@ endif if system == 'linux' tony_main_name = 'tony' +elif system == 'android' + # androiddeployqt looks for lib_.so, and the + # deployment settings name the application-binary Tony + tony_main_name = 'Tony_' + android_abi else tony_main_name = 'Tony' endif # system @@ -1226,207 +1296,246 @@ tony_app_dep = declare_dependency( # link_whole so that the application contains exactly the objects it did # when these sources were compiled straight into it. -executable( - tony_main_name, - qt_resource_files, - tony_entry_files, - rc, - link_whole: [ - tony_app_lib, - tony_core_lib, - ], - dependencies: [ - svcore_dep, - qt_dep, - feature_dependencies, - os_dep, - dl_dep - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'windows', - install: true, -) - -svcore_base_test_exe = executable( - 'test-svcore-base', - svcore_base_test_moc_files, - 'svcore/base/test/svcore-base-test.cpp', - dependencies: [ - svcore_dep, - qt_dep, - feature_dependencies, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) +if system == 'android' + # Qt for Android's Java launcher loads the application as a shared + # library and calls its main(), the one symbol it has to export. + # --exclude-libs keeps whatever comes from a static library (Tony's + # own, svcore, the C libraries) out of its dynamic symbol table. + shared_library( + tony_main_name, + qt_resource_files, + tony_entry_files, + link_whole: [ + tony_app_lib, + tony_core_lib, + ], + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + '-Wl,--exclude-libs,ALL', + ], + install: true, + ) +else + executable( + tony_main_name, + qt_resource_files, + tony_entry_files, + rc, + link_whole: [ + tony_app_lib, + tony_core_lib, + ], + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'windows', + install: true, + ) +endif # system -svcore_system_test_exe = executable( - 'test-svcore-system', - svcore_system_test_moc_files, - 'svcore/system/test/svcore-system-test.cpp', - dependencies: [ - svcore_dep, - qt_dep, - feature_dependencies, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) +# The tests run on the desktop only: the Android build is the +# application and its plugins. +if system != 'android' + + svcore_base_test_exe = executable( + 'test-svcore-base', + svcore_base_test_moc_files, + 'svcore/base/test/svcore-base-test.cpp', + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -svcore_data_model_test_exe = executable( - 'test-svcore-data-model', - svcore_data_model_test_moc_files, - 'svcore/data/model/test/MockWaveModel.cpp', - 'svcore/data/model/test/svcore-data-model-test.cpp', - dependencies: [ - svcore_dep, - qt_dep, - feature_dependencies, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) + svcore_system_test_exe = executable( + 'test-svcore-system', + svcore_system_test_moc_files, + 'svcore/system/test/svcore-system-test.cpp', + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -svcore_data_fileio_test_exe = executable( - 'test-svcore-data-fileio', - svcore_data_fileio_test_moc_files, - 'svcore/data/model/test/MockWaveModel.cpp', - 'svcore/data/fileio/test/UnsupportedFormat.cpp', - 'svcore/data/fileio/test/svcore-data-fileio-test.cpp', - dependencies: [ - svcore_dep, - qt_dep, - feature_dependencies, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) + svcore_data_model_test_exe = executable( + 'test-svcore-data-model', + svcore_data_model_test_moc_files, + 'svcore/data/model/test/MockWaveModel.cpp', + 'svcore/data/model/test/svcore-data-model-test.cpp', + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -# Tony's own tests. Two executables, because they differ in cost and in -# what they link: test-tony-core needs no GUI and no audio device; -# test-tony-app creates widgets (offscreen) and links svgui and svapp. + svcore_data_fileio_test_exe = executable( + 'test-svcore-data-fileio', + svcore_data_fileio_test_moc_files, + 'svcore/data/model/test/MockWaveModel.cpp', + 'svcore/data/fileio/test/UnsupportedFormat.cpp', + 'svcore/data/fileio/test/svcore-data-fileio-test.cpp', + dependencies: [ + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -tony_core_test_moc_files = qt.preprocess( - moc_headers: [ - 'main/test/TestRealtimeYin.h', - 'main/test/TestRealtimePitchTracker.h', - 'main/test/TestLatencyShift.h', - 'main/test/TestCoverage.h', - 'main/test/TestTakeAudio.h', - 'main/test/TestTakeEvents.h', - 'main/test/TestSingingTakes.h', - 'main/test/TestTakesFile.h', - 'main/test/TestTakeTiming.h', -]) + # Tony's own tests. Two executables, because they differ in cost and in + # what they link: test-tony-core needs no GUI and no audio device; + # test-tony-app creates widgets (offscreen) and links svgui and svapp. + + tony_core_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestRealtimeYin.h', + 'main/test/TestRealtimePitchTracker.h', + 'main/test/TestLatencyShift.h', + 'main/test/TestCoverage.h', + 'main/test/TestTakeAudio.h', + 'main/test/TestTakeEvents.h', + 'main/test/TestSingingTakes.h', + 'main/test/TestTakesFile.h', + 'main/test/TestTakeTiming.h', + ]) + + tony_core_test_exe = executable( + 'test-tony-core', + tony_core_test_moc_files, + 'main/test/PyinReference.cpp', + 'pyin/YinUtil.cpp', + 'vamp-plugin-sdk/src/vamp-sdk/FFT.cpp', + 'main/test/tony-core-test.cpp', + dependencies: [ + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -tony_core_test_exe = executable( - 'test-tony-core', - tony_core_test_moc_files, - 'main/test/PyinReference.cpp', - 'pyin/YinUtil.cpp', - 'vamp-plugin-sdk/src/vamp-sdk/FFT.cpp', - 'main/test/tony-core-test.cpp', - dependencies: [ - tony_core_dep, - svcore_dep, - qt_dep, - feature_dependencies, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) + tony_app_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestSingingDocument.h', + 'main/test/TestSingingAnalysis.h', + 'main/test/TestRecordWorkflow.h', + ]) + + tony_app_test_exe = executable( + 'test-tony-app', + tony_app_test_moc_files, + 'main/test/tony-app-test.cpp', + dependencies: [ + tony_app_dep, + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) -tony_app_test_moc_files = qt.preprocess( - moc_headers: [ - 'main/test/TestSingingDocument.h', - 'main/test/TestSingingAnalysis.h', - 'main/test/TestRecordWorkflow.h', -]) + test('tony-core', tony_core_test_exe) + test('tony-app', tony_app_test_exe, + depends: pyin_plugin, + timeout: 900, + env: [ 'QT_QPA_PLATFORM=offscreen' ]) -tony_app_test_exe = executable( - 'test-tony-app', - tony_app_test_moc_files, - 'main/test/tony-app-test.cpp', - dependencies: [ - tony_app_dep, - tony_core_dep, - svcore_dep, - qt_dep, - feature_dependencies, - os_dep, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) + test('svcore-base', svcore_base_test_exe) + test('svcore-system', svcore_system_test_exe) + test('svcore-data-model', svcore_data_model_test_exe) + test('svcore-data-fileio', svcore_data_fileio_test_exe, + args: [ + '--testdir', meson.current_source_dir() / 'svcore/data/fileio/test' + ]) -test('tony-core', tony_core_test_exe) -test('tony-app', tony_app_test_exe, - depends: pyin_plugin, - timeout: 900, - env: [ 'QT_QPA_PLATFORM=offscreen' ]) - -test('svcore-base', svcore_base_test_exe) -test('svcore-system', svcore_system_test_exe) -test('svcore-data-model', svcore_data_model_test_exe) -test('svcore-data-fileio', svcore_data_fileio_test_exe, - args: [ - '--testdir', meson.current_source_dir() / 'svcore/data/fileio/test' - ]) +endif # system summary({'prefix': get_option('prefix'), 'bindir': get_option('bindir'), From 44cae60d3296db431afdeb336167daa04ef6d213 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 00:59:09 +0000 Subject: [PATCH 105/275] fix: lyrics boxes match the word times, and stay in one row Picks up the svgui change that draws each box exactly as long as its word and places the labels on their own, moving them sideways before giving them a second row. Contiguous words went to alternate rows because their touching boxes left no gap. Tests for the placement, the exact box and the single row replace those for the widened box. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- docs/forks.md | 31 ++++-- docs/manual-checklist.md | 24 ++-- docs/open-points.md | 18 +-- main/LyricsTrack.cpp | 4 +- main/test/TestLyricsLayer.h | 212 +++++++++++++++++++++++++++++------- repoint-lock.json | 2 +- 6 files changed, 218 insertions(+), 73 deletions(-) diff --git a/docs/forks.md b/docs/forks.md index 09bff04e..47cd99ad 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -73,23 +73,30 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` - `RegionLayer::PlotStrip` plot style: the coverage strip. Saved through the existing `plotStyle` attribute. - `RegionLayer::PlotLyrics` plot style, after `PlotStrip` so saved numbers keep their - meaning: the lyrics. Each region's label is centred in a light box, dark text whatever - the view's colours, in two rows along the bottom of the view just above `PlotStrip`'s - 8 px (row 0 lowest), with a bar in the base colour under each region. A box spans the - region, or the label centred on it where the label is longer (`getLyricsBoxSpan()`). The - first word of a line (where the value changes) is bold. The font - (`getLyricsFontPixelSize()`) is twice the view's at the least, grows with the zoom up to - four times, and is never more than an eighth of the view's height. No vertical scale, no - feature description, not editable. The static, pure `assignLabelRows()` places the boxes - (Tony's app suite tests it): one that fits in no row is left out. + meaning: the lyrics. Each region is a light box in one row along the bottom of the view + just above `PlotStrip`'s 8 px, **exactly as long as the region** and never widened for + its label: the box edges are the word times, and two words next to each other share the + line between them. The label, dark text with a halo in the box's colour whatever the + view's colours, is centred on the box and runs over its edges where it is longer. The + static, pure `placeLyricsLabels()` places the labels (Tony's app suite tests it): one + that would come closer than a small gap to the label before it is moved right, and the + labels before it left, as little as will do but never so far that a label's middle + leaves its box; one that cannot be goes in a second row above the boxes, and one with no + room there either is left out (its box is still drawn). There is no gap between boxes + to keep, so contiguous words whose labels fit share the first row. The first word of a + line (where the value changes) is bold. The font (`getLyricsFontPixelSize()`) is twice + the view's at the least, up to four times, and never more than an eighth of the view's + height; it grows with the **square root** of the zoom, so that zooming in gives the + words room (their boxes grow with the zoom itself). No vertical scale, no feature + description, not editable. `setHighlightFrame()` draws the region at that frame in amber (the latest to start, where regions overlap) and emits `layerParametersChanged()` only when that region changes: the highlight is painted into the view's cache, so each new word repaints the view, a few times a second at most, and only views listen to that signal, so nothing is marked modified. `getHighlightedEvent()` says which region it is. A highlighted word that was - left out is drawn in row 0 over the others for as long as it is highlighted: at the - usual zoom the larger font leaves many words out, and the one being sung is the one the - singer must be able to read. Where a label goes depends on the labels before it, so the + left out is drawn in the boxes' row, centred on its box, over the others for as long as + it is highlighted, with an amber halo: the one being sung is the one the singer must be + able to read. Where a label goes depends on the labels before it, so the layout is made for the **whole model** at once and cached per zoom level and font; a strip newly scrolled into sight then agrees with what is already on show. That is what lets the layer stay **scrollable**: `View::getNonScrollableFrontLayers()` treats every diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 20e75837..c1917573 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -110,15 +110,19 @@ Launch with `.\build.bat run`. 37. **Legibility**: the words, dark on light boxes along the bottom of the pane, are readable over the waveform, the pitch tracks, the alternate pitch track and the live dots low in the range. The waveform is pale grey (225, 225, 225) while the lyrics are - on show: faint enough for the words, still enough to see where the singing is? Grey - bars under the boxes: right colour? -38. **Rows and font** at the zoom used while singing: can the words be read while - singing, and do the two rows hide too much of the bottom of the pitch range? The font - grows as you zoom in (twice the usual size up to four times, never more than an eighth - of the pane's height): right size at each zoom, words centred in their boxes? -39. **Density**: zoom out until words drop out (they are left out, never drawn over each - other, except the highlighted one) and back in (they return). At the usual zoom the - larger font leaves many words out: how many on a fast song, and are two rows enough? + on show: faint enough for the words, still enough to see where the singing is? Where + a word is longer than its box it runs over the box's edges with a light halo: still + readable over the waveform, and still clearly that box's word? +38. **Box edges and rows** at the zoom used while singing: each box's edges are the + word's start and end, never widened, and words next to each other share an edge. Do + the words stay in one row, and is the space between two labels enough to tell them + apart? The font grows as you zoom in (twice the usual size up to four times, never + more than an eighth of the pane's height), more slowly than the boxes: does zooming in + put the words that were in the second row back in the first? +39. **Density**: zoom out until words go to the second row, above the boxes, and then + drop out (their boxes stay; the highlighted one's word is still drawn) and back in + (they return). At the zoom you sing at, how many words leave the first row on a fast + song? 40. **Bold line starts**: do they read as the start of a phrase, or as noise? 41. **The left edge**: a word in the first ~30 px of the view is under the pane's vertical scale, at the bottom left (scroll so that a word is at the left edge). How much does @@ -126,7 +130,7 @@ Launch with `.\build.bat run`. 42. **Inferred ends**: the exporter writes no end times. With word timing the last word of a line ends at the next line but at most 2 s after it starts, unless a `♪` line marks the end; with line timing a line lasts until the next one, and the last line 5 s. Do - those boxes and bars mislead, and does the last word of a line stay highlighted too + those boxes mislead, and does the last word of a line stay highlighted too long? The start times are exact. 43. **Hover readout**: with the lyrics shown, hovering over the pitch tracks gives the same readout and the same vertical scale as without them, also after turning the alternate diff --git a/docs/open-points.md b/docs/open-points.md index 04820e4a..89f3ded0 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -56,14 +56,16 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. (about 86 px/s) showed 1.6 s before 0 s, so a word at 0 s was clear of the scale, but the half of its box before 0 s was under the pale wash the pane draws before the start of the reference. -- **The lyrics' layout is a guess at what reads well**: two rows (a word with no room is - left out), the bold line starts, the font size, and the 2 s / 5 s caps on inferred ends - are all constants to be judged by eye ([manual checklist](manual-checklist.md)). -- **At the usual zoom many words have no room**: with the font twice the view's at the - least, most labels are wider than the time their word takes on screen, so the boxes - overlap their neighbours and two rows do not hold them all (seen at 100 px/s). Only the - word being sung is always drawn, over the others if need be. Zooming in, a smaller font - or a third row would each help. +- **The lyrics' layout is a guess at what reads well**: the gap between labels (a sixth of + the font size), how far a label may move from its box (until its middle would leave it), + the second row, the bold line starts, the font size and its growth, and the 2 s / 5 s + caps on inferred ends are all constants to be judged by eye + ([manual checklist](manual-checklist.md)). +- **Zoomed out, words still go to the second row or are left out**: with the font twice + the view's at the least, a fast word's label is wider than its box. Rendered with + made-up but realistic timing (about three words a second) and a 26 px font: one row at + 200 px/s and above, a few words in the second row at 150 px/s, many in it and some left + out at 100 px/s. Only the word being sung is always drawn. - **Right after `closeSession()`, Show Lyrics and the alternate pitch actions keep their enabled and checked states** until the next reference or session opens: nothing there calls `updateLayerStatuses()`. Show Lyrics then does nothing when chosen. diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index 3f84e16d..dcf678b7 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -163,8 +163,8 @@ LyricsTrack::configureLayer() m_layer->setVerticalScale(RegionLayer::EqualSpaced); m_layer->setPlotStyle(RegionLayer::PlotLyrics); - // The bar under each word; the words themselves are dark on light - // boxes whatever the colours. The layer's default would be black + // The words are dark on light boxes whatever this is; grey rather + // than the layer's default black wherever else its colour shows m_layer->setBaseColour (ColourDatabase::getInstance()->getColourIndex(tr("Grey"))); diff --git a/main/test/TestLyricsLayer.h b/main/test/TestLyricsLayer.h index 46f7d407..4cac8aed 100644 --- a/main/test/TestLyricsLayer.h +++ b/main/test/TestLyricsLayer.h @@ -16,7 +16,7 @@ // Tier 3: the lyrics plot style of RegionLayer (svgui fork), which // draws the words of the lyrics along the bottom of pane 0. Where each -// box goes and how big its font is are worked out by pure functions, +// label goes and how big its font is are worked out by pure functions, // tested first; then the layer is painted into images, with no // MainWindow. @@ -44,7 +44,16 @@ class TestLyricsLayer : public QObject { Q_OBJECT - typedef std::vector> Spans; + typedef std::vector> Places; + + static Places place(const std::vector &labels, + int rows = 2, double gap = 4) { + Places places; + for (const auto &p : sv::RegionLayer::placeLyricsLabels(labels, rows, gap)) { + places.push_back({ p.row, p.left }); + } + return places; + } QTemporaryDir m_dir; @@ -181,53 +190,65 @@ private slots: // --- Where labels go ------------------------------------------------- - void labels_that_fit_share_the_first_row() { - Spans spans { { 0, 10 }, { 20, 10 }, { 40, 10 } }; - QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), - std::vector({ 0, 0, 0 })); + void contiguous_labels_that_fit_their_boxes_share_the_first_row() { + // Each word ends where the next starts, as in the lyrics + QCOMPARE(place({ { 0, 100, 50 }, { 100, 200, 50 }, { 200, 300, 50 } }), + Places({ { 0, 25 }, { 0, 125 }, { 0, 225 } })); } - void a_label_that_overlaps_goes_to_the_next_row() { - // The second runs into the first; the third clears the first - Spans spans { { 0, 30 }, { 10, 30 }, { 45, 10 } }; - QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), - std::vector({ 0, 1, 0 })); + void a_label_wider_than_its_box_moves_right_into_room() { + QCOMPARE(place({ { 0, 40, 20 }, { 40, 80, 60 } }), + Places({ { 0, 10 }, { 0, 34 } })); } - void a_label_with_no_room_in_any_row_is_left_out() { - Spans spans { { 0, 50 }, { 10, 50 }, { 20, 50 }, { 70, 10 } }; - QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), - std::vector({ 0, 1, -1, 0 })); + void the_labels_before_move_left_to_make_room() { + QCOMPARE(place({ { 0, 40, 30 }, { 40, 60, 50 } }), + Places({ { 0, 1 }, { 0, 35 } })); + // Through more than one of them, each only as far as it must + QCOMPARE(place({ { 0, 40, 30 }, { 40, 80, 30 }, { 80, 100, 60 } }), + Places({ { 0, 2 }, { 0, 36 }, { 0, 70 } })); + QCOMPARE(place({ { 0, 40, 30 }, { 40, 80, 30 }, { 80, 100, 50 } }), + Places({ { 0, 5 }, { 0, 41 }, { 0, 75 } })); } - void the_gap_is_the_least_space_between_labels() { - QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 }, { 14, 10 } }, 2, 4), - std::vector({ 0, 0 })); - QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 }, { 13.5, 10 } }, 2, 4), - std::vector({ 0, 1 })); + void a_label_whose_middle_would_leave_its_box_goes_to_the_next_row() { + // The first would have to move 34 left, but may move only 10; + // it stays where it was + QCOMPARE(place({ { 0, 20, 60 }, { 20, 40, 60 } }), + Places({ { 0, -20 }, { 1, 0 } })); + // A long label centred on a short region reaches back past the + // label before it, and cannot move far enough + QCOMPARE(place({ { 95, 105, 10 }, { 105, 115, 100 }, { 120, 130, 10 } }), + Places({ { 0, 95 }, { 1, 60 }, { 0, 120 } })); + } + + void a_label_with_no_room_in_any_row_is_left_out() { + QCOMPARE(place({ { 0, 20, 60 }, { 20, 40, 60 }, { 40, 60, 60 }, + { 200, 220, 20 } }), + Places({ { 0, -20 }, { 1, 0 }, { -1, 20 }, { 0, 200 } })); } - void a_label_reaching_back_over_the_one_before_goes_to_the_next_row() { - // A long label centred on a short region can start left of the - // label of the region before it - Spans spans { { 100, 10 }, { 60, 100 }, { 115, 10 } }; - QCOMPARE(sv::RegionLayer::assignLabelRows(spans, 2, 4), - std::vector({ 0, 1, 0 })); + void the_gap_is_the_least_space_between_labels() { + // Regions of no width: their labels cannot move + QCOMPARE(place({ { 5, 5, 10 }, { 19, 19, 10 } }), + Places({ { 0, 0 }, { 0, 14 } })); + QCOMPARE(place({ { 5, 5, 10 }, { 18.5, 18.5, 10 } }), + Places({ { 0, 0 }, { 1, 13.5 } })); } - void a_box_is_its_region_or_its_label_centred_on_the_region() { - typedef std::pair Span; - QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 200, 50), Span(100, 100)); - QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 200, 100), Span(100, 100)); - QCOMPARE(sv::RegionLayer::getLyricsBoxSpan(100, 120, 60), Span(80, 60)); + void no_labels_and_no_rows() { + QCOMPARE(place({}), Places()); + QCOMPARE(place({ { 0, 10, 10 } }, 0), Places({ { -1, 0 } })); } void the_font_grows_with_the_zoom_from_twice_to_four_times() { - // Base 13 pixels in a tall view: 26 to 52, a pixel for every 10 - // pixels per second in between + // Base 13 pixels in a tall view: 26 to 52, growing from 260 + // pixels per second with the square root of the zoom QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(50, 13, 800), 26); - QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(300, 13, 800), 30); - QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(400, 13, 800), 40); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(260, 13, 800), 26); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(300, 13, 800), 28); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(585, 13, 800), 39); + QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(1040, 13, 800), 52); QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(5000, 13, 800), 52); // Never more than an eighth of the view, nor less than its font QCOMPARE(sv::RegionLayer::getLyricsFontPixelSize(5000, 13, 200), 25); @@ -242,10 +263,18 @@ private slots: } } - void no_labels_and_no_rows() { - QCOMPARE(sv::RegionLayer::assignLabelRows({}, 2, 4), std::vector()); - QCOMPARE(sv::RegionLayer::assignLabelRows({ { 0, 10 } }, 0, 4), - std::vector({ -1 })); + void the_font_grows_more_slowly_than_the_boxes() { + // Zooming in twice as far makes every box twice as long and its + // word less than one and a half times as large, so that it fits + // better + for (double pps = 260; pps <= 520; pps *= 1.1) { + int size = sv::RegionLayer::getLyricsFontPixelSize(pps, 13, 800); + int zoomedIn = sv::RegionLayer::getLyricsFontPixelSize(pps * 2, 13, 800); + QVERIFY2(zoomedIn <= 1.5 * size, + qPrintable(QString("%1 pixels at %2 pixels a second, %3 at " + "twice the zoom") + .arg(size).arg(pps).arg(zoomedIn))); + } } // --- The layer -------------------------------------------------------- @@ -382,10 +411,113 @@ private slots: QVERIFY(m_layer->getHighlightedEvent(e)); QCOMPARE(e.getLabel(), QString("sanaseppo2")); QImage highlighted = render({ QRect(0, 0, kWidth, kHeight) }); - QVERIFY2(highlightPixels(highlighted) > 100, + + // Its box is there whatever happens; its label, far wider, + // has the halo of a highlighted box around it + int x0 = m_pane->getXForFrame(e.getFrame()); + int x1 = m_pane->getXForFrame(e.getFrame() + e.getDuration()); + QImage outside = highlighted; + QPainter painter(&outside); + painter.fillRect(x0, 0, x1 - x0, kHeight, Qt::white); + painter.end(); + QVERIFY2(highlightPixels(outside) > 50, "the word being sung was not drawn: it had no row"); } + // The top edge of the boxes: the highest row in which this column + // has anything drawn in it + static int topDrawnRow(const QImage &image, int x) { + for (int y = 0; y < image.height(); ++y) { + if (image.pixel(x, y) != qRgb(255, 255, 255)) return y; + } + return -1; + } + + static bool isBoxEdge(QRgb p) { + return qRed(p) < 200 && qBlue(p) - qRed(p) >= 20; + } + + void contiguous_words_that_fit_are_drawn_in_one_row() { + // Each word ends where the next starts, and each label is far + // shorter than its box + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + const double length = 0.6; + for (int i = 0; i < 10; ++i) { + model->add(sv::Event(sv::sv_frame_t(kRate * length * i), 0.f, + sv::sv_frame_t(kRate * length), + QString("w%1").arg(i))); + } + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + + int top = -1; + for (int i = 0; i < 10; ++i) { + int x = m_pane->getXForFrame(sv::sv_frame_t(kRate * length * i)) + 8; + if (x >= kWidth) break; + int y = topDrawnRow(image, x); + QVERIFY2(y > 0, qPrintable(QString("nothing drawn for word %1").arg(i))); + if (top < 0) top = y; + QVERIFY2(y == top, + qPrintable(QString("word %1's box starts at y = %2, the " + "first word's at y = %3") + .arg(i).arg(y).arg(top))); + } + for (int y = 0; y < top; ++y) { + QVERIFY2(rowIsWhite(image, y), + qPrintable(QString("something was drawn at y = %1, above " + "the boxes at y = %2").arg(y).arg(top))); + } + } + + void a_box_is_its_region_even_when_its_label_is_wider() { + // Twenty pixels of region, and a label several times as wide + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + sv::Event e(sv::sv_frame_t(kRate * 3.0), 0.f, sv::sv_frame_t(kRate * 0.2), + QString("sanaseppo")); + model->add(e); + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + + int x0 = m_pane->getXForFrame(e.getFrame()); + int x1 = m_pane->getXForFrame(e.getFrame() + e.getDuration()); + QCOMPARE(x1 - x0, 20); + + // The box's top edge, a row above anything of the label's + int top = -1; + for (int y = 0; y < kHeight; ++y) { + if (isBoxEdge(image.pixel(x0 + 10, y))) { + top = y; + break; + } + } + QVERIFY2(top >= 0, "no box edge was found"); + + int left = -1, right = -1; + for (int x = 0; x < kWidth; ++x) { + if (isBoxEdge(image.pixel(x, top))) { + if (left < 0) left = x; + right = x; + } + } + QVERIFY2(left >= x0 && right <= x1 - 1, + qPrintable(QString("the box's top edge runs from x = %1 to %2, " + "its region from %3 to %4") + .arg(left).arg(right).arg(x0).arg(x1 - 1))); + + // And the label is drawn, past both ends of the box + bool textLeft = false, textRight = false; + for (int y = top; y < kHeight; ++y) { + for (int x = 0; x < x0 - 2; ++x) { + if (qGray(image.pixel(x, y)) < 100) textLeft = true; + } + for (int x = x1 + 2; x < kWidth; ++x) { + if (qGray(image.pixel(x, y)) < 100) textRight = true; + } + } + QVERIFY2(textLeft && textRight, + "the label is not drawn centred over its box, past both ends"); + } + void painting_in_strips_matches_painting_whole() { // A view that scrolls repaints only the strip that comes into // sight; the labels in it must be where they were when the diff --git a/repoint-lock.json b/repoint-lock.json index bef369da..1aace33f 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "e2c736d08ec5150b1d74c5e8b6052360a966777c" + "pin": "11231118534c1099b11554d2414459d51470e0dc" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 47944f27a7d7361c3c7b1f90f9cb435aef3ecf6b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:25:50 +0000 Subject: [PATCH 106/275] feat: takes use the round trip the audio check measured A figure the check measured is stored per device (the Preferences createAudioIO() reads, and the recording's rate) together with the latencies the device reported at the time; if those change, the figure is stale and the reported sum is used again. recordingStarted() now works the round trip out in seconds and converts it at the recording's rate: the reported output latency is in the play source's frames and the input latency in the device's, which were added as they came. latency_measured_round_trip_used lands the sung step on the reference's with wrong reported latencies and a stored figure, and was seen to fail without it; the 48 kHz sum test failed with the old sum; a check, a store and a second check finds no offset. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green (TestAudioCheck 9, TestRecordWorkflow 96, TestSingingAnalysis 20, TestSingingDocument 12). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 30 ++- docs/calibrate-audio.md | 12 +- main/LatencyCalibration.cpp | 192 +++++++++++++++++++ main/LatencyCalibration.h | 135 ++++++++++++++ main/LatencyUtils.h | 21 ++- main/MainWindow.cpp | 132 ++++++++++++- main/MainWindow.h | 46 ++++- main/test/TestAudioCheck.h | 52 ++++++ main/test/TestLatencyCalibration.h | 277 ++++++++++++++++++++++++++++ main/test/TestRecordWorkflow.h | 168 +++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 12 files changed, 1056 insertions(+), 18 deletions(-) create mode 100644 main/LatencyCalibration.cpp create mode 100644 main/LatencyCalibration.h create mode 100644 main/test/TestLatencyCalibration.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 8ae20bc0..91cad742 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (see git log). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -337,6 +337,15 @@ sonification's audibility are set (search `setPlayPan`, `PlayParameters`). - Nothing of the user's own sessions changes. - **A progress signal** on the runner: step, punch-in *k* of *n*, and the seconds left where known, for B4's dialog. +- **Two runner fixes from B2's report:** + - **Reference file name.** The reference WAV gets a new name each run (a counter or a + timestamp), and old ones are removed when no session holds them. **Check Again** + otherwise rewrites the file that the check session it replaces still has open; on + Windows that write can fail. Linux cannot show it. + - **Reported output latency.** `AudioCheckRunner` divides the reported output latency + by the reference's rate. Have `TakeLatency` carry both reported figures **in + seconds**, as `MainWindow::roundTripAt()` works them out, and have the result use + those. Then the take path and the check can never disagree. - **Tests:** - During a check the reference's play parameters are centred at the planned gain, and the sonification is not audible. @@ -373,6 +382,10 @@ Read also: how an existing Tony dialog is built and tested (search - **Forget Measured Latency**. - The calibration plan is 4 punch-ins × 3 events on the calibration layout; it fits since the lead's spacing change. +- **The device key is taken when the check starts.** Carry it in `AudioCheckResult`, + and have `storeMeasuredLatency()` use it, not the Preferences at the moment the + button is pressed. The dialog is non-modal, so the user could change device in + between (B2's report). Also disable both Audio Device menus while a check runs. - Anything that asks the user goes through a virtual seam, as `confirmRecordingOverTake()` does. The app tests drive the dialog's slots directly; the dialog watchdog fails a test on any unexpected modal dialog. @@ -481,3 +494,18 @@ Left open: no test deletes the window mid-check. Seen while proving the session- - Reordered the calibration spacings to `{21,16,25,19,17,23,26,20,24,18,22}` (B1's suggestion) so that 4 × 3 punch-ins fit; `punch_ins_hold_the_events_asked_for` now asks for 4 × 3 and failed on the old order. `judge_only_events_inside_a_punch_in` names its events from the layout instead of 9.1 and 11.4 s. - Split B3 into B3 (the check's playback and progress) and B4 (dialog and menu), after B1 needed 370k tokens. - Calibration sweeps now at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, 17.7, 20.1, 21.9, 24.1 s. + +### Phase B2 — 2026-09-26 +Built: `main/LatencyCalibration.{h,cpp}` (`tony_core`, namespace): `Key`, `currentKey(settings, rate)`, `Figure`, `store`/`load`/`forget` (all take a `QSettings &`), `isStale`, `kStaleToleranceSeconds` = 1 ms, `Source`, `InUse {source, roundTrip, date, stale}`, `roundTripInUse()`, `reportedSeconds()`, `toFrames()`. `TakeLatency::measured`. `MainWindow`: `roundTripAt(rate)`, used by the `recordingStarted()` lambda; B4's API `storeMeasuredLatency(result)`, `forgetMeasuredLatency()`, `latencyInUse()`. Core class `TestLatencyCalibration` (6 tests); app tests `latency_measured_round_trip_used`, `latency_stale_round_trip_ignored`, `latency_reported_at_the_device_rate`, `latency_reported_with_device_opened_first` (TestRecordWorkflow), `check_stored_round_trip_is_used` (TestAudioCheck, 25 s). +Choices / deviations: +- Settings: `LatencyCalibration/||//`. In names only `%`, `/`, `\` and `|` are percent-encoded. Registry key names stop at 255 characters, and full encoding of long non-ASCII names could pass that. Values are stored as text (`'g'`, 17 digits) and the date as ISO UTC. +- The key follows `createAudioIO()`, not `audioDeviceSettingKey()`: they differ for `audio-target` = "auto", which Tony never writes. +- **The output latency's unit** (spec §11, corrected): frames at `m_playSource->getDeviceSampleRate()`, or at the recording's rate when that is 0. This is not always the session's rate. If a device is chosen before any file is opened (or a standalone take is the first action), `ResamplerWrapper` has no source rate yet. It passes the device's figure through unconverted and tells the play source 0. Using the session's rate there gave 13012 frames instead of 12288 at 48 kHz (`latency_reported_with_device_opened_first`). +- `storeMeasuredLatency()` stores only when `calibrationUsable()`. The key uses the current Preferences, the rate is the result's, and the fingerprint is the result's reported pair. +- `latencyInUse()`/`forget` use the rate of the last take placed with a round trip. Before any take they use the session's rate, the only rate at which a usable check stores. That rate is reset when a device is chosen from the menu. The device's rate cannot be known before a take: `AudioCallbackRecordTarget` has no getter for it. +- A stale figure is not deleted; it becomes valid again if the driver goes back to reporting the old pair. +The next phase must know: +- With no figure stored at 44.1 kHz, the round trip is exactly the old sum; this is tested in core across a grid of values. At 48 kHz the check now uses 256 ms, not 242. +- The runner's `reportedOutputLatency` still divides by the reference's rate. It matches the take path at 44.1 kHz, the only rate that is stored. +- `storeMeasuredLatency()` reads the device from the Preferences when "Use this latency" is pressed. If B4's non-modal dialog lets the device change in between, the figure is stored under the new device. +Left open: `computeRecordingLatency()` is unused outside `TestLatencyShift`. The svapp fork could add `AudioCallbackRecordTarget::getRecordSampleRate()` so that `latencyInUse()` knows the rate before the first take. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 94a8c2e5..8ded2211 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -289,7 +289,7 @@ marked "Done" when it is committed. - **B1** The alignment check runner and its app tests. Done. - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, `recordingStarted()`, staleness). This was step 3 below; it moved up because the - dialog needs it. + dialog needs it. Done. - **B3** The check's playback: the reference centred at −12 dBFS, the sonification silent, and a progress signal. - **B4** The Calibrate Audio dialog and menu entry. @@ -376,7 +376,15 @@ Checked on 2026-09-25, so that phases do not re-derive them. measurement. *Found in B1:* `getTargetPlayLatency()` counts frames of the session's rate (bqaudioio's `ResamplerWrapper` converts it), `getSystemRecordLatency()` the device's, and L is taken off the recording, in the device's. They differ only when - the device is not at 44.1 kHz. + the device is not at 44.1 kHz. *Found in B2:* the wrapper converts only if the + session had a rate when the device was opened. A device chosen before any file is + opened gets its figure passed on in its own frames, and the play source is told a + device rate of 0. So the output latency counts frames at the play source's + `getDeviceSampleRate()`, or at the device's rate when that is 0. + *Since B2* the round trip is the stored figure (`LatencyCalibration`) when one is + valid, otherwise the reported pair, each converted to seconds at its own rate. It + is then turned into recording frames. `computeRecordingLatency()` is no longer + called. - **Where the reference is heard** (found in B1). `Analyser` pans the reference hard left and its pitch and notes sonification hard right, so only the left earcup carries the sweeps; the right one carries the synth. diff --git a/main/LatencyCalibration.cpp b/main/LatencyCalibration.cpp new file mode 100644 index 00000000..4e6f6433 --- /dev/null +++ b/main/LatencyCalibration.cpp @@ -0,0 +1,192 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LatencyCalibration.h" + +#include + +#include +#include + +using namespace sv; + +namespace LatencyCalibration { +namespace { + +const char *const settingsGroup = "LatencyCalibration"; + +// A device name as part of a group name. QSettings takes "/" and "\" +// as the start of a subgroup, so they are percent-encoded, and so is +// "%" itself, and the "|" that separates the names. Nothing else is: +// every settings format keeps any other character, non-ASCII ones +// included, and the Windows registry allows a key name 255 characters +// at most, which a name encoded whole could run past +QString encoded(QString name) +{ + name.replace("%", "%25"); + name.replace("/", "%2F"); + name.replace("\\", "%5C"); + name.replace("|", "%7C"); + return name; +} + +QString devicesGroup(const Key &key) +{ + return encoded(key.implementation) + "|" + + encoded(key.playbackDevice) + "|" + + encoded(key.recordDevice); +} + +QString rateGroup(const Key &key) +{ + return QString::number(qint64(std::llround(key.rate))); +} + +// As text, so that every settings format keeps all of the figure +QString number(double value) +{ + return QString::number(value, 'g', 17); +} + +double numberFrom(const QVariant &value, bool *ok = nullptr) +{ + return value.toString().toDouble(ok); +} + +} + +Key +currentKey(QSettings &settings, sv_samplerate_t recordingRate) +{ + Key key; + settings.beginGroup("Preferences"); + key.implementation = settings.value("audio-target", "").toString(); + QString suffix; + if (key.implementation != "") suffix = "-" + key.implementation; + key.playbackDevice = + settings.value("audio-playback-device" + suffix, "").toString(); + key.recordDevice = + settings.value("audio-record-device" + suffix, "").toString(); + settings.endGroup(); + key.rate = recordingRate; + return key; +} + +void +store(QSettings &settings, const Key &key, const Figure &figure) +{ + settings.beginGroup(settingsGroup); + settings.beginGroup(devicesGroup(key)); + settings.beginGroup(rateGroup(key)); + settings.setValue("roundTrip", number(figure.roundTrip)); + settings.setValue("spread", number(figure.spread)); + settings.setValue("date", + figure.date.toUTC().toString(Qt::ISODateWithMs)); + settings.setValue("reportedOutput", number(figure.reportedOutput)); + settings.setValue("reportedInput", number(figure.reportedInput)); + settings.endGroup(); + settings.endGroup(); + settings.endGroup(); +} + +bool +load(QSettings &settings, const Key &key, Figure &figure) +{ + Figure f; + bool ok = false; + settings.beginGroup(settingsGroup); + settings.beginGroup(devicesGroup(key)); + settings.beginGroup(rateGroup(key)); + f.roundTrip = numberFrom(settings.value("roundTrip"), &ok); + f.spread = numberFrom(settings.value("spread")); + f.date = QDateTime::fromString(settings.value("date").toString(), + Qt::ISODateWithMs); + f.reportedOutput = numberFrom(settings.value("reportedOutput")); + f.reportedInput = numberFrom(settings.value("reportedInput")); + settings.endGroup(); + settings.endGroup(); + settings.endGroup(); + + if (!ok || !std::isfinite(f.roundTrip) || f.roundTrip < 0.0) { + return false; + } + figure = f; + return true; +} + +void +forget(QSettings &settings, const Key &key) +{ + settings.beginGroup(settingsGroup); + settings.beginGroup(devicesGroup(key)); + settings.remove(rateGroup(key)); + bool empty = settings.childGroups().isEmpty() && + settings.childKeys().isEmpty(); + settings.endGroup(); + // Figures for the devices at no other rate: nothing left of them + if (empty) settings.remove(devicesGroup(key)); + settings.endGroup(); +} + +bool +isStale(const Figure &figure, double reportedOutput, double reportedInput) +{ + return std::fabs(figure.reportedOutput - reportedOutput) > + kStaleToleranceSeconds || + std::fabs(figure.reportedInput - reportedInput) > + kStaleToleranceSeconds; +} + +const char * +sourceName(Source source) +{ + switch (source) { + case Source::Reported: return "reported"; + case Source::Measured: return "measured"; + } + return ""; +} + +InUse +roundTripInUse(const Figure *stored, + double reportedOutput, double reportedInput) +{ + InUse inUse; + if (stored && !isStale(*stored, reportedOutput, reportedInput)) { + inUse.source = Source::Measured; + inUse.roundTrip = stored->roundTrip; + inUse.date = stored->date; + return inUse; + } + inUse.source = Source::Reported; + inUse.roundTrip = std::max(reportedOutput, 0.0) + + std::max(reportedInput, 0.0); + inUse.stale = (stored != nullptr); + return inUse; +} + +double +reportedSeconds(sv_frame_t frames, sv_samplerate_t rate) +{ + if (frames <= 0 || !(rate > 0)) return 0.0; + return double(frames) / rate; +} + +sv_frame_t +toFrames(double seconds, sv_samplerate_t rate) +{ + if (!(seconds > 0.0) || !(rate > 0)) return 0; + return sv_frame_t(std::llround(seconds * rate)); +} + +} diff --git a/main/LatencyCalibration.h b/main/LatencyCalibration.h new file mode 100644 index 00000000..aabebf8c --- /dev/null +++ b/main/LatencyCalibration.h @@ -0,0 +1,135 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LATENCY_CALIBRATION_H +#define TONY_LATENCY_CALIBRATION_H + +#include "base/BaseTypes.h" + +#include +#include + +class QSettings; + +/** + * The round trip the audio check measured, kept for the devices it was + * measured on, and the choice between it and the latencies the device + * reports, which is what a take is placed with. + * + * A figure is kept per key: the audio driver and the playback and + * record devices, as the Preferences name them for + * MainWindowBase::createAudioIO(), which opens them, and the rate the + * device records at. Next to it are the latencies the device reported + * when it was measured. If the device reports others now, its buffers + * have changed (in the driver's control panel, say), the round trip + * has changed with them, and the figure is stale: takes go back to the + * reported pair until the check is run again. + * + * In the settings group "LatencyCalibration", one group for the driver + * and devices, within it one for the rate. All times are in seconds. + */ +namespace LatencyCalibration +{ + /// How far either reported latency may move before a stored figure + /// is stale. A device reports its latencies in whole frames, and the + /// output latency is rounded again, to frames of the session, when + /// the device runs at another rate: both well under this + constexpr double kStaleToleranceSeconds = 0.001; + + struct Key { + /// As the Preferences have them, "" for the default + QString implementation; + QString playbackDevice; + QString recordDevice; + + /// The rate the device records at + sv::sv_samplerate_t rate; + + Key() : rate(0) { } + }; + + /** + * The key for the devices the Preferences name, read as + * MainWindowBase::createAudioIO() reads them: "audio-target", then + * "audio-playback-device" and "audio-record-device", each suffixed + * with "-" and the driver when one is named. + */ + Key currentKey(QSettings &settings, sv::sv_samplerate_t recordingRate); + + struct Figure { + /// What the check measured + double roundTrip; + double spread; + QDateTime date; + + /// What the device reported then: the staleness fingerprint + double reportedOutput; + double reportedInput; + + Figure() : roundTrip(0), spread(0), + reportedOutput(0), reportedInput(0) { } + }; + + /// Keep the figure for the key, over any kept for it before + void store(QSettings &settings, const Key &key, const Figure &figure); + + /// The figure kept for the key, if there is one + bool load(QSettings &settings, const Key &key, Figure &figure); + + /// Drop the figure kept for the key, if there is one + void forget(QSettings &settings, const Key &key); + + /// Whether either latency the device reports now differs from the + /// one it reported when the figure was measured by more than + /// kStaleToleranceSeconds + bool isStale(const Figure &figure, + double reportedOutput, double reportedInput); + + enum class Source { + Reported, ///< the device's reported output and input latency + Measured ///< a figure the audio check measured + }; + + const char *sourceName(Source source); + + /// The round trip a take is placed with, and where it came from + struct InUse { + Source source; + double roundTrip; + + /// When it was measured; null for the reported pair + QDateTime date; + + /// A figure was stored for the key, but it is stale + bool stale; + + InUse() : source(Source::Reported), roundTrip(0), stale(false) { } + }; + + /** + * The stored figure, if there is one and it is not stale; otherwise + * the sum of the two reported latencies. stored may be null. + */ + InUse roundTripInUse(const Figure *stored, + double reportedOutput, double reportedInput); + + /// A reported latency, counted in frames at the given rate, in + /// seconds. A latency reported as zero or less, or a rate not yet + /// known, counts as none + double reportedSeconds(sv::sv_frame_t frames, sv::sv_samplerate_t rate); + + /// Seconds in whole frames at the given rate, rounded to the nearest + sv::sv_frame_t toFrames(double seconds, sv::sv_samplerate_t rate); +} + +#endif diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index d5dfdf94..7e652a7b 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -35,27 +35,32 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, /** * What a take was placed with: the round trip taken off the front of - * its recording (besides the start gap, which is measured), the output - * and input latencies the device reported, and the rate the device - * recorded at. All 0 until known; the round trip stays 0 for a take - * made without the reference playing, which is placed with none. + * its recording (besides the start gap, which is measured), whether + * that was a figure the audio check measured or the sum of the output + * and input latencies the device reported, those two latencies, and the + * rate the device recorded at. All 0 until known; the round trip stays + * 0 for a take made without the reference playing, which is placed with + * none. * * In frames as the window has them. The round trip is taken off the * recording, and the input latency is the device's, so both count * frames of the recording; but the play source reports the output * latency in frames of the session, converted when it resamples to the - * device. The two kinds differ only when the device's rate is not the - * session's, and the round trip then adds the one to the other. + * device (unless the device was opened before the session had a rate; + * see MainWindow::roundTripAt()). The two kinds differ only when the + * device's rate is not the session's; the round trip is worked out in + * seconds for that reason. */ struct TakeLatency { sv::sv_frame_t roundTrip; + bool measured; sv::sv_frame_t reportedOutput; sv::sv_frame_t reportedInput; sv::sv_samplerate_t recordingRate; - TakeLatency() : roundTrip(0), reportedOutput(0), reportedInput(0), - recordingRate(0) { } + TakeLatency() : roundTrip(0), measured(false), reportedOutput(0), + reportedInput(0), recordingRate(0) { } /// Frames of the recording in seconds; 0 while its rate is unknown double recordingSeconds(sv::sv_frame_t frames) const { diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index fd2a35d6..f0e0fe6d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -203,7 +203,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_awaitingReferenceStart(false), m_takeLatency(), m_audioCheck(nullptr), - m_audioCheckTakes(false) + m_audioCheckTakes(false), + m_lastRecordingRate(0) { setWindowTitle(QApplication::applicationName()); @@ -1455,6 +1456,9 @@ MainWindow::audioDeviceSelected(QAction *action) stop(); } + // Another device may record at another rate + m_lastRecordingRate = 0; + recreateAudioIO(); } @@ -4174,7 +4178,9 @@ MainWindow::recordingStarted() // The singer's response to the reference where playback starts // arrives in the recording at approximately frame // (outputLatency + inputLatency), so that is the frame the splice - // reads the recording from. + // reads the recording from. Drivers often report those two + // wrong; a round trip the audio check measured on this device + // is used instead, while there is one (roundTripAt()). // With a pre-roll, playback starts at the beginning of the // lead-in rather than at the take's position, and the splice // skips the lead-in as well (TakeTiming::spliceOffset()). @@ -4193,14 +4199,47 @@ MainWindow::recordingStarted() // figure, and refineRecordingLatency() picks it up. m_recordingStartGapEstimate = m_recordTarget ? m_recordTarget->getFramesReceived() : 0; + + // The round trip is taken off the front of the recording, so it + // is counted in frames of the recording, at the device's rate. + // The reported latencies are not both counted so: the play + // source usually has the output latency in frames of the + // session. They differ when the device is not at 44.1 kHz, so + // the round trip goes through seconds + sv_samplerate_t recordingRate = 0; + if (auto wfm = ModelById::getAs + (m_currentRecordingModelId)) { + recordingRate = wfm->getSampleRate(); + } + if (recordingRate <= 0) { + recordingRate = sessionRate(); + cerr << "MainWindow::recordingStarted: the recording's rate " + << "is not known; taking the session's, " + << recordingRate << " Hz" << endl; + } + m_lastRecordingRate = recordingRate; + + LatencyCalibration::InUse inUse = roundTripAt(recordingRate); sv_frame_t roundTrip = - computeRecordingLatency(outputLatency, inputLatency); + LatencyCalibration::toFrames(inUse.roundTrip, recordingRate); m_recordingLatencyFrames = roundTrip + m_recordingStartGapEstimate; m_takeLatency.roundTrip = roundTrip; m_takeLatency.reportedOutput = outputLatency; m_takeLatency.reportedInput = inputLatency; - cerr << "MainWindow::recordingStarted: output latency=" << outputLatency + m_takeLatency.measured = + (inUse.source == LatencyCalibration::Source::Measured); + cerr << "MainWindow::recordingStarted: round trip " << roundTrip + << " frames at " << recordingRate << " Hz (" + << inUse.roundTrip * 1000.0 << " ms), " + << LatencyCalibration::sourceName(inUse.source); + if (inUse.source == LatencyCalibration::Source::Measured) { + cerr << " on " << inUse.date.toString(Qt::ISODate).toStdString(); + } else if (inUse.stale) { + cerr << ": the measured one is stale, the device reports " + << "other latencies now"; + } + cerr << "; output latency=" << outputLatency << " input latency=" << inputLatency << " estimated start gap=" << m_recordingStartGapEstimate << " total compensation=" << m_recordingLatencyFrames << " frames" << endl; @@ -4240,6 +4279,91 @@ MainWindow::currentRecordingLatency() const return m_recordingLatencyFrames + (measured - m_recordingStartGapEstimate); } +sv_samplerate_t +MainWindow::sessionRate() const +{ + sv_samplerate_t rate = m_playSource ? m_playSource->getSourceSampleRate() : 0; + if (rate <= 0) rate = Preferences::getInstance()->getFixedSampleRate(); + return rate; +} + +LatencyCalibration::InUse +MainWindow::roundTripAt(sv_samplerate_t recordingRate) const +{ + // The play source counts its output latency in frames at the rate it + // was told the device runs at, as its own getCurrentPlayingFrame() + // does. bqaudioio's ResamplerWrapper tells it the session's rate and + // converts the device's figure to it, when the session had a rate by + // the time the device was opened. If it had none yet (a device chosen + // before any file was opened), the wrapper passed the figure on as the + // device counts it and told the play source 0: the recording's frames + sv_samplerate_t outputRate = + m_playSource ? m_playSource->getDeviceSampleRate() : 0; + if (outputRate <= 0) outputRate = recordingRate; + + double output = m_playSource ? + LatencyCalibration::reportedSeconds + (m_playSource->getTargetPlayLatency(), outputRate) : 0.0; + double input = m_recordTarget ? + LatencyCalibration::reportedSeconds + (m_recordTarget->getSystemRecordLatency(), recordingRate) : 0.0; + + QSettings settings; + LatencyCalibration::Figure figure; + bool stored = LatencyCalibration::load + (settings, LatencyCalibration::currentKey(settings, recordingRate), + figure); + return LatencyCalibration::roundTripInUse + (stored ? &figure : nullptr, output, input); +} + +sv_samplerate_t +MainWindow::expectedRecordingRate() const +{ + return m_lastRecordingRate > 0 ? m_lastRecordingRate : sessionRate(); +} + +LatencyCalibration::InUse +MainWindow::latencyInUse() const +{ + return roundTripAt(expectedRecordingRate()); +} + +bool +MainWindow::storeMeasuredLatency(const AudioCheckResult &result) +{ + if (!result.calibrationUsable()) return false; + + LatencyCalibration::Figure figure; + figure.roundTrip = result.calibratedRoundTrip; + figure.spread = result.summary.spread; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = result.reportedOutputLatency; + figure.reportedInput = result.reportedInputLatency; + + QSettings settings; + LatencyCalibration::store + (settings, LatencyCalibration::currentKey(settings, result.recordingRate), + figure); + cerr << "MainWindow::storeMeasuredLatency: round trip " + << figure.roundTrip * 1000.0 << " ms at " << result.recordingRate + << " Hz, the device reporting " << figure.reportedOutput * 1000.0 + << " ms out and " << figure.reportedInput * 1000.0 << " ms in" + << endl; + return true; +} + +void +MainWindow::forgetMeasuredLatency() +{ + QSettings settings; + LatencyCalibration::forget + (settings, LatencyCalibration::currentKey(settings, + expectedRecordingRate())); + cerr << "MainWindow::forgetMeasuredLatency: at " + << expectedRecordingRate() << " Hz" << endl; +} + TakeTiming MainWindow::currentTakeTiming() const { diff --git a/main/MainWindow.h b/main/MainWindow.h index fd03fd34..49ca203c 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -25,6 +25,7 @@ #include "TakeCommands.h" #include "TakeTiming.h" #include "LatencyUtils.h" +#include "LatencyCalibration.h" #include #include @@ -37,6 +38,7 @@ class QComboBox; class QActionGroup; class AudioCheckRunner; +struct AudioCheckResult; namespace sv { class VersionTester; @@ -89,6 +91,23 @@ class MainWindow : public sv::MainWindowBase void toXml(QTextStream &out, bool asTemplate) override; FileOpenStatus openSession(sv::FileSource source) override; + // The round trip takes are placed with (see LatencyCalibration). + // Keep the one an audio check measured, for the devices the + // Preferences name and the rate the check recorded at; false, with + // nothing kept, unless the check's figure is usable + bool storeMeasuredLatency(const AudioCheckResult &result); + + // Drop the figure latencyInUse() describes + void forgetMeasuredLatency(); + + // What the next take will be placed with, as far as it is known + // before the take starts: the device's rate is known only once it + // has recorded, so until a take has been recorded on these devices + // this assumes the session's rate, the only one a usable check + // stores a figure at. The reported pair is 0 until the device is + // open + LatencyCalibration::InUse latencyInUse() const; + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -829,9 +848,10 @@ protected slots: // what the singer sang would be left unanalysed. Coverage::Range m_takeAnalysisRange; - // Round-trip hardware latency (output + input, in frames at the model - // sample rate) stored when a singing-track recording is made with the - // "play reference while recording" toggle on. The recording is read + // Round-trip hardware latency (the figure the audio check measured, + // or else output + input as the device reports them, in frames of the + // recording; see roundTripAt()) stored when a singing-track recording + // is made with the "play reference while recording" toggle on. The recording is read // from this frame on when it is spliced into the take's audio, so that // what the singer sang in answer to the reference at m_takePosition // lands there; and the live dots are placed with it during the take. @@ -865,6 +885,26 @@ protected slots: AudioCheckRunner *m_audioCheck; bool m_audioCheckTakes; + // The rate the device recorded at, the last time a take was placed + // with a round trip; 0 until then, and again once another device is + // chosen + sv::sv_samplerate_t m_lastRecordingRate; + + // The session's rate, from the play source, or the fixed rate every + // file is read at before there is one + sv::sv_samplerate_t sessionRate() const; + + // The rate the next take is expected to record at: the last one's, + // or before there is one, the session's + sv::sv_samplerate_t expectedRecordingRate() const; + + // The round trip for a take recorded at the given rate, in seconds, + // and where it came from: a stored figure for these devices and this + // rate, unless the latencies the device reports have changed since + // it was measured; otherwise the reported pair, each converted from + // the frames it counts + LatencyCalibration::InUse roundTripAt(sv::sv_samplerate_t recordingRate) const; + void refineRecordingLatency(); // The best figure for the latency as things stand, measurement diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 18124afc..09a37f02 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -213,6 +213,10 @@ private slots: } void cleanup() { + // A round trip a test stored would place the next test's takes. + // First, as the waits below return early when they fail + QSettings().remove("LatencyCalibration"); + if (m_window) { if (m_window->recordTarget()->isRecording()) { m_window->doRecord(); @@ -289,6 +293,49 @@ private slots: QVERIFY(!m_window->audioCheckTakes()); } + // The round trip the check measured, kept: a second check places its + // takes with it, and finds them where they belong. Forgotten, the + // reported pair is in use again + void check_stored_round_trip_is_used() { + makeWindow(loopback()); + + runCheck(); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_result.calibrationUsable(), describe(m_result).constData()); + const double measured = m_result.calibratedRoundTrip; + + QVERIFY(m_window->storeMeasuredLatency(m_result)); + LatencyCalibration::InUse inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Measured); + QCOMPARE(inUse.roundTrip, measured); + QVERIFY(inUse.date.isValid()); + + m_result = AudioCheckResult(); + m_finished = 0; + runCheck(); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QCOMPARE(r.summary.found, 4); + QVERIFY2(std::fabs(r.summary.medianOffset * rate) <= 4.0, + describe(r).constData()); + QCOMPARE(int(r.takes.size()), 2); + for (const TakeLatency &t : r.takes) { + QVERIFY(t.measured); + QCOMPARE(t.roundTrip, sv::sv_frame_t(std::llround(measured * rate))); + } + QVERIFY2(std::fabs(r.calibratedRoundTrip * rate - roundTrip) <= 4.0, + describe(r).constData()); + + m_window->forgetMeasuredLatency(); + inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Reported); + QVERIFY(!inUse.stale); + QCOMPARE(inUse.roundTrip, (reportedOut + reportedIn) / rate); + } + // A device at 48 kHz: the takes are recorded at a rate other than the // reference's, which is reported with both rates, whatever the sweeps // say. What Tony does with such takes is a known bug of its own, so @@ -307,6 +354,11 @@ private slots: QCOMPARE(m_result.referenceRate, rate); QVERIFY(!m_result.calibrationUsable()); QVERIFY(!m_window->audioCheckTakes()); + + // and such a figure is not kept + QVERIFY(!m_window->storeMeasuredLatency(m_result)); + QSettings settings; + QVERIFY(!settings.childGroups().contains("LatencyCalibration")); } // Cancel during a take stops it through the Stop path, clears the diff --git a/main/test/TestLatencyCalibration.h b/main/test/TestLatencyCalibration.h new file mode 100644 index 00000000..b806062c --- /dev/null +++ b/main/test/TestLatencyCalibration.h @@ -0,0 +1,277 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LATENCY_CALIBRATION_H +#define TEST_LATENCY_CALIBRATION_H + +// Tier 2: the stored round trip, its key and its staleness, and the +// choice between it and the reported latencies. No window and no +// device. The settings are an INI file of each test's own, in a +// temporary directory, and never the suite's application settings. + +#include "../LatencyCalibration.h" +#include "../LatencyUtils.h" + +#include +#include +#include +#include +#include + +class TestLatencyCalibration : public QObject +{ + Q_OBJECT + + typedef LatencyCalibration::Key Key; + typedef LatencyCalibration::Figure Figure; + typedef LatencyCalibration::InUse InUse; + typedef LatencyCalibration::Source Source; + + QTemporaryDir m_dir; + int m_counter = 0; + + // The settings file of the test that is running + QString m_path; + + static Key key(QString playback, QString record, double rate = 44100, + QString implementation = "") { + Key k; + k.implementation = implementation; + k.playbackDevice = playback; + k.recordDevice = record; + k.rate = rate; + return k; + } + + static Figure figure(double roundTrip, double output, double input) { + Figure f; + f.roundTrip = roundTrip; + f.spread = 0.0021; + f.date = QDateTime::fromString("2026-09-26T10:30:15.250Z", + Qt::ISODateWithMs); + f.reportedOutput = output; + f.reportedInput = input; + return f; + } + + static double loaded(QSettings &settings, const Key &k) { + Figure f; + if (!LatencyCalibration::load(settings, k, f)) return -1.0; + return f.roundTrip; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void init() { + m_path = m_dir.filePath(QString("settings-%1.ini").arg(++m_counter)); + } + + void cleanup() { + QFile::remove(m_path); + } + + // What is stored comes back, from the file, with device names that + // hold the characters QSettings takes as a subgroup, and more; the + // names make no subgroups of their own; and forget() takes it away + void stored_figure_comes_back() { + const Key k = key(QString::fromUtf8("Line 1/2 (Ger\xc3\xa4t f\xc3\xbcr Audio)"), + QString::fromUtf8("Mic \\ In | 100% (\xc3\xa4)")); + const Figure stored = figure(0.2814285714, 8192 / 44100.0, + 4096 / 44100.0); + { + QSettings settings(m_path, QSettings::IniFormat); + Figure none; + QVERIFY(!LatencyCalibration::load(settings, k, none)); + LatencyCalibration::store(settings, k, stored); + settings.sync(); + QCOMPARE(settings.status(), QSettings::NoError); + } + + QSettings settings(m_path, QSettings::IniFormat); + Figure f; + QVERIFY(LatencyCalibration::load(settings, k, f)); + QCOMPARE(f.roundTrip, stored.roundTrip); + QCOMPARE(f.spread, stored.spread); + QCOMPARE(f.date, stored.date); + QCOMPARE(f.reportedOutput, stored.reportedOutput); + QCOMPARE(f.reportedInput, stored.reportedInput); + + // One group for the devices, one within it for the rate, and + // nothing more + settings.beginGroup("LatencyCalibration"); + QCOMPARE(settings.childGroups().size(), 1); + settings.beginGroup(settings.childGroups().front()); + QCOMPARE(settings.childGroups(), QStringList { "44100" }); + QVERIFY(settings.childKeys().isEmpty()); + settings.endGroup(); + settings.endGroup(); + + LatencyCalibration::forget(settings, k); + QVERIFY(!LatencyCalibration::load(settings, k, f)); + settings.beginGroup("LatencyCalibration"); + QVERIFY(settings.childGroups().isEmpty()); + settings.endGroup(); + + // Forgetting what is not there does nothing + LatencyCalibration::forget(settings, k); + QVERIFY(!LatencyCalibration::load(settings, k, f)); + } + + // Different devices, driver or rate: each figure is its own. The "|" + // between the names is not confused with one inside a name + void keys_stay_apart() { + QSettings settings(m_path, QSettings::IniFormat); + const std::vector keys { + key("Speakers", "Mic"), + key("Headphones", "Mic"), + key("Speakers", "Line In"), + key("Speakers", "Mic", 48000), + key("Speakers", "Mic", 44100, "portaudio"), + key("a|b", "c"), + key("a", "b|c"), + key("a/b", "c"), + key("a", "b/c"), + }; + for (int i = 0; i < int(keys.size()); ++i) { + LatencyCalibration::store(settings, keys[i], + figure(0.1 + 0.01 * i, 0.1, 0.05)); + } + for (int i = 0; i < int(keys.size()); ++i) { + QCOMPARE(loaded(settings, keys[i]), 0.1 + 0.01 * i); + } + QCOMPARE(loaded(settings, key("Speakers", "Mic", 96000)), -1.0); + QCOMPARE(loaded(settings, key("Speakers", "")), -1.0); + + // Forgetting one leaves the rest, the same devices at another + // rate among them + LatencyCalibration::forget(settings, keys[0]); + QCOMPARE(loaded(settings, keys[0]), -1.0); + for (int i = 1; i < int(keys.size()); ++i) { + QCOMPARE(loaded(settings, keys[i]), 0.1 + 0.01 * i); + } + } + + // The key names the devices MainWindowBase::createAudioIO() opens: + // the plain preferences, or, with a driver named, those suffixed + // with it + void key_follows_the_preferences() { + QSettings settings(m_path, QSettings::IniFormat); + Key k = LatencyCalibration::currentKey(settings, 48000); + QCOMPARE(k.implementation, QString()); + QCOMPARE(k.playbackDevice, QString()); + QCOMPARE(k.recordDevice, QString()); + QCOMPARE(k.rate, 48000.0); + + settings.setValue("Preferences/audio-playback-device", "Speakers"); + settings.setValue("Preferences/audio-record-device", "Mic"); + k = LatencyCalibration::currentKey(settings, 44100); + QCOMPARE(k.playbackDevice, QString("Speakers")); + QCOMPARE(k.recordDevice, QString("Mic")); + QCOMPARE(k.rate, 44100.0); + + settings.setValue("Preferences/audio-target", "portaudio"); + settings.setValue("Preferences/audio-playback-device-portaudio", + "Headphones"); + k = LatencyCalibration::currentKey(settings, 44100); + QCOMPARE(k.implementation, QString("portaudio")); + QCOMPARE(k.playbackDevice, QString("Headphones")); + QCOMPARE(k.recordDevice, QString()); + } + + // Stale when either reported latency has moved by more than the + // tolerance, either way; not when it has moved by less + void stale_beyond_the_tolerance() { + const double out = 8192 / 44100.0; + const double in = 4096 / 44100.0; + const Figure f = figure(0.28, out, in); + const double within = 0.9 * LatencyCalibration::kStaleToleranceSeconds; + const double beyond = 1.1 * LatencyCalibration::kStaleToleranceSeconds; + + QVERIFY(!LatencyCalibration::isStale(f, out, in)); + for (double sign : { 1.0, -1.0 }) { + QVERIFY(!LatencyCalibration::isStale(f, out + sign * within, in)); + QVERIFY(!LatencyCalibration::isStale(f, out, in + sign * within)); + QVERIFY(LatencyCalibration::isStale(f, out + sign * beyond, in)); + QVERIFY(LatencyCalibration::isStale(f, out, in + sign * beyond)); + } + } + + // The stored figure while it is fresh, the reported sum otherwise + void round_trip_in_use() { + const double out = 0.2; + const double in = 0.1; + + InUse none = LatencyCalibration::roundTripInUse(nullptr, out, in); + QVERIFY(none.source == Source::Reported); + QCOMPARE(none.roundTrip, out + in); + QVERIFY(none.date.isNull()); + QVERIFY(!none.stale); + + const Figure fresh = figure(0.345, out, in); + InUse measured = LatencyCalibration::roundTripInUse(&fresh, out, in); + QVERIFY(measured.source == Source::Measured); + QCOMPARE(measured.roundTrip, 0.345); + QCOMPARE(measured.date, fresh.date); + QVERIFY(!measured.stale); + + const Figure old = figure(0.345, out + 0.01, in); + InUse stale = LatencyCalibration::roundTripInUse(&old, out, in); + QVERIFY(stale.source == Source::Reported); + QCOMPARE(stale.roundTrip, out + in); + QVERIFY(stale.date.isNull()); + QVERIFY(stale.stale); + + // A latency reported as less than nothing counts as none + InUse negative = LatencyCalibration::roundTripInUse(nullptr, -0.5, in); + QCOMPARE(negative.roundTrip, in); + } + + // With nothing stored and the device at 44.1 kHz, the round trip in + // frames is exactly the reported sum, as it was before there was a + // stored figure. At another rate each latency is converted from the + // frames it counts: the output's are the session's (bqaudioio's + // ResamplerWrapper converts them), the input's the device's + void frames_at_the_recording_rate() { + const double rate = 44100; + const std::vector latencies { + -100, -1, 0, 1, 255, 512, 4096, 8191, 8192, 8820, 12345, + 44100, 88200, 1000003 + }; + for (sv::sv_frame_t out : latencies) { + for (sv::sv_frame_t in : latencies) { + double seconds = + LatencyCalibration::reportedSeconds(out, rate) + + LatencyCalibration::reportedSeconds(in, rate); + QCOMPARE(LatencyCalibration::toFrames(seconds, rate), + computeRecordingLatency(out, in)); + } + } + + // 8192 frames out and 4096 in at 48 kHz; the play source has the + // output latency as round(8192 * 44100 / 48000) = 7526 frames + double seconds = LatencyCalibration::reportedSeconds(7526, rate) + + LatencyCalibration::reportedSeconds(4096, 48000); + QVERIFY(std::llabs(LatencyCalibration::toFrames(seconds, 48000) - + 12288) <= 1); + + QCOMPARE(LatencyCalibration::toFrames(0.0, rate), sv::sv_frame_t(0)); + QCOMPARE(LatencyCalibration::toFrames(-0.1, rate), sv::sv_frame_t(0)); + QCOMPARE(LatencyCalibration::toFrames(0.1, 0), sv::sv_frame_t(0)); + QCOMPARE(LatencyCalibration::reportedSeconds(4096, 0), 0.0); + } +}; + +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index b3f28c78..116509da 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -370,6 +370,22 @@ class TestRecordWorkflow : public QObject stopTake(); } + // A round trip as the audio check would have kept it for the fake + // device (the default devices, the Preferences naming none) at 44.1 + // kHz, measured while the device reported the given latencies, in + // frames. cleanup() forgets it + static void storeRoundTrip(int roundTrip, int reportedOutput, + int reportedInput) { + LatencyCalibration::Figure figure; + figure.roundTrip = roundTrip / rate; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = reportedOutput / rate; + figure.reportedInput = reportedInput / rate; + QSettings settings; + LatencyCalibration::store + (settings, LatencyCalibration::currentKey(settings, rate), figure); + } + static sv::EventVector pitchEvents(sv::Layer *layer) { if (!layer) return {}; auto model = sv::ModelById::getAs @@ -969,6 +985,10 @@ private slots: } void cleanup() { + // A round trip a test stored would place the next test's takes. + // First, as the waits below return early when they fail + QSettings().remove("LatencyCalibration"); + if (m_window) { if (m_window->recordTarget()->isRecording()) { m_window->doRecord(); @@ -1318,6 +1338,154 @@ private slots: QVERIFY2(std::llabs(error) <= 2 * hop, qPrintable(detail)); } + // latency_end_to_end with a device that reports 50 ms less than its + // round trip of K frames, and the round trip the audio check measured + // kept for it: the take is placed with the measured figure, and the + // two pitch tracks line up + void latency_measured_round_trip_used() { + const int K = 3 * 4096; + const int reportedOut = 2 * 4096; + const int reportedIn = 4096 - 2205; + FakeAudioIO::Config config; + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.input = melody(0.75); + config.inputDelay = K; + config.inputFollowsPlayback = true; + storeRoundTrip(K, reportedOut, reportedIn); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(melody(0.75))); + if (QTest::currentTestFailed()) return; + + take(2200); + if (QTest::currentTestFailed()) return; + + sv::sv_frame_t refStep = stepFrame(pitchEvents(m_window->analyser())); + sv::sv_frame_t sungStep = stepFrame(pitchEvents(m_window->analyser2())); + QVERIFY(refStep > 0); + QVERIFY2(sungStep > 0, "the take never reached the second note"); + sv::sv_frame_t error = sungStep - refStep; + QVERIFY2(std::llabs(error) <= 2 * hop, + qPrintable(QString("sung step at %1, reference step at %2: " + "%3 frames (%4 ms) apart; the take was " + "placed with a round trip of %5 frames") + .arg(sungStep).arg(refStep).arg(error) + .arg(1000.0 * double(error) / rate, 0, 'f', 1) + .arg(m_window->takeLatency().roundTrip))); + + TakeLatency used = m_window->takeLatency(); + QVERIFY(used.measured); + QCOMPARE(used.roundTrip, sv::sv_frame_t(K)); + QCOMPARE(used.reportedOutput, sv::sv_frame_t(reportedOut)); + QCOMPARE(used.reportedInput, sv::sv_frame_t(reportedIn)); + } + + // A round trip measured while the device reported other latencies + // (its buffers have been changed since) is stale: the take is placed + // with the reported pair. With the latencies it was measured with, + // the same figure would be in use + void latency_stale_round_trip_ignored() { + const int reportedOut = 2 * 4096; + const int reportedIn = 4096; + FakeAudioIO::Config config; + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.input = tone(highHz, 2.0); + const int stale = reportedOut + + int(2 * LatencyCalibration::kStaleToleranceSeconds * rate); + storeRoundTrip(15000, stale, reportedIn); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + + TakeLatency used = m_window->takeLatency(); + QVERIFY(!used.measured); + QCOMPARE(used.roundTrip, sv::sv_frame_t(reportedOut + reportedIn)); + + LatencyCalibration::InUse inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Reported); + QVERIFY(inUse.stale); + QCOMPARE(inUse.roundTrip, (reportedOut + reportedIn) / rate); + + storeRoundTrip(15000, reportedOut, reportedIn); + inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Measured); + QCOMPARE(inUse.roundTrip, 15000 / rate); + } + + // A device at 48 kHz reporting 2 x 4096 frames out and 4096 in: the + // take is placed with those 3 x 4096 frames of the recording, + // although the play source has the output latency in frames of the + // session, converted as it resamples + void latency_reported_at_the_device_rate() { + const double deviceRate = 48000.0; + const int reportedOut = 2 * 4096; + const int reportedIn = 4096; + FakeAudioIO::Config config; + config.sampleRate = int(deviceRate); + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.input = tone(highHz, 2.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(true); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + take(800); + if (QTest::currentTestFailed()) return; + + TakeLatency used = m_window->takeLatency(); + QCOMPARE(used.recordingRate, deviceRate); + QVERIFY(!used.measured); + QCOMPARE(used.reportedOutput, + sv::sv_frame_t(std::lround(reportedOut * rate / deviceRate))); + QCOMPARE(used.reportedInput, sv::sv_frame_t(reportedIn)); + QVERIFY2(std::llabs(used.roundTrip - (reportedOut + reportedIn)) <= 2, + qPrintable(QString("placed with %1 frames at %2 Hz; the " + "device reports %3 + %4") + .arg(used.roundTrip).arg(used.recordingRate) + .arg(reportedOut).arg(reportedIn))); + } + + // The same device, chosen before any file is open, as from the audio + // device menu at startup. The play source has no rate yet, so its + // resampler passes the output latency on as the device counts it, + // and tells the play source the device's rate is 0: the round trip + // is the same + void latency_reported_with_device_opened_first() { + const double deviceRate = 48000.0; + const int reportedOut = 2 * 4096; + const int reportedIn = 4096; + FakeAudioIO::Config config; + config.sampleRate = int(deviceRate); + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.input = tone(highHz, 2.0); + makeWindow(config); + m_window->recreateAudioIO(); + QVERIFY(m_window->fake()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + m_window->setPlayReferenceWhileRecording(true); + + take(800); + if (QTest::currentTestFailed()) return; + + TakeLatency used = m_window->takeLatency(); + QCOMPARE(used.recordingRate, deviceRate); + QCOMPARE(used.reportedOutput, sv::sv_frame_t(reportedOut)); + QVERIFY2(std::llabs(used.roundTrip - (reportedOut + reportedIn)) <= 2, + qPrintable(QString("placed with %1 frames at %2 Hz; the " + "device reports %3 + %4") + .arg(used.roundTrip).arg(used.recordingRate) + .arg(reportedOut).arg(reportedIn))); + } + void latency_zero_when_toggle_off() { FakeAudioIO::Config config; config.playbackLatency = 4096; diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 850a47b1..da882fdb 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -21,6 +21,7 @@ #include "TestTakesFile.h" #include "TestTakeTiming.h" #include "TestLatencyCheck.h" +#include "TestLatencyCalibration.h" #include "RunSuite.h" @@ -105,6 +106,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLatencyCalibration t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index acbf5ba4..c3729571 100644 --- a/meson.build +++ b/meson.build @@ -1095,6 +1095,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/Coverage.cpp', + 'main/LatencyCalibration.cpp', 'main/LatencyCheck.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', @@ -1360,6 +1361,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', 'main/test/TestLatencyCheck.h', + 'main/test/TestLatencyCalibration.h', ]) tony_core_test_exe = executable( From 2424bd698e4d66bf75567ad221ef6fa9e44335ff Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:25:57 +0000 Subject: [PATCH 107/275] docs: calibrate audio work orders, B2 done Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 91cad742..16b38835 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (see git log). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) From 2fe110052e2b5900a19c902333cb8359b22ca623 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:34:42 +0000 Subject: [PATCH 108/275] test: lyrics laid out again after an edit, and the box row found Picks up the svgui change that lays the lyrics out again and finds the word being sung again whenever the model changes, and that says where the row of boxes is, for editing the words with the mouse. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/test/TestLyricsLayer.h | 64 +++++++++++++++++++++++++++++++++++++ repoint-lock.json | 2 +- 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/main/test/TestLyricsLayer.h b/main/test/TestLyricsLayer.h index 4cac8aed..8ca3a53f 100644 --- a/main/test/TestLyricsLayer.h +++ b/main/test/TestLyricsLayer.h @@ -518,6 +518,70 @@ private slots: "the label is not drawn centred over its box, past both ends"); } + void an_edited_word_is_laid_out_again() { + // The same number of words, over the same stretch of time: all + // that a stale layout would have noticed + addSpacedWords(3); + render({ QRect(0, 0, kWidth, kHeight) }); + + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + QSignalSpy repaints(m_layer, &sv::Layer::layerParametersChanged); + model->remove(spacedWord(1)); + model->add(spacedWord(1).withLabel("sanaseppo")); + QVERIFY2(repaints.count() > 0, "the edit did not ask for a repaint"); + + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + int x0 = m_pane->getXForFrame(spacedWord(1).getFrame()); + int x1 = m_pane->getXForFrame(spacedWord(1).getFrame() + + spacedWord(1).getDuration()); + int dark = 0; + for (int x = x0; x < x1; ++x) { + for (int y = kHeight / 2; y < kHeight; ++y) { + if (qGray(image.pixel(x, y)) < 100) ++dark; + } + } + QVERIFY2(dark > 20, "the edited word's label is not drawn"); + } + + void the_highlight_follows_an_edit_of_the_word_being_sung() { + addSpacedWords(3); + auto model = sv::ModelById::getAs(m_layer->getModel()); + QVERIFY(model); + m_layer->setHighlightFrame(sv::sv_frame_t(kRate * 2.1)); + + sv::Event renamed = spacedWord(1).withLabel("sana"); + model->remove(spacedWord(1)); + model->add(renamed); + sv::Event e(0); + QVERIFY(m_layer->getHighlightedEvent(e)); + QCOMPARE(e.getLabel(), QString("sana")); + + // Shortened to end before the frame: no word is being sung + model->remove(renamed); + model->add(renamed.withDuration(sv::sv_frame_t(kRate * 0.05))); + QVERIFY(!m_layer->getHighlightedEvent(e)); + } + + void the_box_row_is_where_the_boxes_are_painted() { + QVERIFY(m_layer->getLyricsBoxRow(m_pane).isEmpty()); + addSpacedWords(3); + QImage image = render({ QRect(0, 0, kWidth, kHeight) }); + QRect row = m_layer->getLyricsBoxRow(m_pane); + QVERIFY(!row.isEmpty()); + + int x = m_pane->getXForFrame(spacedWord(1).getFrame()) + 8; + QCOMPARE(topDrawnRow(image, x), row.top()); + int bottom = row.top(); + while (bottom + 1 < kHeight && + image.pixel(x, bottom + 1) != qRgb(255, 255, 255)) { + ++bottom; + } + QCOMPARE(bottom, row.bottom()); + QCOMPARE(row.left(), 0); + QCOMPARE(row.width(), m_pane->getPaintWidth()); + } + void painting_in_strips_matches_painting_whole() { // A view that scrolls repaints only the strip that comes into // sight; the labels in it must be where they were when the diff --git a/repoint-lock.json b/repoint-lock.json index 1aace33f..58816ba8 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "11231118534c1099b11554d2414459d51470e0dc" + "pin": "6084cd2ad746971ec068cbcd00d5323728ac0ab4" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 2e003f7339be2053f450b7ade40a842729dc9125 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:55:19 +0000 Subject: [PATCH 109/275] build: package tony for android as a debug apk build-apk.sh stages the application and the plugins (as libpyin.so and libchp.so), strips them, writes the deployment settings, runs androiddeployqt with gradle and legacy packaging, and checks the apk. the manifest is qt's template with the package id, record_audio and landscape. not yet run to the end: gradle needs google's maven on dl.google.com, which the container could not reach. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/android/build-apk.sh | 294 +++++++++++++++++++++ deploy/android/package/AndroidManifest.xml | 64 +++++ docs/android-work-orders.md | 33 ++- docs/port-android.md | 3 + 4 files changed, 392 insertions(+), 2 deletions(-) create mode 100755 deploy/android/build-apk.sh create mode 100644 deploy/android/package/AndroidManifest.xml diff --git a/deploy/android/build-apk.sh b/deploy/android/build-apk.sh new file mode 100755 index 00000000..2a5fa4fd --- /dev/null +++ b/deploy/android/build-apk.sh @@ -0,0 +1,294 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Packages what deploy/android/build-tony.sh built into an APK signed +# with the debug key, which can be installed on a phone by hand: +# +# build-android/apk/Tony-debug.apk +# +# androiddeployqt reads what to package from a deployment settings file, +# which Qt's CMake support writes for a CMake build; this writes it for +# the meson one (build-android/android-Tony-deployment-settings.json). +# androiddeployqt then adds Qt's libraries and plugins and the NDK's C++ +# library to build-android/android-build/, a Gradle project, and runs +# Gradle, which strips the libraries (it knows the NDK's version) and +# signs the APK with ~/.android/debug.keystore, made on its first run. +# +# The manifest is deploy/android/package/AndroidManifest.xml; the icon is +# added from icons/. +# +# The Vamp plugins go in as libpyin.so and libchp.so, beside the +# application's library in lib/arm64-v8a/: Android installs nothing else +# from an APK. They are put there directly rather than named as +# android-extra-libs, which Qt's launcher would load at every start. The +# libraries are packaged the legacy way, compressed and unpacked when the +# app is installed, so that they exist as files for svcore's scan to open; +# main.cpp links the plugins under their own names (see AndroidFiles). +# +# Gradle fetches the Android Gradle plugin and its dependencies from +# Google's repository and Maven Central on its first run, and Maven +# Central has answered 429 Too Many Requests: a Gradle run that fails for +# want of the network is tried again, three times at most. The log is +# /opt/android/logs/tony-apk.log. It ends by checking the APK. +# +# Usage, from anywhere: +# deploy/android/build-apk.sh + +set -eu -o pipefail + +if [ "$#" -ne 0 ]; then + echo "Usage: $0" 1>&2 + exit 2 +fi + +android=/opt/android +sdk=$android/sdk +ndk=$sdk/ndk/27.2.12479018 +build_tools=$sdk/build-tools/36.0.0 +platform=android-36 +qt=$android/qt/6.11.2 +qt_android=$qt/android_arm64_v8a +qt_host=$qt/gcc_64 +logs=$android/logs +abi=arm64-v8a + +repo=$(cd "$(dirname "$0")/../.." && pwd) +build=$repo/build-android +out=$build/android-build +package=$build/android-package +settings=$build/android-Tony-deployment-settings.json +apk_dir=$build/apk +apk=$apk_dir/Tony-debug.apk + +llvm=$ndk/toolchains/llvm/prebuilt/linux-x86_64/bin + +export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-$(dpkg --print-architecture) + +# Qt's tools warn at every start in the container's C locale +export LC_ALL=C.UTF-8 + +if [ ! -x "$JAVA_HOME/bin/java" ] || [ ! -d "$build_tools" ] || + [ ! -d "$sdk/platforms/$platform" ] || [ ! -f "$ndk/source.properties" ]; then + echo "ERROR: no JDK 21, SDK or NDK: run deploy/android/setup-toolchain.sh first" 1>&2 + exit 1 +fi +if [ ! -x "$qt_host/bin/androiddeployqt" ] || [ ! -d "$qt_android/src/android/templates" ]; then + echo "ERROR: no Qt for Android at $qt: run deploy/android/build-qt.sh first" 1>&2 + exit 1 +fi +for f in libTony_arm64-v8a.so pyin.so chp.so version.h; do + if [ ! -f "$build/$f" ]; then + echo "ERROR: no build-android/$f: run deploy/android/build-tony.sh first" 1>&2 + exit 1 + fi +done + +mkdir -p "$logs" + +# 1. Gather the libraries, the manifest and the icon + +echo "Gathering the libraries" + +# Emptied first, so that nothing from an earlier run is packaged by +# mistake; androiddeployqt puts Qt's libraries back +rm -rf "$out/libs/$abi" +mkdir -p "$out/libs/$abi" +cp "$build/libTony_arm64-v8a.so" "$out/libs/$abi/" +cp "$build/pyin.so" "$out/libs/$abi/libpyin.so" +cp "$build/chp.so" "$out/libs/$abi/libchp.so" + +# Stripped here as Gradle strips them, so that the APK is small whatever +# Gradle does (Qt's libraries come without debug information). Those in +# build-android/ keep theirs, for symbolising a crash with the NDK's +# ndk-stack +for so in libTony_arm64-v8a.so libpyin.so libchp.so; do + "$llvm/llvm-strip" --strip-unneeded "$out/libs/$abi/$so" +done + +rm -rf "$package" +cp -r "$repo/deploy/android/package" "$package" +# 128 pixels is about the 48 dp of a launcher icon at xxhdpi (3x) +mkdir -p "$package/res/drawable-xxhdpi" +cp "$repo/icons/tony-128x128.png" "$package/res/drawable-xxhdpi/icon.png" + +# 2. The deployment settings + +# The version name says which commit this is, with a + if the working +# tree had changes; the version code grows with each commit, so that a +# newer build installs over an older one +version=$(sed -n 's/^#define TONY_VERSION "\(.*\)"$/\1/p' "$build/version.h") +commit=$(git -C "$repo" rev-parse --short HEAD) +if ! git -C "$repo" diff --quiet HEAD --; then + commit="$commit+" +fi +version_name="$version ($commit)" +version_code=$(git -C "$repo" rev-list --count HEAD) + +# The Qt plugins are those Qt's CMake support chose for an application +# using the same Qt modules (the model in $logs): without the list, +# androiddeployqt takes every plugin of each module, test platforms and +# all +plugins= +for p in platforms/libplugins_platforms_qtforandroid \ + styles/libplugins_styles_qandroidstyle \ + imageformats/libplugins_imageformats_qgif \ + imageformats/libplugins_imageformats_qico \ + imageformats/libplugins_imageformats_qjpeg \ + imageformats/libplugins_imageformats_qsvg \ + iconengines/libplugins_iconengines_qsvgicon \ + networkinformation/libplugins_networkinformation_qandroidnetworkinformation \ + tls/libplugins_tls_qcertonlybackend; do + f=$qt_android/plugins/${p}_$abi.so + if [ ! -f "$f" ]; then + echo "ERROR: no Qt plugin $f" 1>&2 + exit 1 + fi + plugins="$plugins${plugins:+;}$f" +done + +cat > "$settings" <> "$log" + if "$qt_host/bin/androiddeployqt" \ + --input "$settings" --output "$out" \ + --android-platform "$platform" --jdk "$JAVA_HOME" \ + --apk "$apk" --verbose >> "$log" 2>&1; then + break + fi + this_attempt=$(sed -n "/^=== attempt $attempt\$/,\$p" "$log") + if [ "$attempt" -lt 3 ] && + echo "$this_attempt" | + grep -qE "status code 429|Too Many Requests|Connection reset|timed out"; then + echo " Gradle was turned away for now; trying again in $attempt minute(s)" + sleep $((60 * attempt)) + attempt=$((attempt + 1)) + continue + fi + echo "$this_attempt" | grep -E "error|FAILED|What went wrong|Could not GET" | + sort -u | tail -20 1>&2 || true + refused=$(echo "$this_attempt" | grep "status code 403" | + grep -o "https://[^/']*" | sort -u | tr '\n' ' ' || true) + if [ -n "$refused" ]; then + echo "ERROR: refused with 403 (is the host allowed on this network?): $refused" 1>&2 + fi + echo "ERROR: packaging failed; the whole log is $log" 1>&2 + exit 1 +done + +# 4. Check + +echo +echo "Checking $apk" + +listing=$(unzip -Z1 "$apk") +for f in lib/$abi/libTony_arm64-v8a.so lib/$abi/libpyin.so lib/$abi/libchp.so \ + lib/$abi/libc++_shared.so lib/$abi/libQt6Core_arm64-v8a.so \ + lib/$abi/libplugins_platforms_qtforandroid_arm64-v8a.so; do + if ! echo "$listing" | grep -qxF "$f"; then + echo "ERROR: the APK has no $f" 1>&2 + exit 1 + fi +done +echo " it holds $(echo "$listing" | grep -c "^lib/$abi/.*\.so$") libraries in lib/$abi/" + +# Stripped, and each loading nothing that is neither in the APK nor one +# of Android's own libraries +unpacked=$(mktemp -d) +trap 'rm -rf "$unpacked"' EXIT +unzip -q "$apk" "lib/$abi/*" -d "$unpacked" +system_libraries='lib(c|m|dl|log|z|android|jnigraphics|EGL|GLESv2|GLESv3|vulkan|mediandk)\.so' +for so in "$unpacked/lib/$abi/"*.so; do + name=$(basename "$so") + if "$llvm/llvm-readelf" -SW "$so" | grep -q "\.debug_info"; then + echo "ERROR: $name in the APK is not stripped" 1>&2 + exit 1 + fi + for needed in $("$llvm/llvm-readelf" -d "$so" | sed -n 's/.*(NEEDED).*\[\(.*\)\]/\1/p'); do + if [ ! -f "$unpacked/lib/$abi/$needed" ] && ! echo "$needed" | grep -qxE "$system_libraries"; then + echo "ERROR: $name loads $needed, which is neither in the APK nor Android's" 1>&2 + exit 1 + fi + done +done +echo " they are stripped, and load nothing but each other and Android's libraries" + +"$build_tools/apksigner" verify "$apk" +echo " apksigner: verified" +"$build_tools/zipalign" -c -P 16 4 "$apk" +echo " zipalign: aligned for 16 KB pages" + +badging=$("$build_tools/aapt2" dump badging "$apk") +echo "$badging" | grep -E "^package:|^application-label:|^sdkVersion:|^targetSdkVersion:|^uses-permission:" | sed 's/^/ /' +if ! echo "$badging" | grep -q "name='android.permission.RECORD_AUDIO'"; then + echo "ERROR: the APK does not ask for RECORD_AUDIO" 1>&2 + exit 1 +fi +manifest=$("$build_tools/aapt2" dump xmltree --file AndroidManifest.xml "$apk") +if ! echo "$manifest" | grep -q 'screenOrientation.*=6'; then + echo "ERROR: the activity is not sensorLandscape" 1>&2 + exit 1 +fi +if echo "$manifest" | grep -q 'extractNativeLibs.*=false'; then + echo "ERROR: the libraries would not be unpacked on install" 1>&2 + exit 1 +fi +echo " landscape, and the libraries are unpacked on install" + +cat < + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 4ec6d3ce..4b3d3806 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -78,7 +78,9 @@ first. end, and again only if something failed. The app suite runs in real time (minutes): give it a 10-minute tool timeout. - Network: GitHub (git and release downloads), the Ubuntu archive, PyPI, conda-forge, - `download.qt.io` and `dl.google.com` are reachable; but `download.qt.io` answers every + `download.qt.io` and `dl.google.com` are reachable (but on 2026-09-26 the egress + gateway refused every CONNECT to `dl.google.com` for A3b's whole session; Gradle needs + it for the Android Gradle plugin); but `download.qt.io` answers every Qt binary archive with a redirect to a mirror, and all mirrors are blocked (A2 builds Qt from source). `breakfastquay.com` and `ppa.launchpadcontent.net` are blocked by the environment's policy, and `hg.sr.ht` @@ -151,7 +153,9 @@ builds happen in the container.) - A1 — Sample rate: a device that is not at 44.1 kHz. Done. - A2 — Android toolchain and C libraries. Done. - A3a — Tony builds for Android. Done. -- A3b — Tony as an APK (no audio): the test port. +- A3b — Tony as an APK (no audio): the test port. Built but for the APK itself: Gradle + was blocked (`dl.google.com` refused); run `deploy/android/build-apk.sh` once it is + allowed (see the log). - A4 — Touch gestures on the panes. - A5 — Compact touch mode. - A6 — Oboe audio backend. @@ -412,3 +416,28 @@ Xml, Network, Svg, and libc, libm, libdl, libc++_shared (zlib is linked in); it androiddeployqt 6.11 has no strip option: check that the APK's copies are stripped (Qt's Gradle template sets `ndkVersion`, so the Android Gradle plugin should strip them). Left open: Qt's `Test` module stays in the dependency for Android; `--as-needed` drops it. + +### Phase A3b — 2026-09-26 +Built: `deploy/android/build-apk.sh` (after build-tony.sh; writes +`build-android/android-Tony-deployment-settings.json`, runs androiddeployqt and Gradle, +checks the APK: `build-android/apk/Tony-debug.apk`, log `/opt/android/logs/tony-apk.log`); +`deploy/android/package/AndroidManifest.xml` (`io.github.jhhr.tony`, RECORD_AUDIO, +sensorLandscape). `main/AndroidFiles` (tony_core; `TestAndroidFiles`). main.cpp on Android: +stdout/stderr to logcat, `AUDIO_NONE`, the plugin links, a box if they fail. `MainWindow:: +getOpenFileName()` on Android copies a `content://` pick to `/imported/`. +Choices / deviations: +- Plugins: Android installs only `lib*.so`, svcore names a plugin after its file, and the + native library folder holds all of Qt: so the APK has `libpyin.so`, `libchp.so` (in + `libs/` directly: `android-extra-libs` would load them at every start), legacy + packaging, and VAMP_PATH is `/vamp`, links `pyin.so`, `chp.so` to them, remade + at each start. `applicationDirPath()` is that folder (argv[0] is the library's path). +- Our three libraries are stripped by the script; the Qt plugins are CMake's list (A2). +- The menu bar stays in the window: a native one needs an action bar, taller than it. +- A `.ton` from the picker is refused with a message (it needs its folder). +Blocked: the egress gateway refused every CONNECT to `dl.google.com` (Google's Maven: the +Android Gradle plugin) all session, so there is no APK yet. All up to Gradle was run and +checked (libraries 16 KB aligned, their NEEDED all packaged or Android's; manifest). +The next phase must know: logcat tag `Tony` has Tony's and svcore's cerr; SVDEBUG goes only +to `files/log/sv-debug.log` (`adb shell run-as io.github.jhhr.tony cat ...`). Phone test: +install; File > Open, pick audio: waveform, then pitch and notes; Play is off (no audio). +Left open: saving and sessions through the picker (A7); `imported/` is never emptied. diff --git a/docs/port-android.md b/docs/port-android.md index df246500..a8211ed5 100644 --- a/docs/port-android.md +++ b/docs/port-android.md @@ -175,6 +175,9 @@ common to both platforms. *(snippet)* marks a fact seen only in search results. can be empty, so svcore's scan of `VAMP_PATH` finds nothing. Two fixes: - Turn on legacy packaging (`android-legacy-packaging` for androiddeployqt, or `useLegacyPackaging` in Qt's gradle template) and name the plugin `libpyin.so`. + *(A3b: not enough by itself. svcore names a plugin after its file, + `vamp:libpyin:...`, and Tony asks for `vamp:pyin:...`; A3b links `pyin.so` to it + in app storage.)* - Link pYIN into the app and have svcore load it without a scan (a small svcore fork change). - Precedents link Vamp plugin code straight in rather than loading it, for example From 2abe36b4d240a6374bf193356db396501f2571f5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:55:19 +0000 Subject: [PATCH 110/275] feat: find pyin and open picked files on android vamp_path points at a folder of links named pyin.so and chp.so to the installed libpyin.so and libchp.so, so that the plugin ids stay vamp:pyin; a content:// pick is copied into the app's storage and the copy opened; stdout and stderr go to logcat; no audio yet. desktop behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/AndroidFiles.cpp | 188 ++++++++++++++++++++++++ main/AndroidFiles.h | 77 ++++++++++ main/MainWindow.cpp | 47 ++++++ main/MainWindow.h | 7 + main/main.cpp | 99 ++++++++++++- main/test/TestAndroidFiles.h | 275 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 7 + 8 files changed, 705 insertions(+), 2 deletions(-) create mode 100644 main/AndroidFiles.cpp create mode 100644 main/AndroidFiles.h create mode 100644 main/test/TestAndroidFiles.h diff --git a/main/AndroidFiles.cpp b/main/AndroidFiles.cpp new file mode 100644 index 00000000..9a0be724 --- /dev/null +++ b/main/AndroidFiles.cpp @@ -0,0 +1,188 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AndroidFiles.h" + +#include "base/Debug.h" +#include "system/System.h" + +#include +#include +#include +#include + +#include + +using namespace sv; + +QStringList +AndroidFiles::linkVampPlugins(QString libraryDir, + QString linkDir, + QStringList names, + QStringList &problems) +{ + QStringList links; + + if (!QDir().mkpath(linkDir)) { + problems << QString("Cannot create the plugin folder %1").arg(linkDir); + SVCERR << "AndroidFiles: " << problems.back() << endl; + return links; + } + + for (QString name : names) { + + // Absolute: a relative link would be read from linkDir + QString library = QFileInfo(QDir(libraryDir).filePath + ("lib" + name + ".so")).absoluteFilePath(); + QString link = QDir(linkDir).filePath(name + ".so"); + + // Whatever is there is from an earlier start, and may point into + // a library directory that an update has since removed + QFile::remove(link); + + if (!QFileInfo::exists(library)) { + problems << QString("No %1 plugin: there is no %2") + .arg(name).arg(library); + SVCERR << "AndroidFiles: " << problems.back() << endl; + continue; + } + + QFile file(library); + if (file.link(link)) { + SVCERR << "AndroidFiles: linked " << link << " to " << library + << endl; + } else { + QString linkError = file.errorString(); + if (file.copy(link)) { + SVCERR << "AndroidFiles: could not link " << link << " to " + << library << " (" << linkError + << "), so copied it" << endl; + } else { + problems << QString("Cannot link or copy %1 to %2: %3") + .arg(library).arg(link).arg(file.errorString()); + SVCERR << "AndroidFiles: " << problems.back() << endl; + continue; + } + } + + links << link; + } + + return links; +} + +QString +AndroidFiles::checkVampPlugin(QString path) +{ + QString problem; + + void *handle = DLOPEN(path, RTLD_LAZY | RTLD_LOCAL); + if (!handle) { + const char *error = DLERROR(); + problem = QString("Cannot load %1: %2") + .arg(path) + .arg(QString::fromLocal8Bit(error ? error : "no reason given")); + } else { + if (!DLSYM(handle, "vampGetPluginDescriptor")) { + problem = QString("%1 is not a Vamp plugin library").arg(path); + } + (void)DLCLOSE(handle); + } + + if (problem != "") { + SVCERR << "AndroidFiles: " << problem << endl; + } else { + SVCERR << "AndroidFiles: " << path << " loads" << endl; + } + return problem; +} + +QString +AndroidFiles::copyIn(QString source, QString name, QString dir, + QString &error) +{ + if (!QDir().mkpath(dir)) { + error = QString("cannot create the folder %1").arg(dir); + return ""; + } + + QString target = QDir(dir).filePath(safeFileName(name)); + + QFile in(source); + if (!in.open(QIODevice::ReadOnly)) { + error = QString("cannot read it: %1").arg(in.errorString()); + return ""; + } + + // QSaveFile writes a file of its own and renames it over the target + // at the end, so a copy that fails half-way leaves the earlier one + QSaveFile out(target); + if (!out.open(QIODevice::WriteOnly)) { + error = QString("cannot write %1: %2") + .arg(target).arg(out.errorString()); + return ""; + } + + // Read to the end rather than to the size: a cloud provider may hand + // over a pipe, which has none + std::vector buffer(1 << 20); + qint64 total = 0; + while (true) { + qint64 n = in.read(buffer.data(), qint64(buffer.size())); + if (n < 0) { + error = QString("reading it failed: %1").arg(in.errorString()); + out.cancelWriting(); + return ""; + } + if (n == 0) break; + if (out.write(buffer.data(), n) != n) { + error = QString("writing %1 failed: %2") + .arg(target).arg(out.errorString()); + out.cancelWriting(); + return ""; + } + total += n; + } + + if (!out.commit()) { + error = QString("writing %1 failed: %2") + .arg(target).arg(out.errorString()); + return ""; + } + + SVCERR << "AndroidFiles: copied " << total << " bytes from " << source + << " to " << target << endl; + return target; +} + +QString +AndroidFiles::safeFileName(QString name) +{ + const QString refused("/\\:*?\"<>|"); + + QString safe; + for (QChar c : name) { + if (c.unicode() < 32 || refused.contains(c)) { + safe += QChar('_'); + } else { + safe += c; + } + } + + safe = safe.trimmed(); + if (safe.count(QChar('.')) == safe.size()) { + safe = "imported"; + } + return safe; +} diff --git a/main/AndroidFiles.h b/main/AndroidFiles.h new file mode 100644 index 00000000..4e52b30d --- /dev/null +++ b/main/AndroidFiles.h @@ -0,0 +1,77 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_ANDROID_FILES_H +#define TONY_ANDROID_FILES_H + +#include +#include + +/** + * File work that only the Android build calls: Android hands Tony + * neither its Vamp plugins under their own names nor the files the user + * picks as paths. Plain file operations, so they are built and tested + * on the desktop as well. + */ +class AndroidFiles +{ +public: + /** + * Android installs only libraries named lib*.so with an application, + * so pYIN and CHP go into the APK as libpyin.so and libchp.so, beside + * the application's own library in libraryDir. But svcore names a + * plugin after its library's file name (vamp:pyin:pyin:...), and its + * scan of VAMP_PATH opens every .so in a directory, of which the + * application's has dozens. So this makes linkDir hold only the + * plugins, under their own names: a link .so to + * /lib.so for each of names, or a copy where a link + * cannot be made. A link left from an earlier start is replaced + * first, as the library directory moves with every install. + * + * Returns the paths of the links made. Each plugin that could not be + * set up adds a line to problems, saying why. + */ + static QStringList linkVampPlugins(QString libraryDir, + QString linkDir, + QStringList names, + QStringList &problems); + + /** + * Opens the plugin library at path as svcore's scan does and looks + * for the Vamp entry point. Returns "" if it is there, else why not: + * a library that fails here is one the scan will pass over, telling + * only its own log file. + */ + static QString checkVampPlugin(QString path); + + /** + * Copies the file at source into dir as name, the name the user knows + * it by, and returns the copy's path; on Android source is the + * content:// URI the system file picker gives, which Tony's readers + * cannot open, and Qt's QFile can. An earlier copy of the same name + * is replaced, and kept if the copy fails. Returns "" on failure, + * with error saying why. + */ + static QString copyIn(QString source, QString name, QString dir, + QString &error); + + /** + * name as a file name that can be used in any directory: path + * separators and the characters Windows refuses become '_', and a + * name that is empty or only dots becomes "imported". + */ + static QString safeFileName(QString name); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4edab375..203d33c3 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -24,6 +24,11 @@ #include "TakeLayers.h" #include "TakesFile.h" +#ifdef Q_OS_ANDROID +#include "AndroidFiles.h" +#include +#endif + #include "framework/Document.h" #include "framework/VersionTester.h" @@ -508,6 +513,11 @@ MainWindow::setupMenus() // workaround, to remove the appmenu-qt5 package, but that is // awkward and the problem is so severe that it merits disabling // the system menubar integration altogether. Like this: + // + // Android defines Q_OS_LINUX as well, and there too the bar stays + // in the window: a native one would be an options menu in an + // action bar that Android adds above the window, taller than the + // bar it replaces. menuBar()->setNativeMenuBar(false); #endif @@ -2641,6 +2651,43 @@ MainWindow::closeSession() documentRestored(); } +#ifdef Q_OS_ANDROID +QString +MainWindow::getOpenFileName(FileFinder::FileType type) +{ + QString path = MainWindowBase::getOpenFileName(type); + if (!path.startsWith("content:")) return path; + + // The name the user knows the file by, which Qt asks the file's + // provider for: the URI need not contain it + QString name = QFileInfo(path).fileName(); + + // A session finds its audio and takes beside it, and the picker + // grants access to the one file picked and nothing next to it + if (QFileInfo(name).suffix().toLower() == "ton") { + QMessageBox::warning + (this, tr("Cannot open a session here"), + tr("Sessions cannot be opened from the file picker yet

A session needs its audio and its takes folder beside it, and the picker lets %1 read the one file only. Open an audio file instead.") + .arg(QApplication::applicationName())); + return ""; + } + + QString dir = QStandardPaths::writableLocation + (QStandardPaths::AppDataLocation) + "/imported"; + QString error; + QString copy = AndroidFiles::copyIn(path, name, dir, error); + if (copy == "") { + QMessageBox::critical + (this, tr("Failed to open file"), + tr("File open failed

\"%1\" could not be copied into %2's own storage: %3") + .arg(name.toHtmlEscaped()) + .arg(QApplication::applicationName()) + .arg(error.toHtmlEscaped())); + } + return copy; +} +#endif + void MainWindow::openFile() { diff --git a/main/MainWindow.h b/main/MainWindow.h index 33aff270..dd6b8ee8 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -881,6 +881,13 @@ protected slots: bool checkSaveModified(); bool waitForInitialAnalysis(); +#ifdef Q_OS_ANDROID + // Android's file picker gives content:// URIs, which svcore's readers + // cannot open: the file picked is copied into the app's own storage, + // and the copy's path returned + QString getOpenFileName(sv::FileFinder::FileType type) override; +#endif + // A session must not be saved in the middle of the analysis of a // recorded range: the take's pitch and notes still hold the state // before the merge, and the two models the run works in are in the diff --git a/main/main.cpp b/main/main.cpp index ac90b39b..b0d62d33 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -42,6 +42,16 @@ #include #include +#ifdef Q_OS_ANDROID +#include "AndroidFiles.h" +#include +#include +#include +#include +#include +#include +#endif + #include "../version.h" #include @@ -131,6 +141,44 @@ class TonyApplication : public QApplication } }; +#ifdef Q_OS_ANDROID +// An Android app's stdout and stderr lead nowhere, and Tony and svcore +// report through cerr. Send both into a pipe, and have a thread pass what +// comes out of it on to the system log (logcat), a line at a time, under +// the tag "Tony". Qt's own messages go to the system log already. +static void +sendOutputToSystemLog() +{ + int fds[2]; + if (pipe(fds) != 0) return; + + setvbuf(stdout, nullptr, _IOLBF, 0); + setvbuf(stderr, nullptr, _IONBF, 0); + dup2(fds[1], STDOUT_FILENO); + dup2(fds[1], STDERR_FILENO); + close(fds[1]); + + int readEnd = fds[0]; + std::thread([readEnd]() { + char buffer[1024]; + std::string line; + while (true) { + ssize_t n = read(readEnd, buffer, sizeof(buffer)); + if (n < 0 && errno == EINTR) continue; + if (n <= 0) break; + for (ssize_t i = 0; i < n; ++i) { + // The system log cuts longer lines short + if (buffer[i] == '\n' || line.size() >= 1000) { + __android_log_write(ANDROID_LOG_INFO, "Tony", line.c_str()); + line.clear(); + } + if (buffer[i] != '\n') line += buffer[i]; + } + } + }).detach(); +} +#endif + static QString getEnvQStr(QString variable) { @@ -156,9 +204,13 @@ putEnvQStr(QString assignment) #endif } -static void +// Returns what went wrong in setting up the plugins, for the user to be +// told; only on Android can anything go wrong here +static QStringList setupTonyVampPath() { + QStringList problems; + QString myVampPath = getEnvQStr("TONY_VAMP_PATH"); #ifdef Q_OS_WIN32 @@ -183,6 +235,21 @@ setupTonyVampPath() #ifdef Q_OS_MAC myVampPath = myDir + "/../Resources"; (void)sep; // unused +#elif defined(Q_OS_ANDROID) + // myDir is the folder Android installed the application's own + // library in, and the plugins beside it as libpyin.so and + // libchp.so. Vamp looks in a folder of links to those two + // under their own names instead (see AndroidFiles) + QString linkDir = QStandardPaths::writableLocation + (QStandardPaths::AppDataLocation) + "/vamp"; + QStringList links = AndroidFiles::linkVampPlugins + (myDir, linkDir, { "pyin", "chp" }, problems); + for (QString link : links) { + QString problem = AndroidFiles::checkVampPlugin(link); + if (problem != "") problems << problem; + } + myVampPath = linkDir; + (void)sep; // unused #else if (binaryName != "") { myVampPath = @@ -202,11 +269,17 @@ setupTonyVampPath() // Windows lacks setenv, must use putenv (different arg convention) putEnvQStr(env); + + return problems; } int main(int argc, char **argv) { +#ifdef Q_OS_ANDROID + sendOutputToSystemLog(); +#endif + if (argc == 2 && (QString(argv[1]) == "--version" || QString(argv[1]) == "-v")) { cerr << TONY_VERSION << endl; @@ -229,7 +302,7 @@ main(int argc, char **argv) QApplication::setOrganizationDomain("sonicvisualiser.org"); QApplication::setApplicationName("Tony"); - setupTonyVampPath(); + QStringList pluginProblems = setupTonyVampPath(); QStringList args = application.arguments(); @@ -253,6 +326,11 @@ main(int argc, char **argv) if (args.contains("--no-audio")) audioOutput = false; +#ifdef Q_OS_ANDROID + // There is no audio backend for Android yet + audioOutput = false; +#endif + if (args.contains("--no-sonification")) sonification = false; if (args.contains("--no-spectrogram")) spectrogram = false; @@ -333,6 +411,23 @@ main(int argc, char **argv) gui->show(); +#ifdef Q_OS_ANDROID + if (!pluginProblems.empty()) { + // Without the plugins no audio file can be analysed, and on a + // phone the log that says why is out of the user's sight + QStringList lines; + for (QString problem : pluginProblems) { + lines << problem.toHtmlEscaped(); + } + QMessageBox::warning + (gui, QMessageBox::tr("Plugins not found"), + QMessageBox::tr("The pitch analysis plugins could not be set up

%1

") + .arg(lines.join("
"))); + } +#else + (void)pluginProblems; // none but Android's +#endif + application.readyForFiles(); for (QStringList::iterator i = args.begin(); i != args.end(); ++i) { diff --git a/main/test/TestAndroidFiles.h b/main/test/TestAndroidFiles.h new file mode 100644 index 00000000..b25ae764 --- /dev/null +++ b/main/test/TestAndroidFiles.h @@ -0,0 +1,275 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_ANDROID_FILES_H +#define TEST_ANDROID_FILES_H + +// Tier 1: the file work the Android build does at start (the Vamp plugin +// links) and when a file is picked (the copy into app storage), done here +// on plain files in a temporary directory. What only a phone has -- the +// content:// URI, the installed library directory -- is not here. + +#include "../AndroidFiles.h" + +#include "plugin/PluginIdentifier.h" + +#include +#include +#include +#include +#include +#include +#include + +class TestAndroidFiles : public QObject +{ + Q_OBJECT + + QTemporaryDir m_dir; + int m_counter = 0; + + // A path of its own for each use, created or not + QString newPath(QString name) { + return m_dir.filePath(QString("%1-%2").arg(name).arg(++m_counter)); + } + + QString newDir(QString name) { + QString path = newPath(name); + QDir().mkpath(path); + return path; + } + + static bool writeFile(QString path, QByteArray content) { + QFile file(path); + if (!file.open(QIODevice::WriteOnly)) return false; + bool ok = (file.write(content) == content.size()); + file.close(); + return ok; + } + + static QByteArray readFile(QString path) { + QFile file(path); + if (!file.open(QIODevice::ReadOnly)) return QByteArray("(unreadable)"); + return file.readAll(); + } + + // What svcore's scan of a VAMP_PATH directory finds there + // (NativeVampPluginFactory's getCandidateLibraries()) + static QStringList scanned(QString dir) { + return QDir(dir, "*.so", QDir::Name | QDir::IgnoreCase, + QDir::Files | QDir::Readable).entryList(); + } + + // The folder as Android installs an application's libraries: the + // plugins among them, renamed lib*.so + QString installedLibraries(QByteArray pyin, QByteArray chp) { + QString dir = newDir("lib-arm64"); + writeFile(dir + "/libTony_arm64-v8a.so", "Tony"); + writeFile(dir + "/libQt6Core_arm64-v8a.so", "Qt"); + writeFile(dir + "/libplugins_platforms_qtforandroid_arm64-v8a.so", "Qt"); + writeFile(dir + "/libc++_shared.so", "C++"); + if (pyin != "") writeFile(dir + "/libpyin.so", pyin); + if (chp != "") writeFile(dir + "/libchp.so", chp); + return dir; + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + // --- The copy of a picked file --- + + void a_picked_file_is_copied_under_its_own_name() { + // On Android the source is a content:// URI, named like nothing + // the user knows + QString source = newDir("provider") + "/document-1234"; + QVERIFY(writeFile(source, "RIFF and the rest")); + + QString into = newPath("imported"); // not there yet + QString error; + QString copy = AndroidFiles::copyIn(source, "My Song.wav", into, error); + + QCOMPARE(error, QString()); + QCOMPARE(copy, QDir(into).filePath("My Song.wav")); + QCOMPARE(readFile(copy), QByteArray("RIFF and the rest")); + QCOMPARE(readFile(source), QByteArray("RIFF and the rest")); + } + + void picking_a_file_of_the_same_name_again_replaces_the_copy() { + QString from = newDir("provider"); + QVERIFY(writeFile(from + "/a", "first")); + QVERIFY(writeFile(from + "/b", "second, and longer")); + + QString into = newPath("imported"); + QString error; + QString first = AndroidFiles::copyIn(from + "/a", "take.wav", into, error); + QString second = AndroidFiles::copyIn(from + "/b", "take.wav", into, error); + + QCOMPARE(error, QString()); + QCOMPARE(second, first); + QCOMPARE(readFile(second), QByteArray("second, and longer")); + QCOMPARE(QDir(into).entryList(QDir::Files), QStringList({ "take.wav" })); + } + + void a_failed_copy_says_why_and_keeps_the_earlier_one() { + QString from = newDir("provider"); + QVERIFY(writeFile(from + "/a", "the good one")); + + QString into = newPath("imported"); + QString error; + QString copy = AndroidFiles::copyIn(from + "/a", "take.wav", into, error); + QCOMPARE(error, QString()); + + QString failed = AndroidFiles::copyIn(from + "/gone", "take.wav", + into, error); + QCOMPARE(failed, QString()); + QVERIFY(error != ""); + QCOMPARE(readFile(copy), QByteArray("the good one")); + QCOMPARE(QDir(into).entryList(QDir::Files), QStringList({ "take.wav" })); + } + + void a_name_cannot_put_the_copy_elsewhere() { + QString source = newDir("provider") + "/document"; + QVERIFY(writeFile(source, "x")); + + QString into = newPath("imported"); + QString error; + QString copy = AndroidFiles::copyIn(source, "../../escape.wav", + into, error); + + QCOMPARE(error, QString()); + QCOMPARE(QFileInfo(copy).absolutePath(), QFileInfo(into).absoluteFilePath()); + QCOMPARE(QFileInfo(copy).fileName(), QString(".._.._escape.wav")); + } + + void names_are_made_safe_to_use() { + QCOMPARE(AndroidFiles::safeFileName("Song 1.wav"), QString("Song 1.wav")); + QCOMPARE(AndroidFiles::safeFileName("Ääni – live.mp3"), + QString("Ääni – live.mp3")); + QCOMPARE(AndroidFiles::safeFileName("a/b\\c.wav"), QString("a_b_c.wav")); + QCOMPARE(AndroidFiles::safeFileName("What? : \"x\"|*.ogg"), + QString("What_ _Live__ _x___.ogg")); + QCOMPARE(AndroidFiles::safeFileName("line\nbreak.wav"), + QString("line_break.wav")); + QCOMPARE(AndroidFiles::safeFileName(" padded.wav "), QString("padded.wav")); + QCOMPARE(AndroidFiles::safeFileName(""), QString("imported")); + QCOMPARE(AndroidFiles::safeFileName("."), QString("imported")); + QCOMPARE(AndroidFiles::safeFileName(".."), QString("imported")); + } + + // --- The Vamp plugin links --- + + void the_plugins_are_linked_under_their_own_names() { +#ifdef Q_OS_WIN + QSKIP("Symbolic links: the Android build's, not Windows'"); +#endif + QString libraries = installedLibraries("pyin", "chp"); + QString vamp = newPath("vamp"); // not there yet + + QStringList problems; + QStringList links = AndroidFiles::linkVampPlugins + (libraries, vamp, { "pyin", "chp" }, problems); + + QCOMPARE(problems, QStringList()); + QCOMPARE(links, QStringList({ vamp + "/pyin.so", vamp + "/chp.so" })); + + // The scan opens every .so it finds: the plugins, and nothing else + QCOMPARE(scanned(vamp), QStringList({ "chp.so", "pyin.so" })); + QVERIFY(QFileInfo(links[0]).isSymLink()); + QCOMPARE(readFile(links[0]), QByteArray("pyin")); + QCOMPARE(readFile(links[1]), QByteArray("chp")); + + // svcore names a plugin after its library's file name, and Tony + // asks for pYIN as vamp:pyin:pyin:... (Analyser) + QCOMPARE(sv::PluginIdentifier::createIdentifier("vamp", links[0], "pyin"), + QString("vamp:pyin:pyin")); + } + + void links_left_by_an_earlier_install_are_replaced() { +#ifdef Q_OS_WIN + QSKIP("Symbolic links: the Android build's, not Windows'"); +#endif + QString vamp = newPath("vamp"); + QStringList problems; + + QString before = installedLibraries("old pyin", "old chp"); + AndroidFiles::linkVampPlugins(before, vamp, { "pyin", "chp" }, problems); + QCOMPARE(problems, QStringList()); + + // An update installs the libraries in a new folder and removes + // the old one, leaving the links pointing nowhere + QVERIFY(QDir(before).removeRecursively()); + QString after = installedLibraries("new pyin", "new chp"); + + QStringList links = AndroidFiles::linkVampPlugins + (after, vamp, { "pyin", "chp" }, problems); + + QCOMPARE(problems, QStringList()); + QCOMPARE(links.size(), 2); + QCOMPARE(readFile(links[0]), QByteArray("new pyin")); + QCOMPARE(readFile(links[1]), QByteArray("new chp")); + QCOMPARE(QFileInfo(links[0]).symLinkTarget(), + QFileInfo(after + "/libpyin.so").absoluteFilePath()); + QCOMPARE(scanned(vamp), QStringList({ "chp.so", "pyin.so" })); + } + + void a_missing_plugin_is_reported_and_the_rest_linked() { +#ifdef Q_OS_WIN + QSKIP("Symbolic links: the Android build's, not Windows'"); +#endif + QString vamp = newPath("vamp"); + QStringList problems; + AndroidFiles::linkVampPlugins(installedLibraries("pyin", "chp"), vamp, + { "pyin", "chp" }, problems); + QCOMPARE(problems, QStringList()); + + QStringList links = AndroidFiles::linkVampPlugins + (installedLibraries("pyin", ""), vamp, { "pyin", "chp" }, problems); + + QCOMPARE(links, QStringList({ vamp + "/pyin.so" })); + QCOMPARE(problems.size(), 1); + QVERIFY(problems[0].contains("libchp.so")); + // Not the link from before, to a library that is not there now + QCOMPARE(scanned(vamp), QStringList({ "pyin.so" })); + } + + void a_linked_plugin_is_checked_as_the_scan_would_load_it() { +#ifdef Q_OS_WIN + QSKIP("Symbolic links: the Android build's, not Windows'"); +#endif + // The real pYIN, which the build leaves beside this executable, + // installed as Android installs it; and a library that is not one + QString built = QCoreApplication::applicationDirPath() + "/pyin.so"; + if (!QFileInfo::exists(built)) { + QSKIP("No pyin.so beside the test executable: build it first"); + } + QString libraries = installedLibraries("", "not a library"); + QVERIFY(QFile::copy(built, libraries + "/libpyin.so")); + + QString vamp = newPath("vamp"); + QStringList problems; + QStringList links = AndroidFiles::linkVampPlugins + (libraries, vamp, { "pyin", "chp" }, problems); + QCOMPARE(problems, QStringList()); + QCOMPARE(links.size(), 2); + + QCOMPARE(AndroidFiles::checkVampPlugin(links[0]), QString()); + + QString problem = AndroidFiles::checkVampPlugin(links[1]); + QVERIFY(problem.startsWith("Cannot load " + links[1] + ": ")); + QVERIFY(problem.size() > QString("Cannot load " + links[1] + ": ").size()); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 4a7ed2b5..7e9faf17 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -11,6 +11,7 @@ COPYING included with this distribution for more information. */ +#include "TestAndroidFiles.h" #include "TestRealtimeYin.h" #include "TestRealtimePitchTracker.h" #include "TestLatencyShift.h" @@ -44,6 +45,12 @@ int main(int argc, char *argv[]) app.setOrganizationName("tony-tests"); app.setApplicationName("test-tony-core"); + { + TestAndroidFiles t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestRealtimeYin t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 3896aad1..37147c40 100644 --- a/meson.build +++ b/meson.build @@ -536,6 +536,11 @@ elif system == 'android' '-Wl,--version-script=' + meson.current_source_dir() / 'vamp-plugin-sdk/vamp-plugin.map' ] + # Android's system log, which main.cpp passes stdout and stderr on to + feature_additional_libs = [ + '-llog' + ] + else error('This operating system ("' + system + '") is not supported by this build file') endif # system @@ -1160,6 +1165,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ + 'main/AndroidFiles.cpp', 'main/Coverage.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', @@ -1454,6 +1460,7 @@ if system != 'android' tony_core_test_moc_files = qt.preprocess( moc_headers: [ + 'main/test/TestAndroidFiles.h', 'main/test/TestRealtimeYin.h', 'main/test/TestRealtimePitchTracker.h', 'main/test/TestLatencyShift.h', From 0b8d007461601cbb3e9360412b4af5406921576d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:01:00 +0000 Subject: [PATCH 111/275] feat: read and write TTML lyrics, with an end for every word LRC gives each word a start only; the Moises-Lyric-Exporter's TTML gives a start and an end, and syllables where Moises has them. parseTtml() reads the Apple-style TTML the exporter and AMLL TTML Tool write, joining syllables into words and skipping background vocals and translations; writeTtml() writes it back in the exporter's own style. parseLyrics() tells the two formats apart. The exporter fixtures were made by the exporter's own code. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/Lyrics.cpp | 108 ++- main/Lyrics.h | 45 +- main/LyricsTtml.cpp | 858 +++++++++++++++++++ main/LyricsTtml.h | 49 ++ main/test/TestLyricsTtml.h | 909 +++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + testdata/lyrics/amll-style.ttml | 5 + testdata/lyrics/moises-exporter-lines.ttml | 15 + testdata/lyrics/moises-exporter-words.ttml | 15 + 10 files changed, 1959 insertions(+), 54 deletions(-) create mode 100644 main/LyricsTtml.cpp create mode 100644 main/LyricsTtml.h create mode 100644 main/test/TestLyricsTtml.h create mode 100644 testdata/lyrics/amll-style.ttml create mode 100644 testdata/lyrics/moises-exporter-lines.ttml create mode 100644 testdata/lyrics/moises-exporter-words.ttml diff --git a/main/Lyrics.cpp b/main/Lyrics.cpp index 3f3c09e3..08c30ca2 100644 --- a/main/Lyrics.cpp +++ b/main/Lyrics.cpp @@ -13,6 +13,7 @@ */ #include "Lyrics.h" +#include "LyricsTtml.h" #include #include @@ -135,45 +136,6 @@ bool readMetadata(const QString &row, QString &key, QString &value) return true; } -/** - * XML 1.0 cannot hold the C0 controls other than tab (and the line - * breaks, which never get this far), nor U+FFFE and U+FFFF, which the - * UTF-8 decoder lets through: one in a label would make the session - * file unreadable. DEL is never meant as text either. A tab becomes - * a space, which is what it comes back as from a session file anyway: - * an XML attribute value is read with its tabs as spaces. - */ -QString withoutControls(const QString &s) -{ - QString out; - out.reserve(s.size()); - for (QChar c : s) { - const ushort u = c.unicode(); - if (u == '\t') { - out += QLatin1Char(' '); - continue; - } - if (u < 0x20 || u == 0x7F || u == 0xFFFE || u == 0xFFFF) { - continue; - } - out += c; - } - return out; -} - -// A label as it is shown and saved: trimmed, and not too long -QString labelFrom(const QString &text) -{ - QString s = text.trimmed(); - if (s.size() > Lyrics::maxLabelLength) { - int n = Lyrics::maxLabelLength; - // Never half of a surrogate pair - if (s.at(n - 1).isHighSurrogate()) --n; - s = s.left(n).trimmed(); - } - return s; -} - // Nothing but notes and space: the exporter's mark for a gap bool isOnlyMusic(const QString &text) { @@ -292,7 +254,7 @@ TimedLine readLineText(const QString &text, Ms stamp, int &backwards) previous.endGiven = true; } } - QString label = labelFrom(piece.text); + QString label = lyricsLabel(piece.text); if (label.isEmpty()) continue; Entry entry; entry.start = time; @@ -365,6 +327,40 @@ void inferEnds(QVector &lines) } // namespace +QString +lyricsWithoutControls(const QString &text) +{ + QString out; + out.reserve(text.size()); + for (QChar c : text) { + const ushort u = c.unicode(); + if (u == '\t') { + out += QLatin1Char(' '); + continue; + } + // The line breaks too: an LRC row never has one, and to TTML + // they are whitespace, which its parser has made spaces + if (u < 0x20 || u == 0x7F || u == 0xFFFE || u == 0xFFFF) { + continue; + } + out += c; + } + return out; +} + +QString +lyricsLabel(const QString &text) +{ + QString s = lyricsWithoutControls(text).trimmed(); + if (s.size() > Lyrics::maxLabelLength) { + int n = Lyrics::maxLabelLength; + // Never half of a surrogate pair + if (s.at(n - 1).isHighSurrogate()) --n; + s = s.left(n).trimmed(); + } + return s; +} + int Lyrics::lineCount() const { @@ -401,7 +397,7 @@ parseLrc(const QByteArray &bytes) const QStringList rows = text.split(QLatin1Char('\n')); for (const QString &raw : rows) { - const QString row = withoutControls(raw).trimmed(); + const QString row = lyricsWithoutControls(raw).trimmed(); if (row.isEmpty()) continue; QVector stamps; @@ -433,12 +429,12 @@ parseLrc(const QByteArray &bytes) } else { result.warnings << tr("The offset \"%1\" is not a whole " "number of milliseconds and was " - "ignored.").arg(labelFrom(value)); + "ignored.").arg(lyricsLabel(value)); } } else if (key == QLatin1String("ti")) { - lyrics.title = labelFrom(value); + lyrics.title = lyricsLabel(value); } else if (key == QLatin1String("ar")) { - lyrics.artist = labelFrom(value); + lyrics.artist = lyricsLabel(value); } continue; } @@ -517,6 +513,30 @@ parseLrc(const QByteArray &bytes) return result; } +LyricsParseResult +parseLyrics(const QByteArray &bytes) +{ + // UTF-16 and UTF-32 have to be decoded to be looked at; without a + // BOM, or with UTF-8's, whatever the encoding, the blanks and the + // '<' are single ASCII bytes, as Latin-1 reads them. The decoder + // drops the BOM. + std::optional bom = + QStringConverter::encodingForData(bytes); + QString text; + if (bom) { + text = QStringDecoder(*bom).decode(bytes); + } else { + text = QString::fromLatin1(bytes); + } + + for (QChar c : text) { + if (c.isSpace()) continue; + if (c == QLatin1Char('<')) return parseTtml(bytes); + break; + } + return parseLrc(bytes); +} + EventVector lyricsToEvents(const Lyrics &lyrics, sv_samplerate_t rate) { diff --git a/main/Lyrics.h b/main/Lyrics.h index da57ab83..06bb0d12 100644 --- a/main/Lyrics.h +++ b/main/Lyrics.h @@ -24,13 +24,14 @@ #include /** - * Timed lyrics read from an LRC file, and the events of the region - * model that holds them in a session: one region per word (or per - * line, for a file that times only lines), with the line's index as - * its value. + * Timed lyrics read from an LRC or TTML file, and the events of the + * region model that holds them in a session: one region per word (or + * per line, for a file that times only lines), with the line's index + * as its value. * * These are pure functions over bytes and event lists: they touch no - * model, so they can be tested without a window (TestLyrics). + * model, so they can be tested without a window (TestLyrics, + * TestLyricsTtml). */ /** @@ -62,11 +63,11 @@ struct Lyrics /// Sorted by start QVector words; - /// From [ti:] and [ar:], for the layer's name + /// From [ti:] and [ar:], or TTML's , for the layer's name QString title; QString artist; - /// Any word tags were seen + /// Any word tags, or TTML s with times, were seen bool wordTimed = false; bool isEmpty() const { return words.isEmpty(); } @@ -87,7 +88,7 @@ struct Lyrics /// A longer word or line is cut to this many characters static constexpr int maxLabelLength = 200; - /// LRC files are a few kB; a bigger file is not read at all + /// Lyrics files are a few kB; a bigger file is not read at all static constexpr qint64 maxFileBytes = 1024 * 1024; }; @@ -95,8 +96,8 @@ struct LyricsParseResult { Lyrics lyrics; - /// Non-empty if there is nothing usable (not LRC, no timed lines, - /// too big); lyrics is then empty + /// Non-empty if there is nothing usable (not LRC or TTML, no timed + /// lines, too big); lyrics is then empty QString error; /// Short sentences on what was skipped or changed, for the status bar @@ -109,6 +110,30 @@ struct LyricsParseResult */ LyricsParseResult parseLrc(const QByteArray &bytes); +/** + * Read a lyrics file of either kind: TTML (parseTtml(), in + * LyricsTtml.h) if its first character that is not blank, after a + * byte order mark, is '<', else LRC. + */ +LyricsParseResult parseLyrics(const QByteArray &bytes); + +/** + * The text without the characters a session file cannot hold: XML 1.0 + * has no C0 controls other than tab, nor U+FFFE and U+FFFF, which a + * UTF-8 decoder lets through, and one in a label would make the session + * unreadable. DEL is never meant as text either. A tab becomes a + * space, which is what it comes back as from a session file anyway: an + * XML attribute value is read with its tabs as spaces. + */ +QString lyricsWithoutControls(const QString &text); + +/** + * The text as a word's label is shown and saved: without controls, + * trimmed, and cut to Lyrics::maxLabelLength characters. Every parser + * makes its labels with this. + */ +QString lyricsLabel(const QString &text); + /** * The events of a region model holding the lyrics: frame = start, * duration = end - start (at least 1 frame), value = line, label = diff --git a/main/LyricsTtml.cpp b/main/LyricsTtml.cpp new file mode 100644 index 00000000..9dea9d60 --- /dev/null +++ b/main/LyricsTtml.cpp @@ -0,0 +1,858 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LyricsTtml.h" + +#include +#include +#include +#include + +#include +#include +#include + +namespace { + +QString tr(const char *text) +{ + return QCoreApplication::translate("Lyrics", text); +} + +// "1 line was" or "3 lines were": the status bar shows these +QString counted(int n, const char *one, const char *many) +{ + if (n == 1) return tr(one); + return tr(many).arg(n); +} + +const char *const ttmlNamespace = "http://www.w3.org/ns/ttml"; +const char *const itunesNamespace = "http://music.apple.com/lyric-ttml-internal"; +const char *const metadataNamespace = "http://www.w3.org/ns/ttml#metadata"; + +// Only to keep the arithmetic far from overflowing, as the LRC parser +// has it: no reference is a day long +const qint64 maxSeconds = 24 * 3600; + +// The biggest number a time can have in it: that day in milliseconds +const qint64 maxNumber = maxSeconds * 1000; + +// Digits kept after the point: microseconds are plenty +const int maxFractionDigits = 6; + +bool isAsciiDigit(QChar c) +{ + return c.unicode() >= '0' && c.unicode() <= '9'; +} + +int digitValue(QChar c) +{ + return c.unicode() - '0'; +} + +// What XML counts as white space; nothing else separates words +bool isXmlSpace(QChar c) +{ + const ushort u = c.unicode(); + return u == ' ' || u == '\t' || u == '\n' || u == '\r'; +} + +QStringView xmlTrimmed(QStringView s) +{ + qsizetype b = 0, e = s.size(); + while (b < e && isXmlSpace(s[b])) ++b; + while (e > b && isXmlSpace(s[e - 1])) --e; + return s.mid(b, e - b); +} + +// Each line break or tab a space, so that the cleaning, which drops +// controls, does not glue the words either side of it +QString spaced(QStringView s) +{ + QString out = s.toString(); + for (QChar &c : out) { + if (isXmlSpace(c)) c = QLatin1Char(' '); + } + return out; +} + +// Every run of white space one space +QString collapsed(QStringView s) +{ + QString out; + out.reserve(s.size()); + bool blank = false; + for (QChar c : s) { + if (isXmlSpace(c)) { + blank = true; + continue; + } + if (blank && !out.isEmpty()) out += QLatin1Char(' '); + blank = false; + out += c; + } + return out; +} + +/** + * At least one digit at i, which is moved past them; false if there + * is none, or so many that they could be no time. + */ +bool readNumber(QStringView s, qsizetype &i, qint64 &value) +{ + const qsizetype from = i; + value = 0; + while (i < s.size() && isAsciiDigit(s[i])) { + value = value * 10 + digitValue(s[i]); + if (value > maxNumber) return false; + ++i; + } + return i > from; +} + +/** + * A TTML time in seconds: a clock time [[h:]m:]s[.fraction], where + * the minutes can pass 59 when there are no hours (the exporter writes + * m:ss.mmm, AMLL TTML Tool mm:ss.mmm, Apple sometimes plain seconds), + * or an offset time h, m, s or ms. Frames and ticks (f, t, + * hh:mm:ss:ff) are not read: nothing is returned for them, as for + * anything else. + */ +std::optional readTime(QStringView text) +{ + const QStringView s = xmlTrimmed(text); + const qsizetype n = s.size(); + qsizetype i = 0; + + qint64 fields[3]; + int count = 0; + while (true) { + if (!readNumber(s, i, fields[count])) return {}; + ++count; + if (count < 3 && i < n && s[i] == QLatin1Char(':')) { + ++i; + continue; + } + break; + } + + qint64 fraction = 0; + qint64 scale = 1; + if (i < n && s[i] == QLatin1Char('.')) { + ++i; + int digits = 0; + while (i < n && isAsciiDigit(s[i])) { + if (digits < maxFractionDigits) { + fraction = fraction * 10 + digitValue(s[i]); + scale *= 10; + } + ++digits; + ++i; + } + if (digits == 0) return {}; + } + + // The number is in units of multiplier / divisor seconds + qint64 whole = 0; + qint64 multiplier = 1; + qint64 divisor = 1; + const QStringView metric = s.mid(i); + + if (count == 1) { + whole = fields[0]; + if (metric.isEmpty() || metric == QLatin1String("s")) { + } else if (metric == QLatin1String("ms")) { + divisor = 1000; + } else if (metric == QLatin1String("m")) { + multiplier = 60; + } else if (metric == QLatin1String("h")) { + multiplier = 3600; + } else { + return {}; + } + } else { + if (!metric.isEmpty()) return {}; + const qint64 hours = (count == 3 ? fields[0] : 0); + const qint64 minutes = fields[count - 2]; + const qint64 seconds = fields[count - 1]; + if (seconds >= 60) return {}; + if (count == 3 && minutes >= 60) return {}; + whole = (hours * 60 + minutes) * 60 + seconds; + } + + if (whole > maxSeconds * divisor / multiplier) return {}; + + // Whole milliseconds, as nearly every file has them, come out as + // the nearest double to the decimal, not a step away from it + const qint64 units = whole * scale + fraction; + const double result = + double(units) * double(multiplier) / (double(scale) * double(divisor)); + if (result > double(maxSeconds)) return {}; + return result; +} + +// What was skipped or changed, for the warnings +struct Counts +{ + int unreadableBegins = 0; + int unreadableEnds = 0; + int untimedLines = 0; + int droppedText = 0; + int skippedParts = 0; + int backwards = 0; +}; + +struct Timing +{ + /// There is a begin, whether or not it could be read + bool hasBegin = false; + + std::optional begin; + + /// The end, or else begin + dur + std::optional end; +}; + +/** + * An element's begin, end and dur, matched by local name in any + * namespace. An end that cannot be read is counted only when the + * begin can: a word is left out for its begin, with a warning of its + * own, but a line with timed words needs no begin. + */ +Timing readTiming(const QXmlStreamAttributes &attributes, Counts &counts) +{ + std::optional begin, end, dur; + for (const QXmlStreamAttribute &a : attributes) { + const QStringView name = a.name(); + if (name == QLatin1String("begin")) { + if (!begin) begin = a.value(); + } else if (name == QLatin1String("end")) { + if (!end) end = a.value(); + } else if (name == QLatin1String("dur")) { + if (!dur) dur = a.value(); + } + } + + Timing timing; + if (begin) { + timing.hasBegin = true; + timing.begin = readTime(*begin); + } + + if (end) { + timing.end = readTime(*end); + if (!timing.end && (timing.begin || !timing.hasBegin)) { + ++counts.unreadableEnds; + } + } + if (!timing.end && dur && timing.begin) { + std::optional length = readTime(*dur); + if (length) { + timing.end = *timing.begin + *length; + } else { + ++counts.unreadableEnds; + } + } + return timing; +} + +/** + * Background vocals, translations and romanisations: AMLL TTML Tool + * puts them in the line as spans with these roles. ttm:role is a + * list, in TTML 1. + */ +bool hasSkippedRole(const QXmlStreamAttributes &attributes) +{ + static const QStringList skipped = { + QStringLiteral("x-bg"), QStringLiteral("x-translation"), + QStringLiteral("x-roman"), QStringLiteral("x-romanization") + }; + for (const QXmlStreamAttribute &a : attributes) { + if (a.name() != QLatin1String("role")) continue; + // An attribute's line breaks and tabs are read as spaces + const QStringList roles = + a.value().toString().split(QLatin1Char(' '), Qt::SkipEmptyParts); + for (const QString &role : roles) { + if (skipped.contains(role)) return true; + } + } + return false; +} + +// What a line holds, in file order, before it is made into words +struct Piece +{ + enum Kind { + Timed, ///< A span with a begin + Text, ///< Text outside timed spans, with no blanks + Blank ///< White space, a
or a part left out + }; + + Kind kind = Blank; + QString text; + double begin = 0.0; + std::optional end; +}; + +void addBlank(QVector &pieces) +{ + if (!pieces.isEmpty() && pieces.last().kind == Piece::Blank) return; + pieces.push_back(Piece()); +} + +void addText(QStringView text, QVector &pieces) +{ + const qsizetype n = text.size(); + qsizetype i = 0; + while (i < n) { + const bool blank = isXmlSpace(text[i]); + qsizetype j = i; + while (j < n && isXmlSpace(text[j]) == blank) ++j; + if (blank) { + addBlank(pieces); + } else if (!pieces.isEmpty() && pieces.last().kind == Piece::Text) { + // The reader may hand one run of text over in parts + pieces.last().text += text.mid(i, j - i); + } else { + Piece piece; + piece.kind = Piece::Text; + piece.text = text.mid(i, j - i).toString(); + pieces.push_back(piece); + } + i = j; + } +} + +/** + * All the text in the element the reader is at, which is read to its + * end. A
is a space, and so is a skipped part, so that the + * words either side of it are not joined; elements other than spans + * (metadata, animation) hold no lyrics and are passed over. + */ +QString readAllText(QXmlStreamReader &reader, Counts &counts) +{ + QString text; + int depth = 1; + while (!reader.atEnd()) { + switch (reader.readNext()) { + case QXmlStreamReader::Characters: + text += reader.text(); + break; + case QXmlStreamReader::StartElement: + if (reader.name() == QLatin1String("br")) { + text += QLatin1Char(' '); + reader.skipCurrentElement(); + } else if (reader.name() != QLatin1String("span")) { + reader.skipCurrentElement(); + } else if (hasSkippedRole(reader.attributes())) { + ++counts.skippedParts; + text += QLatin1Char(' '); + reader.skipCurrentElement(); + } else { + ++depth; + } + break; + case QXmlStreamReader::EndElement: + if (--depth == 0) return text; + break; + default: + break; + } + } + return text; +} + +/** + * The content of the element the reader is at, a

or a wrapping + * , read to its end. + */ +void readInline(QXmlStreamReader &reader, QVector &pieces, + Counts &counts) +{ + while (!reader.atEnd()) { + + const QXmlStreamReader::TokenType token = reader.readNext(); + if (token == QXmlStreamReader::EndElement) return; + if (token == QXmlStreamReader::Characters) { + addText(reader.text(), pieces); + continue; + } + if (token != QXmlStreamReader::StartElement) continue; + + if (reader.name() == QLatin1String("br")) { + addBlank(pieces); + reader.skipCurrentElement(); + continue; + } + if (reader.name() != QLatin1String("span")) { + reader.skipCurrentElement(); + continue; + } + + const QXmlStreamAttributes attributes = reader.attributes(); + + if (hasSkippedRole(attributes)) { + // It can come straight after a word, and a word after it + ++counts.skippedParts; + addBlank(pieces); + reader.skipCurrentElement(); + continue; + } + + const Timing timing = readTiming(attributes, counts); + + if (!timing.hasBegin) { + // A wrapper: its content is read as if it were not there + readInline(reader, pieces, counts); + continue; + } + + const QString text = readAllText(reader, counts); + + if (!timing.begin) { + ++counts.unreadableBegins; + addBlank(pieces); + continue; + } + + // Blanks at either end of its text are blanks between words + if (!text.isEmpty() && isXmlSpace(text.front())) addBlank(pieces); + Piece piece; + piece.kind = Piece::Timed; + piece.text = xmlTrimmed(text).toString(); + piece.begin = *timing.begin; + piece.end = timing.end; + pieces.push_back(piece); + if (!text.isEmpty() && isXmlSpace(text.back())) addBlank(pieces); + } +} + +// A word before it has its line index, and perhaps its end +struct Entry +{ + double start = 0.0; + std::optional end; + bool endGiven = false; + QString text; +}; + +struct Line +{ + /// In time order + QVector words; + + /// The

's own end, if it has one + std::optional end; + + /// It had no timed spans: its one word is the whole line + bool lineTimed = false; +}; + +/** + * The

the reader is at, read to its end: one line, unless nothing + * in it is timed and it has no begin either, or it has no text. + */ +void readParagraph(QXmlStreamReader &reader, QVector &lines, + Counts &counts) +{ + const Timing timing = readTiming(reader.attributes(), counts); + + QVector pieces; + readInline(reader, pieces, counts); + + Line line; + line.end = timing.end; + + const bool anyTimed = + std::any_of(pieces.begin(), pieces.end(), [](const Piece &p) { + return p.kind == Piece::Timed; + }); + + if (!anyTimed) { + QString text; + for (const Piece &p : pieces) { + text += (p.kind == Piece::Blank ? QStringLiteral(" ") : p.text); + } + const QString label = lyricsLabel(collapsed(text)); + if (label.isEmpty()) return; + if (timing.hasBegin && !timing.begin) { + ++counts.unreadableBegins; + return; + } + if (!timing.begin) { + ++counts.untimedLines; + return; + } + Entry entry; + entry.start = *timing.begin; + entry.end = timing.end; + entry.text = label; + line.words.push_back(entry); + line.lineTimed = true; + lines.push_back(line); + return; + } + + // Timed spans with nothing but text between them are one word: + // syllables, or a word and the punctuation after it + std::optional current; + auto finish = [&]() { + if (!current) return; + current->text = lyricsLabel(spaced(current->text)); + if (!current->text.isEmpty()) line.words.push_back(*current); + current.reset(); + }; + + for (const Piece &p : pieces) { + switch (p.kind) { + case Piece::Blank: + finish(); + break; + case Piece::Timed: + if (current) { + current->text += p.text; + current->end = p.end; + } else { + Entry entry; + entry.start = p.begin; + entry.end = p.end; + entry.text = p.text; + current = entry; + } + break; + case Piece::Text: + if (current) { + current->text += p.text; + } else { + ++counts.droppedText; + } + break; + } + } + finish(); + + if (line.words.isEmpty()) return; + + std::stable_sort(line.words.begin(), line.words.end(), + [](const Entry &a, const Entry &b) { + return a.start < b.start; + }); + lines.push_back(line); +} + +/** + * Where each word ends when the file does not say: at the next word + * of its line, else at the end of the line, else as the LRC parser + * has it, a while after its start but not past the next line's start + * (a line-timed line: at the next line's start). The lines must be in + * time order. + */ +void inferEnds(QVector &lines, Counts &counts) +{ + for (int k = 0; k < lines.size(); ++k) { + Line &line = lines[k]; + for (int i = 0; i < line.words.size(); ++i) { + Entry &e = line.words[i]; + if (e.end) { + e.endGiven = true; + } else if (i + 1 < line.words.size()) { + e.end = line.words[i + 1].start; + } else if (line.end) { + e.end = line.end; + e.endGiven = true; + } else if (k + 1 < lines.size()) { + const double next = lines[k + 1].words[0].start; + if (line.lineTimed) { + e.end = next; + } else { + e.end = std::min(next, e.start + + Lyrics::inferredWordSeconds); + } + } else { + e.end = e.start + (line.lineTimed ? + Lyrics::inferredLastLineSeconds : + Lyrics::inferredWordSeconds); + } + + if (*e.end < e.start) { + if (e.endGiven) ++counts.backwards; + e.end = e.start; + } + } + } +} + +// Seconds as m:ss.mmm, to the nearest millisecond +QString ttmlTime(double seconds) +{ + const qint64 ms = std::max(qint64(0), qint64(std::llround(seconds * 1000))); + return QString("%1:%2.%3") + .arg(ms / 60000) + .arg((ms % 60000) / 1000, 2, 10, QLatin1Char('0')) + .arg(ms % 1000, 3, 10, QLatin1Char('0')); +} + +} // namespace + +LyricsParseResult +parseTtml(const QByteArray &bytes) +{ + LyricsParseResult result; + + if (bytes.isEmpty()) { + result.error = tr("The file is empty."); + return result; + } + if (bytes.size() > Lyrics::maxFileBytes) { + result.error = tr("The file is over 1 MB, too big to be a TTML " + "lyrics file."); + return result; + } + + // The reader finds the encoding from the BOM or the declaration + QXmlStreamReader reader(bytes); + + Counts counts; + QVector lines; + QString title; + bool inHead = false; + + while (!reader.atEnd()) { + const QXmlStreamReader::TokenType token = reader.readNext(); + + if (token == QXmlStreamReader::DTD) { + result.error = tr("The file has a document type declaration " + "(), which a TTML file never has, " + "so it was not read."); + return result; + } + + if (token == QXmlStreamReader::StartElement) { + const QStringView name = reader.name(); + if (name == QLatin1String("head")) { + inHead = true; + } else if (inHead) { + if (name == QLatin1String("title") && title.isEmpty()) { + title = lyricsLabel(collapsed + (reader.readElementText + (QXmlStreamReader::IncludeChildElements))); + } + } else if (name == QLatin1String("p")) { + readParagraph(reader, lines, counts); + } + } else if (token == QXmlStreamReader::EndElement && + reader.name() == QLatin1String("head")) { + inHead = false; + } + } + + if (reader.hasError()) { + result.error = tr("The file is not well-formed XML: %1 (line %2).") + .arg(reader.errorString()).arg(reader.lineNumber()); + return result; + } + + // Lines in time order; those that start together stay in file order + std::stable_sort(lines.begin(), lines.end(), + [](const Line &a, const Line &b) { + return a.words[0].start < b.words[0].start; + }); + + inferEnds(lines, counts); + + Lyrics &lyrics = result.lyrics; + lyrics.title = title; + for (int k = 0; k < lines.size(); ++k) { + if (!lines[k].lineTimed) lyrics.wordTimed = true; + for (const Entry &e : lines[k].words) { + LyricWord word; + word.start = e.start; + word.end = *e.end; + word.text = e.text; + word.line = k; + word.endGiven = e.endGiven; + lyrics.words.push_back(word); + } + } + + std::stable_sort(lyrics.words.begin(), lyrics.words.end(), + [](const LyricWord &a, const LyricWord &b) { + return a.start < b.start; + }); + + if (counts.unreadableBegins > 0) { + result.warnings << counted + (counts.unreadableBegins, + "1 word had a start time that could not be read and was " + "left out.", + "%1 words had start times that could not be read and were " + "left out."); + } + if (counts.unreadableEnds > 0) { + result.warnings << counted + (counts.unreadableEnds, + "1 end time could not be read and was ignored.", + "%1 end times could not be read and were ignored."); + } + if (counts.untimedLines > 0) { + result.warnings << counted + (counts.untimedLines, + "1 line had no time and was left out.", + "%1 lines had no time and were left out."); + } + if (counts.droppedText > 0) { + result.warnings << counted + (counts.droppedText, + "1 piece of text outside the timed words was left out.", + "%1 pieces of text outside the timed words were left out."); + } + if (counts.skippedParts > 0) { + result.warnings << counted + (counts.skippedParts, + "1 background vocal, translation or romanisation was " + "skipped.", + "%1 background vocals, translations or romanisations were " + "skipped."); + } + if (counts.backwards > 0) { + result.warnings << counted + (counts.backwards, + "1 word ended before it started and was given no length.", + "%1 words ended before they started and were given no " + "length."); + } + + if (lyrics.words.isEmpty()) { + result.lyrics = Lyrics(); + result.error = tr("No timed lyrics were found: this is not a TTML " + "file, or it has no timed lines."); + } + + return result; +} + +QByteArray +writeTtml(const Lyrics &lyrics) +{ + const QString tt = QString::fromLatin1(ttmlNamespace); + const QString itunes = QString::fromLatin1(itunesNamespace); + const QString ttm = QString::fromLatin1(metadataNamespace); + + // A

for each line that has words, in line order + QMap> lines; + for (const LyricWord &w : lyrics.words) lines[w.line].push_back(w); + + double first = 0.0; + double last = 0.0; + for (int i = 0; i < lyrics.words.size(); ++i) { + const LyricWord &w = lyrics.words[i]; + if (i == 0 || w.start < first) first = w.start; + last = std::max(last, std::max(w.start, w.end)); + } + + QByteArray out; + QXmlStreamWriter writer(&out); + + // Laid out by hand, as the exporter lays it out: automatic + // formatting would put white space between the spans of a line + auto newline = [&](int indent) { + writer.writeCharacters(QLatin1String("\n") + + QString(indent * 2, QLatin1Char(' '))); + }; + + writer.writeStartDocument(); + newline(0); + + writer.writeDefaultNamespace(tt); + writer.writeNamespace(itunes, QStringLiteral("itunes")); + writer.writeNamespace(ttm, QStringLiteral("ttm")); + writer.writeStartElement(tt, QStringLiteral("tt")); + writer.writeAttribute(itunes, QStringLiteral("timing"), + QStringLiteral("Word")); + + newline(1); + writer.writeStartElement(tt, QStringLiteral("head")); + newline(2); + writer.writeStartElement(tt, QStringLiteral("metadata")); + if (!lyrics.title.isEmpty()) { + newline(3); + writer.writeTextElement(ttm, QStringLiteral("title"), lyrics.title); + } + newline(3); + writer.writeEmptyElement(ttm, QStringLiteral("agent")); + writer.writeAttribute(QStringLiteral("type"), QStringLiteral("person")); + writer.writeAttribute(QStringLiteral("xml:id"), QStringLiteral("v1")); + newline(2); + writer.writeEndElement(); // metadata + newline(1); + writer.writeEndElement(); // head + + newline(1); + writer.writeStartElement(tt, QStringLiteral("body")); + writer.writeAttribute(QStringLiteral("dur"), ttmlTime(last)); + newline(2); + writer.writeStartElement(tt, QStringLiteral("div")); + writer.writeAttribute(QStringLiteral("begin"), ttmlTime(first)); + writer.writeAttribute(QStringLiteral("end"), ttmlTime(last)); + + int key = 0; + for (auto i = lines.begin(); i != lines.end(); ++i) { + + QVector words = i.value(); + std::stable_sort(words.begin(), words.end(), + [](const LyricWord &a, const LyricWord &b) { + return a.start < b.start; + }); + double begin = words[0].start; + double end = words[0].end; + for (const LyricWord &w : words) { + end = std::max(end, w.end); + } + + newline(3); + writer.writeStartElement(tt, QStringLiteral("p")); + writer.writeAttribute(QStringLiteral("begin"), ttmlTime(begin)); + writer.writeAttribute(QStringLiteral("end"), ttmlTime(end)); + writer.writeAttribute(ttm, QStringLiteral("agent"), + QStringLiteral("v1")); + writer.writeAttribute(itunes, QStringLiteral("key"), + QStringLiteral("L%1").arg(++key)); + + for (int j = 0; j < words.size(); ++j) { + if (j > 0) writer.writeCharacters(QStringLiteral(" ")); + writer.writeStartElement(tt, QStringLiteral("span")); + writer.writeAttribute(QStringLiteral("begin"), + ttmlTime(words[j].start)); + writer.writeAttribute(QStringLiteral("end"), + ttmlTime(words[j].end)); + writer.writeCharacters(words[j].text); + writer.writeEndElement(); + } + + writer.writeEndElement(); // p + } + + newline(2); + writer.writeEndElement(); // div + newline(1); + writer.writeEndElement(); // body + newline(0); + writer.writeEndElement(); // tt + writer.writeEndDocument(); + + // Qt ends the document with a line break; not every version need + if (!out.endsWith('\n')) out += '\n'; + + return out; +} diff --git a/main/LyricsTtml.h b/main/LyricsTtml.h new file mode 100644 index 00000000..3c774ecf --- /dev/null +++ b/main/LyricsTtml.h @@ -0,0 +1,49 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_TTML_H +#define TONY_LYRICS_TTML_H + +#include "Lyrics.h" + +#include + +/** + * Timed lyrics in TTML, as Apple Music has them and as the + * Moises-Lyric-Exporter and AMLL TTML Tool write them: a

per line + * and a per word. Pure functions, as in Lyrics.h + * (TestLyricsTtml). + */ + +/** + * Read a TTML file. Its times are read as absolute, as every lyrics + * tool writes them, although strict TTML makes a child's times + * relative to its parent's. Timed spans with no whitespace between + * them are syllables of one word, and are joined; background vocals, + * translations and romanisations are skipped; a

with no timed + * spans is one word, the whole line. A file with a is + * refused: TTML has none, and it is how entity tricks get in. + */ +LyricsParseResult parseTtml(const QByteArray &bytes); + +/** + * The lyrics as TTML, UTF-8, in the exporter's Apple style: one agent, + * a

per line, a per word, times m:ss.mmm rounded to the + * millisecond. The title, if any, goes in ; the artist is + * not written. parseTtml() gives the same words back, to half a + * millisecond. + */ +QByteArray writeTtml(const Lyrics &lyrics); + +#endif diff --git a/main/test/TestLyricsTtml.h b/main/test/TestLyricsTtml.h new file mode 100644 index 00000000..c1c266c4 --- /dev/null +++ b/main/test/TestLyricsTtml.h @@ -0,0 +1,909 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LYRICS_TTML_H +#define TEST_LYRICS_TTML_H + +// Tier 2: reading and writing TTML lyrics, and parseLyrics() choosing +// between TTML and LRC. No window and no model. Two of the files in +// testdata/lyrics were written by the Moises-Lyric-Exporter's own TTML +// code, run on an invented lyrics.json; the third is written by hand +// as AMLL TTML Tool writes its files. + +#include "../Lyrics.h" +#include "../LyricsTtml.h" + +#include +#include +#include +#include + +#include + +class TestLyricsTtml : public QObject +{ + Q_OBJECT + + // The inferred lengths, as the parser has them + static constexpr double W = Lyrics::inferredWordSeconds; + static constexpr double L = Lyrics::inferredLastLineSeconds; + + // A whole file around the given

content, and head metadata + static QByteArray ttml(const char *div, const char *metadata = "") { + return QByteArray("\n" + "" + "") + metadata + + "
" + div + "
\n"; + } + + static LyricsParseResult parse(const char *div) { + return parseTtml(ttml(div)); + } + + static QByteArray fixture(const char *name) { + QFile file(QString(TONY_TEST_DATA_DIR) + "/lyrics/" + name); + if (!file.open(QIODevice::ReadOnly)) return QByteArray(); + return file.readAll(); + } + + static int wordCount(const LyricsParseResult &r) { + return int(r.lyrics.words.size()); + } + + struct Expected { + double start; + double end; + const char *text; + int line; + bool endGiven; + }; + + static QString describe(const LyricWord &w) { + return QString("\"%1\" %2-%3 line %4%5") + .arg(w.text).arg(w.start, 0, 'f', 6).arg(w.end, 0, 'f', 6) + .arg(w.line).arg(w.endGiven ? ", end given" : ""); + } + + static QString describe(const Lyrics &lyrics) { + QStringList list; + for (const LyricWord &w : lyrics.words) list << describe(w); + return list.join("; "); + } + + // Times of whole milliseconds are read as the nearest double to + // the decimal: this only absorbs the last bit of one + static bool sameTime(double a, double b) { + return std::abs(a - b) < 1e-9; + } + + // All the words, in order. A failure names the word that differs. + static void compareWords(const Lyrics &lyrics, + const QVector &expected) { + QVERIFY2(lyrics.words.size() == expected.size(), + qPrintable(QString("%1 words, expected %2: %3") + .arg(lyrics.words.size()).arg(expected.size()) + .arg(describe(lyrics)))); + for (int i = 0; i < expected.size(); ++i) { + const LyricWord &w = lyrics.words[i]; + LyricWord want; + want.start = expected[i].start; + want.end = expected[i].end; + want.text = QString::fromUtf8(expected[i].text); + want.line = expected[i].line; + want.endGiven = expected[i].endGiven; + bool same = (w.text == want.text && + sameTime(w.start, want.start) && + sameTime(w.end, want.end) && + w.line == want.line && + w.endGiven == want.endGiven); + QVERIFY2(same, qPrintable(QString("word %1 is %2, expected %3") + .arg(i).arg(describe(w)) + .arg(describe(want)))); + } + } + + // One warning, starting with the count it should give + static void verifyOneWarning(const LyricsParseResult &r, + const QString &count) { + QVERIFY2(r.warnings.size() == 1 && r.warnings[0].startsWith(count), + qPrintable(QString("warnings: [%1], expected one " + "starting \"%2\"") + .arg(r.warnings.join(" | ")).arg(count))); + } + + // Back from TTML, the same words, texts and lines, each time + // within half a millisecond + static void verifyRoundTrip(const Lyrics &lyrics, const char *what) { + const QByteArray written = writeTtml(lyrics); + LyricsParseResult r = parseTtml(written); + QVERIFY2(r.error.isEmpty(), + qPrintable(QString("%1: %2").arg(what).arg(r.error))); + QVERIFY2(r.warnings.isEmpty(), + qPrintable(QString("%1: %2").arg(what) + .arg(r.warnings.join(" | ")))); + QCOMPARE(r.lyrics.title, lyrics.title); + QVERIFY(r.lyrics.wordTimed); + QVERIFY2(r.lyrics.words.size() == lyrics.words.size(), + qPrintable(QString("%1: %2 words back, expected %3: %4") + .arg(what).arg(r.lyrics.words.size()) + .arg(lyrics.words.size()) + .arg(describe(r.lyrics)))); + const double halfMs = 0.0005 + 1e-9; + for (int i = 0; i < lyrics.words.size(); ++i) { + const LyricWord &w = r.lyrics.words[i]; + const LyricWord &o = lyrics.words[i]; + bool same = (w.text == o.text && w.line == o.line && + std::abs(w.start - o.start) <= halfMs && + std::abs(w.end - o.end) <= halfMs); + QVERIFY2(same, qPrintable(QString("%1: word %2 came back as " + "%3, was %4") + .arg(what).arg(i).arg(describe(w)) + .arg(describe(o)))); + } + // and writing it again gives the same file + QCOMPARE(writeTtml(r.lyrics), written); + } + +private slots: + // Clock times with and without hours, minutes past 59, plain + // seconds and offset times, all read as absolute + void times_in_every_accepted_form() { + struct Case { + const char *time; + double seconds; + } cases[] = { + { "0:01.000", 1.0 }, // the exporter + { "1:02.500", 62.5 }, + { "01:02.500", 62.5 }, // AMLL TTML Tool + { "1:02.5", 62.5 }, + { "1:02.50", 62.5 }, + { "1:02", 62.0 }, + { "1:2.25", 62.25 }, + { "75:00.000", 4500.0 }, // minutes past 59, no hours + { "1:01:02.5", 3662.5 }, + { "01:01:02.250", 3662.25 }, + { "0:00:00.001", 0.001 }, + { "62.5", 62.5 }, // plain seconds + { "62", 62.0 }, + { "4500.25", 4500.25 }, + { "0", 0.0 }, + { "62.5s", 62.5 }, // offset times + { "62500ms", 62.5 }, + { "250.5ms", 0.2505 }, + { "1.5m", 90.0 }, + { "2m", 120.0 }, + { "0.5h", 1800.0 }, + { " 1:02.500 ", 62.5 }, + { "1:02.1234567", 62.123456 }, // microseconds are plenty + { "24:00:00", 86400.0 }, // a day is the most + }; + for (const Case &c : cases) { + QByteArray div = QByteArray("

sana

"; + LyricsParseResult r = parseTtml(ttml(div.constData())); + QVERIFY2(r.error.isEmpty() && r.warnings.isEmpty(), + qPrintable(QString("\"%1\": %2 %3").arg(c.time) + .arg(r.error).arg(r.warnings.join(" ")))); + QCOMPARE(wordCount(r), 1); + QVERIFY2(sameTime(r.lyrics.words[0].start, c.seconds) && + sameTime(r.lyrics.words[0].end, c.seconds), + qPrintable(QString("\"%1\" read as %2") + .arg(c.time) + .arg(describe(r.lyrics.words[0])))); + } + } + + // Frames, ticks and anything else that is no time: the word is + // left out, with a warning, and the rest of the line is read + void refused_times() { + const char *times[] = { + "00:00:01:12", // hh:mm:ss:ff + "25f", "100t", // frames and ticks + "1:60.000", "1:60:00", "1:02:60", + "-1.0", "", " ", "abc", "1.2.3", "1:02.", ".5", "1e3", + "12 s", "1:02.5s", "1::02", ":02", "1:", + "24:00:00.001", "25:00:00", "1441m", + "99999999999999999999", + }; + for (const char *time : times) { + QByteArray div = QByteArray("

ei " + "kyllä

"; + LyricsParseResult r = parseTtml(ttml(div.constData())); + QVERIFY2(r.error.isEmpty(), time); + compareWords(r.lyrics, { { 1.0, 2.0, "kyllä", 0, true } }); + verifyOneWarning(r, "1 word had a start time"); + if (QTest::currentTestFailed()) { + qWarning() << "time:" << time; + return; + } + + // A line-timed line likewise + div = QByteArray("

" + "ei

kyllä

"; + r = parseTtml(ttml(div.constData())); + QVERIFY2(r.error.isEmpty(), time); + compareWords(r.lyrics, { { 1.0, 2.0, "kyllä", 0, true } }); + verifyOneWarning(r, "1 word had a start time"); + if (QTest::currentTestFailed()) { + qWarning() << "line time:" << time; + return; + } + } + } + + // A word ends at its end, or else at its begin + dur; an end that + // cannot be read is ignored, with one warning for all of them + void end_or_dur() { + LyricsParseResult r = parse + ("

a " + "b " + "c " + "d " + "e " + "f

"); + QCOMPARE(r.error, QString()); + compareWords(r.lyrics, { + { 1.0, 1.5, "a", 0, true }, + { 2.0, 2.25, "b", 0, true }, + { 3.0, 3.5, "c", 0, true }, + { 4.0, 4.25, "d", 0, true }, + { 5.0, 5.5, "e", 0, true }, + { 6.0, 6.0 + W, "f", 0, false }, + }); + verifyOneWarning(r, "2 "); + } + + // Timed spans with no white space between them are one word: + // first start, last end, the texts together + void syllables_joined() { + LyricsParseResult r = parse + ("

" + "kek" + "sit" + "ty " + "laulu

"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QVERIFY(r.lyrics.wordTimed); + compareWords(r.lyrics, { + { 1.0, 2.0, "keksitty", 0, true }, + { 2.1, 3.0, "laulu", 0, true }, + }); + if (QTest::currentTestFailed()) return; + + // The end is the last syllable's: if it has none, the word's + // end is inferred + r = parse("

la" + "lu " + "x

"); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 2.0, "lalu", 0, false }, + { 2.0, 2.5, "x", 0, true }, + }); + } + + // White space separates words wherever it is: line breaks and + // indentation of a laid-out file, CRLF, tabs,
, and blanks + // inside a span at either end of its text + void whitespace_separates_words() { + LyricsParseResult r = parse + ("\n

\r\n" + " yksi\r\n" + " kaksi
" + "kolme " + "neljä" + " viisi " + "kuusi\n" + "seitsemän\n" + "

\n"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 1.5, "yksi", 0, true }, + { 1.5, 2.0, "kaksi", 0, true }, + { 2.0, 2.5, "kolme", 0, true }, + { 2.5, 3.0, "neljä", 0, true }, + { 3.0, 3.5, "viisi", 0, true }, + { 3.5, 4.0, "kuusi", 0, true }, + { 4.0, 4.5, "seitsemän", 0, true }, + }); + } + + // Text straight after a word is part of it; other text in a line + // of timed words is left out, with a warning + void punctuation_appended() { + LyricsParseResult r = parse + ("

sana, " + "toinen! " + "laulu" + ", " + "loppu...

"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 1.5, "sana,", 0, true }, + { 2.0, 2.5, "toinen!", 0, true }, + { 3.0, 3.6, "laulu,", 0, true }, + { 4.0, 4.5, "loppu...", 0, true }, + }); + if (QTest::currentTestFailed()) return; + + r = parse("

\"lainaus\" ja " + "muuta " + "(sivuhuomio)

"); + QCOMPARE(r.error, QString()); + compareWords(r.lyrics, { + { 1.0, 1.5, "lainaus\"", 0, true }, + { 2.0, 2.5, "muuta", 0, true }, + }); + verifyOneWarning(r, "3 "); + } + + // A span with no begin is read through, as if it were not there + void wrappers_read_through() { + LyricsParseResult r = parse + ("

yksi " + "kak" + "si " + "" + "kolme " + "neljä!

"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 1.5, "yksi", 0, true }, + { 1.5, 2.5, "kaksi", 0, true }, + { 3.0, 3.5, "kolme", 0, true }, + { 4.0, 4.5, "neljä!", 0, true }, + }); + } + + // Background vocals, translations and romanisations are left out + // with everything in them, whatever namespace their role is in, + // with one warning that counts them. One between two words does + // not join them. + void skipped_roles_warned_once() { + LyricsParseResult r = parse + ("

yksi" + "(tausta " + "ääni)" + "kaksi" + "one two " + "kolme" + "kolme" + "k " + "tausta " + "ei " + "laulaja" + "

"); + QCOMPARE(r.error, QString()); + compareWords(r.lyrics, { + { 1.0, 1.5, "yksi", 0, true }, + { 2.5, 3.0, "kaksi", 0, true }, + { 3.0, 3.5, "kolme", 0, true }, + { 5.0, 5.5, "laulaja", 0, true }, + }); + verifyOneWarning(r, "6 "); + } + + // A

with no timed spans is one word, the whole line, white + // space made single spaces; with no begin it is left out + void line_timed_paragraphs() { + LyricsParseResult r = parse + ("

Koko rivi\n" + " yhdellä
leimalla

\n" + "

Rivi " + "käärittynä

\n" + "

Ei aikaa

\n" + "

\n" + "

Tekstiä" + "Text

\n"); + QCOMPARE(r.error, QString()); + QVERIFY(!r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 1.0, 3.0, "Koko rivi yhdellä leimalla", 0, true }, + { 4.0, 5.0, "Rivi käärittynä", 1, true }, + { 8.0, 9.0, "Tekstiä", 2, true }, + }); + QCOMPARE(r.warnings.size(), qsizetype(2)); + QVERIFY2(r.warnings[0].startsWith("1 line had no time"), + qPrintable(r.warnings.join(" | "))); + } + + // A missing end is the next word's start in the line, else the + // line's end, else inferred as the LRC parser infers it. A line of + // timed words needs no begin of its own, so one it cannot read + // takes nothing away. + void missing_ends_inferred() { + LyricsParseResult r = parse + ("

a b

" + "

c " + "d

" + "

e

" + "

rivi

" + "

kesto

" + "

viimeinen rivi

"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QVERIFY(r.lyrics.wordTimed); + compareWords(r.lyrics, { + { 1.0, 1.5, "a", 0, false }, + { 1.5, 2.5, "b", 0, false }, // up to the next line + { 2.5, 3.0, "c", 1, false }, + { 3.0, 4.0, "d", 1, true }, // the line's end + { 10.0, 10.0 + W, "e", 2, false }, + { 20.0, 30.0, "rivi", 3, false }, + { 30.0, 31.5, "kesto", 4, true }, + { 40.0, 40.0 + L, "viimeinen rivi", 5, false }, + }); + if (QTest::currentTestFailed()) return; + + r = parse("

a

"); + compareWords(r.lyrics, { { 1.0, 1.0 + W, "a", 0, false } }); + } + + // An end before its start is made the start, with a warning + void end_before_start_becomes_start() { + LyricsParseResult r = parse + ("

a " + "b

" + "

rivi

"); + QCOMPARE(r.error, QString()); + compareWords(r.lyrics, { + { 2.0, 2.0, "a", 0, true }, + { 3.0, 3.5, "b", 0, true }, + { 5.0, 5.0, "rivi", 1, true }, + }); + verifyOneWarning(r, "2 "); + } + + // Entities are decoded, then the text is cleaned as the LRC + // parser cleans it; blanks inside a word stay, as in LRC + void text_cleaned() { + QByteArray div = + "

rock & roll " + "<3 " + "äitiä " + ""'> " + " " + "ab\x7F" "c " + "x y z w " + "kaksi sanaa " + "" + QByteArray(1000, 'x') + + "

"; + LyricsParseResult r = parseTtml(ttml(div.constData())); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 1.0, 1.5, "rock & roll", 0, true }, + { 2.0, 2.5, "<3", 0, true }, + { 3.0, 3.5, "äitiä", 0, true }, + { 4.0, 4.5, "\"'>", 0, true }, + { 5.0, 5.5, "a" + text + + "

").toUtf8().constData())); + QCOMPARE(wordCount(r), 1); + QCOMPARE(r.lyrics.words[0].text, + QString(Lyrics::maxLabelLength - 1, 'z')); + } + + // in the head's metadata is the title: the first one, + // and none from a

's metadata, which is not lyrics either + void title_from_head_metadata() { + LyricsParseResult r = parseTtml + (ttml("

Ei" + "sana

", + "" + "Kesäyön &\n testi" + "Toinen")); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString::fromUtf8("Kesäyön & testi")); + QCOMPARE(r.lyrics.artist, QString()); + compareWords(r.lyrics, { { 1.0, 2.0, "sana", 0, true } }); + + r = parse("

sana

"); + QCOMPARE(r.lyrics.title, QString()); + } + + // Nothing usable is an error, and gives no lyrics + void no_words_is_an_error() { + const QByteArray files[] = { + ttml(""), + ttml("

ilman aikaa

", "Nimi"), + ttml("

"), + ttml("

" + "tausta

"), + ttml("

sana

"), + QByteArray("
Hei
"), + QByteArray(""), + QByteArray(), + }; + for (const QByteArray &file : files) { + LyricsParseResult r = parseTtml(file); + QVERIFY2(!r.error.isEmpty(), file.constData()); + QVERIFY(r.lyrics.isEmpty()); + QCOMPARE(r.lyrics.title, QString()); + } + } + + // Malformed XML is an error that names the line, even when words + // were read before it + void malformed_xml_names_the_line() { + LyricsParseResult r = parseTtml + ("\n\n
\n" + "

a

\n" + "
\n\n\n"); + QVERIFY(r.lyrics.isEmpty()); + QVERIFY2(r.error.contains("line 4"), qPrintable(r.error)); + + QByteArray whole = fixture("moises-exporter-words.ttml"); + QVERIFY(!whole.isEmpty()); + r = parseTtml(whole.left(whole.indexOf("itunes:key=\"L3\""))); + QVERIFY(r.lyrics.isEmpty()); + QVERIFY2(r.error.contains("line 12"), qPrintable(r.error)); + + // An entity XML itself does not have + r = parse("

a b

"); + QVERIFY(r.lyrics.isEmpty()); + QVERIFY2(r.error.contains("line 2"), qPrintable(r.error)); + + // Bytes that are not the UTF-8 it says it is. The reader + // decodes ahead of where it reads, so the line it names is not + // the one they are on. + r = parseTtml("\n" + "

" + "H\xE4m\xE4r\xE4

"); + QVERIFY(r.lyrics.isEmpty()); + QVERIFY2(r.error.contains("line "), qPrintable(r.error)); + } + + // A file with a DTD is refused before anything in it is read: it + // is how entity tricks get in, and TTML has none + void dtd_refused() { + const char *files[] = { + "\n" + "\n" + " ]>\n" + "

&b;

" + "
\n", + + "\n" + "

sana

" + "
\n", + + "\n" + "

sana

" + "
\n", + }; + for (const char *file : files) { + LyricsParseResult r = parseTtml(file); + QVERIFY2(r.error.contains("DOCTYPE"), qPrintable(r.error)); + QVERIFY2(r.lyrics.isEmpty(), qPrintable(describe(r.lyrics))); + } + } + + // Exactly the limit is read; one byte more is not + void too_big_refused() { + QByteArray big = ttml("

sana

"); + big += QByteArray(Lyrics::maxFileBytes - big.size(), '\n'); + LyricsParseResult r = parseTtml(big); + QCOMPARE(r.error, QString()); + QCOMPARE(wordCount(r), 1); + + big += '\n'; + r = parseTtml(big); + QVERIFY(!r.error.isEmpty()); + QVERIFY(r.lyrics.isEmpty()); + QCOMPARE(parseLyrics(big).error, r.error); + } + + // Lines in time order by their first start, those that start + // together in file order; the words of a line in time order + void lines_sorted_by_first_start() { + LyricsParseResult r = parse + ("

kolmas

" + "

toinen " + "ensin

" + "

eka

" + "

toka

"); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { + { 0.0, 1.0, "eka", 0, true }, + { 0.0, 0.5, "toka", 1, true }, + { 5.0, 5.5, "ensin", 2, true }, + { 5.5, 6.0, "toinen", 2, true }, + { 20.0, 21.0, "kolmas", 3, true }, + }); + } + + // Files without the TTML namespace, with a prefix for it, and + // with the time attributes in another namespace + void names_matched_by_local_name() { + const char *files[] = { + "

sana" + "

", + + "" + "sana" + "", + + "" + "

sana" + "

", + }; + for (const char *file : files) { + LyricsParseResult r = parseTtml(file); + QVERIFY2(r.error.isEmpty(), qPrintable(r.error)); + QCOMPARE(r.warnings, QStringList()); + compareWords(r.lyrics, { { 1.0, 2.0, "sana", 0, true } }); + if (QTest::currentTestFailed()) return; + } + } + + // What the exporter's own TTML code writes in word mode, from an + // invented lyrics.json whose words were, in seconds: + // Tämä 0.52-0.8, on 0.84-1.02, keksitty 1.1-1.9 (syllables kek + // 1.1-1.35, sit 1.35-1.62, ty 1.62-1.9), laulu 2.0-2.7, "," + // 2.7-2.75 | Yö 4.64-4.9, on 4.92-5.1, "hämärä," 5.3-6.2, kuu + // 6.25-6.8, "nousee." 6.8-7.5 | Nyt 61.25-61.6, se 61.7-61.9, + // "loppuu!" 62.0-63.5 + // The exporter glues a word that starts with punctuation to the + // one before it, so the comma joins "laulu" as a syllable would. + void exporter_word_fixture() { + QByteArray bytes = fixture("moises-exporter-words.ttml"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseTtml(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QCOMPARE(r.lyrics.title, QString()); + QVERIFY(r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 0.52, 0.8, "Tämä", 0, true }, + { 0.84, 1.02, "on", 0, true }, + { 1.1, 1.9, "keksitty", 0, true }, + { 2.0, 2.75, "laulu,", 0, true }, + { 4.64, 4.9, "Yö", 1, true }, + { 4.92, 5.1, "on", 1, true }, + { 5.3, 6.2, "hämärä,", 1, true }, + { 6.25, 6.8, "kuu", 1, true }, + { 6.8, 7.5, "nousee.", 1, true }, + { 61.25, 61.6, "Nyt", 2, true }, + { 61.7, 61.9, "se", 2, true }, + { 62.0, 63.5, "loppuu!", 2, true }, + }); + } + + // The same in line mode: each line one word, from its first + // word's start to its last word's end + void exporter_line_fixture() { + QByteArray bytes = fixture("moises-exporter-lines.ttml"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseTtml(bytes); + QCOMPARE(r.error, QString()); + QCOMPARE(r.warnings, QStringList()); + QVERIFY(!r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 0.52, 2.75, "Tämä on keksitty laulu,", 0, true }, + { 4.64, 7.5, "Yö on hämärä, kuu nousee.", 1, true }, + { 61.25, 63.5, "Nyt se loppuu!", 2, true }, + }); + } + + // As AMLL TTML Tool writes it: mm:ss.mmm, two agents, syllables, + // a background vocal and translations, which are skipped + void amll_fixture() { + QByteArray bytes = fixture("amll-style.ttml"); + QVERIFY(!bytes.isEmpty()); + LyricsParseResult r = parseTtml(bytes); + QCOMPARE(r.error, QString()); + verifyOneWarning(r, "3 "); + QCOMPARE(r.lyrics.title, QString()); + QVERIFY(r.lyrics.wordTimed); + QCOMPARE(r.lyrics.lineCount(), 3); + compareWords(r.lyrics, { + { 3.2, 3.55, "Pöllö", 0, true }, + { 3.6, 4.05, "huhuilee", 0, true }, + { 4.3, 4.6, "ja", 0, true }, + { 4.6, 5.1, "kuu", 0, true }, + { 7.0, 7.65, "nousee", 1, true }, + { 7.8, 8.5, "metsän", 1, true }, + { 8.6, 9.4, "ylle", 1, true }, + { 65.0, 65.8, "Hiljaa", 2, true }, + { 65.9, 69.5, "hiljaa", 2, true }, + }); + } + + // TTML if the first character that is not blank, after a BOM, is + // '<'; LRC otherwise + void parse_lyrics_chooses_the_format() { + const QByteArray bom("\xEF\xBB\xBF"); + const QByteArray lrc = fixture("moises-exporter-words.lrc"); + const QByteArray tt = fixture("moises-exporter-words.ttml"); + QVERIFY(!lrc.isEmpty() && !tt.isEmpty()); + + const Lyrics fromLrc = parseLrc(lrc).lyrics; + const Lyrics fromTtml = parseTtml(tt).lyrics; + QVERIFY(!fromLrc.isEmpty() && !fromTtml.isEmpty()); + + auto same = [](const Lyrics &a, const Lyrics &b) { + if (a.words.size() != b.words.size()) return false; + for (int i = 0; i < a.words.size(); ++i) { + if (a.words[i].text != b.words[i].text || + a.words[i].start != b.words[i].start || + a.words[i].end != b.words[i].end) return false; + } + return true; + }; + + QVERIFY(same(parseLyrics(lrc).lyrics, fromLrc)); + QVERIFY(same(parseLyrics(bom + lrc).lyrics, fromLrc)); + QVERIFY(same(parseLyrics(" \r\n\t" + lrc).lyrics, fromLrc)); + QVERIFY(same(parseLyrics(tt).lyrics, fromTtml)); + QVERIFY(same(parseLyrics(bom + tt).lyrics, fromTtml)); + + // Blanks before the root are XML too, when there is no + // declaration, which has to come first + QByteArray noDeclaration = tt.mid(tt.indexOf("").error, parseTtml("").error); + QCOMPARE(parseLyrics("sana").error, parseLrc("sana").error); + QVERIFY(parseTtml("").error != parseLrc("sana").error); + QVERIFY(!parseLyrics(QByteArray()).error.isEmpty()); + } + + // The writer's layout, as the exporter's: one agent, a

per + // line and a per word with a space between, m:ss.mmm + void write_format() { + Lyrics lyrics; + lyrics.title = "Testi & laulu"; + lyrics.artist = "Ei kirjoiteta"; + auto word = [](double start, double end, const char *text, int line) { + LyricWord w; + w.start = start; + w.end = end; + w.text = QString::fromUtf8(text); + w.line = line; + return w; + }; + lyrics.words = { + word(0.5, 1.0, "Yö", 0), + word(1.0, 1.25, "on", 0), + word(65.4321, 66.0, "", 1), + word(66.0, 70.0006, "\"kaksi sanaa\"", 1), + }; + const QByteArray expected = + "\n" + "\n" + " \n" + " \n" + " Testi & laulu\n" + " \n" + " \n" + " \n" + " \n" + "

\n" + "

" + "Yö on" + "

\n" + "

" + "<kaunis> "kaksi sanaa"

\n" + "
\n" + " \n" + "\n"; + const QByteArray written = writeTtml(lyrics); + QVERIFY2(written == expected, + qPrintable(QString("written:\n%1\nexpected:\n%2") + .arg(QString::fromUtf8(written)) + .arg(QString::fromUtf8(expected)))); + } + + // What is written reads back as the same words, texts and lines, + // to half a millisecond: from LRC, from TTML, and awkward cases + void round_trip() { + const char *lrcFixtures[] = { + "moises-exporter-words.lrc", "moises-exporter-lines.lrc", + "lrc-with-ends.lrc", + }; + for (const char *name : lrcFixtures) { + LyricsParseResult r = parseLrc(fixture(name)); + QVERIFY2(r.error.isEmpty(), name); + verifyRoundTrip(r.lyrics, name); + if (QTest::currentTestFailed()) return; + } + const char *ttmlFixtures[] = { + "moises-exporter-words.ttml", "moises-exporter-lines.ttml", + "amll-style.ttml", + }; + for (const char *name : ttmlFixtures) { + LyricsParseResult r = parseTtml(fixture(name)); + QVERIFY2(r.error.isEmpty(), name); + verifyRoundTrip(r.lyrics, name); + if (QTest::currentTestFailed()) return; + } + + // Markup characters, blanks inside a word, a character that + // needs two, punctuation as a word of its own, words of no + // length and at the same time, times off the millisecond and + // past an hour + Lyrics lyrics; + lyrics.title = QString::fromUtf8("Hämärä & \"päivä\""); + auto word = [](double start, double end, const QString &text, + int line) { + LyricWord w; + w.start = start; + w.end = end; + w.text = text; + w.line = line; + return w; + }; + const char32_t clef[] = { 0x1D11E }; + lyrics.words = { + word(0.0, 0.0, "a&b", 0), + word(0.0, 1.2344, "<3", 0), + word(1.2346, 2.0, "koko rivi \"lainaus\" 'x'", 0), + word(2.0, 2.0, ",", 0), + word(2.5, 3.0, QString::fromUtf8("äiti ") + + QString::fromUcs4(clef, 1), 1), + word(2.9, 3.5, "]]>", 1), + word(3725.4996, 3726.0004, "tunti", 2), + word(3726.0004, 3726.0004, "yli", 2), + }; + verifyRoundTrip(lyrics, "awkward"); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index df63a235..65f102c5 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -21,6 +21,7 @@ #include "TestTakesFile.h" #include "TestTakeTiming.h" #include "TestLyrics.h" +#include "TestLyricsTtml.h" #include "RunSuite.h" @@ -105,6 +106,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLyricsTtml t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index afe58372..c4642d39 100644 --- a/meson.build +++ b/meson.build @@ -1096,6 +1096,7 @@ tony_entry_files = [ tony_core_files = [ 'main/Coverage.cpp', 'main/Lyrics.cpp', + 'main/LyricsTtml.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', @@ -1360,6 +1361,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', 'main/test/TestLyrics.h', + 'main/test/TestLyricsTtml.h', ]) # Where the suites find the files in testdata/. Forward slashes: a diff --git a/testdata/lyrics/amll-style.ttml b/testdata/lyrics/amll-style.ttml new file mode 100644 index 00000000..88e8dacb --- /dev/null +++ b/testdata/lyrics/amll-style.ttml @@ -0,0 +1,5 @@ +
+

Pöllö huhuilee ja kuuAn owl hoots and the moon(taustalla huuto)

+

nousee metsän yllerises over the forest

+

Hiljaa hiljaa

+
 diff --git a/testdata/lyrics/moises-exporter-lines.ttml b/testdata/lyrics/moises-exporter-lines.ttml new file mode 100644 index 00000000..e91685b8 --- /dev/null +++ b/testdata/lyrics/moises-exporter-lines.ttml @@ -0,0 +1,15 @@ + + + + + + + + +
+

Tämä on keksitty laulu,

+

Yö on hämärä, kuu nousee.

+

Nyt se loppuu!

+
+ + diff --git a/testdata/lyrics/moises-exporter-words.ttml b/testdata/lyrics/moises-exporter-words.ttml new file mode 100644 index 00000000..3aed05da --- /dev/null +++ b/testdata/lyrics/moises-exporter-words.ttml @@ -0,0 +1,15 @@ + + + + + + + + +
+

Tämä on keksitty laulu,

+

Yö on hämärä, kuu nousee.

+

Nyt se loppuu!

+
+ + From 2a306fb896f474ed4275acfb4a13fe90fd242f7c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:11:56 +0000 Subject: [PATCH 112/275] test: waiting for a ranged analysis waits for the range to be done initialAnalysisCompleted() also comes from layerCompletionChanged() when the layers reach 100%, which can happen before the ranged merge, so waitForRange() now and then saw the signal with the range still being analysed and failed ranged_keeps_the_end_of_a_note_past_the_run (once in several full runs on Linux; it passed alone six times). It now waits up to 30 s for the analyser to say the range is done, and still fails if it never does. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- main/test/TestSingingAnalysis.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 67b4a48d..acf03fea 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -217,12 +217,16 @@ class TestSingingAnalysis : public QObject QVERIFY(noteEvents(analyser).empty()); } - // Wait for a ranged analysis to be merged + // Wait for a ranged analysis to be merged. initialAnalysisCompleted() + // also comes from layerCompletionChanged() whenever the layers reach + // 100%, which can be before the merge, so the signal alone does not + // say the range is done: wait for the analyser to say so as well void waitForRange(Analyser &analyser, QSignalSpy &done) { QVERIFY2(done.count() > 0 || done.wait(30000), "the ranged analysis did not complete within 30 seconds"); - QVERIFY2(!analyser.isAnalysingRange(), - "the analyser still says a range is being analysed"); + QTRY_VERIFY2_WITH_TIMEOUT(!analyser.isAnalysingRange(), + "the analyser still says a range is being " + "analysed", 30000); } // Nothing of a ranged analysis may be left behind in the document From 8524d5f33ed3cc7ca0d102b48202878806810518 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:11:56 +0000 Subject: [PATCH 113/275] feat: the audio check plays its reference centred and at -12 dBFS The reference is normalised to full scale as it is read, the analyser pans it hard left and sonifies its pitch on the right, so as built the check played 0 dBFS sweeps into one earcup with a synth tone in the other. For the check's own session the reference now plays centred at the planned level and the sonification is silent, set on the play parameters, never through Analyser::setAudible(), which writes the shared settings; the toolbar's level control is moved with its signals blocked for the same reason. The user's sessions play as before. The runner reports progress, writes its reference under a new name each run so that Check Again never rewrites a file the open session holds, and takes the reported latencies in seconds from the take, as the take path works them out. Skipping the gain, leaving the check's playback in place for the next file, and leaving the sonification audible were each seen to fail a test. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green (TestAudioCheck 12, TestRecordWorkflow 96, TestSingingAnalysis 20, TestSingingDocument 12). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 13 ++ docs/calibrate-audio.md | 14 +- main/AudioCheckRunner.cpp | 177 +++++++++++++-- main/AudioCheckRunner.h | 99 +++++++-- main/LatencyCalibration.cpp | 2 + main/LatencyCalibration.h | 9 +- main/LatencyUtils.h | 18 +- main/MainWindow.cpp | 4 +- main/MainWindow.h | 5 +- main/test/TestAudioCheck.h | 325 +++++++++++++++++++++++++++- main/test/TestLatencyCalibration.h | 9 +- main/test/TestRecordWorkflow.h | 15 +- 12 files changed, 625 insertions(+), 65 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 16b38835..b3d0c088 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -509,3 +509,16 @@ The next phase must know: - The runner's `reportedOutputLatency` still divides by the reference's rate. It matches the take path at 44.1 kHz, the only rate that is stored. - `storeMeasuredLatency()` reads the device from the Preferences when "Use this latency" is pressed. If B4's non-modal dialog lets the device change in between, the figure is stored under the new device. Left open: `computeRecordingLatency()` is unused outside `TestLatencyShift`. The svapp fork could add `AudioCallbackRecordTarget::getRecordSampleRate()` so that `latencyInUse()` knows the rate before the first take. + +### Phase B3 — 2026-09-26 +Built: `AudioCheckRunner::setPlayback()`: on the play parameters of the check session's own models, the reference audible, centred, gain 10^(kPeakDbfs/20) (1 if the normalise preference is off); its pitch and notes muted. Applied once the reference is open and again before every punch-in. Public `Step`, `Progress {step, punchIn, punchIns, secondsLeft}`, signal `progress()`. `referenceDirectory()`, `nextReferencePath(dir, inUse)`. `LatencyCalibration::InUse::reportedOutput/Input` (seconds, whichever source won); `TakeLatency::reportedOutput/Input` are now those seconds, and `end()` copies them. App tests `check_plays_the_reference_centred_and_quiet` (13 s), `check_leaves_the_next_session_alone` (4 s), `check_reference_gets_a_file_of_its_own`; core `round_trip_in_use` extended. Four existing tests now compare the reported pair in seconds. +Choices / deviations: +- The toolbar's reference level control (`m_audioLPW`) answers a gain between its notches (−12 dB lies between −11.25 and −20) by emitting the nearest; `audioGainChanged()` then sets it through `Analyser::setGain()`/`setAudible()`, writing `Analyser/audible-0`. `setPlayback()` moves the control first under a `QSignalBlocker`. Without it both new session tests fail on the settings. +- Reference files `calibrate-audio-reference-N.wav`: every such file in the directory but the open session's main model file is removed, then the lowest free N is taken, so names alternate 1, 2. A timestamp per run would add a dead Recent Files entry per run (`RecentFiles` has no remove, and keeps 20). +- The check session keeps its playback after the run. The reference is made audible even where the user's settings mute it. +- `progress()` comes from `poll()` only, never from inside `start()`/`cancel()`: on each step change, and each whole second less while recording. `secondsLeft` counts recording to come (min(1 s, start) + range per take), not analyses. No metatype: direct connections only. +- "Afterwards" is tested after a cancelled run, against the same file opened before the check: the settings alone do not say how a session plays (below). +The next phase must know: +- Pre-existing, not fixed: (a) in the first file of a window the same feedback moves the pitch and notes gain from 0.5 to 0.562 and forces both audible, writing the settings; (b) `audible-0` is overridden at load by `audible-3`: the spectrogram is a layer on the reference's model, so it plays whenever `audible-3` is true. +- B4: add a few seconds per analysis to `secondsLeft` for a rough total. +Left open: the default reference path is exercised only through `nextReferencePath()`; the tests name their own file. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 8ded2211..b46272d3 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -291,7 +291,7 @@ marked "Done" when it is committed. `recordingStarted()`, staleness). This was step 3 below; it moved up because the dialog needs it. Done. - **B3** The check's playback: the reference centred at −12 dBFS, the sonification - silent, and a progress signal. + silent, and a progress signal. Done. - **B4** The Calibrate Audio dialog and menu entry. **Then you run it on your PC.** Its numbers settle three things: how wrong the @@ -335,7 +335,9 @@ could convert. The button then shows the fix working on each device. - **Loudness.** The sweeps are −12 dBFS with earcups off the ears; the dialog says so before starting. *Found in B1:* not as played. Tony normalises every audio file to full scale as it reads it (`Preferences::setNormaliseAudio(true)`), so the reference - plays at 0 dBFS, in the left channel only (§11). + plays at 0 dBFS, in the left channel only (§11). *Since B3* the check's session + plays it centred, with a play gain that brings it back to −12 dBFS, and its + sonification silent. Other sessions play as before. - **Cursor versus dots.** The cursor subtracts the *reported* output latency. With a measured round trip the dots move to the right place and may sit off the cursor. Item 8's number will show how much. Fixing it needs the round trip split between output @@ -387,7 +389,9 @@ Checked on 2026-09-25, so that phases do not re-derive them. called. - **Where the reference is heard** (found in B1). `Analyser` pans the reference hard left and its pitch and notes sonification hard right, so only the left earcup - carries the sweeps; the right one carries the synth. + carries the sweeps; the right one carries the synth. *Since B3* the check's own + session sets its models' play parameters instead: the reference centred at + −12 dBFS, the sonification muted. - **bqaudioio `PortAudioIO`** (upstream, not a fork): - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; - input goes to the record target **before** output is asked for, in the same @@ -429,8 +433,8 @@ Checked on 2026-09-25, so that phases do not re-derive them. - **Fake device.** `FakeAudioIO::Config::loopback` adds the output to the input `inputDelay` frames late; `TestAudioCheck` uses it. The reported latencies are independent of the real delay. `TestMainWindow::createAudioIO()` installs the fake. - Its output is the mean of the channels, so the hard-left reference loops back at - half level. + Its output is the mean of the channels, so a hard-left reference loops back at + half level; the check's, centred since B3, at the level it is played at. - **Menus.** The Playback menu is built in `MainWindow::setupToolbars()` (`m_playbackMenu`). The audio device submenus are there too. - **Build types.** `build.bat` uses `debugoptimized`; `meson.build` defaults to diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index 1fc9aa67..3cfb7771 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -19,19 +19,28 @@ #include "SingingTakes.h" #include "audio/AudioCallbackRecordTarget.h" +#include "base/PlayParameterRepository.h" +#include "base/PlayParameters.h" +#include "base/Preferences.h" #include "base/Selection.h" #include "data/fileio/FileSource.h" #include "data/fileio/WavFileReader.h" #include "data/fileio/WavFileWriter.h" +#include "data/model/ReadOnlyWaveFileModel.h" #include "data/model/WaveFileModel.h" +#include "layer/Layer.h" #include "transform/ModelTransformerFactory.h" #include "view/ViewManager.h" +#include "widgets/LevelPanToolButton.h" #include +#include #include +#include #include #include +#include #include #include @@ -81,11 +90,36 @@ AudioCheckRunner::~AudioCheckRunner() } QString -AudioCheckRunner::defaultReferencePath() +AudioCheckRunner::referenceDirectory() { - QString dir = - QStandardPaths::writableLocation(QStandardPaths::AppDataLocation); - return QDir(dir).filePath("calibrate-audio-reference.wav"); + return QStandardPaths::writableLocation(QStandardPaths::AppDataLocation); +} + +QString +AudioCheckRunner::nextReferencePath(QString directory, QString inUse) +{ + const QString stem = "calibrate-audio-reference"; + const QDir dir(directory); + const QFileInfo held(inUse); + + // One that cannot be removed (another program has it open, say) + // stays, and its number is passed over below + const QStringList names = + dir.entryList(QStringList() << stem + "*.wav", QDir::Files); + for (const QString &name : names) { + const QString path = dir.absoluteFilePath(name); + if (inUse != "" && QFileInfo(path) == held) continue; + if (!QFile::remove(path)) { + cerr << "AudioCheckRunner: an earlier reference could not be " + << "removed: " << path << endl; + } + } + + for (int n = 1; ; ++n) { + const QString path = + dir.absoluteFilePath(QString("%1-%2.wav").arg(stem).arg(n)); + if (!QFileInfo::exists(path)) return path; + } } bool @@ -108,17 +142,15 @@ AudioCheckRunner::start(const Plan &plan) } m_plan = plan; - if (m_plan.referencePath == "") { - m_plan.referencePath = defaultReferencePath(); - } m_result = AudioCheckResult(); + m_reported = Progress(); m_punchIns = punchIns; m_starts.clear(); m_ends.clear(); m_punchIn = 0; cerr << "AudioCheckRunner::start: " << m_punchIns.size() - << " punch-ins against " << m_plan.referencePath << endl; + << " punch-ins of " << m_plan.eventsEach << " events" << endl; // Nothing is done before the first poll, so that however the run // ends, the caller hears of it through finished() and never from @@ -157,6 +189,11 @@ AudioCheckRunner::poll() if (m_inPoll) return; m_inPoll = true; + // Before the step, so that the first poll says the reference is + // being opened before it is, and after it, so that a step begun here + // is reported as it begins and not a poll later + reportProgress(); + switch (m_step) { case Step::Idle: @@ -200,6 +237,8 @@ AudioCheckRunner::poll() break; } + reportProgress(); + m_inPoll = false; } @@ -217,10 +256,21 @@ AudioCheckRunner::openReference() } if (m_step != Step::OpeningReference) return; - // Written afresh every time, over the one a check before wrote: it - // is made from the plan, and a session opened from it has read it - // whole already + // Made from the plan and written afresh every time, to a file of its + // own: the session open now may be the check before, and on Windows + // the file it plays cannot be written over. A plan that names a + // file has that one written over QString path = m_plan.referencePath; + if (path == "") { + QString inUse; + if (auto model = std::dynamic_pointer_cast + (m_window->getMainModel())) { + inUse = model->getLocalFilename(); + } + path = nextReferencePath(referenceDirectory(), inUse); + m_plan.referencePath = path; + } + cerr << "AudioCheckRunner: writing the reference to " << path << endl; QDir().mkpath(QFileInfo(path).absolutePath()); vector samples = LatencyCheck::generate(m_plan.layout); WavFileWriter writer(path, m_plan.layout.rate, 1, @@ -257,6 +307,9 @@ AudioCheckRunner::openReference() double(m_ends.back()) / rate); } + // Its layers were made as it opened + setPlayback(); + setStep(Step::AnalysingReference, kReferenceTimeoutMs); } @@ -284,6 +337,10 @@ AudioCheckRunner::startPunchIn() viewManager->clearSelections(); viewManager->addSelectionQuietly(Selection(from, to)); + // Again for every take: whatever has happened since the last, the + // takes are all recorded with the same playback + setPlayback(); + m_window->m_audioCheckTakes = true; m_window->record(); if (m_step != step) return; @@ -353,6 +410,93 @@ AudioCheckRunner::judge() end(""); } +void +AudioCheckRunner::setPlayback() +{ + const float gain = float(referenceGain()); + + // The reference's model: its waveform layer, and any other layer on + // it, has these parameters. Audible even if the user has muted the + // reference in their own sessions: the check has to hear it + if (auto params = PlayParameterRepository::getInstance() + ->getPlayParameters(m_window->getMainModelId().untyped)) { + params->setPlayAudible(true); + params->setPlayPan(0.f); + params->setPlayGain(gain); + } + + Analyser *reference = m_window->m_analyser; + for (Analyser::Component c : { Analyser::PitchTrack, Analyser::Notes }) { + Layer *layer = reference ? reference->getLayer(c) : nullptr; + if (!layer) continue; + if (auto params = layer->getPlayParameters()) { + params->setPlayAudible(false); + } + } + + // The toolbar's level control shows the reference's gain. Given one + // between its notches, it moves to the nearest and says so, and the + // window sets that gain through Analyser::setGain() and setAudible(), + // which write the shared settings. Moved here first without a word, + // it has nothing to say when the window shows the gain + if (LevelPanToolButton *control = m_window->m_audioLPW) { + QSignalBlocker quiet(control); + control->setLevel(gain); + control->setPan(0.f); + } + m_window->updateLayerStatuses(); +} + +double +AudioCheckRunner::referenceGain() +{ + // Made with its peak at kPeakDbfs, and read normalised to full scale, + // as MainWindow has every audio file read + if (!Preferences::getInstance()->getNormaliseAudio()) return 1.0; + return std::pow(10.0, LatencyCheck::kPeakDbfs / 20.0); +} + +AudioCheckRunner::Progress +AudioCheckRunner::currentProgress() const +{ + Progress p; + p.step = m_step; + p.punchIns = int(m_punchIns.size()); + if (m_step == Step::Recording || m_step == Step::AnalysingTake) { + p.punchIn = m_punchIn + 1; + } + + // The lead-in of a take to come is the whole of the check's, unless + // its range starts sooner than that + for (int i = m_punchIn; i < int(m_punchIns.size()); ++i) { + const LatencyCheck::PunchIn &range = m_punchIns[i]; + double seconds = std::min(kPreRollSeconds, range.start) + + (range.end - range.start); + if (i == m_punchIn) { + if (m_step == Step::AnalysingTake) continue; + if (m_step == Step::Recording) { + seconds = std::max(0.0, seconds - + double(m_stepClock.elapsed()) / 1000.0); + } + } + p.secondsLeft += seconds; + } + return p; +} + +void +AudioCheckRunner::reportProgress() +{ + if (m_step == Step::Idle) return; + const Progress p = currentProgress(); + if (p.step == m_reported.step && p.punchIn == m_reported.punchIn && + std::ceil(p.secondsLeft) == std::ceil(m_reported.secondsLeft)) { + return; + } + m_reported = p; + emit progress(p); +} + void AudioCheckRunner::setStep(Step step, qint64 limitMs) { @@ -388,15 +532,12 @@ AudioCheckRunner::end(QString failure) m_result.failure = failure; if (!m_result.takes.empty()) { - // Each at the rate it counts in (see TakeLatency) + // The reported pair as the take path worked it out, and compares + // a stored figure's fingerprint with const TakeLatency &first = m_result.takes.front(); m_result.usedRoundTrip = first.recordingSeconds(first.roundTrip); - m_result.reportedInputLatency = - first.recordingSeconds(first.reportedInput); - if (m_result.referenceRate > 0) { - m_result.reportedOutputLatency = - double(first.reportedOutput) / m_result.referenceRate; - } + m_result.reportedOutputLatency = first.reportedOutput; + m_result.reportedInputLatency = first.reportedInput; m_result.recordingRate = first.recordingRate; m_result.rateMismatch = m_result.recordingRate > 0 && m_result.referenceRate > 0 && diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index 6d0cbc80..d23b7ccd 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -44,9 +44,9 @@ struct AudioCheckResult /// What each punch-in was placed with, in the order recorded std::vector takes; - /// The round trip the first punch-in was placed with, as frames of - /// the recording, and the two latencies the device reported, each at - /// the rate it counts in (see TakeLatency) + /// The round trip the first punch-in was placed with, and the two + /// latencies the device reported then, as the take path had them + /// (see TakeLatency) double usedRoundTrip; double reportedOutputLatency; double reportedInputLatency; @@ -94,12 +94,23 @@ struct AudioCheckResult * take stops itself at the end of the selection, through the same path * as the Stop button. * + * The check's session plays the reference centred and at the level it + * was made at, LatencyCheck::kPeakDbfs, and leaves the pitch and notes + * sonification silent: an earcup is held to the microphone, and Tony + * otherwise plays the reference in the left channel only, normalised + * to full scale, with the sonification in the right. This is set on + * the play parameters of the session's own models, never through + * Analyser::setAudible() and the like, which write the settings every + * session reads. It stays so after the run, as the session does, and + * a session opened afterwards plays as before. + * * Driven by a polling timer, like MainWindow's own take polling, and * never by a nested event loop: this runs in every build, and the * window can be closed at any moment. MainWindow owns it, deletes it * first thing in its destructor, and tells it when the session closes. - * A friend of MainWindow: it drives the window's take path, and reads - * what the take was placed with, but changes nothing else there. + * A friend of MainWindow: it drives the window's take path, reads what + * the take was placed with, and sets the playback of the session it + * opened, but changes nothing else there. */ class AudioCheckRunner : public QObject { @@ -126,17 +137,59 @@ class AudioCheckRunner : public QObject int eventsEach; /// Where the reference is written, over whatever is there; "" - /// for defaultReferencePath() + /// for a new file in referenceDirectory() (nextReferencePath()) QString referencePath; Plan() : punchIns(0), eventsEach(0) { } }; + /// The steps of a run, in order; the last two come once for each + /// punch-in + enum class Step { + Idle, + OpeningReference, + AnalysingReference, + Recording, + AnalysingTake + }; + + /// How far a run has got + struct Progress { + Step step; + + /// Punch-in punchIn of punchIns: the one being recorded, or + /// whose take is being analysed, counting from 1; 0 before the + /// first + int punchIn; + int punchIns; + + /// Seconds of recording still to come: the rest of the take + /// being recorded, and the lead-in and range of each one after + /// it. The waits for the analyses between them are not in it: + /// their length is not known + double secondsLeft; + + Progress() : step(Step::Idle), punchIn(0), punchIns(0), + secondsLeft(0) { } + }; + explicit AudioCheckRunner(MainWindow *window); virtual ~AudioCheckRunner(); - /// A file in the application's data directory - static QString defaultReferencePath(); + /// Where the reference is written unless the plan names a file: + /// the application's data directory + static QString referenceDirectory(); + + /** + * A file in the directory to write the next reference to, never + * the one inUse names: that is the session open now, perhaps the + * check before, and on Windows a file that is open cannot be + * written over. The references in the directory that inUse does + * not name are removed first, as no session holds them; and the + * lowest free number is taken, so the names go 1, 2, 1, 2 and + * Recent Files, where each one opened is listed, gets two at most. + */ + static QString nextReferencePath(QString directory, QString inUse); /** * Begin a run. False, with nothing started, if one is running @@ -159,21 +212,21 @@ class AudioCheckRunner : public QObject signals: void finished(const AudioCheckResult &result); -private: - enum class Step { - Idle, - OpeningReference, - AnalysingReference, - Recording, - AnalysingTake - }; + /// When a step begins, and each time the whole seconds left go + /// down while a take is recorded. Never from inside start() or + /// cancel() + void progress(const AudioCheckRunner::Progress &state); +private: MainWindow *m_window; QTimer *m_timer; Step m_step; Plan m_plan; AudioCheckResult m_result; + /// The last progress reported + Progress m_reported; + /// The punch-ins in seconds, from the plan, until the reference is /// open; from then on as the takes record them, in whole frames of /// the session @@ -198,9 +251,23 @@ class AudioCheckRunner : public QObject void takeStopped(); void judge(); + /// The check session's playback, set on the play parameters of the + /// reference and of its pitch and notes: see the class comment + void setPlayback(); + + /// The gain that brings the reference, as the session's model + /// has it, down to the level it was made at + static double referenceGain(); + void setStep(Step step, qint64 limitMs); bool stepTimedOut() const; + Progress currentProgress() const; + + /// Emit progress() if the step, the punch-in or the whole seconds + /// left have changed since it was last emitted + void reportProgress(); + /// Stop a take the check is recording, through the Stop path void stopTake(); diff --git a/main/LatencyCalibration.cpp b/main/LatencyCalibration.cpp index 4e6f6433..8d9feaf5 100644 --- a/main/LatencyCalibration.cpp +++ b/main/LatencyCalibration.cpp @@ -162,6 +162,8 @@ roundTripInUse(const Figure *stored, double reportedOutput, double reportedInput) { InUse inUse; + inUse.reportedOutput = reportedOutput; + inUse.reportedInput = reportedInput; if (stored && !isStale(*stored, reportedOutput, reportedInput)) { inUse.source = Source::Measured; inUse.roundTrip = stored->roundTrip; diff --git a/main/LatencyCalibration.h b/main/LatencyCalibration.h index aabebf8c..4cd349cb 100644 --- a/main/LatencyCalibration.h +++ b/main/LatencyCalibration.h @@ -113,7 +113,14 @@ namespace LatencyCalibration /// A figure was stored for the key, but it is stale bool stale; - InUse() : source(Source::Reported), roundTrip(0), stale(false) { } + /// The latencies the device reports now, which the choice was + /// made against, whichever source won: a figure measured now is + /// stored with these as its fingerprint + double reportedOutput; + double reportedInput; + + InUse() : source(Source::Reported), roundTrip(0), stale(false), + reportedOutput(0), reportedInput(0) { } }; /** diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index 7e652a7b..146a14cf 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -42,21 +42,19 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, * 0 for a take made without the reference playing, which is placed with * none. * - * In frames as the window has them. The round trip is taken off the - * recording, and the input latency is the device's, so both count - * frames of the recording; but the play source reports the output - * latency in frames of the session, converted when it resamples to the - * device (unless the device was opened before the session had a rate; - * see MainWindow::roundTripAt()). The two kinds differ only when the - * device's rate is not the session's; the round trip is worked out in - * seconds for that reason. + * The round trip is in frames of the recording, which it is taken off. + * The two reported latencies are in seconds, as MainWindow::roundTripAt() + * works them out from the frames each counts in (the play source's, the + * device's), and as it compares them with a stored figure's + * fingerprint: in frames they count at two different rates when the + * device's rate is not the session's. */ struct TakeLatency { sv::sv_frame_t roundTrip; bool measured; - sv::sv_frame_t reportedOutput; - sv::sv_frame_t reportedInput; + double reportedOutput; + double reportedInput; sv::sv_samplerate_t recordingRate; TakeLatency() : roundTrip(0), measured(false), reportedOutput(0), diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index f0e0fe6d..59e86563 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -4225,8 +4225,8 @@ MainWindow::recordingStarted() m_recordingLatencyFrames = roundTrip + m_recordingStartGapEstimate; m_takeLatency.roundTrip = roundTrip; - m_takeLatency.reportedOutput = outputLatency; - m_takeLatency.reportedInput = inputLatency; + m_takeLatency.reportedOutput = inUse.reportedOutput; + m_takeLatency.reportedInput = inUse.reportedInput; m_takeLatency.measured = (inUse.source == LatencyCalibration::Source::Measured); cerr << "MainWindow::recordingStarted: round trip " << roundTrip diff --git a/main/MainWindow.h b/main/MainWindow.h index 49ca203c..6b50df6f 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -873,8 +873,9 @@ protected slots: // What the take being recorded, or the last one, was placed with: // cleared when a take starts, the round trip and the latencies the - // device reported filled in when the reference starts to play, and - // the recording's rate when the take is spliced in + // device reported (in seconds, as roundTripAt() has them) filled in + // when the reference starts to play, and the recording's rate when + // the take is spliced in TakeLatency m_takeLatency; // The audio check, and the override it sets for each take of its diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 09a37f02..1168a56e 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -29,6 +29,8 @@ #include "../AudioCheckRunner.h" #include "../LatencyCheck.h" +#include "base/PlayParameterRepository.h" + class TestAudioCheck : public QObject { Q_OBJECT @@ -45,20 +47,27 @@ class TestAudioCheck : public QObject QTimer m_watchdog; QStringList m_dialogs; - // What the runner said when the run ended, and how often it said it + // What the runner said when the run ended, and how often it said it; + // and every progress it reported AudioCheckResult m_result; int m_finished = 0; + std::vector m_progress; void makeWindow(FakeAudioIO::Config config, bool installDevice = true) { delete m_window; m_window = new TestMainWindow(config, installDevice); m_result = AudioCheckResult(); m_finished = 0; + m_progress.clear(); connect(m_window->audioCheck(), &AudioCheckRunner::finished, this, [this](const AudioCheckResult &result) { m_result = result; ++m_finished; }); + connect(m_window->audioCheck(), &AudioCheckRunner::progress, + this, [this](const AudioCheckRunner::Progress &progress) { + m_progress.push_back(progress); + }); } // Speakers into the microphone: the output comes back as input, @@ -149,6 +158,140 @@ class TestAudioCheck : public QObject .toUtf8(); } + // What differs from the check's playback, or "": the reference + // audible, centred, and brought back from full scale to the level it + // was made at; its pitch and notes silent + QString checkPlaybackProblem() { + const float gain = + float(std::pow(10.0, LatencyCheck::kPeakDbfs / 20.0)); + QStringList problems; + auto reference = sv::PlayParameterRepository::getInstance() + ->getPlayParameters(m_window->mainModelId().untyped); + if (!reference) { + problems << "the reference has no play parameters"; + } else { + if (!reference->isPlayAudible()) { + problems << "the reference is muted"; + } + if (reference->getPlayPan() != 0.f) { + problems << QString("the reference is panned %1") + .arg(reference->getPlayPan()); + } + if (std::fabs(reference->getPlayGain() - gain) > 1e-6f) { + problems << QString("the reference plays at %1 dB") + .arg(20.0 * std::log10(reference->getPlayGain())); + } + } + for (Analyser::Component c : { Analyser::PitchTrack, + Analyser::Notes }) { + sv::Layer *layer = m_window->analyser()->getLayer(c); + auto params = layer ? layer->getPlayParameters() : nullptr; + if (!params) { + problems << QString("no layer %1").arg(int(c)); + } else if (params->isPlayAudible()) { + problems << QString("layer %1 is heard").arg(int(c)); + } + } + return problems.join(", "); + } + + // The settings the analysers' show and play toggles are kept in, as + // Analyser::loadState() reads them + static QStringList analyserSettings() { + QSettings settings; + settings.beginGroup("Analyser"); + QStringList state; + for (int c = Analyser::Audio; c <= Analyser::Spectrogram; ++c) { + state << QString("component %1 visible %2 audible %3").arg(c) + .arg(settings.value(QString("visible-%1").arg(c), + c != Analyser::Spectrogram).toBool()) + .arg(settings.value(QString("audible-%1").arg(c), true) + .toBool()); + } + settings.endGroup(); + return state; + } + + // The reference muted in the user's own sessions. Its spectrogram's + // setting as well: both are read for the reference's model, the + // spectrogram's last + static void muteReferenceInSettings() { + QSettings settings; + settings.beginGroup("Analyser"); + settings.setValue(QString("audible-%1").arg(Analyser::Audio), false); + settings.setValue(QString("audible-%1").arg(Analyser::Spectrogram), + false); + settings.endGroup(); + } + + // How the session open now plays its reference, and its pitch and + // notes. Not the gain of those two: the toolbar moves it to a notch + // of its own level control in the first session of a window only + QStringList sessionPlayback() { + QStringList state; + auto reference = sv::PlayParameterRepository::getInstance() + ->getPlayParameters(m_window->mainModelId().untyped); + if (reference) { + state << QString("reference audible %1 pan %2 gain %3") + .arg(reference->isPlayAudible()) + .arg(reference->getPlayPan()) + .arg(reference->getPlayGain()); + } else { + state << "no reference"; + } + for (Analyser::Component c : { Analyser::PitchTrack, + Analyser::Notes }) { + sv::Layer *layer = m_window->analyser()->getLayer(c); + auto params = layer ? layer->getPlayParameters() : nullptr; + if (params) { + state << QString("layer %1 audible %2 pan %3").arg(int(c)) + .arg(params->isPlayAudible()) + .arg(params->getPlayPan()); + } else { + state << QString("no layer %1").arg(int(c)); + } + } + return state; + } + + // An ordinary file, opened as the user opens one, and analysed + void openSong() { + const QString path = m_dir.filePath("song.wav"); + if (!QFileInfo::exists(path)) { + const std::vector samples = TestSignals::sawtooth + (220.5, rate, int(1.5 * rate), 0.5); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *data = samples.data(); + QVERIFY(writer.isOK()); + QVERIFY(writer.writeSamples(&data, + sv::sv_frame_t(samples.size()))); + QVERIFY(writer.close()); + } + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + Analyser *analyser = m_window->analyser(); + QTRY_VERIFY_WITH_TIMEOUT + (analyser->getLayer(Analyser::PitchTrack) && + analyser->getLayer(Analyser::Notes) && + analyser->getInitialAnalysisCompletion() >= 100 && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + } + + static QString stepName(AudioCheckRunner::Step step) { + switch (step) { + case AudioCheckRunner::Step::Idle: return "idle"; + case AudioCheckRunner::Step::OpeningReference: return "opening"; + case AudioCheckRunner::Step::AnalysingReference: + return "analysing the reference"; + case AudioCheckRunner::Step::Recording: return "recording"; + case AudioCheckRunner::Step::AnalysingTake: return "analysing the take"; + } + return ""; + } + // Not a slot: QtTest would run it as a test. As TestRecordWorkflow's void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -270,8 +413,8 @@ private slots: QCOMPARE(int(r.takes.size()), 2); for (const TakeLatency &t : r.takes) { QCOMPARE(t.roundTrip, sv::sv_frame_t(reportedOut + reportedIn)); - QCOMPARE(t.reportedOutput, sv::sv_frame_t(reportedOut)); - QCOMPARE(t.reportedInput, sv::sv_frame_t(reportedIn)); + QCOMPARE(t.reportedOutput, reportedOut / rate); + QCOMPARE(t.reportedInput, reportedIn / rate); QCOMPARE(t.recordingRate, rate); } QCOMPARE(r.usedRoundTrip, (reportedOut + reportedIn) / rate); @@ -445,6 +588,182 @@ private slots: QVERIFY(!m_window->recordTarget()->isRecording()); QCOMPARE(m_finished, 1); } + + // The check's session plays the reference centred, at the level it + // was made at, and not its pitch and notes: from the first take to + // the end of the run, and after it. The user has muted the reference + // in their own sessions, which the check does not follow; its sweeps + // reach the speakers at -12 dBFS, in both channels, and come back at + // the level they were made at. Its progress names each step and + // punch-in in order, with the recording still to come going down + void check_plays_the_reference_centred_and_quiet() { + muteReferenceInSettings(); + const QStringList settingsBefore = analyserSettings(); + makeWindow(loopback()); + + QStringList problems; + int looks = 0; + bool recorded = false; + QTimer sampler; + connect(&sampler, &QTimer::timeout, &sampler, [&]() { + if (!m_window->audioCheck()->isRunning()) return; + if (m_window->recordTarget()->isRecording()) recorded = true; + if (!recorded) return; + ++looks; + const QString problem = checkPlaybackProblem(); + if (problem != "" && !problems.contains(problem)) { + problems << problem; + } + }); + sampler.start(20); + runCheck(); + sampler.stop(); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_result.failure == "", describe(m_result).constData()); + + // What reached the speakers, both channels mixed: the reference + // centred has the same level there as in each channel + const std::vector played = + m_window->fake()->getCapturedOutput(); + float peak = 0.f; + for (float s : played) peak = std::max(peak, std::fabs(s)); + const double peakDb = 20.0 * std::log10(peak); + QVERIFY2(std::fabs(peakDb - LatencyCheck::kPeakDbfs) <= 0.5, + qPrintable(QString("played at %1 dBFS").arg(peakDb))); + + // and each sweep, as the finder heard it come back: 0 dB is the + // level it was made at + QVERIFY2(m_result.summary.found == 4, describe(m_result).constData()); + for (const LatencyCheck::EventResult &e : m_result.summary.events) { + if (!e.arrival.found) continue; + QVERIFY2(std::fabs(e.arrival.levelDb) <= 1.0, + qPrintable(QString("the sweep at %1 s came back at " + "%2 dB") + .arg(e.expectedSeconds) + .arg(e.arrival.levelDb))); + } + + // The playback, looked at every 20 ms from the first take on: + // neither the analyses nor the next take put anything back + QVERIFY2(looks >= 200, + qPrintable(QString("looked %1 times").arg(looks))); + QVERIFY2(problems.isEmpty(), + qPrintable("during the check: " + problems.join(" | "))); + const QString after = checkPlaybackProblem(); + QVERIFY2(after == "", qPrintable("after the check: " + after)); + QCOMPARE(analyserSettings(), settingsBefore); + + // Each step once as it begins, and more often while a take is + // recorded, as the seconds of recording to come go down + QVERIFY(!m_progress.empty()); + const AudioCheckRunner::Plan plan = shortPlan(); + double planned = 0.0; + for (const LatencyCheck::PunchIn &p : LatencyCheck::punchInsFor + (plan.layout, plan.punchIns, plan.eventsEach)) { + planned += std::min(AudioCheckRunner::kPreRollSeconds, p.start) + + (p.end - p.start); + } + QVERIFY2(std::fabs(m_progress.front().secondsLeft - planned) < 0.01, + qPrintable(QString("%1 s to record at first, not %2 s") + .arg(m_progress.front().secondsLeft) + .arg(planned))); + QStringList steps; + int reportsWhileRecording[3] = { 0, 0, 0 }; + double left = m_progress.front().secondsLeft; + for (const AudioCheckRunner::Progress &p : m_progress) { + QCOMPARE(p.punchIns, 2); + QVERIFY(p.punchIn >= 0 && p.punchIn <= 2); + QVERIFY2(p.secondsLeft <= left + 0.001, + qPrintable(QString("%1 s left after %2 s") + .arg(p.secondsLeft).arg(left))); + left = p.secondsLeft; + const QString step = + QString("%1 %2").arg(stepName(p.step)).arg(p.punchIn); + if (steps.isEmpty() || steps.back() != step) steps << step; + if (p.step == AudioCheckRunner::Step::Recording) { + ++reportsWhileRecording[p.punchIn]; + } + } + QCOMPARE(steps, QStringList() + << "opening 0" << "analysing the reference 0" + << "recording 1" << "analysing the take 1" + << "recording 2" << "analysing the take 2"); + QVERIFY2(reportsWhileRecording[1] >= 3 && reportsWhileRecording[2] >= 3, + qPrintable(QString("%1 and %2 reports while recording") + .arg(reportsWhileRecording[1]) + .arg(reportsWhileRecording[2]))); + QCOMPARE(m_progress.back().secondsLeft, 0.0); + } + + // A session opened after a check plays as one opened before it did, + // and the settings every session reads are as they were: the + // check's playback was its session's alone. The user has the + // reference muted and the sonification on, both the other way round + // from the check's + void check_leaves_the_next_session_alone() { + muteReferenceInSettings(); + makeWindow(loopback()); + openSong(); + if (QTest::currentTestFailed()) return; + const QStringList before = sessionPlayback(); + const QStringList settingsBefore = analyserSettings(); + QVERIFY2(checkPlaybackProblem() != "", qPrintable(before.join(", "))); + + startCheck(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QTest::qWait(500); + QCOMPARE(checkPlaybackProblem(), QString()); + m_window->audioCheck()->cancel(); + QCOMPARE(m_finished, 1); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + QCOMPARE(analyserSettings(), settingsBefore); + + openSong(); + if (QTest::currentTestFailed()) return; + QCOMPARE(sessionPlayback(), before); + QCOMPARE(analyserSettings(), settingsBefore); + } + + // Each check writes its reference to a file of its own, never over + // the one the open session plays, which may be the check before's; + // the others are removed, and the lowest free number is taken + void check_reference_gets_a_file_of_its_own() { + QTemporaryDir dir; + QVERIFY(dir.isValid()); + auto touch = [&](QString name) { + QFile file(dir.filePath(name)); + return file.open(QIODevice::WriteOnly); + }; + auto files = [&]() { + return QDir(dir.path()).entryList(QDir::Files, QDir::Name); + }; + const QString one = dir.filePath("calibrate-audio-reference-1.wav"); + const QString two = dir.filePath("calibrate-audio-reference-2.wav"); + + QCOMPARE(AudioCheckRunner::nextReferencePath(dir.path(), ""), one); + QVERIFY(touch("calibrate-audio-reference-1.wav")); + + // Check Again, with the first check's session open + QCOMPARE(AudioCheckRunner::nextReferencePath(dir.path(), one), two); + QVERIFY(touch("calibrate-audio-reference-2.wav")); + QVERIFY(touch("calibrate-audio-reference.wav")); + QVERIFY(touch("song.wav")); + + // and again + QCOMPARE(AudioCheckRunner::nextReferencePath(dir.path(), two), one); + QCOMPARE(files(), QStringList() + << "calibrate-audio-reference-2.wav" << "song.wav"); + + // A check with the user's song open + QCOMPARE(AudioCheckRunner::nextReferencePath + (dir.path(), dir.filePath("song.wav")), one); + QCOMPARE(files(), QStringList() << "song.wav"); + } }; #endif diff --git a/main/test/TestLatencyCalibration.h b/main/test/TestLatencyCalibration.h index b806062c..37223691 100644 --- a/main/test/TestLatencyCalibration.h +++ b/main/test/TestLatencyCalibration.h @@ -209,7 +209,8 @@ private slots: } } - // The stored figure while it is fresh, the reported sum otherwise + // The stored figure while it is fresh, the reported sum otherwise; + // and either way the reported pair it was chosen against void round_trip_in_use() { const double out = 0.2; const double in = 0.1; @@ -219,6 +220,8 @@ private slots: QCOMPARE(none.roundTrip, out + in); QVERIFY(none.date.isNull()); QVERIFY(!none.stale); + QCOMPARE(none.reportedOutput, out); + QCOMPARE(none.reportedInput, in); const Figure fresh = figure(0.345, out, in); InUse measured = LatencyCalibration::roundTripInUse(&fresh, out, in); @@ -226,6 +229,8 @@ private slots: QCOMPARE(measured.roundTrip, 0.345); QCOMPARE(measured.date, fresh.date); QVERIFY(!measured.stale); + QCOMPARE(measured.reportedOutput, out); + QCOMPARE(measured.reportedInput, in); const Figure old = figure(0.345, out + 0.01, in); InUse stale = LatencyCalibration::roundTripInUse(&old, out, in); @@ -233,6 +238,8 @@ private slots: QCOMPARE(stale.roundTrip, out + in); QVERIFY(stale.date.isNull()); QVERIFY(stale.stale); + QCOMPARE(stale.reportedOutput, out); + QCOMPARE(stale.reportedInput, in); // A latency reported as less than nothing counts as none InUse negative = LatencyCalibration::roundTripInUse(nullptr, -0.5, in); diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 116509da..b70d1059 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1377,8 +1377,8 @@ private slots: TakeLatency used = m_window->takeLatency(); QVERIFY(used.measured); QCOMPARE(used.roundTrip, sv::sv_frame_t(K)); - QCOMPARE(used.reportedOutput, sv::sv_frame_t(reportedOut)); - QCOMPARE(used.reportedInput, sv::sv_frame_t(reportedIn)); + QCOMPARE(used.reportedOutput, reportedOut / rate); + QCOMPARE(used.reportedInput, reportedIn / rate); } // A round trip measured while the device reported other latencies @@ -1421,7 +1421,8 @@ private slots: // A device at 48 kHz reporting 2 x 4096 frames out and 4096 in: the // take is placed with those 3 x 4096 frames of the recording, // although the play source has the output latency in frames of the - // session, converted as it resamples + // session, converted as it resamples; the take keeps each latency in + // seconds, the output's from those converted frames void latency_reported_at_the_device_rate() { const double deviceRate = 48000.0; const int reportedOut = 2 * 4096; @@ -1443,8 +1444,8 @@ private slots: QCOMPARE(used.recordingRate, deviceRate); QVERIFY(!used.measured); QCOMPARE(used.reportedOutput, - sv::sv_frame_t(std::lround(reportedOut * rate / deviceRate))); - QCOMPARE(used.reportedInput, sv::sv_frame_t(reportedIn)); + std::lround(reportedOut * rate / deviceRate) / rate); + QCOMPARE(used.reportedInput, reportedIn / deviceRate); QVERIFY2(std::llabs(used.roundTrip - (reportedOut + reportedIn)) <= 2, qPrintable(QString("placed with %1 frames at %2 Hz; the " "device reports %3 + %4") @@ -1456,7 +1457,7 @@ private slots: // device menu at startup. The play source has no rate yet, so its // resampler passes the output latency on as the device counts it, // and tells the play source the device's rate is 0: the round trip - // is the same + // is the same, and so is the output latency in seconds void latency_reported_with_device_opened_first() { const double deviceRate = 48000.0; const int reportedOut = 2 * 4096; @@ -1478,7 +1479,7 @@ private slots: TakeLatency used = m_window->takeLatency(); QCOMPARE(used.recordingRate, deviceRate); - QCOMPARE(used.reportedOutput, sv::sv_frame_t(reportedOut)); + QCOMPARE(used.reportedOutput, reportedOut / deviceRate); QVERIFY2(std::llabs(used.roundTrip - (reportedOut + reportedIn)) <= 2, qPrintable(QString("placed with %1 frames at %2 Hz; the " "device reports %3 + %4") From 3963e5119f6741bee3b4973646e72536f8d05b06 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:12:22 +0000 Subject: [PATCH 114/275] docs: calibrate audio work orders, B3 done Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index b3d0c088..c823b945 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -522,3 +522,7 @@ The next phase must know: - Pre-existing, not fixed: (a) in the first file of a window the same feedback moves the pitch and notes gain from 0.5 to 0.562 and forces both audible, writing the settings; (b) `audible-0` is overridden at load by `audible-3`: the spectrogram is a layer on the reference's model, so it plays whenever `audible-3` is true. - B4: add a few seconds per analysis to `secondsLeft` for a rough total. Left open: the default reference path is exercised only through `nextReferencePath()`; the tests name their own file. + +### Lead — 2026-09-26, after B3 +- De-raced `TestSingingAnalysis::waitForRange()` (`2a306fb`): `initialAnalysisCompleted()` also fires from `layerCompletionChanged()` before a ranged merge; it failed once in a full run. +- For phase D's `open-points.md`, older bugs B3 found: (a) in a window's first file the toolbar level control's notches move the pitch/notes gain 0.5 → 0.562, force both audible and write that to the settings; (b) `audible-0` (Play Audio) is overridden on load by `audible-3` (the spectrogram layer on the same model, loaded last). Also B1's: closing a session during an ordinary take, then Stop, hangs. From 6d2978b2d50c1e5d72766f092b6889c0771ca6d2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:20:08 +0000 Subject: [PATCH 115/275] feat: import TTML lyrics, and export the lyrics as TTML Import Lyrics reads TTML as well as LRC, so the exporter's word ends reach Tony. File > Export Lyrics... writes the words as they are now in the session to a TTML file, named after the reference and beside it by default; not a change to the session. The README now tells to export TTML, word by word, from Moises. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- README.md | 29 ++-- main/MainWindow.cpp | 108 +++++++++++++-- main/MainWindow.h | 26 +++- main/test/TestRecordWorkflow.h | 240 ++++++++++++++++++++++++++++++++- 4 files changed, 369 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 8768e0c7..c220b3fa 100644 --- a/README.md +++ b/README.md @@ -47,19 +47,22 @@ orange pitch track to compare with it. the toolbar): each keeps its own audio, pitch track and notes, and switching between them needs no re-analysis. The audio of a session's takes is kept in a folder named after the session beside it, so the two can be moved together - * timed lyrics: File -> Import Lyrics... reads an LRC file, timed by line or - by word, and shows the words in boxes along the bottom of the pane, at the - time and for as long as each is sung. The word at the playback position is - highlighted, while playing, while recording and wherever you click, and the - waveform is faded while the words are on show so that they can be read over - it. They are saved with the session; View -> Show Lyrics hides them and - File -> Remove Lyrics takes them out. The reference must be the recording - the lyrics were timed to (a Moises stem and its original mix share a - timeline); to change a word or its time, edit the file and import it again - * LRC files can be exported from Moises with the Moises-Lyric-Exporter browser - extension. Set its offset to 0 (otherwise every line is 0.2 s early, and - Tony cannot tell) and its gap threshold as low as it goes (so that it marks - where lines end). It is unofficial and not made by Moises: install it + * timed lyrics: File -> Import Lyrics... reads a TTML or LRC file, timed by + word or by line, and shows the words in boxes along the bottom of the pane, + at the time and for as long as each is sung. The word at the playback + position is highlighted, while playing, while recording and wherever you + click, and the waveform is faded while the words are on show so that they + can be read over it. They are saved with the session; View -> Show Lyrics + hides them, File -> Remove Lyrics takes them out, and File -> Export + Lyrics... writes them, as they are now, to a TTML file. The reference must + be the recording the lyrics were timed to (a Moises stem and its original + mix share a timeline); to change a word or its time, edit the file and + import it again + * lyrics can be exported from Moises with the Moises-Lyric-Exporter browser + extension. Set it to TTML, word by word, with the offset at 0 (otherwise + every line is 0.2 s early, and Tony cannot tell): TTML gives every word + its end as well as its start, which LRC cannot. Its gap threshold only + matters for LRC. It is unofficial and not made by Moises: install it unpacked from a commit that has been reviewed, and do not update it without reviewing the change diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 1cd4f01b..0affb265 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -20,6 +20,7 @@ #include "Analyser.h" #include "LatencyUtils.h" #include "Lyrics.h" +#include "LyricsTtml.h" #include "PaneUtils.h" #include "TakeEvents.h" #include "TakeLayers.h" @@ -107,6 +108,8 @@ #include #include #include +#include +#include #include #include #include @@ -153,6 +156,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_coverageStrip(nullptr), m_lyrics(nullptr), m_importLyricsAction(nullptr), + m_exportLyricsAction(nullptr), m_removeLyricsAction(nullptr), m_showLyrics(nullptr), m_takesMenu(nullptr), @@ -619,11 +623,17 @@ MainWindow::setupFileMenu() // Enabled in updateMenuStates() m_importLyricsAction = new QAction(il.load("fileopen"), tr("Import &Lyrics..."), this); - m_importLyricsAction->setStatusTip(tr("Import timed lyrics from an LRC file, to be shown along the top of the pane")); + m_importLyricsAction->setStatusTip(tr("Import timed lyrics from a TTML or LRC file, to be shown along the bottom of the pane")); m_importLyricsAction->setEnabled(false); connect(m_importLyricsAction, &QAction::triggered, this, &MainWindow::importLyrics); menu->addAction(m_importLyricsAction); + m_exportLyricsAction = new QAction(tr("Expor&t Lyrics..."), this); + m_exportLyricsAction->setStatusTip(tr("Write the lyrics, as they are now, to a TTML file")); + m_exportLyricsAction->setEnabled(false); + connect(m_exportLyricsAction, &QAction::triggered, this, &MainWindow::exportLyrics); + menu->addAction(m_exportLyricsAction); + m_removeLyricsAction = new QAction(tr("Remove Lyrics"), this); m_removeLyricsAction->setStatusTip(tr("Take the imported lyrics out of the session")); m_removeLyricsAction->setEnabled(false); @@ -992,7 +1002,7 @@ MainWindow::setupViewMenu() // Peek Left has the L m_showLyrics = new QAction(tr("Show L&yrics"), this); m_showLyrics->setCheckable(true); - m_showLyrics->setStatusTip(tr("Show or hide the imported lyrics along the top of the pane")); + m_showLyrics->setStatusTip(tr("Show or hide the imported lyrics along the bottom of the pane")); m_showLyrics->setEnabled(false); connect(m_showLyrics, &QAction::triggered, this, &MainWindow::showLyricsToggled); menu->addAction(m_showLyrics); @@ -2245,6 +2255,9 @@ MainWindow::updateMenuStates() if (m_importLyricsAction) { m_importLyricsAction->setEnabled(lyricsImportAllowed()); } + if (m_exportLyricsAction) { + m_exportLyricsAction->setEnabled(m_lyrics && m_lyrics->isShown()); + } if (m_removeLyricsAction) { m_removeLyricsAction->setEnabled(m_lyrics && m_lyrics->isShown()); } @@ -3377,8 +3390,8 @@ MainWindow::askForLyricsFile() return QFileDialog::getOpenFileName (this, tr("Import Lyrics"), dir, - tr("LRC lyrics (*.lrc)") + ";;" + tr("Text files (*.txt)") + ";;" + - tr("All files (*)")); + tr("Lyrics (*.ttml *.lrc)") + ";;" + tr("TTML lyrics (*.ttml)") + + ";;" + tr("LRC lyrics (*.lrc)") + ";;" + tr("All files (*)")); } void @@ -3400,16 +3413,16 @@ MainWindow::importLyricsFrom(QString path) emit activity(tr("Import lyrics \"%1\"").arg(path)); - // An LRC file is a few kB. One chosen by mistake, a recording say, is - // not read at all; and the read stops just past the limit, which - // parseLrc() enforces too, in case the file grows in the meantime + // A lyrics file is a few kB. One chosen by mistake, a recording say, + // is not read at all; and the read stops just past the limit, which + // the parsers enforce too, in case the file grows in the meantime QString error; QByteArray bytes; QFileInfo info(path); if (!info.isFile()) { error = tr("File \"%1\" could not be found.").arg(path); } else if (info.size() > Lyrics::maxFileBytes) { - error = tr("The file is over 1 MB, too big to be an LRC file."); + error = tr("The file is over 1 MB, too big to be a lyrics file."); } else { QFile file(path); if (!file.open(QIODevice::ReadOnly)) { @@ -3422,7 +3435,7 @@ MainWindow::importLyricsFrom(QString path) LyricsParseResult parsed; if (error == "") { - parsed = parseLrc(bytes); + parsed = parseLyrics(bytes); error = parsed.error; } @@ -3492,6 +3505,83 @@ MainWindow::importLyricsFrom(QString path) return true; } +QString +MainWindow::askForLyricsExportFile(QString suggested) +{ + return QFileDialog::getSaveFileName + (this, tr("Export Lyrics"), suggested, + tr("TTML lyrics (*.ttml)") + ";;" + tr("All files (*)")); +} + +void +MainWindow::exportLyrics() +{ + if (!m_lyrics || !m_lyrics->isShown()) return; + + // Named after the reference and beside it, as the lyrics to import + // were looked for there + QString suggested; + if (auto reference = getMainModel()) { + QFileInfo info(reference->getLocation()); + if (info.exists()) { + suggested = QDir(info.absolutePath()) + .filePath(info.completeBaseName() + ".ttml"); + } + } + + QString path = askForLyricsExportFile(suggested); + if (path.isEmpty()) return; + exportLyricsTo(path); +} + +bool +MainWindow::exportLyricsTo(QString path) +{ + if (!m_lyrics || !m_lyrics->isShown()) return false; + auto model = ModelById::getAs(m_lyrics->getModelId()); + if (!model) return false; + + // The words as the model has them now, not as the file they came + // from had them. The title is not in the model: an import named the + // layer after it + Lyrics lyrics = lyricsFromEvents(model->getAllEvents(), + model->getSampleRate()); + if (RegionLayer *layer = m_lyrics->getLayer()) { + QString name = layer->getLayerPresentationName(); + if (name != tr("Lyrics")) lyrics.title = name; + } + QByteArray bytes = writeTtml(lyrics); + + // A file that is there already is replaced only once the new one is + // written in full + QSaveFile file(path); + bool written = file.open(QIODevice::WriteOnly) && + file.write(bytes) == bytes.size() && + file.commit(); + if (!written) { + QMessageBox::warning + (this, tr("Could not export lyrics"), + tr("File \"%1\" could not be written: %2") + .arg(path).arg(file.errorString())); + return false; + } + + emit activity(tr("Export lyrics to \"%1\"").arg(path)); + + // Not a change to the session, so nothing is marked modified. Kept + // as the status message, as an import's is + int wordCount = int(lyrics.words.size()); + QSet lines; + for (const LyricWord &w : lyrics.words) lines.insert(w.line); + int lineCount = int(lines.size()); + m_myStatusMessage = tr("Exported %1 in %2.") + .arg(wordCount == 1 ? tr("1 word") : tr("%1 words").arg(wordCount), + lineCount == 1 ? tr("1 line") : tr("%1 lines").arg(lineCount)); + getStatusLabel()->setText(m_myStatusMessage); + + return true; +} + void MainWindow::removeLyrics() { diff --git a/main/MainWindow.h b/main/MainWindow.h index 8f5d07c9..250824ef 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -181,6 +181,7 @@ protected slots: virtual void syncAlternatePitchTrack(); virtual void importLyrics(); + virtual void exportLyrics(); virtual void removeLyrics(); virtual void showLyricsToggled(); @@ -369,6 +370,7 @@ protected slots: // belong to the song, not to a take. Display only LyricsTrack *m_lyrics; QAction *m_importLyricsAction; + QAction *m_exportLyricsAction; QAction *m_removeLyricsAction; QAction *m_showLyrics; @@ -379,22 +381,32 @@ protected slots: // the reference's analyser has taken its waveform over void updateWaveformFade(); - // Put the lyrics of this LRC file on the reference's timeline, in - // place of any there are. Not undoable, as loading background music - // is not, and nothing goes onto the undo stack. False if the file - // could not be read or holds no timed lyrics, which the user is told - // in a dialog, or if lyricsImportAllowed() says no; nothing has - // changed then + // Put the lyrics of this TTML or LRC file on the reference's + // timeline, in place of any there are. Not undoable, as loading + // background music is not, and nothing goes onto the undo stack. + // False if the file could not be read or holds no timed lyrics, + // which the user is told in a dialog, or if lyricsImportAllowed() + // says no; nothing has changed then bool importLyricsFrom(QString path); // Lyrics can be imported once there is a reference, and not while a // take is being recorded bool lyricsImportAllowed() const; - // Ask for the LRC file to import, "" if the user cancelled. + // Ask for the TTML or LRC file to import, "" if the user cancelled. // Overridden by the tests, which cannot answer a dialog virtual QString askForLyricsFile(); + // Write the lyrics, as the model holds them now, to this TTML file. + // Not a command, and the session is not changed by it. False if + // there are no lyrics, or if the file could not be written, which + // the user is told in a dialog + bool exportLyricsTo(QString path); + + // Ask where to export the lyrics to, offering the suggested path; + // "" if the user cancelled. Overridden by the tests + virtual QString askForLyricsExportFile(QString suggested); + // --- The audio folder of the session (spec 6.4) --- // Where the next combined audio file of a take is to be written: the diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index da8b5538..e9509bd3 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -30,6 +30,7 @@ #include "../CoverageStrip.h" #include "../Lyrics.h" #include "../LyricsTrack.h" +#include "../LyricsTtml.h" #include "../SingingTakes.h" #include "../TakeLayers.h" #include "../TakesFile.h" @@ -219,10 +220,12 @@ class TestMainWindow : public MainWindow QAction *alternatePitchUpAction() { return m_alternatePitchUpAction; } QAction *alternatePitchDownAction() { return m_alternatePitchDownAction; } - // The timed lyrics, and the three menu actions that act on them + // The timed lyrics, and the four menu actions that act on them LyricsTrack *lyrics() { return m_lyrics; } bool doImportLyricsFrom(QString path) { return importLyricsFrom(path); } + bool doExportLyricsTo(QString path) { return exportLyricsTo(path); } QAction *importLyricsAction() { return m_importLyricsAction; } + QAction *exportLyricsAction() { return m_exportLyricsAction; } QAction *removeLyricsAction() { return m_removeLyricsAction; } QAction *showLyricsAction() { return m_showLyrics; } @@ -230,6 +233,11 @@ class TestMainWindow : public MainWindow void setLyricsFileAnswer(QString path) { m_lyricsFileAnswer = path; } int lyricsFileQuestions() const { return m_lyricsFileQuestions; } + // The file Export Lyrics asks for, likewise, and the path it offered + void setLyricsExportAnswer(QString path) { m_lyricsExportAnswer = path; } + int lyricsExportQuestions() const { return m_lyricsExportQuestions; } + QString lyricsExportSuggestion() const { return m_lyricsExportSuggestion; } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { onRealtimePitchDetected(frame, hz); } @@ -268,6 +276,12 @@ class TestMainWindow : public MainWindow return m_lyricsFileAnswer; } + QString askForLyricsExportFile(QString suggested) override { + ++m_lyricsExportQuestions; + m_lyricsExportSuggestion = suggested; + return m_lyricsExportAnswer; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -281,6 +295,9 @@ class TestMainWindow : public MainWindow QString m_takeNameAnswer; QString m_lyricsFileAnswer; int m_lyricsFileQuestions = 0; + QString m_lyricsExportAnswer; + int m_lyricsExportQuestions = 0; + QString m_lyricsExportSuggestion; }; class TestRecordWorkflow : public QObject @@ -1005,10 +1022,11 @@ class TestRecordWorkflow : public QObject return QString(TONY_TEST_DATA_DIR) + "/lyrics/" + name; } + // As the import reads it: TTML or LRC static Lyrics lyricsIn(QString path) { QFile file(path); if (!file.open(QIODevice::ReadOnly)) return {}; - return parseLrc(file.readAll()).lyrics; + return parseLyrics(file.readAll()).lyrics; } // What an import of this file has to put in the model: the words on @@ -1019,9 +1037,9 @@ class TestRecordWorkflow : public QObject return lyricsToEvents(lyricsIn(path), reference->getSampleRate()); } - QString writeLrc(const QByteArray &text) { + QString writeLrc(const QByteArray &text, const char *suffix = "lrc") { QString path = m_dir.filePath - (QString("lyrics-%1.lrc").arg(++m_fileCounter)); + (QString("lyrics-%1.%2").arg(++m_fileCounter).arg(suffix)); QFile file(path); if (!file.open(QIODevice::WriteOnly) || file.write(text) != text.size()) { @@ -5895,7 +5913,10 @@ private slots: { untimed, "No timed lyrics were found" }, { m_dir.filePath("no-such-lyrics.lrc"), "could not be found" }, { writeLrc(QByteArray(int(Lyrics::maxFileBytes) + 1, 'a')), - "over 1 MB" }, + "over 1 MB, too big to be a lyrics file" }, + { writeLrc("\n

Sana\n\n", + "ttml"), + "not well-formed XML" }, }; for (const auto &f : failures) { QVERIFY2(!m_window->doImportLyricsFrom(f.first), @@ -5991,6 +6012,215 @@ private slots: QTRY_VERIFY_WITH_TIMEOUT(import->isEnabled(), 2000); } + // TTML as the exporter writes it word by word: every word ends where + // the file says, not where the next one starts + void lyrics_import_ttml() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + m_window->discardModifications(); + + QString path = lyricsFixture("moises-exporter-words.ttml"); + QVERIFY(m_window->doImportLyricsFrom(path)); + QVERIFY(m_window->lyrics()->isShown()); + QVERIFY(m_window->isDocumentModified()); + QCOMPARE(m_window->lyrics()->getLayer()->getLayerPresentationName(), + QString("Lyrics")); + + sv::EventVector expected = expectedLyricsEvents(path); + QCOMPARE(int(expected.size()), 12); + sv::EventVector events = lyricsEvents(); + QCOMPARE(events, expected); + + // "Tämä" is sung from 0.52 s to 0.80 s and "on" starts at 0.84 s, + // where an inferred end would be. The syllables of "keksitty" are + // one word, and the second

is the second line + auto frameAt = [](double seconds) { + return sv::sv_frame_t(std::llround(seconds * rate)); + }; + QCOMPARE(events[0].getLabel(), QString("Tämä")); + QCOMPARE(events[0].getFrame(), frameAt(0.52)); + QCOMPARE(events[0].getFrame() + events[0].getDuration(), frameAt(0.80)); + QCOMPARE(events[1].getFrame(), frameAt(0.84)); + QCOMPARE(events[2].getLabel(), QString("keksitty")); + QCOMPARE(events[2].getFrame(), frameAt(1.10)); + QCOMPARE(events[2].getFrame() + events[2].getDuration(), frameAt(1.90)); + QCOMPARE(events[3].getValue(), 0.f); + QCOMPARE(events[4].getLabel(), QString("Yö")); + QCOMPARE(events[4].getValue(), 1.f); + + QString status = m_window->statusText(); + QVERIFY2(status.startsWith("Imported 12 words in 3 lines."), + qPrintable(status)); + } + + // File > Export Lyrics... writes the words as the model holds them + // now, not as the file they came from had them, as TTML that reads + // back to the same words. Not a change to the session + void lyrics_export() { + makeWindow(FakeAudioIO::Config()); + QString reference = writeWav(tone(lowHz, 1.0)); + openReference(reference); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + auto model = sv::ModelById::getAs + (m_window->lyrics()->getModelId()); + QVERIFY(model); + + // One word changed in the model since: its text, and its end + sv::EventVector imported = lyricsEvents(); + auto word = std::find_if(imported.begin(), imported.end(), + [](const sv::Event &e) { + return e.getLabel() == "hämärä"; + }); + QVERIFY(word != imported.end()); + model->remove(*word); + model->add(word->withLabel("hämärämpi") + .withDuration(word->getDuration() / 2)); + sv::EventVector events = lyricsEvents(); + QVERIFY(events != imported); + + m_window->discardModifications(); + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + + QString path = m_dir.filePath("exported-lyrics.ttml"); + m_window->setLyricsExportAnswer(path); + QVERIFY(m_window->exportLyricsAction()->isEnabled()); + m_window->exportLyricsAction()->trigger(); + QCOMPARE(m_window->lyricsExportQuestions(), 1); + + // The name offered is the reference's, beside it + QFileInfo info(reference); + QCOMPARE(m_window->lyricsExportSuggestion(), + QDir(info.absolutePath()) + .filePath(info.completeBaseName() + ".ttml")); + + QFile file(path); + QVERIFY2(file.open(QIODevice::ReadOnly), "nothing was written"); + LyricsParseResult parsed = parseTtml(file.readAll()); + QVERIFY2(parsed.error == "", qPrintable(parsed.error)); + + // The same words, texts and lines, at the same times to the + // nearest millisecond, and the title the layer was named after + Lyrics now = lyricsFromEvents(events, rate); + const QVector &back = parsed.lyrics.words; + QCOMPARE(back.size(), now.words.size()); + for (int i = 0; i < back.size(); ++i) { + QString what = QString("word %1, \"%2\"").arg(i) + .arg(now.words[i].text); + QVERIFY2(back[i].text == now.words[i].text, + qPrintable(what + " came back as " + back[i].text)); + QVERIFY2(back[i].line == now.words[i].line, + qPrintable(what + ": another line")); + QVERIFY2(std::fabs(back[i].start - now.words[i].start) < 0.0005001, + qPrintable(what + ": another start")); + QVERIFY2(std::fabs(back[i].end - now.words[i].end) < 0.0005001, + qPrintable(what + ": another end")); + } + QStringList texts; + for (const LyricWord &w : back) texts << w.text; + QVERIFY2(texts.contains("hämärämpi") && !texts.contains("hämärä"), + qPrintable(texts.join(" "))); + QCOMPARE(parsed.lyrics.title, QString("Kesäyön testilaulu")); + + QCOMPARE(m_window->statusText(), + QString("Exported 13 words in %1 lines.") + .arg(now.lineCount())); + QCOMPARE(int(commands.count()), 0); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(lyricsEvents(), events); + } + + // Only when there are lyrics, shown or hidden + void lyrics_export_needs_lyrics() { + makeWindow(FakeAudioIO::Config()); + QAction *exportAction = m_window->exportLyricsAction(); + QVERIFY2(!exportAction->isEnabled(), "there is no reference yet"); + + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY2(!exportAction->isEnabled(), "there are no lyrics yet"); + + QString lrc = lyricsFixture("moises-exporter-words.lrc"); + QVERIFY(m_window->doImportLyricsFrom(lrc)); + QVERIFY(exportAction->isEnabled()); + m_window->showLyricsAction()->trigger(); + QVERIFY(!m_window->lyrics()->isVisible()); + QVERIFY2(exportAction->isEnabled(), "hidden lyrics are lyrics too"); + m_window->showLyricsAction()->trigger(); + + m_window->removeLyricsAction()->trigger(); + QVERIFY(!m_window->lyrics()->isShown()); + QVERIFY(!exportAction->isEnabled()); + QString path = m_dir.filePath("removed-lyrics.ttml"); + m_window->setLyricsExportAnswer(path); + exportAction->trigger(); + QCOMPARE(m_window->lyricsExportQuestions(), 0); + QVERIFY(!m_window->doExportLyricsTo(path)); + QVERIFY(!QFileInfo::exists(path)); + + QVERIFY(m_window->doImportLyricsFrom(lrc)); + QVERIFY(exportAction->isEnabled()); + m_window->doCloseSession(); + QVERIFY2(!exportAction->isEnabled(), "the session has gone"); + } + + void lyrics_export_cancelled() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + QString status = m_window->statusText(); + + const QStringList ttml { "*.ttml" }; + QStringList before = QDir(m_dir.path()).entryList(ttml, QDir::Files); + m_window->setLyricsExportAnswer(""); + m_window->exportLyricsAction()->trigger(); + QCOMPARE(m_window->lyricsExportQuestions(), 1); + QVERIFY(m_window->lyricsExportSuggestion() != ""); + QVERIFY2(!QFileInfo::exists(m_window->lyricsExportSuggestion()), + "written where the dialog would have suggested"); + QCOMPARE(QDir(m_dir.path()).entryList(ttml, QDir::Files), before); + QCOMPARE(m_window->statusText(), status); + } + + // One dialog each, and nothing written or changed + void lyrics_export_failure() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->doImportLyricsFrom + (lyricsFixture("moises-exporter-words.lrc"))); + sv::EventVector events = lyricsEvents(); + QString status = m_window->statusText(); + m_window->discardModifications(); + + // Not a file made read-only: root can write to that. A folder + // that is not there, and the name of a folder + QString noFolder = m_dir.filePath("no-such-folder/lyrics.ttml"); + QString folder = m_dir.filePath("a-folder.ttml"); + QVERIFY(QDir().mkpath(folder)); + + for (QString path : { noFolder, folder }) { + m_window->setLyricsExportAnswer(path); + m_window->exportLyricsAction()->trigger(); + QStringList dialogs = dialogsMatching("Could not export lyrics"); + QCOMPARE(dialogs.size(), 1); + QVERIFY2(dialogs[0].contains(path), qPrintable(dialogs[0])); + QCOMPARE(m_window->statusText(), status); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(lyricsEvents(), events); + } + QCOMPARE(m_window->lyricsExportQuestions(), 2); + QVERIFY(!QFileInfo::exists(m_dir.filePath("no-such-folder"))); + QVERIFY(QFileInfo(folder).isDir()); + QVERIFY(QDir(folder).isEmpty()); + } + // Saved with the session and found again by name when it is opened, // as it was: hidden if it was hidden void lyrics_session_round_trip() { From 92d04b843c7659d4d8e0bfa09c7a54a8884fe4e8 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 05:32:34 +0300 Subject: [PATCH 116/275] fix: pitch candidates make no undo commands The analyser added the pitch candidates of a re-analysis (a selection on the reference, or a note dragged with the Edit tool) as undoable layers. An undo could take them out of the pane while the analyser went on listing them, and the next re-analysis then pushed a command to remove each one while clearing the redo stack that owned and deleted them: the next Ctrl+Z crashed on the freed layer. They are now the analyser's own layers like every other: attached with no command, shown and hidden directly, deleted outright when discarded, and discarded too when the whole file is analysed again. Selections and note drags no longer put "Re-Analyse Selection" or "Show Pitch Candidates" entries into the undo history; choosing a candidate for the pitch track is still undoable. Co-Authored-By: Claude Opus 5.5 --- docs/architecture.md | 15 +++++---- docs/open-points.md | 5 --- main/Analyser.cpp | 49 +++++++++++------------------ main/MainWindow.cpp | 4 --- main/test/TestRecordWorkflow.h | 56 ++++++++++++++++++++++++++++++++++ 5 files changed, 83 insertions(+), 46 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index a9d9d9d0..090310a3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -95,12 +95,15 @@ These were all learned from crashes or wrong behaviour. They hold for any new co ### Tony's own layers make no undo commands -Every layer Tony makes for itself — analysers' layers, live dots, the recording's hidden -waveform, the alternate pitch track, the coverage strip, background music — is added with -**`Document::attachLayerToView()`** (svapp fork): in the view and in the layer-view map, so -the session keeps it, but no command and no modified flag. `addLayerToView()` (the -undoable Add Layer) must not be used for these: Undo after a take has to find the take. -Whoever attaches the layer calls `documentModified()` if the change should count. +Every layer Tony makes for itself — analysers' layers, pitch candidates, live dots, the +recording's hidden waveform, the alternate pitch track, the coverage strip, background +music — is added with **`Document::attachLayerToView()`** (svapp fork): in the view and in +the layer-view map, so the session keeps it, but no command and no modified flag. +`addLayerToView()` (the undoable Add Layer) must not be used for these: Undo after a take +has to find the take. Whoever attaches the layer calls `documentModified()` if the change +should count. Such a layer is shown and hidden with `showLayer()` and removed with +`deleteLayer(layer, true)`, never by command: an undo that takes a layer out of the pane +leaves whoever keeps a pointer to it holding a layer that the redo stack owns and deletes. ### Commands diff --git a/docs/open-points.md b/docs/open-points.md index 4811901b..009dc821 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -11,11 +11,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - **No overwrite question when recording into a selection**: the selection is taken as the consent. Right in use? - **Take operations clear the undo history with no prompt** (all but Rename). -- **A selection is an entry of the undo history**: making one re-analyses the reference in - it (upstream Tony's pitch candidates) and pushes "Re-Analyse Selection". Selecting for - Record into Selection or for Erase therefore puts such entries between "Record Singing" - and "Erase Singing", and if the re-analysis finishes after an erase, Ctrl+Z takes it - back instead of the erase. - **The Edit tool edits the take's note at the time it is used, wherever in the pane**, the band of the coverage strip included. The strip itself takes no edits. Should the band keep the tools off the notes? diff --git a/main/Analyser.cpp b/main/Analyser.cpp index e7cbf079..1e344550 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -33,7 +33,6 @@ #include "layer/LayerFactory.h" #include "layer/SpectrogramLayer.h" #include "layer/Colour3DPlotLayer.h" -#include "layer/ShowLayerCommand.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" @@ -162,10 +161,9 @@ Analyser::analyseExistingFile() QString Analyser::doAllAnalyses(bool withPitchTrack) { - m_reAnalysingSelection = Selection(); - m_reAnalysisCandidates.clear(); - m_currentCandidate = -1; - m_candidatesVisible = false; + // Candidates of the analysis being replaced. Only forgotten, they + // would stay in the pane: nothing takes a layer of ours out but us + discardPitchCandidates(); // Note that we need at least one main-model layer (time ruler, // waveform or what have you). It could be hidden if we don't want @@ -969,10 +967,7 @@ Analyser::reAnalyseSelection(Selection sel, FrequencyRange range) } if (!m_reAnalysisCandidates.empty()) { - CommandHistory::getInstance()->startCompoundOperation - (tr("Discard Previous Candidates"), true); discardPitchCandidates(); - CommandHistory::getInstance()->endCompoundOperation(); } m_reAnalysingSelection = sel; @@ -1432,16 +1427,10 @@ Analyser::showPitchCandidates(bool shown) { if (m_candidatesVisible == shown) return; + // Directly and not by command, as with every layer of our own: see + // layersCreated() foreach (Layer *layer, m_reAnalysisCandidates) { - if (shown) { - CommandHistory::getInstance()->addCommand - (new ShowLayerCommand(m_pane, layer, true, - tr("Show Pitch Candidates"))); - } else { - CommandHistory::getInstance()->addCommand - (new ShowLayerCommand(m_pane, layer, false, - tr("Hide Pitch Candidates"))); - } + layer->showLayer(m_pane, shown); } m_candidatesVisible = shown; @@ -1468,9 +1457,10 @@ Analyser::layersCreated(Document::LayerCreationAsyncHandle handle, } m_currentAsyncHandle = 0; - CommandHistory::getInstance()->startCompoundOperation - (tr("Re-Analyse Selection"), true); - + // Like every layer of our own, the candidates make no undo + // commands. An undo could otherwise take them out of the pane + // while they stay on our list, and removing them again later + // left a command holding a layer already deleted m_reAnalysisCandidates.clear(); vector all; @@ -1491,7 +1481,7 @@ Analyser::layersCreated(Document::LayerCreationAsyncHandle handle, t->setBaseColour (ColourDatabase::getInstance()->getColourIndex(tr("Bright Orange"))); t->setPresentationName("candidate"); - m_document->addLayerToView(m_pane, t); + m_document->attachLayerToView(m_pane, t); m_reAnalysisCandidates.push_back(t); /* cerr << "New re-analysis candidate model has " @@ -1505,8 +1495,6 @@ Analyser::layersCreated(Document::LayerCreationAsyncHandle handle, m_candidatesVisible = !show; // to ensure the following takes effect showPitchCandidates(show); } - - CommandHistory::getInstance()->endCompoundOperation(); } emit layersChanged(); @@ -1628,15 +1616,14 @@ Analyser::clearReAnalysis() void Analyser::discardPitchCandidates() { - if (!m_reAnalysisCandidates.empty()) { - // We don't use a compound command here, because we may be - // already in one. Caller bears responsibility for doing that - foreach (Layer *layer, m_reAnalysisCandidates) { - // This will cause the layer to be deleted later (ownership is - // transferred to the remove command) - m_document->removeLayerFromView(m_pane, layer); + // Deleted outright, with no command: see layersCreated(). They exist + // only once their transform has finished, so there is none to cancel + vector doomed = m_reAnalysisCandidates; + m_reAnalysisCandidates.clear(); // before deleteLayer() tells us of each + if (m_document) { + for (Layer *layer : doomed) { + m_document->deleteLayer(layer, true); } - m_reAnalysisCandidates.clear(); } m_currentCandidate = -1; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index b0c9af7c..4419f024 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -6763,12 +6763,8 @@ MainWindow::octaveShift(bool up) void MainWindow::togglePitchCandidates() { - CommandHistory::getInstance()->startCompoundOperation(tr("Toggle Pitch Candidates"), true); - m_analyser->showPitchCandidates(!m_analyser->arePitchCandidatesShown()); - CommandHistory::getInstance()->endCompoundOperation(); - updateMenuStates(); } diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 553cedc2..e067beeb 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -70,6 +70,7 @@ #include #include #include +#include #include #include #include @@ -2943,6 +2944,61 @@ private slots: "the analyser still points at its deleted note layer"); } + // Pitch candidates are the analyser's own layers: they come and go + // with no entry in the undo history, an undo leaves them alone, and + // the next re-analysis deletes them. As undoable layers they could be + // undone out of the pane while the analyser went on listing them, and + // the next re-analysis then made a command of each that a later undo + // crashed on + void candidates_make_no_undo_entries() { + makeWindow(FakeAudioIO::Config()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + Analyser *a = m_window->analyser(); + sv::Pane *pane = a->getPane(); + auto candidates = [&]() { + std::vector> found; + for (int i = 0; i < pane->getLayerCount(); ++i) { + sv::Layer *layer = pane->getLayer(i); + if (layer->getLayerPresentationName() == "candidate") { + found.push_back(layer); + } + } + return found; + }; + sv::CommandHistory::getInstance()->clear(); + + std::vector> earlier; + for (double start : { 0.5, 0.8 }) { + QString error = a->reAnalyseSelection + (sv::Selection(sv::sv_frame_t(start * rate), + sv::sv_frame_t((start + 0.7) * rate)), + Analyser::FrequencyRange()); + QVERIFY2(error.isEmpty(), qPrintable(error)); + QTRY_VERIFY_WITH_TIMEOUT(a->haveHigherPitchCandidate(), 30000); + QVERIFY(!candidates().empty()); + for (const QPointer &layer : earlier) { + QVERIFY2(!layer, "a re-analysis left the candidates of the " + "one before it alive"); + } + QString undone = undoOnce(); + QVERIFY2(undone.isEmpty(), + qPrintable("the re-analysis left \"" + undone + + "\" in the undo history")); + QVERIFY2(a->haveHigherPitchCandidate() && !candidates().empty(), + "an undo took the pitch candidates away"); + earlier = candidates(); + } + + a->clearReAnalysis(); + QVERIFY2(candidates().empty(), + "clearing the re-analysis left candidates in the pane"); + for (const QPointer &layer : earlier) { + QVERIFY2(!layer, "clearing the re-analysis left its candidates " + "alive"); + } + } + void load_background_music() { makeWindow(FakeAudioIO::Config()); openReference(writeWav(tone(lowHz, 1.0))); From 785be7ceef8ba500399e1037263d64deb6b85a96 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 05:33:39 +0300 Subject: [PATCH 117/275] test: the edit tool may edit the take's note over the coverage strip Decided: the Edit tool acts on the take's note at the time under the pointer at any height in the pane, the band of the coverage strip included, as upstream's note tools do. strip_ignores_the_mouse failed on Windows because a drag there also re-analyses the pitch under the note, which adds pitch candidates and may put one into the take's pitch track. The test now lets an edit of the note change the take's notes, its pitch track and the candidates, waits for the re-analysis to finish before judging, and undoes every entry a gesture added before checking that all else is as it was. Co-Authored-By: Claude Opus 5.5 --- docs/open-points.md | 3 -- docs/takes.md | 4 ++ docs/testing.md | 5 ++ main/test/TestUiChecks.h | 114 ++++++++++++++++++++++++++++----------- 4 files changed, 92 insertions(+), 34 deletions(-) diff --git a/docs/open-points.md b/docs/open-points.md index 009dc821..ecb2d42e 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -11,9 +11,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - **No overwrite question when recording into a selection**: the selection is taken as the consent. Right in use? - **Take operations clear the undo history with no prompt** (all but Rename). -- **The Edit tool edits the take's note at the time it is used, wherever in the pane**, - the band of the coverage strip included. The strip itself takes no edits. Should the band - keep the tools off the notes? - **The alternate pitch track at ±3 octaves** of a 220 Hz reference (28 Hz, 1.8 kHz) is outside the range the pane shows, and nothing scrolls to it; ±2 is in view. - Of the [manual checklist](manual-checklist.md), the device check has been run only in diff --git a/docs/takes.md b/docs/takes.md index 01bac609..7a0db5d0 100644 --- a/docs/takes.md +++ b/docs/takes.md @@ -184,6 +184,10 @@ swap, never before: the swap restores the pane's state as it found it, and makes waveform layer again, on top, where it covers the band; `syncCoverageStrip()` raises the strip above it every time. It also removes the strip's model from the play source. +The strip takes no mouse input of its own. The Edit tool acts on the take's note at the +time under the pointer, at any height in the pane, and that includes the band: decided so, +rather than keeping the tools off the notes there (`strip_ignores_the_mouse`). + ## Files on disk `SingingTakes` tracks three sets: **superseded** paths (kept until the session closes, for diff --git a/docs/testing.md b/docs/testing.md index 04e1ffbf..ad1c5516 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -194,6 +194,11 @@ it draws is judged by pixels: about (246, 126, 114) there: `isSinging()` takes both. Bright orange is the reference's pitch candidates, which a selection makes. - Record puts the view back on the take's position: work out x positions again after it. +- A drag of a note re-analyses the pitch under it. The candidates arrive when that + transform finishes, and one of them may go into the pitch track, so judge only once + `haveRunningTransformers()` is false. The drag may leave more than one entry in the undo + history, and undoing them leaves the candidates in the pane: they are the analyser's own + layers, gone at the next re-analysis. - Timing checks in real time go through the pane's own timers (the pointer moves every 20 ms): allow a tick. diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 47475bac..e8706fce 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -68,6 +68,7 @@ #include #include #include +#include #include class TestUiChecks : public QObject @@ -323,6 +324,17 @@ class TestUiChecks : public QObject QCoreApplication::processEvents(); } + // A drag across pane 0 with the left button, in ten steps + void drag(QPoint from, QPoint to) { + sv::Pane *pane = pane0(); + QTest::mousePress(pane, Qt::LeftButton, Qt::NoModifier, from); + for (int i = 1; i <= 10; ++i) { + QTest::mouseMove(pane, from + (to - from) * i / 10); + QTest::qWait(10); + } + QTest::mouseRelease(pane, Qt::LeftButton, Qt::NoModifier, to); + } + Coverage::Ranges coverage() { return m_window->takes()->getCoverage().getRanges(); } @@ -338,6 +350,9 @@ class TestUiChecks : public QObject sv::ZoomLevel zoom; sv::MultiSelection::SelectionList selections; QString undo; + // Those of the layers that are pitch candidates of a + // re-analysis. Not compared: layers has them + std::set candidates; bool operator==(const PaneState &s) const { return events == s.events && extents == s.extents && @@ -353,6 +368,9 @@ class TestUiChecks : public QObject sv::Layer *layer = pane->getLayer(i); QString key = QString("%1 %2").arg(i).arg(layer->objectName()); s.layers.push_back(key); + if (layer->getLayerPresentationName() == "candidate") { + s.candidates.insert(key); + } sv::ModelId id = layer->getModel(); if (auto m = sv::ModelById::getAs(id)) { s.events[key] = m->getAllEvents(); @@ -706,9 +724,12 @@ private slots: // Checklist: the coverage strip cannot be touched. Clicking, double- // clicking and dragging on it with either tool creates, moves, selects // and edits nothing of its own and does not change the pane's scale. - // The Edit tool acts on the take's note at the time it is used, - // wherever in the pane that is -- the band too -- so an edit of that - // note is what it may do, and one undo takes that back exactly + // The Edit tool acts on the take's note at the time in the song under + // the pointer, at any height in the pane -- the band too, and that is + // meant -- so an edit of that note is what it may do. A drag moves + // the note and re-analyses the pitch under it, which may leave more + // than one entry in the undo history; undoing them takes it all back + // exactly, but for the pitch candidates, which are the analyser's void strip_ignores_the_mouse() { FakeAudioIO::Config config; config.input = tone(highHz, 6.0); @@ -740,17 +761,6 @@ private slots: "the strip is not where the gestures are going to be made"); saveShot("strip", image); - auto drag = [&](int from, int to) { - QTest::mousePress(pane, Qt::LeftButton, Qt::NoModifier, - QPoint(from, y)); - for (int i = 1; i <= 10; ++i) { - QTest::mouseMove(pane, QPoint(from + (to - from) * i / 10, y)); - QTest::qWait(10); - } - QTest::mouseRelease(pane, Qt::LeftButton, Qt::NoModifier, - QPoint(to, y)); - }; - struct Gesture { QString name; std::function act; }; std::vector gestures { { "a click in the band", [&]() { @@ -762,24 +772,56 @@ private slots: { "a click at its end", [&]() { QTest::mouseClick(pane, Qt::LeftButton, Qt::NoModifier, QPoint(atEnd, y)); } }, - { "a drag along it", [&]() { drag(inBand, atEnd); } }, - { "a drag off its end", [&]() { drag(atEnd, beyond); } }, - { "a drag onto it", [&]() { drag(beyond, inBand); } }, + { "a drag along it", [&]() { + drag(QPoint(inBand, y), QPoint(atEnd, y)); } }, + { "a drag off its end", [&]() { + drag(QPoint(atEnd, y), QPoint(beyond, y)); } }, + { "a drag onto it", [&]() { + drag(QPoint(beyond, y), QPoint(inBand, y)); } }, + }; + + // A drag of the take's note also re-analyses the pitch under it. + // That puts pitch candidates into the pane, which stay until the + // next re-analysis whatever is undone: they are the analyser's + // own layers. Everything but them, the other layers numbered as + // if they were not there + auto withoutCandidates = [&](const PaneState &s) { + PaneState r; + int n = 0; + for (const QString &key : s.layers) { + if (s.candidates.count(key)) continue; + QString renumbered = + QString("%1 %2").arg(n++).arg(key.section(' ', 1)); + r.layers.push_back(renumbered); + auto e = s.events.find(key); + if (e != s.events.end()) r.events[renumbered] = e->second; + auto x = s.extents.find(key); + if (x != s.extents.end()) r.extents[renumbered] = x->second; + } + r.zoom = s.zoom; + r.selections = s.selections; + r.undo = s.undo; + return r; }; - // Everything but the take's notes and the undo history - auto apartFromNotes = [&](PaneState s) { - for (auto i = s.events.begin(); i != s.events.end(); ) { - if (i->first.endsWith(TakeLayers::nameFor - (m_window->takes()->getActiveName(), - TakeLayers::Notes))) { - i = s.events.erase(i); + // ... and but what an edit of the take's note may change: the + // note, the undo history, and the take's pitch track, into which + // the drag may put one of the candidates + auto apartFromNoteEdit = [&](const PaneState &s) { + QString take = m_window->takes()->getActiveName(); + QStringList edited { + TakeLayers::nameFor(take, TakeLayers::Notes), + TakeLayers::nameFor(take, TakeLayers::Pitch) }; + PaneState r = withoutCandidates(s); + for (auto i = r.events.begin(); i != r.events.end(); ) { + if (edited.contains(i->first.section(' ', 1))) { + i = r.events.erase(i); } else { ++i; } } - s.undo = ""; - return s; + r.undo = ""; + return r; }; int noteEdits = 0; @@ -790,6 +832,11 @@ private slots: const PaneState before = paneState(); g.act(); QTest::qWait(300); // outlast the double-click interval + // and any re-analysis the gesture started: its candidates + // arrive when it finishes (Analyser::layersCreated()) + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); // The navigate tool scrolls when dragged, as anywhere pane->setCentreFrame(centre); PaneState after = paneState(); @@ -801,20 +848,25 @@ private slots: .arg(describe(before, after)))); continue; } - QVERIFY2(apartFromNotes(after) == apartFromNotes(before), + QVERIFY2(apartFromNoteEdit(after) == apartFromNoteEdit(before), qPrintable(QString("with the edit tool, %1 changed " "%2") .arg(g.name) - .arg(describe(before, after)))); + .arg(describe(apartFromNoteEdit(before), + apartFromNoteEdit(after))))); if (after.undo != before.undo) { ++noteEdits; - press(QKeySequence(tr("Ctrl+Z"))); + for (int i = 0; i < 20 && undoText() != before.undo; ++i) { + press(QKeySequence(tr("Ctrl+Z"))); + } PaneState undone = paneState(); - QVERIFY2(undone == before, + QVERIFY2(withoutCandidates(undone) == + withoutCandidates(before), qPrintable(QString("undoing what %1 did with " "the edit tool left %2") .arg(g.name) - .arg(describe(before, undone)))); + .arg(describe(withoutCandidates(before), + withoutCandidates(undone))))); } } } From 9bcb51a999b5f99c230b4e871d66fd68e83cc541 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 05:33:57 +0300 Subject: [PATCH 118/275] fix: an edited note of a take keeps the take's pitch The Edit tool sets the pitch of a note it has split, dragged or merged from a pitch track, and FlexiNoteLayer took the first pitch track in the pane: the reference's. A split or a drag of a take's note therefore gave it the reference's pitch. The svgui pin now takes the pitch track of the same source model as the notes (svgui a34646a, on this branch's svgui work), and edited_take_notes_keep_the_take_pitch splits and drags a take's note sung a fourth above the reference. Co-Authored-By: Claude Opus 5.5 --- docs/forks.md | 4 ++ docs/testing.md | 5 +++ main/test/TestUiChecks.h | 88 ++++++++++++++++++++++++++++++++++++++++ repoint-lock.json | 2 +- 4 files changed, 98 insertions(+), 1 deletion(-) diff --git a/docs/forks.md b/docs/forks.md index 87189c2c..03ce204d 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -74,6 +74,10 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` `plotStyle` attribute. - `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the notes of a take that is put away. +- `FlexiNoteLayer::getAssociatedPitchModel()`, which the note tools set a note's pitch + from, takes the pitch track with the same source model as the notes, and the first in + the view only when there is none. With the reference first in the pane, an edited + take's note otherwise took the reference's pitch. - `Pane::setWorkModel()` / `getWorkModel()`: which model's extents are blocked off at the ends of the pane (a pale wash and a line), and whose duration, title and alignment are reported. The scan that chooses one now skips layers dormant in that pane. Tony's pane diff --git a/docs/testing.md b/docs/testing.md index ad1c5516..e21e9d06 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -194,6 +194,11 @@ it draws is judged by pixels: about (246, 126, 114) there: `isSinging()` takes both. Bright orange is the reference's pitch candidates, which a selection makes. - Record puts the view back on the take's position: work out x positions again after it. +- What the Edit tool does is decided by where the pointer last hovered over a note + (`FlexiNoteLayer::mouseMoveEvent()`): near its top a drag moves the note, near its bottom + a click splits it, and before any hover a drag moves and a click does nothing. `hover()` + sends that move to the pane; `QTest::mouseMove()` with no button held moves the + platform's cursor instead. - A drag of a note re-analyses the pitch under it. The candidates arrive when that transform finishes, and one of them may go into the pitch track, so judge only once `haveRunningTransformers()` is false. The drag may leave more than one entry in the undo diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index e8706fce..64d6b925 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -36,6 +36,7 @@ #include "view/PaneStack.h" #include "layer/Layer.h" #include "layer/ColourDatabase.h" +#include "layer/CoordinateScale.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" #include "data/model/RegionModel.h" @@ -59,6 +60,7 @@ #include #include #include +#include #include #include #include @@ -324,6 +326,20 @@ class TestUiChecks : public QObject QCoreApplication::processEvents(); } + // The pointer over pane 0 with no button held. Where it was last over + // a note decides what the Edit tool does to that note: near its top a + // drag moves it, near its bottom a click splits it, and with the + // pointer never over a note a drag moves it. Sent to the pane itself: + // QTest::mouseMove() with no button held moves the platform's cursor + // instead, which the offscreen platform need not pass on + void hover(QPoint pos) { + sv::Pane *pane = pane0(); + QMouseEvent move(QEvent::MouseMove, QPointF(pos), + QPointF(pane->mapToGlobal(pos)), + Qt::NoButton, Qt::NoButton, Qt::NoModifier); + QApplication::sendEvent(pane, &move); + } + // A drag across pane 0 with the left button, in ten steps void drag(QPoint from, QPoint to) { sv::Pane *pane = pane0(); @@ -875,6 +891,78 @@ private slots: press(QKeySequence("1")); } + // The Edit tool gives a take's note it has changed the pitch of the + // take's own pitch track there, not that of the first pitch track in + // the pane, which is the reference's: a split gives each half the + // pitch sung in it, a drag the note where it is let go + void edited_take_notes_keep_the_take_pitch() { + FakeAudioIO::Config config; + config.input = tone(highHz, 6.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 4.0))); + if (QTest::currentTestFailed()) return; + + m_window->seekTo(frames(0.5)); + take(1500); + if (QTest::currentTestFailed()) return; + m_window->clearSelections(); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + showSeconds(0.0, 3.0); + + sv::Pane *pane = pane0(); + sv::Layer *layer = m_window->analyser2()->getLayer(Analyser::Notes); + QVERIFY(layer); + auto notes = [&]() { + auto model = sv::ModelById::getAs(layer->getModel()); + return model ? model->getAllEvents() : sv::EventVector(); + }; + auto verifySung = [&](QString what) { + sv::EventVector events = notes(); + QVERIFY2(!events.empty(), + qPrintable(what + " left the take no notes")); + for (const sv::Event &e : events) { + double cents = 1200.0 * std::log2(e.getValue() / highHz); + QVERIFY2(std::fabs(cents) < 50.0, + qPrintable(QString("after %1 a note of the take is " + "at %2 Hz, not at the %3 Hz sung " + "(the reference is at %4 Hz)") + .arg(what).arg(e.getValue()) + .arg(highHz).arg(lowHz))); + } + }; + + const sv::EventVector original = notes(); + QCOMPARE(int(original.size()), 1); + const sv::Event note = original[0]; + const int x = pane->getXForFrame(note.getFrame() + frames(0.3)); + const int noteY = pane->getEffectiveVerticalExtentsForLayer(layer) + .getCoordForValueRounded(pane, note.getValue()); + + press(QKeySequence("2")); + + hover(QPoint(x, noteY + 4)); + QTest::mouseClick(pane, Qt::LeftButton, Qt::NoModifier, + QPoint(x, noteY + 4)); + QCOMPARE(int(notes().size()), 2); + verifySung("a split"); + if (QTest::currentTestFailed()) return; + press(QKeySequence(tr("Ctrl+Z"))); + QVERIFY2(notes() == original, "undo did not take the split back"); + + hover(QPoint(x, noteY - 4)); + drag(QPoint(x, noteY - 4), QPoint(x + 60, noteY - 4)); + // The drag re-analyses the pitch under the note + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + QVERIFY2(notes() != original, "the drag did not move the note"); + verifySung("a drag"); + press(QKeySequence("1")); + } + // Checklist: the band is readable over waveform and dots at every // zoom: where there is singing the band is drawn over whatever else // is there, band high, and nowhere else diff --git a/repoint-lock.json b/repoint-lock.json index 6798fdb4..5869abea 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "15a0bd24fb69d9a03fef6ea8ef014db27d383fc0" + "pin": "a34646ac8888690724d890063789b123f7921885" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 18ac4ea9690e103a73f941cc521ef26abd18e23b Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 05:34:06 +0300 Subject: [PATCH 119/275] docs: a fork's branch follows the tony branch Fork work for tony work committed straight to default goes onto the fork branch of the table; fork work for a tony feature branch goes on a fork branch of the same name and reaches the fork branch when the tony branch is merged. And the fork branches go with the tony branch checked out: one left on another branch builds what the lock file does not say. Co-Authored-By: Claude Opus 5.5 --- docs/forks.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/forks.md b/docs/forks.md index 03ce204d..0f60d1c8 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -18,8 +18,12 @@ forks under `github.com/jhhr` that exist only for this Tony fork: The forks are free to change when Tony needs it; prefer a small, general addition to the library over a workaround in `main/`. -1. Edit and commit inside the library's directory (it is its own git repository, on the - fork branch). Commit messages there follow that repository's style: `area: what`. +1. Edit and commit inside the library's directory (it is its own git repository). The + fork's branch follows Tony's: for work committed straight to Tony's `default`, the fork + branch of the table; for work on a Tony feature branch, a fork branch of the **same + name**, on top of what it already holds (or started from the fork branch of the table), + which reaches the fork branch when the Tony branch is merged. Commit messages there + follow that repository's style: `area: what`. 2. Push to the remote named **`jhhr`**. In `svcore`, `svgui` and `svapp`, `origin` is upstream sonic-visualiser — do not push there. 3. Put the new commit hash in `repoint-lock.json` as that library's `pin`, and commit that @@ -27,6 +31,9 @@ library over a workaround in `main/`. 4. A sub-agent that was told to work only in `main/` does not edit a fork: it reports exactly which change it needs, and the lead session makes it. +When switching Tony branches, check out the fork branches that go with it: a fork left on +another branch builds something the lock file does not say. + **repoint does not run on the development machine** (it needs an SML compiler and none is installed). The checkouts are managed with plain git, and `repoint-project.json` / `repoint-lock.json` are edited by hand. Keep the lock file's pins equal to what is checked From 6c8011193a405912ab368f1afda3f03306794243 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:42:02 +0000 Subject: [PATCH 120/275] feat: the rules for editing lyrics words, as pure functions Which word and which of its edges the pointer is on, with the side of a shared edge picking the word; where a dragged start or end may go, stopping at the neighbouring word and at 20 ms; where Add Word puts a new word and which line it joins; and how typed text is cleaned. In tony_core, so that the editor to come only feeds them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsEdit.cpp | 302 +++++++++++++++ main/LyricsEdit.h | 213 ++++++++++ main/test/TestLyricsEdit.h | 725 +++++++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 5 files changed, 1249 insertions(+) create mode 100644 main/LyricsEdit.cpp create mode 100644 main/LyricsEdit.h create mode 100644 main/test/TestLyricsEdit.h diff --git a/main/LyricsEdit.cpp b/main/LyricsEdit.cpp new file mode 100644 index 00000000..a1b53c24 --- /dev/null +++ b/main/LyricsEdit.cpp @@ -0,0 +1,302 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LyricsEdit.h" + +#include "Lyrics.h" + +#include +#include + +using namespace sv; + +namespace { + +sv_frame_t framesFor(double seconds, sv_samplerate_t rate) +{ + if (rate <= 0) return 0; + return sv_frame_t(std::llround(seconds * rate)); +} + +// One past the box's last column. A word too short to reach the next +// column is still painted one column wide, and can be hit there +int columnAfter(const LyricsEdit::Box &box) +{ + return std::max(box.x1, box.x0 + 1); +} + +sv_frame_t endOf(const Event &word) +{ + return word.getFrame() + word.getDuration(); +} + +bool isWord(const EventVector &words, int word) +{ + return word >= 0 && word < int(words.size()); +} + +} // namespace + +sv_frame_t +LyricsEdit::minWordFrames(sv_samplerate_t rate) +{ + return framesFor(minWordSeconds, rate); +} + +sv_frame_t +LyricsEdit::newWordFrames(sv_samplerate_t rate) +{ + return framesFor(newWordSeconds, rate); +} + +LyricsEdit::Boxes +LyricsEdit::boxesFor(const EventVector &words, + const std::function &xForFrame) +{ + Boxes boxes; + boxes.reserve(words.size()); + for (const Event &w : words) { + boxes.push_back({ xForFrame(w.getFrame()), xForFrame(endOf(w)) }); + } + return boxes; +} + +int +LyricsEdit::wordAt(const Boxes &boxes, int x) +{ + int found = -1; + for (int i = 0; i < int(boxes.size()); ++i) { + const Box &box = boxes[i]; + if (x < box.x0 || x >= columnAfter(box)) continue; + // Later boxes are painted over earlier ones + if (found < 0 || box.x0 >= boxes[found].x0) found = i; + } + return found; +} + +LyricsEdit::Hit +LyricsEdit::hitTest(const Boxes &boxes, int x, int grab) +{ + Hit hit; + + int inside = wordAt(boxes, x); + if (inside >= 0) { + const Box &box = boxes[inside]; + int toStart = x - box.x0; + int toEnd = (columnAfter(box) - 1) - x; + bool onStart = (toStart < grab); + bool onEnd = (toEnd < grab); + hit.word = inside; + if (onStart && (!onEnd || toStart < toEnd)) { + hit.part = Part::Start; + } else if (onEnd) { + hit.part = Part::End; + } else { + hit.part = Part::Inside; + } + return hit; + } + + // In no box, every box is wholly to one side of the pointer: the + // edges that face it are the latest end to its left and the + // earliest start to its right + bool haveLeft = false, haveRight = false; + int leftEnd = 0, rightStart = 0; + for (const Box &box : boxes) { + int after = columnAfter(box); + if (after <= x) { + if (!haveLeft || after > leftEnd) leftEnd = after; + haveLeft = true; + } else { + if (!haveRight || box.x0 < rightStart) rightStart = box.x0; + haveRight = true; + } + } + + int toEnd = x - leftEnd; + int toStart = (rightStart - 1) - x; + bool onEnd = (haveLeft && toEnd < grab); + bool onStart = (haveRight && toStart < grab); + + // The word each edge belongs to is the one a pointer just inside + // it would find, so an edge is the same word's from either side + if (onEnd && (!onStart || toEnd <= toStart)) { + hit.word = wordAt(boxes, leftEnd - 1); + hit.part = Part::End; + } else if (onStart) { + hit.word = wordAt(boxes, rightStart); + hit.part = Part::Start; + } + return hit; +} + +sv_frame_t +LyricsEdit::clampStart(const EventVector &words, int word, + sv_frame_t wanted, sv_samplerate_t rate) +{ + if (!isWord(words, word)) return wanted; + + const sv_frame_t start = words[word].getFrame(); + const sv_frame_t end = endOf(words[word]); + + // Back to the latest end at or before the start; not back at all + // where an earlier word already overlaps it, since that would make + // the overlap longer + sv_frame_t lowest = 0; + for (int i = 0; i < int(words.size()); ++i) { + if (i == word) continue; + const Event &other = words[i]; + if (endOf(other) <= start) { + lowest = std::max(lowest, endOf(other)); + } else if (other.getFrame() < start) { + lowest = start; + } + } + // Every limit is where the start already is or before it, so the + // word can only move the way it is dragged. (A start before frame + // 0, which no parser makes, stays where it is.) + lowest = std::min(lowest, start); + + // Forward to 20 ms before the end, or nowhere for a word that is + // already shorter + const sv_frame_t highest = std::max(start, end - minWordFrames(rate)); + + return std::clamp(wanted, lowest, highest); +} + +sv_frame_t +LyricsEdit::clampEnd(const EventVector &words, int word, + sv_frame_t wanted, sv_samplerate_t rate) +{ + if (!isWord(words, word)) return wanted; + + const sv_frame_t start = words[word].getFrame(); + const sv_frame_t end = endOf(words[word]); + + // Back to 20 ms after the start, or nowhere for a word that is + // already shorter + const sv_frame_t lowest = std::min(end, start + minWordFrames(rate)); + + // Forward to the earliest start at or after the end; not forward at + // all where another word already reaches past the end. Nothing + // after the last word + bool limited = false; + sv_frame_t highest = end; + for (int i = 0; i < int(words.size()); ++i) { + if (i == word) continue; + const Event &other = words[i]; + sv_frame_t limit; + if (other.getFrame() >= end) { + limit = other.getFrame(); + } else if (endOf(other) > end) { + limit = end; + } else { + continue; + } + if (!limited || limit < highest) highest = limit; + limited = true; + } + + if (wanted < lowest) return lowest; + if (limited && wanted > highest) return highest; + return wanted; +} + +Event +LyricsEdit::startDraggedTo(const EventVector &words, int word, + sv_frame_t wanted, sv_samplerate_t rate) +{ + if (!isWord(words, word)) return Event(); + const Event &w = words[word]; + sv_frame_t start = clampStart(words, word, wanted, rate); + return w.withFrame(start).withDuration(endOf(w) - start); +} + +Event +LyricsEdit::endDraggedTo(const EventVector &words, int word, + sv_frame_t wanted, sv_samplerate_t rate) +{ + if (!isWord(words, word)) return Event(); + const Event &w = words[word]; + sv_frame_t end = clampEnd(words, word, wanted, rate); + return w.withDuration(end - w.getFrame()); +} + +bool +LyricsEdit::newWordSpan(const EventVector &words, sv_frame_t frame, + sv_samplerate_t rate, Span &span) +{ + if (rate <= 0 || frame < 0) return false; + + // The gap the frame is in, if it is in one + sv_frame_t gapStart = 0; + bool haveGapEnd = false; + sv_frame_t gapEnd = 0; + for (const Event &w : words) { + if (endOf(w) <= frame) { + gapStart = std::max(gapStart, endOf(w)); + } else if (w.getFrame() <= frame) { + return false; + } else if (!haveGapEnd || w.getFrame() < gapEnd) { + gapEnd = w.getFrame(); + haveGapEnd = true; + } + } + + sv_frame_t length = newWordFrames(rate); + sv_frame_t start = frame; + + if (haveGapEnd) { + sv_frame_t room = gapEnd - gapStart; + if (room < minWordFrames(rate)) return false; + // Up to the next word, and back from the click into the gap + // as far as that makes the word longer + length = std::min(length, room); + start = std::min(frame, gapEnd - length); + } + + span.start = start; + span.end = start + length; + return true; +} + +float +LyricsEdit::newWordLine(const EventVector &words, const Span &span) +{ + int before = -1, after = -1; + for (int i = 0; i < int(words.size()); ++i) { + const Event &w = words[i]; + if (endOf(w) <= span.start) { + if (before < 0 || endOf(w) >= endOf(words[before])) before = i; + } else if (w.getFrame() >= span.end) { + if (after < 0 || w.getFrame() < words[after].getFrame()) after = i; + } + } + + if (before < 0 && after < 0) return 0.f; + if (after < 0) return words[before].getValue(); + if (before < 0) return words[after].getValue(); + + // A word more often continues the line it follows than starts one + sv_frame_t gapBefore = span.start - endOf(words[before]); + sv_frame_t gapAfter = words[after].getFrame() - span.end; + return (gapBefore <= gapAfter ? + words[before].getValue() : words[after].getValue()); +} + +QString +LyricsEdit::cleanText(const QString &typed) +{ + return lyricsLabel(typed); +} diff --git a/main/LyricsEdit.h b/main/LyricsEdit.h new file mode 100644 index 00000000..833b83d4 --- /dev/null +++ b/main/LyricsEdit.h @@ -0,0 +1,213 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_EDIT_H +#define TONY_LYRICS_EDIT_H + +#include "base/BaseTypes.h" +#include "base/Event.h" + +#include + +#include +#include + +/** + * What editing a word of the lyrics may do: which edge the pointer is + * on, how far an edge can be dragged, where a new word goes and which + * line it joins, and what a typed text becomes. + * + * The words are the events of the lyrics' region model, in the order + * the model gives them (by start): a word's start is its frame and its + * end is frame + duration. Pure functions over those values, pixel + * positions and strings: no window and no model, so they are tested + * without either (TestLyricsEdit). The editor asks them and does as + * they say. + */ +namespace LyricsEdit +{ + /// No edit makes a word shorter than this (one already shorter + /// may stay so, but gets no shorter) + constexpr double minWordSeconds = 0.020; + + /// How long Add Word makes a word, where there is room for it + constexpr double newWordSeconds = 0.5; + + /** + * Those two in frames at this rate, to the nearest frame, as + * lyricsToEvents() rounds times: 882 and 22050 frames at 44.1 kHz. + * 0 for a rate that is not positive. + */ + sv::sv_frame_t minWordFrames(sv::sv_samplerate_t rate); + sv::sv_frame_t newWordFrames(sv::sv_samplerate_t rate); + + /** + * A word's box in the box row: the pixel columns x0 to x1 - 1 of + * the view the pointer is in. A box whose x1 is not past its x0 is + * painted, and hit, as the one column x0. + */ + struct Box { + int x0; + int x1; + }; + typedef std::vector Boxes; + + /** + * The boxes of the words, in their order: from the x of the word's + * start to the x of its end, as the lyrics layer paints them. + * xForFrame is the view's getXForFrame(). + */ + Boxes boxesFor(const sv::EventVector &words, + const std::function &xForFrame); + + /** + * The word whose box holds the column x, or -1 for none. Where + * boxes overlap, the one that starts latest, and of those starting + * in one column the last in order: the one painted on top. + */ + int wordAt(const Boxes &boxes, int x); + + enum class Part { + Nothing, ///< Not on a word nor near an edge + Start, ///< On the word's start edge + End, ///< On the word's end edge + Inside ///< In the word's box, away from its edges + }; + + struct Hit { + /// An index into the boxes, and so into the words they were + /// made from; -1 with Nothing + int word = -1; + Part part = Part::Nothing; + + bool isEdge() const { return part == Part::Start || part == Part::End; } + }; + + /** + * What the pointer at column x is on, with grab columns on each + * side of an edge. + * + * An edge is the line between two columns: a start is the line to + * the left of column x0, an end the line to the right of column + * x1 - 1. The pointer is on an edge if it is within grab columns + * of it, on either side: + * + * - In a box (wordAt()): its start if x - x0 < grab, its end if + * (x1 - 1) - x < grab, the nearer of the two if both (a narrow + * box), and the end if they are equally near; else Inside. + * - In no box: the end of the box to the left, if x - x1 < grab for + * the latest x1 at or before x; the start of the box to the + * right, if (x0 - 1) - x < grab for the earliest x0 after x; the + * nearer if both, and the end if they are equally near; else + * Nothing. Where several boxes have their edge there, the word + * is the one wordAt() gives in the column just inside it. + * + * So at an edge two words share, the pointer's side of it picks + * the word: the column to its left is the earlier word's end, the + * column to its right the later word's start. A grab of 0 finds + * no edges at all. + */ + Hit hitTest(const Boxes &boxes, int x, int grab); + + /** + * Where the start of words[word] goes when it is dragged to + * wanted. Pass the words as they were when the drag began, and + * the same ones at every move: the limits come from where the word + * was then, so a drag back puts it back. + * + * - Not before the end of a word that ends at or before its start + * (the previous word's, when words do not overlap), nor before + * frame 0. Where another word, starting earlier, already + * reaches past its start, the start does not move back at all: + * an overlap from a file may stay but does not grow. + * - Not later than 20 ms before its end; a word already shorter + * than 20 ms does not get shorter. + * + * The answer is always between where the start was and wanted, so + * a word never jumps, and its length is never negative. + */ + sv::sv_frame_t clampStart(const sv::EventVector &words, int word, + sv::sv_frame_t wanted, + sv::sv_samplerate_t rate); + + /** + * The end, likewise: not past the start of a word that starts at or + * after its end (the next word's), and not at all where another + * word, starting before its end, already reaches past it; not + * earlier than 20 ms after its start, or where it is for a word + * already shorter. After the last word there is no limit. + */ + sv::sv_frame_t clampEnd(const sv::EventVector &words, int word, + sv::sv_frame_t wanted, + sv::sv_samplerate_t rate); + + /** + * words[word] with its start dragged to wanted, as clampStart() + * allows, and its end where it was. Its text and line are kept. + */ + sv::Event startDraggedTo(const sv::EventVector &words, int word, + sv::sv_frame_t wanted, + sv::sv_samplerate_t rate); + + /// words[word] with its end dragged to wanted, as clampEnd() allows + sv::Event endDraggedTo(const sv::EventVector &words, int word, + sv::sv_frame_t wanted, + sv::sv_samplerate_t rate); + + struct Span { + sv::sv_frame_t start = 0; + sv::sv_frame_t end = 0; + + sv::sv_frame_t duration() const { return end - start; } + }; + + /** + * Where Add Word puts a word for a click at frame: from the frame, + * 0.5 s long, or up to the next word if that is nearer, moved back + * into the gap as far as that makes it longer (at most 0.5 s). + * The gap runs from the latest end at or before the frame (or + * frame 0) to the earliest start after it; after the last word it + * has no end. + * + * False, leaving span alone, if there is no room: the frame is in + * a word (a word holds its start frame but not its end frame, so a + * click exactly at a word's end is in the gap after it and the new + * word starts there), before frame 0, or in a gap shorter than + * 20 ms. + */ + bool newWordSpan(const sv::EventVector &words, sv::sv_frame_t frame, + sv::sv_samplerate_t rate, Span &span); + + /** + * The line (the event value) of a new word over span: that of the + * nearer of its neighbours, by the gap between them, and of the + * word before it if the gaps are equal; 0 with no words. The word + * before is the one ending latest at or before the span's start, + * the word after the one starting earliest at or after its end + * (the last and first in order of those, if several); a word + * overlapping the span, which newWordSpan() never gives, is + * neither. + */ + float newWordLine(const sv::EventVector &words, const Span &span); + + /** + * A typed text as a word's label: cleaned as the parsers clean + * theirs (lyricsLabel(): controls out, tab to space, trimmed, at + * most Lyrics::maxLabelLength characters). Empty if nothing is + * left, which the edit must refuse. + */ + QString cleanText(const QString &typed); +} + +#endif diff --git a/main/test/TestLyricsEdit.h b/main/test/TestLyricsEdit.h new file mode 100644 index 00000000..284f706d --- /dev/null +++ b/main/test/TestLyricsEdit.h @@ -0,0 +1,725 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LYRICS_EDIT_H +#define TEST_LYRICS_EDIT_H + +// Tier 2: what an edit of the lyrics may do, as numbers: which edge +// the pointer is on, how far a dragged edge goes, where a new word +// goes and which line it joins, and what typed text becomes. No window +// and no model: what the editor does with the answers is the app +// suite's business. + +#include "../LyricsEdit.h" +#include "../Lyrics.h" + +#include +#include + +#include + +class TestLyricsEdit : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + typedef LyricsEdit::Boxes Boxes; + typedef LyricsEdit::Span Span; + + static constexpr double kRate = 44100.0; + + // 20 ms and 0.5 s at that rate + static constexpr frame_t kMin = 882; + static constexpr frame_t kNew = 22050; + + static constexpr frame_t kSecond = 44100; + + // The grab width the editor is meant to use, about 6 pixels + static constexpr int kGrab = 6; + + static sv::Event word(frame_t start, frame_t end, float line = 0.f, + const char *text = "word") { + return sv::Event(start, line, end - start, QString(text)); + } + + static QString describe(const sv::Event &e) { + return QString("\"%1\" %2-%3 line %4") + .arg(e.getLabel()).arg(e.getFrame()) + .arg(e.getFrame() + e.getDuration()).arg(e.getValue()); + } + + static QString describe(const LyricsEdit::Hit &hit) { + switch (hit.part) { + case LyricsEdit::Part::Nothing: return QString("nothing %1").arg(hit.word); + case LyricsEdit::Part::Start: return QString("start of %1").arg(hit.word); + case LyricsEdit::Part::End: return QString("end of %1").arg(hit.word); + case LyricsEdit::Part::Inside: return QString("inside %1").arg(hit.word); + } + return QString("?"); + } + + // What the pointer at x is on + static QString at(const Boxes &boxes, int x, int grab = kGrab) { + return describe(LyricsEdit::hitTest(boxes, x, grab)); + } + + // Where Add Word puts a word for a click at frame + static QString newSpan(const sv::EventVector &words, frame_t frame) { + Span s; + if (!LyricsEdit::newWordSpan(words, frame, kRate, s)) return "none"; + return QString("%1-%2").arg(s.start).arg(s.end); + } + + // The same, as expected + static QString span(frame_t start, frame_t end) { + return QString("%1-%2").arg(start).arg(end); + } + + static frame_t startTo(const sv::EventVector &words, int i, frame_t f) { + return LyricsEdit::clampStart(words, i, f, kRate); + } + + static frame_t endTo(const sv::EventVector &words, int i, frame_t f) { + return LyricsEdit::clampEnd(words, i, f, kRate); + } + + static frame_t endOf(const sv::Event &e) { + return e.getFrame() + e.getDuration(); + } + + static frame_t overlap(frame_t s0, frame_t e0, frame_t s1, frame_t e1) { + return std::max(frame_t(0), std::min(e0, e1) - std::max(s0, s1)); + } + +private slots: + // 20 ms and 0.5 s to the nearest frame: whole numbers at the usual + // rates, whatever 0.02 is in binary + void limits_in_frames() { + QCOMPARE(LyricsEdit::minWordSeconds, 0.020); + QCOMPARE(LyricsEdit::newWordSeconds, 0.5); + QCOMPARE(LyricsEdit::minWordFrames(44100), kMin); + QCOMPARE(LyricsEdit::minWordFrames(48000), frame_t(960)); + QCOMPARE(LyricsEdit::minWordFrames(22050), frame_t(441)); + QCOMPARE(LyricsEdit::minWordFrames(96000), frame_t(1920)); + QCOMPARE(LyricsEdit::newWordFrames(44100), kNew); + QCOMPARE(LyricsEdit::newWordFrames(48000), frame_t(24000)); + QCOMPARE(LyricsEdit::minWordFrames(0), frame_t(0)); + QCOMPARE(LyricsEdit::newWordFrames(0), frame_t(0)); + } + + // A box runs from the x of the word's start to the x of its end + void boxes_from_the_word_times() { + sv::EventVector words { word(1000, 2500), word(2500, 2560), + word(5000, 5000) }; + Boxes boxes = LyricsEdit::boxesFor + (words, [](frame_t f) { return int(f / 100) + 7; }); + QCOMPARE(int(boxes.size()), 3); + QCOMPARE(boxes[0].x0, 17); + QCOMPARE(boxes[0].x1, 32); + QCOMPARE(boxes[1].x0, 32); + QCOMPARE(boxes[1].x1, 32); + QCOMPARE(boxes[2].x0, 57); + QCOMPARE(boxes[2].x1, 57); + QVERIFY(LyricsEdit::boxesFor({}, [](frame_t) { return 0; }).empty()); + } + + // A box holds its columns x0 to x1 - 1, and one too short to reach + // a column is painted, and found, in the column x0 + void word_at_a_column() { + Boxes boxes { { 100, 150 }, { 150, 200 }, { 220, 221 }, { 300, 300 } }; + QCOMPARE(LyricsEdit::wordAt(boxes, 99), -1); + QCOMPARE(LyricsEdit::wordAt(boxes, 100), 0); + QCOMPARE(LyricsEdit::wordAt(boxes, 149), 0); + QCOMPARE(LyricsEdit::wordAt(boxes, 150), 1); + QCOMPARE(LyricsEdit::wordAt(boxes, 199), 1); + QCOMPARE(LyricsEdit::wordAt(boxes, 200), -1); + QCOMPARE(LyricsEdit::wordAt(boxes, 220), 2); + QCOMPARE(LyricsEdit::wordAt(boxes, 221), -1); + QCOMPARE(LyricsEdit::wordAt(boxes, 299), -1); + QCOMPARE(LyricsEdit::wordAt(boxes, 300), 3); + QCOMPARE(LyricsEdit::wordAt(boxes, 301), -1); + QCOMPARE(LyricsEdit::wordAt({}, 100), -1); + + // Overlapping: the one that starts latest, painted on top + Boxes crossing { { 100, 200 }, { 150, 250 } }; + QCOMPARE(LyricsEdit::wordAt(crossing, 149), 0); + QCOMPARE(LyricsEdit::wordAt(crossing, 150), 1); + QCOMPARE(LyricsEdit::wordAt(crossing, 199), 1); + QCOMPARE(LyricsEdit::wordAt(crossing, 249), 1); + + Boxes within { { 100, 300 }, { 150, 200 } }; + QCOMPARE(LyricsEdit::wordAt(within, 149), 0); + QCOMPARE(LyricsEdit::wordAt(within, 150), 1); + QCOMPARE(LyricsEdit::wordAt(within, 199), 1); + QCOMPARE(LyricsEdit::wordAt(within, 200), 0); + + // Starting in one column: the later one in order + Boxes together { { 100, 101 }, { 100, 200 } }; + QCOMPARE(LyricsEdit::wordAt(together, 100), 1); + Boxes shortLast { { 100, 150 }, { 100, 100 } }; + QCOMPARE(LyricsEdit::wordAt(shortLast, 100), 1); + QCOMPARE(LyricsEdit::wordAt(shortLast, 101), 0); + } + + // Decision 3: at an edge two words share, the column to its left is + // the earlier word's end, the column to its right the later word's + // start, each for grab columns + void shared_edge_from_each_side() { + Boxes boxes { { 100, 150 }, { 150, 200 } }; + QCOMPARE(at(boxes, 149), QString("end of 0")); + QCOMPARE(at(boxes, 144), QString("end of 0")); + QCOMPARE(at(boxes, 143), QString("inside 0")); + QCOMPARE(at(boxes, 150), QString("start of 1")); + QCOMPARE(at(boxes, 155), QString("start of 1")); + QCOMPARE(at(boxes, 156), QString("inside 1")); + + // The outer edges, from inside and from outside + QCOMPARE(at(boxes, 100), QString("start of 0")); + QCOMPARE(at(boxes, 105), QString("start of 0")); + QCOMPARE(at(boxes, 106), QString("inside 0")); + QCOMPARE(at(boxes, 99), QString("start of 0")); + QCOMPARE(at(boxes, 94), QString("start of 0")); + QCOMPARE(at(boxes, 93), QString("nothing -1")); + QCOMPARE(at(boxes, 199), QString("end of 1")); + QCOMPARE(at(boxes, 194), QString("end of 1")); + QCOMPARE(at(boxes, 193), QString("inside 1")); + QCOMPARE(at(boxes, 200), QString("end of 1")); + QCOMPARE(at(boxes, 205), QString("end of 1")); + QCOMPARE(at(boxes, 206), QString("nothing -1")); + + // With a grab of one column, only the columns touching the edge + QCOMPARE(at(boxes, 149, 1), QString("end of 0")); + QCOMPARE(at(boxes, 150, 1), QString("start of 1")); + QCOMPARE(at(boxes, 148, 1), QString("inside 0")); + QCOMPARE(at(boxes, 151, 1), QString("inside 1")); + } + + // In a box too narrow for both edges' grab columns, the nearer edge, + // and the end where they are equally near + void narrow_boxes() { + Boxes four { { 100, 104 } }; + QCOMPARE(at(four, 100), QString("start of 0")); + QCOMPARE(at(four, 101), QString("start of 0")); + QCOMPARE(at(four, 102), QString("end of 0")); + QCOMPARE(at(four, 103), QString("end of 0")); + + Boxes three { { 100, 103 } }; + QCOMPARE(at(three, 100), QString("start of 0")); + QCOMPARE(at(three, 101), QString("end of 0")); + QCOMPARE(at(three, 102), QString("end of 0")); + + Boxes two { { 100, 102 } }; + QCOMPARE(at(two, 100), QString("start of 0")); + QCOMPARE(at(two, 101), QString("end of 0")); + + // One column wide: its end inside, and each edge from its side + Boxes one { { 100, 101 } }; + QCOMPARE(at(one, 100), QString("end of 0")); + QCOMPARE(at(one, 99), QString("start of 0")); + QCOMPARE(at(one, 94), QString("start of 0")); + QCOMPARE(at(one, 93), QString("nothing -1")); + QCOMPARE(at(one, 101), QString("end of 0")); + QCOMPARE(at(one, 106), QString("end of 0")); + QCOMPARE(at(one, 107), QString("nothing -1")); + QCOMPARE(at(one, 100, 1), QString("end of 0")); + QCOMPARE(at(one, 99, 1), QString("start of 0")); + QCOMPARE(at(one, 101, 1), QString("end of 0")); + QCOMPARE(at(one, 98, 1), QString("nothing -1")); + QCOMPARE(at(one, 102, 1), QString("nothing -1")); + + // No width at all is painted one column wide, and is the same + Boxes none { { 100, 100 } }; + QCOMPARE(at(none, 100), QString("end of 0")); + QCOMPARE(at(none, 99), QString("start of 0")); + QCOMPARE(at(none, 101), QString("end of 0")); + + // A one-column word between two others: its start is under the + // earlier word's end, as painted + Boxes squeezed { { 50, 100 }, { 100, 101 }, { 101, 150 } }; + QCOMPARE(at(squeezed, 99), QString("end of 0")); + QCOMPARE(at(squeezed, 100), QString("end of 1")); + QCOMPARE(at(squeezed, 101), QString("start of 2")); + } + + // Overlapping words from a file: the box that starts latest is the + // one the pointer is in, and its edges are the ones found + void overlapping_boxes() { + Boxes crossing { { 100, 200 }, { 150, 250 } }; + QCOMPARE(at(crossing, 101), QString("start of 0")); + QCOMPARE(at(crossing, 149), QString("inside 0")); + QCOMPARE(at(crossing, 150), QString("start of 1")); + QCOMPARE(at(crossing, 155), QString("start of 1")); + QCOMPARE(at(crossing, 160), QString("inside 1")); + // The earlier word's end is under the later word + QCOMPARE(at(crossing, 199), QString("inside 1")); + QCOMPARE(at(crossing, 200), QString("inside 1")); + QCOMPARE(at(crossing, 249), QString("end of 1")); + QCOMPARE(at(crossing, 252), QString("end of 1")); + + Boxes within { { 100, 300 }, { 150, 200 } }; + QCOMPARE(at(within, 150), QString("start of 1")); + QCOMPARE(at(within, 199), QString("end of 1")); + QCOMPARE(at(within, 200), QString("inside 0")); + QCOMPARE(at(within, 298), QString("end of 0")); + + // Two starting in one column, or ending in one: the later word + // in order, from inside and from outside alike + Boxes together { { 100, 101 }, { 100, 200 } }; + QCOMPARE(at(together, 100), QString("start of 1")); + QCOMPARE(at(together, 99), QString("start of 1")); + Boxes endTogether { { 100, 200 }, { 150, 200 } }; + QCOMPARE(at(endTogether, 199), QString("end of 1")); + QCOMPARE(at(endTogether, 200), QString("end of 1")); + } + + // Between boxes, the nearest edge within grab columns, and the end + // where the two are equally near + void gap_between_boxes() { + Boxes wide { { 100, 150 }, { 170, 200 } }; + QCOMPARE(at(wide, 150), QString("end of 0")); + QCOMPARE(at(wide, 155), QString("end of 0")); + QCOMPARE(at(wide, 156), QString("nothing -1")); + QCOMPARE(at(wide, 163), QString("nothing -1")); + QCOMPARE(at(wide, 164), QString("start of 1")); + QCOMPARE(at(wide, 169), QString("start of 1")); + + // An odd gap: its middle column is as near to both + Boxes odd { { 100, 150 }, { 155, 200 } }; + QCOMPARE(at(odd, 150), QString("end of 0")); + QCOMPARE(at(odd, 151), QString("end of 0")); + QCOMPARE(at(odd, 152), QString("end of 0")); + QCOMPARE(at(odd, 153), QString("start of 1")); + QCOMPARE(at(odd, 154), QString("start of 1")); + + Boxes even { { 100, 150 }, { 154, 200 } }; + QCOMPARE(at(even, 150), QString("end of 0")); + QCOMPARE(at(even, 151), QString("end of 0")); + QCOMPARE(at(even, 152), QString("start of 1")); + QCOMPARE(at(even, 153), QString("start of 1")); + } + + void nothing_near() { + QCOMPARE(at({}, 0), QString("nothing -1")); + QCOMPARE(at({}, 100), QString("nothing -1")); + + Boxes boxes { { 100, 150 }, { 150, 200 } }; + QCOMPARE(at(boxes, 0), QString("nothing -1")); + QCOMPARE(at(boxes, -50), QString("nothing -1")); + QCOMPARE(at(boxes, 1000), QString("nothing -1")); + + // No grab, no edges + QCOMPARE(at(boxes, 100, 0), QString("inside 0")); + QCOMPARE(at(boxes, 149, 0), QString("inside 0")); + QCOMPARE(at(boxes, 150, 0), QString("inside 1")); + QCOMPARE(at(boxes, 99, 0), QString("nothing -1")); + QCOMPARE(at(boxes, 200, 0), QString("nothing -1")); + } + + // Words that do not overlap: a start between the previous word's end + // (or frame 0) and 20 ms before its own end; an end between 20 ms + // after its start and the next word's start, or anywhere after the + // last word + void clamps_between_neighbours() { + sv::EventVector words { word(0, 10000), word(12000, 20000), + word(20000, 30000) }; + + QCOMPARE(startTo(words, 1, 11000), frame_t(11000)); + QCOMPARE(startTo(words, 1, 12000), frame_t(12000)); + QCOMPARE(startTo(words, 1, 10000), frame_t(10000)); + QCOMPARE(startTo(words, 1, 9999), frame_t(10000)); + QCOMPARE(startTo(words, 1, -500), frame_t(10000)); + QCOMPARE(startTo(words, 1, 20000 - kMin), 20000 - kMin); + QCOMPARE(startTo(words, 1, 20000 - kMin + 1), 20000 - kMin); + QCOMPARE(startTo(words, 1, 25000), 20000 - kMin); + + // Touching the word before: no further back at all + QCOMPARE(startTo(words, 2, 19000), frame_t(20000)); + QCOMPARE(startTo(words, 2, 29500), 30000 - kMin); + + // The first word: not before frame 0 + QCOMPARE(startTo(words, 0, -100), frame_t(0)); + QCOMPARE(startTo(words, 0, 5000), frame_t(5000)); + QCOMPARE(startTo(words, 0, 9500), 10000 - kMin); + sv::EventVector later { word(5000, 8000) }; + QCOMPARE(startTo(later, 0, 100), frame_t(100)); + QCOMPARE(startTo(later, 0, -100), frame_t(0)); + + QCOMPARE(endTo(words, 0, 11000), frame_t(11000)); + QCOMPARE(endTo(words, 0, 12000), frame_t(12000)); + QCOMPARE(endTo(words, 0, 12001), frame_t(12000)); + QCOMPARE(endTo(words, 0, 13000), frame_t(12000)); + QCOMPARE(endTo(words, 0, kMin), kMin); + QCOMPARE(endTo(words, 0, kMin - 1), kMin); + QCOMPARE(endTo(words, 0, -500), kMin); + + // Touching the word after: no further forward at all + QCOMPARE(endTo(words, 1, 21000), frame_t(20000)); + QCOMPARE(endTo(words, 1, 19000), frame_t(19000)); + QCOMPARE(endTo(words, 1, 12500), 12000 + kMin); + + // The last word: no limit forward + QCOMPARE(endTo(words, 2, 1000000), frame_t(1000000)); + QCOMPARE(endTo(words, 2, 20100), 20000 + kMin); + } + + // The dragged word keeps its other edge, its text and its line + void dragged_word_keeps_the_rest() { + sv::EventVector words { word(0, 10000, 2.f, "hello"), + word(12000, 20000, 2.f, "world") }; + + QCOMPARE(describe(LyricsEdit::startDraggedTo(words, 1, 11000, kRate)), + describe(word(11000, 20000, 2.f, "world"))); + QCOMPARE(describe(LyricsEdit::startDraggedTo(words, 1, 9000, kRate)), + describe(word(10000, 20000, 2.f, "world"))); + QCOMPARE(describe(LyricsEdit::startDraggedTo(words, 1, 19900, kRate)), + describe(word(20000 - kMin, 20000, 2.f, "world"))); + QCOMPARE(describe(LyricsEdit::endDraggedTo(words, 0, 11000, kRate)), + describe(word(0, 11000, 2.f, "hello"))); + QCOMPARE(describe(LyricsEdit::endDraggedTo(words, 0, 13000, kRate)), + describe(word(0, 12000, 2.f, "hello"))); + QCOMPARE(describe(LyricsEdit::endDraggedTo(words, 1, 50000, kRate)), + describe(word(12000, 50000, 2.f, "world"))); + + // Not dragged anywhere: the same event, which a command folds + // away + QVERIFY(LyricsEdit::startDraggedTo(words, 1, 12000, kRate) == words[1]); + QVERIFY(LyricsEdit::endDraggedTo(words, 1, 20000, kRate) == words[1]); + + // No such word: nothing to clamp, and no crash + QCOMPARE(startTo(words, 2, 500), frame_t(500)); + QCOMPARE(endTo(words, -1, 500), frame_t(500)); + } + + // Decision 6: an overlap already in a file may stay, and shrink, but + // never grows; the limits come from the words as they were when the + // drag began, so a drag back puts the word back + void clamps_with_an_overlap_already_there() { + sv::EventVector crossing { word(0, 15000), word(12000, 20000), + word(25000, 30000) }; + QCOMPARE(startTo(crossing, 1, 11000), frame_t(12000)); + QCOMPARE(startTo(crossing, 1, 13000), frame_t(13000)); + QCOMPARE(startTo(crossing, 1, 16000), frame_t(16000)); + QCOMPARE(endTo(crossing, 0, 16000), frame_t(15000)); + QCOMPARE(endTo(crossing, 0, 14000), frame_t(14000)); + QCOMPARE(endTo(crossing, 0, 11000), frame_t(11000)); + QCOMPARE(endTo(crossing, 1, 22000), frame_t(22000)); + QCOMPARE(endTo(crossing, 1, 26000), frame_t(25000)); + + // One word inside another: the inner one only shrinks, the + // outer one's end is not held by it + sv::EventVector within { word(0, 30000), word(10000, 12000), + word(40000, 50000) }; + QCOMPARE(startTo(within, 1, 5000), frame_t(10000)); + QCOMPARE(startTo(within, 1, 11500), 12000 - kMin); + QCOMPARE(endTo(within, 1, 20000), frame_t(12000)); + QCOMPARE(endTo(within, 1, 11000), frame_t(11000)); + QCOMPARE(endTo(within, 0, 35000), frame_t(35000)); + QCOMPARE(endTo(within, 0, 45000), frame_t(40000)); + QCOMPARE(endTo(within, 0, 11000), frame_t(11000)); + + // Two words at one frame, as an LRC file gives them: the first + // one frame long. A word starting at the same frame holds + // neither start back, but it does hold the short one's end + sv::EventVector together { word(0, 5000), word(10000, 10001), + word(10000, 20000) }; + QCOMPARE(startTo(together, 2, 7000), frame_t(7000)); + QCOMPARE(startTo(together, 2, 4000), frame_t(5000)); + QCOMPARE(startTo(together, 1, 3000), frame_t(5000)); + QCOMPARE(endTo(together, 1, 15000), frame_t(10001)); + QCOMPARE(endTo(together, 2, 25000), frame_t(25000)); + + // The same word twice: each is free of the other + sv::EventVector twice { word(10000, 20000), word(10000, 20000) }; + QCOMPARE(startTo(twice, 0, 5000), frame_t(5000)); + QCOMPARE(endTo(twice, 1, 25000), frame_t(25000)); + } + + // A word already shorter than 20 ms does not get shorter, but can + // be made longer; once it is, the 20 ms hold + void clamps_of_a_word_under_20ms() { + sv::EventVector words { word(0, 9000), word(10000, 10400), + word(11000, 20000) }; + QCOMPARE(startTo(words, 1, 10200), frame_t(10000)); + QCOMPARE(startTo(words, 1, 12000), frame_t(10000)); + QCOMPARE(endTo(words, 1, 10200), frame_t(10400)); + QCOMPARE(endTo(words, 1, 0), frame_t(10400)); + QCOMPARE(startTo(words, 1, 9500), frame_t(9500)); + QCOMPARE(startTo(words, 1, 8000), frame_t(9000)); + QCOMPARE(endTo(words, 1, 10800), frame_t(10800)); + QCOMPARE(endTo(words, 1, 12000), frame_t(11000)); + + sv::EventVector longer { word(0, 9000), word(9500, 10800), + word(11000, 20000) }; + QCOMPARE(startTo(longer, 1, 10500), 10800 - kMin); + + // A word of no length (never made by the parsers) stays at no + // length, not less + sv::EventVector none { word(0, 9000), word(10000, 10000), + word(11000, 20000) }; + QCOMPARE(startTo(none, 1, 10500), frame_t(10000)); + QCOMPARE(endTo(none, 1, 9500), frame_t(10000)); + QCOMPARE(startTo(none, 1, 9500), frame_t(9500)); + QCOMPARE(endTo(none, 1, 10500), frame_t(10500)); + } + + // Whatever the words and wherever the pointer goes: the edge ends up + // between where it was and where it was dragged, the word is never + // shorter than 20 ms or than it was, and no overlap grows + void no_drag_jumps_or_grows_an_overlap() { + const std::vector cases { + { word(0, 10000), word(10000, 20000), word(20000, 30000) }, + { word(1000, 5000), word(8000, 9000), word(15000, 40000) }, + { word(0, 15000), word(12000, 20000), word(25000, 30000) }, + { word(0, 30000), word(10000, 12000), word(40000, 50000) }, + { word(0, 5000), word(10000, 10001), word(10000, 20000) }, + { word(10000, 20000), word(10000, 20000) }, + { word(0, 9000), word(10000, 10400), word(11000, 20000) }, + { word(0, 9000), word(10000, 10000), word(11000, 20000) }, + { word(0, 20000), word(5000, 25000), word(8000, 12000), + word(30000, 30500) }, + }; + + const frame_t minimum = LyricsEdit::minWordFrames(kRate); + + for (int c = 0; c < int(cases.size()); ++c) { + const sv::EventVector &words = cases[c]; + + std::vector targets; + for (frame_t f = -2000; f < 55000; f += 97) targets.push_back(f); + for (const sv::Event &e : words) { + for (frame_t f : { e.getFrame(), endOf(e) }) { + for (frame_t d : { frame_t(-1), frame_t(0), frame_t(1), + -kMin, kMin }) { + targets.push_back(f + d); + } + } + } + + for (int i = 0; i < int(words.size()); ++i) { + const frame_t s = words[i].getFrame(); + const frame_t e = endOf(words[i]); + const frame_t shortest = + std::min(e - s, minimum); + + for (frame_t wanted : targets) { + for (bool start : { true, false }) { + sv::Event moved = start ? + LyricsEdit::startDraggedTo(words, i, wanted, kRate) : + LyricsEdit::endDraggedTo(words, i, wanted, kRate); + const frame_t ms = moved.getFrame(); + const frame_t me = endOf(moved); + const frame_t edge = start ? ms : me; + const frame_t was = start ? s : e; + const frame_t other = start ? me : ms; + const frame_t otherWas = start ? e : s; + + const QString where = + QString("case %1, word %2, %3 to %4: %5-%6") + .arg(c).arg(i).arg(start ? "start" : "end") + .arg(wanted).arg(ms).arg(me); + + QVERIFY2(other == otherWas, qPrintable(where)); + QVERIFY2(edge >= std::min(was, wanted) && + edge <= std::max(was, wanted), + qPrintable(where)); + QVERIFY2(moved.getDuration() >= shortest, + qPrintable(where)); + QVERIFY2(ms >= 0, qPrintable(where)); + for (int j = 0; j < int(words.size()); ++j) { + if (j == i) continue; + const frame_t js = words[j].getFrame(); + const frame_t je = endOf(words[j]); + QVERIFY2(overlap(ms, me, js, je) <= + overlap(s, e, js, je), + qPrintable(where + QString(", word %1") + .arg(j))); + } + } + } + } + } + } + + // Decision 9: 0.5 s from the click where there is room, up to the + // next word if that is nearer, moved back into the gap as far as + // that makes it longer + void new_word_where_there_is_room() { + const frame_t S = kSecond; + sv::EventVector words { word(0, S), word(3 * S, 4 * S) }; + + QCOMPARE(newSpan(words, S + S / 2), span(S + S / 2, S + S / 2 + kNew)); + QCOMPARE(newSpan(words, 2 * S), span(2 * S, 2 * S + kNew)); + + // At exactly a word's end: the gap after it begins there + QCOMPARE(newSpan(words, S), span(S, S + kNew)); + + // Near the next word: 0.5 s ending at it + QCOMPARE(newSpan(words, 3 * S - 8820), span(3 * S - kNew, 3 * S)); + QCOMPARE(newSpan(words, 3 * S - 1), span(3 * S - kNew, 3 * S)); + QCOMPARE(newSpan(words, 3 * S - kNew), span(3 * S - kNew, 3 * S)); + QCOMPARE(newSpan(words, 3 * S - kNew - 1), span(3 * S - kNew - 1, 3 * S - 1)); + } + + void new_word_with_little_room() { + const frame_t S = kSecond; + + // A gap shorter than 0.5 s: the whole of it, wherever the click + sv::EventVector words { word(0, S), word(S + 10000, 2 * S) }; + QCOMPARE(newSpan(words, S), span(S, S + 10000)); + QCOMPARE(newSpan(words, S + 5000), span(S, S + 10000)); + QCOMPARE(newSpan(words, S + 9999), span(S, S + 10000)); + + // 20 ms is room enough; less is not + sv::EventVector just { word(0, S), word(S + kMin, 2 * S) }; + QCOMPARE(newSpan(just, S + 400), span(S, S + kMin)); + sv::EventVector under { word(0, S), word(S + kMin - 1, 2 * S) }; + QCOMPARE(newSpan(under, S), QString("none")); + QCOMPARE(newSpan(under, S + 400), QString("none")); + } + + // Add Word is for empty space: none in a word, before frame 0, or + // without a sample rate + void new_word_none_in_a_word() { + const frame_t S = kSecond; + sv::EventVector words { word(0, S), word(3 * S, 4 * S) }; + QCOMPARE(newSpan(words, 0), QString("none")); + QCOMPARE(newSpan(words, S / 2), QString("none")); + QCOMPARE(newSpan(words, S - 1), QString("none")); + QCOMPARE(newSpan(words, 3 * S), QString("none")); + QCOMPARE(newSpan(words, 4 * S - 1), QString("none")); + QCOMPARE(newSpan({}, -1), QString("none")); + + Span s; + s.start = 7; + s.end = 9; + QVERIFY(!LyricsEdit::newWordSpan(words, 2 * S, 0, s)); + QVERIFY(!LyricsEdit::newWordSpan(words, S / 2, kRate, s)); + QCOMPARE(s.start, frame_t(7)); + QCOMPARE(s.end, frame_t(9)); + } + + // Before the first word the gap starts at frame 0; after the last + // it has no end + void new_word_before_the_first_and_after_the_last() { + const frame_t S = kSecond; + sv::EventVector words { word(3 * S, 4 * S) }; + QCOMPARE(newSpan(words, 0), span(0, kNew)); + QCOMPARE(newSpan(words, S), span(S, S + kNew)); + QCOMPARE(newSpan(words, 4 * S), span(4 * S, 4 * S + kNew)); + QCOMPARE(newSpan(words, 10 * S), span(10 * S, 10 * S + kNew)); + + // A first word 0.3 s in: the new one fills the 0.3 s before it + sv::EventVector early { word(13230, S) }; + QCOMPARE(newSpan(early, 4410), span(0, 13230)); + sv::EventVector tooEarly { word(kMin - 1, S) }; + QCOMPARE(newSpan(tooEarly, 100), QString("none")); + + QCOMPARE(newSpan({}, 1000), span(1000, 1000 + kNew)); + QCOMPARE(newSpan({}, 0), span(0, kNew)); + } + + // Overlapping words from a file: the gap is between the latest end + // before the click and the earliest start after it + void new_word_among_overlapping_words() { + sv::EventVector crossing { word(0, 30000), word(10000, 50000), + word(60000, 100000) }; + QCOMPARE(newSpan(crossing, 40000), QString("none")); + QCOMPARE(newSpan(crossing, 55000), span(50000, 60000)); + + sv::EventVector within { word(0, 50000), word(10000, 20000), + word(60000, 100000) }; + QCOMPARE(newSpan(within, 30000), QString("none")); + QCOMPARE(newSpan(within, 55000), span(50000, 60000)); + } + + // Decision 10: the line of the nearer neighbour by the gap, of the + // word before on a tie, of the only one if one, 0 if none + void new_word_line() { + sv::EventVector words { word(0, 10000, 0.f), word(30000, 40000, 1.f) }; + auto line = [&](frame_t s, frame_t e) { + Span added; + added.start = s; + added.end = e; + return LyricsEdit::newWordLine(words, added); + }; + QCOMPARE(line(12000, 14000), 0.f); + QCOMPARE(line(25000, 28000), 1.f); + QCOMPARE(line(15000, 25000), 0.f); + QCOMPARE(line(15000, 24999), 0.f); + QCOMPARE(line(15001, 25000), 1.f); + QCOMPARE(line(10000, 30000), 0.f); + QCOMPARE(line(50000, 60000), 1.f); + + words = { word(20000, 30000, 3.f) }; + QCOMPARE(line(0, 10000), 3.f); + QCOMPARE(line(40000, 50000), 3.f); + + words = {}; + QCOMPARE(line(0, 10000), 0.f); + + // The word before is the one ending latest, not the last to + // start + words = { word(0, 50000, 4.f), word(10000, 20000, 5.f), + word(80000, 90000, 6.f) }; + QCOMPARE(line(52000, 60000), 4.f); + QCOMPARE(line(70000, 78000), 6.f); + + // From newWordSpan(), as the editor uses them together: just + // after a line's last word, just before the next line's first, + // and filling a gap between the two wholly + const frame_t S = kSecond; + words = { word(0, S, 0.f), word(2 * S, 3 * S, 1.f), + word(6 * S, 7 * S, 2.f), word(7 * S + 10000, 8 * S, 3.f) }; + Span added; + QVERIFY(LyricsEdit::newWordSpan(words, S + 1000, kRate, added)); + QCOMPARE(span(added.start, added.end), span(S + 1000, S + 1000 + kNew)); + QCOMPARE(LyricsEdit::newWordLine(words, added), 0.f); + QVERIFY(LyricsEdit::newWordSpan(words, 2 * S - 1000, kRate, added)); + QCOMPARE(span(added.start, added.end), span(2 * S - kNew, 2 * S)); + QCOMPARE(LyricsEdit::newWordLine(words, added), 1.f); + QVERIFY(LyricsEdit::newWordSpan(words, 4 * S, kRate, added)); + QCOMPARE(LyricsEdit::newWordLine(words, added), 1.f); + QVERIFY(LyricsEdit::newWordSpan(words, 6 * S - 1000, kRate, added)); + QCOMPARE(LyricsEdit::newWordLine(words, added), 2.f); + QVERIFY(LyricsEdit::newWordSpan(words, 7 * S + 5000, kRate, added)); + QCOMPARE(span(added.start, added.end), span(7 * S, 7 * S + 10000)); + QCOMPARE(LyricsEdit::newWordLine(words, added), 2.f); + } + + // Decision 11: typed text is cleaned as the parsers clean labels + void typed_text_cleaned() { + QCOMPARE(LyricsEdit::cleanText("hello"), QString("hello")); + QCOMPARE(LyricsEdit::cleanText(" hello \t"), QString("hello")); + QCOMPARE(LyricsEdit::cleanText("a\tb"), QString("a b")); + QCOMPARE(LyricsEdit::cleanText(QString("a\x01" "b\x7f" "c")), + QString("abc")); + QCOMPARE(LyricsEdit::cleanText(QString("a") + QChar(0xFFFE) + "b"), + QString("ab")); + QCOMPARE(LyricsEdit::cleanText("a\nb"), QString("ab")); + QCOMPARE(LyricsEdit::cleanText("&"), QString("&")); + QCOMPARE(LyricsEdit::cleanText(QString::fromUtf8("syd\xc3\xa4n")), + QString::fromUtf8("syd\xc3\xa4n")); + + QString longText(Lyrics::maxLabelLength + 50, QChar('x')); + QCOMPARE(LyricsEdit::cleanText(longText).size(), + qsizetype(Lyrics::maxLabelLength)); + + // Nothing left: empty, which the edit refuses + QCOMPARE(LyricsEdit::cleanText(""), QString()); + QVERIFY(LyricsEdit::cleanText(" ").isEmpty()); + QVERIFY(LyricsEdit::cleanText("\t\n").isEmpty()); + QVERIFY(LyricsEdit::cleanText(QString("\x01\x02")).isEmpty()); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 65f102c5..b3024eef 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -22,6 +22,7 @@ #include "TestTakeTiming.h" #include "TestLyrics.h" #include "TestLyricsTtml.h" +#include "TestLyricsEdit.h" #include "RunSuite.h" @@ -112,6 +113,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLyricsEdit t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index c4642d39..543f31b9 100644 --- a/meson.build +++ b/meson.build @@ -1096,6 +1096,7 @@ tony_entry_files = [ tony_core_files = [ 'main/Coverage.cpp', 'main/Lyrics.cpp', + 'main/LyricsEdit.cpp', 'main/LyricsTtml.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', @@ -1362,6 +1363,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakeTiming.h', 'main/test/TestLyrics.h', 'main/test/TestLyricsTtml.h', + 'main/test/TestLyricsEdit.h', ]) # Where the suites find the files in testdata/. Forward slashes: a From 9b1fb6c2183b2f5bd9374046682bddd4820780a6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:45:33 +0000 Subject: [PATCH 121/275] feat: Playback > Calibrate Audio, a dialog over the audio check The dialog tells the user what to hold where, shows the check's progress with Cancel, and reads the result out in plain words with the fix for each failure: the measured round trip against the driver's, where each punch-in landed and how far they disagree, both sample rates, the input peak and any echo. Use this latency stores the figure under the devices the check started with, so a device changed while the dialog is open cannot take it. The Playback menu gains the entry, a line saying which latency takes use, and Forget Measured Latency; the entry is greyed during takes and checks, the device menus during a check. Storing under the devices at press time, allowing it during a take, and a close that does not cancel were each seen to fail a test. Core suite: all green but the four known TestTakesFile Windows-path tests on Linux. App suite: green (TestAudioCheck 17, TestRecordWorkflow 96, TestSingingAnalysis 20, TestSingingDocument 12). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 14 + docs/calibrate-audio.md | 2 +- main/AudioCheckRunner.cpp | 19 + main/AudioCheckRunner.h | 12 + main/CalibrateAudioDialog.cpp | 634 ++++++++++++++++++++++++++++ main/CalibrateAudioDialog.h | 150 +++++++ main/MainWindow.cpp | 87 +++- main/MainWindow.h | 27 +- main/test/TestAudioCheck.h | 304 +++++++++++++ main/test/TestRecordWorkflow.h | 12 + meson.build | 2 + 11 files changed, 1253 insertions(+), 10 deletions(-) create mode 100644 main/CalibrateAudioDialog.cpp create mode 100644 main/CalibrateAudioDialog.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index c823b945..55046a62 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -526,3 +526,17 @@ Left open: the default reference path is exercised only through `nextReferencePa ### Lead — 2026-09-26, after B3 - De-raced `TestSingingAnalysis::waitForRange()` (`2a306fb`): `initialAnalysisCompleted()` also fires from `layerCompletionChanged()` before a ranged merge; it failed once in a full run. - For phase D's `open-points.md`, older bugs B3 found: (a) in a window's first file the toolbar level control's notches move the pitch/notes gain 0.5 → 0.562, force both audible and write that to the settings; (b) `audible-0` (Play Audio) is overridden on load by `audible-3` (the spectrogram layer on the same model, loaded last). Also B1's: closing a session during an ordinary take, then Stop, hangs. + +### Phase B4 — 2026-09-26 +Built: `main/CalibrateAudioDialog.{h,cpp}` (`tony_app`), a `QDialog`, not modal, with three pages (instructions, progress, result): `present()`, `startCheck()` (Start and Check Again), `cancelCheck()`, `useLatency()`, `showResult()`, `reject()`; for tests `page()`, `pageText()`, `canUseLatency()`, `setPlan()`; static `describeLatency(InUse)`. `AudioCheckRunner::calibrationPlan()` (4 × 3). `AudioCheckResult::key`: the Preferences' devices taken in `start()`, the rate set in `end()`; `storeMeasuredLatency()` stores under it. `MainWindow`: Playback ▸ Calibrate Audio..., a disabled line "Latency: ...", Forget Measured Latency, after the device submenus; `calibrateAudio()`, `updateLatencyMenuLine()`. `updateMenuStates()` shuts Calibrate Audio during any take or check, and both device menus during a check; the runner's `progress` and `finished` call it. `TestAudioCheck`: 5 tests `calibrate_audio_*`, one full run (13 s); `TestMainWindow` accessors. +Choices / deviations: +- The window owns the dialog, makes it on first use, and deletes it in `~MainWindow` before the runner. The dialog calls only `latencyInUse()` and `storeMeasuredLatency()`, and shows only runs it started itself. +- Closing it (title bar, Esc, Close) while its check runs cancels the check. Opened again, it shows the instructions. +- Menu line: "Latency: measured 281 ms, 26 Sep" (the year only when not this one), "Latency: driver's figure, 279 ms", plus " (the measured one is out of date)" when stale, or "not known yet" while the device reports 0. Forget is enabled while a figure is kept, stale or not. The line is refreshed on the menu's `aboutToShow`, in `updateMenuStates()`, and by store and forget. +- Result page: one sentence for the verdict, then its fix. A rate mismatch replaces the verdict's words, and an echo adds a paragraph. Then a table: the round trip measured (not for NoSignal or a rate mismatch) against the driver's (out + in), what the takes were placed with, each punch-in's offset, the spread, found of judged, both rates, the input peak, the echo, and the devices. The text is selectable, to copy. +- Progress: the runner's seconds left plus `kSecondsPerAnalysis` = 3 s for each analysis to come. The bar never goes back. +- Nothing new asks the user, so there is no new seam: Forget asks nothing. +The next phase must know: +- A run replacing a check session asks "Session modified: save?" (its takes mark it modified). Check Again always meets it; answer No. The runner could skip the question for its own reference's session. +- Not on the result page: §2's mic channel and noise floor. The runner measures neither. +Left open: Record stays enabled during a check. Pressing it there goes through the Stop path and ends the check's take early; what the run then makes of it was not tried. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index b46272d3..ec09d597 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -292,7 +292,7 @@ marked "Done" when it is committed. dialog needs it. Done. - **B3** The check's playback: the reference centred at −12 dBFS, the sonification silent, and a progress signal. Done. - - **B4** The Calibrate Audio dialog and menu entry. + - **B4** The Calibrate Audio dialog and menu entry. Done. **Then you run it on your PC.** Its numbers settle three things: how wrong the driver's figure is, whether the offset holds across stream restarts on MME, and diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index 3cfb7771..da468a1f 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -36,6 +36,7 @@ #include #include #include +#include #include #include #include @@ -89,6 +90,16 @@ AudioCheckRunner::~AudioCheckRunner() } } +AudioCheckRunner::Plan +AudioCheckRunner::calibrationPlan() +{ + Plan plan; + plan.layout = LatencyCheck::calibrationLayout(); + plan.punchIns = 4; + plan.eventsEach = 3; + return plan; +} + QString AudioCheckRunner::referenceDirectory() { @@ -144,6 +155,13 @@ AudioCheckRunner::start(const Plan &plan) m_plan = plan; m_result = AudioCheckResult(); m_reported = Progress(); + + // The devices the takes will be recorded on. The window's device + // menus are shut while the run lasts; the rate comes with the takes + { + QSettings settings; + m_result.key = LatencyCalibration::currentKey(settings, 0); + } m_punchIns = punchIns; m_starts.clear(); m_ends.clear(); @@ -539,6 +557,7 @@ AudioCheckRunner::end(QString failure) m_result.reportedOutputLatency = first.reportedOutput; m_result.reportedInputLatency = first.reportedInput; m_result.recordingRate = first.recordingRate; + m_result.key.rate = first.recordingRate; m_result.rateMismatch = m_result.recordingRate > 0 && m_result.referenceRate > 0 && m_result.recordingRate != m_result.referenceRate; diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index d23b7ccd..6dd5d316 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -15,6 +15,7 @@ #ifndef TONY_AUDIO_CHECK_RUNNER_H #define TONY_AUDIO_CHECK_RUNNER_H +#include "LatencyCalibration.h" #include "LatencyCheck.h" #include "LatencyUtils.h" @@ -66,6 +67,13 @@ struct AudioCheckResult /// (LatencyCheck::calibratedRoundTrip()); see calibrationUsable() double calibratedRoundTrip; + /// What the figure is kept under (MainWindow::storeMeasuredLatency()): + /// the devices as the Preferences named them when the run started, + /// and the rate the takes were recorded at. Not the devices named + /// when the figure is kept: the result is on show for as long as the + /// user likes, and another device may have been chosen by then + LatencyCalibration::Key key; + /// Whether calibratedRoundTrip means anything: the run was judged /// Ok or Unsteady, at the reference's rate bool calibrationUsable() const; @@ -176,6 +184,10 @@ class AudioCheckRunner : public QObject explicit AudioCheckRunner(MainWindow *window); virtual ~AudioCheckRunner(); + /// The plan of Playback > Calibrate Audio: four punch-ins of three + /// events each on the calibration layout + static Plan calibrationPlan(); + /// Where the reference is written unless the plan names a file: /// the application's data directory static QString referenceDirectory(); diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp new file mode 100644 index 00000000..d2978b9d --- /dev/null +++ b/main/CalibrateAudioDialog.cpp @@ -0,0 +1,634 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "CalibrateAudioDialog.h" + +#include "MainWindow.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +using std::cerr; +using std::endl; + +using namespace sv; + +namespace { + +using LatencyCheck::Verdict; + +// How long a run of the plan takes, roughly: the recording, as the +// runner counts what is still to come, and each analysis waited for +double +expectedSeconds(const AudioCheckRunner::Plan &plan) +{ + const std::vector ranges = LatencyCheck::punchInsFor + (plan.layout, plan.punchIns, plan.eventsEach); + double seconds = 0.0; + for (const LatencyCheck::PunchIn &r : ranges) { + seconds += std::min(AudioCheckRunner::kPreRollSeconds, r.start) + + (r.end - r.start); + } + return seconds + CalibrateAudioDialog::kSecondsPerAnalysis * + double(ranges.size() + 1); +} + +// How far the offsets disagree, as judgeTake() looks at it for Unsteady +// and Scattered: across punch-ins or within one, whichever is more +double +timingSpread(const LatencyCheck::TakeSummary &s) +{ + double spread = s.spread; + for (const LatencyCheck::PunchInResult &p : s.punchIns) { + if (p.found > 0) spread = std::max(spread, p.spread); + } + return spread; +} + +QString +paragraph(QString html) +{ + return "

" + html + "

"; +} + +QString +bold(QString html) +{ + return "" + html + ""; +} + +} + +CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, + AudioCheckRunner *runner) : + QDialog(window), + m_window(window), + m_runner(runner), + m_plan(AudioCheckRunner::calibrationPlan()), + m_running(false), + m_latencyKept(false), + m_expectedSeconds(0), + m_shownPermille(0) +{ + setWindowTitle(tr("Calibrate Audio")); + setModal(false); + + QVBoxLayout *layout = new QVBoxLayout; + setLayout(layout); + + m_pages = new QStackedWidget; + layout->addWidget(m_pages); + + auto textLabel = []() { + QLabel *label = new QLabel; + label->setWordWrap(true); + label->setTextFormat(Qt::RichText); + label->setAlignment(Qt::AlignLeft | Qt::AlignTop); + return label; + }; + + m_instructions = textLabel(); + m_pages->addWidget(m_instructions); + + QWidget *progress = new QWidget; + QVBoxLayout *progressLayout = new QVBoxLayout; + progress->setLayout(progressLayout); + m_step = new QLabel; + m_step->setWordWrap(true); + m_bar = new QProgressBar; + m_bar->setRange(0, 1000); + m_bar->setTextVisible(false); + m_timeLeft = new QLabel; + progressLayout->addWidget(m_step); + progressLayout->addWidget(m_bar); + progressLayout->addWidget(m_timeLeft); + progressLayout->addStretch(1); + m_pages->addWidget(progress); + + // Selectable, so that the figures can be copied and passed on + m_resultText = textLabel(); + m_resultText->setTextInteractionFlags(Qt::TextSelectableByMouse); + m_pages->addWidget(m_resultText); + + QHBoxLayout *buttons = new QHBoxLayout; + m_useButton = new QPushButton(tr("Use this latency")); + m_againButton = new QPushButton(tr("Check Again")); + m_startButton = new QPushButton(tr("Start")); + m_cancelButton = new QPushButton(tr("Cancel")); + m_closeButton = new QPushButton(tr("Close")); + buttons->addWidget(m_useButton); + buttons->addStretch(1); + buttons->addWidget(m_againButton); + buttons->addWidget(m_startButton); + buttons->addWidget(m_cancelButton); + buttons->addWidget(m_closeButton); + layout->addLayout(buttons); + + connect(m_startButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::startCheck); + connect(m_againButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::startCheck); + connect(m_cancelButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::cancelCheck); + connect(m_useButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::useLatency); + connect(m_closeButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::reject); + + // Direct: the runner's signals carry types with no metatype + connect(m_runner, &AudioCheckRunner::progress, + this, &CalibrateAudioDialog::runnerProgress); + connect(m_runner, &AudioCheckRunner::finished, + this, &CalibrateAudioDialog::runnerFinished); + + setMinimumWidth(520); + showPage(Page::Instructions); +} + +CalibrateAudioDialog::~CalibrateAudioDialog() +{ +} + +void +CalibrateAudioDialog::setPlan(const AudioCheckRunner::Plan &plan) +{ + m_plan = plan; +} + +CalibrateAudioDialog::Page +CalibrateAudioDialog::page() const +{ + return Page(m_pages->currentIndex()); +} + +QString +CalibrateAudioDialog::pageText() const +{ + QTextDocument document; + switch (page()) { + case Page::Instructions: + document.setHtml(m_instructions->text()); + return document.toPlainText(); + case Page::Progress: + return m_step->text() + "\n" + m_timeLeft->text(); + case Page::Result: + document.setHtml(m_resultText->text()); + return document.toPlainText(); + } + return QString(); +} + +bool +CalibrateAudioDialog::canUseLatency() const +{ + return page() == Page::Result && + !m_useButton->isHidden() && m_useButton->isEnabled(); +} + +void +CalibrateAudioDialog::present() +{ + // The devices and the latency may have changed since it was last + // shown; a check of its own that is running stays on show + if (!m_running) { + m_instructions->setText(instructionsHtml()); + showPage(Page::Instructions); + } + show(); + raise(); + activateWindow(); +} + +void +CalibrateAudioDialog::startCheck() +{ + if (m_running) return; + + m_result = AudioCheckResult(); + m_latencyKept = false; + m_expectedSeconds = expectedSeconds(m_plan); + m_shownPermille = 0; + m_bar->setValue(0); + m_step->setText(tr("Starting the check...")); + m_timeLeft->setText(QString()); + + // The runner says nothing from inside start(), so the page is set + // for what comes after it either way + m_running = true; + showPage(Page::Progress); + + if (!m_runner->start(m_plan)) { + m_running = false; + AudioCheckResult refused; + refused.failure = tr("The check could not start while something " + "is being recorded. Stop the recording, then " + "try again."); + showResult(refused); + } +} + +void +CalibrateAudioDialog::cancelCheck() +{ + // The run ends at once, and its end comes back through + // runnerFinished() + if (m_running) m_runner->cancel(); +} + +void +CalibrateAudioDialog::useLatency() +{ + if (!canUseLatency()) return; + if (m_window->storeMeasuredLatency(m_result)) { + m_latencyKept = true; + } else { + cerr << "CalibrateAudioDialog: the measured latency was not kept" + << endl; + } + m_resultText->setText(resultHtml()); + showPage(Page::Result); +} + +void +CalibrateAudioDialog::showResult(const AudioCheckResult &result) +{ + m_result = result; + m_latencyKept = false; + m_resultText->setText(resultHtml()); + showPage(Page::Result); +} + +void +CalibrateAudioDialog::reject() +{ + // Nothing else would show how the run went, and a check left running + // behind a closed dialog would go on playing chirps + if (m_running) m_runner->cancel(); + QDialog::reject(); +} + +void +CalibrateAudioDialog::runnerProgress(const AudioCheckRunner::Progress &p) +{ + if (!m_running) return; + + QString step; + switch (p.step) { + case AudioCheckRunner::Step::Idle: + return; + case AudioCheckRunner::Step::OpeningReference: + step = tr("Opening the test session..."); + break; + case AudioCheckRunner::Step::AnalysingReference: + step = tr("Getting the test reference ready..."); + break; + case AudioCheckRunner::Step::Recording: + step = tr("Recording punch-in %1 of %2. Keep the earcup against the " + "microphone.").arg(p.punchIn).arg(p.punchIns); + break; + case AudioCheckRunner::Step::AnalysingTake: + step = tr("Analysing punch-in %1 of %2...") + .arg(p.punchIn).arg(p.punchIns); + break; + } + m_step->setText(step); + + // The recording still to come, and a rough time for each analysis + // not yet done: the reference's, and that of every take from the one + // being recorded or analysed on + int analyses = p.punchIns - std::max(p.punchIn, 1) + 1; + if (p.step == AudioCheckRunner::Step::OpeningReference || + p.step == AudioCheckRunner::Step::AnalysingReference) { + analyses = p.punchIns + 1; + } + const double left = p.secondsLeft + kSecondsPerAnalysis * analyses; + if (m_expectedSeconds > 0.0) { + int permille = int(1000.0 * (1.0 - left / m_expectedSeconds)); + m_shownPermille = + std::max(m_shownPermille, std::min(1000, std::max(0, permille))); + m_bar->setValue(m_shownPermille); + } + m_timeLeft->setText(tr("About %1 seconds left") + .arg(int(std::ceil(left)))); +} + +void +CalibrateAudioDialog::runnerFinished(const AudioCheckResult &result) +{ + // A run started elsewhere is not this dialog's to show + if (!m_running) return; + m_running = false; + showResult(result); +} + +void +CalibrateAudioDialog::showPage(Page page) +{ + m_pages->setCurrentIndex(int(page)); + + m_startButton->setVisible(page == Page::Instructions); + m_cancelButton->setVisible(page == Page::Progress); + m_againButton->setVisible(page == Page::Result); + m_closeButton->setVisible(page != Page::Progress); + m_useButton->setVisible(page == Page::Result && + m_result.calibrationUsable()); + m_useButton->setEnabled(!m_latencyKept); + + if (page == Page::Instructions) m_startButton->setDefault(true); + if (page == Page::Result) m_closeButton->setDefault(true); +} + +QString +CalibrateAudioDialog::instructionsHtml() const +{ + QSettings settings; + const LatencyCalibration::Key devices = + LatencyCalibration::currentKey(settings, 0); + + // In fives of seconds: the analyses' share is a guess + const int seconds = 5 * int(std::ceil(expectedSeconds(m_plan) / 5.0)); + + QString html; + html += paragraph + (tr("Tony plays short chirps and records them, to measure how late " + "recordings arrive through your devices. What it measures is " + "used to place your takes on the reference.")); + html += paragraph(bold(tr("Before you start:"))); + html += "
  • " + + tr("Hold one earcup of your headphones against the microphone, " + "%1: the chirps are sharp.").arg(bold(tr("off your ears"))) + + "
  • " + + tr("Set a moderate volume, and keep the room quiet.") + + "
"; + html += ""; + html += ""; + html += ""; + html += ""; + html += "
" + tr("Output:") + "" + + deviceName(devices.playbackDevice).toHtmlEscaped() + "
" + tr("Input:") + "" + + deviceName(devices.recordDevice).toHtmlEscaped() + "
" + tr("Latency in use:") + "" + + describeLatency(m_window->latencyInUse()).toHtmlEscaped() + + "
"; + html += paragraph + (tr("The check replaces the session that is open with a test session " + "of its own, and asks you to save your work first. It takes about " + "%1 seconds. Your song is not changed: open it again from File " + "▸ Open Recent afterwards.").arg(seconds)); + return html; +} + +QString +CalibrateAudioDialog::resultHtml() const +{ + const AudioCheckResult &r = m_result; + + if (r.failure != "") { + return paragraph(bold(tr("The check did not finish."))) + + paragraph(r.failure.toHtmlEscaped()); + } + + const LatencyCheck::TakeSummary &s = r.summary; + const double driver = r.reportedOutputLatency + r.reportedInputLatency; + const QString measured = milliseconds(r.calibratedRoundTrip); + const QString timing = milliseconds(timingSpread(s)); + + // The verdict in plain words, and what to do about it. A rate that + // differs comes first, whatever the sweeps say: they are misplaced + // by it, further the later they come + QString html; + if (r.rateMismatch) { + html += paragraph(bold(tr("The recording device runs at %1 Hz; takes " + "cannot line up until that is fixed.") + .arg(hertz(r.recordingRate)))); + html += paragraph + (tr("The test reference runs at %1 Hz, as everything Tony plays " + "does. A take recorded at another rate lands further off the " + "later in the song it is, so no one latency places it right.") + .arg(hertz(r.referenceRate))); + } else { + switch (s.verdict) { + case Verdict::Ok: + html += paragraph(bold(tr("The test sounds came back steadily, " + "%1 after they were played.") + .arg(measured))); + html += paragraph + (tr("Press Use this latency to place your takes with it.")); + break; + case Verdict::NoSignal: + html += paragraph(bold(tr("Tony could not hear the test sounds: " + "it found %1 of %2.") + .arg(s.found).arg(s.judged))); + html += "
  • " + + tr("Turn the volume up, and hold the earcup right against " + "the microphone.") + "
  • " + + tr("Check that the microphone is not muted, and that it is " + "the one Tony records from (Playback ▸ Audio Input " + "Device).") + "
  • " + + tr("In Windows, turn off Sound settings ▸ your microphone ▸ " + "Audio enhancements.") + "
  • " + + tr("Do not record through a Bluetooth headset's " + "\"Hands-Free\" device: it records at telephone quality.") + + "
"; + break; + case Verdict::Clipped: + html += paragraph(bold(tr("The test sounds were too loud: the " + "recording reached full scale."))); + html += paragraph(tr("Turn the volume down, or hold the earcup a " + "little away from the microphone, and " + "check again.")); + break; + case Verdict::Fading: + html += paragraph(bold(tr("The test sounds got quieter as the " + "check went on, by %1 dB.") + .arg(QLocale().toString + (s.fadingDb, 'f', 0)))); + html += paragraph + (tr("Something is filtering the microphone, such as echo " + "cancellation or audio enhancements. In Windows, turn " + "off Sound settings ▸ your microphone ▸ Audio " + "enhancements, and check again.")); + break; + case Verdict::PositionDependent: + html += paragraph(bold(tr("The delay grew from one punch-in to " + "the next."))); + html += paragraph + (tr("The recording seems to run at another speed than the " + "playback, so no one latency places every take right.")); + break; + case Verdict::Scattered: + html += paragraph(bold(tr("The driver's timing varies from take " + "to take by %1.").arg(timing))); + html += paragraph + (tr("No one latency places every take right when it varies " + "that much. Close other programs that use sound, and " + "check again.")); + break; + case Verdict::Unsteady: + html += paragraph(bold(tr("The driver's timing varies from take " + "to take by %1.").arg(timing))); + html += paragraph + (tr("That is small enough to use: the measured round trip, " + "%1, is the middle of it. Press Use this latency to place " + "your takes with it.").arg(measured)); + break; + } + } + + if (s.echo.heard) { + html += paragraph + (tr("Your microphone is being played back somewhere (Windows " + "\"Listen to this device\", or an interface's direct " + "monitor): every test sound came back a second time, %1 " + "later. Turn that off, or your takes will be heard twice.") + .arg(milliseconds(s.echo.delaySeconds))); + } + + if (m_latencyKept) { + html += paragraph(bold(tr("Kept.")) + " " + + tr("Takes on these devices are now placed with %1.") + .arg(measured)); + } + + // The figures, for whoever wants them, and for passing on + auto row = [](QString name, QString value) { + return "" + name + "" + value.toHtmlEscaped() + + ""; + }; + // What the sweeps found says how far the driver's figure is out even + // when it cannot be used, as long as enough of them were found and + // at the right rate + const bool haveMeasurement = s.found > 0 && !r.rateMismatch && + s.verdict != Verdict::NoSignal; + + html += ""; + html += row(tr("Round trip:"), + (haveMeasurement ? + tr("%1 measured").arg(measured) : tr("not measured")) + + tr("; the driver reports %1 (%2 out, %3 in)") + .arg(milliseconds(driver)) + .arg(milliseconds(r.reportedOutputLatency)) + .arg(milliseconds(r.reportedInputLatency))); + if (!r.takes.empty()) { + html += row(tr("Takes placed with:"), + (r.takes.front().measured ? + tr("%1, measured before") : tr("%1, the driver's figure")) + .arg(milliseconds(r.usedRoundTrip))); + } + QStringList offsets; + for (const LatencyCheck::PunchInResult &p : s.punchIns) { + offsets << (p.found > 0 ? + signedMilliseconds(p.medianOffset) : tr("nothing found")); + } + if (!offsets.isEmpty()) { + html += row(tr("Punch-ins landed:"), + tr("%1 (+ is late)").arg(offsets.join(", "))); + } + html += row(tr("Spread between punch-ins:"), milliseconds(s.spread)); + html += row(tr("Test sounds found:"), + tr("%1 of %2").arg(s.found).arg(s.judged)); + html += row(tr("Sample rates:"), + tr("recorded at %1 Hz, reference at %2 Hz") + .arg(hertz(r.recordingRate)).arg(hertz(r.referenceRate))); + html += row(tr("Input peak:"), + s.inputPeak > 0.0 ? + tr("%1 dBFS").arg(QLocale().toString + (20.0 * std::log10(s.inputPeak), 'f', 1)) : + tr("silence")); + html += row(tr("Echo:"), + s.echo.heard ? + tr("%1 after the sound, %2 dB %3") + .arg(milliseconds(s.echo.delaySeconds)) + .arg(QLocale().toString(std::fabs(s.echo.levelDb), 'f', 0)) + .arg(s.echo.levelDb <= 0.0 ? tr("quieter") : tr("louder")) : + tr("none heard")); + html += row(tr("Devices:"), + tr("output %1; input %2") + .arg(deviceName(r.key.playbackDevice)) + .arg(deviceName(r.key.recordDevice))); + html += "
"; + return html; +} + +QString +CalibrateAudioDialog::describeLatency(const LatencyCalibration::InUse &inUse) +{ + if (inUse.source == LatencyCalibration::Source::Measured) { + if (!inUse.date.isValid()) { + return tr("measured %1").arg(milliseconds(inUse.roundTrip)); + } + // The year only when it is not this one + const QDate day = inUse.date.toLocalTime().date(); + const QString format = + day.year() == QDate::currentDate().year() ? "d MMM" : "d MMM yyyy"; + return tr("measured %1, %2").arg(milliseconds(inUse.roundTrip)) + .arg(QLocale().toString(day, format)); + } + + // The device reports its latencies once it is open + if (!(inUse.roundTrip > 0.0)) { + return tr("driver's figure, not known yet"); + } + QString text = tr("driver's figure, %1").arg(milliseconds(inUse.roundTrip)); + if (inUse.stale) { + text = tr("%1 (the measured one is out of date)").arg(text); + } + return text; +} + +QString +CalibrateAudioDialog::milliseconds(double seconds) +{ + const double ms = seconds * 1000.0; + if (std::fabs(ms) < 9.95) { + double tenths = std::round(ms * 10.0) / 10.0; + if (tenths == 0.0) tenths = 0.0; // not "-0" + const int decimals = (tenths == std::round(tenths)) ? 0 : 1; + return tr("%1 ms").arg(QLocale().toString(tenths, 'f', decimals)); + } + return tr("%1 ms").arg(QLocale().toString(std::round(ms), 'f', 0)); +} + +QString +CalibrateAudioDialog::signedMilliseconds(double seconds) +{ + if (std::round(seconds * 10000.0) > 0.0) { + return "+" + milliseconds(seconds); + } + return milliseconds(seconds); +} + +QString +CalibrateAudioDialog::deviceName(QString name) +{ + // As the device menus call it + return name == "" ? tr("(System Default)") : name; +} + +QString +CalibrateAudioDialog::hertz(sv_samplerate_t rate) +{ + return QString::number(qint64(std::llround(rate))); +} diff --git a/main/CalibrateAudioDialog.h b/main/CalibrateAudioDialog.h new file mode 100644 index 00000000..ed5cdccf --- /dev/null +++ b/main/CalibrateAudioDialog.h @@ -0,0 +1,150 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_CALIBRATE_AUDIO_DIALOG_H +#define TONY_CALIBRATE_AUDIO_DIALOG_H + +#include "AudioCheckRunner.h" +#include "LatencyCalibration.h" + +#include + +class MainWindow; +class QLabel; +class QProgressBar; +class QPushButton; +class QStackedWidget; + +/** + * Playback > Calibrate Audio: what the user sees of the audio check. + * Three pages: what to do before it starts, with the devices and the + * latency takes are placed with now; how far it has got, with Cancel; + * and what it found, in plain words, with the fix for whatever went + * wrong, the figures, and Use this latency when the round trip it + * measured can be used. + * + * Not modal: the check opens a session of its own in the window, which + * can be looked at while the dialog is up. A view over the runner and + * nothing more: it starts and cancels runs, shows what the runner + * reports of the runs it started (not of any other), and hands a result + * to MainWindow::storeMeasuredLatency(). It touches no model, layer or + * take. Closing it while its check runs cancels the check, since + * nothing else would show how the run ended. + * + * The window owns it, and makes it the first time it is asked for. + */ +class CalibrateAudioDialog : public QDialog +{ + Q_OBJECT + +public: + CalibrateAudioDialog(MainWindow *window, AudioCheckRunner *runner); + virtual ~CalibrateAudioDialog(); + + /// A rough time for each analysis a run waits for (the reference's, + /// and each take's), added to the recording still to come for the + /// progress shown: the runner cannot know how long pYIN takes + static constexpr double kSecondsPerAnalysis = 3.0; + + enum class Page { Instructions, Progress, Result }; + Page page() const; + + /// The words of the page on show, as plain text + QString pageText() const; + + /// Whether Use this latency is offered, and not yet pressed + bool canUseLatency() const; + + /// What a check started here records: the calibration + /// (AudioCheckRunner::calibrationPlan()) unless set otherwise, as + /// the tests set a shorter one + void setPlan(const AudioCheckRunner::Plan &plan); + const AudioCheckRunner::Plan &plan() const { return m_plan; } + + /// The latency a take is placed with, and where it came from, in a + /// few words: "measured 187 ms, 25 Sep" or "driver's figure, 400 + /// ms". The Playback menu's line says the same + static QString describeLatency(const LatencyCalibration::InUse &inUse); + +public slots: + /// Show the dialog and bring it to the front: on the instructions, + /// with the devices and the latency as they are now, unless its + /// check is running + void present(); + + /// Start, and Check Again + void startCheck(); + + /// Cancel: the run ends, and how it ended is the result shown + void cancelCheck(); + + /// Keep the round trip the check measured, for the devices it ran on + void useLatency(); + + /// The result page for this result + void showResult(const AudioCheckResult &result); + + /// Escape, the title bar's close button, and Close. A check still + /// running is cancelled first + void reject() override; + +private: + MainWindow *m_window; + AudioCheckRunner *m_runner; + AudioCheckRunner::Plan m_plan; + + /// A run this dialog started is going on + bool m_running; + + AudioCheckResult m_result; + bool m_latencyKept; + + /// The time the run was expected to take when it began, and the + /// share of it the progress bar has shown, which never goes back + double m_expectedSeconds; + int m_shownPermille; + + QStackedWidget *m_pages; + QLabel *m_instructions; + QLabel *m_step; + QProgressBar *m_bar; + QLabel *m_timeLeft; + QLabel *m_resultText; + + QPushButton *m_startButton; + QPushButton *m_cancelButton; + QPushButton *m_useButton; + QPushButton *m_againButton; + QPushButton *m_closeButton; + + void runnerProgress(const AudioCheckRunner::Progress &progress); + void runnerFinished(const AudioCheckResult &result); + + void showPage(Page page); + + QString instructionsHtml() const; + QString resultHtml() const; + + /// Seconds in milliseconds, for reading: tenths below 10 ms, where + /// they say something, whole ones from there on + static QString milliseconds(double seconds); + static QString signedMilliseconds(double seconds); + + /// A device as the Preferences name it, "" being the default + static QString deviceName(QString name); + + static QString hertz(sv::sv_samplerate_t rate); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 59e86563..540c046b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -19,6 +19,7 @@ #include "NetworkPermissionTester.h" #include "Analyser.h" #include "AudioCheckRunner.h" +#include "CalibrateAudioDialog.h" #include "LatencyUtils.h" #include "PaneUtils.h" #include "TakeEvents.h" @@ -204,6 +205,10 @@ MainWindow::MainWindow(AudioMode audioMode, m_takeLatency(), m_audioCheck(nullptr), m_audioCheckTakes(false), + m_calibrateAudioDialog(nullptr), + m_calibrateAudioAction(nullptr), + m_latencyLineAction(nullptr), + m_forgetLatencyAction(nullptr), m_lastRecordingRate(0) { setWindowTitle(QApplication::applicationName()); @@ -400,6 +405,13 @@ MainWindow::MainWindow(AudioMode audioMode, m_coverageStrip = new CoverageStrip(this); m_audioCheck = new AudioCheckRunner(this); + // What may be chosen changes as a check begins and as it ends. The + // first progress comes from its first step, before anything is asked + connect(m_audioCheck, &AudioCheckRunner::progress, + this, [this]() { updateMenuStates(); }); + connect(m_audioCheck, &AudioCheckRunner::finished, + this, [this]() { updateMenuStates(); }); + // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end m_takeTimer = new QTimer(this); @@ -456,7 +468,10 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindow::~MainWindow() { - // A check still running ends here, before anything it reads goes + // The check's dialog first, as it holds the runner; then a check + // still running ends here, before anything it reads goes + delete m_calibrateAudioDialog; + m_calibrateAudioDialog = nullptr; delete m_audioCheck; m_audioCheck = nullptr; @@ -1656,6 +1671,29 @@ MainWindow::setupToolbars() connect(g, SIGNAL(triggered(QAction *)), this, SLOT(audioDeviceSelected(QAction *))); } + + // The audio check, and the latency takes are placed with: a line to + // read, never chosen, brought up to date whenever the menu opens + m_calibrateAudioAction = menu->addAction(tr("&Calibrate Audio...")); + m_calibrateAudioAction->setStatusTip + (tr("Measure how late recordings arrive through these devices, with " + "an earcup held against the microphone")); + connect(m_calibrateAudioAction, &QAction::triggered, + this, &MainWindow::calibrateAudio); + + m_latencyLineAction = menu->addAction(QString()); + m_latencyLineAction->setEnabled(false); + + m_forgetLatencyAction = menu->addAction(tr("&Forget Measured Latency")); + m_forgetLatencyAction->setStatusTip + (tr("Place takes on these devices with the latency the driver " + "reports again")); + connect(m_forgetLatencyAction, &QAction::triggered, + this, [this]() { forgetMeasuredLatency(); }); + + connect(menu, &QMenu::aboutToShow, + this, &MainWindow::updateLatencyMenuLine); + updateLatencyMenuLine(); menu->addSeparator(); m_rightButtonPlaybackMenu->addAction(playAction); @@ -2219,6 +2257,17 @@ MainWindow::updateMenuStates() emit canChangeTakes(canChange); emit canActOnTake(canChange && m_takes->getActiveIndex() >= 0); + // The audio check records takes of its own, and keeps what it + // measures for the devices it started on + bool checking = m_audioCheck && m_audioCheck->isRunning(); + if (m_calibrateAudioAction) { + m_calibrateAudioAction->setEnabled(!inTake && !checking); + } + for (QMenu *m : { m_audioDeviceMenu, m_audioInputDeviceMenu }) { + if (m) m->menuAction()->setEnabled(!checking); + } + updateLatencyMenuLine(); + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -4341,15 +4390,19 @@ MainWindow::storeMeasuredLatency(const AudioCheckResult &result) figure.reportedOutput = result.reportedOutputLatency; figure.reportedInput = result.reportedInputLatency; + // Under the devices the check ran on, which the Preferences may no + // longer name: the result can be on show long after the run + LatencyCalibration::Key key = result.key; + key.rate = result.recordingRate; + QSettings settings; - LatencyCalibration::store - (settings, LatencyCalibration::currentKey(settings, result.recordingRate), - figure); + LatencyCalibration::store(settings, key, figure); cerr << "MainWindow::storeMeasuredLatency: round trip " - << figure.roundTrip * 1000.0 << " ms at " << result.recordingRate + << figure.roundTrip * 1000.0 << " ms at " << key.rate << " Hz, the device reporting " << figure.reportedOutput * 1000.0 << " ms out and " << figure.reportedInput * 1000.0 << " ms in" << endl; + updateLatencyMenuLine(); return true; } @@ -4362,6 +4415,30 @@ MainWindow::forgetMeasuredLatency() expectedRecordingRate())); cerr << "MainWindow::forgetMeasuredLatency: at " << expectedRecordingRate() << " Hz" << endl; + updateLatencyMenuLine(); +} + +void +MainWindow::updateLatencyMenuLine() +{ + if (!m_latencyLineAction || !m_forgetLatencyAction) return; + LatencyCalibration::InUse inUse = latencyInUse(); + m_latencyLineAction->setText + (tr("Latency: %1").arg(CalibrateAudioDialog::describeLatency(inUse))); + + // A stale figure is kept too (it applies again if the device goes + // back to its old buffers), and can be forgotten like any other + m_forgetLatencyAction->setEnabled + (inUse.source == LatencyCalibration::Source::Measured || inUse.stale); +} + +void +MainWindow::calibrateAudio() +{ + if (!m_calibrateAudioDialog) { + m_calibrateAudioDialog = new CalibrateAudioDialog(this, m_audioCheck); + } + m_calibrateAudioDialog->present(); } TakeTiming diff --git a/main/MainWindow.h b/main/MainWindow.h index 6b50df6f..8f0da001 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -39,6 +39,7 @@ class QActionGroup; class AudioCheckRunner; struct AudioCheckResult; +class CalibrateAudioDialog; namespace sv { class VersionTester; @@ -92,12 +93,13 @@ class MainWindow : public sv::MainWindowBase FileOpenStatus openSession(sv::FileSource source) override; // The round trip takes are placed with (see LatencyCalibration). - // Keep the one an audio check measured, for the devices the - // Preferences name and the rate the check recorded at; false, with - // nothing kept, unless the check's figure is usable + // Keep the one an audio check measured, for the devices it started + // on and the rate it recorded at (AudioCheckResult::key); false, with + // nothing kept, unless the check's figure is usable. The Playback + // menu's line about the latency follows bool storeMeasuredLatency(const AudioCheckResult &result); - // Drop the figure latencyInUse() describes + // Drop the figure latencyInUse() describes, and say so in the menu void forgetMeasuredLatency(); // What the next take will be placed with, as far as it is known @@ -282,6 +284,9 @@ protected slots: virtual void rescanAudioDevices(); virtual void audioDeviceSelected(QAction *); + // Playback > Calibrate Audio: the audio check's dialog, not modal + virtual void calibrateAudio(); + virtual void handleOSCMessage(const sv::OSCMessage &); virtual void mouseEnteredWidget(); @@ -886,6 +891,20 @@ protected slots: AudioCheckRunner *m_audioCheck; bool m_audioCheckTakes; + // Playback > Calibrate Audio, made the first time it is chosen, and + // the lines under it: the latency takes are placed with, and Forget + // Measured Latency. Calibrate Audio is shut while a take or a check + // is being recorded, and so are the device menus while a check runs: + // the figure it measures is kept for the devices it started on + CalibrateAudioDialog *m_calibrateAudioDialog; + QAction *m_calibrateAudioAction; + QAction *m_latencyLineAction; + QAction *m_forgetLatencyAction; + + // Say which latency is in use, and let it be forgotten if it is one + // the check measured + void updateLatencyMenuLine(); + // The rate the device recorded at, the last time a take was placed // with a round trip; 0 until then, and again once another device is // chosen diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 1168a56e..3e829a1e 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -27,10 +27,13 @@ #include "TestRecordWorkflow.h" #include "../AudioCheckRunner.h" +#include "../CalibrateAudioDialog.h" #include "../LatencyCheck.h" #include "base/PlayParameterRepository.h" +#include + class TestAudioCheck : public QObject { Q_OBJECT @@ -112,6 +115,73 @@ class TestAudioCheck : public QObject QVERIFY(!m_window->audioCheck()->isRunning()); } + // The devices the Preferences name, as the device menus write them + // when the driver is left alone. The fake device takes no notice + static void setDevices(QString output, QString input) { + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue("audio-playback-device", output); + settings.setValue("audio-record-device", input); + settings.endGroup(); + } + + static LatencyCalibration::Key key(QString output, QString input) { + LatencyCalibration::Key key; + key.playbackDevice = output; + key.recordDevice = input; + key.rate = rate; + return key; + } + + // The Playback menu's line about the latency, as it reads when the + // menu is opened + QString latencyLine() { + emit m_window->playbackMenu()->aboutToShow(); + return m_window->latencyLineAction()->text(); + } + + // Playback > Calibrate Audio chosen, and the dialog it shows, Start + // pressed on it with the short plan + CalibrateAudioDialog *startCheckFromMenu() { + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + if (!dialog) return nullptr; + dialog->setPlan(shortPlan()); + m_window->discardModifications(); + dialog->startCheck(); + return dialog; + } + + // A run judged at the reference's rate, by the fake device's + // figures, to be given a verdict + static AudioCheckResult judgedResult(LatencyCheck::Verdict verdict) { + AudioCheckResult r; + r.recordingRate = rate; + r.referenceRate = rate; + r.reportedOutputLatency = reportedOut / rate; + r.reportedInputLatency = reportedIn / rate; + r.usedRoundTrip = (reportedOut + reportedIn) / rate; + r.takes.resize(4); + r.summary.verdict = verdict; + if (verdict != LatencyCheck::Verdict::Ok) { + r.summary.flags.push_back(verdict); + } + r.summary.judged = 12; + r.summary.found = 12; + r.summary.punchIns.resize(4); + for (LatencyCheck::PunchInResult &p : r.summary.punchIns) { + p.judged = 3; + p.found = 3; + p.medianOffset = 0.003; + p.spread = 0.001; + } + r.summary.medianOffset = 0.003; + r.summary.spread = 0.001; + r.summary.inputPeak = 0.25; + r.calibratedRoundTrip = r.usedRoundTrip + 0.003; + return r; + } + // The three toggles of the take path, and the settings they and the // pre-roll's length are kept in QStringList toggles() { @@ -352,6 +422,10 @@ private slots: settings.beginGroup("Analyser"); settings.remove(""); settings.endGroup(); + settings.beginGroup("Preferences"); + settings.remove("audio-playback-device"); + settings.remove("audio-record-device"); + settings.endGroup(); SingingTakes::setOverwriteConfirmationWanted(true); } @@ -764,6 +838,236 @@ private slots: (dir.path(), dir.filePath("song.wav")), one); QCOMPARE(files(), QStringList() << "song.wav"); } + + // Playback > Calibrate Audio: a dialog, not modal, that names the + // devices and the latency in use and starts a check. While the check + // runs, neither it nor the device menus can be chosen. Other devices + // are named in the Preferences meanwhile; Use this latency keeps the + // round trip measured for the devices the check started on, and the + // menu's line says what the devices named now are placed with + void calibrate_audio_from_the_menu() { + setDevices("Speakers A", "Microphone A"); + makeWindow(loopback()); + + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + QVERIFY(dialog->isVisible()); + QVERIFY(!dialog->isModal()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Instructions); + const QString instructions = dialog->pageText(); + for (QString words : { "Speakers A", "Microphone A", "off your ears", + "moderate volume", "room quiet", + "replaces the session", "save your work", + "driver's figure" }) { + QVERIFY2(instructions.contains(words), + qPrintable(words + " not in: " + instructions)); + } + + // Four punch-ins of three sweeps, which the calibration layout has + // room for, unless the test says otherwise + const AudioCheckRunner::Plan plan = dialog->plan(); + QCOMPARE(plan.punchIns, 4); + QCOMPARE(plan.eventsEach, 3); + QCOMPARE(int(LatencyCheck::punchInsFor + (plan.layout, plan.punchIns, plan.eventsEach).size()), 4); + + dialog->setPlan(shortPlan()); + m_window->discardModifications(); + dialog->startCheck(); + QVERIFY(m_window->audioCheck()->isRunning()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); + setDevices("Speakers B", "Microphone B"); + + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); + QVERIFY(!m_window->audioOutputMenu()->menuAction()->isEnabled()); + QVERIFY(!m_window->audioInputMenu()->menuAction()->isEnabled()); + QVERIFY2(dialog->pageText().contains("Recording punch-in 1 of 2"), + qPrintable(dialog->pageText())); + + QTRY_VERIFY_WITH_TIMEOUT + (dialog->page() == CalibrateAudioDialog::Page::Result, 60000); + QCOMPARE(m_finished, 1); + QVERIFY(m_window->calibrateAudioAction()->isEnabled()); + QVERIFY(m_window->audioOutputMenu()->menuAction()->isEnabled()); + QVERIFY(m_window->audioInputMenu()->menuAction()->isEnabled()); + + // 12411 frames measured, 12288 reported, at 44.1 kHz + QVERIFY2(m_result.calibrationUsable(), describe(m_result).constData()); + QString words = dialog->pageText(); + for (QString w : { "came back steadily", + "281 ms measured; the driver reports 279 ms", + "output Speakers A; input Microphone A" }) { + QVERIFY2(words.contains(w), + qPrintable(w + " not in: " + words)); + } + QVERIFY(dialog->canUseLatency()); + + dialog->useLatency(); + QVERIFY(!dialog->canUseLatency()); + QVERIFY2(dialog->pageText().contains("Kept."), + qPrintable(dialog->pageText())); + + LatencyCalibration::Figure figure; + QSettings settings; + QVERIFY(!LatencyCalibration::load + (settings, key("Speakers B", "Microphone B"), figure)); + QVERIFY(LatencyCalibration::load + (settings, key("Speakers A", "Microphone A"), figure)); + QVERIFY2(std::fabs(figure.roundTrip * rate - roundTrip) <= 4.0, + qPrintable(QString("kept %1 frames") + .arg(figure.roundTrip * rate))); + + QCOMPARE(latencyLine(), QString("Latency: driver's figure, 279 ms")); + QVERIFY(!m_window->forgetLatencyAction()->isEnabled()); + setDevices("Speakers A", "Microphone A"); + const QString line = latencyLine(); + QVERIFY2(line.startsWith("Latency: measured 281 ms, "), + qPrintable(line)); + QVERIFY(m_window->forgetLatencyAction()->isEnabled()); + } + + // Forget Measured Latency: the menu's line goes back to the driver's + // figure, and the figure kept is gone + void calibrate_audio_forget_measured_latency() { + makeWindow(loopback()); + const LatencyCalibration::InUse reported = m_window->latencyInUse(); + QVERIFY(reported.source == LatencyCalibration::Source::Reported); + QVERIFY2(latencyLine().startsWith("Latency: driver's figure"), + qPrintable(latencyLine())); + QVERIFY(!m_window->forgetLatencyAction()->isEnabled()); + + // As the check keeps one, for the devices the device reports now + LatencyCalibration::Figure figure; + figure.roundTrip = 0.3; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = reported.reportedOutput; + figure.reportedInput = reported.reportedInput; + QSettings settings; + LatencyCalibration::store(settings, key("", ""), figure); + + const QString line = latencyLine(); + QCOMPARE(line, QString("Latency: measured 300 ms, %1") + .arg(QLocale().toString(QDate::currentDate(), "d MMM"))); + QVERIFY(m_window->forgetLatencyAction()->isEnabled()); + + m_window->forgetLatencyAction()->trigger(); + QVERIFY(!LatencyCalibration::load(settings, key("", ""), figure)); + QVERIFY(m_window->latencyInUse().source == + LatencyCalibration::Source::Reported); + QVERIFY2(m_window->latencyLineAction()->text() + .startsWith("Latency: driver's figure"), + qPrintable(m_window->latencyLineAction()->text())); + QVERIFY(!m_window->forgetLatencyAction()->isEnabled()); + } + + // What the result page says, for results made here: the verdict in + // plain words with its fix, and Use this latency only when the figure + // can be used + void calibrate_audio_result_words() { + makeWindow(loopback()); + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + + auto verify = [&](const AudioCheckResult &result, bool usable, + QStringList present, QStringList absent) { + dialog->showResult(result); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + const QString words = dialog->pageText(); + for (QString w : present) { + QVERIFY2(words.contains(w), + qPrintable(w + " not in: " + words)); + } + for (QString w : absent) { + QVERIFY2(!words.contains(w), + qPrintable(w + " in: " + words)); + } + QCOMPARE(dialog->canUseLatency(), usable); + }; + + // Found one sweep in twelve + AudioCheckResult silent = + judgedResult(LatencyCheck::Verdict::NoSignal); + silent.summary.found = 1; + verify(silent, false, + { "could not hear the test sounds: it found 1 of 12", + "volume up", "not muted", "Audio enhancements", + "\"Hands-Free\"", "not measured; the driver reports 279 ms" }, + { "Use this latency" }); + if (QTest::currentTestFailed()) return; + + // A device at 48 kHz: what the sweeps say is not the point + AudioCheckResult fast = judgedResult(LatencyCheck::Verdict::Scattered); + fast.recordingRate = 48000; + fast.rateMismatch = true; + fast.summary.spread = 0.6; + verify(fast, false, + { "The recording device runs at 48000 Hz; takes cannot line " + "up until that is fixed.", + "recorded at 48000 Hz, reference at 44100 Hz" }, + { "varies from take to take" }); + if (QTest::currentTestFailed()) return; + + // Unsteady, but usable; and the microphone monitored + AudioCheckResult unsteady = + judgedResult(LatencyCheck::Verdict::Unsteady); + unsteady.summary.spread = 0.008; + unsteady.summary.echo.heard = true; + unsteady.summary.echo.delaySeconds = 0.045; + unsteady.summary.echo.levelDb = -12.0; + verify(unsteady, true, + { "The driver's timing varies from take to take by 8 ms.", + "Your microphone is being played back somewhere", + "Listen to this device", "45 ms later", + "45 ms after the sound, 12 dB quieter" }, + { "Kept." }); + } + + // Not while an ordinary take is being recorded: the check records + // takes of its own + void calibrate_audio_not_during_a_take() { + makeWindow(FakeAudioIO::Config()); + openSong(); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->calibrateAudioAction()->isEnabled()); + + m_window->doRecord(); + QVERIFY(m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); + QTest::qWait(300); + + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT + (m_window->calibrateAudioAction()->isEnabled(), 10000); + } + + // Closing the dialog while its check runs cancels the check; opened + // again, it starts from the instructions + void calibrate_audio_closed_during_a_check() { + makeWindow(loopback()); + CalibrateAudioDialog *dialog = startCheckFromMenu(); + QVERIFY(dialog); + QVERIFY(m_window->audioCheck()->isRunning()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + + QVERIFY(dialog->close()); + QVERIFY(!dialog->isVisible()); + QVERIFY(!m_window->audioCheck()->isRunning()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->audioCheckTakes()); + QCOMPARE(m_finished, 1); + QVERIFY(m_result.failure != ""); + QVERIFY(m_window->calibrateAudioAction()->isEnabled()); + + m_window->calibrateAudioAction()->trigger(); + QVERIFY(dialog->isVisible()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Instructions); + } }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index b70d1059..243e5763 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -166,6 +166,18 @@ class TestMainWindow : public MainWindow // the last take was placed with AudioCheckRunner *audioCheck() { return m_audioCheck; } bool audioCheckTakes() { return m_audioCheckTakes; } + + // Playback > Calibrate Audio, the dialog it shows once it has been + // chosen, the lines under it, and the device menus above it + QAction *calibrateAudioAction() { return m_calibrateAudioAction; } + CalibrateAudioDialog *calibrateAudioDialog() { + return m_calibrateAudioDialog; + } + QAction *latencyLineAction() { return m_latencyLineAction; } + QAction *forgetLatencyAction() { return m_forgetLatencyAction; } + QMenu *playbackMenu() { return m_playbackMenu; } + QMenu *audioOutputMenu() { return m_audioDeviceMenu; } + QMenu *audioInputMenu() { return m_audioInputDeviceMenu; } TakeLatency takeLatency() { return m_takeLatency; } QAction *playSingingAudioAction() { return m_playSingingAudio; } diff --git a/meson.build b/meson.build index c3729571..0b84c84f 100644 --- a/meson.build +++ b/meson.build @@ -1108,6 +1108,7 @@ tony_core_files = [ tony_app_files = [ 'main/AlternatePitchTrack.cpp', 'main/AudioCheckRunner.cpp', + 'main/CalibrateAudioDialog.cpp', 'main/CoverageStrip.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', @@ -1129,6 +1130,7 @@ tony_app_moc_files = qt.preprocess( 'main/Analyser.h', 'main/AlternatePitchTrack.h', 'main/AudioCheckRunner.h', + 'main/CalibrateAudioDialog.h', 'main/CoverageStrip.h', ]) From 7b44cbaa4dcb125b1710e08d47f305de29df04db Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:45:34 +0000 Subject: [PATCH 122/275] docs: calibrate audio work orders, B4 done Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 55046a62..7c4860f8 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -540,3 +540,8 @@ The next phase must know: - A run replacing a check session asks "Session modified: save?" (its takes mark it modified). Check Again always meets it; answer No. The runner could skip the question for its own reference's session. - Not on the result page: §2's mic channel and noise floor. The runner measures neither. Left open: Record stays enabled during a check. Pressing it there goes through the Stop path and ends the check's take early; what the run then makes of it was not tried. + +### Lead — 2026-09-26, after B4 +- The button is complete; the user's Windows run is the checkpoint (spec §7). C0 onwards goes on meanwhile. +- Left for C1 (small, in passing): Record stays enabled during a check, and pressing it ends the check's take early; grey it while a check runs. +- Left for D: spec §2 promises the mic channel and noise floor on the result page, which nothing measures yet (C2's item 5 measures the channel); §8 says "modal progress dialog", true only of the dev run; Check Again always asks to save the check's own session (answer No), a possible later nicety. From 8b752c41fd9e44b3b027b4019b3fe2267c08f324 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:46:09 +0000 Subject: [PATCH 123/275] docs: calibrate audio work orders, C0 refined Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 32 ++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 7c4860f8..fa225751 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -399,7 +399,37 @@ Read also: how an existing Tony dialog is built and tested (search ### C0 — TakeDiff (spec §5 "tony_core") -To be refined by the lead. +Read also: `docs/takes.md` on the splice's edge fades and on ranged analysis's merge +window (search "fade", "±", "W"); `main/TakeAudio.h` and `main/TakeEvents.h` for the +existing vocabulary. `TakeEvents` may already hold part of what is needed: reuse it and +do not duplicate it. + +- **New `main/TakeDiff.{h,cpp}`** in `tony_core`, pure. The comparisons the dev checks + (C1–C4) use on a real take. Each returns a plain result: pass or fail, and the numbers + behind it, so that a check can report them. The inputs are sample buffers, event + vectors and frame ranges; no models. +- **Audio unchanged outside a range.** Two sample buffers (before and after a + punch-in, as read from the take's file) are **bit-identical** outside + `[start, end)`, allowing for where the splice's fades fall. Find that in + `TakeAudio.cpp`; do not guess. Report the first differing frame. +- **Events unchanged outside a range ± margin.** Two event vectors (pitch, and notes + with durations) are identical outside `[start − margin, end + margin]`. A note that + crosses the boundary counts as inside. Report what was added, removed and changed. +- **Pitch continuous across a join.** Given pitch events and a join frame: + - no gap longer than N hops within ±W of the join; + - no two events at the same frame; + - frames strictly increasing. +- **One note across a join.** Exactly one note spans the join frame, and no note + begins or ends within ±X of it. X is a named constant. +- **No step at a join.** The largest first difference of the samples within ±2 ms of + the join, against the typical first difference over the 50 ms around it, in dB. It + should stay near 0 dB when the splice is clean, and a hard cut shows as a large + excess. The threshold is a named constant; justify it. +- **Tests,** in a new core class `TestTakeDiff`, on synthetic data: + - each comparison passing and failing on purpose: one sample changed just outside + the range; one pitch event dropped at the join; a doubled frame; a note split in + two at the join; a hard cut in a sine. + - **Show failure** for two of them by breaking the code. ### C1 — Dev-check framework and first group (spec §3, §4, §5 "development builds only") From c31b63ce8ccbe257c6da5f93b4759bcd7b1d78cf Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:49:13 +0000 Subject: [PATCH 124/275] feat: touch gestures on the panes pinch zooms the time axis about the fingers, two fingers scroll it, and a long press asks for the pane's right-button menu. touchgestures is an event filter each pane gets; one finger stays qt's own mouse events, with the press held back so that a long press or a second finger leaves no drag or selection, and the menu is kept from the finger that opened it. pinchzoom (tony_core) holds the zoom and position arithmetic. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/mobile-port.md | 6 +- main/MainWindow.cpp | 2 + main/PinchZoom.cpp | 104 +++++++ main/PinchZoom.h | 68 ++++ main/TouchGestures.cpp | 537 ++++++++++++++++++++++++++++++++ main/TouchGestures.h | 143 +++++++++ main/test/TestPinchZoom.h | 152 +++++++++ main/test/TestTouchGestures.h | 562 ++++++++++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 7 + main/test/tony-core-test.cpp | 7 + meson.build | 5 + 11 files changed, 1591 insertions(+), 2 deletions(-) create mode 100644 main/PinchZoom.cpp create mode 100644 main/PinchZoom.h create mode 100644 main/TouchGestures.cpp create mode 100644 main/TouchGestures.h create mode 100644 main/test/TestPinchZoom.h create mode 100644 main/test/TestTouchGestures.h diff --git a/docs/mobile-port.md b/docs/mobile-port.md index d9af2dd9..aff3165f 100644 --- a/docs/mobile-port.md +++ b/docs/mobile-port.md @@ -170,7 +170,8 @@ device's rate (svgui's `ViewManager`, an expected failure in `TestRecordWorkflow Navigate mode (`Pane::dragTopLayer()`) and dragging in the selection strip (SelectMode through `setToolModeFor()`) should therefore work. - Pinch to zoom, two-finger scrolling and long-press for the right-button menu (the pane - emits `rightButtonMenuRequested` on a right click) need adding in the svgui fork. + emits `rightButtonMenuRequested` on a right click) are added by `TouchGestures` in + `main/`, an event filter on each pane; the svgui fork needed no change (phase A4). - `main.cpp`: - `--no-audio` selects `AUDIO_NONE`, which is useful for a first test port; - the window is sized from the screen; @@ -201,7 +202,8 @@ device's rate (svgui's `ViewManager`, an expected failure in `TestRecordWorkflow Ctrl+D action as a button), zoom. - The Show and Play toggles and gains in a slide-out panel. - Hidden: the note-editing tools, the audio device menus, perhaps the spectrogram. -- **Gestures in the svgui fork**: pinch zoom, two-finger scroll, long-press menu. +- **Gestures**: pinch zoom, two-finger scroll, long-press menu (`TouchGestures` in `main/`, + not the svgui fork). - **The latency calibration setting** described above, if a test port needs it. - **The sample-rate check** described above. - **Headphones.** diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 203d33c3..a9a30e6e 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -23,6 +23,7 @@ #include "TakeEvents.h" #include "TakeLayers.h" #include "TakesFile.h" +#include "TouchGestures.h" #ifdef Q_OS_ANDROID #include "AndroidFiles.h" @@ -6032,6 +6033,7 @@ void MainWindow::paneAdded(Pane *pane) { pane->setPlaybackFollow(PlaybackScrollPage); + new TouchGestures(pane); // owned by the pane m_paneStack->sizePanesEqually(); if (m_overview) m_overview->registerView(pane); } diff --git a/main/PinchZoom.cpp b/main/PinchZoom.cpp new file mode 100644 index 00000000..0844ab01 --- /dev/null +++ b/main/PinchZoom.cpp @@ -0,0 +1,104 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "PinchZoom.h" + +#include "data/model/RelativelyFineZoomConstraint.h" + +#include + +using namespace sv; + +namespace PinchZoom +{ + +double +framesPerPixel(ZoomLevel z) +{ + if (z.zone == ZoomLevel::PixelsPerFrame) { + return 1.0 / double(z.level); + } + return double(z.level); +} + +ZoomLevel +nearestLevel(double fpp) +{ + // The wheel steps through View::getZoomConstraintLevel(), which is + // this constraint alone: the layers' own constraints count only for + // a layer that does not supportsOtherZoomLevels(), and none in svgui + // says that. Its limits are the wheel's limits too + RelativelyFineZoomConstraint constraint; + + double minFpp = framesPerPixel(constraint.getMinZoomLevel()); + double maxFpp = framesPerPixel(constraint.getMaxZoomLevel()); + if (!(fpp > minFpp)) fpp = minFpp; // NaN as well + if (fpp > maxFpp) fpp = maxFpp; + + ZoomLevel requested; + if (fpp >= 1.0) { + requested = ZoomLevel(ZoomLevel::FramesPerPixel, + int(std::lround(fpp))); + } else { + requested = ZoomLevel(ZoomLevel::PixelsPerFrame, + int(std::lround(1.0 / fpp))); + } + + return constraint.getNearestZoomLevel(requested, + ZoomConstraint::RoundNearest); +} + +ZoomLevel +pinchedLevel(ZoomLevel current, double targetFpp) +{ + if (!(targetFpp > 0.0)) return current; + + ZoomLevel candidate = nearestLevel(targetFpp); + if (candidate == current) return current; + + // Nearer by 2% at least. Adjacent levels are 10% or more apart, and a + // finger resting on a screen wanders by well under 1% of a pinch + const double margin = std::log(1.02); + + double candidateError = + std::fabs(std::log(framesPerPixel(candidate) / targetFpp)); + double currentError = + std::fabs(std::log(framesPerPixel(current) / targetFpp)); + + if (candidateError + margin < currentError) return candidate; + return current; +} + +double +frameAtX(sv_frame_t centre, ZoomLevel z, int width, double x) +{ + double dx = x - double(width / 2); + + if (z.zone == ZoomLevel::FramesPerPixel) { + // View maps x from the centre rounded down to a whole pixel + sv_frame_t rounded = (centre / z.level) * z.level; + return double(rounded) + dx * double(z.level); + } + + return double(centre) + dx / double(z.level); +} + +sv_frame_t +centreFor(double frame, ZoomLevel z, int width, double x) +{ + double dx = x - double(width / 2); + return sv_frame_t(std::llround(frame - dx * framesPerPixel(z))); +} + +} diff --git a/main/PinchZoom.h b/main/PinchZoom.h new file mode 100644 index 00000000..9a2297a7 --- /dev/null +++ b/main/PinchZoom.h @@ -0,0 +1,68 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_PINCH_ZOOM_H +#define TONY_PINCH_ZOOM_H + +#include "base/BaseTypes.h" +#include "base/ZoomLevel.h" + +/** + * The arithmetic of a two-finger gesture on a pane's time axis: which + * zoom level a pinch asks for, and where the pane must be centred for + * a frame to stay under the fingers. TouchGestures does as the + * answers say. + * + * The x mapping is svgui's View::getFrameForX() made continuous: the + * same centre, rounded the same way, so that a frame placed at x by + * centreFor() is the one the pane shows there, to within a pixel. + * + * Nothing here touches a view, so all of it is tested without a + * window (TestPinchZoom). + */ +namespace PinchZoom +{ + /// Frames per pixel at z: below 1 when zoomed in past one to one + double framesPerPixel(sv::ZoomLevel z); + + /** + * The zoom level nearest to fpp frames per pixel among those the + * mouse wheel steps through, and within the same limits. + */ + sv::ZoomLevel nearestLevel(double fpp); + + /** + * The level for a pinch that asks for targetFpp, with the view at + * current: the nearest level, but current until another is clearly + * nearer, so that fingers held still half way between two levels + * do not make the view flicker between them. + */ + sv::ZoomLevel pinchedLevel(sv::ZoomLevel current, double targetFpp); + + /** + * The frame at x (pixels, may be fractional) in a view of the given + * width centred on centre at zoom z. + */ + double frameAtX(sv::sv_frame_t centre, sv::ZoomLevel z, int width, + double x); + + /** + * The centre that puts frame at x in a view of the given width at + * zoom z. + */ + sv::sv_frame_t centreFor(double frame, sv::ZoomLevel z, int width, + double x); +} + +#endif diff --git a/main/TouchGestures.cpp b/main/TouchGestures.cpp new file mode 100644 index 00000000..b22930c1 --- /dev/null +++ b/main/TouchGestures.cpp @@ -0,0 +1,537 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TouchGestures.h" +#include "PinchZoom.h" + +#include "view/Pane.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +using namespace sv; + +// How Qt 6 turns touch into mouse events, as far as this class depends +// on it (qguiapplication.cpp, processTouchEvent(); qapplication.cpp, +// notify() and translateRawTouchEvent()): +// +// - Each touch event is offered to the widgets first. Only if none +// accepts it does Qt make a mouse event of it, from the point that +// came down first, and it goes on doing so for as long as the events +// are not accepted. Accepting the event that lifts that point would +// leave Qt's mouse button pressed for good. +// +// - The points of a TouchBegin that no widget accepts go to the first +// widget, from the one touched up to the window, that is subscribed +// to a gesture. Otherwise that is QScrollArea's viewport, which +// subscribes to PanGesture, and it would take the pane's second +// finger too. Subscribed to a gesture, the pane gets its points +// itself. The gesture is one that recognises nothing: it is there +// only to be subscribed to. +// +// - Updates to those points are not delivered to the pane after an +// unaccepted TouchBegin, except from the event in which another +// finger comes down: from that one to the TouchEnd, the pane gets +// every point on it. +// +// So a lone finger is seen only as Qt's mouse events. The pane's first +// touch event with two points in it is where the fingers take over: +// from there each touch event is accepted, so that Qt makes no mouse +// event of it, except the one that lifts the first finger, so that Qt +// makes the release that matches its press. A TouchBegin with two points +// in it is accepted at once, and then Qt makes no mouse events at all. + +namespace { + +class SubscriptionOnly : public QGestureRecognizer +{ +public: + Result recognize(QGesture *, QObject *, QEvent *) override { + return Ignore; + } +}; + +Qt::GestureType +subscriptionOnlyGesture() +{ + // The gesture manager owns the recognizer + static Qt::GestureType type = + QGestureRecognizer::registerRecognizer(new SubscriptionOnly); + return type; +} + +bool +isTouchScreen(const QPointingDevice *device) +{ + return device && + device->type() == QInputDevice::DeviceType::TouchScreen; +} + +} + +TouchGestures::TouchGestures(Pane *pane) : + QObject(pane), + m_pane(pane) +{ + m_longPressTimer.setSingleShot(true); + m_longPressTimer.setInterval(longPressMs); + connect(&m_longPressTimer, &QTimer::timeout, + this, &TouchGestures::longPressed); + + pane->setAttribute(Qt::WA_AcceptTouchEvents); + pane->grabGesture(subscriptionOnlyGesture()); + pane->installEventFilter(this); +} + +TouchGestures::~TouchGestures() +{ + stopWatchingMenu(); +} + +bool +TouchGestures::eventFilter(QObject *watched, QEvent *event) +{ + if (m_menu && watched == m_menu) return menuEvent(event); + + if (watched != m_pane || m_replaying) return false; + + switch (event->type()) { + + case QEvent::TouchBegin: + case QEvent::TouchUpdate: + case QEvent::TouchEnd: + case QEvent::TouchCancel: { + // A touch pad is a mouse here + auto te = static_cast(event); + if (!isTouchScreen(te->pointingDevice())) return false; + return touchEvent(te); + } + + case QEvent::MouseButtonPress: + case QEvent::MouseButtonDblClick: + case QEvent::MouseMove: + case QEvent::MouseButtonRelease: { + // Qt's from a touch, and a platform's own (Windows), carry the + // touch screen + auto me = static_cast(event); + if (!isTouchScreen(me->pointingDevice())) return false; + return mouseEvent(me); + } + + default: + return false; + } +} + +bool +TouchGestures::touchEvent(QTouchEvent *e) +{ + // Returns true for all of them: the pane has no use for touch + // events. Accepted or not says whether Qt makes mouse events of it + + if (e->type() == QEvent::TouchCancel) { + // Qt sends a release for the press it made. In Passing it is the + // pane's; otherwise it is to be eaten + dropHeld(); + if (m_state != State::Passing && m_state != State::Idle) { + m_state = m_sourcePressSeen ? State::Ending : State::Idle; + } + m_points.clear(); + m_order.clear(); + m_sourceId = -1; + return true; + } + + if (e->type() == QEvent::TouchBegin) { + + restart(); + for (const QEventPoint &p : e->points()) { + setPoint(p.id(), p.position()); + } + + if (m_order.size() > 1) { + e->accept(); + startTwoFingers(); + } else { + m_sourceId = m_order.value(0, -1); + e->ignore(); + } + return true; + } + + bool sourceLifted = false; + + for (const QEventPoint &p : e->points()) { + if (p.state() == QEventPoint::State::Released) { + removePoint(p.id()); + if (p.id() == m_sourceId) { + sourceLifted = true; + m_sourceId = -1; + } + } else { + setPoint(p.id(), p.position()); + } + } + + if (m_state != State::TwoFingers && m_order.size() > 1) { + startTwoFingers(); + } + + if (m_state != State::TwoFingers) { + e->ignore(); + return true; + } + + if (m_order.size() > 1) { + updatePinch(); + } + + if (sourceLifted) { + e->ignore(); + } else { + e->accept(); + } + + if (e->type() == QEvent::TouchEnd) { + m_points.clear(); + m_order.clear(); + m_sourceId = -1; + m_state = (sourceLifted && m_sourcePressSeen) ? + State::Ending : State::Idle; + } + + return true; +} + +bool +TouchGestures::mouseEvent(QMouseEvent *e) +{ + QEvent::Type type = e->type(); + bool press = (type == QEvent::MouseButtonPress || + type == QEvent::MouseButtonDblClick); + bool release = (type == QEvent::MouseButtonRelease); + bool drag = (type == QEvent::MouseMove && + (e->buttons() & Qt::LeftButton)); + + // Hovering, and any button but the one a finger is, are not ours + if (!press && !release && !drag) return false; + if ((press || release) && e->button() != Qt::LeftButton) return false; + + // A lone finger's touch updates do not come here, its mouse events do + if (m_state == State::Idle || m_state == State::Holding || + m_state == State::Passing) { + m_lastMousePosition = e->position(); + if (m_points.contains(m_sourceId)) { + m_points[m_sourceId] = e->position(); + } + } + + switch (m_state) { + + case State::Idle: + if (!press) return false; // Qt's, from a touch that ended elsewhere + m_sourcePressSeen = true; + m_pressPosition = e->position(); + hold(e); + m_state = State::Holding; + m_longPressTimer.start(); + return true; + + case State::Holding: + if (drag && + (e->position() - m_pressPosition).manhattanLength() <= slop()) { + hold(e); + return true; + } + // Moved or lifted: the pane has all of it, this one last + replayHeld(); + m_state = release ? State::Idle : State::Passing; + return false; + + case State::Passing: + if (release) m_state = State::Idle; + return false; + + case State::LongPressed: + case State::Ending: + if (release) m_state = State::Idle; + return true; + + case State::TwoFingers: + return true; + } + + return false; +} + +bool +TouchGestures::menuEvent(QEvent *event) +{ + switch (event->type()) { + + case QEvent::MouseButtonPress: + case QEvent::MouseButtonDblClick: + case QEvent::MouseMove: + case QEvent::MouseButtonRelease: { + auto me = static_cast(event); + if (!isTouchScreen(me->pointingDevice())) return false; + if (event->type() == QEvent::MouseButtonPress || + event->type() == QEvent::MouseButtonDblClick) { + // Another touch, so that one is over: this is a tap + stopWatchingMenu(); + return false; + } + if (event->type() == QEvent::MouseButtonRelease) { + // Lifted: the menu is the user's now + stopWatchingMenu(); + if (m_state == State::LongPressed) m_state = State::Idle; + } + me->accept(); + return true; + } + + default: + return false; + } +} + +void +TouchGestures::longPressed() +{ + if (m_state != State::Holding) return; + + QPointF position = m_pressPosition; + dropHeld(); + m_state = State::LongPressed; + + // A right press is how the pane is asked for its menu + // (Pane::mousePressEvent). It is the mouse's, not the touch screen's, + // so this filter lets it through + QMouseEvent press(QEvent::MouseButtonPress, position, + m_pane->mapToGlobal(position), + Qt::RightButton, Qt::RightButton, + QGuiApplication::keyboardModifiers()); + QPointer self(this); + sendToPane(&press); + if (!self) return; + + // While the finger is down, Qt sends what it makes of it to the menu + // that has just opened (QWindowPrivate::forwardToPopup()) + QWidget *menu = QApplication::activePopupWidget(); + if (menu) { + m_menu = menu; + menu->installEventFilter(this); + } +} + +void +TouchGestures::stopWatchingMenu() +{ + if (m_menu) m_menu->removeEventFilter(this); + m_menu = nullptr; +} + +void +TouchGestures::restart() +{ + // What was left of the last touch, if its end did not come here (a + // menu took it, say). The pane must not be left pressed + if (m_state == State::Holding) { + dropHeld(); + } else if (m_state == State::Passing) { + endPaneDrag(); + } + + stopWatchingMenu(); + + m_state = State::Idle; + m_points.clear(); + m_order.clear(); + m_sourceId = -1; + m_sourcePressSeen = false; +} + +void +TouchGestures::startTwoFingers() +{ + if (m_state == State::Holding) { + dropHeld(); + } else if (m_state == State::Passing) { + endPaneDrag(); + } + + m_state = State::TwoFingers; + beginPinch(); +} + +void +TouchGestures::beginPinch() +{ + if (m_order.size() < 2) return; + + m_pinchA = m_order[0]; + m_pinchB = m_order[1]; + + QPointF a = m_points.value(m_pinchA); + QPointF b = m_points.value(m_pinchB); + + m_startSpan = QLineF(a, b).length(); + m_startFramesPerPixel = + PinchZoom::framesPerPixel(m_pane->getZoomLevel()); + m_zooming = false; + + m_anchorFrame = PinchZoom::frameAtX + (m_pane->getCentreFrame(), m_pane->getZoomLevel(), + m_pane->width(), (a.x() + b.x()) / 2.0); +} + +void +TouchGestures::updatePinch() +{ + if (!m_points.contains(m_pinchA) || !m_points.contains(m_pinchB)) { + // One of the two was lifted and another is down: a new pinch + // from here + beginPinch(); + return; + } + + QPointF a = m_points.value(m_pinchA); + QPointF b = m_points.value(m_pinchB); + double span = QLineF(a, b).length(); + double x = (a.x() + b.x()) / 2.0; + + // Two fingers dragged together change their distance a little: that + // is not yet a pinch. Twice the slop, as Android's own detector has + if (!m_zooming && std::fabs(span - m_startSpan) > 2 * slop()) { + m_zooming = true; + } + + if (m_zooming && m_startSpan > 0.0 && span > 0.0) { + ZoomLevel current = m_pane->getZoomLevel(); + ZoomLevel level = PinchZoom::pinchedLevel + (current, m_startFramesPerPixel * m_startSpan / span); + if (!(level == current)) { + m_pane->setZoomLevel(level); + } + } + + // The frame that was under the middle of the fingers stays there + ZoomLevel level = m_pane->getZoomLevel(); + int width = m_pane->width(); + sv_frame_t wanted = PinchZoom::centreFor(m_anchorFrame, level, width, x); + + // Within the audio, as a one-finger drag keeps it + // (Pane::dragTopLayer). Held at an end, the view stays while the + // fingers go on: what is under them then is what they hold + sv_frame_t centre = wanted; + sv_frame_t end = m_pane->getModelsEndFrame(); + if (centre >= end) centre = end - 1; + if (centre < 0) centre = 0; + if (centre != wanted) { + m_anchorFrame = PinchZoom::frameAtX(centre, level, width, x); + } + + if (centre != m_pane->getCentreFrame()) { + m_pane->setCentreFrame(centre); + } +} + +void +TouchGestures::hold(QMouseEvent *e) +{ + m_held.push_back(std::unique_ptr(e->clone())); +} + +void +TouchGestures::replayHeld() +{ + m_longPressTimer.stop(); + + std::vector> held; + held.swap(m_held); + + QPointer self(this); + for (auto &e : held) { + sendToPane(e.get()); + if (!self) return; + } +} + +void +TouchGestures::dropHeld() +{ + m_longPressTimer.stop(); + m_held.clear(); +} + +void +TouchGestures::endPaneDrag() +{ + // A release where the finger is, as a mouse released there would + // end it + QPointF position = sourcePosition(); + QMouseEvent release(QEvent::MouseButtonRelease, position, + m_pane->mapToGlobal(position), + Qt::LeftButton, Qt::NoButton, + QGuiApplication::keyboardModifiers()); + sendToPane(&release); +} + +void +TouchGestures::sendToPane(QMouseEvent *e) +{ + bool wasReplaying = m_replaying; + m_replaying = true; + QPointer self(this); + QCoreApplication::sendEvent(m_pane, e); + if (self) m_replaying = wasReplaying; +} + +void +TouchGestures::setPoint(int id, QPointF position) +{ + if (!m_points.contains(id)) m_order.push_back(id); + m_points[id] = position; +} + +void +TouchGestures::removePoint(int id) +{ + m_points.remove(id); + m_order.removeAll(id); +} + +QPointF +TouchGestures::sourcePosition() const +{ + return m_points.value(m_sourceId, m_lastMousePosition); +} + +int +TouchGestures::slop() const +{ + // Qt's distance for a drag to start: in logical pixels, which on a + // phone are the platform's density-independent ones + return QGuiApplication::styleHints()->startDragDistance(); +} diff --git a/main/TouchGestures.h b/main/TouchGestures.h new file mode 100644 index 00000000..9e07f595 --- /dev/null +++ b/main/TouchGestures.h @@ -0,0 +1,143 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TOUCH_GESTURES_H +#define TONY_TOUCH_GESTURES_H + +#include "base/BaseTypes.h" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +class QMouseEvent; +class QTouchEvent; + +namespace sv { +class Pane; +} + +/** + * Touch on one pane: pinch to zoom the time axis about the fingers, + * two fingers dragged to scroll it, and a long press for the pane's + * right-button menu. MainWindow gives every pane one; the pane owns it. + * + * One finger is left to Qt, which makes mouse events of a touch that + * nothing accepts: tapping, dragging in Navigate mode and selecting in + * the selection strip go through the pane's mouse handling as they + * always have. What this adds works on that stream of mouse events: + * + * - The press is held back until the finger moves, lifts, or has been + * down long enough to be a long press. Held back, it cannot start a + * selection or a drag, or schedule a move of the playhead, that a + * long press or a second finger would leave behind. When the finger + * moves or lifts, the pane gets everything held, in order. + * + * - When a second finger comes down, a drag the first had started is + * ended where it is, and the fingers drive the view until they are + * all up. The first finger's mouse events are eaten meanwhile. + * + * - The menu a long press opens comes up under the finger, and QMenu + * takes the release of a button that has moved over an item for a + * choice of it. A resting finger moves, and the first item is Undo: + * so the rest of that finger's events are kept from the menu, which + * then waits for a tap, as a phone's own menus do. + * + * How Qt decides what becomes a mouse event, and why the pane + * subscribes to a gesture that never happens, is in TouchGestures.cpp. + * Input from a mouse is not touched. + */ +class TouchGestures : public QObject +{ + Q_OBJECT + +public: + explicit TouchGestures(sv::Pane *pane); + virtual ~TouchGestures(); + + /// How long one finger must rest for a long press, in ms + static const int longPressMs = 500; + +protected: + bool eventFilter(QObject *watched, QEvent *event) override; + +private: + enum class State { + Idle, // no touch, or one whose press has not come + Holding, // one finger resting: its press held back + Passing, // one finger moving: its events go to the pane + LongPressed, // the menu asked for: the finger's events eaten + TwoFingers, // the fingers drive the view: mouse events eaten + Ending // all up: the release Qt makes is still to eat + }; + + bool touchEvent(QTouchEvent *); + bool mouseEvent(QMouseEvent *); + bool menuEvent(QEvent *); + void longPressed(); + void stopWatchingMenu(); + + void restart(); + void startTwoFingers(); + void beginPinch(); + void updatePinch(); + + void hold(QMouseEvent *); + void replayHeld(); + void dropHeld(); + void endPaneDrag(); + void sendToPane(QMouseEvent *); + + void setPoint(int id, QPointF position); + void removePoint(int id); + QPointF sourcePosition() const; + int slop() const; + + sv::Pane *m_pane; + State m_state = State::Idle; + + QTimer m_longPressTimer; + QPointer m_menu; // the one a long press opened, until lifted + std::vector> m_held; + QPointF m_pressPosition; + QPointF m_lastMousePosition; + bool m_replaying = false; + + // The touch points down on the pane, in pane coordinates, and the + // order they came down in + QHash m_points; + QList m_order; + + // The point Qt makes the mouse events from, while it is down, and + // whether its press came to this pane + int m_sourceId = -1; + bool m_sourcePressSeen = false; + + // The two points of the pinch, and the view when they came down + int m_pinchA = -1; + int m_pinchB = -1; + double m_startSpan = 0.0; + double m_startFramesPerPixel = 1.0; + double m_anchorFrame = 0.0; + bool m_zooming = false; +}; + +#endif diff --git a/main/test/TestPinchZoom.h b/main/test/TestPinchZoom.h new file mode 100644 index 00000000..3e58c521 --- /dev/null +++ b/main/test/TestPinchZoom.h @@ -0,0 +1,152 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_PINCH_ZOOM_H +#define TEST_PINCH_ZOOM_H + +// Tier 2: the arithmetic of a pinch on a pane's time axis. Zoom levels +// and frames in, zoom levels and frames out; no view. + +#include "../PinchZoom.h" + +#include "data/model/RelativelyFineZoomConstraint.h" + +#include +#include + +#include +#include + +class TestPinchZoom : public QObject +{ + Q_OBJECT + + typedef sv::ZoomLevel ZoomLevel; + typedef sv::sv_frame_t frame_t; + + static ZoomLevel fpp(int level) { + return ZoomLevel(ZoomLevel::FramesPerPixel, level); + } + + static ZoomLevel ppf(int level) { + return ZoomLevel(ZoomLevel::PixelsPerFrame, level); + } + + static QString text(ZoomLevel z) { + return QString("%1 %2") + .arg(z.zone == ZoomLevel::FramesPerPixel ? "fpp" : "ppf") + .arg(z.level); + } + +private slots: + void frames_per_pixel_in_both_zones() { + QCOMPARE(PinchZoom::framesPerPixel(fpp(64)), 64.0); + QCOMPARE(PinchZoom::framesPerPixel(fpp(1)), 1.0); + QCOMPARE(PinchZoom::framesPerPixel(ppf(4)), 0.25); + } + + // Every level the mouse wheel steps through, from far out to as far + // in as it goes, is one a pinch can land on exactly + void pinch_levels_are_the_wheel_levels() { + sv::RelativelyFineZoomConstraint constraint; + ZoomLevel level = constraint.getNearestZoomLevel(fpp(65536)); + int steps = 0; + while (true) { + ZoomLevel found = + PinchZoom::nearestLevel(PinchZoom::framesPerPixel(level)); + QVERIFY2(found == level, + qPrintable(text(level) + " came back as " + + text(found))); + ZoomLevel next = constraint.getNearestZoomLevel + (level.decremented(), sv::ZoomConstraint::RoundDown); + if (next == level) break; + level = next; + ++steps; + } + QVERIFY(steps > 50); + QVERIFY(level == constraint.getMinZoomLevel()); + } + + void nearest_level_between_two() { + QVERIFY(PinchZoom::nearestLevel(33.0) == fpp(32)); + QVERIFY(PinchZoom::nearestLevel(0.26) == ppf(4)); + QVERIFY(PinchZoom::nearestLevel(1.0) == fpp(1)); + } + + // As far as the wheel goes and no further, whatever is asked + void nearest_level_is_limited_as_the_wheel_is() { + sv::RelativelyFineZoomConstraint constraint; + QVERIFY(PinchZoom::nearestLevel(1.0e12) == + constraint.getMaxZoomLevel()); + QVERIFY(PinchZoom::nearestLevel(1.0e-6) == + constraint.getMinZoomLevel()); + QVERIFY(PinchZoom::nearestLevel(0.0) == + constraint.getMinZoomLevel()); + QVERIFY(PinchZoom::nearestLevel + (std::numeric_limits::quiet_NaN()) == + constraint.getMinZoomLevel()); + } + + // 64 and 72 are neighbours. Between them, whichever the view is at + // stays until the other is nearer by 2%: at 67.2 and 68.6 frames per + // pixel + void pinch_between_two_levels_stays_where_it_is() { + for (double target : { 67.3, 68.0, 68.5 }) { + QVERIFY2(PinchZoom::pinchedLevel(fpp(64), target) == fpp(64), + qPrintable(QString("from 64 at %1").arg(target))); + QVERIFY2(PinchZoom::pinchedLevel(fpp(72), target) == fpp(72), + qPrintable(QString("from 72 at %1").arg(target))); + } + QVERIFY(PinchZoom::pinchedLevel(fpp(64), 68.8) == fpp(72)); + QVERIFY(PinchZoom::pinchedLevel(fpp(72), 67.0) == fpp(64)); + } + + void pinch_goes_as_far_as_it_asks() { + QVERIFY(PinchZoom::pinchedLevel(fpp(64), 16.0) == fpp(16)); + QVERIFY(PinchZoom::pinchedLevel(fpp(64), 256.0) == fpp(256)); + QVERIFY(PinchZoom::pinchedLevel(fpp(2), 0.25) == ppf(4)); + QVERIFY(PinchZoom::pinchedLevel(fpp(64), 0.0) == fpp(64)); + } + + // View::getFrameForX(): from the centre rounded down to a whole + // pixel, and width / 2 rounded down + void frame_at_x_as_the_view_has_it() { + QCOMPARE(PinchZoom::frameAtX(10030, fpp(64), 1000, 500), 9984.0); + QCOMPARE(PinchZoom::frameAtX(10030, fpp(64), 1000, 600), + 9984.0 + 6400.0); + QCOMPARE(PinchZoom::frameAtX(10030, fpp(64), 999, 499), 9984.0); + QCOMPARE(PinchZoom::frameAtX(10030, fpp(64), 1000, 499.5), + 9984.0 - 32.0); + QCOMPARE(PinchZoom::frameAtX(1000, ppf(4), 1000, 520), 1005.0); + } + + // What centreFor() puts at x is shown there, to within a pixel + void centre_for_puts_the_frame_at_x() { + for (ZoomLevel z : { fpp(64), fpp(7), fpp(1), ppf(4) }) { + for (double x : { 0.0, 250.0, 500.0, 731.5, 999.0 }) { + for (double frame : { 100000.0, 123457.0 }) { + frame_t centre = PinchZoom::centreFor(frame, z, 1000, x); + double shown = PinchZoom::frameAtX(centre, z, 1000, x); + double pixel = PinchZoom::framesPerPixel(z); + QVERIFY2(shown > frame - pixel - 0.5 && + shown <= frame + 0.5, + qPrintable(QString("%1 at x %2, %3: shows %4") + .arg(frame).arg(x).arg(text(z)) + .arg(shown))); + } + } + } + } +}; + +#endif diff --git a/main/test/TestTouchGestures.h b/main/test/TestTouchGestures.h new file mode 100644 index 00000000..29684916 --- /dev/null +++ b/main/test/TestTouchGestures.h @@ -0,0 +1,562 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TOUCH_GESTURES_H +#define TEST_TOUCH_GESTURES_H + +// Tier 5: touch on the panes of the real MainWindow (TouchGestures). +// Pinch, two fingers dragged and a long press; one finger and the +// mouse as they were. +// +// The touch goes in where a platform's does (QTest::touchEvent goes +// through QWindowSystemInterface), so Qt makes mouse events of it as it +// does on a phone, and a mouse button left pressed shows in +// QGuiApplication::mouseButtons(). Qt finds the widget under a touch +// only among widgets on show, so the window is shown here. + +#include "TestSignals.h" + +#include "../MainWindow.h" +#include "../Analyser.h" +#include "../PinchZoom.h" +#include "../TouchGestures.h" + +#include "version.h" + +#include "view/Pane.h" +#include "view/PaneStack.h" +#include "view/ViewManager.h" +#include "data/fileio/WavFileWriter.h" +#include "data/model/RelativelyFineZoomConstraint.h" +#include "transform/ModelTransformerFactory.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +/** + * MainWindow without an audio device, and with the pane's right-button + * menu counted instead of shown: a menu on show would take the touch + * events that come after it. A test that wants a menu there gives one. + */ +class TouchTestWindow : public MainWindow +{ +public: + TouchTestWindow() : MainWindow(AUDIO_PLAYBACK_AND_RECORD, true, false) { } + + sv::PaneStack *paneStack() { return m_paneStack; } + sv::ViewManager *viewManager() { return m_viewManager; } + Analyser *analyser() { return m_analyser; } + + void discardModifications() { m_documentModified = false; } + void doCloseSession() { discardModifications(); closeSession(); } + + int menuRequests = 0; + QPoint menuPosition; + QMenu *menu = nullptr; + +protected: + void createAudioIO() override { } + + // As MainWindow's own does, with the menu given + void paneRightButtonMenuRequested(sv::Pane *, QPoint position) override { + ++menuRequests; + menuPosition = position; + if (menu) menu->popup(position); + } +}; + +class TestTouchGestures : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + + // Where each test starts: frames per pixel, and the centre frame + static constexpr int startLevel = 64; + static constexpr sv::sv_frame_t startCentre = 3 * 44100; + + QTemporaryDir m_dir; + TouchTestWindow *m_window = nullptr; + QPointingDevice *m_touch = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + + typedef QTest::QTouchEventWidgetSequence Touch; + + // Points are given in the coordinates of the pane they are on + Touch touch() { return QTest::touchEvent(m_window, m_touch, false); } + + sv::Pane *pane() { return m_window->paneStack()->getPane(0); } + sv::Pane *strip() { return m_window->paneStack()->getPane(1); } + + double framesPerPixel() { + return PinchZoom::framesPerPixel(pane()->getZoomLevel()); + } + + sv::MultiSelection::SelectionList selections() { + return m_window->viewManager()->getSelections(); + } + + bool analysed() { + Analyser *a = m_window->analyser(); + return a && a->getLayer(Analyser::PitchTrack) && + a->getLayer(Analyser::Notes) && + a->getInitialAnalysisCompletion() >= 100 && + !a->isAnalysingRange() && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + } + + // The window on show with six seconds of reference, analysed, and + // the view at startLevel about startCentre + void openWindow() { + m_window = new TouchTestWindow; + m_window->resize(1000, 700); + m_window->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_window)); + + QString path = m_dir.filePath("reference.wav"); + if (!QFile::exists(path)) { + std::vector data = TestSignals::sine + (220.5, rate, int(6 * rate), 0.5); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + QVERIFY(writer.isOK()); + QVERIFY(writer.writeSamples(&ptr, sv::sv_frame_t(data.size()))); + QVERIFY(writer.close()); + } + + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(), 30000); + + QVERIFY(pane()); + QVERIFY(strip()); + pane()->setZoomLevel + (sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, startLevel)); + pane()->setCentreFrame(startCentre); + QCOMPARE(framesPerPixel(), double(startLevel)); + } + + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + m_dialogs.push_back(description); + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + // Otherwise the MainWindow constructor asks, in a dialog + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + + // A device of its own for each test: fingers that a failed test + // left down stay on the last one + m_touch = QTest::createTouchDevice(); + } + + void cleanup() { + if (m_window) { + // A test that failed with Qt's button pressed must not leave + // it so for the next, which may make no mouse events to clear it + if (QGuiApplication::mouseButtons() != Qt::NoButton) { + QTest::mouseRelease(m_window->windowHandle(), Qt::LeftButton); + } + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + } + + // Fingers twice as far apart: twice as far in, with the frame that was + // between them still there. The second finger comes down after the + // first, and the first is lifted first + void pinch_out_zooms_in_about_the_fingers() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2 - 100; // off centre: not just the centre kept + int y = p->height() / 2; + sv::sv_frame_t before = p->getFrameForX(x); + + Touch t = touch(); + t.press(0, QPoint(x - 50, y), p).commit(); + t.stationary(0).press(1, QPoint(x + 50, y), p).commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, QPoint(x - 50 - 5 * i, y), p) + .move(1, QPoint(x + 50 + 5 * i, y), p).commit(); + } + t.release(0, QPoint(x - 100, y), p).stationary(1).commit(); + t.release(1, QPoint(x + 100, y), p).commit(); + + QCOMPARE(framesPerPixel(), startLevel / 2.0); + sv::sv_frame_t after = p->getFrameForX(x); + QVERIFY2(std::abs(after - before) <= startLevel / 2, + qPrintable(QString("frame %1 under the fingers, then %2") + .arg(before).arg(after))); + + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // Fingers half as far apart: twice as far out. Both come down, and + // go up, together + void pinch_in_zooms_out_about_the_fingers() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2 + 150; + int y = p->height() / 2; + sv::sv_frame_t before = p->getFrameForX(x); + + Touch t = touch(); + t.press(0, QPoint(x - 100, y), p).press(1, QPoint(x + 100, y), p) + .commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, QPoint(x - 100 + 5 * i, y), p) + .move(1, QPoint(x + 100 - 5 * i, y), p).commit(); + } + t.release(0, QPoint(x - 50, y), p).release(1, QPoint(x + 50, y), p) + .commit(); + + QCOMPARE(framesPerPixel(), startLevel * 2.0); + sv::sv_frame_t after = p->getFrameForX(x); + QVERIFY2(std::abs(after - before) <= startLevel * 2, + qPrintable(QString("frame %1 under the fingers, then %2") + .arg(before).arg(after))); + + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // Two fingers dragged left by 100 pixels bring 100 pixels' worth of + // what follows into view, and do not zoom. They come down together: + // Qt then makes no mouse events at all, so what moves the view can + // only be the two of them (one finger dragged the same way would + // move it as far) + void two_finger_drag_scrolls() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 2; + sv::sv_frame_t before = p->getCentreFrame(); + + Touch t = touch(); + t.press(0, QPoint(x - 50, y), p).press(1, QPoint(x + 50, y), p) + .commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, QPoint(x - 50 - 10 * i, y), p) + .move(1, QPoint(x + 50 - 10 * i, y), p).commit(); + } + t.release(0, QPoint(x - 150, y), p).release(1, QPoint(x - 50, y), p) + .commit(); + + QCOMPARE(framesPerPixel(), double(startLevel)); + sv::sv_frame_t expected = before + 100 * startLevel; + QVERIFY2(std::abs(p->getCentreFrame() - expected) <= startLevel, + qPrintable(QString("centre %1, expected %2") + .arg(p->getCentreFrame()).arg(expected))); + + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // One finger held still asks for the menu where it is, as a right + // click there does. It does not drag afterwards, and the pane never + // had the press: no move of the playhead is scheduled + void long_press_asks_for_the_menu() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + QPoint at(p->width() / 2 + 120, p->height() / 2); + sv::sv_frame_t centre = p->getCentreFrame(); + sv::sv_frame_t playhead = m_window->viewManager()->getPlaybackFrame(); + QVERIFY(std::abs(p->getFrameForX(at.x()) - playhead) > 10000); + + Touch t = touch(); + t.press(0, at, p).commit(); + QTest::qWait(TouchGestures::longPressMs / 2); + QCOMPARE(m_window->menuRequests, 0); + QTest::qWait(TouchGestures::longPressMs / 2 + 200); + QCOMPARE(m_window->menuRequests, 1); + QCOMPARE(m_window->menuPosition, p->mapToGlobal(at)); + + for (int i = 1; i <= 5; ++i) { + t.move(0, at - QPoint(20 * i, 0), p).commit(); + } + t.release(0, at - QPoint(100, 0), p).commit(); + QTest::qWait(QApplication::doubleClickInterval() + 100); + + QCOMPARE(p->getCentreFrame(), centre); + QCOMPARE(m_window->viewManager()->getPlaybackFrame(), playhead); + QCOMPARE(m_window->menuRequests, 1); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // In the selection strip, where a press starts a selection + void long_press_leaves_no_selection() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *s = strip(); + QPoint at(s->width() / 2, s->height() / 2); + QVERIFY(selections().empty()); + + Touch t = touch(); + t.press(0, at, s).commit(); + QTest::qWait(TouchGestures::longPressMs + 200); + QCOMPARE(m_window->menuRequests, 1); + + for (int i = 1; i <= 5; ++i) { + t.move(0, at + QPoint(20 * i, 0), s).commit(); + } + t.release(0, at + QPoint(100, 0), s).commit(); + + QVERIFY(selections().empty()); + QVERIFY(!m_window->viewManager()->haveInProgressSelection()); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // The menu comes up under the finger. The finger wandering onto its + // first item (Undo, in Tony's) and lifted there does not choose it; + // a tap on it afterwards does + void long_press_menu_waits_for_a_tap() { + openWindow(); + if (QTest::currentTestFailed()) return; + + QMenu *menu = new QMenu(m_window); + QAction *first = menu->addAction("First"); + menu->addAction("Second"); + int chosen = 0; + connect(first, &QAction::triggered, this, [&chosen]() { ++chosen; }); + m_window->menu = menu; + + sv::Pane *p = pane(); + QPoint at(p->width() / 2, p->height() / 2); + + Touch t = touch(); + t.press(0, at, p).commit(); + QTest::qWait(TouchGestures::longPressMs + 200); + QCOMPARE(m_window->menuRequests, 1); + QVERIFY(menu->isVisible()); + + QPoint item = p->mapFromGlobal + (menu->mapToGlobal(menu->actionGeometry(first).center())); + for (int i = 1; i <= 10; ++i) { + t.move(0, at + (item - at) * i / 10, p).commit(); + } + t.release(0, item, p).commit(); + + QCOMPARE(chosen, 0); + QVERIFY(menu->isVisible()); + + QTest::qWait(QApplication::doubleClickInterval() + 100); + Touch tap = touch(); + tap.press(0, item, p).commit(); + tap.release(0, item, p).commit(); + + QTRY_COMPARE(chosen, 1); + QVERIFY(!menu->isVisible()); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // One finger dragged in Navigate mode moves the view exactly as the + // mouse dragged the same way does + void one_finger_drag_pans_as_the_mouse_does() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int y = p->height() / 2; + QPoint from(p->width() / 2 + 100, y); + QPoint to(p->width() / 2, y); + sv::sv_frame_t start = p->getCentreFrame(); + + QTest::mousePress(p, Qt::LeftButton, Qt::NoModifier, from); + for (int i = 1; i <= 10; ++i) { + QTest::mouseMove(p, from + (to - from) * i / 10); + } + QTest::mouseRelease(p, Qt::LeftButton, Qt::NoModifier, to); + sv::sv_frame_t byMouse = p->getCentreFrame() - start; + QVERIFY2(byMouse > 90 * startLevel, + qPrintable(QString("the mouse moved it by %1") + .arg(byMouse))); + + p->setCentreFrame(start); + QCOMPARE(p->getCentreFrame(), start); + + Touch t = touch(); + t.press(0, from, p).commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, from + (to - from) * i / 10, p).commit(); + } + t.release(0, to, p).commit(); + sv::sv_frame_t byTouch = p->getCentreFrame() - start; + + QCOMPARE(byTouch, byMouse); + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // A tap is a click: in Navigate mode the playhead goes there + void one_finger_tap_moves_the_playhead() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + QPoint at(p->width() / 2 + 150, p->height() / 2); + sv::sv_frame_t target = p->getFrameForX(at.x()); + QVERIFY(std::abs(m_window->viewManager()->getPlaybackFrame() - + target) > 10000); + + Touch t = touch(); + t.press(0, at, p).commit(); + t.release(0, at, p).commit(); + + QTRY_VERIFY_WITH_TIMEOUT + (std::abs(m_window->viewManager()->getPlaybackFrame() - target) + <= startLevel, 2000); + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // A finger dragging out a selection, and a second finger coming down: + // the selection ends where the first finger was, and nothing either + // finger does after that changes it or leaves a button pressed + void second_finger_ends_a_selection_drag() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *s = strip(); + sv::ViewManager *vm = m_window->viewManager(); + int y = s->height() / 2; + int x0 = s->width() / 2 - 100; + int x1 = x0 + 100; + sv::sv_frame_t f0 = s->getFrameForX(x0); + sv::sv_frame_t f1 = s->getFrameForX(x1); + + Touch t = touch(); + t.press(0, QPoint(x0, y), s).commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, QPoint(x0 + 10 * i, y), s).commit(); + } + QVERIFY(vm->haveInProgressSelection()); + + t.stationary(0).press(1, QPoint(x1 + 80, y), s).commit(); + QVERIFY(!vm->haveInProgressSelection()); + QCOMPARE(int(selections().size()), 1); + sv::Selection made = *selections().begin(); + QVERIFY2(std::abs(made.getStartFrame() - f0) <= startLevel && + std::abs(made.getEndFrame() - f1) <= startLevel, + qPrintable(QString("selected %1 to %2, dragged %3 to %4") + .arg(made.getStartFrame()) + .arg(made.getEndFrame()).arg(f0).arg(f1))); + + for (int i = 1; i <= 5; ++i) { + t.move(0, QPoint(x1 + 20 * i, y), s) + .move(1, QPoint(x1 + 80 + 20 * i, y), s).commit(); + } + t.release(0, QPoint(x1 + 100, y), s) + .release(1, QPoint(x1 + 180, y), s).commit(); + + QCOMPARE(int(selections().size()), 1); + QVERIFY(*selections().begin() == made); + QVERIFY(!vm->haveInProgressSelection()); + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // The wheel zooms in one step about the centre, as svgui has it + void mouse_wheel_zoom_is_unchanged() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + sv::sv_frame_t centre = p->getCentreFrame(); + QPointF at(p->width() / 2 - 100, p->height() / 2); + + QWheelEvent wheel(at, p->mapToGlobal(at), QPoint(), QPoint(0, 120), + Qt::NoButton, Qt::NoModifier, Qt::NoScrollPhase, + false); + QCoreApplication::sendEvent(p, &wheel); + + sv::ZoomLevel expected = sv::RelativelyFineZoomConstraint() + .getNearestZoomLevel + (sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, + startLevel).decremented(), + sv::ZoomConstraint::RoundDown); + QVERIFY(framesPerPixel() < startLevel); + QVERIFY(p->getZoomLevel() == expected); + QCOMPARE(p->getCentreFrame(), centre); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 6ad3eaf0..79725856 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -14,6 +14,7 @@ #include "TestSingingDocument.h" #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" +#include "TestTouchGestures.h" #include "RunSuite.h" @@ -70,6 +71,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTouchGestures t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 7e9faf17..1cb7ea68 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -16,6 +16,7 @@ #include "TestRealtimePitchTracker.h" #include "TestLatencyShift.h" #include "TestCoverage.h" +#include "TestPinchZoom.h" #include "TestTakeAudio.h" #include "TestTakeEvents.h" #include "TestSingingTakes.h" @@ -75,6 +76,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestPinchZoom t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestTakeAudio t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 37147c40..afe1f3e6 100644 --- a/meson.build +++ b/meson.build @@ -1167,6 +1167,7 @@ tony_entry_files = [ tony_core_files = [ 'main/AndroidFiles.cpp', 'main/Coverage.cpp', + 'main/PinchZoom.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', @@ -1184,6 +1185,7 @@ tony_app_files = [ 'main/PaneUtils.cpp', 'main/TakeCommands.cpp', 'main/TakeLayers.cpp', + 'main/TouchGestures.cpp', ] tony_core_moc_files = qt.preprocess( @@ -1198,6 +1200,7 @@ tony_app_moc_files = qt.preprocess( 'main/Analyser.h', 'main/AlternatePitchTrack.h', 'main/CoverageStrip.h', + 'main/TouchGestures.h', ]) qt_resource_files = qt.preprocess( @@ -1465,6 +1468,7 @@ if system != 'android' 'main/test/TestRealtimePitchTracker.h', 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', + 'main/test/TestPinchZoom.h', 'main/test/TestTakeAudio.h', 'main/test/TestTakeEvents.h', 'main/test/TestSingingTakes.h', @@ -1502,6 +1506,7 @@ if system != 'android' 'main/test/TestSingingDocument.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', + 'main/test/TestTouchGestures.h', ]) tony_app_test_exe = executable( From 42b7650af40bd4ae824c723308b15fd6df0216a0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 02:49:13 +0000 Subject: [PATCH 125/275] docs: phase a4 done, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 4b3d3806..c8bba239 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -156,7 +156,7 @@ builds happen in the container.) - A3b — Tony as an APK (no audio): the test port. Built but for the APK itself: Gradle was blocked (`dl.google.com` refused); run `deploy/android/build-apk.sh` once it is allowed (see the log). -- A4 — Touch gestures on the panes. +- A4 — Touch gestures on the panes. Done. - A5 — Compact touch mode. - A6 — Oboe audio backend. - A7 — Android files, permission and lifecycle. @@ -441,3 +441,28 @@ The next phase must know: logcat tag `Tony` has Tony's and svcore's cerr; SVDEBU to `files/log/sv-debug.log` (`adb shell run-as io.github.jhhr.tony cat ...`). Phone test: install; File > Open, pick audio: waveform, then pitch and notes; Play is off (no audio). Left open: saving and sessions through the picker (A7); `imported/` is never emptied. + +### Phase A4 — 2026-09-26 +Built: `main/TouchGestures` (tony_app; `MainWindow::paneAdded()` gives each pane one): pinch +zooms the time axis about the fingers, two fingers scroll it, a 500 ms long press sends the +pane a right press (its menu path). `main/PinchZoom` (tony_core): the wheel's zoom grid and +limits for a pinch, with 2% hysteresis; View's x mapping made continuous. Tests: +`TestPinchZoom` (core), `TestTouchGestures` (app; window shown, touch via QTest::touchEvent). +Qt 6.11 (qapplication.cpp, qguiapplication.cpp): only an unaccepted touch event becomes mouse +events, from its first finger; an unaccepted TouchBegin's points go to the first widget above +that subscribes to a gesture: QScrollArea's viewport (PanGesture), which then takes the +second finger too. So the pane subscribes to a gesture that recognises nothing; a lone finger +stays unaccepted (Qt's mouse events, as before); the second finger's event reaches the pane +with both points; from there events are accepted except the one lifting the first finger. +Choices / deviations: +- The first press is held until the finger moves past startDragDistance or lifts (then + replayed in order) or 500 ms pass: a long press or second finger leaves no drag, selection + or playhead move. A drag already going is ended by a release where the finger is. +- The long-press menu opens under the finger, its first item Undo; QMenu takes a release + after 7 moves over an item as a choice. That finger's events are eaten on the menu until + it lifts, so the menu waits for a tap (a test shows the lift choosing it otherwise). +- Vertical two-finger movement is ignored. The viewport's own pan gesture still scrolls the + pane stack vertically if it can, as before A4. Pinch starts at 2x startDragDistance. +The next phase must know: touch tests need the window shown and a fresh device per test. +Left open: not tried on a touch screen. Windows desktop touch (OS-made mouse events, its own +press-and-hold right click) untested. For A8: architecture.md, testing.md, manual-checklist. From 1ef24940997197232cbacb47a8c8965d20c56149 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:17:41 +0000 Subject: [PATCH 126/275] feat: TakeDiff, the pure comparisons the dev checks make on a take Audio bit-identical outside a range, events unchanged outside a range widened by 0.25 s, the pitch track and one note running through a join, and no step in the samples at a join. Each returns whether it passed and the numbers behind it, so a check can report them. No model or window: TestTakeDiff runs in milliseconds, two of its tests through the real splice and erase. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 13 + docs/calibrate-audio.md | 2 +- main/TakeDiff.cpp | 333 +++++++++++++++ main/TakeDiff.h | 302 +++++++++++++ main/test/TestTakeDiff.h | 634 ++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 7 files changed, 1292 insertions(+), 1 deletion(-) create mode 100644 main/TakeDiff.cpp create mode 100644 main/TakeDiff.h create mode 100644 main/test/TestTakeDiff.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index fa225751..8ad34471 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -575,3 +575,16 @@ Left open: Record stays enabled during a check. Pressing it there goes through t - The button is complete; the user's Windows run is the checkpoint (spec §7). C0 onwards goes on meanwhile. - Left for C1 (small, in passing): Record stays enabled during a check, and pressing it ends the check's take early; grey it while a check runs. - Left for D: spec §2 promises the mic channel and noise floor on the result page, which nothing measures yet (C2's item 5 measures the channel); §8 says "modal progress dialog", true only of the dev run; Check Again always asks to save the check's own session (answer No), a possible later nicety. + +### Phase C0 — 2026-09-26 +Built: `main/TakeDiff.{h,cpp}` (`tony_core`, namespace). Five comparisons, each returning a struct with `pass` and its numbers: `audioOutside()` → `AudioDiff {firstDifference, differences, largestDifference}`; `eventsOutside()` → `EventDiff {added, removed, changed (before, after), firstDifference, window}`; `pitchAcross()` → `PitchJoin {events, largestGap, largestGapFrom, doubled, firstDoubled, outOfOrder, firstOutOfOrder, window}`; `notesAcross()` → `NoteJoin {spanning, edgesNear, nearestEdge}`; `stepAt()` → `SampleStep {stepDb, largest, typical, largestAt, channel}`. Samples are interleaved (`const float *`, frames, channels); times are named constants in seconds, with the rate passed. Core class `TestTakeDiff`: 19 tests, 10 ms. +Choices / deviations: +- Fades: `weightAt()` in `TakeAudio.cpp` mixes only frames of [start, end), both edge frames included, and copies every other frame; take files are float WAV. So `audioOutside()` excuses nothing outside the range: pass the placed range (the coverage added). A test runs the real `splice()` and `erase()` through files: identical outside, and the range less one frame at either end differs exactly at that frame. Frames past a buffer's end are silence; bits are compared, not values. +- Events: "inside" is what `TakeEvents::eraseNotes()` of the window would touch (a crossing note is inside; a pitch event goes by its frame). Window [start − 0.25 s, end + 0.25 s), half-open. Multiset difference; what is left at one frame is "changed". A note that grew from outside into the window reads as removed. +- Pitch window ±0.5 s (`kPitchWindowSeconds`), not ±0.25: the second run's merge seam is 0.25 s before the join, and would sit on the edge of a ±0.25 s window. Gap limit 1 hop (`kMaxGapHops`). Gaps run to the neighbours beyond the window, or to its edge if there are none, so a hole reaching in, a track stopping inside, or an empty window all fail. +- Notes: X = 0.5 s (`kNoteClearanceSeconds`), closed. A note is [f, f + d), so one ending at the join does not hold it. +- Step: the largest |x[i] − x[i−1]| within ±2 ms, against the 95th percentile over the 50 ms around, in the worst channel. Measured: steady tone 0.03 dB, noise 2.7 dB (6.7 at most in 200 simulated trials), hard cut 36 dB, and the real splice 0.03 dB with its fade, 36 dB without. `kMaxStepDb` = 10 dB; about one random cut in ten reads under it (the two sides nearly meeting). +The next phase must know: +- **Two punch-ins that meet at J leave a 10 ms dip, not a crossfade.** Each fades against what the file held there, which is silence: [J − 5 ms, J) fades out and [J, J + 5 ms) fades in (read from `weightAt()`, not measured). `stepAt()` reads it as no step. +- The notes merge adds a new note only if its onset is in W (`Analyser.cpp`, "Notes go by their onset"). A second run's note that begins before J − 0.25 s is not added, and the dip may split the note at J. Either way, item 10's "one note across the join" may fail on today's code. C3 should measure it, not assume it. +Left open: every threshold untuned; nothing calls `TakeDiff` yet. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index ec09d597..8d481dab 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -300,7 +300,7 @@ marked "Done" when it is committed. thresholds and the restart-jitter remedy wait on those numbers. 3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). 4. **Dev-check framework:** - - **C0** `TakeDiff`, pure. + - **C0** `TakeDiff`, pure. Done. - **C1** Build flag, `DevChecks`, `TakeObserver`, report, friend access. First group: items 1, 2, 7, 12, 13, 14. 5. **C2** Observer group: items 3, 4, 5, 8, 15, 16. diff --git a/main/TakeDiff.cpp b/main/TakeDiff.cpp new file mode 100644 index 00000000..352534fd --- /dev/null +++ b/main/TakeDiff.cpp @@ -0,0 +1,333 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TakeDiff.h" + +#include "TakeEvents.h" + +#include +#include +#include +#include +#include +#include + +using namespace sv; + +namespace { + +sv_frame_t +framesOf(double seconds, sv_samplerate_t rate) +{ + return sv_frame_t(std::llround(seconds * rate)); +} + +bool +sameBits(float a, float b) +{ + uint32_t x, y; + std::memcpy(&x, &a, sizeof x); + std::memcpy(&y, &b, sizeof y); + return x == y; +} + +// A ratio in dB that stays finite, as LatencyCheck has it: silence +// against silence is 0 dB, and anything against silence is 200 +double +decibels(double value, double against) +{ + const double limit = 200.0; + if (value <= 0.0 && against <= 0.0) return 0.0; + if (value <= 0.0) return -limit; + if (against <= 0.0) return limit; + double db = 20.0 * std::log10(value / against); + return std::max(-limit, std::min(limit, db)); +} + +// The events no part of which lies in the window, sorted. What +// erasing the window would touch is what lies in it: a note that +// crosses its edge goes with it, and an event with no duration, a +// pitch event, goes by its frame alone +EventVector +outside(const EventVector &events, const Coverage::Range &window) +{ + EventVector all(events); + EventVector in = TakeEvents::eraseNotes + (events, Coverage::Ranges { window }).removed; + std::sort(all.begin(), all.end()); + std::sort(in.begin(), in.end()); + + EventVector result; + std::set_difference(all.begin(), all.end(), in.begin(), in.end(), + std::back_inserter(result)); + return result; +} + +} // namespace + +TakeDiff::AudioDiff +TakeDiff::audioOutside(const float *before, sv_frame_t beforeFrames, + const float *after, sv_frame_t afterFrames, + int channels, const Coverage::Range &range) +{ + AudioDiff result; + if (channels < 1) return result; + + auto compare = [&](sv_frame_t from, sv_frame_t to) { + for (sv_frame_t i = from; i < to; ++i) { + bool differs = false; + for (int c = 0; c < channels; ++c) { + float a = (i < beforeFrames) ? before[i * channels + c] : 0.f; + float b = (i < afterFrames) ? after[i * channels + c] : 0.f; + if (sameBits(a, b)) continue; + differs = true; + result.largestDifference = std::max + (result.largestDifference, + std::fabs(double(a) - double(b))); + } + if (differs) { + if (result.firstDifference < 0) result.firstDifference = i; + ++result.differences; + } + } + }; + + // An empty or backward range leaves out nothing + sv_frame_t total = std::max(beforeFrames, afterFrames); + sv_frame_t skipFrom = std::max(sv_frame_t(0), std::min(range.start, total)); + sv_frame_t skipTo = std::max(skipFrom, std::min(range.end, total)); + compare(0, skipFrom); + compare(skipTo, total); + + result.pass = (result.differences == 0); + return result; +} + +TakeDiff::EventDiff +TakeDiff::eventsOutside(const EventVector &before, const EventVector &after, + const Coverage::Range &range, sv_samplerate_t rate, + double marginSeconds) +{ + EventDiff result; + + sv_frame_t margin = framesOf(marginSeconds, rate); + result.window = Coverage::Range(range.start - margin, range.end + margin); + + EventVector was = outside(before, result.window); + EventVector is = outside(after, result.window); + + // What only one side has, event for event: the very same event on + // both sides is no change, however many of it there are + EventVector onlyWas, onlyIs; + std::set_difference(was.begin(), was.end(), is.begin(), is.end(), + std::back_inserter(onlyWas)); + std::set_difference(is.begin(), is.end(), was.begin(), was.end(), + std::back_inserter(onlyIs)); + + // Both sorted by frame first: pair what is left at one frame as + // changed, in order, and the rest was removed or added + size_t i = 0, j = 0; + while (i < onlyWas.size() || j < onlyIs.size()) { + if (j == onlyIs.size() || + (i < onlyWas.size() && + onlyWas[i].getFrame() < onlyIs[j].getFrame())) { + result.removed.push_back(onlyWas[i++]); + } else if (i == onlyWas.size() || + onlyIs[j].getFrame() < onlyWas[i].getFrame()) { + result.added.push_back(onlyIs[j++]); + } else { + result.changed.push_back({ onlyWas[i++], onlyIs[j++] }); + } + } + + auto first = [&](sv_frame_t frame) { + if (result.firstDifference < 0 || frame < result.firstDifference) { + result.firstDifference = frame; + } + }; + for (const Event &e : result.removed) first(e.getFrame()); + for (const Event &e : result.added) first(e.getFrame()); + for (const auto &p : result.changed) first(p.first.getFrame()); + + result.pass = result.added.empty() && result.removed.empty() && + result.changed.empty(); + return result; +} + +TakeDiff::PitchJoin +TakeDiff::pitchAcross(const EventVector &pitch, sv_frame_t join, + sv_samplerate_t rate, double windowSeconds, + int maxGapHops) +{ + PitchJoin result; + + sv_frame_t reach = framesOf(windowSeconds, rate); + result.window = Coverage::Range(join - reach, join + reach); + const sv_frame_t from = result.window.start, to = result.window.end; + + // Order, in the order given + std::vector inside; + for (const Event &e : pitch) { + sv_frame_t f = e.getFrame(); + if (f < from || f >= to) continue; + if (!inside.empty() && f < inside.back()) { + if (result.firstOutOfOrder < 0) result.firstOutOfOrder = f; + ++result.outOfOrder; + } + inside.push_back(f); + } + result.events = int(inside.size()); + + // Doubles, whatever the order: each frame counted once however + // many events it holds + std::sort(inside.begin(), inside.end()); + for (size_t k = 1; k < inside.size(); ++k) { + if (inside[k] != inside[k-1]) continue; + if (k >= 2 && inside[k-1] == inside[k-2]) continue; + if (result.firstDoubled < 0) result.firstDoubled = inside[k]; + ++result.doubled; + } + inside.erase(std::unique(inside.begin(), inside.end()), inside.end()); + + // Gaps: from the last event before the window, through those in + // it, to the first after it. With no event beyond an edge, the + // edge stands in for one + sv_frame_t previous = from, next = to; + bool havePrevious = false, haveNext = false; + for (const Event &e : pitch) { + sv_frame_t f = e.getFrame(); + if (f < from && (!havePrevious || f > previous)) { + previous = f; + havePrevious = true; + } + if (f >= to && (!haveNext || f < next)) { + next = f; + haveNext = true; + } + } + + std::vector points; + points.push_back(previous); + points.insert(points.end(), inside.begin(), inside.end()); + points.push_back(next); + + for (size_t k = 1; k < points.size(); ++k) { + sv_frame_t gap = points[k] - points[k-1]; + if (gap > result.largestGap) { + result.largestGap = gap; + result.largestGapFrom = points[k-1]; + } + } + + result.pass = result.largestGap <= maxGapHops * kHopFrames && + result.doubled == 0 && result.outOfOrder == 0; + return result; +} + +TakeDiff::NoteJoin +TakeDiff::notesAcross(const EventVector ¬es, sv_frame_t join, + sv_samplerate_t rate, double clearanceSeconds) +{ + NoteJoin result; + + sv_frame_t clearance = framesOf(clearanceSeconds, rate); + bool haveEdge = false; + + for (const Event ¬e : notes) { + sv_frame_t start = note.getFrame(); + sv_frame_t end = start + note.getDuration(); + + if (start <= join && join < end) result.spanning.push_back(note); + + bool near = false; + for (sv_frame_t edge : { start, end }) { + sv_frame_t offset = edge - join; + if (std::llabs(offset) <= clearance) near = true; + if (!haveEdge || std::llabs(offset) < std::llabs(result.nearestEdge)) { + result.nearestEdge = offset; + haveEdge = true; + } + } + if (near) result.edgesNear.push_back(note); + } + + result.pass = result.spanning.size() == 1 && result.edgesNear.empty(); + return result; +} + +TakeDiff::SampleStep +TakeDiff::stepAt(const float *samples, sv_frame_t frames, int channels, + sv_samplerate_t rate, sv_frame_t join) +{ + SampleStep result; + if (channels < 1) return result; + + // First difference i is x[i] - x[i - 1], so i runs from 1 + auto clamp = [&](sv_frame_t i) { + return std::max(sv_frame_t(1), std::min(i, frames)); + }; + + sv_frame_t reach = framesOf(kStepSeconds, rate); + sv_frame_t half = framesOf(kStepContextSeconds / 2.0, rate); + sv_frame_t stepFrom = clamp(join - reach), stepTo = clamp(join + reach + 1); + sv_frame_t contextFrom = clamp(join - half), contextTo = clamp(join + half); + + // Nothing to read at the join: it is not in the audio + if (stepFrom >= stepTo) return result; + + auto difference = [&](sv_frame_t i, int c) { + return std::fabs(double(samples[i * channels + c]) - + double(samples[(i - 1) * channels + c])); + }; + + bool first = true; + for (int c = 0; c < channels; ++c) { + + double largest = 0.0; + sv_frame_t largestAt = join; + for (sv_frame_t i = stepFrom; i < stepTo; ++i) { + double d = difference(i, c); + if (d > largest) { + largest = d; + largestAt = i; + } + } + + double typical = 0.0; + std::vector context; + for (sv_frame_t i = contextFrom; i < contextTo; ++i) { + context.push_back(difference(i, c)); + } + if (!context.empty()) { + size_t k = size_t(std::llround + (kTypicalQuantile * double(context.size() - 1))); + std::nth_element(context.begin(), context.begin() + k, + context.end()); + typical = context[k]; + } + + double db = decibels(largest, typical); + if (first || db > result.stepDb) { + result.stepDb = db; + result.largest = largest; + result.typical = typical; + result.largestAt = largestAt; + result.channel = c; + first = false; + } + } + + result.pass = result.stepDb <= kMaxStepDb; + return result; +} diff --git a/main/TakeDiff.h b/main/TakeDiff.h new file mode 100644 index 00000000..30dcb1b6 --- /dev/null +++ b/main/TakeDiff.h @@ -0,0 +1,302 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_DIFF_H +#define TONY_TAKE_DIFF_H + +#include "Coverage.h" + +#include "base/BaseTypes.h" +#include "base/Event.h" + +#include +#include + +/** + * What a change did to a take, and what the joins it left look like: + * the comparisons the development checks make on a real take (the + * audio and events outside a punch-in, and the join of two punch-ins + * inside a held tone). Each returns whether it passed and the numbers + * behind the answer, so that a check can report them. + * + * Pure functions over sample buffers, event vectors and frames: no + * model, no window and no device, so all of it is tested without them + * (TestTakeDiff). Samples are interleaved, as WavFileReader gives + * them. Frames are on the take's timeline, which is the reference's, + * and ranges are [start, end) as everywhere else. + */ +namespace TakeDiff +{ + /** + * Whether the audio of a take outside a range is what it was. + */ + struct AudioDiff { + /// Every sample outside the range is bit for bit what it was + bool pass; + + /// The first frame outside the range where any channel + /// differs, or -1 if none does + sv::sv_frame_t firstDifference; + + /// How many frames outside the range differ, and the largest + /// difference of a sample there, full scale being 1 + sv::sv_frame_t differences; + double largestDifference; + + AudioDiff() : pass(false), firstDifference(-1), differences(0), + largestDifference(0) { } + }; + + /** + * Compare a take's audio before and after a change that is to + * touch only the range: a punch-in's placed range, or an erased + * one. Both buffers have the given number of channels. A frame + * past the end of either is silence, as a take is wherever nothing + * was recorded: a splice past the end makes the file longer. + * + * Nothing outside the range is excused. TakeAudio::splice() and + * erase() crossfade over the first and last fade frames *inside* + * the range, the range's own first and last frame included, and + * copy every other frame from the old file (weightAt() in + * TakeAudio.cpp). Take files are 32-bit float, so the copy is + * exact. + */ + AudioDiff audioOutside(const float *before, sv::sv_frame_t beforeFrames, + const float *after, sv::sv_frame_t afterFrames, + int channels, const Coverage::Range &range); + + /** + * How far either side of a changed range its pitch and notes may + * change: a ranged analysis replaces what lies within 0.25 s of + * the range it was asked for (docs/takes.md, "W"), and + * manual-checklist item 10 allows the same. + */ + constexpr double kEventMarginSeconds = 0.25; + + /** + * What changed outside a window. An event is outside when no + * part of it lies in the window; a note that crosses the window's + * edge counts as inside. + */ + struct EventDiff { + /// Nothing outside the window was added, removed or changed + bool pass; + + /// Outside the window: events only the vector after the change + /// has, events only the one before it had, and events at one + /// frame that differ, as (before, after). An event that moved + /// is removed at one frame and added at the other; a note that + /// grew into the window is removed + sv::EventVector added; + sv::EventVector removed; + std::vector> changed; + + /// The first frame of any of those, or -1 + sv::sv_frame_t firstDifference; + + /// The window left out: the range widened by the margin + Coverage::Range window; + + EventDiff() : pass(false), firstDifference(-1) { } + }; + + /** + * Compare pitch events, or notes, before and after a change to + * the range, outside the range widened by the margin either side. + * Events are compared in full (frame, value, duration, level and + * label): outside the window the merge leaves the very same + * events. The vectors need not be sorted. + */ + EventDiff eventsOutside(const sv::EventVector &before, + const sv::EventVector &after, + const Coverage::Range &range, + sv::sv_samplerate_t rate, + double marginSeconds = kEventMarginSeconds); + + /// pYIN's step, in frames of the audio it analyses (Analyser.cpp): + /// over a voiced stretch the pitch track has an event every hop + constexpr sv::sv_frame_t kHopFrames = 256; + + /** + * How far either side of a join the pitch track is looked at. A + * ranged analysis runs over its range widened by 0.5 s and merges + * what lies within 0.25 s of it (docs/takes.md), so a join made + * by a punch-in has the merge's seam 0.25 s before it; this takes + * it in with room, and all that the run could reach. + */ + constexpr double kPitchWindowSeconds = 0.5; + + /// The longest step between neighbouring pitch events that is not + /// a hole, in hops. Over a held tone pYIN stamps every hop, and + /// the hole a bad merge once left was two hops + constexpr int kMaxGapHops = 1; + + /** + * Whether the pitch track runs through a join: no hole, no frame + * twice, frames in order. + */ + struct PitchJoin { + /// No gap longer than the largest allowed, nothing doubled and + /// nothing out of order, within the window + bool pass; + + /// The pitch events within the window + int events; + + /// The longest step within the window from one event to the + /// next, in frames, and the frame it starts from. A step that + /// begins before the window or ends after it counts. Where the + /// track has no event beyond the window's edge, the edge + /// itself counts as one, so a track that stops inside the + /// window, or has nothing in it, shows a long gap + sv::sv_frame_t largestGap; + sv::sv_frame_t largestGapFrom; + + /// Frames within the window that hold more than one event, and + /// the first of them, or -1 + int doubled; + sv::sv_frame_t firstDoubled; + + /// Events within the window that come, in the order given, + /// after one at a later frame; and the first of them, or -1 + int outOfOrder; + sv::sv_frame_t firstOutOfOrder; + + /// The window looked at: [join - reach, join + reach) + Coverage::Range window; + + PitchJoin() : pass(false), events(0), largestGap(0), + largestGapFrom(0), doubled(0), firstDoubled(-1), + outOfOrder(0), firstOutOfOrder(-1) { } + }; + + /** + * Look at the pitch events around a join, in the order given (a + * model gives them sorted). Only the window is looked at: pYIN + * itself stamps one frame twice 100 hops before the end of every + * run, which a whole-file track keeps and a ranged merge drops. + */ + PitchJoin pitchAcross(const sv::EventVector &pitch, sv::sv_frame_t join, + sv::sv_samplerate_t rate, + double windowSeconds = kPitchWindowSeconds, + int maxGapHops = kMaxGapHops); + + /** + * How close to a join no note may begin or end: as far as the + * pitch window, so that a note split at the merge's seam 0.25 s + * from the join is caught. The held tone a join is made in has to + * reach further than this on both sides (the dev layout's reach + * 1.5 s from their middle). + */ + constexpr double kNoteClearanceSeconds = 0.5; + + /** + * Whether one note runs through a join. + */ + struct NoteJoin { + /// Exactly one note holds the join frame, and no note begins + /// or ends within the clearance of it + bool pass; + + /// The notes that hold the join frame + sv::EventVector spanning; + + /// The notes that begin or end within the clearance of the join + sv::EventVector edgesNear; + + /// The onset or end of any note nearest the join, in frames + /// from it (negative before it); 0 if there are no notes + sv::sv_frame_t nearestEdge; + + NoteJoin() : pass(false), nearestEdge(0) { } + }; + + /** + * Look at the notes around a join. A note is [frame, frame + + * duration): one that ends at the join does not hold it, and its + * end is 0 frames from it. + */ + NoteJoin notesAcross(const sv::EventVector ¬es, sv::sv_frame_t join, + sv::sv_samplerate_t rate, + double clearanceSeconds = kNoteClearanceSeconds); + + /// Where a step at a join is looked for: first differences within + /// this of the join ... + constexpr double kStepSeconds = 0.002; + + /// ... against those of this much audio around it, centred on it + constexpr double kStepContextSeconds = 0.05; + + /// The context's typical first difference is this quantile of + /// their sizes. For a steady tone that is within 0.03 dB of its + /// largest, so a clean join in a tone reads about 0 dB; unlike a + /// mean, a click or two in the context does not raise it + constexpr double kTypicalQuantile = 0.95; + + /** + * A join is a step when its largest first difference is this far + * above the typical one. Measured with this code's arithmetic: a + * steady tone of 196 to 294 Hz reads 0.03 dB, and so does the + * splice's crossfade into it at the opposite phase. White noise reads + * 3.6 dB typically and at most 6.7 dB in 200 trials (the largest + * of 177 differences against the 95th percentile of 2205). A hard + * cut in a 220 Hz tone at a random phase reads the jump against + * the tone's steepest step: 27 dB typically, up to 36 dB. About + * one such cut in ten reads under 10 dB: the two sides happened + * to meet within three of the tone's own steps (a tenth of its + * amplitude), which is hardly a click. 10 dB keeps noise well + * clear and misses only those. + */ + constexpr double kMaxStepDb = 10.0; + + /** + * Whether the samples step at a join. + */ + struct SampleStep { + /// stepDb is within kMaxStepDb + bool pass; + + /// The largest first difference within kStepSeconds of the + /// join against the typical one over kStepContextSeconds, in + /// dB, in the channel where that is largest. 0 dB when both + /// are silent, 200 when only the typical one is + double stepDb; + + /// The two first differences, full scale being 1, and the + /// frame the largest leads into (it is x[f] - x[f - 1]) + double largest; + double typical; + sv::sv_frame_t largestAt; + + /// The channel they were read from + int channel; + + SampleStep() : pass(false), stepDb(0), largest(0), typical(0), + largestAt(0), channel(0) { } + }; + + /** + * Look for a step in the samples at a join. Both stretches are + * cut short at the ends of the samples; with no first difference + * to read within kStepSeconds of the join (a join outside the + * audio), it does not pass. + * The context is read as it is: a join between two stretches of + * very different level is read against the whole of it. + */ + SampleStep stepAt(const float *samples, sv::sv_frame_t frames, + int channels, sv::sv_samplerate_t rate, + sv::sv_frame_t join); +} + +#endif diff --git a/main/test/TestTakeDiff.h b/main/test/TestTakeDiff.h new file mode 100644 index 00000000..65b2a53b --- /dev/null +++ b/main/test/TestTakeDiff.h @@ -0,0 +1,634 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TAKE_DIFF_H +#define TEST_TAKE_DIFF_H + +// Tier 2: the comparisons the development checks make on a real take, +// on synthetic samples and events: each passing, and failing on +// purpose. Two tests run the real splice through files, since where +// its fades fall is what the audio comparison relies on. + +#include "../TakeDiff.h" +#include "../TakeAudio.h" +#include "TestSignals.h" + +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" +#include "data/fileio/WavFileWriter.h" + +#include +#include +#include + +#include +#include +#include +#include + +class TestTakeDiff : public QObject +{ + Q_OBJECT + + typedef sv::sv_frame_t frame_t; + typedef Coverage::Range Range; + typedef std::vector Signal; // interleaved + + static constexpr double kRate = 44100.0; + static constexpr float kAmplitude = 0.25f; // -12 dBFS, as the reference + + // 220.5 Hz: exactly 200 samples a period at 44.1 kHz, so a phase + // is a frame and nothing drifts + static constexpr double kToneHz = 220.5; + + QTemporaryDir m_dir; + int m_fileCounter = 0; + + static frame_t framesOf(double seconds) { + return frame_t(std::llround(seconds * kRate)); + } + + // A sine whose phase at frame 0 is "phase" radians + static Signal sine(frame_t frames, double phase = 0.0) { + Signal s(frames); + for (frame_t i = 0; i < frames; ++i) { + s[i] = kAmplitude * float(std::sin(2.0 * TestSignals::kPi * kToneHz + * double(i) / kRate + phase)); + } + return s; + } + + // Distinct and nowhere silent, so that any change shows + static Signal ramp(frame_t frames, int channels = 1) { + Signal s(frames * channels); + for (frame_t i = 0; i < frames * channels; ++i) { + s[i] = 0.1f + 0.5f * float(i) / float(frames * channels); + } + return s; + } + + // Gaussian, from a generator of our own so that every platform + // makes the same noise + static Signal noise(frame_t frames, float sd) { + Signal s(frames); + uint32_t state = 12345; + auto uniform = [&]() { + state = state * 1664525u + 1013904223u; + return (double(state) + 1.0) / 4294967297.0; + }; + for (frame_t i = 0; i < frames; ++i) { + double u = uniform(), v = uniform(); + s[i] = sd * float(std::sqrt(-2.0 * std::log(u)) * + std::cos(2.0 * TestSignals::kPi * v)); + } + return s; + } + + static sv::Event pitchAt(frame_t frame, float hz = 220.f) { + return sv::Event(frame, hz, QString()); + } + + static sv::Event noteAt(frame_t frame, frame_t duration, + float hz = 220.f) { + return sv::Event(frame, hz, duration, "sung"); + } + + // An event every hop from "from" to "to", as pYIN leaves over a + // held tone + static sv::EventVector pitchTrack(frame_t from, frame_t to) { + sv::EventVector events; + for (frame_t f = from; f < to; f += TakeDiff::kHopFrames) { + events.push_back(pitchAt(f)); + } + return events; + } + + static void removeFrame(sv::EventVector &events, frame_t frame) { + events.erase(std::remove_if(events.begin(), events.end(), + [&](const sv::Event &e) { + return e.getFrame() == frame; + }), events.end()); + } + + QString writeWav(const Signal &samples) { + QString path = m_dir.filePath + (QString("source-%1.wav").arg(++m_fileCounter)); + sv::WavFileWriter writer(path, kRate, 1, + sv::WavFileWriter::WriteToTarget); + sv::floatvec_t data(samples.begin(), samples.end()); + if (!writer.isOK() || !writer.putInterleavedFrames(data) || + !writer.close()) { + return ""; + } + return path; + } + + QString newPath() { + return m_dir.filePath(QString("take-%1.wav").arg(++m_fileCounter)); + } + + // All of a mono file, as a check reads a take's file + static Signal read(QString path) { + sv::WavFileReader reader { sv::FileSource(path) }; + if (!reader.isOK() || reader.getChannelCount() != 1) return {}; + auto data = reader.getInterleavedFrames(0, reader.getFrameCount()); + return Signal(data.begin(), data.end()); + } + + static TakeDiff::AudioDiff compare(const Signal &before, + const Signal &after, Range range, + int channels = 1) { + return TakeDiff::audioOutside + (before.data(), frame_t(before.size()) / channels, + after.data(), frame_t(after.size()) / channels, + channels, range); + } + + static TakeDiff::SampleStep stepAt(const Signal &samples, frame_t join, + int channels = 1) { + return TakeDiff::stepAt(samples.data(), + frame_t(samples.size()) / channels, + channels, kRate, join); + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + // ---- Audio outside a range ---- + + // Everything in the range may change, its first and last frames too + void audio_changed_only_inside_passes() { + Signal before = ramp(10000); + Signal after = before; + for (frame_t i = 4000; i < 6000; ++i) after[i] = 0.9f; + + TakeDiff::AudioDiff d = compare(before, after, Range(4000, 6000)); + QVERIFY(d.pass); + QCOMPARE(d.firstDifference, frame_t(-1)); + QCOMPARE(d.differences, frame_t(0)); + } + + // Bit for bit: the smallest change a float can make, one frame + // outside the range at either end, or in one channel only + void audio_changed_just_outside_fails() { + Signal before = ramp(10000); + + Signal after = before; + after[6000] = std::nextafter(after[6000], 1.f); + TakeDiff::AudioDiff d = compare(before, after, Range(4000, 6000)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, frame_t(6000)); + QCOMPARE(d.differences, frame_t(1)); + QVERIFY(d.largestDifference > 0.0); + QVERIFY(d.largestDifference < 1e-6); + + after = before; + after[3999] = 0.f; + d = compare(before, after, Range(4000, 6000)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, frame_t(3999)); + + Signal stereo = ramp(10000, 2); + after = stereo; + after[6000 * 2 + 1] = 0.f; + d = compare(stereo, after, Range(4000, 6000), 2); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, frame_t(6000)); + QCOMPARE(d.differences, frame_t(1)); + } + + // A take is silent wherever nothing was recorded, in the file or + // past its end + void audio_past_the_end_is_silence() { + Signal before = ramp(5000); + + // A punch-in after the old end makes the file longer + Signal after = before; + after.resize(8000, 0.f); + for (frame_t i = 6000; i < 8000; ++i) after[i] = 0.9f; + QVERIFY(compare(before, after, Range(6000, 8000)).pass); + + after[5500] = 0.1f; + TakeDiff::AudioDiff d = compare(before, after, Range(6000, 8000)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, frame_t(5500)); + + // Sound lost off the end is a difference too + after = before; + after.resize(4500); + d = compare(before, after, Range(1000, 2000)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, frame_t(4500)); + QCOMPARE(d.differences, frame_t(500)); + } + + // The real splice and erase, through their files: bit-identical + // outside the range they fill, and their fades reach right to its + // ends, so that the range less one frame at either end differs + // exactly there. Were a fade to fall outside the range, the first + // comparison would fail + void audio_splice_and_erase_fades_lie_inside_the_range() { + QString oldPath = writeWav(sine(20000)); + QString recording = writeWav(Signal(6000, -0.2f)); + QVERIFY(oldPath != "" && recording != ""); + + QString spliced = newPath(); + Range placed; + QCOMPARE(TakeAudio::splice(oldPath, recording, 0, 5000, 4000, + spliced, &placed), QString()); + QCOMPARE(placed.start, frame_t(5000)); + QCOMPARE(placed.end, frame_t(9000)); + + Signal before = read(oldPath), after = read(spliced); + QCOMPARE(frame_t(before.size()), frame_t(20000)); + QCOMPARE(frame_t(after.size()), frame_t(20000)); + + TakeDiff::AudioDiff d = compare(before, after, placed); + QVERIFY2(d.pass, qPrintable(QString("differs from frame %1") + .arg(d.firstDifference))); + + d = compare(before, after, Range(placed.start + 1, placed.end)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, placed.start); + QCOMPARE(d.differences, frame_t(1)); + + d = compare(before, after, Range(placed.start, placed.end - 1)); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, placed.end - 1); + + QString erased = newPath(); + Range gone(12050, 13050); // from a peak: a fade there changes it + QCOMPARE(TakeAudio::erase(spliced, Coverage::Ranges { gone }, erased), + QString()); + Signal later = read(erased); + QVERIFY(compare(after, later, gone).pass); + d = compare(after, later, Range(gone.start + 1, gone.end)); + QCOMPARE(d.firstDifference, gone.start); + } + + // ---- Events outside a range, give or take the margin ---- + + void events_changed_inside_the_margin_pass() { + Range range(100000, 200000); + frame_t margin = framesOf(TakeDiff::kEventMarginSeconds); + + sv::EventVector before = pitchTrack(0, 300000); + sv::EventVector after; + for (const sv::Event &e : before) { + bool inside = e.getFrame() >= range.start - margin && + e.getFrame() < range.end + margin; + after.push_back(inside ? e.withValue(330.f) : e); + } + after.push_back(pitchAt(range.start + 7)); // one more, inside + + TakeDiff::EventDiff d = + TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(d.pass); + QCOMPARE(d.firstDifference, frame_t(-1)); + QCOMPARE(d.window.start, range.start - margin); + QCOMPARE(d.window.end, range.end + margin); + + // A note that crosses the window's edge counts as inside: cut + // back, as the merge may cut an old note a new one overlaps + sv::EventVector notes { noteAt(50000, 30000), noteAt(80000, 20000), + noteAt(150000, 10000), noteAt(230000, 10000) }; + sv::EventVector cut { noteAt(50000, 30000), noteAt(80000, 10000), + noteAt(160000, 5000), noteAt(230000, 10000) }; + QVERIFY(TakeDiff::eventsOutside(notes, cut, range, kRate).pass); + } + + // The window is [start - margin, end + margin), to the frame + void pitch_changed_just_outside_the_margin_fails() { + Range range(100000, 200000); + frame_t margin = framesOf(TakeDiff::kEventMarginSeconds); + frame_t first = range.start - margin, last = range.end + margin; + + sv::EventVector before { pitchAt(first - 1), pitchAt(first), + pitchAt(last - 1), pitchAt(last) }; + + sv::EventVector after { pitchAt(first - 1, 221.f), pitchAt(first, 300.f), + pitchAt(last - 1, 300.f), pitchAt(last) }; + TakeDiff::EventDiff d = + TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(int(d.changed.size()), 1); + QCOMPARE(d.changed[0].first.getFrame(), first - 1); + QCOMPARE(d.changed[0].second.getValue(), 221.f); + QCOMPARE(d.firstDifference, first - 1); + + after = { pitchAt(first - 1), pitchAt(first), pitchAt(last - 1), + pitchAt(last, 221.f) }; + d = TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(d.firstDifference, last); + } + + void events_added_removed_and_changed_are_told_apart() { + Range range(100000, 110000); + sv::EventVector before { pitchAt(1000), pitchAt(2000), pitchAt(3000) }; + sv::EventVector after { pitchAt(4000), pitchAt(2000, 230.f), + pitchAt(1000) }; // and out of order + + TakeDiff::EventDiff d = + TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(int(d.removed.size()), 1); + QCOMPARE(d.removed[0].getFrame(), frame_t(3000)); + QCOMPARE(int(d.added.size()), 1); + QCOMPARE(d.added[0].getFrame(), frame_t(4000)); + QCOMPARE(int(d.changed.size()), 1); + QCOMPARE(d.changed[0].first.getValue(), 220.f); + QCOMPARE(d.changed[0].second.getValue(), 230.f); + QCOMPARE(d.firstDifference, frame_t(2000)); + + // The same event twice on one side is one more event there + before = { pitchAt(1000), pitchAt(1000) }; + after = { pitchAt(1000) }; + d = TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(int(d.removed.size()), 1); + QVERIFY(d.changed.empty()); + } + + void note_changed_outside_the_margin_fails() { + Range range(100000, 200000); + + // Shorter, wholly outside + sv::EventVector before { noteAt(20000, 5000) }; + sv::EventVector after { noteAt(20000, 4000) }; + TakeDiff::EventDiff d = + TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(int(d.changed.size()), 1); + QCOMPARE(d.changed[0].second.getDuration(), frame_t(4000)); + + // Grown into the window: the note that was outside is gone + before = { noteAt(80000, 8000) }; + after = { noteAt(80000, 10000) }; + d = TakeDiff::eventsOutside(before, after, range, kRate); + QVERIFY(!d.pass); + QCOMPARE(int(d.removed.size()), 1); + QVERIFY(d.added.empty()); + } + + // ---- Pitch across a join ---- + + void pitch_continuous_across_a_join_passes() { + frame_t join = framesOf(2.0) + 100; + sv::EventVector pitch = pitchTrack(0, framesOf(5.0)); + + TakeDiff::PitchJoin p = TakeDiff::pitchAcross(pitch, join, kRate); + QVERIFY(p.pass); + QCOMPARE(p.largestGap, TakeDiff::kHopFrames); + QCOMPARE(p.doubled, 0); + QCOMPARE(p.outOfOrder, 0); + + frame_t reach = framesOf(TakeDiff::kPitchWindowSeconds); + QCOMPARE(p.window.start, join - reach); + QCOMPARE(p.window.end, join + reach); + int expected = 0; + for (const sv::Event &e : pitch) { + if (e.getFrame() >= join - reach && e.getFrame() < join + reach) { + ++expected; + } + } + QCOMPARE(p.events, expected); + } + + void pitch_event_dropped_at_the_join_fails() { + frame_t join = framesOf(2.0) + 100; + frame_t atJoin = (join / TakeDiff::kHopFrames) * TakeDiff::kHopFrames; + sv::EventVector pitch = pitchTrack(0, framesOf(5.0)); + removeFrame(pitch, atJoin); + + TakeDiff::PitchJoin p = TakeDiff::pitchAcross(pitch, join, kRate); + QVERIFY(!p.pass); + QCOMPARE(p.largestGap, 2 * TakeDiff::kHopFrames); + QCOMPARE(p.largestGapFrom, atJoin - TakeDiff::kHopFrames); + + // Two hops are allowed when asked for + QVERIFY(TakeDiff::pitchAcross(pitch, join, kRate, + TakeDiff::kPitchWindowSeconds, 2).pass); + } + + void pitch_doubled_frame_fails() { + frame_t join = framesOf(2.0) + 100; + frame_t atJoin = (join / TakeDiff::kHopFrames) * TakeDiff::kHopFrames; + sv::EventVector pitch = pitchTrack(0, framesOf(5.0)); + + sv::EventVector doubled = pitch; + auto at = std::find(doubled.begin(), doubled.end(), pitchAt(atJoin)); + QVERIFY(at != doubled.end()); + doubled.insert(at + 1, pitchAt(atJoin, 221.f)); + + TakeDiff::PitchJoin p = TakeDiff::pitchAcross(doubled, join, kRate); + QVERIFY(!p.pass); + QCOMPARE(p.doubled, 1); + QCOMPARE(p.firstDoubled, atJoin); + QCOMPARE(p.outOfOrder, 0); + QCOMPARE(p.largestGap, TakeDiff::kHopFrames); + + // Two neighbours the wrong way round + sv::EventVector swapped = pitch; + at = std::find(swapped.begin(), swapped.end(), pitchAt(atJoin)); + std::iter_swap(at, at + 1); + p = TakeDiff::pitchAcross(swapped, join, kRate); + QVERIFY(!p.pass); + QCOMPARE(p.outOfOrder, 1); + QCOMPARE(p.firstOutOfOrder, atJoin); + QCOMPARE(p.doubled, 0); + } + + // Only the window counts, but a hole that reaches into it does + void pitch_looks_only_around_the_join() { + const frame_t hop = TakeDiff::kHopFrames; + frame_t join = framesOf(2.0) + 100; + frame_t atJoin = (join / hop) * hop; + sv::EventVector pitch = pitchTrack(0, framesOf(5.0)); + + // A doubled frame and a hole a second away: pYIN's own doubled + // frame near the end of a run is like this + sv::EventVector far = pitch; + far.push_back(pitchAt(atJoin + 172 * hop)); + for (int k = 1; k <= 10; ++k) removeFrame(far, atJoin - 172 * hop + k * hop); + QVERIFY(TakeDiff::pitchAcross(far, join, kRate).pass); + + // The merge's seam, a quarter of a second before the join + sv::EventVector seam = pitch; + frame_t seamFrame = ((join - framesOf(0.25)) / hop) * hop; + removeFrame(seam, seamFrame); + QVERIFY(!TakeDiff::pitchAcross(seam, join, kRate).pass); + + // A hole from before the window into it + frame_t windowStart = join - framesOf(TakeDiff::kPitchWindowSeconds); + sv::EventVector into = pitch; + into.erase(std::remove_if(into.begin(), into.end(), + [&](const sv::Event &e) { + return e.getFrame() > windowStart - 2000 && + e.getFrame() < windowStart + 1000; + }), into.end()); + TakeDiff::PitchJoin p = TakeDiff::pitchAcross(into, join, kRate); + QVERIFY(!p.pass); + QVERIFY(p.largestGapFrom < windowStart); + + // The track stops inside the window + sv::EventVector stops = pitchTrack(0, join + framesOf(0.2)); + p = TakeDiff::pitchAcross(stops, join, kRate); + QVERIFY(!p.pass); + QVERIFY(p.largestGap > framesOf(0.2)); + + // Nothing at all + p = TakeDiff::pitchAcross(sv::EventVector(), join, kRate); + QVERIFY(!p.pass); + QCOMPARE(p.events, 0); + QCOMPARE(p.largestGap, 2 * framesOf(TakeDiff::kPitchWindowSeconds)); + } + + // ---- One note across a join ---- + + void one_note_across_a_join_passes() { + frame_t join = framesOf(10.0); + sv::EventVector notes { noteAt(framesOf(5.0), framesOf(1.0)), + noteAt(join - framesOf(1.5), framesOf(3.0)), + noteAt(framesOf(13.0), framesOf(1.0)) }; + + TakeDiff::NoteJoin n = TakeDiff::notesAcross(notes, join, kRate); + QVERIFY(n.pass); + QCOMPARE(int(n.spanning.size()), 1); + QCOMPARE(n.spanning[0].getFrame(), join - framesOf(1.5)); + QVERIFY(n.edgesNear.empty()); + QCOMPARE(n.nearestEdge, -framesOf(1.5)); + } + + void note_split_at_the_join_fails() { + frame_t join = framesOf(10.0); + frame_t a = join - framesOf(1.5), b = join + framesOf(1.5); + frame_t clearance = framesOf(TakeDiff::kNoteClearanceSeconds); + + // Split at the join: the first ends there, so only the second + // holds it + sv::EventVector notes { noteAt(a, join - a), noteAt(join, b - join) }; + TakeDiff::NoteJoin n = TakeDiff::notesAcross(notes, join, kRate); + QVERIFY(!n.pass); + QCOMPARE(int(n.spanning.size()), 1); + QCOMPARE(n.spanning[0].getFrame(), join); + QCOMPARE(int(n.edgesNear.size()), 2); + QCOMPARE(n.nearestEdge, frame_t(0)); + + // At the merge's seam + frame_t seam = join - framesOf(0.25); + notes = { noteAt(a, seam - a), noteAt(seam, b - seam) }; + n = TakeDiff::notesAcross(notes, join, kRate); + QVERIFY(!n.pass); + QCOMPARE(n.nearestEdge, -framesOf(0.25)); + + // At the clearance, and one frame beyond it + frame_t split = join - clearance; + notes = { noteAt(a, split - a), noteAt(split, b - split) }; + QVERIFY(!TakeDiff::notesAcross(notes, join, kRate).pass); + split = join - clearance - 1; + notes = { noteAt(a, split - a), noteAt(split, b - split) }; + QVERIFY(TakeDiff::notesAcross(notes, join, kRate).pass); + + // None, or two at once + QVERIFY(!TakeDiff::notesAcross(sv::EventVector(), join, kRate).pass); + notes = { noteAt(a, b - a), noteAt(a, b - a, 330.f) }; + n = TakeDiff::notesAcross(notes, join, kRate); + QVERIFY(!n.pass); + QCOMPARE(int(n.spanning.size()), 2); + } + + // ---- No step in the samples at a join ---- + + void a_steady_tone_has_no_step() { + Signal tone = sine(framesOf(0.2)); + TakeDiff::SampleStep s = stepAt(tone, framesOf(0.1)); + QVERIFY2(s.pass && std::fabs(s.stepDb) < 0.1, + qPrintable(QString("%1 dB").arg(s.stepDb))); + + // Silence has no step either + s = stepAt(Signal(framesOf(0.2), 0.f), framesOf(0.1)); + QVERIFY(s.pass); + QCOMPARE(s.stepDb, 0.0); + } + + // The threshold's own reasoning: noise reads a few dB, well under it + void white_noise_has_no_step() { + Signal hiss = noise(framesOf(0.2), 0.05f); + TakeDiff::SampleStep s = stepAt(hiss, framesOf(0.1)); + QVERIFY2(s.pass && s.stepDb > 0.0 && s.stepDb < 7.0, + qPrintable(QString("%1 dB").arg(s.stepDb))); + } + + // The tone at its peak is cut to the same tone at its trough + void a_hard_cut_in_a_sine_is_a_step() { + frame_t join = 4050; // a quarter period past a whole one: the peak + Signal cut = sine(framesOf(0.2)); + Signal opposite = sine(framesOf(0.2), TestSignals::kPi); + std::copy(opposite.begin() + join, opposite.end(), cut.begin() + join); + + TakeDiff::SampleStep s = stepAt(cut, join); + QVERIFY(!s.pass); + QVERIFY2(s.stepDb > 30.0, qPrintable(QString("%1 dB").arg(s.stepDb))); + QCOMPARE(s.largestAt, join); + QVERIFY(s.largest > 1.9 * kAmplitude); + + // Seen in the one channel that has it + Signal stereo(cut.size() * 2); + Signal tone = sine(framesOf(0.2)); + for (size_t i = 0; i < cut.size(); ++i) { + stereo[i * 2] = tone[i]; + stereo[i * 2 + 1] = cut[i]; + } + s = stepAt(stereo, join, 2); + QVERIFY(!s.pass); + QCOMPARE(s.channel, 1); + } + + // The real splice, through its files: a recording of the opposite + // phase crossfaded in reads no step, and cut in without its fade + // reads one + void the_splice_crossfade_leaves_no_step() { + QString oldPath = writeWav(sine(20000)); + QString recording = writeWav(sine(6000, -TestSignals::kPi / 2)); + QVERIFY(oldPath != "" && recording != ""); + + frame_t position = 10050; // the old take at its peak + QString faded = newPath(), hard = newPath(); + QCOMPARE(TakeAudio::splice(oldPath, recording, 0, position, 4000, + faded), QString()); + QCOMPARE(TakeAudio::splice(oldPath, recording, 0, position, 4000, + hard, nullptr, 0), QString()); + + TakeDiff::SampleStep s = stepAt(read(faded), position); + QVERIFY2(s.pass && s.stepDb < 1.0, + qPrintable(QString("%1 dB").arg(s.stepDb))); + s = stepAt(read(faded), position + 4000); + QVERIFY2(s.pass, qPrintable(QString("%1 dB").arg(s.stepDb))); + + s = stepAt(read(hard), position); + QVERIFY2(!s.pass && s.stepDb > 30.0, + qPrintable(QString("%1 dB").arg(s.stepDb))); + QCOMPARE(s.largestAt, position); + } + + void a_join_outside_the_audio_does_not_pass() { + Signal tone = sine(framesOf(0.2)); + QVERIFY(!stepAt(tone, framesOf(0.2) + 1000).pass); + QVERIFY(!stepAt(Signal(), 0).pass); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index da882fdb..adb5ba12 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -22,6 +22,7 @@ #include "TestTakeTiming.h" #include "TestLatencyCheck.h" #include "TestLatencyCalibration.h" +#include "TestTakeDiff.h" #include "RunSuite.h" @@ -112,6 +113,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTakeDiff t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 0b84c84f..9abd5f5c 100644 --- a/meson.build +++ b/meson.build @@ -1100,6 +1100,7 @@ tony_core_files = [ 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/TakeAudio.cpp', + 'main/TakeDiff.cpp', 'main/TakeEvents.cpp', 'main/TakesFile.cpp', 'main/TakeTiming.cpp', @@ -1364,6 +1365,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakeTiming.h', 'main/test/TestLatencyCheck.h', 'main/test/TestLatencyCalibration.h', + 'main/test/TestTakeDiff.h', ]) tony_core_test_exe = executable( From c784877033dd809066b302efcd086adc25111939 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:19:55 +0000 Subject: [PATCH 127/275] feat: Edit Lyrics, to drag a word's start or end Edit > Edit Lyrics switches on editing of the words along the bottom of the pane. The start or end of a word can then be dragged, one word at a time: where two words touch, the side of the edge the pointer is on picks the word. An edge stops at the neighbouring word and at a 20 ms word; each drag is one step to undo. Everything else in the pane works as before. Edit mode goes off while recording, when the lyrics are hidden, removed or replaced, and when the session closes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsEditor.cpp | 392 ++++++++++++++++++ main/LyricsEditor.h | 159 ++++++++ main/LyricsTrack.h | 6 +- main/MainWindow.cpp | 71 ++++ main/MainWindow.h | 19 + main/test/TestRecordWorkflow.h | 700 +++++++++++++++++++++++++++++++++ meson.build | 2 + 7 files changed, 1348 insertions(+), 1 deletion(-) create mode 100644 main/LyricsEditor.cpp create mode 100644 main/LyricsEditor.h diff --git a/main/LyricsEditor.cpp b/main/LyricsEditor.cpp new file mode 100644 index 00000000..1d350ca7 --- /dev/null +++ b/main/LyricsEditor.cpp @@ -0,0 +1,392 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LyricsEditor.h" + +#include "LyricsTrack.h" + +#include "view/Pane.h" +#include "layer/RegionLayer.h" +#include "data/model/RegionModel.h" +#include "data/model/EventCommands.h" +#include "widgets/CommandHistory.h" + +#include +#include + +using namespace sv; + +LyricsEditor::LyricsEditor(LyricsTrack *lyrics, QObject *parent) : + QObject(parent), + m_lyrics(lyrics), + m_enabled(false), + m_dragging(false), + m_dragWord(-1), + m_dragPart(LyricsEdit::Part::Nothing), + m_dragEdgeFrame(0), + m_dragPressFrame(0), + m_dragCommand(nullptr), + m_cursorSet(false), + m_helpShown(false) +{ +} + +LyricsEditor::~LyricsEditor() +{ + // Nothing is pushed from here: the history may be going as well + abandonDrag(); + m_dragging = false; + restoreCursor(); + if (m_pane) m_pane->removeEventFilter(this); +} + +void +LyricsEditor::setEnabled(bool enabled) +{ + if (enabled == m_enabled) return; + + if (enabled) { + Pane *pane = m_lyrics ? m_lyrics->getPane() : nullptr; + if (!pane || !m_lyrics->getLayer()) return; + m_pane = pane; + m_enabled = true; + pane->installEventFilter(this); + return; + } + + // Off before anything else, so that whatever the push of the drag's + // command sets off finds edit mode off already, and does not come + // back here half way through + m_enabled = false; + finishDrag(); + restoreCursor(); + clearHelp(); + if (m_pane) m_pane->removeEventFilter(this); + m_pane = nullptr; +} + +RegionLayer * +LyricsEditor::currentLayer() const +{ + if (!m_pane || !m_lyrics || m_lyrics->getPane() != m_pane) return nullptr; + return m_lyrics->getLayer(); +} + +std::shared_ptr +LyricsEditor::dragModel() const +{ + RegionLayer *layer = currentLayer(); + if (!layer || m_dragModel.isNone() || layer->getModel() != m_dragModel) { + return {}; + } + return ModelById::getAs(m_dragModel); +} + +bool +LyricsEditor::hitAt(QPoint pos, LyricsEdit::Hit &hit, + EventVector &words) const +{ + // Hidden lyrics keep the box row they were last painted with, which + // is not there to be clicked + RegionLayer *layer = currentLayer(); + if (!layer || !m_lyrics->isVisible()) return false; + + // As last painted, in the pane's own coordinates, which are the + // mouse's. Empty until the layer has been painted in the pane + QRect row = layer->getLyricsBoxRow(m_pane); + if (!row.contains(pos)) return false; + + auto model = ModelById::getAs(layer->getModel()); + if (!model) return false; + + words = model->getAllEvents(); + Pane *pane = m_pane; + LyricsEdit::Boxes boxes = LyricsEdit::boxesFor + (words, [pane](sv_frame_t frame) { return pane->getXForFrame(frame); }); + hit = LyricsEdit::hitTest(boxes, pos.x(), grabPixels); + return true; +} + +bool +LyricsEditor::eventFilter(QObject *object, QEvent *event) +{ + if (!m_enabled || !m_pane || object != m_pane) return false; + + switch (event->type()) { + + case QEvent::MouseButtonPress: + case QEvent::MouseButtonDblClick: + // A double-click comes in place of the second press: on an edge, + // that is where a drag begins as well + return mousePressed(static_cast(event)); + + case QEvent::MouseMove: + return mouseMoved(static_cast(event)); + + case QEvent::MouseButtonRelease: + return mouseReleased(static_cast(event)); + + case QEvent::Leave: + if (!m_dragging) leaveRow(); + return false; + + default: + return false; + } +} + +bool +LyricsEditor::mousePressed(QMouseEvent *e) +{ + // The pane never saw the press that began the drag, and must not see + // another button in the middle of it: a right press would open its + // menu with our button still held + if (m_dragging) return true; + + if (e->button() != Qt::LeftButton) return false; + + QPoint pos = e->position().toPoint(); + LyricsEdit::Hit hit; + EventVector words; + if (!hitAt(pos, hit, words) || !hit.isEdge()) return false; + + RegionLayer *layer = currentLayer(); + if (!layer || layer->getModel().isNone()) return false; + + m_dragging = true; + m_dragModel = layer->getModel(); + m_dragWords = words; + m_dragWord = hit.word; + m_dragPart = hit.part; + m_dragOriginal = words[hit.word]; + m_dragCurrent = m_dragOriginal; + m_dragEdgeFrame = m_dragOriginal.getFrame(); + if (m_dragPart == LyricsEdit::Part::End) { + m_dragEdgeFrame += m_dragOriginal.getDuration(); + } + m_dragPressFrame = m_pane->getFrameForX(pos.x()); + + // Nothing is made until the word moves: a click on an edge is no + // edit at all + m_dragCommand = nullptr; + + setEdgeCursor(); + return true; +} + +bool +LyricsEditor::mouseMoved(QMouseEvent *e) +{ + QPoint pos = e->position().toPoint(); + + if (m_dragging) { + if (e->buttons() & Qt::LeftButton) { + dragTo(pos.x()); + return true; + } + // The release went somewhere else, a dialog that opened in the + // middle of the drag perhaps: the drag ends where it got to + finishDrag(); + } + + // Someone else's drag, which began outside the row: the pane's + if (e->buttons() != Qt::NoButton) return false; + + return hover(pos); +} + +bool +LyricsEditor::mouseReleased(QMouseEvent *e) +{ + if (!m_dragging) return false; + + if (e->button() == Qt::LeftButton) { + QPoint pos = e->position().toPoint(); + dragTo(pos.x()); + finishDrag(); + hover(pos); + } + + // Any release in the middle of the drag is ours, as its press was + return true; +} + +bool +LyricsEditor::hover(QPoint pos) +{ + LyricsEdit::Hit hit; + EventVector words; + if (!hitAt(pos, hit, words)) { + leaveRow(); + return false; + } + + if (hit.isEdge()) { + setEdgeCursor(); + QString word = words[hit.word].getLabel(); + if (hit.part == LyricsEdit::Part::Start) { + showHelp(tr("Drag to move the start of \"%1\"").arg(word)); + } else { + showHelp(tr("Drag to move the end of \"%1\"").arg(word)); + } + } else { + restoreCursor(); + showHelp(tr("Drag a word's start or end to move it")); + } + + // The row is ours while edit mode is on: the pane would only put its + // own help and cursor over ours + return true; +} + +void +LyricsEditor::leaveRow() +{ + // Cleared here: the pane's own help need not reach the status bar + // (Tony shows none for the reference's pane), and if it does, the + // pane gets the same event next and says what it has to say + restoreCursor(); + clearHelp(); +} + +void +LyricsEditor::dragTo(int x) +{ + if (!m_dragging || !m_pane || m_dragModel.isNone()) return; + + // The model can change under a drag: an undo from the keyboard, or + // the lyrics removed or replaced. The word being dragged is then not + // what the drag last made it, and nothing it would do now is right + auto model = dragModel(); + if (!model || !model->containsEvent(m_dragCurrent)) { + abandonDrag(); + return; + } + + // The edge moves as far as the pointer has, in time: the pointer + // need not have been exactly on it, and the view may scroll + sv_frame_t wanted = m_dragEdgeFrame + + (m_pane->getFrameForX(x) - m_dragPressFrame); + + sv_samplerate_t rate = model->getSampleRate(); + Event moved = (m_dragPart == LyricsEdit::Part::Start ? + LyricsEdit::startDraggedTo(m_dragWords, m_dragWord, + wanted, rate) : + LyricsEdit::endDraggedTo(m_dragWords, m_dragWord, + wanted, rate)); + if (moved == m_dragCurrent) return; + + // Each move takes the word out and puts the moved one in, and the + // command folds the two steps of each move before into this one: at + // the end it holds the original out and the last one in. The layer + // lays the words out again and repaints as the model changes + if (!m_dragCommand) { + m_dragCommand = new ChangeEventsCommand + (m_dragModel.untyped, + m_dragPart == LyricsEdit::Part::Start ? + tr("Move Word Start") : tr("Move Word End")); + } + m_dragCommand->remove(m_dragCurrent); + m_dragCurrent = moved; + m_dragCommand->add(m_dragCurrent); +} + +void +LyricsEditor::finishDrag() +{ + if (!m_dragging) return; + m_dragging = false; + + ChangeEventsCommand *command = m_dragCommand; + m_dragCommand = nullptr; + m_dragWords.clear(); + if (!command) return; + + // As in dragTo(): if the model has changed under the drag, there is + // nothing right to put on the history + auto model = dragModel(); + if (!model || !model->containsEvent(m_dragCurrent)) { + delete command; + return; + } + + // Dragged away and back: the model holds the word as it was, and + // there is nothing to undo + if (m_dragCurrent == m_dragOriginal) { + delete command; + return; + } + + // Done already, as the drag went. CommandHistory marks the session + // modified + command = command->finish(); + if (command) { + CommandHistory::getInstance()->addCommand(command, false); + } +} + +void +LyricsEditor::abandonDrag() +{ + // Deleting a command does not touch the model: whatever the drag had + // done to it stays, as whatever changed it since left it + delete m_dragCommand; + m_dragCommand = nullptr; + m_dragWords.clear(); + + // The rest of the drag, up to the release, moves nothing, and is + // still not the pane's: it never saw the press + m_dragModel = ModelId(); +} + +void +LyricsEditor::setEdgeCursor() +{ + if (!m_pane) return; + if (!m_cursorSet) { + m_savedCursor = m_pane->cursor(); + m_cursorSet = true; + } + m_pane->setCursor(Qt::SizeHorCursor); +} + +void +LyricsEditor::restoreCursor() +{ + if (!m_cursorSet) return; + m_cursorSet = false; + + // Unless the pane has put another of its own in the meantime, for a + // change of tool say: that one stays + if (m_pane && m_pane->cursor().shape() == Qt::SizeHorCursor) { + m_pane->setCursor(m_savedCursor); + } +} + +void +LyricsEditor::showHelp(const QString &help) +{ + // Every time, as the pane does: something else may have written the + // status bar since + m_helpShown = true; + emit contextHelpChanged(help); +} + +void +LyricsEditor::clearHelp() +{ + if (!m_helpShown) return; + m_helpShown = false; + emit contextHelpChanged(""); +} diff --git a/main/LyricsEditor.h b/main/LyricsEditor.h new file mode 100644 index 00000000..fc5bba09 --- /dev/null +++ b/main/LyricsEditor.h @@ -0,0 +1,159 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_EDITOR_H +#define TONY_LYRICS_EDITOR_H + +#include "LyricsEdit.h" + +#include "base/BaseTypes.h" +#include "base/Event.h" +#include "data/model/Model.h" + +#include +#include +#include +#include +#include + +#include + +class QMouseEvent; +class LyricsTrack; + +namespace sv { +class Pane; +class RegionLayer; +class RegionModel; +class ChangeEventsCommand; +} + +/** + * Edit mode for the lyrics (Edit > Edit Lyrics): the mouse in the box + * row of the lyrics, along the bottom of their pane, moves a word's + * start or end. + * + * The lyrics layer is never the pane's top layer, so the pane's tools + * never reach it: this watches the pane's mouse events through an + * event filter, which is on only while edit mode is. Only what it acts + * on is kept from the pane: a left press on an edge and the drag it + * starts, and moves with no button held in the box row, where the + * cursor and the context help are this one's. Everything else, a + * click anywhere to move the playback cursor included, goes to the pane + * as it would without edit mode. + * + * What an edit may do is LyricsEdit's to say; this does as it says. + * A drag edits the model as it goes, so the words move under the + * pointer, and is one command on the undo history when the button is + * let go, or nothing at all if the word is where it was. + * + * The layer and the model are found through LyricsTrack at every + * event, and nothing of them is kept between drags: an import, a + * remove or another session can replace them at any time. + * + * Like the lyrics track itself, MainWindow owns it and only wires it. + */ +class LyricsEditor : public QObject +{ + Q_OBJECT + +public: + LyricsEditor(LyricsTrack *lyrics, QObject *parent = nullptr); + virtual ~LyricsEditor(); + + /** + * Switch edit mode on, in the pane the lyrics are in, or off. On + * does nothing if there are no lyrics. Off finishes a drag that is + * in progress first, as letting go of the button would, then puts + * the pane's cursor back and takes the event filter off. + */ + void setEnabled(bool enabled); + bool isEnabled() const { return m_enabled; } + + /// True from the press on an edge to the release + bool isDragging() const { return m_dragging; } + + /// How near an edge, in logical pixels on either side, grabs it + static constexpr int grabPixels = 6; + +signals: + /// What the mouse does where the pointer is, "" when that is over + void contextHelpChanged(const QString &); + +protected: + bool eventFilter(QObject *, QEvent *) override; + +private: + LyricsTrack *m_lyrics; + bool m_enabled; + + // The pane whose events are filtered, while edit mode is on + QPointer m_pane; + + // The drag. The words are those of the model at the press, and the + // same ones go to LyricsEdit at every move: the limits of the edge + // come from where the word was then, so a drag back puts it back + bool m_dragging; + sv::ModelId m_dragModel; + sv::EventVector m_dragWords; + int m_dragWord; + LyricsEdit::Part m_dragPart; + sv::sv_frame_t m_dragEdgeFrame; + sv::sv_frame_t m_dragPressFrame; + sv::Event m_dragOriginal; + sv::Event m_dragCurrent; + sv::ChangeEventsCommand *m_dragCommand; + + // The pane's own cursor, while ours is shown over an edge + bool m_cursorSet; + QCursor m_savedCursor; + + // Whether the context help is ours + bool m_helpShown; + + // The lyrics layer, if it is in the pane being edited, shown or + // hidden; and its model, if that is the one the drag began in + sv::RegionLayer *currentLayer() const; + std::shared_ptr dragModel() const; + + // What the pointer is on, if it is in the box row: false if it is + // not, or there are no lyrics on show. The words are the model's now + bool hitAt(QPoint pos, LyricsEdit::Hit &hit, + sv::EventVector &words) const; + + bool mousePressed(QMouseEvent *); + bool mouseMoved(QMouseEvent *); + bool mouseReleased(QMouseEvent *); + + // Cursor and context help for a pointer here with no button held. + // True if it is in the box row + bool hover(QPoint pos); + void leaveRow(); + + void dragTo(int x); + + // Push the drag's command, if the word has moved + void finishDrag(); + + // Throw the drag's command away, without touching the model: the + // model has changed under the drag + void abandonDrag(); + + void setEdgeCursor(); + void restoreCursor(); + void showHelp(const QString &); + void clearHelp(); +}; + +#endif diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index e39ff4fa..3c83c643 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -43,7 +43,8 @@ class RegionLayer; * The layer is display only. It is never the pane's top layer, because * the pane takes the hover readout and the vertical scale from that one * and this style has neither; it cannot be played (a RegionModel has no - * play parameters), and it takes no edits. + * play parameters), and it takes no edits itself: LyricsEditor edits + * its words. * * It looks like the coverage strip's class and is used the same way: * MainWindow only wires it. @@ -97,6 +98,9 @@ class LyricsTrack : public QObject sv::RegionLayer *getLayer() const { return m_layer; } + /// The pane the layer is in, null when there are no lyrics + sv::Pane *getPane() const { return m_layer ? m_pane : nullptr; } + /// The model the words are in sv::ModelId getModelId() const; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0affb265..4a15339d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -159,6 +159,8 @@ MainWindow::MainWindow(AudioMode audioMode, m_exportLyricsAction(nullptr), m_removeLyricsAction(nullptr), m_showLyrics(nullptr), + m_lyricsEditor(nullptr), + m_editLyricsAction(nullptr), m_takesMenu(nullptr), m_takeCombo(nullptr), m_newTakeAction(nullptr), @@ -407,6 +409,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_takes = new SingingTakes(this); m_coverageStrip = new CoverageStrip(this); m_lyrics = new LyricsTrack(this); + m_lyricsEditor = new LyricsEditor(m_lyrics, this); + connect(m_lyricsEditor, &LyricsEditor::contextHelpChanged, + this, &MainWindow::contextHelpChanged); // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end @@ -501,6 +506,9 @@ MainWindow::~MainWindow() m_alternatePitch = nullptr; delete m_coverageStrip; m_coverageStrip = nullptr; + // Before the lyrics track, which it finds the lyrics through + delete m_lyricsEditor; + m_lyricsEditor = nullptr; delete m_lyrics; m_lyrics = nullptr; delete m_analyser; @@ -921,6 +929,19 @@ MainWindow::setupEditMenu() m_eraseSingingAction->setEnabled(false); menu->addAction(m_eraseSingingAction); m_rightButtonMenu->addAction(m_eraseSingingAction); + + menu->addSeparator(); + + // A mode, not a tool: the lyrics are never the pane's top layer, which + // is what the tools act on. Enabled and checked in updateMenuStates(). + // No shortcut: it is not switched on and off in the middle of things + m_editLyricsAction = new QAction(tr("Edit L&yrics"), this); + m_editLyricsAction->setCheckable(true); + m_editLyricsAction->setStatusTip(tr("Drag the start or end of a word of the lyrics, along the bottom of the pane, to move it")); + m_editLyricsAction->setEnabled(false); + connect(m_editLyricsAction, &QAction::triggered, + this, &MainWindow::editLyricsToggled); + menu->addAction(m_editLyricsAction); } void @@ -2262,6 +2283,22 @@ MainWindow::updateMenuStates() m_removeLyricsAction->setEnabled(m_lyrics && m_lyrics->isShown()); } + // Edit mode goes off here whenever it is no longer to be had: Remove + // Lyrics, Show Lyrics, the base class's record() once the take has + // started, and closeSession() (by documentRestored()) all come + // through here. None of those is an undo or a redo, which come + // through here as well, and during which the drag that this finishes + // could not push its command + bool lyricsEditable = lyricsEditAllowed(); + if (!lyricsEditable && m_lyricsEditor && m_lyricsEditor->isEnabled()) { + setLyricsEditing(false); + } + if (m_editLyricsAction) { + m_editLyricsAction->setEnabled(lyricsEditable); + m_editLyricsAction->setChecked + (m_lyricsEditor && m_lyricsEditor->isEnabled()); + } + if (pitchCandidatesVisible) { m_showCandidatesAction->setText(tr("Hide Pitch Candidates")); m_showCandidatesAction->setStatusTip(tr("Remove the display of alternate pitch candidates for the selected region")); @@ -3457,6 +3494,10 @@ MainWindow::importLyricsFrom(QString path) return e.getFrame() >= end; })); + // New words are not what edit mode was switched on for, and the ones + // there are go now: a drag of one of them ends first + setLyricsEditing(false); + QString name = (lyrics.title != "" ? lyrics.title : tr("Lyrics")); if (!m_lyrics->show(m_document, pane, events, name)) { // Only if the layer could not be made @@ -3606,6 +3647,36 @@ MainWindow::showLyricsToggled() } updateWaveformFade(); updateLayerStatuses(); + + // Edit Lyrics goes with the words out of sight + updateMenuStates(); +} + +bool +MainWindow::lyricsEditAllowed() const +{ + if (!m_lyrics || !m_lyrics->isShown() || !m_lyrics->isVisible()) { + return false; + } + if (m_recordTarget && m_recordTarget->isRecording()) return false; + return true; +} + +void +MainWindow::setLyricsEditing(bool on) +{ + if (!m_lyricsEditor) return; + m_lyricsEditor->setEnabled(on && lyricsEditAllowed()); + if (m_editLyricsAction) { + m_editLyricsAction->setChecked(m_lyricsEditor->isEnabled()); + } +} + +void +MainWindow::editLyricsToggled() +{ + if (!m_editLyricsAction) return; + setLyricsEditing(m_editLyricsAction->isChecked()); } void diff --git a/main/MainWindow.h b/main/MainWindow.h index 250824ef..63ea6728 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -22,6 +22,7 @@ #include "AlternatePitchTrack.h" #include "CoverageStrip.h" #include "LyricsTrack.h" +#include "LyricsEditor.h" #include "SingingTakes.h" #include "TakeCommands.h" #include "TakeTiming.h" @@ -184,6 +185,7 @@ protected slots: virtual void exportLyrics(); virtual void removeLyrics(); virtual void showLyricsToggled(); + virtual void editLyricsToggled(); virtual void editDisplayExtents(); @@ -374,6 +376,23 @@ protected slots: QAction *m_removeLyricsAction; QAction *m_showLyrics; + // Edit > Edit Lyrics: the mouse moves the words' starts and ends in + // the lyrics' box row while it is on. Off, and not to be had, + // without lyrics on show or while a take is being recorded, which + // updateMenuStates() sees to; and off after an import, which + // importLyricsFrom() sees to + LyricsEditor *m_lyricsEditor; + QAction *m_editLyricsAction; + + // The lyrics are there to be edited: shown, visible, and no take + // being recorded (the singer is reading them) + bool lyricsEditAllowed() const; + + // Edit mode on, if lyricsEditAllowed(), or off, and the action to + // match. Off finishes a drag in progress, pushing its command, so + // this must not be reached from an undo or a redo + void setLyricsEditing(bool on); + // Fade the waveforms of both analysers while the lyrics are on show // over them, and not otherwise. Called after anything that shows or // hides the lyrics, and after anything that makes an analyser: a new diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index e9509bd3..1b322cb9 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -29,6 +29,7 @@ #include "../Analyser.h" #include "../CoverageStrip.h" #include "../Lyrics.h" +#include "../LyricsEditor.h" #include "../LyricsTrack.h" #include "../LyricsTtml.h" #include "../SingingTakes.h" @@ -229,6 +230,10 @@ class TestMainWindow : public MainWindow QAction *removeLyricsAction() { return m_removeLyricsAction; } QAction *showLyricsAction() { return m_showLyrics; } + // Edit > Edit Lyrics, and the editor it switches on + QAction *editLyricsAction() { return m_editLyricsAction; } + LyricsEditor *lyricsEditor() { return m_lyricsEditor; } + // The file Import Lyrics asks for, answered from here: "" is Cancel void setLyricsFileAnswer(QString path) { m_lyricsFileAnswer = path; } int lyricsFileQuestions() const { return m_lyricsFileQuestions; } @@ -991,6 +996,150 @@ class TestRecordWorkflow : public QObject return e.getLabel(); } + // --- Editing the lyrics with the mouse in pane 0 --- + + sv::Pane *pane0() { return m_window->paneStack()->getPane(0); } + + // The lyrics' box row in pane 0, the pane painted first: the layer + // knows where the row is only once it has painted it there + QRect lyricsBoxRow() { + sv::Pane *pane = pane0(); + sv::RegionLayer *layer = m_window->lyrics()->getLayer(); + if (!pane || !layer) return {}; + pane->grab(); + return layer->getLyricsBoxRow(pane); + } + + // The word with this text as the model holds it now; frame -1 if + // there is none + sv::Event lyricsWord(QString text) { + for (const sv::Event &e : lyricsEvents()) { + if (e.getLabel() == text) return e; + } + return sv::Event(-1); + } + + static sv::sv_frame_t endOf(const sv::Event &e) { + return e.getFrame() + e.getDuration(); + } + + // The column of pane 0 that a frame falls in: a word's box runs from + // the column of its start to the one before the column of its end + int columnOf(sv::sv_frame_t frame) { return pane0()->getXForFrame(frame); } + + // A mouse event as Qt gives it to the pane: to its event filters + // first, then to the pane. Not QTest::mouseMove, which does not + // carry the buttons held + void sendMouse(QEvent::Type type, QPoint pos, Qt::MouseButton button, + Qt::MouseButtons buttons) { + sv::Pane *pane = pane0(); + QVERIFY(pane); + QMouseEvent event(type, QPointF(pos), QPointF(pane->mapToGlobal(pos)), + button, buttons, Qt::NoModifier); + QApplication::sendEvent(pane, &event); + } + + void hoverAt(QPoint pos) { + sendMouse(QEvent::MouseMove, pos, Qt::NoButton, Qt::NoButton); + } + void pressAt(QPoint pos) { + sendMouse(QEvent::MouseButtonPress, pos, Qt::LeftButton, Qt::LeftButton); + } + void moveHeldTo(QPoint pos) { + sendMouse(QEvent::MouseMove, pos, Qt::NoButton, Qt::LeftButton); + } + void releaseAt(QPoint pos) { + sendMouse(QEvent::MouseButtonRelease, pos, Qt::LeftButton, Qt::NoButton); + } + + // Press at one point, move to the other a few pixels at a time with + // the button held, and let go there + void dragFromTo(QPoint from, QPoint to) { + pressAt(from); + int steps = std::max(1, std::abs(to.x() - from.x()) / 3); + for (int i = 1; i <= steps; ++i) { + moveHeldTo(QPoint(from.x() + (to.x() - from.x()) * i / steps, + from.y() + (to.y() - from.y()) * i / steps)); + } + releaseAt(to); + } + + // A reference with the gapped lyrics on it, painted. Yksi and kaksi + // share an edge at 0.6 s; kolme comes after a gap. The row's middle + // is where the tests point, at the columns the words are in. + // + // The window is never shown, and its layout gives pane 0 what a + // 640x480 window leaves over, a few pixels high, or never lays a new + // pane out at all. A size of its own, then, and a zoom at which the + // whole reference is in view and a word is about a hundred pixels + // wide + QRect m_row; + void showEditableLyrics() { + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->doImportLyricsFrom(writeLrc(gappedLyrics()))); + + sv::Pane *pane = pane0(); + pane->setFixedSize(1000, 120); + pane->setZoomLevel(sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, 128)); + pane->setCentreFrame(sv::sv_frame_t(rate * 1.0)); + m_row = lyricsBoxRow(); + QVERIFY2(!m_row.isEmpty(), "the lyrics were not painted"); + QVERIFY2(m_row.top() > 12 && m_row.bottom() < pane->height(), + qPrintable(QString("the box row is at %1 to %2 of %3") + .arg(m_row.top()).arg(m_row.bottom()) + .arg(pane->height()))); + + // All in view, and wide enough for the grab not to reach across + // a word + int x0 = columnOf(lyricsWord("Yksi").getFrame()); + int x1 = columnOf(endOf(lyricsWord("kolme"))); + QVERIFY2(x0 > 100 && x1 < pane->width() - 100, + qPrintable(QString("the words are at %1 to %2") + .arg(x0).arg(x1))); + int width = columnOf(endOf(lyricsWord("kaksi"))) - + columnOf(lyricsWord("kaksi").getFrame()); + QVERIFY2(width >= 80, qPrintable(QString("kaksi is %1 pixels wide") + .arg(width))); + + m_window->discardModifications(); + } + + // Edit > Edit Lyrics, as the user switches it on + void switchLyricsEditingOn() { + QAction *edit = m_window->editLyricsAction(); + QVERIFY(edit->isEnabled()); + QVERIFY(!edit->isChecked()); + edit->trigger(); + QVERIFY(edit->isChecked()); + QVERIFY(m_window->lyricsEditor()->isEnabled()); + } + + void lyricsEditFixture(FakeAudioIO::Config config = FakeAudioIO::Config()) { + makeWindow(config); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + switchLyricsEditingOn(); + } + + // A drag from the shared edge, which with edit mode off is the pane's + // own: it moves the view, and no word + void dragIsThePanes(const sv::EventVector &before) { + int edge = columnOf(lyricsWord("kaksi").getFrame()); + dragFromTo(inRow(edge), inRow(edge + 30)); + QCOMPARE(lyricsEvents(), before); + QVERIFY2(columnOf(lyricsWord("kaksi").getFrame()) != edge, + "the pane did not get the drag"); + } + + // How far the pane's frames move for a move of the pointer from one + // column to another + sv::sv_frame_t framesBetween(int x0, int x1) { + return pane0()->getFrameForX(x1) - pane0()->getFrameForX(x0); + } + + QPoint inRow(int x) { return QPoint(x, m_row.center().y()); } + // Every key of the settings, with its value, in the form the test // messages show static QMap allSettings() { @@ -6677,6 +6826,557 @@ private slots: QCOMPARE(waveformColour(m_window->analyser()), QString("Grey")); } + // Without Edit Lyrics the mouse is the pane's everywhere: drags from + // either side of the shared edge move no word + void lyrics_edit_off_leaves_words_alone() { + makeWindow(FakeAudioIO::Config()); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + sv::EventVector before = lyricsEvents(); + + QAction *edit = m_window->editLyricsAction(); + QVERIFY(edit->isEnabled()); + QVERIFY(!edit->isChecked()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + + // The pane's navigate drags, which move the view + int edge = columnOf(lyricsWord("kaksi").getFrame()); + dragFromTo(inRow(edge - 1), inRow(edge - 30)); + QVERIFY2(columnOf(lyricsWord("kaksi").getFrame()) != edge, + "the pane did not get the drag"); + edge = columnOf(lyricsWord("kaksi").getFrame()); + dragFromTo(inRow(edge), inRow(edge + 30)); + QCOMPARE(lyricsEvents(), before); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString()); + } + + // Yksi's end and kaksi's start are one edge on screen. The column + // left of it is Yksi's, the one right of it kaksi's, and the side the + // pointer is on picks the one word that moves (decision 3) + void lyrics_edit_shared_edge_each_side() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kaksi = lyricsWord("kaksi"); + QCOMPARE(endOf(yksi), kaksi.getFrame()); + int edge = columnOf(kaksi.getFrame()); + + dragFromTo(inRow(edge - 1), inRow(edge - 21)); + sv::Event moved = lyricsWord("Yksi"); + QCOMPARE(endOf(moved), endOf(yksi) + framesBetween(edge - 1, edge - 21)); + QCOMPARE(moved.getFrame(), yksi.getFrame()); + QCOMPARE(moved.getValue(), yksi.getValue()); + QCOMPARE(moved.getLabel(), yksi.getLabel()); + QCOMPARE(lyricsWord("kaksi"), kaksi); + QCOMPARE(int(lyricsEvents().size()), 3); + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsWord("Yksi"), yksi); + + dragFromTo(inRow(edge), inRow(edge + 20)); + moved = lyricsWord("kaksi"); + QCOMPARE(moved.getFrame(), kaksi.getFrame() + framesBetween(edge, edge + 20)); + QCOMPARE(endOf(moved), endOf(kaksi)); + QCOMPARE(moved.getValue(), kaksi.getValue()); + QCOMPARE(lyricsWord("Yksi"), yksi); + QCOMPARE(int(lyricsEvents().size()), 3); + QCOMPARE(undoOnce(), QString("Move Word Start")); + QCOMPARE(lyricsWord("kaksi"), kaksi); + } + + // An edge stops at the neighbouring word and 20 ms from the word's + // other edge, however far the pointer goes (decision 6) + void lyrics_edit_clamps() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + sv::EventVector before = lyricsEvents(); + + // kaksi's end, over the gap and past the whole of kolme + int kaksiEnd = columnOf(endOf(kaksi)) - 1; + dragFromTo(inRow(kaksiEnd), inRow(columnOf(endOf(kolme)) + 30)); + QCOMPARE(endOf(lyricsWord("kaksi")), kolme.getFrame()); + QCOMPARE(lyricsWord("kolme"), kolme); + QCOMPARE(undoOnce(), QString("Move Word End")); + + // kolme's start, back over the gap and past Yksi + int kolmeStart = columnOf(kolme.getFrame()); + dragFromTo(inRow(kolmeStart), inRow(columnOf(yksi.getFrame()) - 30)); + QCOMPARE(lyricsWord("kolme").getFrame(), endOf(kaksi)); + QCOMPARE(endOf(lyricsWord("kolme")), endOf(kolme)); + QCOMPARE(lyricsWord("kaksi"), kaksi); + QCOMPARE(undoOnce(), QString("Move Word Start")); + + // Yksi's end, back past its own start: 20 ms after it + int yksiEnd = columnOf(endOf(yksi)) - 1; + dragFromTo(inRow(yksiEnd), inRow(columnOf(yksi.getFrame()) - 30)); + QCOMPARE(endOf(lyricsWord("Yksi")), + yksi.getFrame() + LyricsEdit::minWordFrames(rate)); + QCOMPARE(lyricsWord("Yksi").getFrame(), yksi.getFrame()); + QCOMPARE(undoOnce(), QString("Move Word End")); + + // kaksi's start, on past its own end + int kaksiStart = columnOf(kaksi.getFrame()); + dragFromTo(inRow(kaksiStart), inRow(columnOf(endOf(kaksi)) + 30)); + QCOMPARE(lyricsWord("kaksi").getFrame(), + endOf(kaksi) - LyricsEdit::minWordFrames(rate)); + QCOMPARE(undoOnce(), QString("Move Word Start")); + + // Nothing after the last word: kolme's end goes where it is taken + int kolmeEnd = columnOf(endOf(kolme)) - 1; + dragFromTo(inRow(kolmeEnd), inRow(kolmeEnd + 40)); + QCOMPARE(endOf(lyricsWord("kolme")), + endOf(kolme) + framesBetween(kolmeEnd, kolmeEnd + 40)); + QCOMPARE(undoOnce(), QString("Move Word End")); + + // and a drag past the neighbour and back leaves the edge where the + // pointer is: the limit is not where the drag got to + pressAt(inRow(kaksiEnd)); + moveHeldTo(inRow(columnOf(endOf(kolme)) + 30)); + QCOMPARE(endOf(lyricsWord("kaksi")), kolme.getFrame()); + moveHeldTo(inRow(kaksiEnd + 20)); + releaseAt(inRow(kaksiEnd + 20)); + QCOMPARE(endOf(lyricsWord("kaksi")), + endOf(kaksi) + framesBetween(kaksiEnd, kaksiEnd + 20)); + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + } + + // A drag edits the words as it goes and is one step on the history + // when let go (decision 12); a click, or a drag back to where it + // began, is none + void lyrics_edit_one_step_per_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + auto *history = sv::CommandHistory::getInstance(); + QSignalSpy commands(history, qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::Event kaksi = lyricsWord("kaksi"); + int end = columnOf(endOf(kaksi)) - 1; + + pressAt(inRow(end)); + QVERIFY(editor->isDragging()); + releaseAt(inRow(end)); + QVERIFY(!editor->isDragging()); + + pressAt(inRow(end)); + for (int x = end; x <= end + 40; x += 4) moveHeldTo(inRow(x)); + QVERIFY(endOf(lyricsWord("kaksi")) > endOf(kaksi)); + for (int x = end + 40; x >= end; x -= 4) moveHeldTo(inRow(x)); + releaseAt(inRow(end)); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(int(commands.count()), 0); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString()); + + // Many moves, each one seen in the model at once, and nothing on + // the history until the button is let go + pressAt(inRow(end)); + for (int x = end + 2; x <= end + 60; x += 2) { + moveHeldTo(inRow(x)); + QCOMPARE(endOf(lyricsWord("kaksi")), + endOf(kaksi) + framesBetween(end, x)); + } + QCOMPARE(int(commands.count()), 0); + releaseAt(inRow(end + 60)); + QCOMPARE(int(commands.count()), 1); + QVERIFY(m_window->isDocumentModified()); + sv::EventVector afterFirst = lyricsEvents(); + + int yksiStart = columnOf(lyricsWord("Yksi").getFrame()); + dragFromTo(inRow(yksiStart), inRow(yksiStart + 30)); + QCOMPARE(int(commands.count()), 2); + sv::EventVector afterSecond = lyricsEvents(); + QVERIFY(afterSecond != afterFirst); + + QCOMPARE(undoOnce(), QString("Move Word Start")); + QCOMPARE(lyricsEvents(), afterFirst); + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + QCOMPARE(redoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), afterFirst); + QCOMPARE(redoOnce(), QString("Move Word Start")); + QCOMPARE(lyricsEvents(), afterSecond); + QCOMPARE(redoOnce(), QString()); + + // Still the lyrics' own model, out of the play source + QVERIFY(m_window->playSource()->getModels().count + (m_window->lyrics()->getModelId()) == 0); + verifyPlaySourceClean(); + } + + // An edit is a change to the session, and the session keeps it + void lyrics_edit_saved_with_the_session() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::EventVector before = lyricsEvents(); + + int end = columnOf(endOf(lyricsWord("kaksi"))) - 1; + dragFromTo(inRow(end), inRow(end + 50)); + int start = columnOf(lyricsWord("kolme").getFrame()); + dragFromTo(inRow(start), inRow(start + 25)); + QVERIFY(m_window->isDocumentModified()); + sv::EventVector edited = lyricsEvents(); + QVERIFY(edited != before); + + QString session = m_dir.filePath("lyrics-edited.ton"); + QVERIFY(m_window->saveSessionFile(session)); + reopenSession(session); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->lyrics()->isShown()); + verifyEventsSurvived(edited, lyricsEvents(), "the edited lyrics"); + if (QTest::currentTestFailed()) return; + QCOMPARE(lyricsWord("kaksi").getLabel(), QString("kaksi")); + + // Edit mode is not part of the session + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(m_window->editLyricsAction()->isEnabled()); + QVERIFY(!m_window->editLyricsAction()->isChecked()); + QVERIFY(m_window->playSource()->getModels().count + (m_window->lyrics()->getModelId()) == 0); + verifyPlaySourceClean(); + } + + // The word at the cursor is found again as its edge moves, while the + // button is held as well + void lyrics_edit_highlight_follows() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + sv::sv_frame_t gap = (endOf(kaksi) + kolme.getFrame()) / 2; + m_window->seekTo(gap); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + int end = columnOf(endOf(kaksi)) - 1; + int past = columnOf(gap) + 10; + pressAt(inRow(end)); + moveHeldTo(inRow(past)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("kaksi"), 1000); + releaseAt(inRow(past)); + QCOMPARE(highlightedWord(), QString("kaksi")); + + QCOMPARE(undoOnce(), QString("Move Word End")); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + int start = columnOf(kolme.getFrame()); + dragFromTo(inRow(start), inRow(columnOf(gap) - 10)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("kolme"), 1000); + } + + // Nothing to edit: the action is there, disabled, and its trigger + // does nothing + void lyrics_edit_needs_lyrics() { + makeWindow(FakeAudioIO::Config()); + QAction *edit = m_window->editLyricsAction(); + QVERIFY(!edit->isEnabled()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(!edit->isEnabled()); + QVERIFY(!edit->isChecked()); + edit->trigger(); + QVERIFY(!edit->isChecked()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + + QVERIFY(m_window->doImportLyricsFrom(writeLrc(gappedLyrics()))); + QVERIFY(edit->isEnabled()); + QVERIFY(!edit->isChecked()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + } + + // Turned off, and not to be had, with the lyrics hidden; shown again, + // it is to be had but stays off, and the mouse is the pane's + void lyrics_edit_off_when_hidden() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QAction *edit = m_window->editLyricsAction(); + QAction *show = m_window->showLyricsAction(); + sv::EventVector before = lyricsEvents(); + + show->trigger(); + QVERIFY(!m_window->lyrics()->isVisible()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(!edit->isEnabled()); + QVERIFY(!edit->isChecked()); + + show->trigger(); + QVERIFY(m_window->lyrics()->isVisible()); + QVERIFY(edit->isEnabled()); + QVERIFY(!edit->isChecked()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + dragIsThePanes(before); + } + + void lyrics_edit_off_on_remove() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QAction *edit = m_window->editLyricsAction(); + + m_window->removeLyricsAction()->trigger(); + QVERIFY(!m_window->lyrics()->isShown()); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(!edit->isEnabled()); + QVERIFY(!edit->isChecked()); + } + + // New lyrics are not what edit mode was switched on for + void lyrics_edit_off_on_import() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QAction *edit = m_window->editLyricsAction(); + + QVERIFY(m_window->doImportLyricsFrom(writeLrc(gappedLyrics()))); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(edit->isEnabled()); + QVERIFY(!edit->isChecked()); + m_row = lyricsBoxRow(); + dragIsThePanes(lyricsEvents()); + } + + // Off with the session, and off in the next one until switched on, + // which then edits in the new pane 0 + void lyrics_edit_off_on_close() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QAction *edit = m_window->editLyricsAction(); + + m_window->doCloseSession(); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(!edit->isEnabled()); + QVERIFY(!edit->isChecked()); + + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(!edit->isChecked()); + sv::EventVector before = lyricsEvents(); + dragIsThePanes(before); + if (QTest::currentTestFailed()) return; + + switchLyricsEditingOn(); + int edge = columnOf(lyricsWord("kaksi").getFrame()); + dragFromTo(inRow(edge), inRow(edge + 30)); + QCOMPARE(lyricsWord("kaksi").getFrame(), + before[1].getFrame() + framesBetween(edge, edge + 30)); + } + + // Off while a take is recorded, when the singer is reading the words. + // A drag going on when the take starts is finished first, and so is + // on the history before the take + void lyrics_edit_off_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + lyricsEditFixture(config); + if (QTest::currentTestFailed()) return; + QAction *edit = m_window->editLyricsAction(); + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + + int end = columnOf(endOf(lyricsWord("kaksi"))) - 1; + pressAt(inRow(end)); + moveHeldTo(inRow(end + 30)); + sv::EventVector dragged = lyricsEvents(); + QVERIFY(dragged != before); + + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!editor->isEnabled()); + QVERIFY(!editor->isDragging()); + QVERIFY(!edit->isEnabled()); + QVERIFY(!edit->isChecked()); + + // The rest of that drag moves nothing + moveHeldTo(inRow(end + 60)); + releaseAt(inRow(end + 60)); + QCOMPARE(lyricsEvents(), dragged); + + QTest::qWait(300); + stopTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(edit->isEnabled(), 2000); + QVERIFY(!edit->isChecked()); + QVERIFY(!editor->isEnabled()); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), before); + } + + // Edit mode switched off in the middle of a drag: the drag ends as a + // release would end it, and the rest of it is not an edit + void lyrics_edit_off_in_the_middle_of_a_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + + int end = columnOf(endOf(lyricsWord("kaksi"))) - 1; + pressAt(inRow(end)); + moveHeldTo(inRow(end + 30)); + sv::EventVector dragged = lyricsEvents(); + QVERIFY(dragged != before); + + m_window->editLyricsAction()->trigger(); + QVERIFY(!editor->isEnabled()); + QVERIFY(!editor->isDragging()); + QVERIFY(!m_window->editLyricsAction()->isChecked()); + QVERIFY(m_window->isDocumentModified()); + + moveHeldTo(inRow(end + 60)); + releaseAt(inRow(end + 60)); + QCOMPARE(lyricsEvents(), dragged); + + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + } + + // The words change under a drag: removed, or an undo (Ctrl+Z with the + // button held) takes away the word being dragged. The drag ends, the + // model is left as that made it, and nothing goes on the history + void lyrics_edit_words_change_under_a_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + sv::Event yksi = lyricsWord("Yksi"); + + // An earlier drag of Yksi's end, undone after a press on that end + int end = columnOf(endOf(yksi)) - 1; + dragFromTo(inRow(end), inRow(end - 20)); + sv::EventVector moved = lyricsEvents(); + end = columnOf(endOf(lyricsWord("Yksi"))) - 1; + pressAt(inRow(end)); + QVERIFY(editor->isDragging()); + QCOMPARE(undoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), before); + moveHeldTo(inRow(end - 10)); + releaseAt(inRow(end - 10)); + QVERIFY(!editor->isDragging()); + QCOMPARE(lyricsEvents(), before); + + // Nothing was pushed over the undone drag, which can be redone + QCOMPARE(redoOnce(), QString("Move Word End")); + QCOMPARE(lyricsEvents(), moved); + QCOMPARE(redoOnce(), QString()); + + // Removed in the middle of a drag + int start = columnOf(lyricsWord("kolme").getFrame()); + pressAt(inRow(start)); + moveHeldTo(inRow(start + 20)); + m_window->removeLyricsAction()->trigger(); + QVERIFY(!editor->isDragging()); + QVERIFY(!editor->isEnabled()); + moveHeldTo(inRow(start + 40)); + releaseAt(inRow(start + 40)); + + // On top of the history is still the redone drag of Yksi, whose + // model has gone with the lyrics (and which does nothing now), not + // a Move Word Start + QCOMPARE(undoOnce(), QString("Move Word End")); + QVERIFY(!m_window->lyrics()->isShown()); + verifyPlaySourceClean(); + } + + // The resize cursor over an edge, from either side of a shared one, + // and the pane's own back wherever else the pointer goes; the status + // bar says what the mouse does in the row + void lyrics_edit_cursor_and_help() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Pane *pane = pane0(); + Qt::CursorShape own = pane->cursor().shape(); + QVERIFY(own != Qt::SizeHorCursor); + int edge = columnOf(lyricsWord("kaksi").getFrame()); + QPoint above(edge + 40, m_row.top() - 8); + + hoverAt(inRow(edge - 1)); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + QCOMPARE(m_window->statusText(), + QString("Drag to move the end of \"Yksi\"")); + hoverAt(inRow(edge + 2)); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + QCOMPARE(m_window->statusText(), + QString("Drag to move the start of \"kaksi\"")); + + hoverAt(inRow(edge + 40)); + QCOMPARE(pane->cursor().shape(), own); + QCOMPARE(m_window->statusText(), + QString("Drag a word's start or end to move it")); + + hoverAt(inRow(edge)); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + hoverAt(above); + QCOMPARE(pane->cursor().shape(), own); + QVERIFY2(!m_window->statusText().startsWith("Drag"), + qPrintable(m_window->statusText())); + + // Kept through a drag that leaves the row, and given back after + pressAt(inRow(edge)); + moveHeldTo(above); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + releaseAt(above); + QCOMPARE(pane->cursor().shape(), own); + QCOMPARE(undoOnce(), QString("Move Word Start")); + + // Given back when edit mode goes off over an edge, with the help + hoverAt(inRow(edge)); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + m_window->editLyricsAction()->trigger(); + QCOMPARE(pane->cursor().shape(), own); + QVERIFY2(!m_window->statusText().startsWith("Drag"), + qPrintable(m_window->statusText())); + hoverAt(inRow(edge - 1)); + QCOMPARE(pane->cursor().shape(), own); + } + + // Everything the editor does not act on is the pane's: a click in a + // word moves the playback cursor as before, a click on an edge does + // not. A double-click on an edge is a press there + void lyrics_edit_clicks_elsewhere_are_the_panes() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event kolme = lyricsWord("kolme"); + int edge = columnOf(kolme.getFrame()); + int inside = edge + 50; + sv::sv_frame_t insideFrame = pane0()->getFrameForX(inside); + sv::sv_frame_t start = m_window->playbackFrame(); + QVERIFY(std::abs(start - insideFrame) > 10000); + + // The pane moves the cursor a double-click interval after a + // click, unless a second press comes first and takes its place: + // so the edge's click has its time to show it did nothing + pressAt(inRow(edge)); + releaseAt(inRow(edge)); + QTest::qWait(QApplication::doubleClickInterval() + 200); + QCOMPARE(m_window->playbackFrame(), start); + + pressAt(inRow(inside)); + releaseAt(inRow(inside)); + QTRY_VERIFY_WITH_TIMEOUT + (std::abs(m_window->playbackFrame() - insideFrame) < 1000, 3000); + QCOMPARE(lyricsWord("kolme"), kolme); + QCOMPARE(undoOnce(), QString()); + + pressAt(inRow(edge)); + releaseAt(inRow(edge)); + sendMouse(QEvent::MouseButtonDblClick, inRow(edge), + Qt::LeftButton, Qt::LeftButton); + moveHeldTo(inRow(edge + 20)); + releaseAt(inRow(edge + 20)); + QCOMPARE(lyricsWord("kolme").getFrame(), + kolme.getFrame() + framesBetween(edge, edge + 20)); + QCOMPARE(undoOnce(), QString("Move Word Start")); + QCOMPARE(undoOnce(), QString()); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform diff --git a/meson.build b/meson.build index 543f31b9..eacee66e 100644 --- a/meson.build +++ b/meson.build @@ -1110,6 +1110,7 @@ tony_app_files = [ 'main/AlternatePitchTrack.cpp', 'main/CoverageStrip.cpp', 'main/LyricsTrack.cpp', + 'main/LyricsEditor.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', @@ -1131,6 +1132,7 @@ tony_app_moc_files = qt.preprocess( 'main/AlternatePitchTrack.h', 'main/CoverageStrip.h', 'main/LyricsTrack.h', + 'main/LyricsEditor.h', ]) qt_resource_files = qt.preprocess( From 33df1b5cb374d65ba6b1d964c03bbbd6ac4d807e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:23:16 +0000 Subject: [PATCH 128/275] docs: calibrate audio work orders, C1 split and refined C1 becomes C1a (build flag, DevChecks, report, the dialog's dev run, items 1 and 2) and C1b (TakeObserver, items 7, 12, 13, 14). The dev checks run as timer-driven stages like the runner, not a nested event loop, and scratch folders stay with the session left open. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 129 +++++++++++++++++++++++++++- docs/calibrate-audio.md | 29 ++++--- 2 files changed, 142 insertions(+), 16 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 8ad34471..8a43b6f1 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -127,7 +127,7 @@ push, amend, stash, or `git add -A`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`). ### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) @@ -431,9 +431,125 @@ do not duplicate it. two at the join; a hard cut in a sine. - **Show failure** for two of them by breaking the code. -### C1 — Dev-check framework and first group (spec §3, §4, §5 "development builds only") - -To be refined by the lead. +### C1a — Dev-check framework, items 1 and 2 (spec §2 point 5, §3, §4, §5 "development builds only") + +Read also: `main/AudioCheckRunner.{h,cpp}` whole (about 1000 lines together; you extend +it), `main/CalibrateAudioDialog.h`, `main/test/TestAudioCheck.h` for the fixture +(`loopback()`, `shortPlan()`, `makeWindow()`), `docs/takes.md` on where take files go +before and after a save, and spec §4's rows for items 1 and 2. + +- **Build flag** (spec §3). In `meson.build`, a build type not starting with `release` + adds `-DTONY_DEV_CHECKS` to `general_defines`, and only then are `main/dev/*.cpp` + compiled into `tony_app` and their headers moc'd. `build_linux` is `debugoptimized`, so + it has them. Everything under `main/dev/`, and every use of it elsewhere (a `friend` + line, a member, a dialog widget, the test class's registration), is inside + `#ifdef TONY_DEV_CHECKS`. A release build must compile with no `main/dev/` file; the + lead builds one later, so keep the `#ifdef`s tidy. +- **Runner extensions** (every build; small; each tested in `TestAudioCheck`): + - **`Plan` ranges.** Explicit punch-ins in seconds; when given, they replace + `punchInsFor()`. `start()` refuses ranges that overlap, are out of order or lie + outside the layout. + - **`Plan` keeps the session.** Record into the session open now, which the caller + says is a check reference of the plan's layout: no reference written or opened, no + save question; straight on to the reference's analysis wait and the punch-ins. The + take keeps what earlier runs recorded; only this plan's punch-ins are judged. + - **`Plan` round trip, for the run only** (spec §10 "a dev run uses the new figure + for itself only"). Seconds; unset means the window's own. The window uses it for the + check's takes only, where `recordingStarted()` takes `roundTripAt()` now. It never + touches the stored figure or the Playback menu's line. Log it as the check's own. + - **`TakeLatency` gets the start gap** each take used, and whether it was measured or + only estimated (item 2 reports it per punch-in). + - **No save question for the check's own session.** When the session open has never + been saved and its main file is in `referenceDirectory()`, replacing it asks + nothing: it is the check before. This also ends B4's Check Again prompt. + - **Record during a check** (from B4): greyed while a check runs, and `record()` + ignores the user's press then. Pressing it today ends the check's take early. +- **`main/dev/DevChecks.{h,cpp}`**, a `QObject`. + - **No nested event loop.** Spec §5 said `waitUntil()` with a `QEventLoop`; the lead + changed it: a list of stages driven by the runner's `finished()` and a polling timer, + as the runner is driven, for the runner's reason (the window can be closed at any + moment). Each stage starts something and says when it is done; a stage that times + out fails the run. + - `start(Options)`, `cancel()`, `isRunning()`, `sessionClosing()` (as the runner's: + ends the run unless the run itself is replacing the session). Signals `progress` + (stage name, n of m) and `finished(DevReport)`, once however the run ends. + - `Options`: the round trip for the run (seconds), the report directory ("" for + `TONY_TEST_LOG_DIR` if set, else `AppDataLocation`), the scratch directory ("" for + `AppDataLocation`). + - `CheckResult { item, name, verdict (Pass, Fail, Measured, Skipped), numbers (label and + value pairs, as text), message }`; `DevReport { checks, failure, reportPath, + sessionPath }`. A run that ends early marks the checks it did not reach Skipped, + with the reason. + - Owned by `MainWindow` in dev builds, like the runner; a `friend` of it under the + `#ifdef`. `~MainWindow` deletes it after the dialog and before the runner; + `closeSession()` calls its `sessionClosing()`. +- **The stages of C1a** (C1b inserts more before the last): + 1. **Fresh punch-ins.** A runner run on `devLayout()`, opening a new reference (no + save question: see above), with two explicit punch-ins in separate regions of the + calibration part, each holding two sweeps, and the run's round trip. Choose ranges + that leave the held tones (after 25 s) and the start (before 3 s) free for later + stages, and say which you chose. + 2. **Save and reopen.** Save the session into a scratch folder (below) with + `MainWindow`'s own save path, no dialog; reopen it; wait for the analyses; read the + take's file again. +- **Items:** + - **1** Pass when every judged sweep of every punch-in lands within ±2 ms (a named + constant), and after the reopen the take's audio judged again gives the same offsets + and its pitch and notes are the same events as before the save. Numbers: offsets + per punch-in, largest offset, the round trip used. + - **2** Pass when every punch-in was added to the take, each placed within ±2 ms, each + with a measured start gap of its own. Numbers: per punch-in its median offset and + start gap. +- **Scratch folders.** Not deleted at the end: the session open afterwards lives in it + (spec §2: the test session stays open). As `nextReferencePath()` does for references: + numbered folders, and at the start of a run every one that the open session does not + use is removed. The report names the folder. +- **Report.** + - Text file `DevChecks.txt` in the report directory: the run's date, devices and round + trip, then one block per check grouped by checklist item (verdict, message, + numbers), ending `Totals: N passed, N failed, N measured, N skipped`. + - Tests pass a report directory of their own: a failing run's report must not land + among the suites' own files, where the lead greps for `^FAIL`. +- **Dialog** (dev builds only). The instructions page gets a checkbox, "Run the dev + checks after calibrating", on by default and not remembered. With it on and the + calibration usable, the progress page goes straight on into the dev checks with + `calibratedRoundTrip`, Cancel cancels whichever is running, and the result page + shows the calibration as today plus the dev report: one line per check, then the + report file's path. With the calibration unusable, the dev checks do not run and the + page says so. +- **Tests,** new class `TestDevChecks` in the app suite, compiled and registered only + in dev builds; the fixture copied from `TestAudioCheck`, not shared by editing it: + - passing on a loopback fake with its true round trip: items 1 and 2 pass, the report + file ends with `Totals:`, and the session open afterwards is the one in the scratch + folder; + - failing with the round trip 20 ms off: items 1 and 2 fail, and the report shows the + offsets; + - cancelled mid-run, and the session closed mid-run: `finished` once, no take left + recording, the user's three toggles and stored latency untouched; + - the dialog with the checkbox on runs the dev checks after the calibration (the short + plan for the calibration). + - **Show failure** for item 1's tolerance and for the run round trip being ignored. + - Report the real time `TestDevChecks` adds. Spec §6 says more than about a minute + over all the dev phases moves them to a third executable, which the lead will ask + the user about; do not create one. + +### C1b — Observer, items 7, 12, 13, 14 (spec §4, §5 `TakeObserver`) + +To be refined by the lead after C1a. Outline: + +- `main/dev/TakeObserver`: polls every 20 ms while the runner reports a take recording + and records, with the time: playback frame, output and input levels (left and right, + as `getOutputLevels()` and `getInputLevels()` give them; find out and say exactly what + one reading covers), status text, frames received, any modal widget up, and when the + take stopped. Live dots, pane centre and action states wait for C2. +- Before and after each punch-in: the take's samples from its file, and its pitch and + notes, compared with `TakeDiff`. +- New stages before the save and reopen: re-record over one of stage 1's punch-ins + starting inside it, so its lead-in plays over earlier material (items 7, 12, 14), with + no overwrite question for the check's takes; then a punch-in at P = 1 s with a 3 s + pre-roll for that plan (item 13: playback from 0, a shorter countdown, placement + right; checklist item 13 is "pre-roll less than 3 s from the start"). +- Items 1 and 2 then cover every punch-in of the run. ### C2 — Observer group (spec §4 items 3, 4, 5, 8, 15, 16) @@ -588,3 +704,8 @@ The next phase must know: - **Two punch-ins that meet at J leave a 10 ms dip, not a crossfade.** Each fades against what the file held there, which is silence: [J − 5 ms, J) fades out and [J, J + 5 ms) fades in (read from `weightAt()`, not measured). `stepAt()` reads it as no step. - The notes merge adds a new note only if its onset is in W (`Analyser.cpp`, "Notes go by their onset"). A second run's note that begins before J − 0.25 s is not added, and the dip may split the note at J. Either way, item 10's "one note across the join" may fail on today's code. C3 should measure it, not assume it. Left open: every threshold untuned; nothing calls `TakeDiff` yet. + +### Lead — 2026-09-26, after C0 +- C1 is split into C1a (framework, items 1 and 2) and C1b (observer, items 7, 12, 13, 14); spec §7 says so. +- DevChecks runs as stages driven by the runner and a timer, not a nested event loop; scratch folders stay with the open session and the next run removes the old ones. Spec §5 and §8 changed to match. +- C0's warning about item 10 (a 10 ms dip at two punch-ins' join; notes merged by onset) stands for C3. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 8d481dab..4a56aeb2 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -62,7 +62,8 @@ with less noise. **Use this latency** stores the figure. 5. **Development builds only:** the dialog carries on into the **dev checks**, about - 4 minutes (section 5). It ends with a report page and a report file. + 4 minutes (section 5), unless a checkbox on the instructions page (on by default) + is cleared. It ends with a report page and a report file. The test session stays open afterwards, so the reference's and the takes' pitch tracks can be looked at. You go back to your song through Recent Files. @@ -206,9 +207,10 @@ goes in `tony_core`. - **`DevChecks`.** A list of checks. Each returns a plain `CheckResult { item, name, verdict (Pass/Fail/Measured/Skipped), numbers, message }`. - Waiting is done with a small `waitUntil(predicate, timeout)`, a `QEventLoop` with a - timer, behind the modal progress dialog. Cancel stops the take and closes the test - session cleanly. + The run is a list of stages, each starting something and saying when it is done, + driven by the runner's `finished()` and a polling timer as the runner itself is, and + never by a nested event loop: the window can be closed at any moment. Cancel stops + the take and leaves the test session open. They are **not** QtTest functions. A QVERIFY failure cannot be asserted from inside another QtTest, and the app suite has to prove each check can fail. @@ -217,9 +219,10 @@ goes in `tony_core`. - **Report.** A page in the dialog, grouped by checklist item, with the measured numbers. A text file goes to `TONY_TEST_LOG_DIR` if that is set, else to the app data directory, and ends with a `Totals:` line like the suites. -- **Scratch files.** The run saves its test session into a temporary folder, so every - take file lands in `.takes/`. It deletes the folder at the end, unless - something failed; then the report names it. +- **Scratch files.** The run saves its test session into a numbered scratch folder, so + every take file lands in `.takes/`, and the report names it. The folder stays + after the run, since the session open then lives in it; the next run removes every + one the open session does not use. ### The dev run, one scripted sequence (~4 min) @@ -301,8 +304,9 @@ marked "Done" when it is committed. 3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). 4. **Dev-check framework:** - **C0** `TakeDiff`, pure. Done. - - **C1** Build flag, `DevChecks`, `TakeObserver`, report, friend access. First - group: items 1, 2, 7, 12, 13, 14. + - **C1a** Build flag, `DevChecks`, report, friend access, the dialog's dev run. + Items 1 and 2. + - **C1b** `TakeObserver`. Items 7, 12, 13, 14. 5. **C2** Observer group: items 3, 4, 5, 8, 15, 16. 6. **C3** Join and long-song group: items 9 and 10. 7. **C4** Smoke group. @@ -329,9 +333,10 @@ could convert. The button then shows the fix working on each device. verdicts point to *Sound settings ▸ device ▸ Audio enhancements: Off*. - **Thresholds are guesses** until real runs exist. Every check reports its numbers as well as pass or fail, and the report file is what tunes them. -- **Nested event loops in the live app.** The run stays behind a modal progress dialog, - and Cancel must always leave a clean state. `closeSession()` already stops take - polling. +- **The live app during a run.** No nested event loops: the runner and the dev checks + are driven by timers, and the dialog is not modal, so the window can be used, and + closed, while they run. Cancel, and a session closed mid-run, must always leave a + clean state. `closeSession()` already stops take polling. - **Loudness.** The sweeps are −12 dBFS with earcups off the ears; the dialog says so before starting. *Found in B1:* not as played. Tony normalises every audio file to full scale as it reads it (`Preferences::setNormaliseAudio(true)`), so the reference From 85b5b69a40767233cad15cbe44a0eb79e9d8cee3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:26:48 +0000 Subject: [PATCH 129/275] feat: compact touch layout for a phone one toolbar of large buttons in place of the menu bar, the other toolbars and the overview: a menu button holding the window's menus, play, record, record into selection, the take box, undo, redo, erase, zoom, and a button for the show and play panel. on at start on android and with --compact, switched by view > compact layout; switching off puts back what was there. the app tests now link the icons. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/CompactLayout.cpp | 277 ++++++++++++++ main/CompactLayout.h | 157 ++++++++ main/MainWindow.cpp | 73 ++++ main/MainWindow.h | 22 ++ main/main.cpp | 7 +- main/test/TestCompactLayout.h | 688 ++++++++++++++++++++++++++++++++++ main/test/tony-app-test.cpp | 7 + meson.build | 6 + 8 files changed, 1236 insertions(+), 1 deletion(-) create mode 100644 main/CompactLayout.cpp create mode 100644 main/CompactLayout.h create mode 100644 main/test/TestCompactLayout.h diff --git a/main/CompactLayout.cpp b/main/CompactLayout.cpp new file mode 100644 index 00000000..5e317c50 --- /dev/null +++ b/main/CompactLayout.cpp @@ -0,0 +1,277 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "CompactLayout.h" + +#include "widgets/CommandHistory.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +CompactLayout::CompactLayout(QMainWindow *window) : + QObject(window), + m_window(window) +{ + m_action = new QAction(tr("&Compact Layout"), this); + m_action->setCheckable(true); + m_action->setStatusTip(tr("Show one toolbar of large buttons in place " + "of the menus and toolbars, as on a phone")); + connect(m_action, &QAction::toggled, this, &CompactLayout::setOn); +} + +CompactLayout::~CompactLayout() +{ + // The toolbar and its menu are the window's children, and go with it +} + +void +CompactLayout::setParts(const Parts &parts) +{ + m_parts = parts; +} + +bool +CompactLayout::isWantedAtStart(const QStringList &arguments) +{ +#ifdef Q_OS_ANDROID + (void)arguments; + return true; +#else + return arguments.contains("--compact"); +#endif +} + +void +CompactLayout::setOn(bool on) +{ + if (on != m_on) { + if (on) switchOn(); + else switchOff(); + } + m_action->setChecked(m_on); +} + +void +CompactLayout::makeToolBar() +{ + // Made the first time it is wanted, so that a window that is never + // compact has nothing of it + m_toolBar = new QToolBar(tr("Compact Toolbar"), m_window); + m_toolBar->setObjectName("Compact Toolbar"); + m_toolBar->setIconSize(QSize(iconSize, iconSize)); + + // Not to be dragged away by a finger, nor hidden from the window's + // menu of toolbars: the menus would go with it + m_toolBar->setMovable(false); + m_toolBar->toggleViewAction()->setVisible(false); + + // No icon to hand for the menus: the button shows the menu's title. + // It opens its popup on a tap, not on a press held down + m_menu = new QMenu(tr("Menu"), m_toolBar); + m_toolBar->addAction(m_menu->menuAction()); + m_menuButton = qobject_cast + (m_toolBar->widgetForAction(m_menu->menuAction())); + if (m_menuButton) { + m_menuButton->setPopupMode(QToolButton::InstantPopup); + } + m_toolBar->addSeparator(); + + for (QAction *action: { m_parts.play, m_parts.record, + m_parts.recordIntoSelection }) { + if (action) m_toolBar->addAction(action); + } + + // The take box is not the toolbar's for good: see moveTakeBoxIn() + m_takeBoxPlace = m_toolBar->addSeparator(); + + // The same Undo and Redo as the window's own toolbar, with their + // menus of what there is to undo and redo + sv::CommandHistory::getInstance()->registerToolbar(m_toolBar); + + if (m_parts.erase) m_toolBar->addAction(m_parts.erase); + m_toolBar->addSeparator(); + + for (QAction *action: { m_parts.zoomIn, m_parts.zoomOut }) { + if (action) m_toolBar->addAction(action); + } + m_toolBar->addSeparator(); + + // Again no icon: the name of the bar it brings up + m_panelAction = m_toolBar->addAction(tr("Show and Play")); + m_panelAction->setCheckable(true); + m_panelAction->setToolTip(tr("Show or hide what is shown and played, " + "the gains and the playback speed")); + connect(m_panelAction, &QAction::triggered, + this, &CompactLayout::showPanel); + + // A button that shows text is only as tall as the text: these are the + // toolbar's own buttons, and may grow to the height of the others + for (QAction *action: m_toolBar->actions()) { + if (!action->icon().isNull()) continue; + if (auto button = qobject_cast + (m_toolBar->widgetForAction(action))) { + button->setSizePolicy(QSizePolicy::Fixed, QSizePolicy::Preferred); + } + } +} + +void +CompactLayout::switchOn() +{ + if (!m_toolBar) makeToolBar(); + + // Before its action is hidden, which also disables it + if (m_parts.navigateTool && !m_parts.navigateTool->isChecked()) { + m_parts.navigateTool->trigger(); + } + + m_saved = Saved(); + + // isHidden(), not isVisible(): on Android this happens before the + // window is first shown, when nothing in it is visible yet + QMenuBar *menuBar = m_window->menuBar(); + m_saved.menuBarHidden = menuBar->isHidden(); + menuBar->hide(); + + for (QToolBar *toolBar: m_window->findChildren + (QString(), Qt::FindDirectChildrenOnly)) { + if (toolBar == m_toolBar) continue; + m_saved.toolBars.push_back({ toolBar, toolBar->isHidden() }); + toolBar->hide(); + } + + for (QWidget *widget: m_parts.hiddenWidgets) { + if (!widget) continue; + m_saved.widgets.push_back({ widget, widget->isHidden() }); + widget->hide(); + } + + for (QAction *action: m_parts.hiddenActions) { + if (!action) continue; + m_saved.actions.push_back({ action, action->isVisible() }); + action->setVisible(false); + } + + // The popup holds the menu bar's own menus, the same objects + for (QAction *action: menuBar->actions()) { + if (QMenu *menu = action->menu()) disableTearOff(menu); + m_menu->addAction(action); + } + + moveTakeBoxIn(); + + m_panelAction->setChecked(false); + + m_window->addToolBar(Qt::TopToolBarArea, m_toolBar); + + // Explicitly: it was hidden when last taken away, and a toolbar + // added to a window on show is otherwise shown only later + m_toolBar->show(); + + m_on = true; +} + +void +CompactLayout::switchOff() +{ + moveTakeBoxBack(); + + // The menus are the menu bar's alone again. clear() deletes only + // actions of the popup's own, and these are the menus' + m_menu->clear(); + + m_window->removeToolBar(m_toolBar); + + for (const auto &saved: m_saved.tearOffs) { + if (saved.first) saved.first->setTearOffEnabled(saved.second); + } + for (const auto &saved: m_saved.actions) { + if (saved.first) saved.first->setVisible(saved.second); + } + for (const auto &saved: m_saved.widgets) { + if (saved.first) saved.first->setHidden(saved.second); + } + for (const auto &saved: m_saved.toolBars) { + if (saved.first) saved.first->setHidden(saved.second); + } + m_window->menuBar()->setHidden(m_saved.menuBarHidden); + + m_saved = Saved(); + m_on = false; +} + +void +CompactLayout::disableTearOff(QMenu *menu) +{ + m_saved.tearOffs.push_back({ menu, menu->isTearOffEnabled() }); + menu->setTearOffEnabled(false); + for (QAction *action: menu->actions()) { + if (QMenu *submenu = action->menu()) disableTearOff(submenu); + } +} + +void +CompactLayout::moveTakeBoxIn() +{ + // A widget added to a toolbar is held by a QWidgetAction of the + // toolbar's making, and it can be in one toolbar at a time: the + // action is taken out of the one it is in and put in this one, and + // the widget goes with it + QComboBox *box = m_parts.takeBox; + if (!box) return; + QToolBar *home = qobject_cast(box->parentWidget()); + if (!home) return; + + QList actions = home->actions(); + for (int i = 0; i < actions.size(); ++i) { + QWidgetAction *action = qobject_cast(actions[i]); + if (!action || action->defaultWidget() != box) continue; + m_saved.takeBoxAction = action; + m_saved.takeBoxHome = home; + m_saved.takeBoxNext = (i + 1 < actions.size() ? actions[i+1] : nullptr); + home->removeAction(action); + m_toolBar->insertAction(m_takeBoxPlace, action); + return; + } +} + +void +CompactLayout::moveTakeBoxBack() +{ + QAction *action = m_saved.takeBoxAction; + QToolBar *home = m_saved.takeBoxHome; + if (!action || !home) return; + + m_toolBar->removeAction(action); + + // Where it was: before the action that followed it, if that is still + // there, else at the end + QAction *next = m_saved.takeBoxNext; + if (next && !home->actions().contains(next)) next = nullptr; + home->insertAction(next, action); +} + +void +CompactLayout::showPanel(bool shown) +{ + for (QToolBar *toolBar: m_parts.panel) { + if (toolBar) toolBar->setVisible(shown); + } +} diff --git a/main/CompactLayout.h b/main/CompactLayout.h new file mode 100644 index 00000000..6ddeece2 --- /dev/null +++ b/main/CompactLayout.h @@ -0,0 +1,157 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_COMPACT_LAYOUT_H +#define TONY_COMPACT_LAYOUT_H + +#include +#include +#include +#include + +#include + +class QAction; +class QComboBox; +class QMainWindow; +class QMenu; +class QToolBar; +class QToolButton; +class QWidget; + +/** + * The window laid out for a phone held in landscape: one toolbar of + * touch-sized buttons in place of the menu bar and every other toolbar. + * It is on from the start on Android; on the desktop the View menu + * switches it (getAction()), and so does --compact at start. + * + * The toolbar holds the window's own actions, handed over by MainWindow + * in Parts: a menu button whose popup holds every menu of the menu bar, + * so that nothing becomes unreachable (and the menus' shortcuts keep + * working, which they would not with only the hidden menu bar holding + * them); Play, Record and Record into Selection; the take box; Undo and + * Redo; Erase; Zoom In and Zoom Out; and a button that shows and hides + * the panel, the Show and Play and speed toolbars at the bottom. + * + * The take box is moved into the toolbar and back rather than copied: + * it is the same widget, so it cannot drift out of step with the takes. + * + * Hidden while compact: the note-editing tools and the audio device + * menus (Parts::hiddenActions), the overview (Parts::hiddenWidgets) and + * the tear-off handles of the menus, which a finger would catch. + * + * Switching off puts back exactly what was there when it was switched + * on: the menu bar, which toolbars were on show, the hidden actions and + * widgets, the tear-off handles and the take box in its place. The + * compact toolbar leaves the window's layout, and so its state. The + * tool mode is the one exception: switching on selects Navigate, as the + * note-editing tool that could be selected is hidden, and switching off + * leaves it so. + */ +class CompactLayout : public QObject +{ + Q_OBJECT + +public: + /// The window's own actions and widgets that compact mode uses + struct Parts { + QAction *play = nullptr; + QAction *record = nullptr; + QAction *recordIntoSelection = nullptr; + QComboBox *takeBox = nullptr; + QAction *erase = nullptr; + QAction *zoomIn = nullptr; + QAction *zoomOut = nullptr; + + /// Selected on switching on: every other tool is hidden + QAction *navigateTool = nullptr; + + /// The toolbars the Show and Play button shows and hides + QList panel; + + /// Hidden while compact (a submenu is hidden by its menuAction()) + QList hiddenActions; + QList hiddenWidgets; + }; + + explicit CompactLayout(QMainWindow *window); + virtual ~CompactLayout(); + + /// Before it is first switched on + void setParts(const Parts &parts); + + /// The checkable switch, for the View menu + QAction *getAction() const { return m_action; } + + bool isOn() const { return m_on; } + + /// The compact toolbar and two of its buttons: null until first on + QToolBar *getToolBar() const { return m_toolBar; } + QToolButton *getMenuButton() const { return m_menuButton; } + QAction *getPanelAction() const { return m_panelAction; } + + /// The size of the compact toolbar's icons, in logical pixels: on a + /// phone Qt makes those Android's dp, where the style's 24 is small + /// for a finger + static const int iconSize = 40; + + /// Whether the window starts compact: always on Android, elsewhere + /// when the command line says --compact + static bool isWantedAtStart(const QStringList &arguments); + +public slots: + void setOn(bool on); + +private slots: + void showPanel(bool shown); + +private: + void makeToolBar(); + void switchOn(); + void switchOff(); + void moveTakeBoxIn(); + void moveTakeBoxBack(); + void disableTearOff(QMenu *menu); + + QMainWindow *m_window; + Parts m_parts; + QAction *m_action; + bool m_on = false; + + QToolBar *m_toolBar = nullptr; + QMenu *m_menu = nullptr; + QToolButton *m_menuButton = nullptr; + QAction *m_panelAction = nullptr; + QAction *m_takeBoxPlace = nullptr; // the take box goes before this + + // What switching on changed, to put back. The flags say what it + // was: for a widget whether it was hidden, for an action whether it + // was visible, for a menu whether it could be torn off + struct Saved { + bool menuBarHidden = false; + QList, bool>> toolBars; + QList, bool>> widgets; + QList, bool>> actions; + QList, bool>> tearOffs; + + // The take box's action, its toolbar and the action after it + // there (null if it was the last) + QPointer takeBoxAction; + QPointer takeBoxHome; + QPointer takeBoxNext; + }; + Saved m_saved; +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index a9a30e6e..baef45aa 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -18,6 +18,7 @@ #include "MainWindow.h" #include "NetworkPermissionTester.h" #include "Analyser.h" +#include "CompactLayout.h" #include "LatencyUtils.h" #include "PaneUtils.h" #include "TakeEvents.h" @@ -139,6 +140,15 @@ MainWindow::MainWindow(AudioMode audioMode, m_realtimePitchTracker(nullptr), m_realtimePitchLayer(nullptr), m_overview(0), + m_compactLayout(nullptr), + m_playAction(nullptr), + m_recordAction(nullptr), + m_zoomInAction(nullptr), + m_zoomOutAction(nullptr), + m_navigateToolAction(nullptr), + m_noteEditToolAction(nullptr), + m_playbackControlsToolBar(nullptr), + m_showAndPlayToolBar(nullptr), m_showSingingPitch(nullptr), m_showSingingNotes(nullptr), m_playSingingAudio(nullptr), @@ -410,6 +420,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_takeTimer->setInterval(100); connect(m_takeTimer, SIGNAL(timeout()), this, SLOT(pollTakeProgress())); + // Before the menus: its switch is in the View menu + m_compactLayout = new CompactLayout(this); + setupMenus(); setupToolbars(); setupHelpMenu(); @@ -418,6 +431,8 @@ MainWindow::MainWindow(AudioMode audioMode, finaliseMenus(); + setupCompactLayout(); + connect(m_viewManager, SIGNAL(activity(QString)), m_activityLog, SLOT(activityHappened(QString))); connect(m_playSource, SIGNAL(activity(QString)), @@ -547,6 +562,9 @@ MainWindow::setupFileMenu() QMenu *menu = menuBar()->addMenu(tr("&File")); menu->setTearOffEnabled(true); QToolBar *toolbar = addToolBar(tr("File Toolbar")); + // The toolbars are named for QMainWindow::saveState(), which tells + // them apart by name (CompactLayout's tests compare what it saves) + toolbar->setObjectName("File Toolbar"); m_keyReference->setCategory(tr("File and Session Management")); @@ -680,6 +698,7 @@ MainWindow::setupEditMenu() tr("Double-click left button to select the region of time corresponding to a note")); QToolBar *toolbar = addToolBar(tr("Tools Toolbar")); + toolbar->setObjectName("Tools Toolbar"); CommandHistory::getInstance()->registerToolbar(toolbar); @@ -699,6 +718,7 @@ MainWindow::setupEditMenu() group->addAction(action); menu->addAction(action); m_keyReference->registerShortcut(action); + m_navigateToolAction = action; m_keyReference->setCategory (tr("Navigate Tool Mouse Actions")); @@ -722,6 +742,7 @@ MainWindow::setupEditMenu() group->addAction(action); menu->addAction(action); m_keyReference->registerShortcut(action); + m_noteEditToolAction = action; m_keyReference->setCategory (tr("Note Edit Tool Mouse Actions")); @@ -891,6 +912,9 @@ MainWindow::setupEditMenu() // Ctrl+Backspace, the obvious partner to the Backspace of Delete // Notes, is upstream Tony's Remove Pitches m_eraseSingingAction->setShortcut(tr("Ctrl+D")); + // What a toolbar button shows when there is no icon: only the compact + // toolbar has one for it + m_eraseSingingAction->setIconText(tr("Erase")); m_eraseSingingAction->setStatusTip (tr("Remove the recorded singing within the selected region, leaving silence")); m_keyReference->registerShortcut(m_eraseSingingAction); @@ -944,6 +968,7 @@ MainWindow::setupViewMenu() connect(this, SIGNAL(canZoom(bool)), action, SLOT(setEnabled(bool))); m_keyReference->registerShortcut(action); menu->addAction(action); + m_zoomInAction = action; action = new QAction(il.load("zoom-out"), tr("Zoom &Out"), this); @@ -953,6 +978,7 @@ MainWindow::setupViewMenu() connect(this, SIGNAL(canZoom(bool)), action, SLOT(setEnabled(bool))); m_keyReference->registerShortcut(action); menu->addAction(action); + m_zoomOutAction = action; action = new QAction(tr("Restore &Default Zoom"), this); action->setStatusTip(tr("Restore the zoom level to the default")); @@ -975,6 +1001,10 @@ MainWindow::setupViewMenu() action->setStatusTip(tr("Set the minimum and maximum frequencies in the visible display")); connect(action, SIGNAL(triggered()), this, SLOT(editDisplayExtents())); menu->addAction(action); + + menu->addSeparator(); + + menu->addAction(m_compactLayout->getAction()); } void @@ -1477,6 +1507,7 @@ MainWindow::setupToolbars() m_rightButtonPlaybackMenu = m_rightButtonMenu->addMenu(tr("Playback")); QToolBar *toolbar = addToolBar(tr("Playback Toolbar")); + toolbar->setObjectName("Playback Toolbar"); QAction *rwdStartAction = toolbar->addAction(il.load("rewind-start"), tr("Rewind to Start")); @@ -1503,6 +1534,7 @@ MainWindow::setupToolbars() connect(m_playSource, SIGNAL(playStatusChanged(bool)), playAction, SLOT(setChecked(bool))); connect(this, SIGNAL(canPlay(bool)), playAction, SLOT(setEnabled(bool))); + m_playAction = playAction; m_ffwdAction = toolbar->addAction(il.load("ffwd"), tr("Fast Forward")); @@ -1530,6 +1562,7 @@ MainWindow::setupToolbars() this, SLOT(analyseNow())); connect(this, SIGNAL(canRecord(bool)), recordAction, SLOT(setEnabled(bool))); + m_recordAction = recordAction; // The takes of the session, beside the recording controls: choosing // one shows it, with its audio, pitch track and notes (spec 5.3). @@ -1557,6 +1590,7 @@ MainWindow::setupToolbars() toolbar->addWidget(m_takeCombo); toolbar = addToolBar(tr("Play Mode Toolbar")); + toolbar->setObjectName("Play Mode Toolbar"); QAction *psAction = toolbar->addAction(il.load("playselection"), tr("Constrain Playback to Selection")); @@ -1705,13 +1739,17 @@ MainWindow::setupToolbars() m_rightButtonPlaybackMenu->addAction(normalAction); toolbar = new QToolBar(tr("Playback Controls")); + toolbar->setObjectName("Playback Controls"); addToolBar(Qt::BottomToolBarArea, toolbar); + m_playbackControlsToolBar = toolbar; toolbar->addWidget(m_playSpeed); toolbar->addWidget(m_fader); toolbar = addToolBar(tr("Show and Play")); + toolbar->setObjectName("Show and Play"); addToolBar(Qt::BottomToolBarArea, toolbar); + m_showAndPlayToolBar = toolbar; // "Reference:" label before the reference-track button group { @@ -2024,6 +2062,41 @@ MainWindow::setupToolbars() // QTimer::singleShot(500, this, SLOT(betaReleaseWarning())); } +void +MainWindow::setupCompactLayout() +{ + CompactLayout::Parts parts; + + parts.play = m_playAction; + parts.record = m_recordAction; + parts.recordIntoSelection = m_recordIntoSelection; + parts.takeBox = m_takeCombo; + parts.erase = m_eraseSingingAction; + parts.zoomIn = m_zoomInAction; + parts.zoomOut = m_zoomOutAction; + + parts.navigateTool = m_navigateToolAction; + + parts.panel = { m_playbackControlsToolBar, m_showAndPlayToolBar }; + + // Notes are edited on the desktop; a phone has no device to choose + parts.hiddenActions = { m_navigateToolAction, m_noteEditToolAction }; + for (QMenu *menu: { m_audioDeviceMenu, m_audioInputDeviceMenu }) { + if (menu) parts.hiddenActions.push_back(menu->menuAction()); + } + + // For room: the panes are what a phone's height is wanted for + parts.hiddenWidgets = { m_overview }; + + m_compactLayout->setParts(parts); +} + +void +MainWindow::setCompactLayout(bool on) +{ + m_compactLayout->setOn(on); +} + void MainWindow::moveOneNoteRight() diff --git a/main/MainWindow.h b/main/MainWindow.h index dd6b8ee8..2362b64b 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -34,6 +34,8 @@ class QTimer; class QComboBox; class QActionGroup; +class QToolBar; +class CompactLayout; namespace sv { class VersionTester; @@ -82,6 +84,11 @@ class MainWindow : public sv::MainWindowBase void toXml(QTextStream &out, bool asTemplate) override; FileOpenStatus openSession(sv::FileSource source) override; + // Switch the layout for a phone on or off (CompactLayout), as View > + // Compact Layout does: main() switches it on at start on Android and + // with --compact + void setCompactLayout(bool on); + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -313,6 +320,21 @@ protected slots: sv::Overview *m_overview; + // The layout for a phone: one toolbar of touch-sized buttons in place + // of the menu bar and the other toolbars. MainWindow only hands it + // the parts (setupCompactLayout()): the actions below, which are made + // with the menus and toolbars, and others that have members already + CompactLayout *m_compactLayout; + QAction *m_playAction; + QAction *m_recordAction; + QAction *m_zoomInAction; + QAction *m_zoomOutAction; + QAction *m_navigateToolAction; + QAction *m_noteEditToolAction; + QToolBar *m_playbackControlsToolBar; + QToolBar *m_showAndPlayToolBar; + void setupCompactLayout(); + // Actions/toolbar items for the singing track QAction *m_showSingingPitch; QAction *m_showSingingNotes; diff --git a/main/main.cpp b/main/main.cpp index b0d62d33..2d1a69bd 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -14,6 +14,7 @@ */ #include "MainWindow.h" +#include "CompactLayout.h" #include "system/System.h" #include "system/Init.h" @@ -320,7 +321,7 @@ main(int argc, char **argv) if (args.contains("--help") || args.contains("-h") || args.contains("-?")) { std::cerr << QApplication::tr( - "\nTony is a program for interactive note and pitch analysis and annotation.\n\nUsage:\n\n %1 [--no-audio] [--no-sonification] [--no-spectrogram] [ ...]\n\n --no-audio: Do not attempt to open an audio output device\n --no-sonification: Disable sonification of pitch tracks and notes and hide their toggles.\n --no-spectrogram: Disable spectrogram.\n : One or more Tony (.ton) and audio files may be provided.").arg(argv[0]).toStdString() << std::endl; + "\nTony is a program for interactive note and pitch analysis and annotation.\n\nUsage:\n\n %1 [--no-audio] [--no-sonification] [--no-spectrogram] [--compact] [ ...]\n\n --no-audio: Do not attempt to open an audio output device\n --no-sonification: Disable sonification of pitch tracks and notes and hide their toggles.\n --no-spectrogram: Disable spectrogram.\n --compact: Start with the layout for a phone: one toolbar of large buttons in place of the menus and toolbars.\n : One or more Tony (.ton) and audio files may be provided.").arg(argv[0]).toStdString() << std::endl; exit(2); } @@ -389,6 +390,10 @@ main(int argc, char **argv) QObject::connect(gui, SIGNAL(hideSplash()), splash, SLOT(hide())); } + // Before the window is shown, so that a phone never shows the + // desktop layout + gui->setCompactLayout(CompactLayout::isWantedAtStart(args)); + QScreen *screen = QApplication::primaryScreen(); QRect available = screen->availableGeometry(); diff --git a/main/test/TestCompactLayout.h b/main/test/TestCompactLayout.h new file mode 100644 index 00000000..0d6c45ca --- /dev/null +++ b/main/test/TestCompactLayout.h @@ -0,0 +1,688 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_COMPACT_LAYOUT_H +#define TEST_COMPACT_LAYOUT_H + +// Tier 5: the compact layout (CompactLayout) of the real MainWindow, +// switched on and off as the View menu does, on a window on show at +// about a phone's size in landscape. Whether a widget is visible is +// only known on a window on show. + +#include "TestSignals.h" + +#include "../MainWindow.h" +#include "../Analyser.h" +#include "../CompactLayout.h" + +#include "version.h" + +#include "view/Overview.h" +#include "view/ViewManager.h" +#include "data/fileio/WavFileWriter.h" +#include "transform/ModelTransformerFactory.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/** + * MainWindow without an audio device, with what the tests look at + */ +class CompactTestWindow : public MainWindow +{ +public: + CompactTestWindow() : MainWindow(AUDIO_PLAYBACK_AND_RECORD, true, false) { } + + CompactLayout *compact() { return m_compactLayout; } + QComboBox *takeBox() { return m_takeCombo; } + QWidget *overview() { return m_overview; } + SingingTakes *takes() { return m_takes; } + sv::ViewManager *viewManager() { return m_viewManager; } + Analyser *analyser() { return m_analyser; } + + // What the compact toolbar should hold, in order, but for the take + // box, the menu button and the panel button, and Undo and Redo + QList ownActions() { + return { m_playAction, m_recordAction, m_recordIntoSelection, + m_eraseSingingAction, m_zoomInAction, m_zoomOutAction }; + } + + // What it hides: the note-editing tools and the audio device menus + QList hiddenActions() { + return { m_navigateToolAction, m_noteEditToolAction, + m_audioDeviceMenu->menuAction(), + m_audioInputDeviceMenu->menuAction() }; + } + + QAction *navigateTool() { return m_navigateToolAction; } + QAction *noteEditTool() { return m_noteEditToolAction; } + QAction *eraseAction() { return m_eraseSingingAction; } + + void discardModifications() { m_documentModified = false; } + void doCloseSession() { discardModifications(); closeSession(); } + void doNewEmptyTake() { newEmptyTake(); } + +protected: + void createAudioIO() override { } +}; + +class TestCompactLayout : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + + QTemporaryDir m_dir; + CompactTestWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + + // Everything switching off must put back as it was + struct Layout { + QByteArray state; // QMainWindow::saveState() + QSize iconSize; + bool menuBarShown = false; + bool overviewShown = false; + // Of the toolbars in the window's layout, by name + QMap toolBarsShown; + QMap toolBarIconSizes; + QMap> toolBarActions; + // Of the actions compact mode hides + QList actionsVisible; + // Of the menus of the menu bar and their submenus, in order + QList tearOffs; + QWidget *takeBoxParent = nullptr; + bool takeBoxShown = false; + }; + + static void addTearOffs(QMenu *menu, QList &tearOffs) { + tearOffs.push_back(menu->isTearOffEnabled()); + for (QAction *action: menu->actions()) { + if (QMenu *submenu = action->menu()) addTearOffs(submenu, tearOffs); + } + } + + QList layoutToolBars() { + QList toolBars; + for (QToolBar *toolBar: m_window->findChildren + (QString(), Qt::FindDirectChildrenOnly)) { + if (m_window->toolBarArea(toolBar) != Qt::NoToolBarArea) { + toolBars.push_back(toolBar); + } + } + return toolBars; + } + + QToolBar *toolBar(QString name) { + return m_window->findChild + (name, Qt::FindDirectChildrenOnly); + } + + Layout layout() { + Layout layout; + layout.state = m_window->saveState(); + layout.iconSize = m_window->iconSize(); + layout.menuBarShown = m_window->menuBar()->isVisible(); + layout.overviewShown = m_window->overview()->isVisible(); + for (QToolBar *toolBar: layoutToolBars()) { + QString name = toolBar->objectName(); + layout.toolBarsShown[name] = toolBar->isVisible(); + layout.toolBarIconSizes[name] = toolBar->iconSize(); + layout.toolBarActions[name] = toolBar->actions(); + } + for (QAction *action: m_window->hiddenActions()) { + layout.actionsVisible.push_back(action->isVisible()); + } + for (QAction *action: m_window->menuBar()->actions()) { + if (QMenu *menu = action->menu()) addTearOffs(menu, layout.tearOffs); + } + layout.takeBoxParent = m_window->takeBox()->parentWidget(); + layout.takeBoxShown = m_window->takeBox()->isVisible(); + return layout; + } + + void verifySameLayout(const Layout &now, const Layout &before) { + QCOMPARE(now.menuBarShown, before.menuBarShown); + QCOMPARE(now.overviewShown, before.overviewShown); + QCOMPARE(now.toolBarsShown, before.toolBarsShown); + QCOMPARE(now.toolBarIconSizes, before.toolBarIconSizes); + QCOMPARE(now.iconSize, before.iconSize); + QVERIFY2(now.toolBarActions == before.toolBarActions, + "the toolbars do not hold the actions they held"); + QCOMPARE(now.actionsVisible, before.actionsVisible); + QCOMPARE(now.tearOffs, before.tearOffs); + QCOMPARE(now.takeBoxParent, before.takeBoxParent); + QCOMPARE(now.takeBoxShown, before.takeBoxShown); + QCOMPARE(now.state, before.state); + } + + // Let the window lay itself out again + void settle() { + QCoreApplication::sendPostedEvents(); + QTest::qWait(20); + } + + void switchCompact() { + m_window->compact()->getAction()->trigger(); + settle(); + } + + // The compact toolbar's action that holds the take box, if it has it + QWidgetAction *takeBoxAction(QToolBar *toolBar) { + for (QAction *action: toolBar->actions()) { + auto widgetAction = qobject_cast(action); + if (widgetAction && + widgetAction->defaultWidget() == m_window->takeBox()) { + return widgetAction; + } + } + return nullptr; + } + + static QString names(const QList &actions) { + QStringList names; + for (QAction *action: actions) { + if (qobject_cast(action)) names << "(widget)"; + else names << action->iconText(); + } + return names.join(", "); + } + + bool analysed() { + Analyser *a = m_window->analyser(); + return a && a->getLayer(Analyser::PitchTrack) && + a->getLayer(Analyser::Notes) && + a->getInitialAnalysisCompletion() >= 100 && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + } + + // At about a phone's size in landscape, once Qt has scaled for the + // screen's density + void openWindow() { + m_window = new CompactTestWindow; + m_window->resize(900, 400); + m_window->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_window)); + settle(); + } + + // Two seconds of reference, analysed: takes need a main model + void openReference() { + QString path = m_dir.filePath("reference.wav"); + if (!QFile::exists(path)) { + std::vector data = TestSignals::sine + (220.5, rate, int(2 * rate), 0.5); + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + QVERIFY(writer.isOK()); + QVERIFY(writer.writeSamples(&ptr, sv::sv_frame_t(data.size()))); + QVERIFY(writer.close()); + } + m_window->discardModifications(); + QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(), 30000); + } + + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + m_dialogs.push_back(description); + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + // Otherwise the MainWindow constructor asks, in a dialog + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + } + + void cleanup() { + if (m_window) { + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + } + + // From the View menu: the menu bar, every toolbar and the overview + // go, and one toolbar of large buttons comes, holding the window's + // own actions and its take box + void switching_on_leaves_one_toolbar() { + openWindow(); + if (QTest::currentTestFailed()) return; + + CompactLayout *compact = m_window->compact(); + QVERIFY(!compact->isOn()); + QVERIFY2(!compact->getToolBar(), "made before it was wanted"); + + QMenu *viewMenu = nullptr; + for (QAction *action: m_window->menuBar()->actions()) { + if (action->menu() && + action->menu()->actions().contains(compact->getAction())) { + viewMenu = action->menu(); + } + } + QVERIFY(viewMenu); + QCOMPARE(viewMenu->title(), QString("&View")); + + // The note-editing tool, which compact mode hides, selected + m_window->noteEditTool()->trigger(); + QCOMPARE(m_window->viewManager()->getToolMode(), + sv::ViewManager::NoteEditMode); + + switchCompact(); + QVERIFY(compact->isOn()); + QVERIFY(compact->getAction()->isChecked()); + + QToolBar *bar = compact->getToolBar(); + QVERIFY(bar); + QVERIFY(bar->isVisible()); + QCOMPARE(m_window->toolBarArea(bar), Qt::TopToolBarArea); + + QVERIFY(!m_window->menuBar()->isVisible()); + QVERIFY(!m_window->overview()->isVisible()); + QCOMPARE(int(layoutToolBars().size()), 7); // the window's six and this + for (QToolBar *toolBar: layoutToolBars()) { + if (toolBar == bar) continue; + QVERIFY2(!toolBar->isVisible(), qPrintable(toolBar->objectName())); + } + + // Menu, Play, Record, Record into Selection, the take box, Undo, + // Redo, Erase, Zoom In, Zoom Out, Show and Play. Undo and Redo are + // the ones of the window's own toolbar, with their menus + QToolBar *tools = toolBar("Tools Toolbar"); + QVERIFY(tools); + QVERIFY(compact->getMenuButton()); + QList own = m_window->ownActions(); + QList expected { + compact->getMenuButton()->defaultAction(), + own[0], own[1], own[2], + takeBoxAction(bar), + tools->actions()[0], tools->actions()[1], + own[3], own[4], own[5], + compact->getPanelAction() + }; + QVERIFY(!expected.contains(nullptr)); + QList actions; + for (QAction *action: bar->actions()) { + if (!action->isSeparator()) actions.push_back(action); + } + QVERIFY2(actions == expected, + qPrintable(QString("the toolbar holds %1, not %2") + .arg(names(actions)).arg(names(expected)))); + QVERIFY(tools->actions()[0]->menu()); + QCOMPARE(m_window->eraseAction()->iconText(), QString("Erase")); + + // Every button on show, large; the take box the window's own, on + // show and working + QCOMPARE(bar->iconSize(), QSize(CompactLayout::iconSize, + CompactLayout::iconSize)); + for (QAction *action: actions) { + QWidget *widget = bar->widgetForAction(action); + QVERIFY2(widget && widget->isVisible(), + qPrintable(action->iconText())); + // and as tall as the icons, those that show text too + if (auto button = qobject_cast(widget)) { + QCOMPARE(button->iconSize(), bar->iconSize()); + QVERIFY2(button->height() >= CompactLayout::iconSize, + qPrintable(QString("%1 is %2 high") + .arg(action->iconText()) + .arg(button->height()))); + } + } + QCOMPARE(bar->widgetForAction(takeBoxAction(bar)), + static_cast(m_window->takeBox())); + QCOMPARE(m_window->takeBox()->parentWidget(), + static_cast(bar)); + + // The note-editing tools and the audio device menus hidden, and + // the tool the one that edits nothing + for (QAction *action: m_window->hiddenActions()) { + QVERIFY2(!action->isVisible(), qPrintable(action->text())); + } + QVERIFY(m_window->navigateTool()->isChecked()); + QCOMPARE(m_window->viewManager()->getToolMode(), + sv::ViewManager::NavigateMode); + } + + // Every menu of the menu bar is in the menu button's popup, which a + // tap opens; the switch back is in there, and nothing can be torn off + void menu_button_holds_every_menu() { + openWindow(); + if (QTest::currentTestFailed()) return; + + CompactLayout *compact = m_window->compact(); + switchCompact(); + + QToolButton *button = compact->getMenuButton(); + QVERIFY(button); + QVERIFY(button->isVisible()); + QCOMPARE(button->popupMode(), QToolButton::InstantPopup); + QMenu *popup = button->defaultAction()->menu(); + QVERIFY(popup); + + QList menus = m_window->menuBar()->actions(); + QCOMPARE(int(menus.size()), 7); // File, Edit, View, Analysis, + // Takes, Playback, Help + QVERIFY2(popup->actions() == menus, + qPrintable(QString("the popup holds %1, the menu bar %2") + .arg(names(popup->actions())).arg(names(menus)))); + bool haveSwitch = false; + for (QAction *action: menus) { + QVERIFY(action->menu()); + QVERIFY(action->isVisible()); + QVERIFY2(!action->menu()->isTearOffEnabled(), + qPrintable(action->text())); + if (action->menu()->actions().contains(compact->getAction())) { + haveSwitch = true; + } + } + QVERIFY(haveSwitch); + + // A tap opens it (and a timer closes it again) + bool checked = false, shown = false; + QTimer timer; + timer.setSingleShot(true); + connect(&timer, &QTimer::timeout, this, [&]() { + shown = popup->isVisible(); + popup->close(); + checked = true; + }); + timer.start(200); + QTest::mouseClick(button, Qt::LeftButton); + QTRY_VERIFY_WITH_TIMEOUT(checked, 5000); + QVERIFY(shown); + } + + // The shortcuts of the menus work with the menu bar hidden: Select All + // is in the Edit menu and in no toolbar + void menu_shortcuts_work_while_compact() { + openWindow(); + if (QTest::currentTestFailed()) return; + openReference(); + if (QTest::currentTestFailed()) return; + + switchCompact(); + QVERIFY(!m_window->menuBar()->isVisible()); + + m_window->activateWindow(); + QVERIFY(QTest::qWaitForWindowActive(m_window)); + + QVERIFY(m_window->viewManager()->getSelections().empty()); + QTest::keyClick(m_window, Qt::Key_A, Qt::ControlModifier); + QCOMPARE(int(m_window->viewManager()->getSelections().size()), 1); + } + + // The Show and Play button brings the bottom toolbars, the toggles + // and gains and the playback speed, and takes them away again + void show_and_play_button_shows_and_hides_the_panel() { + openWindow(); + if (QTest::currentTestFailed()) return; + + CompactLayout *compact = m_window->compact(); + switchCompact(); + + QAction *panel = compact->getPanelAction(); + QVERIFY(panel); + QVERIFY(panel->isCheckable()); + QVERIFY(!panel->isChecked()); + auto button = qobject_cast + (compact->getToolBar()->widgetForAction(panel)); + QVERIFY(button); + QVERIFY(button->isVisible()); + + QStringList panelNames { "Playback Controls", "Show and Play" }; + QStringList otherNames { "File Toolbar", "Tools Toolbar", + "Playback Toolbar", "Play Mode Toolbar" }; + for (QString name: panelNames + otherNames) { + QVERIFY2(toolBar(name), qPrintable(name)); + QVERIFY2(!toolBar(name)->isVisible(), qPrintable(name)); + } + + QTest::mouseClick(button, Qt::LeftButton); + settle(); + QVERIFY(panel->isChecked()); + for (QString name: panelNames) { + QVERIFY2(toolBar(name)->isVisible(), qPrintable(name)); + } + // and nothing else with it + for (QString name: otherNames) { + QVERIFY2(!toolBar(name)->isVisible(), qPrintable(name)); + } + QVERIFY(!m_window->menuBar()->isVisible()); + QVERIFY(compact->getToolBar()->isVisible()); + + QTest::mouseClick(button, Qt::LeftButton); + settle(); + QVERIFY(!panel->isChecked()); + for (QString name: panelNames) { + QVERIFY2(!toolBar(name)->isVisible(), qPrintable(name)); + } + } + + // Switching off puts back what was there: here a layout of the user's + // own, with one bottom toolbar hidden, which the panel showed while + // compact + void switching_off_restores_the_layout() { + openWindow(); + if (QTest::currentTestFailed()) return; + + toolBar("Playback Controls")->hide(); + settle(); + Layout before = layout(); + QVERIFY(before.menuBarShown); + QVERIFY(before.overviewShown); + QVERIFY(!before.toolBarsShown["Playback Controls"]); + QVERIFY(before.toolBarsShown["Show and Play"]); + QCOMPARE(int(before.toolBarsShown.size()), 6); + + CompactLayout *compact = m_window->compact(); + switchCompact(); + compact->getPanelAction()->trigger(); + settle(); + QVERIFY(toolBar("Playback Controls")->isVisible()); + + switchCompact(); + QVERIFY(!compact->isOn()); + QVERIFY(!compact->getAction()->isChecked()); + QCOMPARE(m_window->toolBarArea(compact->getToolBar()), + Qt::NoToolBarArea); + verifySameLayout(layout(), before); + } + + // On, off, on and off: the same toolbar and popup the second time, + // nothing added twice, and the layout of the start at the end + void switching_twice_leaves_nothing_duplicated() { + openWindow(); + if (QTest::currentTestFailed()) return; + + Layout before = layout(); + int toolBars = int(m_window->findChildren().size()); + int menus = int(m_window->findChildren().size()); + int boxes = int(m_window->findChildren().size()); + + CompactLayout *compact = m_window->compact(); + switchCompact(); + QToolBar *bar = compact->getToolBar(); + QList actions = bar->actions(); + QList popup = compact->getMenuButton()->defaultAction() + ->menu()->actions(); + int buttons = int(bar->findChildren().size()); + + switchCompact(); + switchCompact(); + QCOMPARE(compact->getToolBar(), bar); + QVERIFY2(bar->actions() == actions, + qPrintable(QString("the toolbar holds %1, not %2") + .arg(names(bar->actions())).arg(names(actions)))); + QVERIFY(compact->getMenuButton()->defaultAction()->menu()->actions() + == popup); + QCOMPARE(int(bar->findChildren().size()), buttons); + + switchCompact(); + verifySameLayout(layout(), before); + if (QTest::currentTestFailed()) return; + + // The compact toolbar and its popup are kept for the next time, + // and that is all there is more of + QCOMPARE(int(m_window->findChildren().size()), + toolBars + 1); + QCOMPARE(int(m_window->findChildren().size()), menus + 1); + QCOMPARE(int(m_window->findChildren().size()), boxes); + } + + // The take box in the compact toolbar is the window's own: choosing a + // take there switches to it, as it does in the desktop layout + void take_box_chooses_the_take_while_compact() { + openWindow(); + if (QTest::currentTestFailed()) return; + openReference(); + if (QTest::currentTestFailed()) return; + + m_window->doNewEmptyTake(); + m_window->doNewEmptyTake(); + QCOMPARE(m_window->takes()->getTakeNames(), + QStringList({ "Take 1", "Take 2" })); + QCOMPARE(m_window->takes()->getActiveIndex(), 1); + auto home = qobject_cast + (m_window->takeBox()->parentWidget()); + QVERIFY(home); + + switchCompact(); + QToolBar *bar = m_window->compact()->getToolBar(); + QVERIFY(takeBoxAction(bar)); + QComboBox *box = qobject_cast + (bar->widgetForAction(takeBoxAction(bar))); + QVERIFY(box); + QVERIFY(box->isVisible()); + QVERIFY(box->isEnabled()); + QCOMPARE(box->count(), 2); + QCOMPARE(box->currentText(), QString("Take 2")); + + // As from a keyboard: the take above. Page Up, as Up is Zoom In's + QTest::keyClick(box, Qt::Key_PageUp); + QCOMPARE(m_window->takes()->getActiveIndex(), 0); + QCOMPARE(box->currentText(), QString("Take 1")); + + // and back in its own toolbar the box says so + switchCompact(); + QCOMPARE(box->parentWidget(), static_cast(home)); + QVERIFY(box->isVisible()); + QCOMPARE(box->currentText(), QString("Take 1")); + } + + // main() switches on at start with --compact, and always on Android: + // before the window is first shown + void compact_at_start() { +#ifdef Q_OS_ANDROID + QVERIFY(CompactLayout::isWantedAtStart({ "Tony" })); +#else + QVERIFY(CompactLayout::isWantedAtStart({ "Tony", "--compact" })); + QVERIFY(CompactLayout::isWantedAtStart + ({ "Tony", "--no-audio", "--compact", "song.wav" })); + QVERIFY(!CompactLayout::isWantedAtStart({ "Tony" })); + QVERIFY(!CompactLayout::isWantedAtStart + ({ "Tony", "--no-audio", "song.wav" })); +#endif + + m_window = new CompactTestWindow; + m_window->resize(900, 400); + m_window->setCompactLayout(true); + QVERIFY(m_window->compact()->isOn()); + QVERIFY(m_window->compact()->getAction()->isChecked()); + + m_window->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_window)); + settle(); + + QToolBar *bar = m_window->compact()->getToolBar(); + QVERIFY(bar->isVisible()); + QVERIFY(m_window->takeBox()->isVisible()); + QVERIFY(!m_window->menuBar()->isVisible()); + QVERIFY(!m_window->overview()->isVisible()); + for (QToolBar *toolBar: layoutToolBars()) { + if (toolBar == bar) continue; + QVERIFY2(!toolBar->isVisible(), qPrintable(toolBar->objectName())); + } + + // Off: as a window that was never compact has it + m_window->setCompactLayout(false); + settle(); + QVERIFY(!m_window->compact()->getAction()->isChecked()); + QVERIFY(m_window->menuBar()->isVisible()); + QVERIFY(m_window->overview()->isVisible()); + QCOMPARE(int(layoutToolBars().size()), 6); + for (QToolBar *toolBar: layoutToolBars()) { + QVERIFY2(toolBar->isVisible(), qPrintable(toolBar->objectName())); + } + QVERIFY(m_window->takeBox()->isVisible()); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 79725856..8bd8e2b9 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -15,6 +15,7 @@ #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" #include "TestTouchGestures.h" +#include "TestCompactLayout.h" #include "RunSuite.h" @@ -77,6 +78,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestCompactLayout t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index afe1f3e6..2ba1c5d2 100644 --- a/meson.build +++ b/meson.build @@ -1178,6 +1178,7 @@ tony_core_files = [ tony_app_files = [ 'main/AlternatePitchTrack.cpp', + 'main/CompactLayout.cpp', 'main/CoverageStrip.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', @@ -1199,6 +1200,7 @@ tony_app_moc_files = qt.preprocess( 'main/MainWindow.h', 'main/Analyser.h', 'main/AlternatePitchTrack.h', + 'main/CompactLayout.h', 'main/CoverageStrip.h', 'main/TouchGestures.h', ]) @@ -1507,12 +1509,16 @@ if system != 'android' 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', 'main/test/TestTouchGestures.h', + 'main/test/TestCompactLayout.h', ]) tony_app_test_exe = executable( 'test-tony-app', tony_app_test_moc_files, 'main/test/tony-app-test.cpp', + # The icons: without them every toolbar button shows its text, and + # is as wide as that (TestCompactLayout needs the buttons' real size) + qt_resource_files, dependencies: [ tony_app_dep, tony_core_dep, From d3e0ac18459da127936f092a83b1d2709d664e65 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:26:48 +0000 Subject: [PATCH 130/275] docs: phase a5 done, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index c8bba239..7c1d46ed 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -157,7 +157,7 @@ builds happen in the container.) was blocked (`dl.google.com` refused); run `deploy/android/build-apk.sh` once it is allowed (see the log). - A4 — Touch gestures on the panes. Done. -- A5 — Compact touch mode. +- A5 — Compact touch mode. Done. - A6 — Oboe audio backend. - A7 — Android files, permission and lifecycle. - A8 — Documentation pass. @@ -466,3 +466,28 @@ Choices / deviations: The next phase must know: touch tests need the window shown and a fresh device per test. Left open: not tried on a touch screen. Windows desktop touch (OS-made mouse events, its own press-and-hold right click) untested. For A8: architecture.md, testing.md, manual-checklist. + +### Phase A5 — 2026-09-26 +Built: `main/CompactLayout` (tony_app): one toolbar in place of the menu bar, every other +toolbar and the overview. View > Compact Layout switches it; main() switches it on before the +window is shown on Android and with `--compact` (`isWantedAtStart()`). Buttons: Menu (a popup +holding the menu bar's own menus), Play, Record, Record into Selection, the take box, Undo, +Redo (`CommandHistory::registerToolbar()`), Erase, Zoom In, Zoom Out, Show and Play (shows +the two bottom toolbars). 40 px icons; the text-only buttons as tall. Hidden while compact: +the Navigate and Edit tools (Navigate is selected first), the two audio device submenus, the +menus' tear-off handles. `MainWindow`: members for what were locals, `setupCompactLayout()`, +`setCompactLayout()`; toolbars named (saveState); Erase's iconText "Erase". `TestCompactLayout`. +Choices / deviations: +- The take box is moved (its QWidgetAction, back before the action that followed it), not + copied: one widget, nothing to keep in step. +- A shortcut needs a visible widget holding its action or its menu: with only the hidden menu + bar holding the menus, every menu-only shortcut dies. The popup's button carries them. +- Kept: the status bar (pre-roll countdown, sung note). No property stacks to hide (Tony uses + `NoPropertyStacks`). The spectrogram stays a panel toggle; the Edit and long-press menus + keep their pitch and note actions (only the tool modes go). Not saved in the settings. +- `test-tony-app` now links `tony.qrc`: without icons every button is its text, and the bar + was 1023 px wide (703 with icons); the other app suites pass with the icons too. +The next phase must know: switching off restores what switching on saved; the tool mode stays. +Left open: not seen on a phone: the take box's height (22 px in Fusion), popup menu rows, and +under ~710 dp wide the last buttons go into the toolbar's extension. For A8: architecture.md, +testing.md (icons in the app tests), mobile-port.md, manual-checklist.md. From d775cc6d21cb12e0776ad73729e92c84a21635a6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:28:12 +0000 Subject: [PATCH 131/275] docs: the work order for phase a6, the oboe audio backend Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 7c1d46ed..3ba8ba3e 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -292,7 +292,25 @@ Read: [mobile-port.md](mobile-port.md) "The window", "Work common to both ports" ### A6 — Oboe audio backend -After the user's phone test of A3b. Detailed when it starts. +Read: [port-android.md](port-android.md) "Audio", "Permissions and lifecycle"; +[mobile-port.md](mobile-port.md) "Audio I/O", "Sample rate"; [recording.md](recording.md) +all of it; the A1 log entry. + +- Oboe (github.com/google/oboe, a pinned 1.x release) built as a static library into the + A2 prefix by `deploy/android/build-deps.sh`, with whatever meson needs to find it. +- `OboeAudioIO` in `main/`, compiled for Android only: a `breakfastquay::SystemAudioIO` + as `PortAudioIO` (bqaudioio) and the tests' `FakeAudioIO` are. One full-duplex callback + (`oboe::FullDuplexStream`), input handed over before output is asked for, low-latency + performance mode, the device's native rate (A1 made Tony handle any rate), latencies + reported through `setSystemRecordLatency()` / `setSystemPlaybackLatency()` from Oboe's + estimates, `suppressRecordSide()` honoured, playback-only when there is no input. +- `MainWindow::createAudioIO()` overridden on Android to install it, as the tests install + `FakeAudioIO`; `AUDIO_NONE` on Android removed. +- The microphone permission (`QMicrophonePermission`) asked for before input is opened; + without it, playback only, and the user told why. +- A device change (headphones in or out) reopens the streams rather than going silent. +- Pure arithmetic (latency from timestamps and the like) in `tony_core` with core tests; + the rest can only be judged on the phone. ### A7 — Android files, permission and lifecycle From 2cf1698f18cc302c3642092b3e229959dc7769b5 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 06:29:06 +0300 Subject: [PATCH 132/275] test: the throttle's stream is paced by the clock a_stream_is_told_once_an_interval waited qWait(2) between changes and required more than 100 of them over ten intervals. On Windows a wait that short lasts a timer tick of about 15 ms, so the stream had some 30 changes and the test failed before it looked at the throttle. Each change is now 2 ms after the last by the clock, with events processed meanwhile so that the throttle's timer runs. Co-Authored-By: Claude Opus 5.5 --- main/test/TestModelChangeThrottle.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/main/test/TestModelChangeThrottle.h b/main/test/TestModelChangeThrottle.h index 355bad6f..5a51a2d2 100644 --- a/main/test/TestModelChangeThrottle.h +++ b/main/test/TestModelChangeThrottle.h @@ -104,7 +104,14 @@ private slots: while (timer.elapsed() < kInterval * 10) { throttle.changed(changes * 256, changes * 256 + 256); ++changes; - QTest::qWait(2); + // A change every 2 ms by the clock, the throttle's timer + // running meanwhile. Not qWait(2): on Windows a wait that + // short lasts a timer tick of about 15 ms + QElapsedTimer step; + step.start(); + while (step.nsecsElapsed() < 2000000) { + QCoreApplication::processEvents(); + } } QVERIFY(changes > 100); QVERIFY2(m_told.size() >= 5 && m_told.size() <= 13, From 92b8b5f122a6653061c4f13c18067ee34eb3240e Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 06:29:06 +0300 Subject: [PATCH 133/275] test: a take's snapshot waits for its audio to be read snapshotTake() read the frame count of the take's audio model at once, and the model of a file just opened -- the one an erase swaps in -- says 0 frames until it has read the file. Under load the snapshot after the erase took 0, and undo_redo_an_erase failed when the redo brought back the real length (seen in one of two full runs on Windows, and in 3 of 10 runs under load). It now waits for the model to be ready, as verifyTakeMatches() already waited: 15 of 15 runs under load passed. Co-Authored-By: Claude Opus 5.5 --- main/test/TestRecordWorkflow.h | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index e067beeb..b2ddb64f 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -68,6 +68,7 @@ #include #include #include +#include #include #include #include @@ -630,7 +631,17 @@ class TestRecordWorkflow : public QObject s.pitch = pitchEvents(a2); s.notes = a2 ? noteEvents(a2->getLayer(Analyser::Notes)) : sv::EventVector(); - if (takeAudio()) s.frames = takeAudio()->getFrameCount(); + // Waited for, as verifyTakeMatches() waits: the model of a file + // just opened (as after an erase) says 0 frames until it has read + // the file, and under load that can outlast the call + if (auto audio = takeAudio()) { + QElapsedTimer waited; + waited.start(); + while (!audio->isReady() && waited.elapsed() < 30000) { + QTest::qWait(10); + } + s.frames = audio->getFrameCount(); + } return s; } From 40037cfb6c5643d7448c8abe671ee98b3cb29a4a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:46:55 +0000 Subject: [PATCH 134/275] feat: change, add and delete lyrics words With Edit Lyrics on, a double-click on a word asks for its new text, and a right-click in the row of words gives Edit Word Text... and Delete Word on a word, Add Word... between words. A new word goes where the click was, half a second long or up to the next word, and joins the line of the nearer neighbour. Typed text is cleaned as the parsers clean it; empty text changes nothing. Each is one step to undo. Elsewhere, a right-click still gives Tony's own menu. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- main/LyricsEditor.cpp | 251 +++++++++++- main/LyricsEditor.h | 95 ++++- main/MainWindow.cpp | 18 +- main/MainWindow.h | 14 +- main/test/TestRecordWorkflow.h | 688 ++++++++++++++++++++++++++++++++- 5 files changed, 1045 insertions(+), 21 deletions(-) diff --git a/main/LyricsEditor.cpp b/main/LyricsEditor.cpp index 1d350ca7..5d38a225 100644 --- a/main/LyricsEditor.cpp +++ b/main/LyricsEditor.cpp @@ -22,11 +22,30 @@ #include "data/model/EventCommands.h" #include "widgets/CommandHistory.h" +#include #include +#include #include +#include + using namespace sv; +namespace { + +// An edit that is done already, as the command was filled, goes on the +// history as it is. CommandHistory marks the session modified +void +pushDone(ChangeEventsCommand *command) +{ + command = command->finish(); + if (command) { + CommandHistory::getInstance()->addCommand(command, false); + } +} + +} + LyricsEditor::LyricsEditor(LyricsTrack *lyrics, QObject *parent) : QObject(parent), m_lyrics(lyrics), @@ -93,9 +112,21 @@ LyricsEditor::dragModel() const return ModelById::getAs(m_dragModel); } +std::shared_ptr +LyricsEditor::editedModel(ModelId id) const +{ + // Edit mode gone off, or other lyrics than the ones the edit began + // in: an import or a new session in the meantime + RegionLayer *layer = currentLayer(); + if (!m_enabled || !layer || id.isNone() || layer->getModel() != id) { + return {}; + } + return ModelById::getAs(id); +} + bool -LyricsEditor::hitAt(QPoint pos, LyricsEdit::Hit &hit, - EventVector &words) const +LyricsEditor::wordsAt(QPoint pos, EventVector &words, + LyricsEdit::Boxes &boxes) const { // Hidden lyrics keep the box row they were last painted with, which // is not there to be clicked @@ -112,12 +143,194 @@ LyricsEditor::hitAt(QPoint pos, LyricsEdit::Hit &hit, words = model->getAllEvents(); Pane *pane = m_pane; - LyricsEdit::Boxes boxes = LyricsEdit::boxesFor + boxes = LyricsEdit::boxesFor (words, [pane](sv_frame_t frame) { return pane->getXForFrame(frame); }); + return true; +} + +bool +LyricsEditor::hitAt(QPoint pos, LyricsEdit::Hit &hit, + EventVector &words) const +{ + LyricsEdit::Boxes boxes; + if (!wordsAt(pos, words, boxes)) return false; hit = LyricsEdit::hitTest(boxes, pos.x(), grabPixels); return true; } +std::vector +LyricsEditor::menuEntriesAt(QPoint pos) const +{ + std::vector entries; + if (!m_enabled || m_dragging) return entries; + + EventVector words; + LyricsEdit::Boxes boxes; + if (!wordsAt(pos, words, boxes)) return entries; + + ModelId modelId = currentLayer()->getModel(); + auto model = ModelById::getAs(modelId); + if (!model) return entries; + + // On a word, anywhere in its box, near its edges too: the menu is for + // the word the pointer is on + int word = LyricsEdit::wordAt(boxes, pos.x()); + if (word >= 0) { + MenuEntry edit; + edit.text = tr("Edit Word Text..."); + edit.enabled = true; + edit.operation = MenuEntry::Operation::EditText; + edit.model = modelId; + edit.word = words[word]; + entries.push_back(edit); + + MenuEntry remove = edit; + remove.text = tr("Delete Word"); + remove.operation = MenuEntry::Operation::DeleteWord; + entries.push_back(remove); + return entries; + } + + // Between words, at the last frame the pointer's column shows. Its + // first can be inside the word before: that word's end is somewhere + // in the column just after its box + sv_frame_t frame = std::max(m_pane->getFrameForX(pos.x()), + m_pane->getFrameForX(pos.x() + 1) - 1); + LyricsEdit::Span span; + + MenuEntry add; + add.text = tr("Add Word..."); + add.enabled = LyricsEdit::newWordSpan(words, frame, + model->getSampleRate(), span); + add.operation = MenuEntry::Operation::AddWord; + add.model = modelId; + add.frame = frame; + entries.push_back(add); + return entries; +} + +void +LyricsEditor::choose(const MenuEntry &entry) +{ + if (!entry.enabled || m_dragging) return; + + switch (entry.operation) { + case MenuEntry::Operation::EditText: + editText(entry.model, entry.word); + break; + case MenuEntry::Operation::DeleteWord: + deleteWord(entry.model, entry.word); + break; + case MenuEntry::Operation::AddWord: + addWord(entry.model, entry.frame); + break; + } +} + +void +LyricsEditor::editText(ModelId modelId, Event word) +{ + if (!m_askText) return; + { + auto model = editedModel(modelId); + if (!model || !model->containsEvent(word)) return; + } + + QString text = word.getLabel(); + if (!m_askText(text, false)) return; + + // The question had an event loop of its own, and the lyrics may have + // gone, or the word been changed, meanwhile: then this edit is of + // something that is not there + auto model = editedModel(modelId); + if (!model || !model->containsEvent(word)) return; + + // Nothing left of the text is refused, and the word stays as it was: + // Delete Word is for deleting it + QString cleaned = LyricsEdit::cleanText(text); + if (cleaned == "" || cleaned == word.getLabel()) return; + + auto command = new ChangeEventsCommand + (modelId.untyped, tr("Change Word Text")); + command->remove(word); + command->add(word.withLabel(cleaned)); + pushDone(command); +} + +void +LyricsEditor::deleteWord(ModelId modelId, Event word) +{ + auto model = editedModel(modelId); + if (!model || !model->containsEvent(word)) return; + + auto command = new ChangeEventsCommand + (modelId.untyped, tr("Delete Word")); + command->remove(word); + pushDone(command); +} + +void +LyricsEditor::addWord(ModelId modelId, sv_frame_t frame) +{ + if (!m_askText) return; + { + auto model = editedModel(modelId); + LyricsEdit::Span span; + if (!model || + !LyricsEdit::newWordSpan(model->getAllEvents(), frame, + model->getSampleRate(), span)) { + return; + } + } + + QString text; + if (!m_askText(text, true)) return; + QString cleaned = LyricsEdit::cleanText(text); + if (cleaned == "") return; + + // Where the word goes, and the line it joins, from the words as they + // are after the question, which may not be as they were before it + auto model = editedModel(modelId); + if (!model) return; + EventVector words = model->getAllEvents(); + LyricsEdit::Span span; + if (!LyricsEdit::newWordSpan(words, frame, model->getSampleRate(), span)) { + return; + } + float line = LyricsEdit::newWordLine(words, span); + + // The layer draws the word bold if it is the first of its line now + auto command = new ChangeEventsCommand + (modelId.untyped, tr("Add Word")); + command->add(Event(span.start, line, span.duration(), cleaned)); + pushDone(command); +} + +bool +LyricsEditor::showMenu(QPoint pos) +{ + std::vector entries = menuEntriesAt(pos); + if (entries.empty()) return false; + + // As the pane does for its own menu + clearHelp(); + + // Shown and left to run by itself, as Tony's own menu is: the entry + // chosen is done from the menu's event handling, and its question + // is not asked from inside the pane's. Deleted once it is hidden, + // after the entry chosen is done + QMenu *menu = new QMenu(m_pane); + connect(menu, &QMenu::aboutToHide, menu, &QObject::deleteLater); + for (const MenuEntry &entry : entries) { + QAction *action = menu->addAction(entry.text); + action->setEnabled(entry.enabled); + connect(action, &QAction::triggered, this, + [this, entry]() { choose(entry); }); + } + menu->popup(m_pane->mapToGlobal(pos)); + return true; +} + bool LyricsEditor::eventFilter(QObject *object, QEvent *event) { @@ -154,16 +367,36 @@ LyricsEditor::mousePressed(QMouseEvent *e) // menu with our button still held if (m_dragging) return true; + QPoint pos = e->position().toPoint(); + + // A right press in the row is for the words' menu, and never reaches + // the pane, which would open its own as well. Outside the row it is + // the pane's, and so is its menu + if (e->button() == Qt::RightButton) return showMenu(pos); + if (e->button() != Qt::LeftButton) return false; - QPoint pos = e->position().toPoint(); LyricsEdit::Hit hit; EventVector words; - if (!hitAt(pos, hit, words) || !hit.isEdge()) return false; + if (!hitAt(pos, hit, words)) return false; RegionLayer *layer = currentLayer(); if (!layer || layer->getModel().isNone()) return false; + if (!hit.isEdge()) { + + // A double-click in a word, away from its edges, is for its + // text. The pane has had the first click of it, and moves the + // playback cursor there as for any click: it never sees the + // second, which would have called that off + if (e->type() == QEvent::MouseButtonDblClick && + hit.part == LyricsEdit::Part::Inside) { + editText(layer->getModel(), words[hit.word]); + return true; + } + return false; + } + m_dragging = true; m_dragModel = layer->getModel(); m_dragWords = words; @@ -240,9 +473,15 @@ LyricsEditor::hover(QPoint pos) } else { showHelp(tr("Drag to move the end of \"%1\"").arg(word)); } + } else if (hit.part == LyricsEdit::Part::Inside) { + restoreCursor(); + showHelp(tr("Double-click to change the text of \"%1\", " + "right-click to delete it") + .arg(words[hit.word].getLabel())); } else { restoreCursor(); - showHelp(tr("Drag a word's start or end to move it")); + showHelp(tr("Right-click to add a word, " + "drag a word's start or end to move it")); } // The row is ours while edit mode is on: the pane would only put its diff --git a/main/LyricsEditor.h b/main/LyricsEditor.h index fc5bba09..4585ceef 100644 --- a/main/LyricsEditor.h +++ b/main/LyricsEditor.h @@ -27,7 +27,9 @@ #include #include +#include #include +#include class QMouseEvent; class LyricsTrack; @@ -42,25 +44,32 @@ class ChangeEventsCommand; /** * Edit mode for the lyrics (Edit > Edit Lyrics): the mouse in the box * row of the lyrics, along the bottom of their pane, moves a word's - * start or end. + * start or end, changes its text on a double-click, and on a right + * click offers a small menu to change or delete the word there, or to + * add one in the space between words. * * The lyrics layer is never the pane's top layer, so the pane's tools * never reach it: this watches the pane's mouse events through an * event filter, which is on only while edit mode is. Only what it acts * on is kept from the pane: a left press on an edge and the drag it - * starts, and moves with no button held in the box row, where the - * cursor and the context help are this one's. Everything else, a - * click anywhere to move the playback cursor included, goes to the pane - * as it would without edit mode. + * starts, a double-click on a word, a right press anywhere in the box + * row, and moves with no button held in the box row, where the cursor + * and the context help are this one's. Everything else, a click + * anywhere to move the playback cursor included, goes to the pane as it + * would without edit mode. * * What an edit may do is LyricsEdit's to say; this does as it says. * A drag edits the model as it goes, so the words move under the * pointer, and is one command on the undo history when the button is - * let go, or nothing at all if the word is where it was. + * let go, or nothing at all if the word is where it was. A change of + * text, an added word and a deleted one are a command each. * * The layer and the model are found through LyricsTrack at every * event, and nothing of them is kept between drags: an import, a - * remove or another session can replace them at any time. + * remove or another session can replace them at any time. For the + * same reason a menu entry holds the word as a value, and the word is + * looked for again when the entry is chosen and again once its text + * has been asked for. * * Like the lyrics track itself, MainWindow owns it and only wires it. */ @@ -87,6 +96,53 @@ class LyricsEditor : public QObject /// How near an edge, in logical pixels on either side, grabs it static constexpr int grabPixels = 6; + /** + * How a word's text is asked for: given the text the word has ("" for + * a new word), true with the text the user gave in its place, false + * if they cancelled. MainWindow asks with a dialog, whose event loop + * runs while the question is open. With none set, no text is changed + * and no word added. + */ + typedef std::function TextQuestion; + void setTextQuestion(TextQuestion question) { m_askText = question; } + + /** + * One entry of the menu a right press in the box row opens. Values + * only: the word as it was when the menu was made, and the model it + * was in, which choose() looks for again. + */ + struct MenuEntry { + enum class Operation { EditText, DeleteWord, AddWord }; + + QString text; + bool enabled = false; + Operation operation = Operation::EditText; + sv::ModelId model; + + /// EditText, DeleteWord: the word the pointer was on + sv::Event word; + + /// AddWord: the frame the new word goes at + sv::sv_frame_t frame = 0; + }; + + /** + * The entries of the menu for a right press at this point of the + * pane, in the order shown: on a word "Edit Word Text..." and "Delete + * Word", elsewhere in the box row "Add Word...", disabled where + * there is no room for a word. None if the point is not in the box + * row, or edit mode is off: the press is then the pane's. + */ + std::vector menuEntriesAt(QPoint pos) const; + + /** + * Do what an entry says, as choosing it in the menu does: nothing + * if it is disabled, if edit mode is off, or if the lyrics have + * changed so that its word is not there as it was. The text is + * asked for first, where the entry needs one. + */ + void choose(const MenuEntry &entry); + signals: /// What the mouse does where the pointer is, "" when that is over void contextHelpChanged(const QString &); @@ -127,11 +183,32 @@ class LyricsEditor : public QObject sv::RegionLayer *currentLayer() const; std::shared_ptr dragModel() const; - // What the pointer is on, if it is in the box row: false if it is - // not, or there are no lyrics on show. The words are the model's now + TextQuestion m_askText; + + // The words and their boxes, if the point is in the box row: false + // if it is not, or there are no lyrics on show. The words are the + // model's now + bool wordsAt(QPoint pos, sv::EventVector &words, + LyricsEdit::Boxes &boxes) const; + + // What the pointer is on, if it is in the box row, as wordsAt() bool hitAt(QPoint pos, LyricsEdit::Hit &hit, sv::EventVector &words) const; + // The lyrics' model, if it is this one and is still being edited + std::shared_ptr editedModel(sv::ModelId) const; + + // The three edits. Each looks for its word in the model again, asks + // for a text where it needs one, and looks again after the question, + // whose event loop may have let anything happen + void editText(sv::ModelId model, sv::Event word); + void deleteWord(sv::ModelId model, sv::Event word); + void addWord(sv::ModelId model, sv::sv_frame_t frame); + + // The menu of menuEntriesAt(), shown at this point. False if there + // is none there + bool showMenu(QPoint pos); + bool mousePressed(QMouseEvent *); bool mouseMoved(QMouseEvent *); bool mouseReleased(QMouseEvent *); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4a15339d..fe8dbeda 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -412,6 +412,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_lyricsEditor = new LyricsEditor(m_lyrics, this); connect(m_lyricsEditor, &LyricsEditor::contextHelpChanged, this, &MainWindow::contextHelpChanged); + m_lyricsEditor->setTextQuestion([this](QString &text, bool isNew) { + return askForLyricsWordText(text, isNew); + }); // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end @@ -937,7 +940,7 @@ MainWindow::setupEditMenu() // No shortcut: it is not switched on and off in the middle of things m_editLyricsAction = new QAction(tr("Edit L&yrics"), this); m_editLyricsAction->setCheckable(true); - m_editLyricsAction->setStatusTip(tr("Drag the start or end of a word of the lyrics, along the bottom of the pane, to move it")); + m_editLyricsAction->setStatusTip(tr("Edit the words of the lyrics along the bottom of the pane: drag a start or end, double-click a word to change its text, right-click to add or delete one")); m_editLyricsAction->setEnabled(false); connect(m_editLyricsAction, &QAction::triggered, this, &MainWindow::editLyricsToggled); @@ -3554,6 +3557,19 @@ MainWindow::askForLyricsExportFile(QString suggested) tr("TTML lyrics (*.ttml)") + ";;" + tr("All files (*)")); } +bool +MainWindow::askForLyricsWordText(QString &text, bool isNew) +{ + bool ok = false; + QString typed = QInputDialog::getText + (this, isNew ? tr("Add Word") : tr("Edit Word Text"), + isNew ? tr("Text of the new word:") : tr("Text of the word:"), + QLineEdit::Normal, text, &ok); + if (!ok) return false; + text = typed; + return true; +} + void MainWindow::exportLyrics() { diff --git a/main/MainWindow.h b/main/MainWindow.h index 63ea6728..dcce73e2 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -377,10 +377,10 @@ protected slots: QAction *m_showLyrics; // Edit > Edit Lyrics: the mouse moves the words' starts and ends in - // the lyrics' box row while it is on. Off, and not to be had, - // without lyrics on show or while a take is being recorded, which - // updateMenuStates() sees to; and off after an import, which - // importLyricsFrom() sees to + // the lyrics' box row while it is on, and changes, adds and deletes + // words there. Off, and not to be had, without lyrics on show or + // while a take is being recorded, which updateMenuStates() sees to; + // and off after an import, which importLyricsFrom() sees to LyricsEditor *m_lyricsEditor; QAction *m_editLyricsAction; @@ -426,6 +426,12 @@ protected slots: // "" if the user cancelled. Overridden by the tests virtual QString askForLyricsExportFile(QString suggested); + // Ask for the text of a word of the lyrics, offering the one it has + // ("" for a new word): true with the text typed in its place, false + // if the user cancelled. The lyrics editor asks through this. + // Overridden by the tests + virtual bool askForLyricsWordText(QString &text, bool isNew); + // --- The audio folder of the session (spec 6.4) --- // Where the next combined audio file of a take is to be written: the diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 1b322cb9..3c061fd3 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -82,6 +83,7 @@ #include #include #include +#include #include /** @@ -243,6 +245,19 @@ class TestMainWindow : public MainWindow int lyricsExportQuestions() const { return m_lyricsExportQuestions; } QString lyricsExportSuggestion() const { return m_lyricsExportSuggestion; } + // The texts the lyrics editor asks for, answered from here in turn. + // A question with no answer left is cancelled + void answerWordText(QString text) { m_wordTextAnswers.push_back({true, text}); } + void cancelWordText() { m_wordTextAnswers.push_back({false, QString()}); } + int wordTextQuestions() const { return m_wordTextQuestions; } + int wordTextAnswersLeft() const { return int(m_wordTextAnswers.size()); } + // The text the last question offered, and whether it was for a new word + QString wordTextOffered() const { return m_wordTextOffered; } + bool wordTextWasNew() const { return m_wordTextWasNew; } + // Done while the next question is open, as anything can be while its + // dialog runs an event loop + void whileAskingWordText(std::function f) { m_whileAskingWordText = f; } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { onRealtimePitchDetected(frame, hz); } @@ -287,6 +302,22 @@ class TestMainWindow : public MainWindow return m_lyricsExportAnswer; } + bool askForLyricsWordText(QString &text, bool isNew) override { + ++m_wordTextQuestions; + m_wordTextOffered = text; + m_wordTextWasNew = isNew; + if (m_whileAskingWordText) { + auto during = m_whileAskingWordText; + m_whileAskingWordText = nullptr; + during(); + } + if (m_wordTextAnswers.isEmpty()) return false; + auto answer = m_wordTextAnswers.takeFirst(); + if (!answer.first) return false; + text = answer.second; + return true; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -303,6 +334,11 @@ class TestMainWindow : public MainWindow QString m_lyricsExportAnswer; int m_lyricsExportQuestions = 0; QString m_lyricsExportSuggestion; + QList> m_wordTextAnswers; + int m_wordTextQuestions = 0; + QString m_wordTextOffered; + bool m_wordTextWasNew = false; + std::function m_whileAskingWordText; }; class TestRecordWorkflow : public QObject @@ -1140,6 +1176,89 @@ class TestRecordWorkflow : public QObject QPoint inRow(int x) { return QPoint(x, m_row.center().y()); } + // A double-click as Qt gives it: a press and a release, then the + // second press as the double-click, and its release + void doubleClickAt(QPoint pos) { + pressAt(pos); + releaseAt(pos); + sendMouse(QEvent::MouseButtonDblClick, pos, Qt::LeftButton, + Qt::LeftButton); + releaseAt(pos); + } + + void rightPressAt(QPoint pos) { + sendMouse(QEvent::MouseButtonPress, pos, Qt::RightButton, + Qt::RightButton); + } + + // The texts of the entries of the words' menu at a point, with a "-" + // in front of a disabled one; none where the menu would not open + QStringList menuAt(QPoint pos) { + QStringList texts; + for (const auto &e : m_window->lyricsEditor()->menuEntriesAt(pos)) { + texts << (e.enabled ? "" : "-") + e.text; + } + return texts; + } + + // The entry of the words' menu at a point with this text, chosen as a + // click on it chooses it; false if there is none + bool chooseAt(QPoint pos, QString text) { + for (const auto &e : m_window->lyricsEditor()->menuEntriesAt(pos)) { + if (e.text == text) { + m_window->lyricsEditor()->choose(e); + return true; + } + } + return false; + } + + // The words' menu the editor has popped up over pane 0, if it is on + // show + QMenu *wordsMenu() { + for (QMenu *menu : pane0()->findChildren()) { + if (menu->isVisible()) return menu; + } + return nullptr; + } + + static QStringList actionTexts(QMenu *menu) { + QStringList texts; + if (menu) for (QAction *a : menu->actions()) texts << a->text(); + return texts; + } + + // Close every menu on show, the words' and Tony's own: both are + // popped up, with no event loop of their own to end. How many + int closeMenus() { + int n = 0; + for (QWidget *w : QApplication::topLevelWidgets()) { + QMenu *menu = qobject_cast(w); + if (menu && menu->isVisible()) { + menu->close(); + ++n; + } + } + return n; + } + + // The lyrics' model, to change the words straight in it, as a test's + // setup: no command, and nothing marked modified + std::shared_ptr lyricsModel() { + return sv::ModelById::getAs + (m_window->lyrics()->getModelId()); + } + + // One word in the model replaced by another, and the pane painted + // again: the editor goes by the boxes as painted + void replaceWord(const sv::Event &from, const sv::Event &to) { + auto model = lyricsModel(); + QVERIFY(model && model->containsEvent(from)); + model->remove(from); + model->add(to); + m_row = lyricsBoxRow(); + } + // Every key of the settings, with its value, in the form the test // messages show static QMap allSettings() { @@ -6849,6 +6968,14 @@ private slots: QCOMPARE(lyricsEvents(), before); QVERIFY(!m_window->isDocumentModified()); QCOMPARE(undoOnce(), QString()); + + // and a double-click on a word asks nothing. The pane's own may + // open the edit dialog of the pitch point there, which the + // watchdog closes + doubleClickAt(inRow(columnOf(lyricsWord("kaksi").getFrame()) + 40)); + takeDialogs(); + QCOMPARE(m_window->wordTextQuestions(), 0); + QCOMPARE(lyricsEvents(), before); } // Yksi's end and kaksi's start are one edge on screen. The column @@ -7309,7 +7436,14 @@ private slots: hoverAt(inRow(edge + 40)); QCOMPARE(pane->cursor().shape(), own); QCOMPARE(m_window->statusText(), - QString("Drag a word's start or end to move it")); + QString("Double-click to change the text of \"kaksi\", " + "right-click to delete it")); + int gap = columnOf(endOf(lyricsWord("kaksi"))) + 20; + hoverAt(inRow(gap)); + QCOMPARE(pane->cursor().shape(), own); + QCOMPARE(m_window->statusText(), + QString("Right-click to add a word, " + "drag a word's start or end to move it")); hoverAt(inRow(edge)); QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); @@ -7373,10 +7507,562 @@ private slots: releaseAt(inRow(edge + 20)); QCOMPARE(lyricsWord("kolme").getFrame(), kolme.getFrame() + framesBetween(edge, edge + 20)); + QCOMPARE(m_window->wordTextQuestions(), 0); QCOMPARE(undoOnce(), QString("Move Word Start")); QCOMPARE(undoOnce(), QString()); } + // A double-click on a word, away from its edges, asks for its text + // and changes it, and nothing else of the word: one step on the + // history (decisions 8, 12). A double-click between words is the + // pane's, and asks nothing + void lyrics_edit_text_by_double_click() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::Event kaksi = lyricsWord("kaksi"); + int inside = columnOf(kaksi.getFrame()) + 40; + + m_window->answerWordText("kaksikko"); + doubleClickAt(inRow(inside)); + QCOMPARE(m_window->wordTextQuestions(), 1); + QCOMPARE(m_window->wordTextOffered(), QString("kaksi")); + QVERIFY(!m_window->wordTextWasNew()); + QCOMPARE(lyricsWord("kaksikko"), kaksi.withLabel("kaksikko")); + QCOMPARE(lyricsWord("kaksi").getFrame(), sv::sv_frame_t(-1)); + QCOMPARE(int(lyricsEvents().size()), 3); + QCOMPARE(int(commands.count()), 1); + QVERIFY(m_window->isDocumentModified()); + sv::EventVector changed = lyricsEvents(); + + QCOMPARE(undoOnce(), QString("Change Word Text")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + QCOMPARE(redoOnce(), QString("Change Word Text")); + QCOMPARE(lyricsEvents(), changed); + + // The menu's Edit Word Text... is the same edit + m_window->discardModifications(); + m_window->answerWordText("kaksi"); + QVERIFY(chooseAt(inRow(inside), "Edit Word Text...")); + QCOMPARE(m_window->wordTextQuestions(), 2); + QCOMPARE(m_window->wordTextOffered(), QString("kaksikko")); + QCOMPARE(lyricsEvents(), before); + QVERIFY(m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString("Change Word Text")); + QCOMPARE(lyricsEvents(), changed); + + // Last, as the pane's double-click moves the view. It may open + // the edit dialog of the pitch point there, which the watchdog + // closes + int gap = columnOf(endOf(kaksi)) + 20; + doubleClickAt(inRow(gap)); + takeDialogs(); + QCOMPARE(m_window->wordTextQuestions(), 2); + QCOMPARE(lyricsEvents(), changed); + } + + // Cancelled, nothing left of the text once cleaned, or the same text: + // the word stays as it was, and nothing goes on the history + // (decision 11). A new word likewise is not added + void lyrics_edit_text_refused() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + + m_window->cancelWordText(); + m_window->answerWordText(""); + m_window->answerWordText(" "); + m_window->answerWordText(QString("\t") + QChar(0x07) + " \r\n"); + m_window->answerWordText(" kaksi\t"); + for (int i = 1; i <= 5; ++i) { + doubleClickAt(inRow(inside)); + QCOMPARE(m_window->wordTextQuestions(), i); + QCOMPARE(lyricsEvents(), before); + } + + m_window->cancelWordText(); + QVERIFY(chooseAt(inRow(inside), "Edit Word Text...")); + QCOMPARE(m_window->wordTextQuestions(), 6); + + int gap = columnOf(endOf(lyricsWord("kaksi"))) + 20; + m_window->cancelWordText(); + m_window->answerWordText(QString(" ") + QChar(0x7F) + "\t"); + QVERIFY(chooseAt(inRow(gap), "Add Word...")); + QVERIFY(chooseAt(inRow(gap), "Add Word...")); + QCOMPARE(m_window->wordTextQuestions(), 8); + QVERIFY(m_window->wordTextWasNew()); + QCOMPARE(m_window->wordTextOffered(), QString()); + + QCOMPARE(lyricsEvents(), before); + QCOMPARE(int(commands.count()), 0); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString()); + } + + // The text is cleaned as the parsers clean a label: control + // characters out, a tab a space, blanks at the ends trimmed, at most + // 200 characters (decision 11) + void lyrics_edit_text_cleaned() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event kaksi = lyricsWord("kaksi"); + int inside = columnOf(kaksi.getFrame()) + 40; + + m_window->answerWordText(QString(" kak") + QChar(0x07) + + "si\tvaan \r\n"); + doubleClickAt(inRow(inside)); + QCOMPARE(lyricsWord("kaksi vaan"), kaksi.withLabel("kaksi vaan")); + + m_window->answerWordText(QString(250, QChar('a'))); + doubleClickAt(inRow(inside)); + QCOMPARE(lyricsWord(QString(200, QChar('a'))), + kaksi.withLabel(QString(200, QChar('a')))); + + int gap = columnOf(endOf(kaksi)) + 20; + m_window->answerWordText(QString(" ja") + QChar(0x7F) + "\t"); + QVERIFY(chooseAt(inRow(gap), "Add Word...")); + QVERIFY(lyricsWord("ja").getFrame() >= 0); + QCOMPARE(int(lyricsEvents().size()), 4); + + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(undoOnce(), QString("Change Word Text")); + QCOMPARE(undoOnce(), QString("Change Word Text")); + QCOMPARE(lyricsWord("kaksi"), kaksi); + } + + // The menu at a point: on a word, anywhere in its box, its edges as + // well, the word's two entries; between words Add Word..., disabled + // left of frame 0; outside the box row, or with edit mode off, none + // (decision 8) + void lyrics_edit_menu_entries() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + const QStringList onWord = { "Edit Word Text...", "Delete Word" }; + const QStringList add = { "Add Word..." }; + + int start = columnOf(kaksi.getFrame()); + int last = columnOf(endOf(kaksi)) - 1; + for (int x : { start, start + 2, start + 40, last - 2, last }) { + QCOMPARE(menuAt(inRow(x)), onWord); + auto entries = editor->menuEntriesAt(inRow(x)); + QCOMPARE(entries[0].word, kaksi); + QCOMPARE(entries[1].word, kaksi); + } + QCOMPARE(menuAt(inRow(start - 1)), onWord); + QCOMPARE(editor->menuEntriesAt(inRow(start - 1))[0].word, yksi); + + // Between words. The first column after kaksi's box shows the + // end of kaksi too, and where it starts is inside kaksi: a new word + // goes after it all the same + int after = columnOf(endOf(kaksi)); + int before = columnOf(kolme.getFrame()) - 1; + QVERIFY2(pane0()->getFrameForX(after) < endOf(kaksi), + "the column after kaksi starts after kaksi's end: " + "the case of this test is not set up"); + for (int x : { after, after + 2, (after + before) / 2, before }) { + QCOMPARE(menuAt(inRow(x)), add); + } + QCOMPARE(menuAt(inRow(columnOf(endOf(kolme)) + 50)), add); + QCOMPARE(menuAt(inRow(columnOf(0) - 5)), QStringList{ "-Add Word..." }); + + QVERIFY(menuAt(QPoint(start + 40, m_row.top() - 3)).isEmpty()); + QVERIFY(menuAt(QPoint(start + 40, m_row.bottom() + 3)).isEmpty()); + m_window->editLyricsAction()->trigger(); + QVERIFY(!editor->isEnabled()); + QVERIFY(menuAt(inRow(start + 40)).isEmpty()); + QVERIFY(menuAt(inRow(after)).isEmpty()); + } + + // A right press in the box row pops the words' menu up, and the pane + // never sees it, so Tony's own menu stays shut; the entries of the + // menu shown do what they say. Outside the row, or with edit mode + // off, the press is the pane's, and opens Tony's menu + void lyrics_edit_right_press() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy panes(pane0(), &sv::Pane::rightButtonMenuRequested); + sv::Event kolme = lyricsWord("kolme"); + int inside = columnOf(kolme.getFrame()) + 40; + + rightPressAt(inRow(inside)); + QCOMPARE(int(panes.count()), 0); + QMenu *menu = wordsMenu(); + QVERIFY2(menu, "the words' menu is not on show"); + QCOMPARE(actionTexts(menu), + (QStringList{ "Edit Word Text...", "Delete Word" })); + menu->actions()[1]->trigger(); + QCOMPARE(lyricsWord("kolme").getFrame(), sv::sv_frame_t(-1)); + QCOMPARE(int(lyricsEvents().size()), 2); + QCOMPARE(closeMenus(), 1); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(lyricsWord("kolme"), kolme); + + int gap = columnOf(kolme.getFrame()) - 20; + rightPressAt(inRow(gap)); + QCOMPARE(int(panes.count()), 0); + menu = wordsMenu(); + QVERIFY2(menu, "the words' menu is not on show"); + QCOMPARE(actionTexts(menu), QStringList{ "Add Word..." }); + QVERIFY(menu->actions()[0]->isEnabled()); + m_window->answerWordText("ja"); + menu->actions()[0]->trigger(); + QCOMPARE(m_window->wordTextQuestions(), 1); + QVERIFY(lyricsWord("ja").getFrame() >= 0); + QCOMPARE(closeMenus(), 1); + QCOMPARE(undoOnce(), QString("Add Word")); + + // Above the row, where the pane's menu is + rightPressAt(QPoint(inside, m_row.top() - 8)); + QCOMPARE(int(panes.count()), 1); + QVERIFY(!wordsMenu()); + QCOMPARE(closeMenus(), 1); + + m_window->editLyricsAction()->trigger(); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + rightPressAt(inRow(inside)); + QCOMPARE(int(panes.count()), 2); + QVERIFY(!wordsMenu()); + QCOMPARE(closeMenus(), 1); + QCOMPARE(lyricsWord("kolme"), kolme); + } + + // Add Word... puts the word at the click: 0.5 s long, or up to the + // next word, back into the gap as far as that makes it longer + // (decision 9), and on the line of the nearer neighbour (decision 10) + void lyrics_edit_add_word() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + QVERIFY(kaksi.getValue() != kolme.getValue()); + QCOMPARE(kolme.getFrame() - endOf(kaksi), LyricsEdit::newWordFrames(rate)); + + // The gap is 0.5 s, and the word fills it wherever the click is. + // Its neighbours are as near as each other, and the word before + // gives it its line + m_window->answerWordText("ja"); + QVERIFY(chooseAt(inRow(columnOf(kolme.getFrame()) - 20), "Add Word...")); + QCOMPARE(m_window->wordTextQuestions(), 1); + QVERIFY(m_window->wordTextWasNew()); + QCOMPARE(m_window->wordTextOffered(), QString()); + QCOMPARE(lyricsWord("ja"), + sv::Event(endOf(kaksi), kaksi.getValue(), + kolme.getFrame() - endOf(kaksi), "ja")); + QCOMPARE(int(lyricsEvents().size()), 4); + QCOMPARE(int(commands.count()), 1); + QVERIFY(m_window->isDocumentModified()); + sv::EventVector added = lyricsEvents(); + + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(redoOnce(), QString("Add Word")); + QCOMPARE(lyricsEvents(), added); + QCOMPARE(undoOnce(), QString("Add Word")); + + // kaksi 0.2 s shorter, for a wider gap. Near kolme, the word is + // moved back from it to be 0.5 s long, takes its line and is the + // line's first word now, which the layer draws bold + sv::Event shorter = kaksi.withDuration + (kaksi.getDuration() - sv::sv_frame_t(0.2 * rate)); + replaceWord(kaksi, shorter); + if (QTest::currentTestFailed()) return; + m_window->answerWordText("ja"); + QVERIFY(chooseAt(inRow(columnOf(kolme.getFrame()) - 5), "Add Word...")); + sv::Event ja = lyricsWord("ja"); + QCOMPARE(endOf(ja), kolme.getFrame()); + QCOMPARE(ja.getDuration(), LyricsEdit::newWordFrames(rate)); + QCOMPARE(ja.getValue(), kolme.getValue()); + Lyrics now = lyricsFromEvents(lyricsEvents(), rate); + QCOMPARE(now.words.size(), 4); + QCOMPARE(now.words[1].text, QString("kaksi")); + QCOMPARE(now.words[2].text, QString("ja")); + QVERIFY(now.words[2].line != now.words[1].line); + QCOMPARE(now.words[3].line, now.words[2].line); + QCOMPARE(undoOnce(), QString("Add Word")); + + // Near kaksi: at the click, 0.5 s long, on kaksi's line + int x = columnOf(endOf(shorter)) + 10; + m_window->answerWordText("ja"); + QVERIFY(chooseAt(inRow(x), "Add Word...")); + ja = lyricsWord("ja"); + QCOMPARE(columnOf(ja.getFrame()), x); + QCOMPARE(ja.getDuration(), LyricsEdit::newWordFrames(rate)); + QCOMPARE(ja.getValue(), kaksi.getValue()); + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(m_window->wordTextQuestions(), 3); + } + + // Before the first word the gap runs from frame 0, and a word that + // would be too long for it is cut to fit; after the last word there + // is no limit. Each takes the line of its one neighbour + void lyrics_edit_add_word_at_either_end() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::EventVector before = lyricsEvents(); + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kolme = lyricsWord("kolme"); + QVERIFY(yksi.getFrame() < LyricsEdit::newWordFrames(rate)); + + m_window->answerWordText("Nyt"); + QVERIFY(chooseAt(inRow(columnOf(yksi.getFrame() / 2)), "Add Word...")); + QCOMPARE(lyricsWord("Nyt"), + sv::Event(0, yksi.getValue(), yksi.getFrame(), "Nyt")); + QCOMPARE(lyricsFromEvents(lyricsEvents(), rate).words[0].text, + QString("Nyt")); + + int after = columnOf(endOf(kolme)) + 20; + m_window->answerWordText("loppu"); + QVERIFY(chooseAt(inRow(after), "Add Word...")); + sv::Event loppu = lyricsWord("loppu"); + QCOMPARE(columnOf(loppu.getFrame()), after); + QCOMPARE(loppu.getDuration(), LyricsEdit::newWordFrames(rate)); + QCOMPARE(loppu.getValue(), kolme.getValue()); + QCOMPARE(int(lyricsEvents().size()), 5); + + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(lyricsEvents(), before); + verifyPlaySourceClean(); + } + + // No room: a gap under 20 ms. The entry is there, disabled, and + // chosen asks nothing; a gap of 20 ms is room. Room taken after the + // menu was made, or while the text is asked for: nothing is added + void lyrics_edit_add_word_needs_room() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + sv::sv_frame_t minimum = LyricsEdit::minWordFrames(rate); + + sv::Event narrow = kaksi.withDuration + (kolme.getFrame() - (minimum - 1) - kaksi.getFrame()); + replaceWord(kaksi, narrow); + if (QTest::currentTestFailed()) return; + int x = columnOf(endOf(narrow)) + 1; + QVERIFY(x < columnOf(kolme.getFrame())); + sv::EventVector before = lyricsEvents(); + QCOMPARE(menuAt(inRow(x)), QStringList{ "-Add Word..." }); + m_window->answerWordText("ei"); + QVERIFY(chooseAt(inRow(x), "Add Word...")); + QCOMPARE(m_window->wordTextQuestions(), 0); + QCOMPARE(lyricsEvents(), before); + + sv::Event fits = narrow.withDuration(narrow.getDuration() - 1); + replaceWord(narrow, fits); + if (QTest::currentTestFailed()) return; + QCOMPARE(menuAt(inRow(x)), QStringList{ "Add Word..." }); + QVERIFY(chooseAt(inRow(x), "Add Word...")); + QCOMPARE(m_window->wordTextQuestions(), 1); + QCOMPARE(lyricsWord("ei"), + sv::Event(endOf(fits), kaksi.getValue(), minimum, "ei")); + QCOMPARE(undoOnce(), QString("Add Word")); + + // The menu made, and the gap closed before its entry is chosen + auto entries = editor->menuEntriesAt(inRow(x)); + QCOMPARE(int(entries.size()), 1); + QVERIFY(entries[0].enabled); + sv::Event closed = fits.withDuration(kolme.getFrame() - fits.getFrame()); + replaceWord(fits, closed); + if (QTest::currentTestFailed()) return; + before = lyricsEvents(); + m_window->answerWordText("ei"); + editor->choose(entries[0]); + QCOMPARE(m_window->wordTextQuestions(), 1); + QCOMPARE(lyricsEvents(), before); + + // and closed while the text is asked for + replaceWord(closed, fits); + if (QTest::currentTestFailed()) return; + m_window->whileAskingWordText([this, fits, closed]() { + replaceWord(fits, closed); + }); + editor->choose(entries[0]); + QCOMPARE(m_window->wordTextQuestions(), 2); + QCOMPARE(m_window->wordTextAnswersLeft(), 0); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + } + + // Delete Word takes the word away: the only word of its line, and so + // the line; then every word, after which Add Word still adds one. + // Undone, each comes back as it was + void lyrics_edit_delete_word() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::Event kolme = lyricsWord("kolme"); + QCOMPARE(lyricsFromEvents(before, rate).lineCount(), 2); + int inside = columnOf(kolme.getFrame()) + 40; + + QVERIFY(chooseAt(inRow(inside), "Delete Word")); + QCOMPARE(m_window->wordTextQuestions(), 0); + QCOMPARE(lyricsWord("kolme").getFrame(), sv::sv_frame_t(-1)); + QCOMPARE(int(lyricsEvents().size()), 2); + QCOMPARE(lyricsFromEvents(lyricsEvents(), rate).lineCount(), 1); + QCOMPARE(int(commands.count()), 1); + QVERIFY(m_window->isDocumentModified()); + sv::EventVector deleted = lyricsEvents(); + + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(redoOnce(), QString("Delete Word")); + QCOMPARE(lyricsEvents(), deleted); + + QVERIFY(chooseAt(inRow(columnOf(lyricsWord("Yksi").getFrame()) + 30), + "Delete Word")); + QVERIFY(chooseAt(inRow(columnOf(lyricsWord("kaksi").getFrame()) + 30), + "Delete Word")); + QVERIFY(lyricsEvents().empty()); + QVERIFY(m_window->lyrics()->isShown()); + QVERIFY(m_window->lyricsEditor()->isEnabled()); + m_row = lyricsBoxRow(); + QCOMPARE(menuAt(inRow(inside)), QStringList{ "Add Word..." }); + m_window->answerWordText("uusi"); + QVERIFY(chooseAt(inRow(inside), "Add Word...")); + QCOMPARE(lyricsWord("uusi").getValue(), 0.f); + QCOMPARE(lyricsWord("uusi").getDuration(), LyricsEdit::newWordFrames(rate)); + + QCOMPARE(undoOnce(), QString("Add Word")); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(lyricsEvents(), before); + } + + // The question runs an event loop of its own, and anything can + // happen while it is open: the word changed by an undo, edit mode + // switched off, the lyrics removed. The answer then changes nothing, + // and nothing is pushed over what is on the history + void lyrics_edit_words_change_during_question() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + + m_window->answerWordText("kaksikko"); + doubleClickAt(inRow(inside)); + sv::EventVector changed = lyricsEvents(); + QVERIFY(changed != before); + + m_window->whileAskingWordText([this]() { undoOnce(); }); + m_window->answerWordText("kaksikkoko"); + doubleClickAt(inRow(inside)); + QCOMPARE(m_window->wordTextQuestions(), 2); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(redoOnce(), QString("Change Word Text")); + QCOMPARE(lyricsEvents(), changed); + QCOMPARE(redoOnce(), QString()); + + // Deleted while its menu entry asks + m_window->whileAskingWordText([this, inside]() { + chooseAt(inRow(inside), "Delete Word"); + }); + m_window->answerWordText("kaksi"); + QVERIFY(chooseAt(inRow(inside), "Edit Word Text...")); + QCOMPARE(m_window->wordTextQuestions(), 3); + QCOMPARE(lyricsWord("kaksikko").getFrame(), sv::sv_frame_t(-1)); + QCOMPARE(int(lyricsEvents().size()), 2); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(lyricsEvents(), changed); + + m_window->whileAskingWordText([this]() { + m_window->editLyricsAction()->trigger(); + }); + m_window->answerWordText("kaksi"); + doubleClickAt(inRow(inside)); + QCOMPARE(m_window->wordTextQuestions(), 4); + QVERIFY(!editor->isEnabled()); + QCOMPARE(lyricsEvents(), changed); + switchLyricsEditingOn(); + if (QTest::currentTestFailed()) return; + + int gap = columnOf(endOf(lyricsWord("kaksikko"))) + 20; + m_window->whileAskingWordText([this]() { + m_window->removeLyricsAction()->trigger(); + }); + m_window->answerWordText("ja"); + QVERIFY(chooseAt(inRow(gap), "Add Word...")); + QCOMPARE(m_window->wordTextQuestions(), 5); + QVERIFY(!m_window->lyrics()->isShown()); + QVERIFY(!editor->isEnabled()); + + // On top of the history, the first change, whose model has gone + QCOMPARE(undoOnce(), QString("Change Word Text")); + verifyPlaySourceClean(); + } + + // Export Lyrics after edits writes the words as edited: a text + // changed, an end moved, a word added on the line of the word after + // it, and a word deleted + void lyrics_edit_export_writes_edits() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kolme = lyricsWord("kolme"); + + m_window->answerWordText("Yksin"); + doubleClickAt(inRow(columnOf(yksi.getFrame()) + 30)); + int end = columnOf(endOf(lyricsWord("kaksi"))) - 1; + dragFromTo(inRow(end), inRow(end - 30)); + m_window->answerWordText("ja"); + QVERIFY(chooseAt(inRow(columnOf(kolme.getFrame()) - 5), "Add Word...")); + QVERIFY(chooseAt(inRow(columnOf(kolme.getFrame()) + 40), "Delete Word")); + QCOMPARE(undoOnce(), QString("Delete Word")); + QCOMPARE(redoOnce(), QString("Delete Word")); + + sv::EventVector events = lyricsEvents(); + Lyrics now = lyricsFromEvents(events, rate); + QStringList texts; + for (const LyricWord &w : now.words) texts << w.text; + QCOMPARE(texts, (QStringList{ "Yksin", "kaksi", "ja" })); + QCOMPARE(now.words[2].line, 1); + + m_window->discardModifications(); + QString path = m_dir.filePath("edited-lyrics.ttml"); + m_window->setLyricsExportAnswer(path); + m_window->exportLyricsAction()->trigger(); + QFile file(path); + QVERIFY2(file.open(QIODevice::ReadOnly), "nothing was written"); + LyricsParseResult parsed = parseTtml(file.readAll()); + QVERIFY2(parsed.error == "", qPrintable(parsed.error)); + + const QVector &back = parsed.lyrics.words; + QCOMPARE(back.size(), now.words.size()); + for (int i = 0; i < back.size(); ++i) { + QString what = QString("word %1, \"%2\"").arg(i) + .arg(now.words[i].text); + QVERIFY2(back[i].text == now.words[i].text, + qPrintable(what + " came back as " + back[i].text)); + QVERIFY2(back[i].line == now.words[i].line, + qPrintable(what + ": another line")); + QVERIFY2(std::fabs(back[i].start - now.words[i].start) < 0.0005001, + qPrintable(what + ": another start")); + QVERIFY2(std::fabs(back[i].end - now.words[i].end) < 0.0005001, + qPrintable(what + ": another end")); + } + QVERIFY(std::fabs(back[1].end - 0.9) > 0.05); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(lyricsEvents(), events); + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform From 0c43b2cb7226aaf9cd3b578455945ab6aeb2cbaf Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:50:42 +0000 Subject: [PATCH 135/275] docs: the work order for phase a7, sessions in place with all files access the user chose ordinary folders mirrored by a sync app, worked on in place with android's all files access; the first phone test's findings (menus off-screen, the save through a content uri) go in the same phase. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 3ba8ba3e..53b4396c 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -159,7 +159,7 @@ builds happen in the container.) - A4 — Touch gestures on the panes. Done. - A5 — Compact touch mode. Done. - A6 — Oboe audio backend. -- A7 — Android files, permission and lifecycle. +- A7 — Sessions in place on the phone, and fixes from the first phone test. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -312,11 +312,36 @@ all of it; the A1 log entry. - Pure arithmetic (latency from timestamps and the like) in `tony_core` with core tests; the rest can only be judged on the phone. -### A7 — Android files, permission and lifecycle - -Detailed when it starts: microphone permission, stopping audio and saving on suspend, -opening a reference or session through the picker by copying it into app storage, and -exporting. +### A7 — Sessions in place on the phone, and fixes from the first phone test + +Decided with the user (2026-09-26): sessions live in ordinary folders on the phone, which +a sync app (Syncthing, FolderSync, Dropsync, OneSync) mirrors with the desktop, and Tony +works on them in place with Android's "All files access" (`MANAGE_EXTERNAL_STORAGE`, fine +for a sideloaded app). No session bundle. + +Read: [port-android.md](port-android.md) "Files, storage and cloud apps", "Permissions and +lifecycle"; [mobile-port.md](mobile-port.md) "Files and sessions"; [takes.md](takes.md) +"Files on disk", "The session file"; the A3b log entry (`AndroidFiles`, the +`getOpenFileName()` override). + +- All files access: declared in the manifest; asked for (a short explanation, then the + system's settings page for it) the first time a session is opened or saved, or a picked + file lies in the phone's own storage; checked with `Environment.isExternalStorageManager()`. +- A picked `content://` URI from the phone's own storage (the external storage provider, + and the downloads provider's `raw:` ids) is turned back into a real path (pure function + in `tony_core`, core tests). With access, that path is what Tony opens and saves: + sessions open and save in place with their takes folder, audio opens in place. +- Without a path (Drive, Dropbox, OneDrive, or no access): audio is copied in as now; a + session is refused with a clear message, and a save does not leave an empty file + behind (the picker creates the document before Tony writes). +- First phone test (user, A3b/A5 APK): no crash; loading, analysis, zoom and scroll work. + Fix: menus taller than the screen cannot be scrolled (items off-screen) — make menus + scrollable on Android; Save Session As suggests no file name and accepts an empty one; + the save error message came out garbled because the `content://` URI's `%3A`/`%2F` + were taken for `QString::arg()` placeholders by chained `.arg()` calls. +- On `Qt::ApplicationSuspended`: stop playback, and finish a take being recorded as Stop + does; save the session if it has a path and is modified. +- The 48 kHz play cursor (A1) is not this phase's: the user expects a fix on `default`. ### A8 — Documentation pass From 1cf602a061fe36c98375d71c92e1cb44986b5c87 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 03:57:00 +0000 Subject: [PATCH 136/275] docs: TTML lyrics, Export Lyrics and editing the words Architecture: how TTML is read and why its times are taken as absolute, why the editor is an event filter behind a mode, where edit mode goes off, one command per edit, and why the box row comes from the layer. Forks, testing, the manual checklist (items for the dialogs, dragging, the words' menu and high-DPI on Windows), open points and the README follow. A comment on what a drag notices of a keyboard undo is made accurate. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- README.md | 8 ++- docs/README.md | 2 +- docs/architecture.md | 120 ++++++++++++++++++++++++++++++++++----- docs/forks.md | 14 ++++- docs/manual-checklist.md | 64 +++++++++++++++++---- docs/open-points.md | 41 +++++++++---- docs/testing.md | 61 +++++++++++++++++--- main/LyricsEditor.cpp | 8 ++- 8 files changed, 267 insertions(+), 51 deletions(-) diff --git a/README.md b/README.md index c220b3fa..2ff60ccc 100644 --- a/README.md +++ b/README.md @@ -56,8 +56,12 @@ orange pitch track to compare with it. hides them, File -> Remove Lyrics takes them out, and File -> Export Lyrics... writes them, as they are now, to a TTML file. The reference must be the recording the lyrics were timed to (a Moises stem and its original - mix share a timeline); to change a word or its time, edit the file and - import it again + mix share a timeline) + * the lyrics can be corrected by ear, with the song there to listen to: with + Edit -> Edit Lyrics on, drag the start or end of a word in its box, + double-click a word to change its text, or right-click to delete a word or + add one between words. Each edit can be undone; the edits are saved with + the session, and Export Lyrics writes them * lyrics can be exported from Moises with the Moises-Lyric-Exporter browser extension. Set it to TTML, word by word, with the offset at 0 (otherwise every line is 0.2 s early, and Tony cannot tell): TTML gives every word diff --git a/docs/README.md b/docs/README.md index 8b19d1af..4d4a9803 100644 --- a/docs/README.md +++ b/docs/README.md @@ -13,7 +13,7 @@ methods, and they do not tell the story of fixed bugs. | --- | --- | | [building.md](building.md) | The MinGW build, and every way the environment has gone wrong; the Linux build of a cloud session | | [testing.md](testing.md) | The two test executables, the fakes and helpers, how tests turned out to be worthless, races | -| [architecture.md](architecture.md) | What the fork adds, `tony_core` / `tony_app`, who owns what, and the rules for layers, models, commands, playback and session files | +| [architecture.md](architecture.md) | What the fork adds, `tony_core` / `tony_app`, who owns what, and the rules for layers, models, commands, playback and session files; the smaller features, the lyrics and their editor among them | | [recording.md](recording.md) | Record to Stop, step by step and why in that order; latency; pre-roll; record into selection; the live tracker | | [takes.md](takes.md) | Takes: decisions, the audio swap, ranged analysis and merge, undo, the coverage strip, files and sessions, limitations | | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | diff --git a/docs/architecture.md b/docs/architecture.md index 7e0820ec..caf8ed1d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -15,8 +15,9 @@ Upstream Tony analyses the pitch of one recording. This fork makes it a singing the rest, erase, undo, and keep several takes. See [takes.md](takes.md). 5. Around that: play the reference while recording, latency compensation, pre-roll, record into selection, an octave-shifted "alternate" pitch track to follow, timed - lyrics along the bottom of the pane with the word being sung highlighted, and a - background music track that is played but never analysed. + lyrics along the bottom of the pane with the word being sung highlighted and every + word editable in place, and a background music track that is played but never + analysed. The user-facing description is in the [README](../README.md). @@ -31,8 +32,8 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LatencyUtils.h` | -| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `TakeCommands`, `TakeLayers`, `PaneUtils` | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LyricsTtml`, `LyricsEdit`, `LatencyUtils.h` | +| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `LyricsEditor`, `TakeCommands`, `TakeLayers`, `PaneUtils` | When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` list **only if** it has `Q_OBJECT`. Logic that can be written as pure functions or a plain @@ -56,7 +57,8 @@ follow. `MainWindow` then only fills the struct in and puts the answer on screen `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`. All three watch `Document::layerAboutToBeDeleted` in case someone else deletes their layer, and all three must be deleted in `~MainWindow` **before** the base class deletes the document - (as must `m_analyser2`). + (as must `m_analyser2`). `LyricsEditor` owns no layer: it finds the lyrics through + `LyricsTrack` at every event, and is deleted before it. - `RealtimePitchTracker` is a `QThread` that only **reads** the recording's `WritableWaveFileModel` and emits `pitchDetected(frame, hz)`. It never touches the pitch model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued @@ -205,10 +207,11 @@ after `openPath()`, and only then prune the extra pane — the imported waveform is the only reference to the model until `m_analyser2` has a layer of its own. It ends with `clearTakeHistory()`, which also disposes of the "Import" command for the pruned pane. -**Lyrics** (`Lyrics` parses the LRC file, `LyricsTrack` owns the layer): one `RegionModel` -in pane 0 on the reference's timeline, a region per word (or per line, for a line with no -word times): frame = start, duration = end - start (at least one frame), label = the word, -value = the line's index, which is what the bold line starts go by. It is drawn by the +**Lyrics** (`Lyrics` and `LyricsTtml` read the file, `LyricsTrack` owns the layer, +`LyricsEditor` edits the words): one `RegionModel` in pane 0 on the reference's timeline, +a region per word (or per line, for a line with no word times): frame = start, +duration = end - start (at least one frame), label = the word, value = the line's index, +which is what the bold line starts go by. It is drawn by the svgui fork's `PlotLyrics` style ([forks.md](forks.md)), in boxes along the bottom of the pane just above the coverage strip, because a session restores only layers `LayerFactory` can make. Found again after a session load by its untranslated object @@ -223,10 +226,38 @@ because that layer is still in the pane when `layerAboutToBeDeleted` arrives. No takes: the take code finds layers by take name, source model or extra pane, so it never finds this one, and it stays on show during a take, when the singer needs the words most. Import and Remove push no command and leave the undo history alone, like Load Background -Music: the simpler option, and the file is still there to import again. Show Lyrics is the layer's own -visibility, which the session saves, not a QSettings key. The parser strips control -characters (and U+FFFE, U+FFFF, which the UTF-8 decoder lets through) from every label: -XML 1.0 cannot hold them, and one in a label would make the `.ton` unreadable. +Music: the simpler option, and the file is still there to import again (edits made in +Tony since are not: Export Lyrics keeps them). Show Lyrics is the layer's own +visibility, which the session saves, not a QSettings key. Both parsers strip control +characters (and U+FFFE, U+FFFF, which the UTF-8 decoder lets through) from every label, +and the editor cleans a typed text the same way: XML 1.0 cannot hold them, and one in a +label would make the `.ton` unreadable. + +**Lyrics files.** `parseLyrics()` reads the file as TTML if its first character that is +not blank, after a byte order mark, is `<`, and as LRC otherwise: an LRC file never +starts with `<`, so the content decides whatever the file is called. TTML is read as +Apple Music, the Moises-Lyric-Exporter and AMLL TTML Tool write it, a `

` per line and +a timed `` per word, with elements matched by local name in any namespace (files +without the TTML namespace exist). Its times are read as **absolute**: strict TTML makes +a child's time relative to its parent's, but no lyrics tool writes them so, and read that +way every word would move by its line's start. Timed spans with no white space between +them are syllables of one word and are joined: Tony edits words, and the exporter writes +punctuation as a span of its own straight after the word. Background vocals, +translations and romanisations (`ttm:role` `x-bg`, `x-translation`, `x-roman`, +`x-romanization`) are skipped with a warning: a translation is not what is sung, and +background vocals overlap the lead's words in a row that has room for one word at a time. +A `

` with no timed spans is one word, the whole line. Ends the file does not give are +inferred as for LRC; the exporter's TTML gives them all, which is why the README says to +set the exporter to TTML. A file with a ` Edit Lyrics is on. In the box row it keeps from the pane only +what it acts on: a left press on an edge and the drag it starts, up to the release; a +double-click inside a word; a right press, for the words' menu; and moves with no button +held, where the cursor and the context help are its own. Everything else reaches the pane +as it would without edit mode, so a click still moves the playback cursor. Editing needs +a mode because words touch: the row is edges nearly everywhere, and an editor always on +would take the clicks meant for the cursor. + +- **Edit mode goes off in one place**, `updateMenuStates()`, whenever + `lyricsEditAllowed()` is false: the lyrics removed or hidden, recording started, the + session closed (through `documentRestored()`). Each of those ends in + `updateMenuStates()`. An import switches it off itself, before the words there go. + Going off finishes a drag in progress, as a release would, and so pushes its command; + when the lyrics have gone first (closing the session deletes the document before + `updateMenuStates()` runs), the drag is dropped unpushed instead, as below. + `updateMenuStates()` runs during undo and redo too, where a push would delete the + command running, so nothing an undo or redo does may make `lyricsEditAllowed()` false: + the lyrics' presence and visibility stay out of commands. +- **One `ChangeEventsCommand` per edit** ("Move Word Start", "Move Word End", "Change Word + Text", "Add Word", "Delete Word"), holding the model's id and `Event` values, pushed + done with `addCommand(command, false)`; `CommandHistory` marks the session modified, and + the session saves the model as it is. A drag changes the model at every move, so that + the word follows the pointer and the layer lays the words out again (svgui fork), and + its command is pushed **only at the release**; a drag that ends where it began pushes + nothing. The editor pushes only from a mouse event or a menu choice, never from + anything an undo or redo reaches. Take operations clear the history, lyrics steps with + it; Import and Remove do not, and a lyrics step left from before them does nothing, its + model being gone. +- **The drag rules are fed the words as they were at the press**, at every move: the + limits (the neighbour, the 20 ms minimum) come from where the word was then, so a drag + back puts the word back. Fed the moved words, the limits would travel with the word. + The edge moves as far as the pointer has, in frames, since the press, so a press a few + pixels off the edge makes no jump, and a view that scrolls in the middle of a drag + changes nothing. +- **The model can change under a drag** (a keyboard undo that takes the word away, the + lyrics removed or replaced). The editor finds the layer and model through `LyricsTrack` at every event + and checks that the word is still what the drag made it; if not, the drag's command is + deleted unpushed and the rest of the drag, to the release, is swallowed, as the pane + never saw its press. +- **The box row is the layer's** (`getLyricsBoxRow()`), never worked out again from the + pane: the layer knows where it painted, in the pane's logical coordinates, the ones a + mouse event has, which on a high-DPI screen are half those of the proxy it paints + through. The row is empty before the first paint, and hidden lyrics keep the row they + were last painted in, so the editor checks visibility as well. The boxes' x come from + the pane's `getXForFrame()`, as the layer paints them. +- **After a question, look again.** The text dialog runs an event loop of its own, in + which anything can happen: an import, a session closed, an undo. So after it the edit + looks the model up again (the same id, edit mode still on) and the word in it (still + there, unchanged), and does nothing if either has gone; Add Word works its span and line + out again from the words as they are then. The words' menu is `popup()`ed, as Tony's + own menu is, so that the question is not asked from inside the pane's event handling; + its entries hold values (the model's id, the word, a frame) and are looked up again + when chosen. +- Add Word goes at the **last** frame of the column clicked: the first can be inside the + word before, whose end lies in the column after its box. +- Pane 0 of a reference has no context-help connection (only `newSession()` makes one), + so the editor's help goes straight to the status bar, and the editor clears it itself + when the pointer leaves the row. diff --git a/docs/forks.md b/docs/forks.md index 47cd99ad..951680e8 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -88,7 +88,8 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` the view's at the least, up to four times, and never more than an eighth of the view's height; it grows with the **square root** of the zoom, so that zooming in gives the words room (their boxes grow with the zoom itself). No vertical scale, no feature - description, not editable. + description, and not editable by the pane's tools: Tony's `LyricsEditor` edits the + model itself. `setHighlightFrame()` draws the region at that frame in amber (the latest to start, where regions overlap) and emits `layerParametersChanged()` only when that region changes: the highlight is painted into the view's cache, so each new word repaints the view, a few @@ -102,6 +103,17 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` lets the layer stay **scrollable**: `View::getNonScrollableFrontLayers()` treats every layer in front of a non-scrollable one as non-scrollable too, so the pitch tracks above the lyrics would repaint on every cursor update. + The layout is made again, the highlighted region found again and the whole view + repainted on **any** change to the model (member-pointer connections to `modelChanged` + and `modelChangedWithin`): an edited word keeps the count of regions and often the + extent of the whole, which with the zoom and the font is all the cache otherwise checks; + a label that changes or moves can move the labels before it and the rows of those after, + anywhere in the view; and the word being sung may be the one edited, or another one now. + `getLyricsBoxRow(view)` says where the boxes' row was last painted in that view, empty + before the first paint, for Tony's editor to tell whether the pointer is over a box. It + is in the view's own **logical** coordinates, the ones a mouse event has: on a high-DPI + screen the layer paints through a proxy at twice the size, so the row cannot be worked + out again from the pane. - `Pane::getTopFlexiNoteLayer()` skips dormant layers, so note tools cannot edit the notes of a take that is put away. - `Pane::setWorkModel()` / `getWorkModel()`: which model's extents are blocked off at the diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index c1917573..84c6f0ee 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -127,31 +127,35 @@ Launch with `.\build.bat run`. 41. **The left edge**: a word in the first ~30 px of the view is under the pane's vertical scale, at the bottom left (scroll so that a word is at the left edge). How much does that matter in use? -42. **Inferred ends**: the exporter writes no end times. With word timing the last word of - a line ends at the next line but at most 2 s after it starts, unless a `♪` line marks - the end; with line timing a line lasts until the next one, and the last line 5 s. Do - those boxes mislead, and does the last word of a line stay highlighted too - long? The start times are exact. +42. **Inferred ends**, with an LRC file, which has no end times (the exporter's TTML has + them). With word timing the last word of a line ends at the next line but at most 2 s + after it starts, unless a `♪` line marks the end; with line timing a line lasts until + the next one, and the last line 5 s. Do those boxes mislead, and does the last word of + a line stay highlighted too long? The start times are exact. 43. **Hover readout**: with the lyrics shown, hovering over the pitch tracks gives the same readout and the same vertical scale as without them, also after turning the alternate pitch track off and after deleting a take. -44. **A real Moises export** of one of your songs (exporter offset 0, gap threshold low), +44. **A real Moises export** of one of your songs, as TTML, word by word, offset 0, imported onto the Moises stem or the original mix: the words line up with the vocal, by - eye and while playing, and the highlight moves with the voice. All early or late by the - same amount means the reference is not the recording Moises timed; an `[offset:]` line - in the file moves them. + eye and while playing, each box ends where the word does, and the highlight moves with + the voice and goes off in the gaps. A word Moises split into syllables is one box; + punctuation is on its word. All early or late by the same amount means the reference + is not the recording Moises timed (Tony cannot shift the whole song; an `[offset:]` + line added to an LRC file can). Compare with an LRC export of the same song, gap + threshold low. 45. **Finnish text**: ä and ö come out right in the pane, and again after save and reopen. 46. **Show Lyrics and Remove Lyrics**: hiding keeps the words for later, Remove takes them out, a second import replaces the first; each makes Close ask whether to save. The waveform is pale while the words are on show and grey again when they are hidden or removed. The status bar after an import counts words and lines and names anything - skipped. + skipped (a TTML file's background vocals and translations, say). Edit Lyrics goes off + and is greyed out when the words are hidden or removed. 47. **Session**: save and reopen: the same words, hidden or shown as saved, and the waveform pale or grey to match; playback still ends at the end of the song, even with words past it. 48. **During a take**: the words stay on show and readable while recording, with the - countdown and the live dots, and the take's waveform is pale as well; Import Lyrics is - greyed out while recording. + countdown and the live dots, and the take's waveform is pale as well; Import Lyrics + and Edit Lyrics are greyed out while recording, and Edit Lyrics is off after it. 49. **The highlight while playing**: the word being sung turns amber as the reference reaches it and light again when it ends, in step with the voice, without flicker or visible lag. With playback stopped, a click or seek into a word highlights it at once, @@ -160,3 +164,39 @@ Launch with `.\build.bat run`. reference through the lead-in, while the countdown is on the status bar, and on through the take from its position; after Stop it is back on the word at the take's position. The same with Play Reference While Recording off. + +## Lyrics: import and export dialogs, editing + +51. **The file dialogs on Windows.** Import Lyrics opens beside the reference and offers + "Lyrics (*.ttml *.lrc)" first, then TTML, LRC and all files. Export Lyrics offers + the reference's name with `.ttml`, beside it; a name typed without `.ttml` gets it + from the dialog (Tony adds none); replacing a file asks first. A folder that cannot + be written gives "Could not export lyrics" and leaves any file there as it was. After + an export the status bar says how many words and lines, and Close does not ask to + save because of it. +52. **Export after edits, imported again**: edit a few words, Export Lyrics, Remove + Lyrics, import the file: the same words, times, bold line starts and title. Does + another tool (AMLL TTML Tool, say) read the file? +53. **Edit mode**: with Edit > Edit Lyrics on, the pointer over a box edge in the row + shows the horizontal-resize cursor and nowhere else; the status bar says what the + mouse does there and clears when the pointer leaves the row. A click still moves the + playback cursor, and above the row, or with edit mode off, the mouse does what it + always did. Is Edit Lyrics easy to find, with no shortcut? +54. **Dragging an edge**: where two words touch, just left of the line moves the earlier + word's end and just right of it the later word's start. Is the grab (6 px each side) + right? An edge stops at the neighbouring word and a word at 20 ms, without jumping; + the words, their labels and the highlight follow the pointer; one Ctrl+Z takes back + one whole drag, and Close asks whether to save. +55. **The text dialog and the words' menu**: a double-click in a word asks for its text + (the first click of it also moves the playback cursor there: acceptable?); empty text + or Cancel changes nothing. A right-click in the row gives "Edit Word Text..." and + "Delete Word" on a word, "Add Word..." between words (greyed where the gap is under + 20 ms); a new word is 0.5 s long or reaches the next word, joins the nearer + neighbour's line and is bold if it starts it. A right-click above the row still gives + Tony's own menu. Is the wording right? +56. **High-DPI**: on a screen scaled to 150 % or 200 %, the resize cursor appears exactly + over the box edges, the row the mouse edits is the row of boxes (not above or below + it), and a drag keeps the edge under the pointer. +57. **Drag smoothness**: with a whole song's words (hundreds) in view, and again while the + reference plays, a drag follows the pointer without stutter and playback does not + break up. diff --git a/docs/open-points.md b/docs/open-points.md index 89f3ded0..33f7b76f 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -13,8 +13,11 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - **Constrain Playback to Selection + pre-roll**: the play source constrains playback to the selection, the lead-in is outside it, so it is cut short. Nothing keeps the two apart. - **Take operations clear the undo history with no prompt** (all but Rename). -- **A shortcut for Show Lyrics?** There is none; one would have to be checked against - `KeyReference` for clashes first. +- **A shortcut for Show Lyrics or Edit Lyrics?** There is none; one would have to be + checked against `KeyReference` for clashes first. +- **The editing constants** were defaults taken without the user: the 20 ms shortest + word, the 0.5 s new word, the 6 px grab on each side of an edge, the menu's wording, + and Edit Lyrics living in the Edit menu. - None of the [manual checklist](manual-checklist.md) has been run. ## Not built @@ -24,15 +27,14 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - Background music is not saved in the session; it is reloaded by hand. - An old session (before takes) loses its singing track without telling the user why. - Recording that starts before frame 0 of the reference. -- **Lyrics cannot be shifted or edited in Tony.** The remedy is to edit the LRC file and - import it again; its `[offset:]` tag is the only shift. That includes lyrics that are - all off by the same amount because the reference is not the recording they were timed - to. Import and Remove are not undoable, and a new import replaces the lyrics without - asking. -- **LRC only**: no SRT, TTML or Moises JSON. The exporter's TTML carries real word (and - syllable) end times where its LRC has none, so it is the natural second format if the - inferred ends turn out misleading; another format is another function beside - `parseLrc()`. +- **Lyrics are edited a word at a time**: no shifting of a line or of the whole song, no + splitting or merging of words, no editing of line breaks, no syllables (a file's + syllables are joined into their word). Not wanted for now. Lyrics that are all off by + the same amount, because the reference is not the recording they were timed to, can + only be moved by an LRC file's `[offset:]` tag. Import and Remove are not undoable, and + a new import replaces the lyrics, edits made in Tony included, without asking. +- **TTML and LRC only**: no SRT or Moises JSON. Another format is another parser that + `parseLyrics()` chooses. ## Weak spots @@ -66,6 +68,23 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. made-up but realistic timing (about three words a second) and a 26 px font: one row at 200 px/s and above, a few words in the second row at 150 px/s, many in it and some left out at 100 px/s. Only the word being sung is always drawn. +- **An undo from the keyboard in the middle of a lyrics drag can leave the word twice**: + once the word has moved, undoing an earlier step of the same word (Ctrl+Z with the + mouse button still held) removes the word as that step left it, which is not in the + model, and adds the old one back beside the moved one. The word the drag made is still + there, so the drag's own check does not see the change. There is no hook before an + undo to end the drag first. +- **The first click of a double-click on a word moves the playback cursor** there, as + any click does; only the second is the editor's. Accepted for now. +- **A double-click the pane handles itself** (edit mode off, or between words in edit + mode) can open the edit dialog of the pitch point under it: upstream Tony's Navigate + mode, not new, but easier to meet now that double-clicks are in use there. +- **Lyrics steps left on the history after Remove Lyrics or an import do nothing** when + undone or redone: their model is gone (the command logs a warning). The menu still + offers them. +- **Pane 0 of a reference has no context-help connection**: only `newSession()` connects + its pane to the status bar, so the pane's own help never shows there; the lyrics editor + sends its help itself and clears it on leaving the row. - **Right after `closeSession()`, Show Lyrics and the alternate pitch actions keep their enabled and checked states** until the next reference or session opens: nothing there calls `updateLayerStatuses()`. Show Lyrics then does nothing when chosen. diff --git a/docs/testing.md b/docs/testing.md index 01a31aa3..ec0504c3 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,7 +5,7 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics` | seconds | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow` | about 4.5 minutes (measured 2026-09-20), nearly all of it `TestRecordWorkflow`: takes are recorded in real time | `meson test` / `build.bat test` runs both plus four svcore suites. @@ -55,7 +55,9 @@ Windows path would start an escape in the C string. `range_analysis_torn_down_while_running`, `save_during_ranged_analysis`, `undo_during_analysis_then_redo` and `analyse_now_reanalyses_the_take`, where the analysis finishes before the race they need can be set up. Which of those five fail - changes from run to run. + changes from run to run, and so can where: `undo_during_analysis_then_redo` fails + either before the undo, with no ranged analysis left running, or after the redo, with + `analysedRangeStart()` already 0 — the same race. ## Design principles @@ -88,13 +90,18 @@ Windows path would start an escape in the C string. `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the fake device through `createAudioIO()`, and **answers dialogs through virtual seams**: `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, - `askForLyricsFile()`, each with a `set...Answer()` and a counter of questions asked. A + `askForLyricsFile()`, `askForLyricsExportFile()` (which also keeps the path it was + offered), each with a `set...Answer()` and a counter of questions asked. + `askForLyricsWordText()` takes a queue of answers (`answerWordText()`, + `cancelWordText()`; none left is Cancel) and can run something while the question is + open (`whileAskingWordText()`), as a real dialog's event loop lets anything happen. A test cannot answer a real dialog: anything new that asks the user needs such a virtual. - Fixture helpers: `makeWindow(config)`, `writeWav()`, `openReference()`, `startTake()` / `stopTake()` / `take(ms)`, `verifyPlaySourceClean()`, `layersOnModel()`, `paneHasLayer()`, `documentHasLayer()`, `reopenAsSession()` / `reopenSession()`, `verifyEventsSurvived()`; for the lyrics `lyricsFixture()`, `writeLrc()`, - `verifyLyricsUntouched()`. + `verifyLyricsUntouched()`; for editing them `lyricsEditFixture()` and the mouse helpers + below. - A **dialog watchdog**: a 50 ms timer closes any modal dialog and records it, and `cleanup()` fails the test for one that was not expected. `dialogsMatching()` is for the dialogs a test does expect. @@ -107,15 +114,53 @@ Windows path would start an escape in the C string. of its own because the Vamp *plugin* SDK headers must not meet the *host* SDK headers svcore uses. - `testdata/happy_birthday_gp_masked.wav`: a real sung recording. -- `testdata/lyrics/`: LRC files with invented text. Two are in the exact format of the - Moises lyrics exporter (word timing and line timing: no end times, a `♪` gap line, a - word with punctuation glued to the one before, a line its clamp stamped 0); the third is - a generic LRC that does give ends. +- `testdata/lyrics/`: LRC and TTML files with invented text. Two LRC files are in the + exact format of the Moises lyrics exporter (word timing and line timing: no end times, a + `♪` gap line, a word with punctuation glued to the one before, a line its clamp stamped + 0); the third is a generic LRC that does give ends. `moises-exporter-words.ttml` and + `moises-exporter-lines.ttml` are the exporter's TTML, word by word and line by line, + offset 0, **made by the exporter's own code**: a small node script copied its input + handling and TTML branch verbatim and ran them on an invented Moises-style JSON (segment + format, one word in syllables, punctuation as a word of its own). The script is not in + the repository, because it is the exporter's code; to make the files again, do the same + from the exporter's reviewed commit. `amll-style.ttml` is written by hand in the style + of AMLL TTML Tool: times `mm:ss.mmm`, two agents, a background-vocal span and + translation spans. Prefer signals that describe themselves: `TestTakeAudio` uses constants and ramps so that every sample says where it came from. Assert **identity** as well as equality where the point is that something survived: the same layer and model objects before and after. +### The mouse in pane 0 (`lyrics_edit_*`) + +The lyrics editor is an event filter on pane 0, so its tests send it real mouse events. +The rules of the edits themselves are tested without a window, in `TestLyricsEdit`. + +- **`QApplication::sendEvent()` to the pane** (`sendMouse()`, and `hoverAt()`, + `pressAt()`, `moveHeldTo()`, `releaseAt()`, `dragFromTo()`, `doubleClickAt()`, + `rightPressAt()` on top of it): an event sent so goes to the pane's event filters first + and then to the pane, as real input does. Not `QTest::mouseMove`, which does not carry + the buttons held. A double-click is sent as Qt makes one: a press and a release, then + `MouseButtonDblClick` in place of the second press, and its release. +- **Positions from what was painted**: y from `getLyricsBoxRow()`, x from the pane's + `getXForFrame()` (`inRow()`, `columnOf()`). The layer knows where the row is only once + it has painted it, so the helpers paint the pane first (`grab()`). +- **The pane gets a size and a zoom of its own** (`showEditableLyrics()`: 1000 x 120, + 128 frames a pixel). The test window is never shown, and its layout leaves pane 0 a few + pixels high, or never lays a new pane out at all. At that zoom every word is in view and + about a hundred pixels wide, so an edge's grab never reaches across a word; the fixture + checks all of that before the test relies on it. +- The words' menu: `menuEntriesAt()` / `choose()` are the seam for what it offers and + does (`menuAt()`, `chooseAt()`); `lyrics_edit_right_press` also finds the menu really + popped up (`wordsMenu()`) and triggers its entries. A popped-up menu has no event loop + to end and the dialog watchdog leaves it alone: close it with `closeMenus()`. +- A double-click that the pane handles itself (edit mode off, or between words) can open + the edit dialog of the pitch point there, as upstream Tony does in Navigate mode. The + watchdog closes it, and a test that sends one takes it with `takeDialogs()`, or + `cleanup()` fails it. +- `lyricsModel()` changes the words straight in the model, as setup: no command, nothing + marked modified. + ### Setup that fails confusingly when it is missing - An `Analyser` without a `MainWindow` needs `qRegisterMetaType` for `"ModelId"`, diff --git a/main/LyricsEditor.cpp b/main/LyricsEditor.cpp index 5d38a225..c37cdef6 100644 --- a/main/LyricsEditor.cpp +++ b/main/LyricsEditor.cpp @@ -504,9 +504,11 @@ LyricsEditor::dragTo(int x) { if (!m_dragging || !m_pane || m_dragModel.isNone()) return; - // The model can change under a drag: an undo from the keyboard, or - // the lyrics removed or replaced. The word being dragged is then not - // what the drag last made it, and nothing it would do now is right + // The model can change under a drag: a keyboard undo that takes the + // word away, or the lyrics removed or replaced. The word being + // dragged is then not what the drag last made it, and nothing it + // would do now is right. An undo that leaves the word as the drag + // made it goes unnoticed: see docs/open-points.md auto model = dragModel(); if (!model || !model->containsEvent(m_dragCurrent)) { abandonDrag(); From 276e8e1db6e4a81125eca5c10b923411d7c3719f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 04:18:15 +0000 Subject: [PATCH 137/275] build: oboe 1.11.0 in the android prefix a static oboe with a .pc file, for the audio backend; the link check uses it and allows liblog. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/android/build-deps.sh | 47 ++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-) diff --git a/deploy/android/build-deps.sh b/deploy/android/build-deps.sh index 7716c8ec..41fb44fe 100755 --- a/deploy/android/build-deps.sh +++ b/deploy/android/build-deps.sh @@ -43,6 +43,8 @@ # bzip2 svcore's BZipFileDevice, which includes bzlib.h # whatever the defines say; the NDK has no bzip2 # Boost headers only: pYIN's boost/math +# Oboe Google's audio library over AAudio and OpenSL ES, for +# Tony's audio backend (main/OboeAudioIO) # Left out, as on a phone they have no use: oggz and fishsound (Ogg # Vorbis), JACK, PulseAudio, ALSA and PortAudio, liblo (never enabled). # @@ -341,6 +343,37 @@ build_boost() { grep -q '^#define BOOST_LIB_VERSION "1_83"' "$prefix/include/boost/version.hpp" } +build_oboe() { + # The newest 1.x release. Oboe opens libaaudio.so and libOpenSLES.so + # with dlopen() and defines OpenSL ES's interface IDs itself, so it + # links against liblog alone. Its CMake install puts the library in + # lib//, moved here to lib/ beside the others; it writes no .pc + # file. + git clone -q -c advice.detachedHead=false --depth 1 --branch 1.11.0 \ + "$github/google/oboe.git" "$work/oboe" + if [ "$(git -C "$work/oboe" rev-parse HEAD)" != b115f47593969fd67a21e9f63640ffef749b5067 ]; then + echo "ERROR: tag 1.11.0 of oboe is not at the expected commit" 1>&2 + exit 1 + fi + rm -rf "$prefix/include/oboe" "$prefix/lib/$abi" + cmake_build "$work/oboe" + mv "$prefix/lib/$abi/liboboe.a" "$prefix/lib/liboboe.a" + rmdir "$prefix/lib/$abi" + cat > "$prefix/lib/pkgconfig/oboe.pc" < "$check/meson.build" <<'EOF' project('tony-deps-check', 'cpp', default_options: ['cpp_std=c++17']) deps = [dependency('boost')] foreach name : ['sndfile', 'samplerate', 'fftw3', 'rubberband', 'sord-0', - 'serd-0', 'mad', 'id3tag', 'opusfile', 'bzip2'] + 'serd-0', 'mad', 'id3tag', 'opusfile', 'bzip2', 'oboe'] deps += dependency(name, static: true) endforeach shared_library('tonydepscheck', 'check.cpp', dependencies: deps) @@ -418,6 +452,7 @@ cat > "$check/check.cpp" <<'EOF' #include #include #include +#include // Something from each library, so that the static linker has to find it extern "C" int tony_deps_check(const unsigned char *data, int size) @@ -464,6 +499,14 @@ extern "C" int tony_deps_check(const unsigned char *data, int size) boost::math::normal_distribution normal(0.0, 1.0); n += int(boost::math::cdf(normal, double(size)) * 10); + std::shared_ptr audio; + oboe::AudioStreamBuilder builder; + builder.setDirection(oboe::Direction::Output); + if (size > 0 && builder.openStream(audio) == oboe::Result::OK) { + n += audio->getSampleRate(); + audio->close(); + } + return n; } EOF @@ -493,7 +536,7 @@ case "$kind" in esac for lib in $needed; do case "$lib" in - libc.so|libm.so|libdl.so|libz.so|libc++_shared.so) ;; + libc.so|libm.so|libdl.so|libz.so|liblog.so|libc++_shared.so) ;; *) echo "ERROR: it loads $lib, which is not part of Android or the NDK" 1>&2; exit 1 ;; esac done From f326bfbcdccfd4a1c09100f4ce9c3933cfc1f2dd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 04:18:15 +0000 Subject: [PATCH 138/275] feat: oboe audio backend on android a bqaudioio systemaudioio over oboe's full-duplex stream at the device's own rate, installed by overriding createaudioio() on android; latency from the streams' timestamps, with the arithmetic in tony_core and tested; the microphone asked for before the first take; a device that goes away is reopened. android no longer forces no audio. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/MainWindow.cpp | 138 +++++++ main/MainWindow.h | 20 ++ main/OboeAudioIO.cpp | 652 ++++++++++++++++++++++++++++++++++ main/OboeAudioIO.h | 136 +++++++ main/StreamLatency.cpp | 103 ++++++ main/StreamLatency.h | 100 ++++++ main/main.cpp | 5 - main/test/TestStreamLatency.h | 209 +++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 11 + 10 files changed, 1376 insertions(+), 5 deletions(-) create mode 100644 main/OboeAudioIO.cpp create mode 100644 main/OboeAudioIO.h create mode 100644 main/StreamLatency.cpp create mode 100644 main/StreamLatency.h create mode 100644 main/test/TestStreamLatency.h diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index baef45aa..83977877 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -28,6 +28,8 @@ #ifdef Q_OS_ANDROID #include "AndroidFiles.h" +#include "OboeAudioIO.h" +#include #include #endif @@ -238,6 +240,14 @@ MainWindow::MainWindow(AudioMode audioMode, }); } +#ifdef Q_OS_ANDROID + m_audioDeviceReopens = 0; + m_audioDeviceCheck = new QTimer(this); + connect(m_audioDeviceCheck, &QTimer::timeout, + this, &MainWindow::checkAudioDevice); + m_audioDeviceCheck->start(250); +#endif + #ifdef Q_OS_MAC #if (QT_VERSION >= QT_VERSION_CHECK(5, 2, 0)) setUnifiedTitleAndToolBarOnMac(true); @@ -2760,6 +2770,124 @@ MainWindow::getOpenFileName(FileFinder::FileType type) } return copy; } + +void +MainWindow::createAudioIO() +{ + if (m_playTarget || m_audioIO) return; + if (m_audioMode == AUDIO_NONE) return; + + breakfastquay::ApplicationPlaybackSource *source = + m_playSource->getApplicationPlaybackSource(); + std::string error; + + // As MainWindowBase's: input and output once recording has been + // asked for, else the output alone, and the output alone too if + // there is no input to be had. Nor is there before the microphone + // may be used, which record() asks for first + if (m_audioMode == AUDIO_PLAYBACK_AND_RECORD && m_recordTarget && + microphoneAllowed()) { + OboeAudioIO *io = new OboeAudioIO(m_recordTarget, source); + if (io->isOK()) { + m_audioIO = io; + } else { + error = io->getStartupError(); + delete io; + } + } + + if (!m_audioIO) { + OboeAudioIO *io = new OboeAudioIO(nullptr, source); + if (io->isOK()) { + m_playTarget = io; + } else { + error = io->getStartupError(); + delete io; + } + } + + if (m_audioIO) { + m_audioIO->suspend(); + m_playSource->setSystemPlaybackTarget(m_audioIO); + } else if (m_playTarget) { + m_playTarget->suspend(); + m_playSource->setSystemPlaybackTarget(m_playTarget); + } else { + emit hideSplash(); + QMessageBox::warning + (this, tr("Couldn't open audio device"), + tr("No audio available

%1

Audio playback and recording will not be available.

") + .arg(QString::fromStdString(error).toHtmlEscaped())); + } +} + +bool +MainWindow::microphoneAllowed() const +{ + return qApp->checkPermission(QMicrophonePermission()) == + Qt::PermissionStatus::Granted; +} + +void +MainWindow::askForMicrophone() +{ + qApp->requestPermission + (QMicrophonePermission(), this, + [this](const QPermission &permission) { + if (permission.status() == Qt::PermissionStatus::Granted) { + // The press of Record that asked, answered at last + if (microphoneAllowed()) record(); + return; + } + QMessageBox::information + (this, tr("Microphone not allowed"), + tr("Recording needs the microphone

%1 may not use the microphone, so it can play but not record.

To allow it, open the phone's Settings, then Apps, %1, Permissions, Microphone.

") + .arg(QApplication::applicationName())); + }); +} + +void +MainWindow::checkAudioDevice() +{ + breakfastquay::SystemPlaybackTarget *device = m_audioIO; + if (!device) device = m_playTarget; + OboeAudioIO *io = dynamic_cast(device); + if (!io || !io->hasFailed()) return; + + // Whatever was going on stops. A take keeps what was sung, through + // the Stop path of the Record button + if (m_recordTarget && m_recordTarget->isRecording()) { + record(); + } else if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + + // Reopened up to three times in ten seconds, as a headset can move + // the output and then the input. More is a device that fails as + // soon as it is open: it is left closed rather than opened over and + // over, and the next Record or file opened tries again + if (!m_audioDeviceReopened.isValid() || + m_audioDeviceReopened.elapsed() > 10000) { + m_audioDeviceReopened.start(); + m_audioDeviceReopens = 0; + } + if (++m_audioDeviceReopens > 3) { + cerr << "MainWindow::checkAudioDevice: the audio device keeps " + << "failing; closing it" << endl; + m_audioDeviceReopened.invalidate(); + deleteAudioIO(); + updateMenuStates(); + QMessageBox::warning + (this, tr("Audio device failed"), + tr("The audio device stopped working

It failed again each time it was opened. Recording, or opening a file, will try to open it again.

")); + return; + } + + cerr << "MainWindow::checkAudioDevice: the audio device failed or " + << "went away; opening it again" << endl; + recreateAudioIO(); + updateMenuStates(); +} #endif void @@ -3943,6 +4071,16 @@ MainWindow::record() return; } +#ifdef Q_OS_ANDROID + // The microphone is asked for when it is first needed, and the + // answer comes later: the take is started then, from the top + if (!microphoneAllowed()) { + if (m_recordAction) m_recordAction->setChecked(false); + askForMicrophone(); + return; + } +#endif + // If a reference track is already loaded, record the microphone input // into the singing track rather than replacing the whole session. We // do that by switching to RecordCreateAdditionalModel for the call, so diff --git a/main/MainWindow.h b/main/MainWindow.h index 2362b64b..f9cce290 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -31,6 +31,10 @@ #include "data/model/SparseTimeValueModel.h" +#ifdef Q_OS_ANDROID +#include +#endif + class QTimer; class QComboBox; class QActionGroup; @@ -908,6 +912,22 @@ protected slots: // cannot open: the file picked is copied into the app's own storage, // and the copy's path returned QString getOpenFileName(sv::FileFinder::FileType type) override; + + // The audio device is Oboe's (OboeAudioIO): bqaudioio has no + // Android backend. Its input only once the microphone may be used + void createAudioIO() override; + + // The microphone is asked for when Record is first pressed, and the + // take is started once it is given + bool microphoneAllowed() const; + void askForMicrophone(); + + // A device that goes away (headphones in or out) leaves the device + // failed: looked for on a timer, and the device opened afresh + void checkAudioDevice(); + QTimer *m_audioDeviceCheck; + QElapsedTimer m_audioDeviceReopened; + int m_audioDeviceReopens; #endif // A session must not be saved in the middle of the analysis of a diff --git a/main/OboeAudioIO.cpp b/main/OboeAudioIO.cpp new file mode 100644 index 00000000..3c1e1983 --- /dev/null +++ b/main/OboeAudioIO.cpp @@ -0,0 +1,652 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "OboeAudioIO.h" + +#include +#include + +#include +#include + +#include + +#include +#include +#include +#include +#include +#include + +#include + +using namespace breakfastquay; +using std::cerr; +using std::endl; + +namespace { + +// Callbacks that must have reached the application since a start +// before the streams count as running as they will go on: by then +// FullDuplexStream has finished draining the input +const int steadyCallbacks = 8; + +// How long the constructor runs the streams, at most, for them to +// have timestamps +const int openWaitMillis = 1000; + +// Readings taken for one measurement of the latency +const int readingsWanted = 9; + +int64_t +monotonicNanos() +{ + // The clock of Oboe's timestamps + timespec ts; + clock_gettime(CLOCK_MONOTONIC, &ts); + return int64_t(ts.tv_sec) * 1000000000 + ts.tv_nsec; +} + +std::string +describe(StreamLatency::Estimate latency, bool withInput, int rate) +{ + auto ms = [rate](int frames) { return double(frames) * 1000.0 / rate; }; + std::ostringstream os; + os.setf(std::ios::fixed); + os.precision(1); + os << "output " << latency.output << " frames (" + << ms(latency.output) << " ms)"; + if (withInput) { + os << ", input " << latency.input << " frames (" + << ms(latency.input) << " ms), round trip " + << latency.roundTrip() << " frames (" + << ms(latency.roundTrip()) << " ms)"; + } + return os.str(); +} + +} + +// The output stream's data callback: input and output together once +// FullDuplexStream is done draining, or output alone +class OboeAudioIO::Engine : public oboe::FullDuplexStream +{ +public: + explicit Engine(OboeAudioIO *io) : m_io(io) { } + + // Whether the input is read. Changed only while the streams stop + std::atomic duplex { false }; + + // Odd while a callback runs, so that a latency reading can tell + // whether one ran while it was taken + std::atomic callbackSeq { 0 }; + + // Callbacks that have reached the application since the last start + std::atomic processed { 0 }; + + // The input could not be read, and FullDuplexStream has stopped + // both streams. Oboe reports no error for a stream that has no + // callback of its own, as the input has not + std::atomic inputFailed { false }; + + oboe::DataCallbackResult onAudioReady(oboe::AudioStream *stream, + void *audioData, + int32_t numFrames) override { + callbackSeq.fetch_add(1, std::memory_order_acq_rel); + oboe::DataCallbackResult result = + oboe::DataCallbackResult::Continue; + if (duplex.load(std::memory_order_relaxed)) { + result = FullDuplexStream::onAudioReady + (stream, audioData, numFrames); + if (result != oboe::DataCallbackResult::Continue) { + inputFailed.store(true); + } + } else { + m_io->process(nullptr, 0, + static_cast(audioData), numFrames); + processed.fetch_add(1, std::memory_order_relaxed); + } + callbackSeq.fetch_add(1, std::memory_order_acq_rel); + return result; + } + + oboe::DataCallbackResult onBothStreamsReady(const void *inputData, + int numInputFrames, + void *outputData, + int numOutputFrames) + override { + m_io->process(static_cast(inputData), + numInputFrames, + static_cast(outputData), numOutputFrames); + processed.fetch_add(1, std::memory_order_relaxed); + return oboe::DataCallbackResult::Continue; + } + +private: + OboeAudioIO *m_io; +}; + +// The output stream's error callback. Oboe calls it on a thread of its +// own, which holds the stream, and the stream holds this: so it may +// run after the OboeAudioIO has gone, and touches nothing of it +class OboeAudioIO::ErrorFlag : public oboe::AudioStreamErrorCallback +{ +public: + std::atomic failed { false }; + + void onErrorAfterClose(oboe::AudioStream *, oboe::Result) override { + failed.store(true); + } +}; + +OboeAudioIO::OboeAudioIO(ApplicationRecordTarget *target, + ApplicationPlaybackSource *source) : + SystemAudioIO(target, source), + m_engine(new Engine(this)), + m_errors(std::make_shared()), + m_epoch(std::chrono::steady_clock::now()), + m_rate(0), + m_sourceChannels(2), + m_outputChannels(0), + m_inputChannels(0), + m_maxFrames(0), + m_outBufferChannels(0), + m_outBuffers(nullptr), + m_inBuffers(nullptr), + m_gain(1.f), + m_balance(0.f), + m_suspended(true), + m_inputRunning(false), + m_recordSuppressed(false), + m_startFailed(false), + m_outputXRuns(0) +{ + if (m_source && m_source->getApplicationChannelCount() > 0) { + m_sourceChannels = m_source->getApplicationChannelCount(); + } + + // The output first, at the device's own rate; the input is opened + // at the same rate, the lowest latency path for both. Exclusive + // where the device allows it: AAudio opens a shared stream if not + oboe::AudioStreamBuilder outBuilder; + outBuilder.setDirection(oboe::Direction::Output) + ->setPerformanceMode(oboe::PerformanceMode::LowLatency) + ->setSharingMode(oboe::SharingMode::Exclusive) + ->setFormat(oboe::AudioFormat::Float) + ->setFormatConversionAllowed(true) + ->setChannelCount(2) + ->setChannelConversionAllowed(true) + ->setUsage(oboe::Usage::Media) + ->setContentType(oboe::ContentType::Music) + ->setDataCallback(m_engine.get()) + ->setErrorCallback(m_errors); + + oboe::Result result = outBuilder.openStream(m_output); + if (result == oboe::Result::OK && + m_output->getFormat() != oboe::AudioFormat::Float) { + result = oboe::Result::ErrorInvalidFormat; + m_output->close(); + } + if (result != oboe::Result::OK) { + m_startupError = std::string("Failed to open the audio output: ") + + oboe::convertToText(result); + cerr << "OboeAudioIO: " << m_startupError << endl; + m_output.reset(); + return; + } + + m_rate = m_output->getSampleRate(); + m_outputChannels = m_output->getChannelCount(); + + if (m_target) { + oboe::AudioStreamBuilder inBuilder; + inBuilder.setDirection(oboe::Direction::Input) + ->setPerformanceMode(oboe::PerformanceMode::LowLatency) + ->setSharingMode(oboe::SharingMode::Exclusive) + ->setFormat(oboe::AudioFormat::Float) + ->setFormatConversionAllowed(true) + ->setChannelCount(1) + ->setChannelConversionAllowed(true) + ->setSampleRate(m_rate) + // Low latency and no automatic gain or noise suppression. + // Android 10 and later; before that Oboe asks for + // VoiceRecognition, the nearest + ->setInputPreset(oboe::InputPreset::VoicePerformance) + // FullDuplexStream's advice + ->setBufferCapacityInFrames + (m_output->getBufferCapacityInFrames() * 2); + + result = inBuilder.openStream(m_input); + // FullDuplexStream reads the input into a buffer that has room + // for as many channels as the output has + if (result == oboe::Result::OK && + (m_input->getSampleRate() != m_rate || + m_input->getFormat() != oboe::AudioFormat::Float || + m_input->getChannelCount() > m_outputChannels)) { + result = oboe::Result::ErrorInvalidFormat; + m_input->close(); + } + if (result != oboe::Result::OK) { + m_startupError = std::string("Failed to open the audio input: ") + + oboe::convertToText(result); + cerr << "OboeAudioIO: " << m_startupError << endl; + m_input.reset(); + return; + } + m_inputChannels = m_input->getChannelCount(); + } + + int burst = m_output->getFramesPerBurst(); + m_maxFrames = std::max({ m_output->getBufferCapacityInFrames(), + burst, 1024 }); + m_outBufferChannels = std::max(m_sourceChannels, m_outputChannels); + m_outBuffers = allocate_and_zero_channels + (m_outBufferChannels, m_maxFrames); + if (m_input) { + m_inBuffers = allocate_and_zero_channels + (m_inputChannels, m_maxFrames); + m_engine->setSharedInputStream(m_input); + } + m_engine->setSharedOutputStream(m_output); + + // All of this before the first callback, as PortAudioIO does. The + // latency is a guess until it is measured below + m_latency = StreamLatency::guess + (m_output->getBufferSizeInFrames(), + m_input ? m_input->getFramesPerBurst() : 0); + + if (m_source) { + m_source->setSystemPlaybackBlockSize(burst); + m_source->setSystemPlaybackSampleRate(m_rate); + m_source->setSystemPlaybackLatency(m_latency.output); + // Not the stream's channel count: the wrappers svapp puts round + // its play source take this for the number of channels they + // will be asked for, and refuse any other. The mix to the + // stream's channels is done here + m_source->setSystemPlaybackChannelCount(m_sourceChannels); + } + if (m_target) { + m_target->setSystemRecordBlockSize(m_input->getFramesPerBurst()); + m_target->setSystemRecordSampleRate(m_rate); + m_target->setSystemRecordLatency(m_latency.input); + m_target->setSystemRecordChannelCount(m_inputChannels); + } + + logStream("output", m_output.get()); + if (m_input) logStream("input", m_input.get()); + + // Run the streams until they can say what their latency is, so that + // the first take is compensated by a measurement too, and leave + // them suspended: the application suspends a new device at once + // anyway (MainWindowBase::createAudioIO() does) + auto started = std::chrono::steady_clock::now(); + resume(); + if (m_startFailed) { + // Without the input, the caller can still have playback + m_startupError = "Failed to start the audio streams"; + cerr << "OboeAudioIO: " << m_startupError << endl; + if (m_input) { + m_input->close(); + m_input.reset(); + } else { + m_output->close(); + m_output.reset(); + } + return; + } + + bool measurable = waitUntilMeasurable(openWaitMillis); + int waited = int(std::chrono::duration_cast + (std::chrono::steady_clock::now() - started).count()); + StreamLatency::Estimate latency; + bool withInput = m_inputRunning; + bool measured = measurable && measureLatency(latency); + stopStreams(); + m_suspended = true; + + if (measured) { + report(latency, withInput); + cerr << "OboeAudioIO: latency measured " << waited + << " ms after starting: " + << describe(m_latency, m_input != nullptr, m_rate) << endl; + } else { + cerr << "OboeAudioIO: no timestamps " << waited + << " ms after starting; latency guessed: " + << describe(m_latency, m_input != nullptr, m_rate) << endl; + } +} + +OboeAudioIO::~OboeAudioIO() +{ + if (!m_suspended) stopStreams(); + + // Closing waits for a callback that is running; none comes after + if (m_output) m_output->close(); + if (m_input) m_input->close(); + m_engine.reset(); + + if (m_outBuffers) deallocate_channels(m_outBuffers, m_outBufferChannels); + if (m_inBuffers) deallocate_channels(m_inBuffers, m_inputChannels); +} + +bool +OboeAudioIO::isSourceOK() const +{ + // Without a record target the input is not wanted + return !m_target || m_input != nullptr; +} + +bool +OboeAudioIO::isTargetOK() const +{ + return m_output != nullptr; +} + +double +OboeAudioIO::getCurrentTime() const +{ + // The play source asks in the callback too: this reads a clock and + // nothing else + return std::chrono::duration + (std::chrono::steady_clock::now() - m_epoch).count(); +} + +bool +OboeAudioIO::hasFailed() const +{ + return m_errors->failed.load() || m_engine->inputFailed.load() || + m_startFailed; +} + +void +OboeAudioIO::resume() +{ + if (!m_suspended || !m_output || hasFailed()) return; + + bool duplex = (m_input && !m_recordSuppressed); + m_engine->duplex.store(duplex); + m_engine->processed.store(0); + + // FullDuplexStream starts the input, then the output, and starts + // draining the input again + oboe::Result result = + (duplex ? m_engine->start() : m_output->requestStart()); + m_inputRunning = duplex; + + if (result != oboe::Result::OK) { + cerr << "OboeAudioIO: failed to start: " + << oboe::convertToText(result) << endl; + stopStreams(); + m_startFailed = true; + return; + } + + m_suspended = false; +} + +void +OboeAudioIO::suspend() +{ + if (m_suspended || !m_output) return; + + // Measured while the streams still run, reported once they have + // stopped, so that nothing the audio thread reads changes under it. + // The application reads the figures when a take starts, so the next + // take is compensated by what the device did now + StreamLatency::Estimate latency; + bool withInput = m_inputRunning; + bool measured = (m_engine->processed.load() >= steadyCallbacks && + measureLatency(latency)); + + stopStreams(); + m_suspended = true; + + if (measured) { + StreamLatency::Estimate before = m_latency; + report(latency, withInput); + if (m_latency.output != before.output || + m_latency.input != before.input) { + cerr << "OboeAudioIO: latency now " + << describe(m_latency, m_input != nullptr, m_rate) << endl; + } + } + + oboe::ResultWithValue xruns = m_output->getXRunCount(); + if (xruns && xruns.value() != m_outputXRuns) { + m_outputXRuns = xruns.value(); + cerr << "OboeAudioIO: " << m_outputXRuns + << " output underrun(s) since the device was opened" << endl; + } +} + +void +OboeAudioIO::suppressRecordSide(bool suppress) +{ + if (suppress == m_recordSuppressed) return; + bool wasRunning = !m_suspended; + if (wasRunning) suspend(); + m_recordSuppressed = suppress; + if (wasRunning) resume(); +} + +void +OboeAudioIO::setOutputGain(float gain) +{ + SystemAudioIO::setOutputGain(gain); + m_gain.store(gain); +} + +void +OboeAudioIO::setOutputBalance(float balance) +{ + SystemAudioIO::setOutputBalance(balance); + m_balance.store(balance); +} + +void +OboeAudioIO::stopStreams() +{ + // The output first, as its callback reads the input. stop() waits + // until the stream has stopped, so no callback runs after this. + // A stream Oboe has closed after an error just says so + if (m_output) m_output->stop(); + if (m_input && m_inputRunning) m_input->stop(); + m_inputRunning = false; +} + +bool +OboeAudioIO::waitUntilMeasurable(int maxMillis) const +{ + auto deadline = std::chrono::steady_clock::now() + + std::chrono::milliseconds(maxMillis); + while (!m_suspended && !hasFailed()) { + if (m_engine->processed.load() >= steadyCallbacks) { + bool stamped = bool(m_output->getTimestamp(CLOCK_MONOTONIC)); + if (stamped && m_inputRunning) { + stamped = bool(m_input->getTimestamp(CLOCK_MONOTONIC)); + } + if (stamped) return true; + } + if (std::chrono::steady_clock::now() >= deadline) break; + std::this_thread::sleep_for(std::chrono::milliseconds(5)); + } + return false; +} + +bool +OboeAudioIO::measureLatency(StreamLatency::Estimate &latency) const +{ + // Readings are taken here, not in the callback: Oboe advises + // against timestamps there before Android 11. A reading that a + // callback ran through is thrown away, as it would pair one + // callback's output count with another's input count; so is one + // taken just after a callback returned and before Oboe counted what + // it wrote, by the median. + bool withInput = m_inputRunning; + std::vector readings; + for (int attempt = 0; + attempt < readingsWanted * 5 && int(readings.size()) < readingsWanted; + ++attempt) { + + if (attempt > 0) { + std::this_thread::sleep_for(std::chrono::microseconds(700)); + } + + uint32_t before = m_engine->callbackSeq.load(std::memory_order_acquire); + if (before & 1) continue; + + StreamLatency::Position output, input; + output.appFrames = m_output->getFramesWritten(); + if (withInput) input.appFrames = m_input->getFramesRead(); + + auto outputStamp = m_output->getTimestamp(CLOCK_MONOTONIC); + if (!outputStamp) continue; + output.hardwareFrame = outputStamp.value().position; + output.hardwareNanos = outputStamp.value().timestamp; + + if (withInput) { + auto inputStamp = m_input->getTimestamp(CLOCK_MONOTONIC); + if (!inputStamp) continue; + input.hardwareFrame = inputStamp.value().position; + input.hardwareNanos = inputStamp.value().timestamp; + } + + int64_t now = monotonicNanos(); + if (m_engine->callbackSeq.load(std::memory_order_acquire) != before) { + continue; + } + + readings.push_back(StreamLatency::fromReading + (StreamLatency::outputLatency(output, now, m_rate), + withInput ? + StreamLatency::inputLatency(input, now, m_rate) : + 0.0)); + } + + return StreamLatency::median(readings, m_rate, latency); +} + +void +OboeAudioIO::report(StreamLatency::Estimate latency, bool withInput) +{ + // Only while the streams are stopped: the play source's wrappers + // are not safe to change under the callback + m_latency.output = latency.output; + if (m_source) m_source->setSystemPlaybackLatency(m_latency.output); + if (withInput) { + m_latency.input = latency.input; + if (m_target) m_target->setSystemRecordLatency(m_latency.input); + } +} + +void +OboeAudioIO::logStream(std::string name, oboe::AudioStream *stream) const +{ + // Only an AAudio stream can be asked about MMAP + bool mmap = (stream->getAudioApi() == oboe::AudioApi::AAudio && + oboe::OboeExtensions::isMMapUsed(stream)); + cerr << "OboeAudioIO: " << name << ": " + << oboe::convertToText(stream->getAudioApi()) + << (mmap ? " (MMAP)" : "") + << ", " << stream->getSampleRate() << " Hz, " + << stream->getChannelCount() << " channel(s), " + << oboe::convertToText(stream->getFormat()) << ", " + << oboe::convertToText(stream->getPerformanceMode()) << ", " + << oboe::convertToText(stream->getSharingMode()) + << ", burst " << stream->getFramesPerBurst() + << ", buffer " << stream->getBufferSizeInFrames() + << " of " << stream->getBufferCapacityInFrames() << " frames"; + if (stream->getDirection() == oboe::Direction::Input) { + cerr << ", preset " << oboe::convertToText(stream->getInputPreset()); + } + cerr << endl; +} + +void +OboeAudioIO::process(const float *input, int inputFrames, + float *output, int outputFrames) +{ + // On the audio thread: no allocation, no lock, no logging. + // + // The input first. The application counts on having a block's + // input before it is asked for the block's output (the start gap, + // recording.md "Latency") + if (m_target && input && inputFrames > 0) { + float peakLeft = 0.f, peakRight = 0.f; + for (int done = 0; done < inputFrames; ) { + int n = std::min(inputFrames - done, m_maxFrames); + v_deinterleave(m_inBuffers, + input + size_t(done) * m_inputChannels, + m_inputChannels, n); + for (int c = 0; c < m_inputChannels && c < 2; ++c) { + float peak = 0.f; + for (int i = 0; i < n; ++i) { + peak = std::max(peak, std::fabs(m_inBuffers[c][i])); + } + if (c == 0) peakLeft = std::max(peakLeft, peak); + if (c == 1 || m_inputChannels == 1) { + peakRight = std::max(peakRight, peak); + } + } + m_target->putSamples(m_inBuffers, m_inputChannels, n); + done += n; + } + m_target->setInputLevels(peakLeft, peakRight); + } + + if (!output || outputFrames <= 0) return; + + if (!m_source) { + v_zero(output, outputFrames * m_outputChannels); + return; + } + + // As bqaudioio's Gains: the balance turns down the other side + float gain = m_gain.load(std::memory_order_relaxed); + float balance = m_balance.load(std::memory_order_relaxed); + float leftGain = (balance > 0.f ? gain * (1.f - balance) : gain); + float rightGain = (balance < 0.f ? gain * (1.f + balance) : gain); + + float peakLeft = 0.f, peakRight = 0.f; + for (int done = 0; done < outputFrames; ) { + int n = std::min(outputFrames - done, m_maxFrames); + int got = m_source->getSourceSamples + (m_outBuffers, m_sourceChannels, n); + got = std::max(0, std::min(got, n)); + if (got < n) { + for (int c = 0; c < m_sourceChannels; ++c) { + v_zero(m_outBuffers[c] + got, n - got); + } + } + v_reconfigure_channels_inplace + (m_outBuffers, m_outputChannels, m_sourceChannels, n); + for (int c = 0; c < m_outputChannels; ++c) { + float g = (c == 0 ? leftGain : c == 1 ? rightGain : gain); + v_scale(m_outBuffers[c], g, n); + if (c < 2) { + float peak = 0.f; + for (int i = 0; i < n; ++i) { + peak = std::max(peak, std::fabs(m_outBuffers[c][i])); + } + if (c == 0) peakLeft = std::max(peakLeft, peak); + if (c == 1 || m_outputChannels == 1) { + peakRight = std::max(peakRight, peak); + } + } + } + v_interleave(output + size_t(done) * m_outputChannels, + m_outBuffers, m_outputChannels, n); + done += n; + } + m_source->setOutputLevels(peakLeft, peakRight); +} diff --git a/main/OboeAudioIO.h b/main/OboeAudioIO.h new file mode 100644 index 00000000..871fdfe1 --- /dev/null +++ b/main/OboeAudioIO.h @@ -0,0 +1,136 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_OBOE_AUDIO_IO_H +#define TONY_OBOE_AUDIO_IO_H + +#include "StreamLatency.h" + +#include + +#include +#include +#include +#include + +namespace oboe { +class AudioStream; +} + +/** + * The audio device on Android, through Google's Oboe (AAudio): a + * bqaudioio SystemAudioIO, as PortAudioIO is on the desktop, which + * MainWindow::createAudioIO() installs; bqaudioio has no Android + * backend. Built for Android only. + * + * A stereo output stream at the device's own rate and, given a record + * target, a mono input stream at the same rate, read in the output + * stream's callback (oboe::FullDuplexStream). Every callback hands the + * target its input before it asks the source for output, as the + * start-gap measurement assumes (recording.md, "Latency"). Without a + * record target, or with the record side suppressed, the output runs + * alone. + * + * FullDuplexStream spends its first 50 or so callbacks after each + * start draining and discarding input, with the output silent, so that + * input is read as soon as it comes in: output starts some 100 ms + * after resume(), and the first input the target gets is from then. + * + * Latency, in frames at the device's rate like PortAudioIO's, is + * worked out from the streams' timestamps (StreamLatency): when the + * device is opened, for which the constructor runs the streams until + * they have timestamps and leaves them suspended, and again each time + * it is suspended after running, so that the next take is compensated + * by what the device did last. Timestamps are read on the calling + * thread, never in the callback. + * + * A stream that fails (a device disconnected: headphones plugged in + * or out) is stopped by Oboe; hasFailed() then says so, and the owner + * must delete this and open another. Every method but the callback's + * is for the GUI thread, which is the only one that logs, to stderr. + */ +class OboeAudioIO : public breakfastquay::SystemAudioIO +{ +public: + /** + * Open the output and, if target is not null, the input. Check + * isOK() afterwards: without the input it is false, and the caller + * is expected to try again without a target, for playback only. + */ + OboeAudioIO(breakfastquay::ApplicationRecordTarget *target, + breakfastquay::ApplicationPlaybackSource *source); + ~OboeAudioIO() override; + + bool isSourceOK() const override; + bool isTargetOK() const override; + double getCurrentTime() const override; + + void suspend() override; + void resume() override; + void suppressRecordSide(bool suppress) override; + + void setOutputGain(float gain) override; + void setOutputBalance(float balance) override; + + /// Why a stream could not be opened, or "" if both were + std::string getStartupError() const { return m_startupError; } + + /// Whether a stream has failed, so that this must be replaced + bool hasFailed() const; + +private: + class Engine; + class ErrorFlag; + + std::shared_ptr m_output; + std::shared_ptr m_input; + std::unique_ptr m_engine; + std::shared_ptr m_errors; + std::string m_startupError; + + std::chrono::steady_clock::time_point m_epoch; + int m_rate; + int m_sourceChannels; + int m_outputChannels; + int m_inputChannels; + int m_maxFrames; + int m_outBufferChannels; + float **m_outBuffers; + float **m_inBuffers; + + std::atomic m_gain; + std::atomic m_balance; + + bool m_suspended; + bool m_inputRunning; + bool m_recordSuppressed; + bool m_startFailed; + StreamLatency::Estimate m_latency; + int m_outputXRuns; + + // The callback: the input first, then the output + friend class Engine; + void process(const float *input, int inputFrames, + float *output, int outputFrames); + + void stopStreams(); + bool waitUntilMeasurable(int maxMillis) const; + bool measureLatency(StreamLatency::Estimate &latency) const; + void report(StreamLatency::Estimate latency, bool withInput); + void logStream(std::string name, oboe::AudioStream *stream) const; + + OboeAudioIO(const OboeAudioIO &) = delete; + OboeAudioIO &operator=(const OboeAudioIO &) = delete; +}; + +#endif diff --git a/main/StreamLatency.cpp b/main/StreamLatency.cpp new file mode 100644 index 00000000..79dacd24 --- /dev/null +++ b/main/StreamLatency.cpp @@ -0,0 +1,103 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "StreamLatency.h" + +#include +#include + +namespace StreamLatency +{ + +static double +framesSince(int64_t thenNanos, int64_t nowNanos, double rate) +{ + return double(nowNanos - thenNanos) * rate / 1.0e9; +} + +double +outputLatency(Position output, int64_t nowNanos, double rate) +{ + // The hardware has played up to hardwareFrame plus whatever has + // gone out since its timestamp; the rest of what was written is + // still to be heard + return double(output.appFrames - output.hardwareFrame) - + framesSince(output.hardwareNanos, nowNanos, rate); +} + +double +inputLatency(Position input, int64_t nowNanos, double rate) +{ + // The next frame to read came in (appFrames - hardwareFrame) frames + // after the timestamp's + return framesSince(input.hardwareNanos, nowNanos, rate) - + double(input.appFrames - input.hardwareFrame); +} + +Estimate +fromReading(double output, double input) +{ + // The sum is rounded once, so that it is the nearest to the exact + // round trip whichever way the parts round + Estimate e; + int roundTrip = int(std::lround(output + input)); + e.output = int(std::lround(output)); + e.input = roundTrip - e.output; + if (e.input < 0) { + e.output = roundTrip; + e.input = 0; + } + if (e.output < 0) { + e.input = roundTrip; + e.output = 0; + } + if (e.input < 0) { + e.input = 0; // nothing to keep: the round trip is negative + } + return e; +} + +bool +isPlausible(Estimate estimate, double rate) +{ + int roundTrip = estimate.roundTrip(); + return roundTrip > 0 && roundTrip <= rate; +} + +bool +median(std::vector readings, double rate, Estimate &chosen) +{ + readings.erase(std::remove_if(readings.begin(), readings.end(), + [rate](const Estimate &e) { + return !isPlausible(e, rate); + }), + readings.end()); + if (readings.empty()) return false; + std::sort(readings.begin(), readings.end(), + [](const Estimate &a, const Estimate &b) { + return a.roundTrip() < b.roundTrip(); + }); + chosen = readings[(readings.size() - 1) / 2]; + return true; +} + +Estimate +guess(int outputBufferFrames, int inputBurstFrames) +{ + Estimate e; + e.output = std::max(0, outputBufferFrames); + e.input = std::max(0, inputBurstFrames); + return e; +} + +} diff --git a/main/StreamLatency.h b/main/StreamLatency.h new file mode 100644 index 00000000..440469be --- /dev/null +++ b/main/StreamLatency.h @@ -0,0 +1,100 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_STREAM_LATENCY_H +#define TONY_STREAM_LATENCY_H + +#include +#include + +/** + * The arithmetic of a device's latency from its streams' timestamps, + * for OboeAudioIO: Android has no call that says what the latency is, + * only timestamps that say when a frame met the hardware. + * + * Every figure is in frames at the device's rate, the unit bqaudioio's + * backends report latency in (setSystemPlaybackLatency() and + * setSystemRecordLatency()). + * + * Nothing here touches a device, so all of it is tested without one + * (TestStreamLatency). + */ +namespace StreamLatency +{ + /** + * Where a stream is: the frames the application has written to an + * output stream, or read from an input stream, so far, and a + * timestamp: the frame at hardwareFrame left for the speaker, or + * came in from the microphone, at hardwareNanos (CLOCK_MONOTONIC). + */ + struct Position { + int64_t appFrames = 0; + int64_t hardwareFrame = 0; + int64_t hardwareNanos = 0; + }; + + /** + * An output stream's latency at nowNanos: how long the next frame + * the application writes will take to be heard. + */ + double outputLatency(Position output, int64_t nowNanos, double rate); + + /** + * An input stream's latency at nowNanos: how long ago the next + * frame the application reads came in from the microphone. + * Negative if it has not come in yet. + */ + double inputLatency(Position input, int64_t nowNanos, double rate); + + /// Output and input latency, in whole frames + struct Estimate { + int output = 0; + int input = 0; + int roundTrip() const { return output + input; } + }; + + /** + * One reading of the two latencies, taken at the same moment, as + * an estimate: rounded, and neither of them negative. + * + * Taken between two callbacks, the output latency comes out high + * and the input latency low by the same amount, the time to the + * next callback, so only their sum is exact. The sum is what a + * recording is compensated by (LatencyUtils.h) and is kept: a + * negative part is taken off the other. + */ + Estimate fromReading(double output, double input); + + /** + * Whether an estimate can be believed: a round trip longer than + * nothing and no longer than a second at the given rate. + */ + bool isPlausible(Estimate estimate, double rate); + + /** + * Of several readings, the plausible one with the median round + * trip: a reading spoilt by a callback that ran while it was taken + * is off by a callback's worth of frames, and is left out that + * way. False if none is plausible. + */ + bool median(std::vector readings, double rate, + Estimate &chosen); + + /** + * A guess for streams with no timestamps: what the output buffer + * holds, and one burst of input. + */ + Estimate guess(int outputBufferFrames, int inputBurstFrames); +} + +#endif diff --git a/main/main.cpp b/main/main.cpp index 2d1a69bd..e589582a 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -327,11 +327,6 @@ main(int argc, char **argv) if (args.contains("--no-audio")) audioOutput = false; -#ifdef Q_OS_ANDROID - // There is no audio backend for Android yet - audioOutput = false; -#endif - if (args.contains("--no-sonification")) sonification = false; if (args.contains("--no-spectrogram")) spectrogram = false; diff --git a/main/test/TestStreamLatency.h b/main/test/TestStreamLatency.h new file mode 100644 index 00000000..7d90f818 --- /dev/null +++ b/main/test/TestStreamLatency.h @@ -0,0 +1,209 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_STREAM_LATENCY_H +#define TEST_STREAM_LATENCY_H + +// Tier 2: a device's latency from its streams' timestamps, as the +// Android backend works it out. Counters and timestamps in, frames out; +// no device: a simulated one whose true latencies are known. + +#include "../StreamLatency.h" + +#include +#include + +#include +#include + +class TestStreamLatency : public QObject +{ + Q_OBJECT + + // A duplex device at 48 kHz with callbacks of 96 frames, every 2 ms + // from t0. Each callback reads 96 frames of input and writes 96 of + // output. The frame written in callback k is heard trueOutput frames + // after the callback; the frame read in callback k came in trueInput + // frames before it. The hardware's timestamps are exact. + struct Device { + static constexpr double rate = 48000.0; + static constexpr int burst = 96; + static constexpr int64_t t0 = 5000000000; // ns + double trueOutput = 250.0; + double trueInput = 130.0; + + static int64_t nanos(double frames) { + return int64_t(std::llround(frames * 1.0e9 / rate)); + } + // When output frame f is heard, and input frame f came in + int64_t heard(int64_t f) const { + return t0 + nanos(double(f) + trueOutput); + } + int64_t cameIn(int64_t f) const { + return t0 + nanos(double(f) - trueInput); + } + // What the streams report at a moment after callback k has run + // and before k + 1 does: every frame of callback k is written + // and read, and the timestamps are of frames some time back + StreamLatency::Position output(int k) const { + StreamLatency::Position p; + p.appFrames = int64_t(k + 1) * burst; + p.hardwareFrame = p.appFrames - 400; + p.hardwareNanos = heard(p.hardwareFrame); + return p; + } + StreamLatency::Position input(int k) const { + StreamLatency::Position p; + p.appFrames = int64_t(k + 1) * burst; + p.hardwareFrame = p.appFrames - 700; + p.hardwareNanos = cameIn(p.hardwareFrame); + return p; + } + // The moment a fraction of the way from callback k to k + 1 + int64_t between(int k, double fraction) const { + return t0 + nanos((double(k) + fraction) * burst); + } + }; + + static StreamLatency::Estimate est(int output, int input) { + StreamLatency::Estimate e; + e.output = output; + e.input = input; + return e; + } + +private slots: + void output_latency_is_what_is_written_and_not_yet_heard() { + // 1000 frames written beyond the timestamp's frame, which went + // out 10 ms (480 frames) ago + StreamLatency::Position p; + p.appFrames = 10000; + p.hardwareFrame = 9000; + p.hardwareNanos = 1000000000; + QCOMPARE(StreamLatency::outputLatency(p, 1010000000, 48000.0), 520.0); + } + + void input_latency_is_the_age_of_the_next_frame_read() { + // The next frame to read came in 1000 frames after the + // timestamp's, and 25 ms (1200 frames) have gone by since it + StreamLatency::Position p; + p.appFrames = 10000; + p.hardwareFrame = 9000; + p.hardwareNanos = 1000000000; + QCOMPARE(StreamLatency::inputLatency(p, 1025000000, 48000.0), 200.0); + // Read so eagerly that the next frame is not in yet + QCOMPARE(StreamLatency::inputLatency(p, 1015000000, 48000.0), -280.0); + } + + // Read at any moment between two callbacks, each part is off by the + // time to the next callback, and their sum is the true round trip + void round_trip_does_not_depend_on_when_it_is_read() { + Device d; + for (int k : { 10, 11, 500 }) { + for (double fraction : { 0.05, 0.3, 0.6, 0.95 }) { + int64_t now = d.between(k, fraction); + double out = StreamLatency::outputLatency + (d.output(k), now, Device::rate); + double in = StreamLatency::inputLatency + (d.input(k), now, Device::rate); + QVERIFY(std::fabs(out + in - (d.trueOutput + d.trueInput)) + < 0.01); + // the time to the next callback, one way and the other + double toNext = (1.0 - fraction) * Device::burst; + QVERIFY(std::fabs(out - (d.trueOutput + toNext)) < 0.01); + QVERIFY(std::fabs(in - (d.trueInput - toNext)) < 0.01); + } + } + } + + // What the figure is for: the singer answers output frame W the + // moment it is heard, and the answer comes in at input frame R plus + // the round trip, R and W being where the two streams stand at the + // same callback (recording.md, "Latency") + void round_trip_is_where_an_answer_to_the_output_lands() { + Device d; + d.trueOutput = 1234.0; + d.trueInput = 321.0; + int k = 40; + int64_t now = d.between(k, 0.7); + StreamLatency::Estimate e = StreamLatency::fromReading + (StreamLatency::outputLatency(d.output(k), now, Device::rate), + StreamLatency::inputLatency(d.input(k), now, Device::rate)); + + int64_t w = d.output(k).appFrames; // the next frame to be written + int64_t r = d.input(k).appFrames; // and to be read + // The input frame that came in when output frame w was heard + int64_t answer = r; + while (d.cameIn(answer) < d.heard(w)) ++answer; + QCOMPARE(r + e.roundTrip(), answer); + } + + void reading_is_rounded_once() { + StreamLatency::Estimate e = StreamLatency::fromReading(100.4, 50.4); + QCOMPARE(e.roundTrip(), 151); + QCOMPARE(e.output, 100); + QCOMPARE(e.input, 51); + } + + // Bqaudioio's users take a negative latency as none (LatencyUtils.h + // does), which would lose it from the sum + void negative_part_is_taken_off_the_other() { + StreamLatency::Estimate e = StreamLatency::fromReading(300.4, -50.2); + QCOMPARE(e.output, 250); + QCOMPARE(e.input, 0); + + e = StreamLatency::fromReading(-20.0, 500.0); + QCOMPARE(e.output, 0); + QCOMPARE(e.input, 480); + + e = StreamLatency::fromReading(-20.0, -30.0); + QCOMPARE(e.output, 0); + QCOMPARE(e.input, 0); + } + + void plausible_is_more_than_nothing_and_at_most_a_second() { + QVERIFY(StreamLatency::isPlausible(est(200, 100), 48000.0)); + QVERIFY(StreamLatency::isPlausible(est(48000, 0), 48000.0)); + QVERIFY(!StreamLatency::isPlausible(est(48000, 1), 48000.0)); + QVERIFY(!StreamLatency::isPlausible(est(0, 0), 48000.0)); + } + + void median_takes_the_middle_round_trip() { + StreamLatency::Estimate chosen; + // One reading spoilt by a callback, one burst low; one absurd + std::vector readings { + est(260, 120), est(250, 130), est(160, 124), + est(900000, 0), est(270, 110), est(255, 127), + }; + QVERIFY(StreamLatency::median(readings, 48000.0, chosen)); + QCOMPARE(chosen.roundTrip(), 380); + + readings = { est(100, 0) }; + QVERIFY(StreamLatency::median(readings, 48000.0, chosen)); + QCOMPARE(chosen.output, 100); + + readings = { est(0, 0), est(-5, 0) }; + QVERIFY(!StreamLatency::median(readings, 48000.0, chosen)); + QVERIFY(!StreamLatency::median({}, 48000.0, chosen)); + } + + void guess_is_the_output_buffer_and_a_burst_of_input() { + StreamLatency::Estimate e = StreamLatency::guess(192, 96); + QCOMPARE(e.output, 192); + QCOMPARE(e.input, 96); + e = StreamLatency::guess(-1, -1); + QCOMPARE(e.roundTrip(), 0); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 1cb7ea68..ae9dab5a 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -17,6 +17,7 @@ #include "TestLatencyShift.h" #include "TestCoverage.h" #include "TestPinchZoom.h" +#include "TestStreamLatency.h" #include "TestTakeAudio.h" #include "TestTakeEvents.h" #include "TestSingingTakes.h" @@ -82,6 +83,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestStreamLatency t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestTakeAudio t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 2ba1c5d2..bf4e5a71 100644 --- a/meson.build +++ b/meson.build @@ -498,6 +498,8 @@ elif system == 'android' mad_dep = dependency('mad', version: '>= 0.15.0', static: true) id3tag_dep = dependency('id3tag', version: '>= 0.15.0', static: true) opus_dep = dependency('opusfile', static: true) + # The audio device: bqaudioio has no Android backend (main/OboeAudioIO) + oboe_dep = dependency('oboe', static: true) feature_dependencies = [ vamphostsdk_dep, @@ -511,6 +513,7 @@ elif system == 'android' mad_dep, id3tag_dep, opus_dep, + oboe_dep, ] feature_defines = [ @@ -1170,6 +1173,7 @@ tony_core_files = [ 'main/PinchZoom.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', + 'main/StreamLatency.cpp', 'main/TakeAudio.cpp', 'main/TakeEvents.cpp', 'main/TakesFile.cpp', @@ -1189,6 +1193,12 @@ tony_app_files = [ 'main/TouchGestures.cpp', ] +if system == 'android' + tony_app_files += [ + 'main/OboeAudioIO.cpp', + ] +endif + tony_core_moc_files = qt.preprocess( moc_headers: [ 'main/RealtimePitchTracker.h', @@ -1471,6 +1481,7 @@ if system != 'android' 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', 'main/test/TestPinchZoom.h', + 'main/test/TestStreamLatency.h', 'main/test/TestTakeAudio.h', 'main/test/TestTakeEvents.h', 'main/test/TestSingingTakes.h', From bd3524cdc0846bce35c1f1c994f3fcb047c38519 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 04:18:15 +0000 Subject: [PATCH 139/275] docs: phase a6 done, its log entry, and a3b's apk built Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 53b4396c..7599daee 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -153,12 +153,12 @@ builds happen in the container.) - A1 — Sample rate: a device that is not at 44.1 kHz. Done. - A2 — Android toolchain and C libraries. Done. - A3a — Tony builds for Android. Done. -- A3b — Tony as an APK (no audio): the test port. Built but for the APK itself: Gradle - was blocked (`dl.google.com` refused); run `deploy/android/build-apk.sh` once it is - allowed (see the log). +- A3b — Tony as an APK (no audio): the test port. Done; the lead built the APK once + `dl.google.com` was allowed again, and the user's first phone test passed (findings in + A7). - A4 — Touch gestures on the panes. Done. - A5 — Compact touch mode. Done. -- A6 — Oboe audio backend. +- A6 — Oboe audio backend. Done. - A7 — Sessions in place on the phone, and fixes from the first phone test. - A8 — Documentation pass. @@ -534,3 +534,29 @@ The next phase must know: switching off restores what switching on saved; the to Left open: not seen on a phone: the take box's height (22 px in Fusion), popup menu rows, and under ~710 dp wide the last buttons go into the toolbar's extension. For A8: architecture.md, testing.md (icons in the app tests), mobile-port.md, manual-checklist.md. + +### Phase A6 — 2026-09-26 +Built: `build-deps.sh` builds Oboe 1.11.0 (`liboboe.a`, `oboe.pc` with `-llog`; Oboe dlopens +AAudio and OpenSL ES). `main/OboeAudioIO` (Android only): stereo float output at the device's +rate, mono float input at that rate (`VoicePerformance`), low latency, exclusive (AAudio falls +back to shared), `FullDuplexStream`; each callback hands over its input, then asks for output. +`main/StreamLatency` (tony_core, `TestStreamLatency`). `MainWindow` on Android: `createAudioIO()` +(duplex once recording is asked for and the microphone allowed, else output only, as the base); +`record()` asks for `QMicrophonePermission` first, starts the take on Granted, else says where +to allow it; a 250 ms timer finds a failed device, stops the take (Stop path) or playback, and +reopens it (3 times in 10 s at most). main.cpp: the `AUDIO_NONE` forcing is gone. +Choices / deviations: +- Latency in device frames at the device's rate, as PortAudioIO. Read from the timestamps on + the GUI thread (Oboe: not in the callback), 9 readings, those a callback ran through dropped, + median round trip. Each part is off by the time to the next callback, the sum is exact + (tested). Measured when opened (the constructor runs the streams up to 1 s and leaves them + suspended) and at every suspend(): a take uses the figures of the device's last run. +- `setSystemPlaybackChannelCount()` gets the play source's count, not the stream's: svapp's + wrappers refuse (or throw on) any other count in `getSourceSamples()`. Mixed to 2 here. +- FullDuplexStream drains and discards ~50 callbacks after each start: sound starts ~0.1-0.25 s + after Play or Record. A callback may read fewer input frames than it writes; the start gap + subtracts the output block, so it can be up to one burst (2-5 ms) short. +- No error callback for the input (it has no callback): FullDuplexStream's Stop is flagged. +The next phase must know: once a take is recorded the device stays duplex (svapp), so Play +opens the microphone too; nothing calls `suppressRecordSide()`. Logcat tag Tony: "OboeAudioIO:". +Left open: not run on a phone; the first Record press blocks for the open (up to ~1 s). From 437013149905d32c185a013ae21f483680b3c403 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 04:32:38 +0000 Subject: [PATCH 140/275] feat: dev checks for latency after save and reopen, and several phrases Development builds (any build type but release) get TONY_DEV_CHECKS and main/dev/DevChecks: stages driven by a timer and the audio check's runner, never a nested event loop. After a calibration, Calibrate Audio carries on into them with the round trip it measured, for the run only: two punch-ins on the dev reference, then the session saved into a scratch folder and reopened. Items 1 and 2 of the manual checklist are judged from the take's file; the report goes to DevChecks.txt with a Totals line. The runner gains explicit ranges, recording into the session open, and a round trip of its own. A check's own session is replaced without the save question, and Record is shut while a check runs, since a press stopped the check's take. The reference in use is found through the model's location: getLocalFilename() is the decoded copy, so Check Again wrote over the reference that was open. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 16 + docs/calibrate-audio.md | 8 +- main/AudioCheckRunner.cpp | 204 +++++-- main/AudioCheckRunner.h | 84 ++- main/CalibrateAudioDialog.cpp | 203 ++++++- main/CalibrateAudioDialog.h | 57 +- main/LatencyUtils.h | 17 +- main/MainWindow.cpp | 122 ++++- main/MainWindow.h | 34 ++ main/dev/DevChecks.cpp | 788 ++++++++++++++++++++++++++++ main/dev/DevChecks.h | 301 +++++++++++ main/test/TestAudioCheck.h | 340 +++++++++++- main/test/TestDevChecks.h | 631 ++++++++++++++++++++++ main/test/TestRecordWorkflow.h | 20 + main/test/tony-app-test.cpp | 11 + meson.build | 50 +- 16 files changed, 2801 insertions(+), 85 deletions(-) create mode 100644 main/dev/DevChecks.cpp create mode 100644 main/dev/DevChecks.h create mode 100644 main/test/TestDevChecks.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 8a43b6f1..00b34dac 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -709,3 +709,19 @@ Left open: every threshold untuned; nothing calls `TakeDiff` yet. - C1 is split into C1a (framework, items 1 and 2) and C1b (observer, items 7, 12, 13, 14); spec §7 says so. - DevChecks runs as stages driven by the runner and a timer, not a nested event loop; scratch folders stay with the open session and the next run removes the old ones. Spec §5 and §8 changed to match. - C0's warning about item 10 (a 10 ms dip at two punch-ins' join; notes merged by onset) stands for C3. + +### Phase C1a — 2026-09-26 +Built: `meson.build`: `dev_checks` = build type not starting with `release`; then `-DTONY_DEV_CHECKS` in `general_defines` and moc, `main/dev/DevChecks.cpp`, `TestDevChecks.h`. Runner: `Plan::ranges` (`punchInsOf()`), `keepSession`, `roundTrip` (window's `m_audioCheckRoundTrip`, read only in the `recordingStarted()` lambda), `abandon()`, public `analysing()` and `readTakeFile()`, no save question for a check's own session. `TakeLatency::startGap/startGapMeasured`. `MainWindow`: Record's action calls `recordPressed()`, which ignores presses while `audioCheckRunning()`; Record greyed then; `m_devChecks` (friend). `main/dev/DevChecks.{h,cpp}`: stages, `CheckResult`, `DevReport`, items 1 `latency_on_this_machine` and 2 `several_phrases_in_one_take`, `DevChecks.txt`, `nextScratchFolder()`. Dialog: checkbox, dev run, `setDevChecks()`, `setDevOptions()`. Tests: 6 in `TestAudioCheck` (~27 s), `TestDevChecks` 7 (~66 s). +Choices / deviations: +- Stage 1 ranges [6.3, 10.2] and [16.8, 21.2] s (sweeps 7.2/9.1, 17.7/20.1; 50 ms spare). Free: before 4.3 s (P = 1 s judging 3.1 s), 10.2–16.8 and 21.2–25 s, the held tones. A re-record starting at 17.9–19.2 s has its lead-in over B's first sweep and still judges its second. +- A stage that fails or times out ends the run: `failure` names it ("Stage 2 of 2, "Save and reopen", did not finish within 60 s."), and every check whose data is missing is Skipped with that text. Checks are worked out at the end: item 2 needs stage 1 only, item 1 both. +- Record goes through `recordPressed()`, not a guard in `record()`: the runner and `pollTakeProgress()` call `record()` during the check's take too, and it cannot tell them from a press. +- After the reopen pitch and notes are restored, not analysed: compared by value with values rounded as `Event::toXml()` writes them; offsets to the frame. Save: `saveSessionToPath()` (a dialog only on failure). +- A run's own round trip counts as `TakeLatency::measured`. The dev reference goes to `referenceDirectory()`, so a cancelled dev run leaves a session the next check replaces without asking. +- `TestAudioCheck`'s fixture deletes each window's `DevChecks`: in a dev build its B4 dialog tests would carry on into them (the checkbox is on) and write `DevChecks.txt` into the log directory. +The next phase must know: +- **B3 bug fixed:** `getLocalFilename()` is the decoded cache copy (spec §11), so `nextReferencePath()` never kept the open reference: always `-1`, written over while open. `mainModelFile()` now. +- On the fake with its true round trip every sweep lands at 0 frames; the fake's reported pair is 2.8 ms short, so a run ignoring its round trip fails item 1. +- `latencyInUse()`'s reported pair differs before any file is open: store test fingerprints after opening one. +- The watchdog answers "Session modified" with No; `QStandardPaths::setTestModeEnabled()` keeps references out of the test app's data directory. +Left open: the app suite now takes about 7m40s; the dev checks add about 66 s, over spec §6's minute. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 4a56aeb2..e92c3490 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -40,6 +40,8 @@ with less noise. 2. **Test session.** Tony writes a generated test reference and opens it the way File ▸ Open does. You are asked to save your work first. The run never touches your song's takes or undo history. + *Since C1a:* not when the session open is a check's own (never saved, its reference + in the check's directory): that is replaced without asking. 3. **Calibration, about 30 s.** Four punch-ins, each spanning three of the reference's sweeps with the room the finder needs around them. The ranges come from the layout, not from fixed times: a 5 s punch-in judges only one or two sweeps. They use the @@ -305,7 +307,7 @@ marked "Done" when it is committed. 4. **Dev-check framework:** - **C0** `TakeDiff`, pure. Done. - **C1a** Build flag, `DevChecks`, report, friend access, the dialog's dev run. - Items 1 and 2. + Items 1 and 2. Done. - **C1b** `TakeObserver`. Items 7, 12, 13, 14. 5. **C2** Observer group: items 3, 4, 5, 8, 15, 16. 6. **C3** Join and long-song group: items 9 and 10. @@ -426,6 +428,10 @@ Checked on 2026-09-25, so that phases do not re-derive them. - `checkSaveModified()` is what asks the user to save. - The reference is analysed when `Analyser::getInitialAnalysisCompletion() >= 100` and the layers exist. See `analysed()` in `TestRecordWorkflow.h`. + - *Found in C1a:* with "normalise audio" on, a model's `getLocalFilename()` is the + reader's decoded copy in the temporary directory, not the file opened. The file + opened is `getLocation()` resolved through `FileSource` + (`AudioCheckRunner::mainModelFile()`). - **The take after Stop.** - Its audio is the model `analyser2()->getMainModelId()`, and its file is `m_takes->getAudioPath()`. *Found in B1:* the model is normalised to full scale diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index da468a1f..a81842ff 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -77,17 +77,26 @@ AudioCheckRunner::AudioCheckRunner(MainWindow *window) : AudioCheckRunner::~AudioCheckRunner() { - // Deleted by the window first thing in its destructor, so the window - // is whole here. A run ends without a word: whoever was waiting for - // it goes with the window. A take in progress is left to the window, - // as any take is when it is deleted: stopping it here would splice - // it and start its analysis in the middle of the window's teardown - m_timer->stop(); + // Deleted by the window in its destructor, before anything it reads + // goes, so the window is whole here. A run ends without a word: + // whoever was waiting for it goes with the window. A take in + // progress is left to the window, as any take is when it is deleted: + // stopping it here would splice it and start its analysis in the + // middle of the window's teardown if (m_step != Step::Idle) { cerr << "AudioCheckRunner: the window is going; the run ends" << endl; - m_window->m_audioCheckTakes = false; - m_step = Step::Idle; } + abandon(); +} + +void +AudioCheckRunner::abandon() +{ + m_timer->stop(); + if (m_step == Step::Idle) return; + cerr << "AudioCheckRunner: the run is abandoned" << endl; + clearOverride(); + m_step = Step::Idle; } AudioCheckRunner::Plan @@ -100,6 +109,29 @@ AudioCheckRunner::calibrationPlan() return plan; } +vector +AudioCheckRunner::punchInsOf(const Plan &plan) +{ + if (plan.ranges.empty()) { + return LatencyCheck::punchInsFor + (plan.layout, plan.punchIns, plan.eventsEach); + } + if (plan.layout.rate <= 0) return {}; + + // One after another along the timeline, as judgeTake() is told they + // were recorded. Two may meet, as takes into adjacent selections do. + // Written so that a range of NaNs fails too + const double length = double(plan.layout.length) / plan.layout.rate; + double free = 0.0; + for (const LatencyCheck::PunchIn &p : plan.ranges) { + if (!(p.start >= free) || !(p.end > p.start) || !(p.end <= length)) { + return {}; + } + free = p.end; + } + return plan.ranges; +} + QString AudioCheckRunner::referenceDirectory() { @@ -143,12 +175,23 @@ AudioCheckRunner::start(const Plan &plan) return false; } - vector punchIns = LatencyCheck::punchInsFor - (plan.layout, plan.punchIns, plan.eventsEach); + vector punchIns = punchInsOf(plan); if (punchIns.empty()) { - cerr << "AudioCheckRunner::start: the layout has no room for " - << plan.punchIns << " punch-ins of " << plan.eventsEach - << " events" << endl; + if (plan.ranges.empty()) { + cerr << "AudioCheckRunner::start: the layout has no room for " + << plan.punchIns << " punch-ins of " << plan.eventsEach + << " events" << endl; + } else { + cerr << "AudioCheckRunner::start: the punch-ins given overlap, " + << "are out of order, are empty or reach outside the layout" + << endl; + } + return false; + } + + if (plan.keepSession && !m_window->getMainModel()) { + cerr << "AudioCheckRunner::start: there is no session to record into" + << endl; return false; } @@ -167,8 +210,16 @@ AudioCheckRunner::start(const Plan &plan) m_ends.clear(); m_punchIn = 0; - cerr << "AudioCheckRunner::start: " << m_punchIns.size() - << " punch-ins of " << m_plan.eventsEach << " events" << endl; + cerr << "AudioCheckRunner::start: " << m_punchIns.size() << " punch-ins"; + if (m_plan.ranges.empty()) { + cerr << " of " << m_plan.eventsEach << " events"; + } + if (m_plan.keepSession) cerr << ", into the session open now"; + if (m_plan.roundTrip >= 0.0) { + cerr << ", placed with a round trip of its own, " + << m_plan.roundTrip * 1000.0 << " ms"; + } + cerr << endl; // Nothing is done before the first poll, so that however the run // ends, the caller hears of it through finished() and never from @@ -263,9 +314,26 @@ AudioCheckRunner::poll() void AudioCheckRunner::openReference() { + // The session open now is the reference, as the caller says, and + // stays open with its take + if (m_plan.keepSession) { + if (!m_window->getMainModel()) { + end(tr("There is no session to record into.")); + return; + } + referenceOpen(); + return; + } + // Each call below may show a dialog, and while one is up the run may - // end: the session closed, or the check cancelled - if (!m_window->checkSaveModified()) { + // end: the session closed, or the check cancelled. A check's own + // session holds nothing of the user's, and its takes have marked it + // modified: it is let go as answering "No" does + if (sessionIsACheck()) { + cerr << "AudioCheckRunner: replacing the session of an earlier " + << "check without asking" << endl; + m_window->m_documentModified = false; + } else if (!m_window->checkSaveModified()) { if (m_step == Step::OpeningReference) { end(tr("Your session was kept open, so the check did not " "replace it.")); @@ -280,12 +348,7 @@ AudioCheckRunner::openReference() // file has that one written over QString path = m_plan.referencePath; if (path == "") { - QString inUse; - if (auto model = std::dynamic_pointer_cast - (m_window->getMainModel())) { - inUse = model->getLocalFilename(); - } - path = nextReferencePath(referenceDirectory(), inUse); + path = nextReferencePath(referenceDirectory(), mainModelFile()); m_plan.referencePath = path; } cerr << "AudioCheckRunner: writing the reference to " << path << endl; @@ -308,15 +371,49 @@ AudioCheckRunner::openReference() m_openingReference = false; if (m_step != Step::OpeningReference) return; - auto model = m_window->getMainModel(); - if (status != MainWindowBase::FileOpenSucceeded || !model) { + if (status != MainWindowBase::FileOpenSucceeded || + !m_window->getMainModel()) { end(tr("The test reference \"%1\" could not be opened.").arg(path)); return; } + referenceOpen(); +} + +bool +AudioCheckRunner::sessionIsACheck() const +{ + // Saved, it is the user's to keep, whatever it holds + if (m_window->m_sessionFile != "") return false; + + const QString file = mainModelFile(); + if (file == "") return false; + + // Canonical paths, where both exist + return QFileInfo(file).absoluteDir() == QDir(referenceDirectory()); +} + +QString +AudioCheckRunner::mainModelFile() const +{ + // Not ReadOnlyWaveFileModel::getLocalFilename(): with the "normalise + // audio" preference on, as Tony has it, that is the file the reader + // decoded the audio into, in the temporary directory. The location + // is what the model was opened from, resolved here as it was then + auto model = std::dynamic_pointer_cast + (m_window->getMainModel()); + if (!model) return ""; + FileSource source(model->getLocation()); + if (source.isRemote()) return ""; + return source.getLocalFilename(); +} + +void +AudioCheckRunner::referenceOpen() +{ // The punch-ins from here on are as the takes record them: in whole // frames of the session, which is what the selections are made of - const sv_samplerate_t rate = model->getSampleRate(); + const sv_samplerate_t rate = m_window->getMainModel()->getSampleRate(); m_result.referenceRate = rate; for (LatencyCheck::PunchIn &p : m_punchIns) { m_starts.push_back(sv_frame_t(std::llround(p.start * rate))); @@ -325,7 +422,7 @@ AudioCheckRunner::openReference() double(m_ends.back()) / rate); } - // Its layers were made as it opened + // Its layers were made as it opened, or are there already setPlayback(); setStep(Step::AnalysingReference, kReferenceTimeoutMs); @@ -360,6 +457,7 @@ AudioCheckRunner::startPunchIn() setPlayback(); m_window->m_audioCheckTakes = true; + m_window->m_audioCheckRoundTrip = m_plan.roundTrip; m_window->record(); if (m_step != step) return; @@ -380,7 +478,7 @@ AudioCheckRunner::startPunchIn() void AudioCheckRunner::takeStopped() { - m_window->m_audioCheckTakes = false; + clearOverride(); m_result.takes.push_back(m_window->m_takeLatency); // The splice went wrong (the window has said so), or the recording @@ -398,34 +496,49 @@ AudioCheckRunner::takeStopped() void AudioCheckRunner::judge() { - // The take's own file, and not its model: the model is normalised to - // full scale as it is read (the "normalise audio" preference), which - // would have every take clipped, and resampled to the session's rate. // The file holds what was recorded, at the rate it was recorded at - QString path = m_window->m_takes->getAudioPath(); + vector mono; + sv_samplerate_t rate = 0; + const QString error = + readTakeFile(m_window->m_takes->getAudioPath(), mono, rate); + if (error != "") { + end(error); + return; + } + + m_result.summary = LatencyCheck::judgeTake + (m_plan.layout, mono.data(), sv_frame_t(mono.size()), rate, + m_punchIns); + + end(""); +} + +QString +AudioCheckRunner::readTakeFile(QString path, vector &mono, + sv_samplerate_t &rate) +{ + mono.clear(); + rate = 0; + FileSource source(path); WavFileReader reader(source); if (!reader.isOK() || reader.getChannelCount() < 1) { - end(tr("The take's audio file \"%1\" could not be read: %2") - .arg(path).arg(reader.getError())); - return; + return tr("The take's audio file \"%1\" could not be read: %2") + .arg(path).arg(reader.getError()); } const int channels = reader.getChannelCount(); const floatvec_t data = reader.getInterleavedFrames (0, reader.getFrameCount()); const sv_frame_t count = sv_frame_t(data.size()) / channels; - vector mono(count, 0.f); + mono.assign(count, 0.f); for (sv_frame_t i = 0; i < count; ++i) { float sum = 0.f; for (int c = 0; c < channels; ++c) sum += data[i * channels + c]; mono[i] = sum / float(channels); } - - m_result.summary = LatencyCheck::judgeTake - (m_plan.layout, mono.data(), count, reader.getSampleRate(), m_punchIns); - - end(""); + rate = reader.getSampleRate(); + return ""; } void @@ -546,7 +659,7 @@ AudioCheckRunner::end(QString failure) m_timer->stop(); m_step = Step::Idle; - m_window->m_audioCheckTakes = false; + clearOverride(); m_result.failure = failure; if (!m_result.takes.empty()) { @@ -587,6 +700,13 @@ AudioCheckRunner::end(QString failure) emit finished(result); } +void +AudioCheckRunner::clearOverride() +{ + m_window->m_audioCheckTakes = false; + m_window->m_audioCheckRoundTrip = -1.0; +} + bool AudioCheckRunner::analysing(Analyser *a) { diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index 6dd5d316..4b5bb2c2 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -115,7 +115,9 @@ struct AudioCheckResult * Driven by a polling timer, like MainWindow's own take polling, and * never by a nested event loop: this runs in every build, and the * window can be closed at any moment. MainWindow owns it, deletes it - * first thing in its destructor, and tells it when the session closes. + * in its destructor after those who drive it (the dialog, the dev + * checks) and before anything it reads, and tells it when the session + * closes. * A friend of MainWindow: it drives the window's take path, reads what * the take was placed with, and sets the playback of the session it * opened, but changes nothing else there. @@ -144,11 +146,32 @@ class AudioCheckRunner : public QObject int punchIns; int eventsEach; + /// The punch-ins themselves, in seconds on the reference's + /// timeline, in the order they are recorded. When there are + /// any, they are what is recorded, and punchIns and eventsEach + /// are not used. They may meet but not overlap, and lie within + /// the layout (punchInsOf()) + std::vector ranges; + /// Where the reference is written, over whatever is there; "" /// for a new file in referenceDirectory() (nextReferencePath()) QString referencePath; - Plan() : punchIns(0), eventsEach(0) { } + /// Record into the session open now, which the caller says is + /// a reference made from this plan's layout, and into its take: + /// no reference is written or opened, and nothing is asked. + /// What earlier runs recorded stays in the take, and only this + /// run's punch-ins are judged + bool keepSession; + + /// The round trip this run's takes are placed with, in seconds, + /// in place of the one the window places every take with; + /// negative for the window's own. For the run only: nothing is + /// stored, and the window goes on saying it uses its own + double roundTrip; + + Plan() : punchIns(0), eventsEach(0), keepSession(false), + roundTrip(-1.0) { } }; /// The steps of a run, in order; the last two come once for each @@ -188,6 +211,14 @@ class AudioCheckRunner : public QObject /// events each on the calibration layout static Plan calibrationPlan(); + /// The punch-ins a run of the plan records, in seconds: its ranges + /// if it has any, else as many as it asks for from its layout + /// (LatencyCheck::punchInsFor()). Empty if they cannot be + /// recorded: ranges that overlap, come out of order, are empty or + /// reach outside the layout, or a layout with no room for the + /// punch-ins asked for + static std::vector punchInsOf(const Plan &plan); + /// Where the reference is written unless the plan names a file: /// the application's data directory static QString referenceDirectory(); @@ -205,9 +236,16 @@ class AudioCheckRunner : public QObject /** * Begin a run. False, with nothing started, if one is running - * already, if a take is being recorded, or if the plan's punch-ins - * do not fit its layout (LatencyCheck::punchInsFor()). Otherwise - * finished() comes once, at the end, however the run ends. + * already, if a take is being recorded, if the plan's punch-ins + * cannot be recorded (punchInsOf()), or if it keeps the session and + * there is none. Otherwise finished() comes once, at the end, + * however the run ends. + * + * A run that replaces the session asks the user whether to save it + * first (MainWindow::checkSaveModified()), unless it is a check's + * own: never saved, and playing a reference in referenceDirectory(). + * Nothing of the user's is in that, and Check Again would otherwise + * ask every time, the takes having changed it. */ bool start(const Plan &plan); @@ -215,12 +253,32 @@ class AudioCheckRunner : public QObject /// path, and finished() says the run was cancelled void cancel(); + /// End the run at once, with no finished() and no Stop path: for + /// the window's teardown, where whoever waits for the run goes as + /// well and a take in progress is left to the window, as any take + /// is when it is deleted. The destructor does this + void abandon(); + bool isRunning() const; /// Called by MainWindow::closeSession() once the session is sure to /// close: a run ends then, unless it is the run replacing it void sessionClosing(); + /// Whether the analyser, or any transform, is still at work: what + /// a run waits for after opening the reference and after each take + static bool analysing(Analyser *analyser); + + /** + * A take's audio file, mixed to one channel, at the rate it was + * recorded at. The file, and not the take's model: the model is + * normalised to full scale as it is read (the "normalise audio" + * preference), which would have every take clipped, and resampled + * to the session's rate. "" on success, else what went wrong. + */ + static QString readTakeFile(QString path, std::vector &mono, + sv::sv_samplerate_t &rate); + signals: void finished(const AudioCheckResult &result); @@ -259,6 +317,18 @@ class AudioCheckRunner : public QObject void poll(); void openReference(); + + /// The session open now is a check's own (see start()), and can be + /// replaced without asking + bool sessionIsACheck() const; + + /// The audio file the session's main model was opened from, or "" + QString mainModelFile() const; + + /// The session is open with the reference in it: the punch-ins in + /// frames of the session, its playback, and on to its analysis + void referenceOpen(); + void startPunchIn(); void takeStopped(); void judge(); @@ -286,8 +356,8 @@ class AudioCheckRunner : public QObject /// End the run and say so; failure is empty for a run judged void end(QString failure); - /// Whether the analyser, or any transform, is still at work - static bool analysing(Analyser *analyser); + /// Take the override for the check's takes away from the window + void clearOverride(); }; #endif diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index d2978b9d..daabd8b7 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -27,6 +27,10 @@ #include #include +#ifdef TONY_DEV_CHECKS +#include +#endif + #include #include #include @@ -45,8 +49,8 @@ using LatencyCheck::Verdict; double expectedSeconds(const AudioCheckRunner::Plan &plan) { - const std::vector ranges = LatencyCheck::punchInsFor - (plan.layout, plan.punchIns, plan.eventsEach); + const std::vector ranges = + AudioCheckRunner::punchInsOf(plan); double seconds = 0.0; for (const LatencyCheck::PunchIn &r : ranges) { seconds += std::min(AudioCheckRunner::kPreRollSeconds, r.start) + @@ -92,6 +96,12 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, m_latencyKept(false), m_expectedSeconds(0), m_shownPermille(0) +#ifdef TONY_DEV_CHECKS + , + m_devChecksBox(nullptr), + m_devRunning(false), + m_haveDevReport(false) +#endif { setWindowTitle(tr("Calibrate Audio")); setModal(false); @@ -110,8 +120,19 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, return label; }; + QWidget *instructions = new QWidget; + QVBoxLayout *instructionsLayout = new QVBoxLayout; + instructionsLayout->setContentsMargins(0, 0, 0, 0); + instructions->setLayout(instructionsLayout); m_instructions = textLabel(); - m_pages->addWidget(m_instructions); + instructionsLayout->addWidget(m_instructions); +#ifdef TONY_DEV_CHECKS + m_devChecksBox = new QCheckBox(tr("Run the dev checks after calibrating")); + m_devChecksBox->setChecked(true); + instructionsLayout->addWidget(m_devChecksBox); +#endif + instructionsLayout->addStretch(1); + m_pages->addWidget(instructions); QWidget *progress = new QWidget; QVBoxLayout *progressLayout = new QVBoxLayout; @@ -178,6 +199,117 @@ CalibrateAudioDialog::setPlan(const AudioCheckRunner::Plan &plan) m_plan = plan; } +#ifdef TONY_DEV_CHECKS + +void +CalibrateAudioDialog::setDevChecks(DevChecks *devChecks) +{ + if (m_devChecks) disconnect(m_devChecks, nullptr, this, nullptr); + m_devChecks = devChecks; + if (!devChecks) return; + + // Direct, as the runner's: the report has no metatype + connect(devChecks, &DevChecks::progress, + this, &CalibrateAudioDialog::devProgress); + connect(devChecks, &DevChecks::finished, + this, &CalibrateAudioDialog::devFinished); +} + +bool +CalibrateAudioDialog::devChecksWanted() const +{ + return m_devChecksBox->isChecked(); +} + +void +CalibrateAudioDialog::setDevChecksWanted(bool wanted) +{ + m_devChecksBox->setChecked(wanted); +} + +void +CalibrateAudioDialog::setDevOptions(const DevChecks::Options &options) +{ + m_devOptions = options; +} + +bool +CalibrateAudioDialog::startDevChecks(const AudioCheckResult &calibration) +{ + if (!m_devChecksBox->isChecked() || !m_devChecks) return false; + + // They place their takes with what the calibration measured, which + // means nothing unless it can be used + if (!calibration.calibrationUsable()) { + m_devNote = tr("The dev checks did not run: they need a calibration " + "that can be used."); + return false; + } + + DevChecks::Options options = m_devOptions; + options.roundTrip = calibration.calibratedRoundTrip; + if (!m_devChecks->start(options)) { + m_devNote = tr("The dev checks could not start."); + return false; + } + + m_devRunning = true; + m_step->setText(tr("Calibrated. Starting the dev checks...")); + m_timeLeft->setText(QString()); + m_bar->setRange(0, 0); + return true; +} + +void +CalibrateAudioDialog::devProgress(QString stage, int stageNumber, int stages) +{ + if (!m_devRunning) return; + m_step->setText(tr("Dev checks, stage %1 of %2: %3...") + .arg(stageNumber).arg(stages).arg(stage)); + m_timeLeft->setText(QString()); +} + +void +CalibrateAudioDialog::devFinished(const DevReport &report) +{ + // Only those that carried on from a calibration of this dialog's + if (!m_devRunning) return; + m_devRunning = false; + m_running = false; + m_devReport = report; + m_haveDevReport = true; + m_bar->setRange(0, 1000); + showResultPage(); +} + +QString +CalibrateAudioDialog::devHtml() const +{ + QString html; + if (m_devNote != "") html += paragraph(m_devNote.toHtmlEscaped()); + if (!m_haveDevReport) return html; + + html += paragraph(bold(tr("Dev checks"))); + if (m_devReport.failure != "") { + html += paragraph(tr("They ended early: %1") + .arg(m_devReport.failure.toHtmlEscaped())); + } + html += "
    "; + for (const CheckResult &c : m_devReport.checks) { + html += "
  • " + tr("Item %1, %2: %3").arg(c.item) + .arg(c.name.toHtmlEscaped()) + .arg(CheckResult::verdictName(c.verdict)) + "
    • "; + } + html += "
"; + html += paragraph(m_devReport.reportPath != "" ? + tr("Report: %1").arg(m_devReport.reportPath + .toHtmlEscaped()) : + tr("The report could not be written.")); + return html; +} + +#endif + CalibrateAudioDialog::Page CalibrateAudioDialog::page() const { @@ -215,6 +347,10 @@ CalibrateAudioDialog::present() // shown; a check of its own that is running stays on show if (!m_running) { m_instructions->setText(instructionsHtml()); +#ifdef TONY_DEV_CHECKS + // On each time the instructions are shown afresh + m_devChecksBox->setChecked(true); +#endif showPage(Page::Instructions); } show(); @@ -231,7 +367,14 @@ CalibrateAudioDialog::startCheck() m_latencyKept = false; m_expectedSeconds = expectedSeconds(m_plan); m_shownPermille = 0; + m_bar->setRange(0, 1000); m_bar->setValue(0); +#ifdef TONY_DEV_CHECKS + m_devRunning = false; + m_haveDevReport = false; + m_devReport = DevReport(); + m_devNote = QString(); +#endif m_step->setText(tr("Starting the check...")); m_timeLeft->setText(QString()); @@ -253,6 +396,21 @@ CalibrateAudioDialog::startCheck() void CalibrateAudioDialog::cancelCheck() { +#ifdef TONY_DEV_CHECKS + // Its dev checks, which end at once and report through devFinished() + if (m_devRunning) { + if (m_devChecks) { + m_devChecks->cancel(); + } else { + // Gone with nothing said (the window is going) + m_devRunning = false; + m_running = false; + showResultPage(); + } + return; + } +#endif + // The run ends at once, and its end comes back through // runnerFinished() if (m_running) m_runner->cancel(); @@ -276,6 +434,17 @@ void CalibrateAudioDialog::showResult(const AudioCheckResult &result) { m_result = result; +#ifdef TONY_DEV_CHECKS + m_haveDevReport = false; + m_devReport = DevReport(); + m_devNote = QString(); +#endif + showResultPage(); +} + +void +CalibrateAudioDialog::showResultPage() +{ m_latencyKept = false; m_resultText->setText(resultHtml()); showPage(Page::Result); @@ -286,7 +455,7 @@ CalibrateAudioDialog::reject() { // Nothing else would show how the run went, and a check left running // behind a closed dialog would go on playing chirps - if (m_running) m_runner->cancel(); + if (m_running) cancelCheck(); QDialog::reject(); } @@ -294,6 +463,10 @@ void CalibrateAudioDialog::runnerProgress(const AudioCheckRunner::Progress &p) { if (!m_running) return; +#ifdef TONY_DEV_CHECKS + // The runner's part in the dev checks, which show their own stages + if (m_devRunning) return; +#endif QString step; switch (p.step) { @@ -340,8 +513,18 @@ CalibrateAudioDialog::runnerFinished(const AudioCheckResult &result) { // A run started elsewhere is not this dialog's to show if (!m_running) return; +#ifdef TONY_DEV_CHECKS + // Nor is the runner's part in the dev checks, which report for + // themselves. A calibration carries on into them if they are + // wanted, and is shown with their report when they end + if (m_devRunning) return; + m_result = result; + if (startDevChecks(result)) return; +#else + m_result = result; +#endif m_running = false; - showResult(result); + showResultPage(); } void @@ -402,6 +585,16 @@ CalibrateAudioDialog::instructionsHtml() const QString CalibrateAudioDialog::resultHtml() const +{ +#ifdef TONY_DEV_CHECKS + return calibrationHtml() + devHtml(); +#else + return calibrationHtml(); +#endif +} + +QString +CalibrateAudioDialog::calibrationHtml() const { const AudioCheckResult &r = m_result; diff --git a/main/CalibrateAudioDialog.h b/main/CalibrateAudioDialog.h index ed5cdccf..d0c18511 100644 --- a/main/CalibrateAudioDialog.h +++ b/main/CalibrateAudioDialog.h @@ -18,6 +18,12 @@ #include "AudioCheckRunner.h" #include "LatencyCalibration.h" +#ifdef TONY_DEV_CHECKS +#include "dev/DevChecks.h" +#include +class QCheckBox; +#endif + #include class MainWindow; @@ -43,6 +49,13 @@ class QStackedWidget; * nothing else would show how the run ended. * * The window owns it, and makes it the first time it is asked for. + * + * In development builds the instructions page has a checkbox, on by + * default and not remembered, to carry on into the dev checks + * (DevChecks) once the calibration is done and usable, with the round + * trip it measured. The progress page then follows them, Cancel ends + * whichever is running, and the result page adds a line for each check + * and the report file's path to the calibration's. */ class CalibrateAudioDialog : public QDialog { @@ -77,6 +90,19 @@ class CalibrateAudioDialog : public QDialog /// ms". The Playback menu's line says the same static QString describeLatency(const LatencyCalibration::InUse &inUse); +#ifdef TONY_DEV_CHECKS + /// The dev checks a calibration carries on into; none until given + void setDevChecks(DevChecks *devChecks); + + /// "Run the dev checks after calibrating", on the instructions page + bool devChecksWanted() const; + void setDevChecksWanted(bool wanted); + + /// Where the dev checks write their report and scratch folders, as + /// the tests set them; the round trip is always the calibration's + void setDevOptions(const DevChecks::Options &options); +#endif + public slots: /// Show the dialog and bring it to the front: on the instructions, /// with the devices and the latency as they are now, unless its @@ -86,7 +112,8 @@ public slots: /// Start, and Check Again void startCheck(); - /// Cancel: the run ends, and how it ended is the result shown + /// Cancel: the run ends, and how it ended is the result shown. In + /// its dev checks, they end, and the calibration is shown with them void cancelCheck(); /// Keep the round trip the check measured, for the devices it ran on @@ -133,8 +160,36 @@ public slots: void showPage(Page page); + /// The result page for m_result as it stands + void showResultPage(); + QString instructionsHtml() const; QString resultHtml() const; + QString calibrationHtml() const; + +#ifdef TONY_DEV_CHECKS + QPointer m_devChecks; + QCheckBox *m_devChecksBox; + DevChecks::Options m_devOptions; + + /// The run this dialog started is in its dev checks + bool m_devRunning; + + /// What they reported, or why they did not run + bool m_haveDevReport; + DevReport m_devReport; + QString m_devNote; + + /// Carry a calibration on into the dev checks, if they are wanted + /// and it can be used; false if the run ends here, with m_devNote + /// saying why when they were wanted + bool startDevChecks(const AudioCheckResult &calibration); + + void devProgress(QString stage, int stageNumber, int stages); + void devFinished(const DevReport &report); + + QString devHtml() const; +#endif /// Seconds in milliseconds, for reading: tenths below 10 ms, where /// they say something, whole ones from there on diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index 146a14cf..0ccfec92 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -35,8 +35,9 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, /** * What a take was placed with: the round trip taken off the front of - * its recording (besides the start gap, which is measured), whether - * that was a figure the audio check measured or the sum of the output + * its recording (besides the start gap, below), whether + * that was a figure the audio check measured (a stored one, or one a + * check brought for its own run) or the sum of the output * and input latencies the device reported, those two latencies, and the * rate the device recorded at. All 0 until known; the round trip stays * 0 for a take made without the reference playing, which is placed with @@ -48,6 +49,13 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, * device's), and as it compares them with a stored figure's * fingerprint: in frames they count at two different rates when the * device's rate is not the session's. + * + * The start gap is the rest of what is taken off the front: the frames + * of the recording made before the reference began to play, also in + * frames of the recording. It is estimated when the reference is + * started, and measured once the audio callback has handed the device + * its first block; startGapMeasured says whether that happened, as it + * does for every take that plays the reference. */ struct TakeLatency { @@ -56,9 +64,12 @@ struct TakeLatency double reportedOutput; double reportedInput; sv::sv_samplerate_t recordingRate; + sv::sv_frame_t startGap; + bool startGapMeasured; TakeLatency() : roundTrip(0), measured(false), reportedOutput(0), - reportedInput(0), recordingRate(0) { } + reportedInput(0), recordingRate(0), startGap(0), + startGapMeasured(false) { } /// Frames of the recording in seconds; 0 while its rate is unknown double recordingSeconds(sv::sv_frame_t frames) const { diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 540c046b..83738050 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -26,6 +26,10 @@ #include "TakeLayers.h" #include "TakesFile.h" +#ifdef TONY_DEV_CHECKS +#include "dev/DevChecks.h" +#endif + #include "framework/Document.h" #include "framework/VersionTester.h" @@ -205,6 +209,11 @@ MainWindow::MainWindow(AudioMode audioMode, m_takeLatency(), m_audioCheck(nullptr), m_audioCheckTakes(false), + m_audioCheckRoundTrip(-1.0), +#ifdef TONY_DEV_CHECKS + m_devChecks(nullptr), +#endif + m_recordAction(nullptr), m_calibrateAudioDialog(nullptr), m_calibrateAudioAction(nullptr), m_latencyLineAction(nullptr), @@ -412,6 +421,16 @@ MainWindow::MainWindow(AudioMode audioMode, connect(m_audioCheck, &AudioCheckRunner::finished, this, [this]() { updateMenuStates(); }); +#ifdef TONY_DEV_CHECKS + // Likewise for the development checks, which run the check among + // stages of their own + m_devChecks = new DevChecks(this, m_audioCheck); + connect(m_devChecks, &DevChecks::progress, + this, [this]() { updateMenuStates(); }); + connect(m_devChecks, &DevChecks::finished, + this, [this]() { updateMenuStates(); }); +#endif + // Often enough to stop a take that records into a selection well // within the margin that follows the selection's end m_takeTimer = new QTimer(this); @@ -468,10 +487,15 @@ MainWindow::MainWindow(AudioMode audioMode, MainWindow::~MainWindow() { - // The check's dialog first, as it holds the runner; then a check + // The check's dialog first, as it holds the runner and the dev + // checks; then the dev checks, which drive the runner; then a check // still running ends here, before anything it reads goes delete m_calibrateAudioDialog; m_calibrateAudioDialog = nullptr; +#ifdef TONY_DEV_CHECKS + delete m_devChecks; + m_devChecks = nullptr; +#endif delete m_audioCheck; m_audioCheck = nullptr; @@ -1536,7 +1560,9 @@ MainWindow::setupToolbars() recordAction->setCheckable(true); recordAction->setShortcut(tr("Ctrl+Space")); recordAction->setStatusTip(tr("Record a new audio file. If a reference track is already loaded, the recording is added as the singing track alongside it.")); - connect(recordAction, SIGNAL(triggered()), this, SLOT(record())); + connect(recordAction, &QAction::triggered, + this, &MainWindow::recordPressed); + m_recordAction = recordAction; connect(m_recordTarget, SIGNAL(recordStatusChanged(bool)), recordAction, SLOT(setChecked(bool))); connect(m_recordTarget, SIGNAL(recordCompleted()), @@ -2258,8 +2284,11 @@ MainWindow::updateMenuStates() emit canActOnTake(canChange && m_takes->getActiveIndex() >= 0); // The audio check records takes of its own, and keeps what it - // measures for the devices it started on - bool checking = m_audioCheck && m_audioCheck->isRunning(); + // measures for the devices it started on. Record is shut after the + // base class has opened it: a press would stop the check's take, or + // record one of the user's into the check's session + bool checking = audioCheckRunning(); + if (checking) emit canRecord(false); if (m_calibrateAudioAction) { m_calibrateAudioAction->setEnabled(!inTake && !checking); } @@ -2596,8 +2625,14 @@ MainWindow::closeSession() if (!checkSaveModified()) return; // A check has nothing left to record into; a take of its own that is - // running is stopped through the Stop path, as the check's Cancel does + // running is stopped through the Stop path, as the check's Cancel does. + // The runner first: one that is replacing the session for the dev + // checks carries on, and the dev checks see it running and carry on + // with it, as they do through a reopen of their own if (m_audioCheck) m_audioCheck->sessionClosing(); +#ifdef TONY_DEV_CHECKS + if (m_devChecks) m_devChecks->sessionClosing(); +#endif // Nothing of a take that is still running outlives its session stopTakePolling(); @@ -4083,6 +4118,36 @@ MainWindow::record() } } +void +MainWindow::recordPressed() +{ + // The check starts and stops its takes through record() itself, and + // a press would stop its take early, or start one of the user's in + // the check's session. The button is shut while it runs; a press + // that arrives all the same (queued before it was shut, say) leaves + // the button showing what is really happening + if (audioCheckRunning()) { + cerr << "MainWindow::recordPressed: the audio check is running; " + << "Record is ignored" << endl; + if (m_recordAction) { + m_recordAction->setChecked + (m_recordTarget && m_recordTarget->isRecording()); + } + return; + } + record(); +} + +bool +MainWindow::audioCheckRunning() const +{ + if (m_audioCheck && m_audioCheck->isRunning()) return true; +#ifdef TONY_DEV_CHECKS + if (m_devChecks && m_devChecks->isRunning()) return true; +#endif + return false; +} + void MainWindow::startTakePolling() { @@ -4269,6 +4334,16 @@ MainWindow::recordingStarted() m_lastRecordingRate = recordingRate; LatencyCalibration::InUse inUse = roundTripAt(recordingRate); + + // A check that brings a round trip of its own places its takes + // with that, for the run only (the dev checks, with the figure + // the calibration before them measured): nothing is stored, + // and latencyInUse() goes on describing roundTripAt()'s. The + // reported pair stays the device's + const bool checkOwn = + m_audioCheckTakes && m_audioCheckRoundTrip >= 0.0; + if (checkOwn) inUse.roundTrip = m_audioCheckRoundTrip; + sv_frame_t roundTrip = LatencyCalibration::toFrames(inUse.roundTrip, recordingRate); m_recordingLatencyFrames = roundTrip + m_recordingStartGapEstimate; @@ -4276,17 +4351,24 @@ MainWindow::recordingStarted() m_takeLatency.roundTrip = roundTrip; m_takeLatency.reportedOutput = inUse.reportedOutput; m_takeLatency.reportedInput = inUse.reportedInput; - m_takeLatency.measured = + m_takeLatency.measured = checkOwn || (inUse.source == LatencyCalibration::Source::Measured); + m_takeLatency.startGap = m_recordingStartGapEstimate; + m_takeLatency.startGapMeasured = false; cerr << "MainWindow::recordingStarted: round trip " << roundTrip << " frames at " << recordingRate << " Hz (" - << inUse.roundTrip * 1000.0 << " ms), " - << LatencyCalibration::sourceName(inUse.source); - if (inUse.source == LatencyCalibration::Source::Measured) { - cerr << " on " << inUse.date.toString(Qt::ISODate).toStdString(); - } else if (inUse.stale) { - cerr << ": the measured one is stale, the device reports " - << "other latencies now"; + << inUse.roundTrip * 1000.0 << " ms), "; + if (checkOwn) { + cerr << "the audio check's own, for its run only"; + } else { + cerr << LatencyCalibration::sourceName(inUse.source); + if (inUse.source == LatencyCalibration::Source::Measured) { + cerr << " on " + << inUse.date.toString(Qt::ISODate).toStdString(); + } else if (inUse.stale) { + cerr << ": the measured one is stale, the device reports " + << "other latencies now"; + } } cerr << "; output latency=" << outputLatency << " input latency=" << inputLatency @@ -4309,7 +4391,16 @@ void MainWindow::refineRecordingLatency() { sv_frame_t measured = m_recordingStartGapMeasured; - if (measured < 0 || measured == m_recordingStartGapEstimate) return; + if (measured < 0) return; + + // Measured, whether or not the estimate was right: the audio check + // reports for each take which of the two it was placed with. Kept + // here, on the GUI thread, and not by the audio callback that + // measures it + m_takeLatency.startGap = measured; + m_takeLatency.startGapMeasured = true; + + if (measured == m_recordingStartGapEstimate) return; cerr << "MainWindow::refineRecordingLatency: start gap was " << measured << " frames, not the estimated " << m_recordingStartGapEstimate << endl; m_recordingLatencyFrames = currentRecordingLatency(); @@ -4437,6 +4528,9 @@ MainWindow::calibrateAudio() { if (!m_calibrateAudioDialog) { m_calibrateAudioDialog = new CalibrateAudioDialog(this, m_audioCheck); +#ifdef TONY_DEV_CHECKS + m_calibrateAudioDialog->setDevChecks(m_devChecks); +#endif } m_calibrateAudioDialog->present(); } diff --git a/main/MainWindow.h b/main/MainWindow.h index 8f0da001..39678e2c 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -40,6 +40,9 @@ class QActionGroup; class AudioCheckRunner; struct AudioCheckResult; class CalibrateAudioDialog; +#ifdef TONY_DEV_CHECKS +class DevChecks; +#endif namespace sv { class VersionTester; @@ -55,6 +58,11 @@ class MainWindow : public sv::MainWindowBase // The audio check drives the take path of the window, and reads what // each take was placed with; see AudioCheckRunner friend class AudioCheckRunner; +#ifdef TONY_DEV_CHECKS + // The development checks save and reopen the session, and read the + // take's pitch and notes; see DevChecks + friend class DevChecks; +#endif public: MainWindow(AudioMode audioMode, @@ -136,6 +144,12 @@ protected slots: // causing the recording to be treated as the singing track. virtual void record(); + // The Record button, and its shortcut: record() or Stop, except + // while the audio check runs, which records and stops takes of its + // own through record(); the button is shut then, and a press that + // gets here all the same is ignored + virtual void recordPressed(); + protected slots: virtual void openFile(); virtual void openSingingTrack(); @@ -891,6 +905,26 @@ protected slots: AudioCheckRunner *m_audioCheck; bool m_audioCheckTakes; + // The round trip the check's takes are placed with in place of + // roundTripAt()'s, in seconds, for a run that brings one of its own + // (AudioCheckRunner::Plan::roundTrip); negative when it brings none. + // Read with m_audioCheckTakes, and never by latencyInUse() + double m_audioCheckRoundTrip; + +#ifdef TONY_DEV_CHECKS + // The development checks, which drive the audio check and the + // session; made with the window, deleted in its destructor after the + // dialog and before the runner, and told when the session closes + DevChecks *m_devChecks; +#endif + + // The audio check, or the development checks, are running: the take + // path and the devices are theirs until they end + bool audioCheckRunning() const; + + // The Record button, shut while audioCheckRunning() + QAction *m_recordAction; + // Playback > Calibrate Audio, made the first time it is chosen, and // the lines under it: the latency takes are placed with, and Forget // Measured Latency. Calibrate Audio is shut while a take or a check diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp new file mode 100644 index 00000000..bcee21b3 --- /dev/null +++ b/main/dev/DevChecks.cpp @@ -0,0 +1,788 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifdef TONY_DEV_CHECKS + +#include "DevChecks.h" + +#include "../MainWindow.h" +#include "../SingingTakes.h" + +#include "audio/AudioCallbackRecordTarget.h" +#include "data/model/NoteModel.h" +#include "data/model/SparseTimeValueModel.h" +#include "layer/Layer.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +using std::cerr; +using std::endl; +using std::vector; + +using namespace sv; + +namespace { + +// Signed, to a tenth of a millisecond: the tolerance is 2 ms +QString +signedMs(double seconds) +{ + const double ms = std::round(seconds * 10000.0) / 10.0; + return QString("%1%2 ms").arg(ms > 0.0 ? "+" : "") + .arg(ms == 0.0 ? 0.0 : ms, 0, 'f', 1); +} + +QString +unsignedMs(double seconds) +{ + return QString("%1 ms").arg(seconds * 1000.0, 0, 'f', 1); +} + +QString +secondsText(double seconds) +{ + return QString("%1").arg(seconds, 0, 'f', 2); +} + +// A value as the session file holds it: Event::toXml() writes it with +// QString::arg(), which keeps six significant figures +float +asSaved(float value) +{ + return QString("%1").arg(value).toFloat(); +} + +// The events as a session saved and read back has them, in order, so +// that they can be compared with those of the session read back +EventVector +asSaved(const EventVector &events) +{ + EventVector saved; + saved.reserve(events.size()); + for (const Event &e : events) { + Event s = e; + if (s.hasValue()) s = s.withValue(asSaved(s.getValue())); + if (s.hasLevel()) s = s.withLevel(asSaved(s.getLevel())); + saved.push_back(s); + } + std::sort(saved.begin(), saved.end()); + return saved; +} + +// Whether the events after the reopen are those before the save, and +// in words how many there are, or the first that differs +bool +sameEvents(const EventVector &before, const EventVector &after, + sv_samplerate_t rate, QString what, QString &description) +{ + const EventVector b = asSaved(before); + const EventVector a = asSaved(after); + if (a == b) { + description = QString("%1 %2, the same").arg(a.size()).arg(what); + return true; + } + size_t i = 0; + while (i < a.size() && i < b.size() && a[i] == b[i]) ++i; + sv_frame_t frame = (i < b.size() ? b[i].getFrame() : + i < a.size() ? a[i].getFrame() : 0); + description = QString("%1 %2, %3 before the save; the first difference " + "at %4 s") + .arg(a.size()).arg(what).arg(b.size()) + .arg(rate > 0 ? secondsText(double(frame) / rate) : QString("?")); + return false; +} + +} // namespace + +QString +CheckResult::verdictName(Verdict verdict) +{ + switch (verdict) { + case Verdict::Pass: return "Pass"; + case Verdict::Fail: return "Fail"; + case Verdict::Measured: return "Measured"; + case Verdict::Skipped: return "Skipped"; + } + return ""; +} + +int +DevReport::count(CheckResult::Verdict verdict) const +{ + int n = 0; + for (const CheckResult &c : checks) { + if (c.verdict == verdict) ++n; + } + return n; +} + +DevChecks::DevChecks(MainWindow *window, AudioCheckRunner *runner) : + QObject(window), + m_window(window), + m_runner(runner), + m_timer(new QTimer(this)), + m_stage(-1), + m_begun(false), + m_inPoll(false), + m_runnerRunning(false), + m_reopening(false), + m_saved(false), + m_haveFresh(false), + m_reopened(false) +{ + m_timer->setInterval(kPollMs); + connect(m_timer, &QTimer::timeout, this, &DevChecks::poll); + + // Direct: the result has no metatype, and the stage waiting for it + // is looked at on the next poll + connect(runner, &AudioCheckRunner::finished, + this, &DevChecks::runnerFinished); +} + +DevChecks::~DevChecks() +{ + // Deleted by the window in its destructor, before the runner. The + // run ends without a word, as the runner's does, and so does the + // runner's run it started: without the Stop path, which would splice + // a take and start its analysis in the middle of the window's + // teardown. The take in progress is left to the window + m_timer->stop(); + if (m_stage >= 0) { + cerr << "DevChecks: the window is going; the run ends" << endl; + if (m_runnerRunning && m_runner) m_runner->abandon(); + m_runnerRunning = false; + m_stage = -1; + } +} + +vector +DevChecks::freshPunchIns() +{ + return { LatencyCheck::PunchIn(6.3, 10.2), + LatencyCheck::PunchIn(16.8, 21.2) }; +} + +QString +DevChecks::nextScratchFolder(QString directory, QString inUse) +{ + const QDir dir(directory); + const QRegularExpression name("^dev-checks-[0-9]+$"); + + // Only folders by our own name: the directory may hold anything else + const QStringList folders = + dir.entryList(QDir::Dirs | QDir::NoDotAndDotDot); + for (const QString &folder : folders) { + if (!name.match(folder).hasMatch()) continue; + const QString path = dir.absoluteFilePath(folder); + if (inUse != "" && QFileInfo(inUse).absoluteDir() == QDir(path)) { + continue; + } + if (!QDir(path).removeRecursively()) { + cerr << "DevChecks: an earlier scratch folder could not be " + << "removed: " << path << endl; + } + } + + for (int n = 1; ; ++n) { + const QString path = + dir.absoluteFilePath(QString("dev-checks-%1").arg(n)); + if (QFileInfo::exists(path)) continue; + if (!QDir().mkpath(path)) return ""; + return path; + } +} + +bool +DevChecks::start(const Options &options) +{ + if (isRunning()) return false; + if (!m_runner || m_runner->isRunning()) { + cerr << "DevChecks::start: the audio check is running" << endl; + return false; + } + if (m_window->m_recordTarget && m_window->m_recordTarget->isRecording()) { + cerr << "DevChecks::start: a take is being recorded" << endl; + return false; + } + + // The folder the session open now lives in stays, as it does + const QString scratch = nextScratchFolder + (scratchDirectory(options.scratchDirectory), m_window->m_sessionFile); + if (scratch == "") { + cerr << "DevChecks::start: no scratch folder could be made in " + << scratchDirectory(options.scratchDirectory) << endl; + return false; + } + + m_options = options; + m_scratchFolder = scratch; + m_sessionPath = QDir(scratch).filePath(kSessionFileName); + m_saved = false; + m_startedAt = QDateTime::currentDateTime(); + { + QSettings settings; + m_devices = LatencyCalibration::currentKey(settings, 0); + } + + m_layout = LatencyCheck::devLayout(); + m_haveFresh = false; + m_fresh = AudioCheckResult(); + m_coverageAfterFresh = Coverage(); + m_pitchBefore.clear(); + m_notesBefore.clear(); + m_pitchAfter.clear(); + m_notesAfter.clear(); + m_reopened = false; + m_rejudged = LatencyCheck::TakeSummary(); + m_runnerRunning = false; + m_runnerResult = AudioCheckResult(); + + m_stages.clear(); + m_stages.push_back({ tr("Fresh punch-ins"), + [this]() { beginFreshPunchIns(); }, + [this]() { return freshPunchInsDone(); }, + kCheckStageTimeoutMs }); + m_stages.push_back({ tr("Save and reopen"), + [this]() { beginReopen(); }, + [this]() { return reopenDone(); }, + kReopenTimeoutMs }); + + cerr << "DevChecks::start: " << m_stages.size() << " stages, scratch " + << "folder " << m_scratchFolder << ", round trip "; + if (m_options.roundTrip >= 0.0) { + cerr << m_options.roundTrip * 1000.0 << " ms" << endl; + } else { + cerr << "the window's own" << endl; + } + + // Nothing is done before the first poll, so that however the run + // ends, the caller hears of it through finished() and never from + // inside this call + m_stage = 0; + m_begun = false; + m_timer->start(); + return true; +} + +void +DevChecks::cancel() +{ + if (!isRunning()) return; + end(tr("The dev checks were cancelled.")); +} + +bool +DevChecks::isRunning() const +{ + return m_stage >= 0; +} + +void +DevChecks::sessionClosing() +{ + // Its own reopen replaces the session, and so does its runner as it + // opens the dev reference. The window tells the runner first: a run + // of the runner's still going after that is one replacing the session + if (!isRunning() || m_reopening) return; + if (m_runnerRunning && m_runner && m_runner->isRunning()) return; + end(tr("The session was closed during the dev checks.")); +} + +void +DevChecks::poll() +{ + if (m_inPoll) return; + if (!isRunning()) { + m_timer->stop(); + return; + } + m_inPoll = true; + + // A copy: whatever a stage calls may end the run + const int index = m_stage; + const Stage stage = m_stages[index]; + + if (!m_begun) { + m_begun = true; + m_stageClock.start(); + cerr << "DevChecks: stage " << (index + 1) << " of " + << m_stages.size() << ": " << stage.name << endl; + emit progress(stage.name, index + 1, int(m_stages.size())); + if (m_stage == index) stage.begin(); + } else if (stage.done()) { + if (m_stage == index) { + m_begun = false; + if (++m_stage >= int(m_stages.size())) { + m_stage = index; + end(""); + } + } + } else if (m_stage == index && + m_stageClock.elapsed() > qint64(stage.limitMs)) { + end(tr("Stage %1 of %2, \"%3\", did not finish within %4 s.") + .arg(index + 1).arg(m_stages.size()).arg(stage.name) + .arg(stage.limitMs / 1000)); + } + + m_inPoll = false; +} + +void +DevChecks::runnerFinished(const AudioCheckResult &result) +{ + // A run the dialog started, or anyone else, is not ours + if (!m_runnerRunning) return; + m_runnerRunning = false; + m_runnerResult = result; +} + +void +DevChecks::beginFreshPunchIns() +{ + AudioCheckRunner::Plan plan; + plan.layout = m_layout; + plan.ranges = freshPunchIns(); + plan.roundTrip = m_options.roundTrip; + + m_runnerResult = AudioCheckResult(); + m_runnerRunning = m_runner && m_runner->start(plan); + if (!m_runnerRunning) { + end(tr("The audio check could not start.")); + } +} + +bool +DevChecks::freshPunchInsDone() +{ + if (m_runnerRunning) return false; + + if (m_runnerResult.failure != "") { + end(tr("The punch-ins could not be recorded: %1") + .arg(m_runnerResult.failure)); + return false; + } + + m_fresh = m_runnerResult; + m_haveFresh = true; + m_coverageAfterFresh = m_window->m_takes->getCoverage(); + return true; +} + +void +DevChecks::beginReopen() +{ + // Read from the models before the save, and again from the models + // there are after the reopen + m_pitchBefore = takeEvents(Analyser::PitchTrack); + m_notesBefore = takeEvents(Analyser::Notes); + + // The way Save As saves once it has a name, with no dialog but for a + // failure: the take's audio is copied into the session's folder + if (!m_window->saveSessionToPath(m_sessionPath)) { + if (isRunning()) { + end(tr("The session could not be saved to \"%1\".") + .arg(m_sessionPath)); + } + return; + } + m_saved = true; + if (!isRunning()) return; + + m_reopening = true; + const MainWindowBase::FileOpenStatus status = + m_window->openPath(m_sessionPath, MainWindowBase::ReplaceSession); + m_reopening = false; + if (!isRunning()) return; + + if (status != MainWindowBase::FileOpenSucceeded) { + end(tr("The session saved in \"%1\" could not be opened again.") + .arg(m_sessionPath)); + } +} + +bool +DevChecks::reopenDone() +{ + // The session restores its analyses rather than running them, but + // waits as the runner does, in case something runs all the same + if (AudioCheckRunner::analysing(m_window->m_analyser)) return false; + if (!m_window->m_takes->haveTake()) { + end(tr("The session opened again has no take.")); + return false; + } + if (!m_window->m_analyser2 || + AudioCheckRunner::analysing(m_window->m_analyser2)) { + return false; + } + + m_pitchAfter = takeEvents(Analyser::PitchTrack); + m_notesAfter = takeEvents(Analyser::Notes); + + // The take's file as the session names it now: the copy in its + // folder. Judged over the same ranges as before the save + vector mono; + sv_samplerate_t rate = 0; + const QString error = AudioCheckRunner::readTakeFile + (m_window->m_takes->getAudioPath(), mono, rate); + if (error != "") { + end(error); + return false; + } + vector ranges; + for (const LatencyCheck::PunchInResult &p : m_fresh.summary.punchIns) { + ranges.push_back(p.range); + } + m_rejudged = LatencyCheck::judgeTake + (m_layout, mono.data(), sv_frame_t(mono.size()), rate, ranges); + m_reopened = true; + return true; +} + +EventVector +DevChecks::takeEvents(Analyser::Component component) const +{ + Analyser *analyser = m_window->m_analyser2; + Layer *layer = analyser ? analyser->getLayer(component) : nullptr; + if (!layer) return {}; + if (auto model = ModelById::getAs + (layer->getModel())) { + return model->getAllEvents(); + } + if (auto model = ModelById::getAs(layer->getModel())) { + return model->getAllEvents(); + } + return {}; +} + +void +DevChecks::end(QString failure) +{ + if (!isRunning()) return; + + m_timer->stop(); + m_stage = -1; + m_begun = false; + + // A take of its own is stopped through the Stop path, as the check's + // Cancel does; what the runner says then is not waited for + if (m_runnerRunning) { + m_runnerRunning = false; + if (m_runner) m_runner->cancel(); + } + + DevReport report; + report.failure = failure; + report.checks = evaluate(failure); + if (m_saved) report.sessionPath = m_sessionPath; + report.reportPath = writeReport(report); + + cerr << "DevChecks: the run ended" + << (failure != "" ? ": " + failure.toStdString() : std::string()) + << "; " << report.count(CheckResult::Verdict::Pass) << " passed, " + << report.count(CheckResult::Verdict::Fail) << " failed, " + << report.count(CheckResult::Verdict::Measured) << " measured, " + << report.count(CheckResult::Verdict::Skipped) << " skipped; " + << "report in " << report.reportPath << endl; + + emit finished(report); +} + +vector +DevChecks::evaluate(QString reason) const +{ + return { latencyCheck(reason), phrasesCheck(reason) }; +} + +CheckResult +DevChecks::latencyCheck(QString reason) const +{ + CheckResult c; + c.item = 1; + c.name = "latency_on_this_machine"; + + if (!m_haveFresh) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Every judged sweep of every punch-in, found and within the + // tolerance + const LatencyCheck::TakeSummary &s = m_fresh.summary; + QStringList problems; + double largest = 0.0; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const LatencyCheck::PunchInResult &p = s.punchIns[i]; + QStringList offsets; + for (const LatencyCheck::EventResult &e : s.events) { + if (e.punchIn != i) continue; + if (!e.arrival.found) { + offsets << tr("not found"); + problems << tr("the sweep at %1 s was not found") + .arg(secondsText(e.expectedSeconds)); + continue; + } + const double offset = e.arrival.errorSeconds; + offsets << signedMs(offset); + if (std::fabs(offset) > std::fabs(largest)) largest = offset; + if (std::fabs(offset) > kPlacementSeconds) { + problems << tr("the sweep at %1 s landed %2 off") + .arg(secondsText(e.expectedSeconds)) + .arg(signedMs(offset)); + } + } + if (p.judged == 0) { + problems << tr("punch-in %1 judged no sweep").arg(i + 1); + } + c.numbers.push_back + ({ tr("offsets, punch-in %1 (%2 to %3 s)").arg(i + 1) + .arg(secondsText(p.range.start)) + .arg(secondsText(p.range.end)), + offsets.isEmpty() ? tr("none judged") : offsets.join(", ") }); + } + if (s.punchIns.empty()) problems << tr("no punch-in was judged"); + c.numbers.push_back({ tr("largest offset"), signedMs(largest) }); + c.numbers.push_back({ tr("round trip used"), + unsignedMs(m_fresh.usedRoundTrip) }); + + if (!m_reopened) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + if (!problems.isEmpty()) { + c.message += " " + tr("Before that: %1.").arg(problems.join("; ")); + } + return c; + } + + // The same file, copied into the session's folder by the save: every + // sweep where it was, to the frame + const LatencyCheck::TakeSummary &r = m_rejudged; + QString offsetsAfter = tr("the same"); + bool same = (r.events.size() == s.events.size()); + for (size_t i = 0; same && i < s.events.size(); ++i) { + const LatencyCheck::EventResult &a = s.events[i]; + const LatencyCheck::EventResult &b = r.events[i]; + if (a.event != b.event || a.arrival.found != b.arrival.found || + a.arrival.errorFrames != b.arrival.errorFrames) { + same = false; + offsetsAfter = tr("the sweep at %1 s: %2, %3 before the save") + .arg(secondsText(a.expectedSeconds)) + .arg(b.arrival.found ? signedMs(b.arrival.errorSeconds) + : tr("not found")) + .arg(a.arrival.found ? signedMs(a.arrival.errorSeconds) + : tr("not found")); + } + } + if (r.events.size() != s.events.size()) { + offsetsAfter = tr("%1 sweeps judged, %2 before the save") + .arg(r.events.size()).arg(s.events.size()); + } + if (!same) problems << tr("the take judged again after reopening differs"); + c.numbers.push_back({ tr("offsets after reopening"), offsetsAfter }); + + const sv_samplerate_t rate = m_fresh.referenceRate; + QString pitch, notes; + if (!sameEvents(m_pitchBefore, m_pitchAfter, rate, tr("pitch events"), + pitch)) { + problems << tr("the take's pitch changed after reopening"); + } + if (!sameEvents(m_notesBefore, m_notesAfter, rate, tr("notes"), notes)) { + problems << tr("the take's notes changed after reopening"); + } + if (m_pitchBefore.empty()) { + problems << tr("the take had no pitch track to compare"); + } + c.numbers.push_back({ tr("pitch after reopening"), pitch }); + c.numbers.push_back({ tr("notes after reopening"), notes }); + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Every sweep landed within %1 of where the reference " + "has it, and the session saved and opened again " + "gives the same offsets, pitch and notes.") + .arg(unsignedMs(kPlacementSeconds)); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::phrasesCheck(QString reason) const +{ + CheckResult c; + c.item = 2; + c.name = "several_phrases_in_one_take"; + + if (!m_haveFresh) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Each punch-in in the take where it was recorded, placed right, and + // with the start gap of its own stream measured + const LatencyCheck::TakeSummary &s = m_fresh.summary; + const sv_samplerate_t rate = m_fresh.referenceRate; + QStringList problems; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const LatencyCheck::PunchInResult &p = s.punchIns[i]; + + // As the runner selected it, in whole frames of the session + const sv_frame_t start = sv_frame_t(std::llround(p.range.start * rate)); + const sv_frame_t end = sv_frame_t(std::llround(p.range.end * rate)); + Coverage::Range held; + if (!m_coverageAfterFresh.getRangeAt(start, held) || + held.start > start || held.end < end) { + problems << tr("punch-in %1 is not all in the take").arg(i + 1); + } + + if (p.found == 0) { + problems << tr("punch-in %1: no sweep found").arg(i + 1); + } else if (std::fabs(p.medianOffset) > kPlacementSeconds) { + problems << tr("punch-in %1 landed %2 off").arg(i + 1) + .arg(signedMs(p.medianOffset)); + } + + const TakeLatency t = (i < int(m_fresh.takes.size()) ? + m_fresh.takes[i] : TakeLatency()); + if (!t.startGapMeasured) { + problems << tr("punch-in %1's start gap was not measured") + .arg(i + 1); + } + + c.numbers.push_back + ({ tr("punch-in %1, median offset").arg(i + 1), + p.found > 0 ? signedMs(p.medianOffset) : tr("nothing found") }); + c.numbers.push_back + ({ tr("punch-in %1, start gap").arg(i + 1), + tr("%1 (%2 frames), %3") + .arg(unsignedMs(t.recordingSeconds(t.startGap))) + .arg(t.startGap) + .arg(t.startGapMeasured ? tr("measured") : + tr("estimated only")) }); + } + if (s.punchIns.size() < 2) { + problems << tr("%1 punch-ins, not several").arg(s.punchIns.size()); + } + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Every punch-in is in the take, placed within %1, " + "with a start gap measured for its own stream.") + .arg(unsignedMs(kPlacementSeconds)); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +QString +DevChecks::writeReport(const DevReport &report) const +{ + const QString directory = reportDirectory(m_options.reportDirectory); + if (!QDir().mkpath(directory)) { + cerr << "DevChecks: the report directory " << directory + << " could not be made" << endl; + return ""; + } + const QString path = QDir(directory).filePath(kReportFileName); + QFile file(path); + if (!file.open(QIODevice::WriteOnly | QIODevice::Truncate | + QIODevice::Text)) { + cerr << "DevChecks: the report " << path << " could not be written" + << endl; + return ""; + } + + auto device = [](QString name) { + return name == "" ? QString("(System Default)") : name; + }; + + QTextStream out(&file); + out << "Tony development checks\n"; + out << "Run: " << m_startedAt.toString(Qt::ISODate) << "\n"; + out << "Output device: " << device(m_devices.playbackDevice) << "\n"; + out << "Input device: " << device(m_devices.recordDevice) << "\n"; + out << "Round trip for the run: " + << (m_options.roundTrip >= 0.0 ? unsignedMs(m_options.roundTrip) + : QString("the one takes are placed with")) << "\n"; + out << "Scratch folder: " << m_scratchFolder << "\n"; + out << "Session: " << (report.sessionPath != "" ? report.sessionPath + : QString("not saved")) << "\n"; + if (report.failure != "") { + out << "The run ended early: " << report.failure << "\n"; + } + + // Grouped by checklist item, in the order of the checklist + vector checks = report.checks; + std::stable_sort(checks.begin(), checks.end(), + [](const CheckResult &a, const CheckResult &b) { + return a.item < b.item; + }); + int item = -1; + for (const CheckResult &c : checks) { + if (c.item != item) { + item = c.item; + out << "\nItem " << item << "\n"; + } + out << " " << c.name << ": " + << CheckResult::verdictName(c.verdict).toUpper() << "\n"; + if (c.message != "") out << " " << c.message << "\n"; + for (const auto &n : c.numbers) { + out << " " << n.first << ": " << n.second << "\n"; + } + } + + out << "\nTotals: " << report.count(CheckResult::Verdict::Pass) + << " passed, " << report.count(CheckResult::Verdict::Fail) + << " failed, " << report.count(CheckResult::Verdict::Measured) + << " measured, " << report.count(CheckResult::Verdict::Skipped) + << " skipped\n"; + out.flush(); + file.close(); + + return QFileInfo(path).absoluteFilePath(); +} + +QString +DevChecks::reportDirectory(QString given) +{ + if (given != "") return given; + const QString logs = qEnvironmentVariable("TONY_TEST_LOG_DIR"); + if (logs != "") return logs; + return QStandardPaths::writableLocation(QStandardPaths::AppDataLocation); +} + +QString +DevChecks::scratchDirectory(QString given) +{ + if (given != "") return given; + return QStandardPaths::writableLocation(QStandardPaths::AppDataLocation); +} + +#endif diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h new file mode 100644 index 00000000..1d5815f9 --- /dev/null +++ b/main/dev/DevChecks.h @@ -0,0 +1,301 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_DEV_CHECKS_H +#define TONY_DEV_CHECKS_H + +#ifdef TONY_DEV_CHECKS + +#include "../Analyser.h" +#include "../AudioCheckRunner.h" +#include "../Coverage.h" +#include "../LatencyCalibration.h" +#include "../LatencyCheck.h" + +#include "base/Event.h" + +#include +#include +#include +#include +#include + +#include +#include +#include + +class MainWindow; +class QTimer; + +/** + * What one development check found. Not a QtTest function: the app + * suite has to show that each check can fail, which a QVERIFY inside + * another test cannot. + */ +struct CheckResult +{ + enum class Verdict { Pass, Fail, Measured, Skipped }; + + /// The item of docs/manual-checklist.md it settles, and its name + int item; + QString name; + + Verdict verdict; + + /// The figures behind the verdict, as label and value, in words + std::vector> numbers; + + /// What the verdict means, or why there is none + QString message; + + /// "Pass", "Fail", "Measured" or "Skipped" + static QString verdictName(Verdict verdict); + + CheckResult() : item(0), verdict(Verdict::Skipped) { } +}; + +/** + * What a run of the development checks found, however it ended. + */ +struct DevReport +{ + /// Every check, in the order of the checklist's items. A run that + /// ended early has those it did not reach Skipped, with the reason + std::vector checks; + + /// Why the run ended before its last stage was done; empty if not + QString failure; + + /// The report file, and the session the run saved in its scratch + /// folder; empty if they were not written + QString reportPath; + QString sessionPath; + + int count(CheckResult::Verdict verdict) const; +}; + +/** + * The development checks (development builds only): every item of the + * manual checklist that a speaker-to-microphone loopback can settle, + * run on the live window through the audio check's ordinary takes, + * after a calibration, with the round trip it measured. + * + * A run is a list of stages, each of which starts something and then + * says, when polled, whether it is done. Driven by a polling timer + * and the runner's finished(), as the runner itself is, and never by a + * nested event loop: the window can be closed, and the session + * replaced, at any moment, and a run ends cleanly then. + * + * The stages: + * 1. Fresh punch-ins: a run of the audio check on the dev layout, in + * a reference of its own (replacing the calibration's session + * without asking), with the two punch-ins of freshPunchIns() and + * the round trip given. + * 2. Save and reopen: the session saved into a scratch folder of this + * run, the way Save As saves once it has a name, then opened again + * and its take's file judged again. + * + * The checks are worked out when the run ends, from what the stages + * kept: item 1 (latency, also after save and reopen) and item 2 + * (several phrases in one take). + * + * The session saved stays open afterwards, so that the takes can be + * looked at; its scratch folder stays with it, and the next run + * removes every such folder the session open then does not use. The + * report goes to a text file, DevChecks.txt, ending with a Totals: + * line like the suites'. + * + * MainWindow owns it, and deletes it after the Calibrate Audio dialog + * and before the runner. A friend of MainWindow: it saves and opens + * sessions, and reads the take's coverage, file and events. + */ +class DevChecks : public QObject +{ + Q_OBJECT + +public: + /// Items 1 and 2: how far from where the reference has it a sweep + /// may land, either way + static constexpr double kPlacementSeconds = 0.002; + + /// How often a stage is looked at + static constexpr int kPollMs = 50; + + /// How long a stage may take. A stage that runs the audio check + /// waits for it, and the runner has limits of its own for each + /// step, which end its run with a reason: this is a backstop. + /// Save and reopen is quick + static constexpr int kCheckStageTimeoutMs = 240000; + static constexpr int kReopenTimeoutMs = 60000; + + /// The report's file name, in the report directory + static constexpr const char *kReportFileName = "DevChecks.txt"; + + /// The name of the session saved in the scratch folder + static constexpr const char *kSessionFileName = "dev-checks.ton"; + + struct Options { + /// The round trip the run's takes are placed with, in seconds, + /// for the run only (AudioCheckRunner::Plan::roundTrip); + /// negative for the one the window uses + double roundTrip; + + /// Where the report goes: "" for $TONY_TEST_LOG_DIR if that is + /// set, else the application's data directory + QString reportDirectory; + + /// Where the scratch folders go: "" for the application's data + /// directory + QString scratchDirectory; + + Options() : roundTrip(-1.0) { } + }; + + DevChecks(MainWindow *window, AudioCheckRunner *runner); + virtual ~DevChecks(); + + /** + * Stage 1's punch-ins on the dev layout, in seconds: [6.3, 10.2] + * and [16.8, 21.2], each judging two sweeps (7.2 and 9.1 s; 17.7 + * and 20.1 s) with 50 ms to spare. In two separate regions of the + * calibration part, clear of what later stages need: the start + * (before 4.3 s) for a punch-in at 1 s whose range judges the sweep + * at 3.1 s; the held tones from 26.9 s on; 10.2 to 16.8 s and 21.2 + * to 25 s for fresh punch-ins; and each range holds two sweeps, so + * that a re-recording can start inside it, past its first sweep, + * with its lead-in over what was recorded there, and still judge + * the second. + */ + static std::vector freshPunchIns(); + + /** + * A new scratch folder in the directory, made: dev-checks-1, + * dev-checks-2 and so on, the lowest number free. Every such + * folder in the directory is removed first, with all it holds, + * except the one the session file inUse is in: that session, open + * now, was saved there by an earlier run. "" if none could be + * made. + */ + static QString nextScratchFolder(QString directory, QString inUse); + + /** + * Begin a run. False, with nothing started, if one is running + * already, if the audio check is, if a take is being recorded, or + * if no scratch folder could be made. Otherwise finished() comes + * once, at the end, however the run ends. + */ + bool start(const Options &options); + + /// End the run: its take is stopped through the Stop path, as the + /// audio check's Cancel does, and finished() says why + void cancel(); + + bool isRunning() const; + + /// Called by MainWindow::closeSession(), after the runner has been + /// told: a run ends then, unless it is replacing the session itself + /// (its reopen, or its runner opening the dev reference) + void sessionClosing(); + +signals: + /// When a stage begins: its name, and which of how many. Never + /// from inside start() or cancel() + void progress(QString stage, int stageNumber, int stages); + + void finished(const DevReport &report); + +private: + struct Stage { + QString name; + std::function begin; + std::function done; + int limitMs; + }; + + MainWindow *m_window; + QPointer m_runner; + QTimer *m_timer; + + Options m_options; + std::vector m_stages; + + /// The stage going on, or -1; whether it has begun, and since when + int m_stage; + bool m_begun; + QElapsedTimer m_stageClock; + + /// Set while poll() runs: a dialog shown from inside a stage runs + /// an event loop of its own, in which the timer goes on firing + bool m_inPoll; + + /// A run of the runner's that this run started is going on; and + /// what it said when it ended + bool m_runnerRunning; + AudioCheckResult m_runnerResult; + + /// Set while the run opens the session it saved + bool m_reopening; + + /// What the run was started with, for the report + QDateTime m_startedAt; + LatencyCalibration::Key m_devices; + QString m_scratchFolder; + QString m_sessionPath; + bool m_saved; + + /// Stage 1: the layout, what the runner found, and the take's + /// coverage straight after + LatencyCheck::Layout m_layout; + bool m_haveFresh; + AudioCheckResult m_fresh; + Coverage m_coverageAfterFresh; + + /// Stage 2: the take's pitch and notes before the save and after + /// the reopen, and its file judged again after it + sv::EventVector m_pitchBefore; + sv::EventVector m_notesBefore; + sv::EventVector m_pitchAfter; + sv::EventVector m_notesAfter; + bool m_reopened; + LatencyCheck::TakeSummary m_rejudged; + + void poll(); + void runnerFinished(const AudioCheckResult &result); + + void beginFreshPunchIns(); + bool freshPunchInsDone(); + void beginReopen(); + bool reopenDone(); + + /// The events of the take's pitch track or notes, from its model + /// as it is now + sv::EventVector takeEvents(Analyser::Component component) const; + + /// End the run, work the checks out, write the report and say so; + /// failure is empty for a run that got through every stage + void end(QString failure); + + std::vector evaluate(QString reason) const; + CheckResult latencyCheck(QString reason) const; + CheckResult phrasesCheck(QString reason) const; + + /// The report file written, or "" if it could not be + QString writeReport(const DevReport &report) const; + + static QString reportDirectory(QString given); + static QString scratchDirectory(QString given); +}; + +#endif +#endif diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 3e829a1e..1c3f2459 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -18,7 +18,8 @@ // the real MainWindow, recording from the fake device with its output // looped back into its input, as an earcup held to the mic. A run is // two punch-ins of two sweeps each on the first 11 s of the calibration -// reference, about 13 s of real time. +// reference, about 13 s of real time. The windows have no development +// checks, even in a development build: TestDevChecks has those. // // The same dialog watchdog as TestRecordWorkflow's: a dialog would // block the test for ever, so it is dismissed, and the test fails in @@ -33,6 +34,7 @@ #include "base/PlayParameterRepository.h" #include +#include class TestAudioCheck : public QObject { @@ -59,6 +61,12 @@ class TestAudioCheck : public QObject void makeWindow(FakeAudioIO::Config config, bool installDevice = true) { delete m_window; m_window = new TestMainWindow(config, installDevice); +#ifdef TONY_DEV_CHECKS + // The calibration alone, as a release build has it: with the + // dev checks there, the dialog carries on into them by default + // (TestDevChecks) + m_window->doDeleteDevChecks(); +#endif m_result = AudioCheckResult(); m_finished = 0; m_progress.clear(); @@ -99,22 +107,89 @@ class TestAudioCheck : public QObject return plan; } - void startCheck() { + void startCheck() { startCheck(shortPlan()); } + void runCheck() { runCheck(shortPlan()); } + + void startCheck(const AudioCheckRunner::Plan &plan, + bool discard = true) { // As answering "No" to "do you want to save?", which the check // asks before it replaces the session - m_window->discardModifications(); - QVERIFY(m_window->audioCheck()->start(shortPlan())); + if (discard) m_window->discardModifications(); + QVERIFY(m_window->audioCheck()->start(plan)); QVERIFY(m_window->audioCheck()->isRunning()); } - void runCheck() { - startCheck(); + void runCheck(const AudioCheckRunner::Plan &plan, bool discard = true) { + m_result = AudioCheckResult(); + m_finished = 0; + startCheck(plan, discard); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0, 60000); QCOMPARE(m_finished, 1); QVERIFY(!m_window->audioCheck()->isRunning()); } + // The short plan with one punch-in of its own, which judges the sweep + // at 7.2 s; its lead-in starts in the tone of the sweep before, so + // that the fake device's first audible sample is the first played + AudioCheckRunner::Plan onePunchIn() { + AudioCheckRunner::Plan plan = shortPlan(); + plan.ranges = { LatencyCheck::PunchIn(6.3, 8.3) }; + return plan; + } + + // Every measured round trip kept, as the settings hold it + static QStringList storedLatency() { + QSettings settings; + settings.beginGroup("LatencyCalibration"); + QStringList all; + for (const QString &key : settings.allKeys()) { + all << key + "=" + settings.value(key).toString(); + } + settings.endGroup(); + all.sort(); + return all; + } + + // A reference in the directory the check writes its own to, as a + // check before this one left it, opened and analysed; its path + QString openEarlierCheckReference() { + QDir().mkpath(AudioCheckRunner::referenceDirectory()); + const QString path = AudioCheckRunner::nextReferencePath + (AudioCheckRunner::referenceDirectory(), ""); + const AudioCheckRunner::Plan plan = shortPlan(); + const std::vector samples = LatencyCheck::generate(plan.layout); + sv::WavFileWriter writer(path, plan.layout.rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *data = samples.data(); + if (!writer.isOK() || + !writer.writeSamples(&data, sv::sv_frame_t(samples.size())) || + !writer.close()) { + return {}; + } + m_window->discardModifications(); + if (m_window->openPath(path, MainWindow::ReplaceSession) != + MainWindow::FileOpenSucceeded) { + return {}; + } + Analyser *analyser = m_window->analyser(); + QTest::qWaitFor([analyser]() { + return analyser->getLayer(Analyser::PitchTrack) && + analyser->getInitialAnalysisCompletion() >= 100 && + !sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(); + }, 30000); + return path; + } + + // The application's data directory, where a check writes its + // references unless told otherwise, is Qt's test location while one + // of these lives + struct TestDataLocation { + TestDataLocation() { QStandardPaths::setTestModeEnabled(true); } + ~TestDataLocation() { QStandardPaths::setTestModeEnabled(false); } + }; + // The devices the Preferences name, as the device menus write them // when the driver is left alone. The fake device takes no notice static void setDevices(QString output, QString input) { @@ -839,6 +914,259 @@ private slots: QCOMPARE(files(), QStringList() << "song.wav"); } + // Punch-ins given in the plan must be recordable one after the other + // within the layout: overlapping, out of order, empty or outside it, + // the run does not start. Ranges that meet are fine. A plan that + // keeps the session needs one + void check_refuses_ranges_that_do_not_fit() { + makeWindow(loopback()); + AudioCheckRunner::Plan plan = shortPlan(); + using P = LatencyCheck::PunchIn; + const std::vector> refused = { + { P(2.2, 4.2), P(4.0, 6.3) }, + { P(6.3, 8.3), P(2.2, 4.2) }, + { P(-0.1, 2.0) }, + { P(9.0, 11.0) }, + { P(3.0, 3.0) }, + { P(4.0, 3.0) }, + }; + for (const std::vector

&ranges : refused) { + plan.ranges = ranges; + QVERIFY(AudioCheckRunner::punchInsOf(plan).empty()); + QVERIFY(!m_window->audioCheck()->start(plan)); + QVERIFY(!m_window->audioCheck()->isRunning()); + } + + plan.ranges = { P(2.2, 4.2), P(4.2, 6.3), P(6.3, 10.8) }; + const std::vector

given = AudioCheckRunner::punchInsOf(plan); + QCOMPARE(int(given.size()), 3); + QCOMPARE(given[1].start, 4.2); + QCOMPARE(given[2].end, 10.8); + + // Without ranges, as many as it asks for from the layout + plan.ranges.clear(); + QCOMPARE(int(AudioCheckRunner::punchInsOf(plan).size()), 2); + + plan = onePunchIn(); + plan.keepSession = true; + QVERIFY(!m_window->audioCheck()->start(plan)); + QVERIFY(!m_window->audioCheck()->isRunning()); + + QTest::qWait(200); + QCOMPARE(m_finished, 0); + } + + // A round trip of the run's own: its takes are placed with it and + // land where they belong, although another is kept for the devices + // and the window places every other take with that. The one kept, + // and the Playback menu's line about it, are as they were. The take + // measured its own start gap, as the device says it was + void check_uses_the_round_trip_it_is_given() { + makeWindow(loopback()); + + // With a file open, the device reports what it will during takes + openSong(); + if (QTest::currentTestFailed()) return; + const LatencyCalibration::InUse reported = m_window->latencyInUse(); + LatencyCalibration::Figure figure; + figure.roundTrip = 0.3; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = reported.reportedOutput; + figure.reportedInput = reported.reportedInput; + { + QSettings settings; + LatencyCalibration::store(settings, key("", ""), figure); + } + const QString lineBefore = latencyLine(); + const QStringList storedBefore = storedLatency(); + QVERIFY2(lineBefore.startsWith("Latency: measured 300 ms"), + qPrintable(lineBefore)); + + AudioCheckRunner::Plan plan = onePunchIn(); + plan.roundTrip = roundTrip / rate; + runCheck(plan); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QCOMPARE(r.summary.judged, 1); + QCOMPARE(r.summary.found, 1); + QVERIFY2(std::fabs(r.summary.medianOffset * rate) <= 4.0, + describe(r).constData()); + QCOMPARE(int(r.takes.size()), 1); + const TakeLatency &t = r.takes[0]; + QCOMPARE(t.roundTrip, sv::sv_frame_t(roundTrip)); + QVERIFY(t.measured); + QCOMPARE(t.reportedOutput, reportedOut / rate); + QCOMPARE(t.reportedInput, reportedIn / rate); + QCOMPARE(r.usedRoundTrip, roundTrip / rate); + + QVERIFY(t.startGapMeasured); + const long gap = m_window->fake()->getFramesBeforePlayStart(); + QVERIFY2(gap >= 0 && std::labs(long(t.startGap) - gap) <= 16, + qPrintable(QString("start gap %1 frames; the device says %2") + .arg(t.startGap).arg(gap))); + + QCOMPARE(latencyLine(), lineBefore); + QCOMPARE(storedLatency(), storedBefore); + const LatencyCalibration::InUse inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Measured); + QCOMPARE(inUse.roundTrip, 0.3); + } + + // A run that keeps the session records into the one open, the + // reference and take of the run before: nothing is written, opened + // or asked, though the session is modified. The take keeps the + // earlier punch-in, and only the run's own is judged + void check_records_into_the_session_open() { + makeWindow(loopback()); + + AudioCheckRunner::Plan first = shortPlan(); + first.ranges = { LatencyCheck::PunchIn(2.2, 4.2) }; + runCheck(first); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_result.summary.verdict == LatencyCheck::Verdict::Ok, + describe(m_result).constData()); + QCOMPARE(m_result.summary.judged, 1); + const sv::ModelId reference = m_window->mainModelId(); + QVERIFY(m_window->isDocumentModified()); + + AudioCheckRunner::Plan second = onePunchIn(); + second.keepSession = true; + second.referencePath = m_dir.filePath("not-written.wav"); + runCheck(second, false); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QVERIFY(!QFileInfo::exists(second.referencePath)); + QVERIFY(m_window->mainModelId() == reference); + QCOMPARE(r.referenceRate, rate); + + QCOMPARE(int(r.summary.punchIns.size()), 1); + QVERIFY(std::fabs(r.summary.punchIns[0].range.start - 6.3) < 1e-4); + QCOMPARE(r.summary.judged, 1); + QCOMPARE(int(r.summary.events.size()), 1); + QVERIFY(std::fabs(r.summary.events[0].expectedSeconds - 7.2) < 1e-4); + QCOMPARE(int(r.takes.size()), 1); + + const Coverage &coverage = m_window->takes()->getCoverage(); + QCOMPARE(int(coverage.getRanges().size()), 2); + QVERIFY(coverage.contains(sv::sv_frame_t(2.3 * rate))); + QVERIFY(coverage.contains(sv::sv_frame_t(6.4 * rate))); + } + + // The session of an earlier check, never saved and changed by its + // takes, is replaced without asking; and the new reference goes to a + // file of its own, the earlier one being open + void check_replaces_its_own_session_without_asking() { + TestDataLocation testData; + makeWindow(loopback()); + const QString earlier = openEarlierCheckReference(); + QVERIFY(earlier != ""); + const sv::ModelId reference = m_window->mainModelId(); + m_window->markModified(); + QVERIFY(m_window->isDocumentModified()); + + AudioCheckRunner::Plan plan = onePunchIn(); + plan.referencePath = ""; + startCheck(plan, false); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY2(m_dialogs.isEmpty(), qPrintable(m_dialogs.join(" | "))); + QVERIFY(m_window->mainModelId() != reference); + + // The earlier one was open, so it stayed, and the new one has a + // name of its own + const QStringList references = + QDir(AudioCheckRunner::referenceDirectory()).entryList + (QStringList() << "calibrate-audio-reference*.wav", QDir::Files, + QDir::Name); + QCOMPARE(references, QStringList() + << "calibrate-audio-reference-1.wav" + << "calibrate-audio-reference-2.wav"); + QCOMPARE(QFileInfo(earlier).fileName(), + QString("calibrate-audio-reference-1.wav")); + + m_window->audioCheck()->cancel(); + QCOMPARE(m_finished, 1); + } + + // A session of the user's is asked about before it is replaced: their + // song, and a check's session they saved. The watchdog answers + void check_asks_before_replacing_the_users_session() { + TestDataLocation testData; + makeWindow(loopback()); + + openSong(); + if (QTest::currentTestFailed()) return; + m_window->markModified(); + startCheck(onePunchIn(), false); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(!m_dialogs.isEmpty(), 10000); + QVERIFY2(m_dialogs.first().startsWith("Session modified"), + qPrintable(m_dialogs.join(" | "))); + QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0 || + m_window->recordTarget()->isRecording(), + 30000); + m_window->audioCheck()->cancel(); + QCOMPARE(m_finished, 1); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_dialogs.clear(); + + const QString earlier = openEarlierCheckReference(); + QVERIFY(earlier != ""); + QVERIFY(m_window->doSaveSessionAs(m_dir.filePath("kept.ton"))); + m_window->markModified(); + m_finished = 0; + startCheck(onePunchIn(), false); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(!m_dialogs.isEmpty(), 10000); + QVERIFY2(m_dialogs.first().startsWith("Session modified"), + qPrintable(m_dialogs.join(" | "))); + QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0 || + m_window->recordTarget()->isRecording(), + 30000); + m_window->audioCheck()->cancel(); + QCOMPARE(m_finished, 1); + m_dialogs.clear(); + } + + // Record is shut while a check runs. A press that reaches the window + // all the same does not stop the check's take, which records to the + // end of its range as if nothing had happened + void check_ignores_the_record_button() { + makeWindow(loopback()); + QVERIFY(m_window->recordAction()->isEnabled()); + + startCheck(onePunchIn()); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY(!m_window->recordAction()->isEnabled()); + + m_window->recordAction()->trigger(); + emit m_window->recordAction()->triggered(false); + QVERIFY(m_window->recordTarget()->isRecording()); + QVERIFY(m_window->recordAction()->isChecked()); + QTest::qWait(300); + QVERIFY(m_window->recordTarget()->isRecording()); + + QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0, 60000); + QVERIFY2(m_result.summary.verdict == LatencyCheck::Verdict::Ok, + describe(m_result).constData()); + QCOMPARE(m_result.summary.found, 1); + QCOMPARE(int(m_result.takes.size()), 1); + QVERIFY(m_window->recordAction()->isEnabled()); + } + // Playback > Calibrate Audio: a dialog, not modal, that names the // devices and the latency in use and starts a check. While the check // runs, neither it nor the device menus can be chosen. Other devices diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h new file mode 100644 index 00000000..c0ade9e5 --- /dev/null +++ b/main/test/TestDevChecks.h @@ -0,0 +1,631 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_DEV_CHECKS_H +#define TEST_DEV_CHECKS_H + +#ifdef TONY_DEV_CHECKS + +// Tier 5, as TestAudioCheck: the development checks (DevChecks) on the +// real MainWindow, recording from the fake device with its output +// looped back into its input. A run records two punch-ins against the +// 40 s dev reference, then saves the session and opens it again: about +// 20 s of real time. +// +// The fixture is TestAudioCheck's, copied rather than shared. The +// application's data directory, where the check writes its references, +// is Qt's test location while this class runs; the report and the +// scratch folders go to directories of the tests' own, so that a +// failing run's report never lands among the suites' results. + +#include "TestRecordWorkflow.h" + +#include "../AudioCheckRunner.h" +#include "../CalibrateAudioDialog.h" +#include "../LatencyCheck.h" +#include "../dev/DevChecks.h" + +#include + +class TestDevChecks : public QObject +{ + Q_OBJECT + + static constexpr double rate = 44100.0; + + // What the device reports, and what the round trip really is: as + // TestAudioCheck's, 123 frames (2.8 ms) more than reported + static constexpr int reportedOut = 2 * 4096; + static constexpr int reportedIn = 4096; + static constexpr int roundTrip = 3 * 4096 + 123; + + QTemporaryDir m_dir; + TestMainWindow *m_window = nullptr; + QTimer m_watchdog; + QStringList m_dialogs; + + // What the dev checks said when the run ended, and how often; the + // stages they began; and every result of the runner's + DevReport m_report; + int m_finished = 0; + QStringList m_stages; + std::vector m_checks; + + void makeWindow(FakeAudioIO::Config config) { + delete m_window; + m_window = new TestMainWindow(config); + m_report = DevReport(); + m_finished = 0; + m_stages.clear(); + m_checks.clear(); + connect(m_window->devChecks(), &DevChecks::finished, + this, [this](const DevReport &report) { + m_report = report; + ++m_finished; + }); + connect(m_window->devChecks(), &DevChecks::progress, + this, [this](QString stage, int n, int of) { + m_stages << QString("%1 of %2: %3").arg(n).arg(of) + .arg(stage); + }); + connect(m_window->audioCheck(), &AudioCheckRunner::finished, + this, [this](const AudioCheckResult &result) { + m_checks.push_back(result); + }); + } + + static FakeAudioIO::Config loopback() { + FakeAudioIO::Config config; + config.playbackLatency = reportedOut; + config.recordLatency = reportedIn; + config.inputDelay = roundTrip; + config.loopback = true; + return config; + } + + QString reportDirectory() { return m_dir.filePath("report"); } + QString scratchDirectory() { return m_dir.filePath("scratch"); } + + DevChecks::Options options(double roundTripSeconds) { + DevChecks::Options o; + o.roundTrip = roundTripSeconds; + o.reportDirectory = reportDirectory(); + o.scratchDirectory = scratchDirectory(); + return o; + } + + // TestAudioCheck's short calibration, with its reference where the + // check writes one of its own: in the application's data directory + AudioCheckRunner::Plan shortPlan() { + AudioCheckRunner::Plan plan; + plan.layout = LatencyCheck::calibrationLayout(); + plan.layout.length = plan.layout.events[5].sweepStart; + plan.layout.events.resize(5); + plan.punchIns = 2; + plan.eventsEach = 2; + return plan; + } + + void startDevChecks(double roundTripSeconds) { + m_window->discardModifications(); + QVERIFY(m_window->devChecks()->start(options(roundTripSeconds))); + QVERIFY(m_window->devChecks()->isRunning()); + } + + void runDevChecks(double roundTripSeconds) { + startDevChecks(roundTripSeconds); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0, 120000); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->devChecks()->isRunning()); + QVERIFY(!m_window->audioCheck()->isRunning()); + } + + const CheckResult *check(int item) const { + for (const CheckResult &c : m_report.checks) { + if (c.item == item) return &c; + } + return nullptr; + } + + static QString number(const CheckResult &c, QString label) { + for (const auto &n : c.numbers) { + if (n.first == label) return n.second; + } + return QString(); + } + + // "-20.1 ms" as -20.1; NaN for anything else + static double milliseconds(QString text) { + bool ok = false; + double ms = text.endsWith(" ms") ? text.chopped(3).toDouble(&ok) : 0.0; + return ok ? ms : std::nan(""); + } + + QString reportText() { + QFile file(m_report.reportPath); + if (!file.open(QIODevice::ReadOnly | QIODevice::Text)) return {}; + return QString::fromUtf8(file.readAll()); + } + + QString lastReportLine() { + const QStringList lines = reportText().trimmed().split('\n'); + return lines.isEmpty() ? QString() : lines.last(); + } + + QByteArray describe() { + QStringList words; + words << "failure: " + m_report.failure; + for (const CheckResult &c : m_report.checks) { + QStringList numbers; + for (const auto &n : c.numbers) numbers << n.first + ": " + n.second; + words << QString("item %1 %2 %3 (%4) [%5]").arg(c.item).arg(c.name) + .arg(CheckResult::verdictName(c.verdict)).arg(c.message) + .arg(numbers.join("; ")); + } + return words.join(" | ").toUtf8(); + } + + // The three toggles of the take path, and the settings they and the + // pre-roll's length are kept in + QStringList toggles() { + QSettings settings; + settings.beginGroup("MainWindow"); + QStringList state; + state << QString("play reference %1, setting %2") + .arg(m_window->playReferenceWhileRecordingAction()->isChecked()) + .arg(settings.value("playrefwhilerecording").toString()); + state << QString("pre-roll %1, setting %2, %3 s") + .arg(m_window->preRollAction()->isChecked()) + .arg(settings.value("preroll").toString()) + .arg(settings.value("prerollseconds").toString()); + state << QString("record into selection %1, setting %2") + .arg(m_window->recordIntoSelectionAction()->isChecked()) + .arg(settings.value("recordintoselection").toString()); + settings.endGroup(); + return state; + } + + // Every measured round trip kept, as the settings hold it + static QStringList storedLatency() { + QSettings settings; + settings.beginGroup("LatencyCalibration"); + QStringList all; + for (const QString &key : settings.allKeys()) { + all << key + "=" + settings.value(key).toString(); + } + settings.endGroup(); + all.sort(); + return all; + } + + // A round trip of 300 ms kept for the fake device, as Use this + // latency keeps one, and the user's toggles set otherwise than the + // check sets them for its takes + void setUserState() { + LatencyCalibration::InUse reported = m_window->latencyInUse(); + LatencyCalibration::Figure figure; + figure.roundTrip = 0.3; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = reported.reportedOutput; + figure.reportedInput = reported.reportedInput; + LatencyCalibration::Key key; + key.rate = rate; + QSettings settings; + LatencyCalibration::store(settings, key, figure); + settings.setValue("MainWindow/prerollseconds", 3.0); + m_window->setPlayReferenceWhileRecording(false); + m_window->setPreRoll(true); + m_window->setRecordIntoSelection(false); + } + + // A run ended early: finished once, every check skipped, nothing + // recording, and none of the user's state changed + void verifyEndedEarly(QStringList togglesBefore, + QStringList storedBefore) { + QCOMPARE(m_finished, 1); + QVERIFY(m_report.failure != ""); + QVERIFY(!m_window->devChecks()->isRunning()); + QVERIFY(!m_window->audioCheck()->isRunning()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->audioCheckTakes()); + QCOMPARE(int(m_report.checks.size()), 2); + for (const CheckResult &c : m_report.checks) { + QVERIFY2(c.verdict == CheckResult::Verdict::Skipped, describe()); + QVERIFY2(c.message.contains(m_report.failure), describe()); + } + QCOMPARE(lastReportLine(), + QString("Totals: 0 passed, 0 failed, 0 measured, 2 skipped")); + + QTest::qWait(500); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->recordTarget()->isRecording()); + QCOMPARE(toggles(), togglesBefore); + QCOMPARE(storedLatency(), storedBefore); + } + + // Not a slot: QtTest would run it as a test. As TestAudioCheck's + void dismissDialog() { + QWidget *modal = QApplication::activeModalWidget(); + if (!modal) return; + QString description = modal->windowTitle(); + if (auto box = qobject_cast(modal)) { + description += ": " + box->text(); + m_dialogs.push_back(description); + QList buttons = box->buttons(); + if (!buttons.isEmpty()) { + buttons.last()->click(); + return; + } + } else { + m_dialogs.push_back(description); + } + if (auto dialog = qobject_cast(modal)) { + dialog->reject(); + } else { + modal->close(); + } + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + + // The check's references go where Qt keeps the tests' data + QStandardPaths::setTestModeEnabled(true); + + QSettings().clear(); + + // Otherwise the MainWindow constructor asks, in a dialog + QSettings settings; + settings.beginGroup("Preferences"); + settings.setValue(QString("network-permission-%1").arg(TONY_VERSION), + false); + settings.endGroup(); + + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + + // Recordings and takes go here and not among the user's + sv::RecordDirectory::setRecordContainerDirectory + (m_dir.filePath("recorded")); + + connect(&m_watchdog, &QTimer::timeout, + this, [this]() { dismissDialog(); }); + m_watchdog.start(50); + } + + void init() { + m_dialogs.clear(); + QSettings settings; + settings.beginGroup("MainWindow"); + settings.setValue("playrefwhilerecording", false); + settings.setValue("preroll", false); + settings.setValue("recordintoselection", false); + settings.remove("prerollseconds"); + settings.endGroup(); + settings.beginGroup("Analyser"); + settings.remove(""); + settings.endGroup(); + settings.beginGroup("Preferences"); + settings.remove("audio-playback-device"); + settings.remove("audio-record-device"); + settings.endGroup(); + SingingTakes::setOverwriteConfirmationWanted(true); + } + + void cleanup() { + QSettings().remove("LatencyCalibration"); + + if (m_window) { + if (m_window->recordTarget()->isRecording()) { + m_window->doRecord(); + } + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + } + QVERIFY2(m_dialogs.isEmpty(), + qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); + } + + void cleanupTestCase() { + m_watchdog.stop(); + sv::RecordDirectory::setRecordContainerDirectory(""); + QStandardPaths::setTestModeEnabled(false); + } + + // Each run gets a folder of its own, and the folders of earlier runs + // go, but the one the session open now lives in, and anything not + // named as they are + void dev_checks_scratch_folders() { + QTemporaryDir dir; + QVERIFY(dir.isValid()); + const QString one = dir.filePath("dev-checks-1"); + const QString two = dir.filePath("dev-checks-2"); + + QCOMPARE(DevChecks::nextScratchFolder(dir.path(), ""), one); + QVERIFY(QFileInfo(one).isDir()); + { + QFile session(QDir(one).filePath("dev-checks.ton")); + QVERIFY(session.open(QIODevice::WriteOnly)); + } + QVERIFY(QDir().mkpath(dir.filePath("dev-checks-x"))); + QVERIFY(QDir().mkpath(dir.filePath("mine"))); + + // With the first run's session open + const QString session = QDir(one).filePath("dev-checks.ton"); + QCOMPARE(DevChecks::nextScratchFolder(dir.path(), session), two); + QVERIFY(QFileInfo::exists(session)); + + // With the second's, whose folder holds a takes folder + QVERIFY(QDir().mkpath(QDir(two).filePath("dev-checks.takes"))); + QCOMPARE(DevChecks::nextScratchFolder + (dir.path(), QDir(two).filePath("dev-checks.ton")), one); + QVERIFY(!QFileInfo::exists(session)); + QVERIFY(QFileInfo(QDir(two).filePath("dev-checks.takes")).isDir()); + QCOMPARE(QDir(dir.path()).entryList(QDir::Dirs | QDir::NoDotAndDotDot, + QDir::Name), + QStringList() << "dev-checks-1" << "dev-checks-2" + << "dev-checks-x" << "mine"); + } + + // The loopback fake, placed with its true round trip: every sweep of + // both punch-ins lands within 2 ms, the session saved and opened + // again holds the same take, and each punch-in measured its own start + // gap. The report ends with its totals, and the session open + // afterwards is the one saved in the scratch folder + void dev_checks_pass_with_the_true_round_trip() { + makeWindow(loopback()); + + runDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + + // Line by line, so that none begins a line of the suite's log: + // its Totals would be taken for the suite's + for (const QString &line : reportText().split('\n')) { + qDebug().noquote() << "report:" << line; + } + QVERIFY2(m_report.failure == "", describe()); + QCOMPARE(int(m_report.checks.size()), 2); + const CheckResult *latency = check(1); + const CheckResult *phrases = check(2); + QVERIFY(latency && phrases); + QCOMPARE(latency->name, QString("latency_on_this_machine")); + QCOMPARE(phrases->name, QString("several_phrases_in_one_take")); + QVERIFY2(latency->verdict == CheckResult::Verdict::Pass, describe()); + QVERIFY2(phrases->verdict == CheckResult::Verdict::Pass, describe()); + QCOMPARE(number(*latency, "round trip used"), + QString("%1 ms").arg(roundTrip * 1000.0 / rate, 0, 'f', 1)); + QVERIFY2(number(*latency, "pitch after reopening") + .endsWith("pitch events, the same"), describe()); + + QCOMPARE(m_stages, QStringList() << "1 of 2: Fresh punch-ins" + << "2 of 2: Save and reopen"); + + // Two punch-ins into a reference of the dev layout, recorded in + // the order given + QCOMPARE(int(m_checks.size()), 1); + const LatencyCheck::TakeSummary &s = m_checks[0].summary; + QCOMPARE(int(s.punchIns.size()), 2); + QCOMPARE(s.judged, 4); + QCOMPARE(s.found, 4); + + QVERIFY(QFileInfo(m_report.reportPath).fileName() == "DevChecks.txt"); + QVERIFY(TakesFile::isInFolder(reportDirectory(), m_report.reportPath)); + QCOMPARE(lastReportLine(), + QString("Totals: 2 passed, 0 failed, 0 measured, 0 skipped")); + + QVERIFY(m_report.sessionPath != ""); + QCOMPARE(m_window->sessionFile(), m_report.sessionPath); + QVERIFY2(TakesFile::isInFolder(scratchDirectory(), m_report.sessionPath), + qPrintable(m_report.sessionPath)); + QVERIFY2(TakesFile::isInFolder + (TakesFile::takesFolder(m_report.sessionPath), + m_window->takes()->getAudioPath()), + qPrintable(m_window->takes()->getAudioPath())); + QVERIFY(!m_window->isDocumentModified()); + QVERIFY(!m_window->audioCheckTakes()); + QVERIFY(m_window->recordAction()->isEnabled()); + } + + // The same with a round trip 20 ms too long: every take is spliced + // from 20 ms too late, and lands 20 ms early. Both items fail, and + // the report shows where the sweeps landed + void dev_checks_fail_with_the_round_trip_off() { + makeWindow(loopback()); + + runDevChecks(roundTrip / rate + 0.020); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_report.failure == "", describe()); + const CheckResult *latency = check(1); + const CheckResult *phrases = check(2); + QVERIFY(latency && phrases); + QVERIFY2(latency->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(phrases->verdict == CheckResult::Verdict::Fail, describe()); + + const QString largest = number(*latency, "largest offset"); + QVERIFY2(std::fabs(milliseconds(largest) + 20.0) <= 1.0, describe()); + for (QString label : { QString("punch-in 1, median offset"), + QString("punch-in 2, median offset") }) { + QVERIFY2(std::fabs(milliseconds(number(*phrases, label)) + 20.0) + <= 1.0, describe()); + } + + // What the save and reopen kept is still right: only the placing + QCOMPARE(number(*latency, "offsets after reopening"), + QString("the same")); + + const QString text = reportText(); + for (QString words : { QString("offsets, punch-in 1 (6.30 to 10.20 " + "s): "), + QString("offsets, punch-in 2 (16.80 to 21.20 " + "s): "), + "largest offset: " + largest, + "punch-in 1, median offset: " + + number(*phrases, "punch-in 1, median offset"), + QString("latency_on_this_machine: FAIL"), + QString("several_phrases_in_one_take: FAIL") }) { + QVERIFY2(text.contains(words), qPrintable(words + " not in:\n" + + text)); + } + QCOMPARE(lastReportLine(), + QString("Totals: 0 passed, 2 failed, 0 measured, 0 skipped")); + } + + // Cancelled during a take: the take stops, the run ends once with + // every check skipped, and the user's toggles and stored round trip + // are as they were + void dev_checks_cancelled() { + makeWindow(loopback()); + setUserState(); + const QStringList togglesBefore = toggles(); + const QStringList storedBefore = storedLatency(); + QVERIFY(!storedBefore.isEmpty()); + + startDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY(m_window->audioCheckTakes()); + QVERIFY(!m_window->recordAction()->isEnabled()); + QTest::qWait(500); + + m_window->devChecks()->cancel(); + verifyEndedEarly(togglesBefore, storedBefore); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->recordAction()->isEnabled()); + } + + // The same when the session is closed during a take + void dev_checks_end_when_the_session_closes() { + makeWindow(loopback()); + setUserState(); + const QStringList togglesBefore = toggles(); + const QStringList storedBefore = storedLatency(); + + startDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QTest::qWait(500); + + m_window->doCloseSession(); + verifyEndedEarly(togglesBefore, storedBefore); + } + + // Deleted during a take, as the window's destructor deletes them: + // the audio check's run they started ends with them, silently, and + // the take in progress is left to the window, which stops it at the + // end of its range as any take into a selection. Then the window + // itself, deleted during a take of theirs + void dev_checks_deleted_during_a_run() { + makeWindow(loopback()); + startDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + + m_window->doDeleteDevChecks(); + QVERIFY(!m_window->audioCheck()->isRunning()); + QVERIFY(!m_window->audioCheckTakes()); + QCOMPARE(m_finished, 0); + QTRY_VERIFY_WITH_TIMEOUT(!m_window->recordTarget()->isRecording(), + 30000); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordAction()->isEnabled(), + 10000); + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + + makeWindow(loopback()); + startDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + delete m_window; + m_window = nullptr; + QCOMPARE(m_finished, 0); + } + + // Playback > Calibrate Audio with the checkbox on, as it is by + // default. A calibration that cannot be used (cancelled) is shown + // with a word that the dev checks did not run. Check Again, run to + // the end: the dev checks carry on from it, with the round trip it + // measured, and Cancel then ends them; the result page shows the + // calibration, a line for each check and the report's path + void dev_checks_after_calibrating_from_the_dialog() { + makeWindow(loopback()); + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + QVERIFY(dialog->devChecksWanted()); + dialog->setPlan(shortPlan()); + dialog->setDevOptions(options(-1.0)); + + m_window->discardModifications(); + dialog->startCheck(); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + dialog->cancelCheck(); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + QVERIFY2(dialog->pageText().contains("The dev checks did not run"), + qPrintable(dialog->pageText())); + QVERIFY(!m_window->devChecks()->isRunning()); + QCOMPARE(m_finished, 0); + + dialog->startCheck(); + QTRY_VERIFY_WITH_TIMEOUT(m_window->devChecks()->isRunning(), 60000); + QCOMPARE(int(m_checks.size()), 2); + const AudioCheckResult calibration = m_checks.back(); + QVERIFY(calibration.calibrationUsable()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); + QTRY_VERIFY_WITH_TIMEOUT + (dialog->pageText().contains("Dev checks, stage 1 of 2"), 10000); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); + QVERIFY(!m_window->recordAction()->isEnabled()); + + dialog->cancelCheck(); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->devChecks()->isRunning()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + + const QString words = dialog->pageText(); + for (QString w : { QString("came back steadily"), + QString("They ended early: The dev checks were " + "cancelled."), + QString("Item 1, latency_on_this_machine: Skipped"), + QString("Item 2, several_phrases_in_one_take: " + "Skipped"), + "Report: " + m_report.reportPath }) { + QVERIFY2(words.contains(w), qPrintable(w + " not in: " + words)); + } + QVERIFY(dialog->canUseLatency()); + + // The dev checks were given the round trip the calibration measured + QVERIFY2(reportText().contains + (QString("Round trip for the run: %1 ms") + .arg(calibration.calibratedRoundTrip * 1000.0, 0, 'f', 1)), + qPrintable(reportText())); + } +}; + +#endif +#endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 243e5763..bae1e00a 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -32,6 +32,10 @@ #include "../TakeLayers.h" #include "../TakesFile.h" +#ifdef TONY_DEV_CHECKS +#include "../dev/DevChecks.h" +#endif + #include "version.h" #include "framework/Document.h" @@ -147,6 +151,9 @@ class TestMainWindow : public MainWindow // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } bool isDocumentModified() { return m_documentModified; } + + // As any edit does + void markModified() { documentModified(); } void doCloseSession() { discardModifications(); closeSession(); } void setPlayReferenceWhileRecording(bool on) { @@ -167,6 +174,19 @@ class TestMainWindow : public MainWindow AudioCheckRunner *audioCheck() { return m_audioCheck; } bool audioCheckTakes() { return m_audioCheckTakes; } + // The Record button, as the user presses it + QAction *recordAction() { return m_recordAction; } + +#ifdef TONY_DEV_CHECKS + // The development checks; deleted as the window's destructor deletes + // them, with the window left, and then as a release build has it + DevChecks *devChecks() { return m_devChecks; } + void doDeleteDevChecks() { + delete m_devChecks; + m_devChecks = nullptr; + } +#endif + // Playback > Calibrate Audio, the dialog it shows once it has been // chosen, the lines under it, and the device menus above it QAction *calibrateAudioAction() { return m_calibrateAudioAction; } diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index e8f310ef..7a05c07d 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -15,6 +15,9 @@ #include "TestSingingAnalysis.h" #include "TestRecordWorkflow.h" #include "TestAudioCheck.h" +#ifdef TONY_DEV_CHECKS +#include "TestDevChecks.h" +#endif #include "RunSuite.h" @@ -77,6 +80,14 @@ int main(int argc, char *argv[]) else ++bad; } +#ifdef TONY_DEV_CHECKS + { + TestDevChecks t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } +#endif + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 9abd5f5c..8e9941c8 100644 --- a/meson.build +++ b/meson.build @@ -71,6 +71,21 @@ elif buildtype.startswith('debug') ] endif # get_option('buildtype') +# The development checks (main/dev/, docs/calibrate-audio.md) are in +# every build but a release one: debug, debugoptimized, plain, minsize +# and custom alike. Packages are release builds and have none of it. +# moc has to see the flag too, for what is inside its #ifdefs +dev_checks = not buildtype.startswith('release') +dev_moc_args = [] +if dev_checks + general_defines += [ + '-DTONY_DEV_CHECKS', + ] + dev_moc_args += [ + '-DTONY_DEV_CHECKS', + ] +endif + svcore_moc_args = [] feature_additional_libs = [] feature_include_dirs = [] @@ -1125,15 +1140,28 @@ tony_core_moc_files = qt.preprocess( 'main/SingingTakes.h', ]) -tony_app_moc_files = qt.preprocess( - moc_headers: [ +tony_app_moc_headers = [ 'main/MainWindow.h', 'main/Analyser.h', 'main/AlternatePitchTrack.h', 'main/AudioCheckRunner.h', 'main/CalibrateAudioDialog.h', 'main/CoverageStrip.h', -]) +] + +if dev_checks + tony_app_files += [ + 'main/dev/DevChecks.cpp', + ] + tony_app_moc_headers += [ + 'main/dev/DevChecks.h', + ] +endif + +tony_app_moc_files = qt.preprocess( + moc_headers: tony_app_moc_headers, + moc_extra_arguments: dev_moc_args, +) qt_resource_files = qt.preprocess( qresources: [ @@ -1393,13 +1421,23 @@ tony_core_test_exe = executable( win_subsystem: 'console' ) -tony_app_test_moc_files = qt.preprocess( - moc_headers: [ +tony_app_test_moc_headers = [ 'main/test/TestSingingDocument.h', 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', 'main/test/TestAudioCheck.h', -]) +] + +if dev_checks + tony_app_test_moc_headers += [ + 'main/test/TestDevChecks.h', + ] +endif + +tony_app_test_moc_files = qt.preprocess( + moc_headers: tony_app_test_moc_headers, + moc_extra_arguments: dev_moc_args, +) tony_app_test_exe = executable( 'test-tony-app', From d0d242d0d35ada743aeb6d6e2cd8f68393b61df9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:06:41 +0000 Subject: [PATCH 141/275] fix: one arg() call for messages and names holding paths chained qstring::arg() calls fill a %n that the text substituted by an earlier one contains: a take named "verse %1 %3" was written into the session file wrong, and a path holding %3a or %2f (as android's content uris do) garbled its message. the remaining chains in mainwindow go with the android change that follows. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/Analyser.cpp | 4 ++-- main/TakeAudio.cpp | 6 +++--- main/TakesFile.cpp | 15 +++++++++------ main/test/TestTakesFile.h | 22 ++++++++++++++++++++++ 4 files changed, 36 insertions(+), 11 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index e7cbf079..41d27b7b 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -758,8 +758,8 @@ Analyser::addEmptyAnalyses() QString transformName = tf->getTransformFriendlyName(pyinBase + w.output); if (sourceName != "" && transformName != "") { - model->setObjectName(tr("%1: %2").arg(sourceName) - .arg(transformName)); + model->setObjectName(tr("%1: %2").arg(sourceName, + transformName)); } else if (transformName != "") { model->setObjectName(transformName); } diff --git a/main/TakeAudio.cpp b/main/TakeAudio.cpp index 0c608889..8b97aa85 100644 --- a/main/TakeAudio.cpp +++ b/main/TakeAudio.cpp @@ -58,7 +58,7 @@ std::unique_ptr openWav(QString path, QString &error) std::unique_ptr reader(new WavFileReader(FileSource(path))); if (!reader->isOK()) { error = tr("Failed to read audio file \"%1\": %2") - .arg(path).arg(reader->getError()); + .arg(path, reader->getError()); return {}; } return reader; @@ -162,7 +162,7 @@ QString write(WavFileReader *old, sv_samplerate_t rate, int channels, // The writer puts its file in place even when it is abandoned if (error != "") { QFile::remove(outPath); - return tr("Failed to write \"%1\": %2").arg(outPath).arg(error); + return tr("Failed to write \"%1\": %2").arg(outPath, error); } return ""; @@ -377,7 +377,7 @@ TakeAudio::resample(QString inPath, sv_samplerate_t rate, QString outPath) if (error != "") { QFile::remove(outPath); return tr("Failed to convert \"%1\" to %2 Hz: %3") - .arg(inPath).arg(rate).arg(error); + .arg(inPath, QString::number(rate), error); } return ""; diff --git a/main/TakesFile.cpp b/main/TakesFile.cpp index 7f034228..f2590950 100644 --- a/main/TakesFile.cpp +++ b/main/TakesFile.cpp @@ -81,10 +81,12 @@ TakesFile::toXml(const SingingTakes &takes, QString sessionPath, QString indent) .arg(XmlExportable::encodeEntities(takes.getActiveName())); for (const SingingTakes::Take &take : takes.getTakes()) { + // One arg() for all three: with one each, a "%1" or "%3" in the + // take's name would be filled in by the next xml += QString("%1 \n") - .arg(indent) - .arg(XmlExportable::encodeEntities(take.name)) - .arg(XmlExportable::encodeEntities + .arg(indent, + XmlExportable::encodeEntities(take.name), + XmlExportable::encodeEntities (relativeAudioPath(sessionPath, take.audioPath))); } @@ -257,7 +259,8 @@ TakesFile::freeCopyPath(QString folder, QString fileName) } for (int i = 2; i < 1000; ++i) { - QString name = QString("%1-%2%3").arg(base).arg(i).arg(suffix); + QString name = QString("%1-%2%3") + .arg(base, QString::number(i), suffix); if (!dir.exists(name)) return cleaned(dir.filePath(name)); } @@ -295,13 +298,13 @@ TakesFile::copyTakeAudioInto(SingingTakes &takes, QString folder) freeCopyPath(folder, QFileInfo(take->audioPath).fileName()); if (target == "") { error = tr("Could not find a name to copy the audio of the take " - "\"%1\" under, in \"%2\"").arg(take->name).arg(folder); + "\"%1\" under, in \"%2\"").arg(take->name, folder); break; } if (!QFile::copy(take->audioPath, target)) { error = tr("Could not copy the audio of the take \"%1\" to " - "\"%2\"").arg(take->name).arg(target); + "\"%2\"").arg(take->name, target); break; } diff --git a/main/test/TestTakesFile.h b/main/test/TestTakesFile.h index c968a3d0..da2ff465 100644 --- a/main/test/TestTakesFile.h +++ b/main/test/TestTakesFile.h @@ -150,6 +150,21 @@ private slots: QCOMPARE(read.active, QString("Rock & \"Roll\" <2>")); } + // A name the user gave, and a file name, holding what QString::arg() + // takes for its placeholders: written as they are, and neither taken + // for the other + void names_like_placeholders() { + SingingTakes takes; + takes.addTake("Verse %1 %3"); + takes.restoreTake("C:/songs/100%2 take.wav", Coverage()); + + TakesFile::Takes read = readString(inDocument(TakesFile::toXml(takes))); + QCOMPARE(int(read.takes.size()), 1); + QCOMPARE(read.takes[0].name, QString("Verse %1 %3")); + QCOMPARE(read.takes[0].audioPath, QString("C:/songs/100%2 take.wav")); + QCOMPARE(read.active, QString("Verse %1 %3")); + } + // From a file: bzip2, as a .ton is, and plain XML, which the session // reader also accepts void from_a_file() { @@ -355,6 +370,13 @@ private slots: QCOMPARE(TakesFile::freeCopyPath("", "take-1.wav"), QString()); QCOMPARE(TakesFile::freeCopyPath(folder, ""), QString()); + + // A loaded file's own name, which may hold a '%' + QString percent = TakesFile::freeCopyPath(folder, "100%2.wav"); + QCOMPARE(percent, QDir::cleanPath(folder + "/100%2.wav")); + QVERIFY(writeAudio(percent)); + QCOMPARE(TakesFile::freeCopyPath(folder, "100%2.wav"), + QDir::cleanPath(folder + "/100%2-2.wav")); } // The copy a save makes: one per file, the takes pointing at the From 6cb3c38ce23c942c93079c1c174dfe1a9fb313a2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:06:41 +0000 Subject: [PATCH 142/275] feat: sessions in place on android, scrollable menus, save on suspend with android's all files access, a file picked from the phone's own storage is opened and saved at its real path, so a session keeps its audio and takes folder beside it, as in a folder a sync app mirrors. save session as suggests a name and refuses an empty one; a session picked or saved elsewhere (drive, dropbox) is refused with a message, and the picker's empty document is removed. menus scroll by a dragged finger. on suspend, playback stops, a take is finished as stop does, and a session with a file is saved. build-apk.sh no longer repackages over the previous apk. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/android/build-apk.sh | 15 +- deploy/android/package/AndroidManifest.xml | 14 +- main/AndroidFiles.cpp | 132 ++++++++++- main/AndroidFiles.h | 44 ++++ main/AndroidStorage.cpp | 190 ++++++++++++++++ main/AndroidStorage.h | 68 ++++++ main/MainWindow.cpp | 248 +++++++++++++++++++-- main/MainWindow.h | 27 ++- main/TouchMenuStyle.cpp | 164 ++++++++++++++ main/TouchMenuStyle.h | 77 +++++++ main/main.cpp | 7 + main/test/TestAndroidFiles.h | 186 +++++++++++++++- main/test/TestTouchMenuStyle.h | 207 +++++++++++++++++ main/test/tony-app-test.cpp | 7 + meson.build | 4 + 15 files changed, 1353 insertions(+), 37 deletions(-) create mode 100644 main/AndroidStorage.cpp create mode 100644 main/AndroidStorage.h create mode 100644 main/TouchMenuStyle.cpp create mode 100644 main/TouchMenuStyle.h create mode 100644 main/test/TestTouchMenuStyle.h diff --git a/deploy/android/build-apk.sh b/deploy/android/build-apk.sh index 2a5fa4fd..cf79970b 100755 --- a/deploy/android/build-apk.sh +++ b/deploy/android/build-apk.sh @@ -198,6 +198,11 @@ log=$logs/tony-apk.log rm -f "$log" mkdir -p "$apk_dir" rm -f "$apk" +# Gradle packages a debug APK incrementally, over its last one: a library +# that changed is written anew and the space of the old one stays in the +# file (7 MB each time Tony's own library changes). Without its last one +# it writes the APK afresh +rm -rf "$out/build/outputs/apk" attempt=1 while true; do @@ -273,10 +278,12 @@ echo " zipalign: aligned for 16 KB pages" badging=$("$build_tools/aapt2" dump badging "$apk") echo "$badging" | grep -E "^package:|^application-label:|^sdkVersion:|^targetSdkVersion:|^uses-permission:" | sed 's/^/ /' -if ! echo "$badging" | grep -q "name='android.permission.RECORD_AUDIO'"; then - echo "ERROR: the APK does not ask for RECORD_AUDIO" 1>&2 - exit 1 -fi +for permission in RECORD_AUDIO MANAGE_EXTERNAL_STORAGE; do + if ! echo "$badging" | grep -q "name='android.permission.$permission'"; then + echo "ERROR: the APK does not ask for $permission" 1>&2 + exit 1 + fi +done manifest=$("$build_tools/aapt2" dump xmltree --file AndroidManifest.xml "$apk") if ! echo "$manifest" | grep -q 'screenOrientation.*=6'; then echo "ERROR: the activity is not sensorLandscape" 1>&2 diff --git a/deploy/android/package/AndroidManifest.xml b/deploy/android/package/AndroidManifest.xml index d5fe4a9d..ac61a4f2 100644 --- a/deploy/android/package/AndroidManifest.xml +++ b/deploy/android/package/AndroidManifest.xml @@ -3,11 +3,13 @@ Tony's Android manifest: Qt 6.11's template (src/android/templates/AndroidManifest.xml in Qt for Android), which androiddeployqt fills in where it says "INSERT", with the package - name, the icon, RECORD_AUDIO and landscape orientation set here. - deploy/android/build-apk.sh gives androiddeployqt the folder this is - in, with the icon added from icons/. + name, the icon, RECORD_AUDIO, MANAGE_EXTERNAL_STORAGE and landscape + orientation set here. deploy/android/build-apk.sh gives + androiddeployqt the folder this is in, with the icon added from + icons/. --> + + #include #include +#include #include +#include #include @@ -69,8 +71,10 @@ AndroidFiles::linkVampPlugins(QString libraryDir, << library << " (" << linkError << "), so copied it" << endl; } else { + // One arg() for all: a path may hold a "%1" that a second + // arg() would fill problems << QString("Cannot link or copy %1 to %2: %3") - .arg(library).arg(link).arg(file.errorString()); + .arg(library, link, file.errorString()); SVCERR << "AndroidFiles: " << problems.back() << endl; continue; } @@ -91,8 +95,8 @@ AndroidFiles::checkVampPlugin(QString path) if (!handle) { const char *error = DLERROR(); problem = QString("Cannot load %1: %2") - .arg(path) - .arg(QString::fromLocal8Bit(error ? error : "no reason given")); + .arg(path, + QString::fromLocal8Bit(error ? error : "no reason given")); } else { if (!DLSYM(handle, "vampGetPluginDescriptor")) { problem = QString("%1 is not a Vamp plugin library").arg(path); @@ -130,7 +134,7 @@ AndroidFiles::copyIn(QString source, QString name, QString dir, QSaveFile out(target); if (!out.open(QIODevice::WriteOnly)) { error = QString("cannot write %1: %2") - .arg(target).arg(out.errorString()); + .arg(target, out.errorString()); return ""; } @@ -148,7 +152,7 @@ AndroidFiles::copyIn(QString source, QString name, QString dir, if (n == 0) break; if (out.write(buffer.data(), n) != n) { error = QString("writing %1 failed: %2") - .arg(target).arg(out.errorString()); + .arg(target, out.errorString()); out.cancelWriting(); return ""; } @@ -157,7 +161,7 @@ AndroidFiles::copyIn(QString source, QString name, QString dir, if (!out.commit()) { error = QString("writing %1 failed: %2") - .arg(target).arg(out.errorString()); + .arg(target, out.errorString()); return ""; } @@ -186,3 +190,119 @@ AndroidFiles::safeFileName(QString name) } return safe; } + +// The path of relative under root, or "" if relative would lead out of +// it: a provider's ids do not, but a URI is only a string +static QString +pathUnder(QString root, QString relative) +{ + if (!root.startsWith('/')) return ""; + for (QString part : relative.split('/')) { + if (part == "..") return ""; + } + return QDir::cleanPath(root + "/" + relative); +} + +QString +AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) +{ + const QString scheme("content://"); + if (!uri.startsWith(scheme, Qt::CaseInsensitive)) return ""; + + // Nothing the picker gives has a query or a fragment + QString rest = uri.mid(scheme.size()); + static const QRegularExpression queryOrFragment("[?#]"); + int end = rest.indexOf(queryOrFragment); + if (end >= 0) rest = rest.left(end); + + // Split before decoding: a '/' inside the id is %2F in every form of + // the URI, Android's and QUrl's, while spaces and letters such as 'ä' + // may come either way + QStringList parts = rest.split('/'); + QString authority = parts.takeFirst(); + for (QString &part : parts) { + part = QUrl::fromPercentEncoding(part.toUtf8()); + } + + QString id; + if (parts.size() == 2 && parts[0] == "document") { + id = parts[1]; + } else if (parts.size() == 4 && parts[0] == "tree" && + parts[2] == "document") { + id = parts[3]; + } else { + return ""; + } + + if (authority == "com.android.externalstorage.documents") { + + int colon = id.indexOf(':'); + if (colon <= 0) return ""; + QString volume = id.left(colon); + QString relative = id.mid(colon + 1); + + if (volume == "primary") { + if (primaryRoot == "") return ""; + return pathUnder(primaryRoot, relative); + } + + // A card or USB drive is named by its file system's UUID, which is + // also its folder in /storage + static const QRegularExpression uuid("^[0-9A-Fa-f]+(-[0-9A-Fa-f]+)*$"); + if (uuid.match(volume).hasMatch()) { + return pathUnder("/storage/" + volume, relative); + } + return ""; + } + + if (authority == "com.android.providers.downloads.documents") { + // Only these carry a path; the rest are numbers in a database + const QString raw("raw:/"); + if (!id.startsWith(raw)) return ""; + return pathUnder("/", id.mid(raw.size())); + } + + return ""; +} + +QString +AndroidFiles::suggestedSessionName(QString sessionPath, QString audioPath) +{ + QString from = (sessionPath != "" ? sessionPath : audioPath); + QString base = QFileInfo(from).completeBaseName(); + if (base == "") return ""; + return base + ".ton"; +} + +QString +AndroidFiles::sessionFileName(QString picked) +{ + QString name = picked.trimmed(); + + // Android's storage names a document created with no name "(invalid)" + // (FileUtils.buildValidFatFilename()) + if (name == "(invalid)") return ""; + + int dot = name.lastIndexOf('.'); + if (dot < 0) return (name == "" ? QString() : name + ".ton"); + if (name.left(dot).trimmed().count(QChar('.')) == + name.left(dot).trimmed().size()) { + return ""; // ".ton", ".", "..ton" and the like + } + return name; +} + +bool +AndroidFiles::removeIfEmpty(QString path) +{ + QFileInfo info(path); + if (path == "" || !info.exists() || info.isDir() || info.size() != 0) { + return false; + } + if (!QFile::remove(path)) { + SVCERR << "AndroidFiles: could not remove the empty " << path << endl; + return false; + } + SVCERR << "AndroidFiles: removed the empty " << path << endl; + return true; +} diff --git a/main/AndroidFiles.h b/main/AndroidFiles.h index 4e52b30d..ec7c2326 100644 --- a/main/AndroidFiles.h +++ b/main/AndroidFiles.h @@ -72,6 +72,50 @@ class AndroidFiles * name that is empty or only dots becomes "imported". */ static QString safeFileName(QString name); + + /** + * The real path of the file a content:// URI from Android's file + * picker names, when it lies in the phone's own storage; "" for any + * other URI (a cloud provider's, the media provider's, a download + * known only by number), which has no path Tony could use. + * + * The external storage provider's documents, alone or under a folder + * grant (.../document/ and .../tree//document/), have ids + * "primary:", under the phone's own shared storage, whose root + * primaryRoot is (Environment.getExternalStorageDirectory(): normally + * /storage/emulated/0), and ":" on a card or USB + * drive, under /storage/. The downloads provider names + * some files "raw:". The id is percent-encoded in the + * URI, and may be partly decoded in the string Qt hands over. + */ + static QString pathFromContentUri(QString uri, QString primaryRoot); + + /** + * The name Save Session As suggests on Android, where the system's + * picker suggests none of its own: the session's name if it has a + * file, else the reference audio's with the session extension, else + * "". + */ + static QString suggestedSessionName(QString sessionPath, + QString audioPath); + + /** + * The name to save a session under when the picker returned picked: + * with the session extension added if it has none, as the desktop's + * file dialog adds it; or "" for a name that names nothing (empty, + * only an extension, or the "(invalid)" Android's storage gives a + * document created with an empty name). + */ + static QString sessionFileName(QString picked); + + /** + * Removes the file at path if it is there and empty: the document the + * system's picker makes for a save, when the save is not going to be + * written there. A file with anything in it is left alone. path may + * be a content:// URI, which Qt's QFile removes through the file's + * provider. True if it removed one. + */ + static bool removeIfEmpty(QString path); }; #endif diff --git a/main/AndroidStorage.cpp b/main/AndroidStorage.cpp new file mode 100644 index 00000000..3893f1c4 --- /dev/null +++ b/main/AndroidStorage.cpp @@ -0,0 +1,190 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AndroidStorage.h" +#include "AndroidFiles.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +using std::cerr; +using std::endl; + +AndroidStorage::AndroidStorage(QWidget *parent) : + m_parent(parent), + m_asked(false) +{ +} + +bool +AndroidStorage::hasAllFilesAccess() +{ + if (QNativeInterface::QAndroidApplication::sdkVersion() < 30) { + return false; + } + return QJniObject::callStaticMethod + ("android/os/Environment", "isExternalStorageManager", "()Z"); +} + +QString +AndroidStorage::primaryRoot() +{ + QJniObject directory = QJniObject::callStaticObjectMethod + ("android/os/Environment", "getExternalStorageDirectory", + "()Ljava/io/File;"); + if (!directory.isValid()) return ""; + return directory.callObjectMethod + ("getAbsolutePath", "()Ljava/lang/String;").toString(); +} + +QString +AndroidStorage::pathFor(QString uri) const +{ + return AndroidFiles::pathFromContentUri(uri, primaryRoot()); +} + +bool +AndroidStorage::openSettings(bool thisApp) +{ + QJniObject context = QNativeInterface::QAndroidApplication::context(); + if (!context.isValid()) return false; + + QJniObject action = QJniObject::getStaticObjectField + ("android/provider/Settings", + thisApp ? "ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION" + : "ACTION_MANAGE_ALL_FILES_ACCESS_PERMISSION", + "Ljava/lang/String;"); + if (!action.isValid()) return false; + + QJniObject intent("android/content/Intent", "(Ljava/lang/String;)V", + action.object()); + if (!intent.isValid()) return false; + + if (thisApp) { + QString package = context.callObjectMethod + ("getPackageName", "()Ljava/lang/String;").toString(); + QJniObject uri = QJniObject::callStaticObjectMethod + ("android/net/Uri", "parse", + "(Ljava/lang/String;)Landroid/net/Uri;", + QJniObject::fromString("package:" + package).object()); + intent.callObjectMethod("setData", + "(Landroid/net/Uri;)Landroid/content/Intent;", + uri.object()); + } + + if (!QNativeInterface::QAndroidApplication::isActivityContext()) { + const jint newTask = 0x10000000; // Intent.FLAG_ACTIVITY_NEW_TASK + intent.callObjectMethod("addFlags", "(I)Landroid/content/Intent;", + newTask); + } + + // Called through JNI directly: QJniObject clears the exception a phone + // without the page throws (ActivityNotFoundException) and tells no one + QJniEnvironment env; + jclass contextClass = env->GetObjectClass(context.object()); + jmethodID start = env->GetMethodID(contextClass, "startActivity", + "(Landroid/content/Intent;)V"); + env->DeleteLocalRef(contextClass); + if (!start) { + env.checkAndClearExceptions(); + return false; + } + env->CallVoidMethod(context.object(), start, intent.object()); + if (env.checkAndClearExceptions()) { + cerr << "AndroidStorage: could not open the settings page " + << (thisApp ? "for this app" : "listing the apps") << endl; + return false; + } + return true; +} + +bool +AndroidStorage::ask(QString why) +{ + m_asked = true; + if (hasAllFilesAccess()) return true; + + QString app = QApplication::applicationName(); + + if (QNativeInterface::QAndroidApplication::sdkVersion() < 30) { + QMessageBox::information + (m_parent, tr("Files in the phone's storage"), + tr("%1 cannot use files where they are on this phone" + "

%2

That needs All files access, which Android " + "has from version 11 on.

").arg(app, why)); + return false; + } + + QMessageBox box(QMessageBox::Question, tr("All files access"), + tr("Allow %1 to use files where they are?" + "

%2

This needs All files access. " + "Open Settings, switch on Allow access to manage " + "all files for %1, and come back.

") + .arg(app, why), + QMessageBox::NoButton, m_parent); + QAbstractButton *open = + box.addButton(tr("Open Settings"), QMessageBox::AcceptRole); + box.addButton(tr("Not Now"), QMessageBox::RejectRole); + box.exec(); + if (box.clickedButton() != open) return false; + + // Up while the settings page is, and there when the user comes back: + // it goes by itself if access was given there, and otherwise waits + // to be told, which also covers a settings page shown beside Tony + // (split screen), when Tony never goes away. Qt's own box, not + // Android's: Qt cannot close Android's from here + bool away = false; + QMessageBox wait(QMessageBox::Information, tr("All files access"), + tr("Waiting for All files access

Switch on " + "Allow access to manage all files for %1 on " + "the settings page, then come back here.

") + .arg(app), + QMessageBox::NoButton, m_parent); + wait.setOption(QMessageBox::Option::DontUseNativeDialog); + wait.addButton(tr("Continue"), QMessageBox::AcceptRole); + wait.addButton(tr("Cancel"), QMessageBox::RejectRole); + QObject::connect(qApp, &QGuiApplication::applicationStateChanged, &wait, + [&](Qt::ApplicationState state) { + if (state != Qt::ApplicationActive) { + away = true; + } else if (away && hasAllFilesAccess()) { + wait.accept(); + } + }); + + if (!openSettings(true) && !openSettings(false)) { + QMessageBox::warning + (m_parent, tr("All files access"), + tr("The settings page could not be opened

Open the " + "phone's Settings, then Apps, %1, and allow All files " + "access there (on some phones it is under Special app " + "access).

").arg(app)); + return hasAllFilesAccess(); + } + + wait.exec(); + + bool allowed = hasAllFilesAccess(); + cerr << "AndroidStorage: All files access " + << (allowed ? "given" : "not given") << endl; + return allowed; +} diff --git a/main/AndroidStorage.h b/main/AndroidStorage.h new file mode 100644 index 00000000..633ce855 --- /dev/null +++ b/main/AndroidStorage.h @@ -0,0 +1,68 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_ANDROID_STORAGE_H +#define TONY_ANDROID_STORAGE_H + +#include +#include + +class QWidget; + +/** + * Android's "All files access" (MANAGE_EXTERNAL_STORAGE), with which + * Tony reads and writes the phone's own storage by path, as on the + * desktop: a session opens and saves where it is, with its audio and + * takes folder beside it, in a folder a sync app keeps in step with a + * computer. Android only; the path of a picked file is + * AndroidFiles::pathFromContentUri(), tested on the desktop. + */ +class AndroidStorage +{ + Q_DECLARE_TR_FUNCTIONS(AndroidStorage) + +public: + // parent is what the boxes asking for access are shown over + AndroidStorage(QWidget *parent); + + // Whether Tony may use the phone's shared storage by path. Never + // before Android 11, which has no such permission + static bool hasAllFilesAccess(); + + // The root of the phone's own shared storage, normally + // /storage/emulated/0; "" if Android does not say + static QString primaryRoot(); + + // The real path of a content:// URI the file picker gave, if it names + // a file in the phone's own storage; else "" + QString pathFor(QString uri) const; + + // Asks for All files access: why, in a box, and then the system's + // settings page for it; back from there, checks again. True if Tony + // has it now + bool ask(QString why); + + // Whether ask() has been called since Tony started + bool hasAsked() const { return m_asked; } + +private: + QWidget *m_parent; + bool m_asked; + + // The settings page for All files access, Tony's own or the list of + // apps; false if it could not be opened + static bool openSettings(bool thisApp); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 83977877..1b18a580 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -28,9 +28,13 @@ #ifdef Q_OS_ANDROID #include "AndroidFiles.h" +#include "AndroidStorage.h" #include "OboeAudioIO.h" +#include +#include #include #include +#include #endif #include "framework/Document.h" @@ -246,6 +250,15 @@ MainWindow::MainWindow(AudioMode audioMode, connect(m_audioDeviceCheck, &QTimer::timeout, this, &MainWindow::checkAudioDevice); m_audioDeviceCheck->start(250); + + m_storage = new AndroidStorage(this); + + m_suspendSaveTimer = new QTimer(this); + m_suspendSaveTimer->setInterval(250); + connect(m_suspendSaveTimer, &QTimer::timeout, + this, &MainWindow::saveWhenSuspended); + connect(qApp, &QGuiApplication::applicationStateChanged, + this, &MainWindow::applicationStateChanged); #endif #ifdef Q_OS_MAC @@ -524,6 +537,9 @@ MainWindow::~MainWindow() m_coverageStrip = nullptr; delete m_analyser; delete m_keyReference; +#ifdef Q_OS_ANDROID + delete m_storage; +#endif Profiles::getInstance()->dump(); } @@ -2742,17 +2758,61 @@ MainWindow::getOpenFileName(FileFinder::FileType type) QString path = MainWindowBase::getOpenFileName(type); if (!path.startsWith("content:")) return path; + QString app = QApplication::applicationName(); + // The name the user knows the file by, which Qt asks the file's // provider for: the URI need not contain it QString name = QFileInfo(path).fileName(); + bool session = (QFileInfo(name).suffix().toLower() == "ton"); + + // A file in the phone's own storage has a path, and with All files + // access that is what is opened, as on the desktop: a session finds + // its audio and takes folder beside it, and a session saved beside + // audio opened so finds the audio. Asked for with a session, which + // cannot do without it, and the first time with audio, which can + QString local = m_storage->pathFor(path); + if (local != "") { + if (!AndroidStorage::hasAllFilesAccess() && + (session || !m_storage->hasAsked())) { + m_storage->ask + (session ? + tr("A session keeps its audio and its takes folder beside " + "it, and %1 opens and saves it there, where it is.") + .arg(app) : + tr("Audio opened where it is can have its session saved " + "beside it. Otherwise %1 copies the audio into its own " + "storage.").arg(app)); + } + if (AndroidStorage::hasAllFilesAccess()) { + if (QFileInfo(local).isFile()) { + cerr << "MainWindow::getOpenFileName: opening " << path + << " where it is, " << local << endl; + return local; + } + cerr << "MainWindow::getOpenFileName: " << path << " should be " + << local << ", which is not there" << endl; + } + } - // A session finds its audio and takes beside it, and the picker - // grants access to the one file picked and nothing next to it - if (QFileInfo(name).suffix().toLower() == "ton") { - QMessageBox::warning - (this, tr("Cannot open a session here"), - tr("Sessions cannot be opened from the file picker yet

A session needs its audio and its takes folder beside it, and the picker lets %1 read the one file only. Open an audio file instead.") - .arg(QApplication::applicationName())); + // A session read through the picker comes without the audio and + // takes beside it: the picker lets Tony read the one file only + if (session) { + if (local == "") { + QMessageBox::warning + (this, tr("Cannot open the session"), + tr("The session cannot be opened from here

A session needs its audio and its takes folder beside it, and from here (Downloads, or a cloud app such as Drive) %1 is given the one file only.

Keep sessions in a folder of the phone's own storage, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer, and open them by browsing to that folder in the picker.

") + .arg(app)); + } else if (!AndroidStorage::hasAllFilesAccess()) { + QMessageBox::warning + (this, tr("Cannot open the session"), + tr("%1 may not open the session where it is

A session needs its audio and its takes folder beside it, and %1 can read those only with All files access. Allow it when %1 asks, or in the phone's Settings, Apps, %1.

") + .arg(app)); + } else { + QMessageBox::warning + (this, tr("Cannot open the session"), + tr("The session was not found where it should be

%1 looked for it at \"%2\".

") + .arg(app, local.toHtmlEscaped())); + } return ""; } @@ -2764,13 +2824,155 @@ MainWindow::getOpenFileName(FileFinder::FileType type) QMessageBox::critical (this, tr("Failed to open file"), tr("File open failed

\"%1\" could not be copied into %2's own storage: %3") - .arg(name.toHtmlEscaped()) - .arg(QApplication::applicationName()) - .arg(error.toHtmlEscaped())); + .arg(name.toHtmlEscaped(), app, error.toHtmlEscaped())); } return copy; } +QString +MainWindow::getSaveFileName(FileFinder::FileType type) +{ + // Exported layers, audio and images go through svgui's dialog as + // before + if (type != FileFinder::SessionFile) { + return MainWindowBase::getSaveFileName(type); + } + + QString app = QApplication::applicationName(); + + // Asked before the picker: without it there is nowhere a session can + // be saved, and the picker makes the document it is asked for + if (!AndroidStorage::hasAllFilesAccess() && + !m_storage->ask(tr("A session keeps its audio and its takes folder " + "beside it, and %1 saves it there, where it is.") + .arg(app))) { + return ""; + } + + QFileDialog dialog(this, tr("Select a session file")); + dialog.setAcceptMode(QFileDialog::AcceptSave); + dialog.setFileMode(QFileDialog::AnyFile); + + // The type of document the picker makes: Android knows nothing of + // .ton, and "*.ton" would come to this anyway + dialog.setMimeTypeFilters({ "application/octet-stream" }); + + // What the picker's name field starts with (EXTRA_TITLE). There is no + // default suffix: Qt adds that to the path of the URI, where it names + // another document + QString suggested = + AndroidFiles::suggestedSessionName(m_sessionFile, m_audioFile); + if (suggested != "") dialog.selectFile(suggested); + + if (!dialog.exec()) return ""; + QStringList selected = dialog.selectedFiles(); + if (selected.empty() || selected[0] == "") return ""; + QString picked = selected[0]; + + // The picker has made an empty document of that name by now + QString local = (picked.startsWith("content:") ? + m_storage->pathFor(picked) : picked); + if (local == "") { + AndroidFiles::removeIfEmpty(picked); + QMessageBox::warning + (this, tr("Cannot save the session there"), + tr("The session cannot be saved there

A session keeps its audio and its takes folder beside it, which %1 can write only in a folder of the phone's own storage, not in Downloads or through a cloud app.

Browse to a folder of the phone's own storage in the picker, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer.

") + .arg(app)); + return ""; + } + + QString name = AndroidFiles::sessionFileName(QFileInfo(local).fileName()); + if (name == "") { + AndroidFiles::removeIfEmpty(local); + QMessageBox::warning + (this, tr("No file name"), + tr("The session was not saved

It needs a name: type one in the picker's name field.

")); + return ""; + } + + // Named without the extension, the document the picker made is not + // the one saved: the extension is added, as the desktop's dialog adds + // it + QString path = QFileInfo(local).dir().filePath(name); + if (path != local) AndroidFiles::removeIfEmpty(local); + + // The picker makes an empty document, so one with something in it + // was there already, and the picker may not have asked + QFileInfo target(path); + if (target.exists() && target.size() > 0 && + QMessageBox::question + (this, tr("File exists"), + tr("File exists

The file \"%1\" already exists.\nDo you want to overwrite it?").arg(path), + QMessageBox::Ok, QMessageBox::Cancel) != QMessageBox::Ok) { + return ""; + } + + cerr << "MainWindow::getSaveFileName: the session goes to " << path + << " (picked as " << picked << ")" << endl; + return path; +} + +void +MainWindow::applicationStateChanged(Qt::ApplicationState state) +{ + if (state != Qt::ApplicationSuspended) return; + + cerr << "MainWindow::applicationStateChanged: going into the " + << "background" << endl; + + // A take keeps what was sung, through the Stop path of the Record + // button, as when the audio device fails (checkAudioDevice()) + if (m_recordTarget && m_recordTarget->isRecording()) { + record(); + } else if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + + // Only a session that has a file of its own: one never saved stays as + // it is, as there is no one to ask where it should go + if (m_sessionFile == "" || !m_documentModified) return; + + m_suspendSavePath = m_sessionFile; + saveWhenSuspended(); +} + +void +MainWindow::saveWhenSuspended() +{ + m_suspendSaveTimer->stop(); + + // Another session since, or saved since + if (m_suspendSavePath == "" || m_suspendSavePath != m_sessionFile || + !m_documentModified) { + m_suspendSavePath = ""; + return; + } + + // Not in the middle of something that shows a box or the picker (the + // picker and the settings page send Tony into the background too), + // which may be about to save, or to decide not to + if (QThread::currentThread()->loopLevel() > 1) { + cerr << "MainWindow::saveWhenSuspended: a dialog is open; the " + << "session is not saved" << endl; + m_suspendSavePath = ""; + return; + } + + // Saving waits for the analysis of a take to be merged, which needs + // the event loop that Android is about to hold: the save is made + // once it is done, which is when Tony is back + if (m_analyser2 && m_analyser2->isAnalysingRange()) { + cerr << "MainWindow::saveWhenSuspended: the session is saved when " + << "the analysis of the take is done" << endl; + m_suspendSaveTimer->start(); + return; + } + + cerr << "MainWindow::saveWhenSuspended: saving " << m_sessionFile << endl; + m_suspendSavePath = ""; + saveSession(); +} + void MainWindow::createAudioIO() { @@ -4833,7 +5035,7 @@ MainWindow::rebuildSingingTrackFromTake(const Coverage::Range &placed) tr("The recording was added to the singing track, but it " "could not be shown

%1

What is on screen is the " "singing track as it was. The recording is in the take's " - "audio file, \"%2\".

").arg(error).arg(path), + "audio file, \"%2\".

").arg(error, path), QMessageBox::Ok); return false; } @@ -5080,7 +5282,7 @@ MainWindow::eraseSingingInSelection() tr("The singing was erased, but the result could not be " "shown

%1

What is on screen is the singing track " "as it was. The erased audio is in the take's audio file, " - "\"%2\".

").arg(showError).arg(path), + "\"%2\".

").arg(showError, path), QMessageBox::Ok); } @@ -5631,7 +5833,7 @@ MainWindow::applyTakeState(SingingTakeCommand *command, const TakeState &state) tr("Failed to show the singing track"), tr("The singing track could not be shown as it was" "

%1

The take's audio is in the file \"%2\".

") - .arg(error).arg(state.path), + .arg(error, state.path), QMessageBox::Ok); } @@ -5915,7 +6117,7 @@ MainWindow::activateTake(bool warnIfNoAudio) (this, tr("Failed to open the take's audio"), tr("The take \"%1\" is shown without its audio

%2

") - .arg(name).arg(error), + .arg(name, error), QMessageBox::Ok); } @@ -6067,7 +6269,7 @@ MainWindow::duplicateTake() documentModified(); emit activity(tr("Copied the take \"%1\" into \"%2\"") - .arg(from).arg(name)); + .arg(from, name)); } void @@ -6113,7 +6315,7 @@ MainWindow::renameTake() documentModified(); emit activity(tr("The take \"%1\" is called \"%2\" now") - .arg(current).arg(name)); + .arg(current, name)); } void @@ -6552,6 +6754,10 @@ bool MainWindow::saveSessionToPath(QString path) { if (!saveSessionFile(path)) { +#ifdef Q_OS_ANDROID + // Save As picked it, and the picker made it, empty + AndroidFiles::removeIfEmpty(path); +#endif QMessageBox::critical(this, tr("Failed to save file"), tr("Session file \"%1\" could not be saved.").arg(path)); return false; @@ -6586,6 +6792,10 @@ MainWindow::saveSessionAs() } if (!waitForInitialAnalysis()) { +#ifdef Q_OS_ANDROID + // The picker made it, empty + AndroidFiles::removeIfEmpty(path); +#endif QMessageBox::warning(this, tr("File not saved"), tr("Wait cancelled: the session has not been saved.")); return; @@ -7733,14 +7943,14 @@ MainWindow::modelRegenerationFailed(QString layerName, (this, tr("Failed to regenerate layer"), tr("Layer generation failed

Failed to regenerate derived layer \"%1\" using new data model as input.

The layer transform \"%2\" failed:

%3") - .arg(layerName).arg(transformName).arg(message), + .arg(layerName, transformName, message), QMessageBox::Ok); } else { QMessageBox::warning (this, tr("Failed to regenerate layer"), tr("Layer generation failed

Failed to regenerate derived layer \"%1\" using new data model as input.

The layer transform \"%2\" failed.

No error information is available.") - .arg(layerName).arg(transformName), + .arg(layerName, transformName), QMessageBox::Ok); } } @@ -7751,7 +7961,7 @@ MainWindow::modelRegenerationWarning(QString layerName, QString message) { QMessageBox::warning - (this, tr("Warning"), tr("Warning when regenerating layer

When regenerating the derived layer \"%1\" using new data model as input:

%2").arg(layerName).arg(message), QMessageBox::Ok); + (this, tr("Warning"), tr("Warning when regenerating layer

When regenerating the derived layer \"%1\" using new data model as input:

%2").arg(layerName, message), QMessageBox::Ok); } void diff --git a/main/MainWindow.h b/main/MainWindow.h index f9cce290..9ffa0293 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -33,6 +33,7 @@ #ifdef Q_OS_ANDROID #include +class AndroidStorage; #endif class QTimer; @@ -909,10 +910,32 @@ protected slots: #ifdef Q_OS_ANDROID // Android's file picker gives content:// URIs, which svcore's readers - // cannot open: the file picked is copied into the app's own storage, - // and the copy's path returned + // cannot open. A file in the phone's own storage is opened where it + // is, by its path, once Tony has All files access (AndroidStorage), + // so that a session finds its audio and takes beside it. Other audio + // is copied into the app's own storage and the copy's path returned; + // other sessions are refused QString getOpenFileName(sv::FileFinder::FileType type) override; + // Save Session As: Tony's own picker, which suggests a name (svgui's + // suggests none), and then the path of the file picked in the phone's + // own storage, or nothing. The picker has made an empty document by + // then, which is removed if it is not to be the session. Other files + // are saved as before + QString getSaveFileName(sv::FileFinder::FileType type) override; + AndroidStorage *m_storage; + + // Android sends Tony to the background: playback stops, a take being + // recorded is finished as Stop finishes it, and the session is saved + // as Save saves it, if it has a file of its own. Android holds Tony's + // event loop from the moment this returns until Tony is back, so + // nothing here may wait on it: a save that has to wait for the + // analysis of a take is made when that is done + void applicationStateChanged(Qt::ApplicationState state); + void saveWhenSuspended(); + QTimer *m_suspendSaveTimer; + QString m_suspendSavePath; + // The audio device is Oboe's (OboeAudioIO): bqaudioio has no // Android backend. Its input only once the microphone may be used void createAudioIO() override; diff --git a/main/TouchMenuStyle.cpp b/main/TouchMenuStyle.cpp new file mode 100644 index 00000000..07137b76 --- /dev/null +++ b/main/TouchMenuStyle.cpp @@ -0,0 +1,164 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "TouchMenuStyle.h" + +#include +#include +#include +#include +#include + +#include + +TouchMenuStyle::TouchMenuStyle(QStyle *base) : + QProxyStyle(base), + m_pressY(0), + m_scrolledToY(0), + m_dragging(false) +{ +} + +int +TouchMenuStyle::styleHint(StyleHint hint, + const QStyleOption *option, + const QWidget *widget, + QStyleHintReturn *returnData) const +{ + if (hint == SH_Menu_Scrollable) return 1; + return QProxyStyle::styleHint(hint, option, widget, returnData); +} + +int +TouchMenuStyle::pixelMetric(PixelMetric metric, + const QStyleOption *option, + const QWidget *widget) const +{ + int value = QProxyStyle::pixelMetric(metric, option, widget); + + // A strip ten pixels high at the menu's edge is where a finger has to + // press for the arrows to scroll it; missing it chooses the item + // beside it + if (metric == PM_MenuScrollerHeight) return 3 * value; + + return value; +} + +void +TouchMenuStyle::polish(QWidget *widget) +{ + QProxyStyle::polish(widget); + if (QMenu *menu = qobject_cast(widget)) { + menu->installEventFilter(this); + } +} + +void +TouchMenuStyle::unpolish(QWidget *widget) +{ + if (QMenu *menu = qobject_cast(widget)) { + menu->removeEventFilter(this); + } + QProxyStyle::unpolish(widget); +} + +int +TouchMenuStyle::rowHeight(QMenu *menu) +{ + // The first item that is not a separator, which are thinner + for (QAction *action : menu->actions()) { + if (!action->isVisible() || action->isSeparator()) continue; + int height = menu->actionGeometry(action).height(); + if (height > 0) return height; + } + return std::max(8, menu->fontMetrics().height()); +} + +bool +TouchMenuStyle::eventFilter(QObject *object, QEvent *event) +{ + QMenu *menu = qobject_cast(object); + if (!menu) return false; + + switch (event->type()) { + + case QEvent::MouseButtonPress: { + // Passed on: the press may be a tap, and QMenu takes one on its + // scroll arrows as its own + QMouseEvent *e = static_cast(event); + if (e->button() != Qt::LeftButton) break; + m_menu = menu; + m_pressY = m_scrolledToY = e->globalPosition().y(); + m_dragging = false; + break; + } + + case QEvent::MouseMove: { + QMouseEvent *e = static_cast(event); + if (m_menu != menu || !(e->buttons() & Qt::LeftButton)) break; + + double y = e->globalPosition().y(); + + if (!m_dragging) { + // Until the finger has gone this far, QMenu follows it + if (std::abs(y - m_pressY) < QApplication::startDragDistance()) { + break; + } + m_dragging = true; + // Nothing is chosen by a drag, and a submenu the press opened + // is closed + menu->setActiveAction(nullptr); + } + + // A row for each row's height the finger has moved: the finger + // moving down pulls the items above into view, as a wheel turned + // up does. The wheel event goes where QMenu takes it, inside it + int row = rowHeight(menu); + QPointF centre(menu->rect().center()); + while (std::abs(y - m_scrolledToY) >= row) { + int direction = (y > m_scrolledToY ? 1 : -1); + QWheelEvent wheel(centre, menu->mapToGlobal(centre), QPoint(), + QPoint(0, 120 * direction), Qt::NoButton, + Qt::NoModifier, Qt::NoScrollPhase, false); + QCoreApplication::sendEvent(menu, &wheel); + m_scrolledToY += direction * row; + } + + // Not QMenu's: it would pick out the item under the finger + return true; + } + + case QEvent::MouseButtonRelease: { + if (m_menu != menu) break; + bool dragged = m_dragging; + m_menu = nullptr; + m_dragging = false; + // The end of a drag is not a choice + if (dragged) return true; + break; + } + + case QEvent::Hide: + if (m_menu == menu) { + m_menu = nullptr; + m_dragging = false; + } + break; + + default: + break; + } + + return false; +} diff --git a/main/TouchMenuStyle.h b/main/TouchMenuStyle.h new file mode 100644 index 00000000..6c776278 --- /dev/null +++ b/main/TouchMenuStyle.h @@ -0,0 +1,77 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TOUCH_MENU_STYLE_H +#define TONY_TOUCH_MENU_STYLE_H + +#include +#include + +class QMenu; + +/** + * Menus for a touch screen, as the application's style on Android (the + * style it wraps does everything else). + * + * A menu taller than the screen scrolls, one column high, where Qt's + * styles would lay it out in columns and let the ones that do not fit + * run off the screen: QMenu scrolls if its style answers + * SH_Menu_Scrollable. The scroll arrows it then shows are made tall + * enough for a finger. A finger dragged up or down a menu scrolls it as + * a phone's lists scroll, a row at a time, through the wheel events + * QMenu scrolls by; the lift that ends a drag chooses nothing. A tap + * chooses an item as before. + * + * Built everywhere and tested on the desktop, whose style is left as it + * is: only main() on Android installs it. + */ +class TouchMenuStyle : public QProxyStyle +{ + Q_OBJECT + +public: + // The application's style, or base if given, with menus for touch + TouchMenuStyle(QStyle *base = nullptr); + + int styleHint(StyleHint hint, + const QStyleOption *option = nullptr, + const QWidget *widget = nullptr, + QStyleHintReturn *returnData = nullptr) const override; + + int pixelMetric(PixelMetric metric, + const QStyleOption *option = nullptr, + const QWidget *widget = nullptr) const override; + + // Each menu the style is given watches for a finger dragged over it + void polish(QWidget *widget) override; + void unpolish(QWidget *widget) override; + using QProxyStyle::polish; + using QProxyStyle::unpolish; + +protected: + bool eventFilter(QObject *object, QEvent *event) override; + +private: + // The height a finger moves to scroll the menu by one row + static int rowHeight(QMenu *menu); + + // The menu pressed on, where, how far the scrolling has followed the + // finger, and whether the press has become a drag + QPointer m_menu; + double m_pressY; + double m_scrolledToY; + bool m_dragging; +}; + +#endif diff --git a/main/main.cpp b/main/main.cpp index e589582a..cbeef1ee 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -45,6 +45,7 @@ #ifdef Q_OS_ANDROID #include "AndroidFiles.h" +#include "TouchMenuStyle.h" #include #include #include @@ -299,6 +300,12 @@ main(int argc, char **argv) QGuiApplication::styleHints()->setColorScheme(Qt::ColorScheme::Light); #endif +#ifdef Q_OS_ANDROID + // Menus that scroll, by a finger dragged over them, rather than run + // off the screen. Before any widget is made, so that every menu has it + QApplication::setStyle(new TouchMenuStyle); +#endif + QApplication::setOrganizationName("sonic-visualiser"); QApplication::setOrganizationDomain("sonicvisualiser.org"); QApplication::setApplicationName("Tony"); diff --git a/main/test/TestAndroidFiles.h b/main/test/TestAndroidFiles.h index b25ae764..54e82b7c 100644 --- a/main/test/TestAndroidFiles.h +++ b/main/test/TestAndroidFiles.h @@ -15,9 +15,12 @@ #define TEST_ANDROID_FILES_H // Tier 1: the file work the Android build does at start (the Vamp plugin -// links) and when a file is picked (the copy into app storage), done here -// on plain files in a temporary directory. What only a phone has -- the -// content:// URI, the installed library directory -- is not here. +// links) and when a file is picked (the copy into app storage, the path +// of a picked content:// URI, the names Save Session As suggests and +// accepts), done here on plain files in a temporary directory and on +// URIs written as Android writes them. What only a phone has -- a +// provider behind the URI, the installed library directory -- is not +// here. #include "../AndroidFiles.h" @@ -30,6 +33,7 @@ #include #include #include +#include class TestAndroidFiles : public QObject { @@ -168,6 +172,182 @@ private slots: QCOMPARE(AndroidFiles::safeFileName(".."), QString("imported")); } + // --- A picked document's path --- + + // As Android's Uri.toString() gives it, which QFileDialog then passes + // through QUrl: the string Tony gets from the picker. Tested in both + // forms, since QUrl decodes some of the id and not the rest + static QString pickedAsQtGivesIt(QString uri) { + return QUrl(uri).toString(QUrl::PreferLocalFile); + } + + void a_document_in_the_phones_storage_has_its_path() { + QString root = "/storage/emulated/0"; + QString uri = "content://com.android.externalstorage.documents/" + "document/primary%3AMusic%2Ftest.ton"; + + QCOMPARE(AndroidFiles::pathFromContentUri(uri, root), + QString("/storage/emulated/0/Music/test.ton")); + QCOMPARE(AndroidFiles::pathFromContentUri(pickedAsQtGivesIt(uri), root), + QString("/storage/emulated/0/Music/test.ton")); + + // The root is the phone's, whatever it is + QCOMPARE(AndroidFiles::pathFromContentUri(uri, "/storage/emulated/10"), + QString("/storage/emulated/10/Music/test.ton")); + } + + void names_with_spaces_and_finnish_letters_are_decoded() { + QString root = "/storage/emulated/0"; + // "Music/Laulut/Sävel äänessä 100%.ton", as Android encodes it + QString uri = "content://com.android.externalstorage.documents/" + "document/primary%3AMusic%2FLaulut%2FS%C3%A4vel%20%C3%A4%C3%A4ness" + "%C3%A4%20100%25.ton"; + QString expected("/storage/emulated/0/Music/Laulut/" + "Sävel äänessä 100%.ton"); + + QCOMPARE(AndroidFiles::pathFromContentUri(uri, root), expected); + QString given = pickedAsQtGivesIt(uri); + QCOMPARE(AndroidFiles::pathFromContentUri(given, root), expected); + } + + void a_document_under_a_folder_grant_has_its_path() { + QString uri = "content://com.android.externalstorage.documents/" + "tree/primary%3ASync%2FSongs/document/" + "primary%3ASync%2FSongs%2FMy%20Song.ton"; + QCOMPARE(AndroidFiles::pathFromContentUri(uri, "/storage/emulated/0"), + QString("/storage/emulated/0/Sync/Songs/My Song.ton")); + } + + void the_top_of_the_phones_storage_is_its_root() { + QString uri = "content://com.android.externalstorage.documents/" + "document/primary%3A"; + QCOMPARE(AndroidFiles::pathFromContentUri(uri, "/storage/emulated/0/"), + QString("/storage/emulated/0")); + } + + void a_document_on_a_card_is_under_its_volume() { + QString uri = "content://com.android.externalstorage.documents/" + "document/1A2B-3C4D%3AMusic%2FSong.mp3"; + QCOMPARE(AndroidFiles::pathFromContentUri(uri, "/storage/emulated/0"), + QString("/storage/1A2B-3C4D/Music/Song.mp3")); + } + + void a_download_with_a_raw_id_has_that_path() { + QString uri = "content://com.android.providers.downloads.documents/" + "document/raw%3A%2Fstorage%2Femulated%2F0%2FDownload%2FSong.mp3"; + QCOMPARE(AndroidFiles::pathFromContentUri(uri, "/storage/emulated/0"), + QString("/storage/emulated/0/Download/Song.mp3")); + QCOMPARE(AndroidFiles::pathFromContentUri(pickedAsQtGivesIt(uri), + "/storage/emulated/0"), + QString("/storage/emulated/0/Download/Song.mp3")); + } + + void documents_elsewhere_have_no_path() { + QString root = "/storage/emulated/0"; + QStringList none = { + // Google Drive, Dropbox: a cloud provider's own ids + "content://com.google.android.apps.docs.storage/document/" + "acc%3D1%3Bdoc%3Dencoded%3DabcDEF", + "content://com.dropbox.android.document/document/" + "%2FSongs%2Ftest.ton", + // The media provider and a download, by number + "content://com.android.providers.media.documents/document/audio%3A42", + "content://com.android.providers.downloads.documents/document/msf%3A1234", + "content://com.android.providers.downloads.documents/document/1234", + // A raw id that is not a path + "content://com.android.providers.downloads.documents/document/raw%3ASong.mp3", + // The external storage provider, but no volume Tony knows + "content://com.android.externalstorage.documents/document/home%3ADocuments", + "content://com.android.externalstorage.documents/document/Music%2Ftest.ton", + "content://com.android.externalstorage.documents/document/%3AMusic", + // Not a document + "content://com.android.externalstorage.documents/tree/primary%3AMusic", + "content://com.android.externalstorage.documents/root/primary", + // Not content:// at all + "/storage/emulated/0/Music/test.ton", + "file:///storage/emulated/0/Music/test.ton", + "", + }; + for (QString uri : none) { + QCOMPARE(AndroidFiles::pathFromContentUri(uri, root), QString()); + } + + // Without the root there is nowhere to put primary's documents + QCOMPARE(AndroidFiles::pathFromContentUri + ("content://com.android.externalstorage.documents/" + "document/primary%3AMusic%2Ftest.ton", ""), + QString()); + } + + void an_id_cannot_lead_out_of_its_volume() { + QString root = "/storage/emulated/0"; + QCOMPARE(AndroidFiles::pathFromContentUri + ("content://com.android.externalstorage.documents/" + "document/primary%3A..%2F..%2F..%2Fdata%2Fx", root), + QString()); + QCOMPARE(AndroidFiles::pathFromContentUri + ("content://com.android.externalstorage.documents/" + "document/primary%3AMusic%2F..%2F..%2Fx", root), + QString()); + QCOMPARE(AndroidFiles::pathFromContentUri + ("content://com.android.providers.downloads.documents/" + "document/raw%3A%2Fstorage%2F..%2Fdata%2Fx", root), + QString()); + } + + // --- The name Save Session As suggests and accepts --- + + void the_suggested_name_is_the_sessions_or_the_references() { + QCOMPARE(AndroidFiles::suggestedSessionName + ("/storage/emulated/0/Music/My Song.ton", + "/storage/emulated/0/Music/Other.mp3"), + QString("My Song.ton")); + QCOMPARE(AndroidFiles::suggestedSessionName + ("", "/data/user/0/io.github.jhhr.tony/files/imported/" + "Ääni v1.2.mp3"), + QString("Ääni v1.2.ton")); + QCOMPARE(AndroidFiles::suggestedSessionName("", ""), QString()); + } + + void a_picked_name_without_an_extension_gets_one() { + QCOMPARE(AndroidFiles::sessionFileName("test.ton"), QString("test.ton")); + QCOMPARE(AndroidFiles::sessionFileName("test"), QString("test.ton")); + QCOMPARE(AndroidFiles::sessionFileName("My Song v1.2"), + QString("My Song v1.2")); + QCOMPARE(AndroidFiles::sessionFileName("test (1).ton"), + QString("test (1).ton")); + } + + void a_picked_name_that_names_nothing_is_refused() { + QCOMPARE(AndroidFiles::sessionFileName(""), QString()); + QCOMPARE(AndroidFiles::sessionFileName(" "), QString()); + QCOMPARE(AndroidFiles::sessionFileName(".ton"), QString()); + QCOMPARE(AndroidFiles::sessionFileName(" .ton"), QString()); + QCOMPARE(AndroidFiles::sessionFileName("."), QString()); + QCOMPARE(AndroidFiles::sessionFileName("..ton"), QString()); + QCOMPARE(AndroidFiles::sessionFileName("(invalid)"), QString()); + } + + void only_an_empty_document_is_removed() { + QString dir = newDir("picked"); + + // What the picker leaves for a save that is not made there + QString empty = dir + "/test.ton"; + QVERIFY(writeFile(empty, "")); + QVERIFY(AndroidFiles::removeIfEmpty(empty)); + QVERIFY(!QFileInfo::exists(empty)); + + // A session that was there before is not to be touched + QString session = dir + "/Song.ton"; + QVERIFY(writeFile(session, "BZh91AY&SY")); + QVERIFY(!AndroidFiles::removeIfEmpty(session)); + QCOMPARE(readFile(session), QByteArray("BZh91AY&SY")); + + QVERIFY(!AndroidFiles::removeIfEmpty(dir + "/not-there.ton")); + QVERIFY(!AndroidFiles::removeIfEmpty(newDir("folder"))); + QVERIFY(!AndroidFiles::removeIfEmpty("")); + } + // --- The Vamp plugin links --- void the_plugins_are_linked_under_their_own_names() { diff --git a/main/test/TestTouchMenuStyle.h b/main/test/TestTouchMenuStyle.h new file mode 100644 index 00000000..8b638139 --- /dev/null +++ b/main/test/TestTouchMenuStyle.h @@ -0,0 +1,207 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_TOUCH_MENU_STYLE_H +#define TEST_TOUCH_MENU_STYLE_H + +// Tier 5: menus taller than the screen, as a phone has them +// (TouchMenuStyle): on the screen in one column, scrolled by a finger +// dragged over them, every item reachable, and a tap still a choice. +// The menus are real popups on the offscreen platform's screen; the +// mouse events are what Qt makes of a finger on Android. + +#include "../TouchMenuStyle.h" + +#include +#include +#include +#include +#include +#include +#include + +#include + +class TestTouchMenuStyle : public QObject +{ + Q_OBJECT + + // Taller than the screen by some way: the phone's Edit menu is about + // two screens high in landscape + static constexpr int itemCount = 80; + + std::unique_ptr m_style; + std::unique_ptr m_menu; + + QRect screen() const { + return QGuiApplication::primaryScreen()->availableGeometry(); + } + + // A menu of itemCount items with the touch style, popped up at the + // top of the screen. The items are as wide as Tony's longest: laid + // out in columns, as Qt's styles lay out a menu taller than the + // screen, the columns would not fit across it either + QMenu *longMenu() { + m_menu.reset(new QMenu); + m_menu->setStyle(m_style.get()); + for (int i = 0; i < itemCount; ++i) { + m_menu->addAction(QString("Item %1: Save Session to Audio File " + "Path").arg(i)); + } + m_menu->popup(screen().topLeft()); + return m_menu.get(); + } + + QAction *item(int i) { return m_menu->actions().at(i); } + + int top(int i) { return m_menu->actionGeometry(item(i)).top(); } + + // On show: inside the menu, and across the screen, not off its side + // as columns that do not fit are. (The menu itself fits the screen's + // height, or nearly: the offscreen platform puts a popup two pixels + // in from where Qt asks) + bool shows(int i) { + QRect r = m_menu->actionGeometry(item(i)); + QRect global(m_menu->mapToGlobal(r.topLeft()), r.size()); + return r.top() >= 0 && r.bottom() < m_menu->height() && + global.left() >= screen().left() && + global.right() <= screen().right(); + } + + // A finger put down at from and moved to to, a few pixels at a time, + // then lifted there + void drag(QPoint from, QPoint to) { + QTest::mousePress(m_menu.get(), Qt::LeftButton, {}, from); + QPoint step = (to - from) / 20; + for (int i = 1; i <= 20; ++i) { + QTest::mouseMove(m_menu.get(), from + step * i); + } + QTest::mouseRelease(m_menu.get(), Qt::LeftButton, {}, from + step * 20); + } + +private slots: + void init() { + m_style.reset(new TouchMenuStyle); + } + + void cleanup() { + m_menu.reset(); + m_style.reset(); + } + + void a_long_menu_is_on_the_screen_in_one_column() { + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + + QVERIFY2(menu->height() <= screen().height(), + qPrintable(QString("%1 high on a screen %2 high") + .arg(menu->height()).arg(screen().height()))); + + // One column: every item at the same place across, and the ones + // that do not fit below, to be scrolled to + for (int i = 1; i < itemCount; ++i) { + QCOMPARE(menu->actionGeometry(item(i)).left(), + menu->actionGeometry(item(0)).left()); + } + QVERIFY(shows(0)); + QVERIFY(!shows(itemCount - 1)); + QVERIFY(top(itemCount - 1) >= menu->height()); + + // Arrows a finger can hit + QVERIFY(m_style->pixelMetric(QStyle::PM_MenuScrollerHeight) >= 30); + } + + void a_finger_dragged_up_scrolls_the_menu_and_chooses_nothing() { + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + QSignalSpy triggered(menu, &QMenu::triggered); + + int row = menu->actionGeometry(item(0)).height(); + QVERIFY(row > 0); + int before = top(10); + + QPoint centre = menu->rect().center(); + drag(centre, centre - QPoint(0, 10 * row)); + + // Ten rows' drag, ten rows further on (give or take the arrow + // strip that appears at the top once it has scrolled) + int moved = before - top(10); + QVERIFY2(moved >= 8 * row && moved <= 11 * row, + qPrintable(QString("moved %1 for rows of %2") + .arg(moved).arg(row))); + QCOMPARE(triggered.count(), 0); + QVERIFY(menu->isVisible()); + QVERIFY(!shows(0)); + } + + void every_item_can_be_reached_and_the_way_back_too() { + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + QSignalSpy triggered(menu, &QMenu::triggered); + + QPoint low(menu->width() / 2, menu->height() * 3 / 4); + QPoint high(menu->width() / 2, menu->height() / 4); + + for (int i = 0; i < 10 && !shows(itemCount - 1); ++i) { + drag(low, high); + } + QVERIFY(shows(itemCount - 1)); + + for (int i = 0; i < 10 && !shows(0); ++i) { + drag(high, low); + } + QVERIFY(shows(0)); + + QCOMPARE(triggered.count(), 0); + QVERIFY(menu->isVisible()); + } + + void a_tap_chooses_an_item_as_before() { + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + QSignalSpy triggered(menu, &QMenu::triggered); + + QPoint at = menu->actionGeometry(item(3)).center(); + QTest::mousePress(menu, Qt::LeftButton, {}, at); + QTest::mouseRelease(menu, Qt::LeftButton, {}, at); + + QCOMPARE(triggered.count(), 1); + QCOMPARE(triggered.at(0).at(0).value(), item(3)); + } + + void a_tap_after_scrolling_chooses_the_item_shown_there() { + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + QSignalSpy triggered(menu, &QMenu::triggered); + + QPoint centre = menu->rect().center(); + int row = menu->actionGeometry(item(0)).height(); + int before = menu->actions().indexOf(menu->actionAt(centre)); + drag(centre, centre - QPoint(0, 20 * row)); + + QAction *shown = menu->actionAt(centre); + QVERIFY(shown); + QVERIFY2(menu->actions().indexOf(shown) >= before + 18, + qPrintable(QString("item %1 there before, %2 after") + .arg(before) + .arg(menu->actions().indexOf(shown)))); + + QTest::mousePress(menu, Qt::LeftButton, {}, centre); + QTest::mouseRelease(menu, Qt::LeftButton, {}, centre); + + QCOMPARE(triggered.count(), 1); + QCOMPARE(triggered.at(0).at(0).value(), shown); + } +}; + +#endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 8bd8e2b9..e887af41 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -16,6 +16,7 @@ #include "TestRecordWorkflow.h" #include "TestTouchGestures.h" #include "TestCompactLayout.h" +#include "TestTouchMenuStyle.h" #include "RunSuite.h" @@ -84,6 +85,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestTouchMenuStyle t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index bf4e5a71..9d3bed38 100644 --- a/meson.build +++ b/meson.build @@ -1191,10 +1191,12 @@ tony_app_files = [ 'main/TakeCommands.cpp', 'main/TakeLayers.cpp', 'main/TouchGestures.cpp', + 'main/TouchMenuStyle.cpp', ] if system == 'android' tony_app_files += [ + 'main/AndroidStorage.cpp', 'main/OboeAudioIO.cpp', ] endif @@ -1213,6 +1215,7 @@ tony_app_moc_files = qt.preprocess( 'main/CompactLayout.h', 'main/CoverageStrip.h', 'main/TouchGestures.h', + 'main/TouchMenuStyle.h', ]) qt_resource_files = qt.preprocess( @@ -1521,6 +1524,7 @@ if system != 'android' 'main/test/TestRecordWorkflow.h', 'main/test/TestTouchGestures.h', 'main/test/TestCompactLayout.h', + 'main/test/TestTouchMenuStyle.h', ]) tony_app_test_exe = executable( From 3062d252d084678a5767e9f8e6744683a483bbaa Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:06:41 +0000 Subject: [PATCH 143/275] docs: phase a7 done, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 7599daee..a2fadeb9 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -159,7 +159,7 @@ builds happen in the container.) - A4 — Touch gestures on the panes. Done. - A5 — Compact touch mode. Done. - A6 — Oboe audio backend. Done. -- A7 — Sessions in place on the phone, and fixes from the first phone test. +- A7 — Sessions in place on the phone, and fixes from the first phone test. Done. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -560,3 +560,29 @@ Choices / deviations: The next phase must know: once a take is recorded the device stays duplex (svapp), so Play opens the microphone too; nothing calls `suppressRecordSide()`. Logcat tag Tony: "OboeAudioIO:". Left open: not run on a phone; the first Record press blocks for the open (up to ~1 s). + +### Phase A7 — 2026-09-26 +Built: `AndroidFiles` (core): `pathFromContentUri()` (externalstorage `primary:`/volume-UUID ids, +alone or under a tree; downloads `raw:`; split before decoding; `..` refused), `suggestedSessionName()`, +`sessionFileName()`, `removeIfEmpty()`. `AndroidStorage` (Android only): `isExternalStorageManager()`, +the root, `ask()` (a box, Settings' page for Tony or else the list, then a Qt box that closes when Tony +is back with access). `TouchMenuStyle` (app; main() installs it on Android only). `MainWindow` on +Android: Open maps a pick to its path (asks with a session, once with audio), else copies audio or +refuses a session; Save As runs its own QFileDialog; `applicationStateChanged()`. Manifest: +MANAGE_EXTERNAL_STORAGE (`tools:ignore="ScopedStorage"`; debug builds run no lint). Chained `.arg()` +with paths or names made single calls in main/: TakesFile wrote a take named "a %1" or "%3" wrong. +Choices / deviations: +- Qt 6.11 (androidjnimain.cpp): on Suspended Qt posts the event, then stops the GUI dispatcher until + Active (no `android.app.background_running`): a nested loop in the slot blocks until Tony is back. + So nothing there waits; a save that needs a take's ranged analysis merged is made after return. + A never-saved session is not saved. No save while a dialog or the picker is up (`loopLevel() > 1`). +- QMenu wraps a tall menu into columns (off the side on a phone); scrollable, it has hover arrows + (a tap scrolls to the end), so a finger's drag scrolls it too, through wheel events. +- Save As: `selectFile()` gives EXTRA_TITLE; no default suffix (Qt appends it to the URI's decoded + path); MIME octet-stream. Exports still go through svgui's dialog and the URI. `build-apk.sh` + now deletes Gradle's last APK (incremental packaging left 7 MB holes). +The next phase must know: Android's native QMessageBox cannot be closed from code (its helper's +hide() does not end exec()): DontUseNativeDialog. `activeModalWidget()` misses native dialogs. +Left open: not run on a phone. Below Android 11 no in-place sessions. Downloads (`msf:` ids) +refused for sessions. Save to Audio Path with copied audio saves into app storage. For A8: +port-android.md "Files, storage..." (bundle superseded) and "Permissions and lifecycle". From 6745b2f46d47ec245520e8db84ede60b8b9e644e Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 08:15:27 +0300 Subject: [PATCH 144/275] fix: no variables named near or far, which windows headers define away Windows headers define near and far as empty macros, so bool near = false; reached the compiler as bool = false; and TakeDiff.cpp did not build under MinGW. The variable in notesAcross() is now edgeNear, and the one in TestTakeDiff is now distant. Co-Authored-By: Claude Opus 5.5 --- main/TakeDiff.cpp | 6 +++--- main/test/TestTakeDiff.h | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/main/TakeDiff.cpp b/main/TakeDiff.cpp index 352534fd..054185d5 100644 --- a/main/TakeDiff.cpp +++ b/main/TakeDiff.cpp @@ -250,16 +250,16 @@ TakeDiff::notesAcross(const EventVector ¬es, sv_frame_t join, if (start <= join && join < end) result.spanning.push_back(note); - bool near = false; + bool edgeNear = false; for (sv_frame_t edge : { start, end }) { sv_frame_t offset = edge - join; - if (std::llabs(offset) <= clearance) near = true; + if (std::llabs(offset) <= clearance) edgeNear = true; if (!haveEdge || std::llabs(offset) < std::llabs(result.nearestEdge)) { result.nearestEdge = offset; haveEdge = true; } } - if (near) result.edgesNear.push_back(note); + if (edgeNear) result.edgesNear.push_back(note); } result.pass = result.spanning.size() == 1 && result.edgesNear.empty(); diff --git a/main/test/TestTakeDiff.h b/main/test/TestTakeDiff.h index 65b2a53b..fbdbb5d3 100644 --- a/main/test/TestTakeDiff.h +++ b/main/test/TestTakeDiff.h @@ -460,10 +460,10 @@ private slots: // A doubled frame and a hole a second away: pYIN's own doubled // frame near the end of a run is like this - sv::EventVector far = pitch; - far.push_back(pitchAt(atJoin + 172 * hop)); - for (int k = 1; k <= 10; ++k) removeFrame(far, atJoin - 172 * hop + k * hop); - QVERIFY(TakeDiff::pitchAcross(far, join, kRate).pass); + sv::EventVector distant = pitch; + distant.push_back(pitchAt(atJoin + 172 * hop)); + for (int k = 1; k <= 10; ++k) removeFrame(distant, atJoin - 172 * hop + k * hop); + QVERIFY(TakeDiff::pitchAcross(distant, join, kRate).pass); // The merge's seam, a quarter of a second before the join sv::EventVector seam = pitch; From 1744406b99fcb18a8c5463399dcbe0bbbbf8a9b0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:35:49 +0000 Subject: [PATCH 145/275] docs: calibrate audio re-planned after the merge of default default's TestUiChecks covers the checklist's screen and smoke items, and its test-tony-device the device items; the dev run takes the device items over and test-tony-device is to be retired. The remaining phases are C1b (observer; items 3, 4, 5), C1c (items 7, 12, 13, 14), C2 (items 9, 10), C3 (retire test-tony-device), a release build, then docs. The finished phases' work orders and log move to calibrate-audio-log.md, and the work orders' state of the code is brought up to date, with the user's first Windows run. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-log.md | 572 ++++++++++++++++++++ docs/calibrate-audio-work-orders.md | 803 +++++++--------------------- docs/calibrate-audio.md | 48 +- 3 files changed, 795 insertions(+), 628 deletions(-) create mode 100644 docs/calibrate-audio-log.md diff --git a/docs/calibrate-audio-log.md b/docs/calibrate-audio-log.md new file mode 100644 index 00000000..cb70be20 --- /dev/null +++ b/docs/calibrate-audio-log.md @@ -0,0 +1,572 @@ +# Calibrate Audio: work orders and log of the finished phases + +The work orders of the phases already built, and their hand-over log, moved out of +[calibrate-audio-work-orders.md](calibrate-audio-work-orders.md) once they were done. +**Phase agents do not read this file**; the work orders' section 3 says what they need of +it. The documentation phase (D) reads the log, and deletes this file with the work orders. + +## Work orders of the finished phases + +### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) + +- New `main/LatencyCheck.{h,cpp}` in `tony_core`, pure. +- **Generator.** + - A layout gives the length and the events. Make three: *calibration* (about 26 s), + *dev* (adds 3 s held tones) and *long* (4 minutes). + - Each event: a sweep of 1 → 8 kHz, 200 ms, 10 ms raised-cosine edges, −12 dBFS + peak; then a tone at a pitch with a whole number of samples per period at 44.1 kHz + (196, 220.5, 245, 294 Hz; see `docs/testing.md` on pYIN subharmonics). + - Gaps between events are irregular, 1.6–2.6 s, all different by at least 0.1 s. + - Deterministic. It returns the samples and each sweep's exact frame. + - Exponential or linear sweep: choose one and say why. The spec leans neither way. +- **Finder.** Given take audio (samples and rate) and an expected event time, search + ±0.8 s: + - FFT matched filter against the sweep (bqfft; see how `RealtimePitchTracker` uses + it), weighted to the sweep's band; + - the envelope of the result; + - the **earliest** peak within 6 dB of the largest; + - two confidences: the peak over the window's median in dB, and the peak over the + second-best peak outside ±10 ms in dB; + - the error in frames and in ms. + + The thresholds are named constants (15 dB and 6 dB to start). +- **Not in this phase:** aggregation over events and punch-ins, verdicts, second arrivals. +- **Tests,** in a new core test class `TestLatencyCheck`: + - generator: deterministic, events where it says, gaps all different, peak level; + - finder on synthetic takes, made by shifting the reference: + - shifts across ±0.75 s, plus one fractional shift (resample or interpolate); + - white noise at 0 and −10 dB SNR; + - high-pass at 1 kHz and low-pass at 4 kHz; + - polarity inverted; + - a reflection 6 dB **stronger** 7 ms after the direct sound: the direct sound + must be found; + - silence: no confident peak. + - **Show that the reflection test fails** when the finder takes the largest peak. + +### A2 — Verdicts and calibration arithmetic (spec §5 "tony_core") + +- In `LatencyCheck`: given the events inside a take's coverage ranges (one range per + punch-in), and the finder's results, compute: + - median and spread per punch-in and across punch-ins; + - the slope of offset over position; + - the input peak, for clipping; + - whether confidences fall steadily over time, for fading; + - a **second arrival**: a second confident peak at a consistent extra delay across + events, which means monitoring echo. +- `Verdict`: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent, with + the thresholds of spec §5 as named constants. +- The calibration arithmetic as a pure function: new round trip = used round trip + + median offset, in seconds. A take that lands late was spliced from too early a frame. +- **Refined by the lead after A1:** + - **Entry point for B1.** One function takes a layout, the take's samples and rate, + and the punch-ins. Each punch-in is its timeline range in seconds, in the order + recorded. The function returns a summary with the verdict, all flags that applied, + per-punch-in figures, and per-event results. + - **Which events are judged.** An event is *judged* in a punch-in when its sweep and + the finder's window sit inside the range with a margin, a named constant. The splice + cuts content at the range ends, and a sweep cut in half is not a failure of the + path. Count judged and found separately; NoSignal is about found out of judged. + - **Second arrivals.** `findSweep()` computes the second peak but does not return + its position; add it, and its level against the chosen one, to `Arrival`. Monitoring + echo is a second arrival at a consistent extra delay (a few ms of spread) across + most found events, not more than some named dB below the direct sound. + - **Verdict order.** When several apply, pick one and document it, with all flags kept. + Suggested: NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok. + - **PositionDependent against Scattered.** Fit offset over punch-in position. + PositionDependent is a slope above 0.5 % whose fit leaves little residual; large + residuals are Scattered. +- **Tests:** + - one event missing, the rest found; + - fading; + - clipped; + - misplacement that grows with position (the 48000/44100 case) → PositionDependent; + - two punch-ins 20 ms apart → Scattered or Unsteady by threshold; + - monitoring echo detected; + - an event cut by a range end is not judged; + - the arithmetic with both signs. **Show that the sign test fails** when flipped. + - A1 found that a take of 48 kHz frames read as 44.1 kHz finds nothing, because the + sweeps are stretched. Build the rate case as punch-ins displaced by + P·(1 − 44100/48000), not as a stretch. + +### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) + +Read also: `docs/recording.md` whole (154 lines), `docs/architecture.md` sections on +layers, models and commands (search the headings), `docs/testing.md` "What there is to +reuse". In `MainWindow.cpp`, read by range: `record()`, the deferred lambda in +`recordingStarted()`, `pollTakeProgress()`, `finishSingingTake()` and +`wantedPreRollFrames()`. + +- **Pure helper first**, in `LatencyCheck`, with a core test: + - `punchInsFor(layout, count, eventsEach)` returns `count` consecutive punch-in ranges + in seconds. Each holds `eventsEach` events that `judgeTake()` will judge, using its + margins. + - The calibration uses 4 × 3 on the calibration layout. Tests use a short layout of + 2 × 2 (≈ 8 s) to keep real time down. +- **New `main/AudioCheckRunner.{h,cpp}`** in `tony_app_files`. A QObject owned and wired + by `MainWindow`. It is driven by signals and a polling timer, like + `pollTakeProgress()`, **not by nested event loops**: it runs in every build, and the + user can close the window at any moment. +- **Steps:** + 1. Write the reference WAV to the app data directory, overwriting the old one. Mono, + 44.1 kHz. + 2. `checkSaveModified()`, then `openPath(path, ReplaceSession)`. Wait for the + reference's analysis, as `openReference()` in the tests does. + 3. For each punch-in: select its range and call `record()`; the take stops itself. + Wait for the take's analysis before the next punch-in. It is not strictly needed, + but it keeps pYIN's CPU load out of the next take's timing. + 4. Read the take's audio: the model `analyser2()->getMainModelId()`, mixed to mono, + at its own rate. Call `judgeTake()`. +- **Record into Selection, Play Reference While Recording and a 1 s pre-roll** apply to + the check's own takes through an override in `MainWindow`. `record()`, + `recordingStarted()` and `wantedPreRollFrames()` consult it. **Never through + `setChecked()`**: those actions write QSettings. +- **What each take used.** Keep, per take, the round trip it used: today + `computeRecordingLatency(out, in)`, which B2 will change. Also keep the reported + output and input latency, and the **recording's sample rate**. +- **The result** carries: + - the `TakeSummary`; + - the round trip used, and the reported pair; + - the recording's rate and the reference's; + - a **rate-mismatch flag**, set when the two rates differ, whatever the sweeps say + (A2's finding: a 48 kHz take runs off the finder's reach); + - the calibrated round trip from `calibratedRoundTrip()`, meaningful only on Ok or + Unsteady. + + Emitted as a signal when done. Failures (no device, recording refused, the session + closed mid-run) end the run with a reason. +- **Cancel** stops a take in progress through the normal Stop path and clears the + override. `closeSession()` and `~MainWindow()` cancel a running check. +- **Not in this phase:** storing or using the result (B2), any dialog or menu entry (B3). +- **App tests.** Choose between a new class and adding to `TestRecordWorkflow`; say + which. `TestMainWindow` and the fixtures live in `TestRecordWorkflow.h`. Use + `FakeAudioIO` `loopback = true` and the short layout. + - Wrong reported latencies (e.g. 2·4096 and 4096), `inputDelay` = 3·4096 + 123. The + median offset equals the difference, in seconds at 44.1 kHz, to within a few + frames. The verdict is Ok. The calibrated round trip equals `inputDelay`. + - Fake at 48 kHz: the rate mismatch is flagged and both rates are given. Assert only + what the runner reports, and that nothing crashes. What Tony does with 48 kHz takes + is a separate, known bug. + - After a check, the three toggles and their QSettings values are as before. + - Cancel during a take leaves no recording in progress and the override cleared. + - **Show failure** for the first test with the override's Play Reference half + removed: nothing is heard, so NoSignal. + +### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") + +Read also: `docs/recording.md` "Latency"; `main/LatencyUtils.h` whole; in +`MainWindow.cpp`, the deferred lambda in `recordingStarted()` (search +`m_takeLatency.roundTrip`) and `refineRecordingLatency()`; `AudioCheckRunner.h` +(`AudioCheckResult`, `calibrationUsable()`). + +- **New `main/LatencyCalibration.{h,cpp}`** in `tony_core`: + - **Key.** The three Preferences values `createAudioIO()` reads (`audio-target`, + then `audio-playback-device` and `audio-record-device`, each suffixed with the + implementation when one is pinned; see `audioDeviceSettingKey()` in + `MainWindow.cpp`) and the recording's rate. Device names can hold `/` and non-ASCII + characters: encode them, so that QSettings does not make subgroups. + - **Stored:** round trip and spread in seconds, the date, and the reported output + and input latency, each **in seconds**. The reported pair is the staleness + fingerprint: a stored figure is stale when either differs by more than a named + tolerance. + - `store`, `load`, `forget`, and a staleness test. QSettings group + `LatencyCalibration`. +- **Use in `recordingStarted()`.** The round trip is the stored one when there is one + for this key and it is not stale; otherwise the reported sum. Either way it is + **converted to frames at the recording's rate**, from seconds. + - Fix the reported sum's units while you are there. `getTargetPlayLatency()` counts + frames at the session's rate (`ResamplerWrapper` converts it), and + `getSystemRecordLatency()` at the device's. Today the two are added as they come; + B1 measured 242 ms instead of 256 at 48 kHz. + - The recording's model (`m_currentRecordingModelId`) exists by the time the lambda + runs, and gives the rate. + - Keep in `m_takeLatency` which source was used (reported or measured), and log it. + - The start gap and everything downstream stay as they are. +- **MainWindow API for B4.** Store a check's result, forget the stored figure, and + describe the figure in use (source, milliseconds, date). +- **Not in this phase:** any dialog, menu entry or playback change. +- **Tests:** + - **Core.** Store, load and forget round trip, with a device name holding `/` and + `ä`; the staleness tolerance on both sides; different keys stay apart. Use a + QSettings scope the tests own and clear. + - **App.** + - `latency_end_to_end`'s recipe with wrong reported latencies and a stored figure + equal to `inputDelay`: the sung step lands on the reference's. **The same test + without the stored figure must fail**; show it. + - A stale stored figure (fingerprint differs): the reported sum is used. + - 48 kHz fake: the reported sum, in recording frames, equals + `playbackLatency + recordLatency`, both device frames, to within a frame or two. + - Check, store, check again (short plan): the second check's median offset is + within a few frames of 0. This is the strongest test in the feature, and it + takes about 25 s. + +### B3 — The check's playback, and progress (spec §2, §8 "Loudness") + +Read also: `docs/architecture.md` on play parameters and on `Analyser::setAudible()` +(search "audible"); in `Analyser.cpp`, where the reference's pan and the +sonification's audibility are set (search `setPlayPan`, `PlayParameters`). + +- For the check's session only, the reference plays **centred** and at **−12 dBFS + peak** after normalisation: the model is normalised to full scale when it is read, so + the gain has to come off at playback. The pitch-track sonification is **silent**. + - Do it through the play parameters of the check session's models and layers, never + `Analyser::setAudible()`, which writes the shared settings (see `AGENTS.md`). + - Make sure nothing puts them back later in the run, for example the analysis + finishing, or the next take. + - Nothing of the user's own sessions changes. +- **A progress signal** on the runner: step, punch-in *k* of *n*, and the seconds left + where known, for B4's dialog. +- **Two runner fixes from B2's report:** + - **Reference file name.** The reference WAV gets a new name each run (a counter or a + timestamp), and old ones are removed when no session holds them. **Check Again** + otherwise rewrites the file that the check session it replaces still has open; on + Windows that write can fail. Linux cannot show it. + - **Reported output latency.** `AudioCheckRunner` divides the reported output latency + by the reference's rate. Have `TakeLatency` carry both reported figures **in + seconds**, as `MainWindow::roundTripAt()` works them out, and have the result use + those. Then the take path and the check can never disagree. +- **Tests:** + - During a check the reference's play parameters are centred at the planned gain, + and the sonification is not audible. + - Afterwards, a newly opened ordinary file plays as before: reference pan and + sonification as the settings say. + - The sweeps reach `FakeAudioIO`'s captured output at about −12 dBFS on the mixed + channel. + - Progress reports each punch-in in order. + +### B4 — Calibrate Audio dialog and menu (spec §2) + +Read also: how an existing Tony dialog is built and tested (search +`confirmRecordingOverTake` and `askForTakeName` in `MainWindow.cpp` and +`TestRecordWorkflow.h`). + +- **`main/CalibrateAudioDialog.{h,cpp}`**, non-modal and thin. Its pages: + 1. **Instructions:** the output and input device names, the latency in use with its + source, "hold one earcup against the mic, off your ears", moderate volume. + 2. **Progress,** from B3's signal, with Cancel. + 3. **Result:** + - the verdict in plain words, with the fix for each failure (spec §2 and §8); + - the measured round trip against the driver's figure; + - the spread; + - both rates, with a plain sentence when they differ; + - the input peak; + - the echo, if one was heard. + + **Use this latency** (only when `calibrationUsable()`), **Check Again** and + **Close**. +- **Menu, Playback:** + - **Calibrate Audio…**, disabled while recording; + - a disabled line saying the latency in use ("Latency: measured 187 ms, 25 Sep" or + "Latency: driver's figure, 400 ms"); + - **Forget Measured Latency**. +- The calibration plan is 4 punch-ins × 3 events on the calibration layout; it fits + since the lead's spacing change. +- **The device key is taken when the check starts.** Carry it in `AudioCheckResult`, + and have `storeMeasuredLatency()` use it, not the Preferences at the moment the + button is pressed. The dialog is non-modal, so the user could change device in + between (B2's report). Also disable both Audio Device menus while a check runs. +- Anything that asks the user goes through a virtual seam, as + `confirmRecordingOverTake()` does. The app tests drive the dialog's slots directly; + the dialog watchdog fails a test on any unexpected modal dialog. +- **Tests:** + - the menu starts a check; + - Use this latency stores (B2's API), and the menu line changes; + - Forget clears it; + - the dialog's result words for NoSignal and for a rate mismatch; + - Calibrate Audio is disabled during an ordinary take. +- **After B4 the user runs it on Windows** (the checkpoint in spec §7). + +### C0 — TakeDiff (spec §5 "tony_core") + +Read also: `docs/takes.md` on the splice's edge fades and on ranged analysis's merge +window (search "fade", "±", "W"); `main/TakeAudio.h` and `main/TakeEvents.h` for the +existing vocabulary. `TakeEvents` may already hold part of what is needed: reuse it and +do not duplicate it. + +- **New `main/TakeDiff.{h,cpp}`** in `tony_core`, pure. The comparisons the dev checks + (C1–C4) use on a real take. Each returns a plain result: pass or fail, and the numbers + behind it, so that a check can report them. The inputs are sample buffers, event + vectors and frame ranges; no models. +- **Audio unchanged outside a range.** Two sample buffers (before and after a + punch-in, as read from the take's file) are **bit-identical** outside + `[start, end)`, allowing for where the splice's fades fall. Find that in + `TakeAudio.cpp`; do not guess. Report the first differing frame. +- **Events unchanged outside a range ± margin.** Two event vectors (pitch, and notes + with durations) are identical outside `[start − margin, end + margin]`. A note that + crosses the boundary counts as inside. Report what was added, removed and changed. +- **Pitch continuous across a join.** Given pitch events and a join frame: + - no gap longer than N hops within ±W of the join; + - no two events at the same frame; + - frames strictly increasing. +- **One note across a join.** Exactly one note spans the join frame, and no note + begins or ends within ±X of it. X is a named constant. +- **No step at a join.** The largest first difference of the samples within ±2 ms of + the join, against the typical first difference over the 50 ms around it, in dB. It + should stay near 0 dB when the splice is clean, and a hard cut shows as a large + excess. The threshold is a named constant; justify it. +- **Tests,** in a new core class `TestTakeDiff`, on synthetic data: + - each comparison passing and failing on purpose: one sample changed just outside + the range; one pitch event dropped at the join; a doubled frame; a note split in + two at the join; a hard cut in a sine. + - **Show failure** for two of them by breaking the code. + +### C1a — Dev-check framework, items 1 and 2 (spec §2 point 5, §3, §4, §5 "development builds only") + +Read also: `main/AudioCheckRunner.{h,cpp}` whole (about 1000 lines together; you extend +it), `main/CalibrateAudioDialog.h`, `main/test/TestAudioCheck.h` for the fixture +(`loopback()`, `shortPlan()`, `makeWindow()`), `docs/takes.md` on where take files go +before and after a save, and spec §4's rows for items 1 and 2. + +- **Build flag** (spec §3). In `meson.build`, a build type not starting with `release` + adds `-DTONY_DEV_CHECKS` to `general_defines`, and only then are `main/dev/*.cpp` + compiled into `tony_app` and their headers moc'd. `build_linux` is `debugoptimized`, so + it has them. Everything under `main/dev/`, and every use of it elsewhere (a `friend` + line, a member, a dialog widget, the test class's registration), is inside + `#ifdef TONY_DEV_CHECKS`. A release build must compile with no `main/dev/` file; the + lead builds one later, so keep the `#ifdef`s tidy. +- **Runner extensions** (every build; small; each tested in `TestAudioCheck`): + - **`Plan` ranges.** Explicit punch-ins in seconds; when given, they replace + `punchInsFor()`. `start()` refuses ranges that overlap, are out of order or lie + outside the layout. + - **`Plan` keeps the session.** Record into the session open now, which the caller + says is a check reference of the plan's layout: no reference written or opened, no + save question; straight on to the reference's analysis wait and the punch-ins. The + take keeps what earlier runs recorded; only this plan's punch-ins are judged. + - **`Plan` round trip, for the run only** (spec §10 "a dev run uses the new figure + for itself only"). Seconds; unset means the window's own. The window uses it for the + check's takes only, where `recordingStarted()` takes `roundTripAt()` now. It never + touches the stored figure or the Playback menu's line. Log it as the check's own. + - **`TakeLatency` gets the start gap** each take used, and whether it was measured or + only estimated (item 2 reports it per punch-in). + - **No save question for the check's own session.** When the session open has never + been saved and its main file is in `referenceDirectory()`, replacing it asks + nothing: it is the check before. This also ends B4's Check Again prompt. + - **Record during a check** (from B4): greyed while a check runs, and `record()` + ignores the user's press then. Pressing it today ends the check's take early. +- **`main/dev/DevChecks.{h,cpp}`**, a `QObject`. + - **No nested event loop.** Spec §5 said `waitUntil()` with a `QEventLoop`; the lead + changed it: a list of stages driven by the runner's `finished()` and a polling timer, + as the runner is driven, for the runner's reason (the window can be closed at any + moment). Each stage starts something and says when it is done; a stage that times + out fails the run. + - `start(Options)`, `cancel()`, `isRunning()`, `sessionClosing()` (as the runner's: + ends the run unless the run itself is replacing the session). Signals `progress` + (stage name, n of m) and `finished(DevReport)`, once however the run ends. + - `Options`: the round trip for the run (seconds), the report directory ("" for + `TONY_TEST_LOG_DIR` if set, else `AppDataLocation`), the scratch directory ("" for + `AppDataLocation`). + - `CheckResult { item, name, verdict (Pass, Fail, Measured, Skipped), numbers (label and + value pairs, as text), message }`; `DevReport { checks, failure, reportPath, + sessionPath }`. A run that ends early marks the checks it did not reach Skipped, + with the reason. + - Owned by `MainWindow` in dev builds, like the runner; a `friend` of it under the + `#ifdef`. `~MainWindow` deletes it after the dialog and before the runner; + `closeSession()` calls its `sessionClosing()`. +- **The stages of C1a** (C1b inserts more before the last): + 1. **Fresh punch-ins.** A runner run on `devLayout()`, opening a new reference (no + save question: see above), with two explicit punch-ins in separate regions of the + calibration part, each holding two sweeps, and the run's round trip. Choose ranges + that leave the held tones (after 25 s) and the start (before 3 s) free for later + stages, and say which you chose. + 2. **Save and reopen.** Save the session into a scratch folder (below) with + `MainWindow`'s own save path, no dialog; reopen it; wait for the analyses; read the + take's file again. +- **Items:** + - **1** Pass when every judged sweep of every punch-in lands within ±2 ms (a named + constant), and after the reopen the take's audio judged again gives the same offsets + and its pitch and notes are the same events as before the save. Numbers: offsets + per punch-in, largest offset, the round trip used. + - **2** Pass when every punch-in was added to the take, each placed within ±2 ms, each + with a measured start gap of its own. Numbers: per punch-in its median offset and + start gap. +- **Scratch folders.** Not deleted at the end: the session open afterwards lives in it + (spec §2: the test session stays open). As `nextReferencePath()` does for references: + numbered folders, and at the start of a run every one that the open session does not + use is removed. The report names the folder. +- **Report.** + - Text file `DevChecks.txt` in the report directory: the run's date, devices and round + trip, then one block per check grouped by checklist item (verdict, message, + numbers), ending `Totals: N passed, N failed, N measured, N skipped`. + - Tests pass a report directory of their own: a failing run's report must not land + among the suites' own files, where the lead greps for `^FAIL`. +- **Dialog** (dev builds only). The instructions page gets a checkbox, "Run the dev + checks after calibrating", on by default and not remembered. With it on and the + calibration usable, the progress page goes straight on into the dev checks with + `calibratedRoundTrip`, Cancel cancels whichever is running, and the result page + shows the calibration as today plus the dev report: one line per check, then the + report file's path. With the calibration unusable, the dev checks do not run and the + page says so. +- **Tests,** new class `TestDevChecks` in the app suite, compiled and registered only + in dev builds; the fixture copied from `TestAudioCheck`, not shared by editing it: + - passing on a loopback fake with its true round trip: items 1 and 2 pass, the report + file ends with `Totals:`, and the session open afterwards is the one in the scratch + folder; + - failing with the round trip 20 ms off: items 1 and 2 fail, and the report shows the + offsets; + - cancelled mid-run, and the session closed mid-run: `finished` once, no take left + recording, the user's three toggles and stored latency untouched; + - the dialog with the checkbox on runs the dev checks after the calibration (the short + plan for the calibration). + - **Show failure** for item 1's tolerance and for the run round trip being ignored. + - Report the real time `TestDevChecks` adds. Spec §6 says more than about a minute + over all the dev phases moves them to a third executable, which the lead will ask + the user about; do not create one. + +## Log + +### Phase A1 — 2026-09-25 +Built: `main/LatencyCheck.{h,cpp}` in `tony_core`: `calibrationLayout()`, `devLayout()`, `longLayout()` (rate defaults to 44100), `sweep(rate)`, `generate(layout)`, `findSweep(samples, count, rate, expectedSeconds)` → `Arrival {found, errorFrames, errorSeconds, peakOverMedianDb, peakOverSecondDb, levelDb, inputPeak}`. Thresholds are `k…` constants in the header. `main/test/TestLatencyCheck.h`: 16 tests, 1.4 s. +Choices / deviations: +- Linear sweep: flat spectrum, narrowest peak; an exponential sweep's harmonics match it 67/106 ms *early*, where the earliest-peak rule looks. +- Envelope = magnitude of the analytic signal (a second inverse FFT gives the Hilbert part). +- Earlier peak counts if it is a local maximum, within 6 dB, and ≥ 1 ms before the largest. No dip rule: one arrival with a hole in its band beats, with deep dips, so only time tells arrivals apart (`finder_takes_one_arrival_as_one`). +- No band mask beyond the matched filter itself: a 0 dBFS 100 Hz hum already comes through 100 dB down; a mask changed nothing measurable. +- Confidences are the chosen (earliest) peak's; "second" is the envelope's maximum more than 10 ms from it. +- A gap is sweep to sweep. Calibration sweeps at 1.0, 3.1, 4.7, 7.2, 9.1, 11.4, 13.1, 15.7, 17.7, 19.5, 21.9, 24.1 s; each tone starts 0.3 s after its sweep, 0.8 s long. Dev adds 3 s held tones after sweeps at 26.9, 30.9, 35.2 s (40 s). Long: 113 events, 240 s. +- Reflection test at 5.5 dB (direct found) and 7 dB (reflection taken): exactly 6 dB passes here only by rounding (6.1 dB flips). +The next phase must know: +- Pass the take's samples at their own rate, and the expected time in seconds (layout frame / layout rate). +- A take of 48 kHz samples read as 44.1 kHz (sweeps stretched 8.8%) finds nothing: level −22 dB, 0.1 dB over the second. `findSweep` at 48000 finds them. A2's "resampled by 48000/44100" case must be built as misplaced frames, or try both rates. +- Measured: noise alone 6–12 dB over the median (threshold 15); SNR 0 / −10 dB: 38 / 28 dB over the median, 26 / 16 dB over the second. In digital silence the median is ~0, so over-the-median reads up to the 200 dB clamp. +- About 20 ms per call. The second peak's position is computed but not returned; second arrivals (A2) need it. +Left open: every threshold untuned; nothing reads `inputPeak` yet. + +### Phase A2 — 2026-09-25 +Built: in `main/LatencyCheck.{h,cpp}`: `Arrival::secondDelaySeconds`, `secondLevelDb`; `judgeTake(layout, take, count, rate, punchIns)` → `TakeSummary {verdict, flags, punchIns[], events[], judged, found, medianOffset, spread, slope, slopeResidual, inputPeak, fadingDb, echo}`; `PunchIn {start, end}` in timeline seconds; `Verdict`, declared in precedence order (NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok); `verdictName()`; `calibratedRoundTrip(used, offset)` = used + offset. 9 tests in `TestLatencyCheck` itself (reusing its helpers); the class now takes 2.9 s. +Choices / deviations: +- Judged: the finder's window, plus a sweep's length past it, inside the range with 50 ms to spare (the splice crossfades 5 ms). An event under a later, overlapping punch-in is judged in that one only. Ranges stop at the take's end. +- Across = median of the punch-ins' medians (each stream start counts once); spread = their max − min. Unsteady/Scattered take the larger of that and any spread within one punch-in. PositionDependent: 3 punch-ins at least (a line through two always fits), |slope| > 0.5 %, and what the least-squares line leaves ≤ 5 ms. NoSignal also when nothing is found, or nothing judged. +- Fading reads `levelDb`, not a confidence: over the median of near silence a confidence runs up to the 200 dB clamp. Median of the first half of the judged events (as recorded) minus that of the second ≥ 10 dB; needs 6 events. +- Echo is judged over events *heard* (≥ 15 dB over the median), not found: an echo within 6 dB leaves nothing found. Added `kEchoMinDelaySeconds` = 20 ms: a reflection 9 ms after the direct sound and 5.5 dB stronger leaves the tail of its peak at 10.1 ms, 9–23 dB down, after every sweep, and was reported as an echo. Also ≤ 30 dB down, within 3 ms of the median delay, in more than half of the heard events and 3 at least. +- Clipped: the largest sample inside the ranges ≥ −0.2 dBFS. +The next phase must know: +- **At 48 kHz, §2's punch-ins at 14 and 20 s land 1.14 and 1.63 s early, beyond the finder's 0.8 s reach.** Measured: the finder takes the neighbouring sweeps, fully confident (+662, +575 ms), and the verdict is Scattered. PositionDependent needs punch-ins that start before about 9.8 s, so §6's "a 48 kHz fake reports PositionDependent" fails with §2's punch-ins. The rates themselves can name the rate. +- §2's punch-ins as [2,7], [8,13], [14,19], [20,25] s judge 7 events (2, 2, 2, 1). A punch-in shorter than 1.95 s judges none. +- An echo under 20 ms (an interface's direct monitor) is not seen. +Left open: every threshold untuned. The verdict thresholds came from the lead's brief; spec §5 has none of them. + +### Phase B1 — 2026-09-26 +Built: `LatencyCheck::punchInsFor()` (+ `kPunchInSlackSeconds`, 10 ms) and core test `punch_ins_hold_the_events_asked_for`. `TakeLatency` in `LatencyUtils.h`. `main/AudioCheckRunner.{h,cpp}` (`tony_app`): `Plan`, `start()`, `cancel()`, `sessionClosing()`, `finished(AudioCheckResult)`. `MainWindow`: `friend class AudioCheckRunner`, the override `m_audioCheckTakes` (read by `record()`, the `recordingStarted()` lambda, `wantedPreRollFrames()`), `m_takeLatency`, the runner made in the constructor, deleted first in `~MainWindow`, told by `closeSession()`. New app class `main/test/TestAudioCheck.h`: 6 tests, 32 s. +Choices / deviations: +- **4 × 3 does not fit the calibration layout** (spec §2 now says why). `punchInsFor()` returns nothing then; 4 × 2 and 3 × 3 fit. B3 needs the lead's choice. +- The take is read from its **file**, not its model: the model is peak-normalised as read (measured: every take Clipped) and resampled to 44.1 kHz. +- `friend` over accessors: the runner needs about nine internals. A new test class, not `TestRecordWorkflow` (5475 lines); its watchdog and init/cleanup are copied. +- Selection: `clearSelections()` + `addSelectionQuietly()`, so the reference is not re-analysed during a take; each punch-in adds one or two "Select" undo steps, as a user's selection does. +- Waits: poll every 50 ms for "nothing being analysed", not "analysed": with auto-analysis off the reference never gets layers (test `check_runs_without_automatic_analysis`). Limits 60 s reference, 30 s a take's analysis, take length + 10 s to stop (then the Stop path); each ends the run with a reason. +- `~MainWindow` deletes the runner: the run ends silently and a take in progress is left to the destructor (the Stop path would splice and start pYIN mid-teardown). `closeSession()` → Stop path + `finished()`. +- Reference: `AppDataLocation/calibrate-audio-reference.wav` unless the plan names a path (tests: their temp dir). Save question first, then write, then open. +- `calibrationUsable()`: Ok or Unsteady, and no rate mismatch. +- No loopback gain was needed (see below). +The next phase must know: +- `Analyser` pans the reference hard left, sonification hard right: only the **left earcup** carries sweeps. Normalised, the reference plays at 0 dBFS, not −12. The fake averages channels: sweeps loop back at half level (peak 0.905 with the synth). +- `getTargetPlayLatency()` counts session frames, `getSystemRecordLatency()` device frames; the result converts each at its own rate. At 48 kHz the round trip used was 242 ms, not 256. +- 48 kHz fake: Scattered, 3 of 3 found, offsets −200 ms median, as A2 foresaw. +- No progress signal yet; B3's dialog may want one (`m_punchIn`). +Left open: no test deletes the window mid-check. Seen while proving the session-close hook: closing a session during an **ordinary** take, then pressing Stop, hangs (pre-existing). + +### Lead — 2026-09-26, after B1 +- Reordered the calibration spacings to `{21,16,25,19,17,23,26,20,24,18,22}` (B1's suggestion) so that 4 × 3 punch-ins fit; `punch_ins_hold_the_events_asked_for` now asks for 4 × 3 and failed on the old order. `judge_only_events_inside_a_punch_in` names its events from the layout instead of 9.1 and 11.4 s. +- Split B3 into B3 (the check's playback and progress) and B4 (dialog and menu), after B1 needed 370k tokens. +- Calibration sweeps now at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, 17.7, 20.1, 21.9, 24.1 s. + +### Phase B2 — 2026-09-26 +Built: `main/LatencyCalibration.{h,cpp}` (`tony_core`, namespace): `Key`, `currentKey(settings, rate)`, `Figure`, `store`/`load`/`forget` (all take a `QSettings &`), `isStale`, `kStaleToleranceSeconds` = 1 ms, `Source`, `InUse {source, roundTrip, date, stale}`, `roundTripInUse()`, `reportedSeconds()`, `toFrames()`. `TakeLatency::measured`. `MainWindow`: `roundTripAt(rate)`, used by the `recordingStarted()` lambda; B4's API `storeMeasuredLatency(result)`, `forgetMeasuredLatency()`, `latencyInUse()`. Core class `TestLatencyCalibration` (6 tests); app tests `latency_measured_round_trip_used`, `latency_stale_round_trip_ignored`, `latency_reported_at_the_device_rate`, `latency_reported_with_device_opened_first` (TestRecordWorkflow), `check_stored_round_trip_is_used` (TestAudioCheck, 25 s). +Choices / deviations: +- Settings: `LatencyCalibration/||//`. In names only `%`, `/`, `\` and `|` are percent-encoded. Registry key names stop at 255 characters, and full encoding of long non-ASCII names could pass that. Values are stored as text (`'g'`, 17 digits) and the date as ISO UTC. +- The key follows `createAudioIO()`, not `audioDeviceSettingKey()`: they differ for `audio-target` = "auto", which Tony never writes. +- **The output latency's unit** (spec §11, corrected): frames at `m_playSource->getDeviceSampleRate()`, or at the recording's rate when that is 0. This is not always the session's rate. If a device is chosen before any file is opened (or a standalone take is the first action), `ResamplerWrapper` has no source rate yet. It passes the device's figure through unconverted and tells the play source 0. Using the session's rate there gave 13012 frames instead of 12288 at 48 kHz (`latency_reported_with_device_opened_first`). +- `storeMeasuredLatency()` stores only when `calibrationUsable()`. The key uses the current Preferences, the rate is the result's, and the fingerprint is the result's reported pair. +- `latencyInUse()`/`forget` use the rate of the last take placed with a round trip. Before any take they use the session's rate, the only rate at which a usable check stores. That rate is reset when a device is chosen from the menu. The device's rate cannot be known before a take: `AudioCallbackRecordTarget` has no getter for it. +- A stale figure is not deleted; it becomes valid again if the driver goes back to reporting the old pair. +The next phase must know: +- With no figure stored at 44.1 kHz, the round trip is exactly the old sum; this is tested in core across a grid of values. At 48 kHz the check now uses 256 ms, not 242. +- The runner's `reportedOutputLatency` still divides by the reference's rate. It matches the take path at 44.1 kHz, the only rate that is stored. +- `storeMeasuredLatency()` reads the device from the Preferences when "Use this latency" is pressed. If B4's non-modal dialog lets the device change in between, the figure is stored under the new device. +Left open: `computeRecordingLatency()` is unused outside `TestLatencyShift`. The svapp fork could add `AudioCallbackRecordTarget::getRecordSampleRate()` so that `latencyInUse()` knows the rate before the first take. + +### Phase B3 — 2026-09-26 +Built: `AudioCheckRunner::setPlayback()`: on the play parameters of the check session's own models, the reference audible, centred, gain 10^(kPeakDbfs/20) (1 if the normalise preference is off); its pitch and notes muted. Applied once the reference is open and again before every punch-in. Public `Step`, `Progress {step, punchIn, punchIns, secondsLeft}`, signal `progress()`. `referenceDirectory()`, `nextReferencePath(dir, inUse)`. `LatencyCalibration::InUse::reportedOutput/Input` (seconds, whichever source won); `TakeLatency::reportedOutput/Input` are now those seconds, and `end()` copies them. App tests `check_plays_the_reference_centred_and_quiet` (13 s), `check_leaves_the_next_session_alone` (4 s), `check_reference_gets_a_file_of_its_own`; core `round_trip_in_use` extended. Four existing tests now compare the reported pair in seconds. +Choices / deviations: +- The toolbar's reference level control (`m_audioLPW`) answers a gain between its notches (−12 dB lies between −11.25 and −20) by emitting the nearest; `audioGainChanged()` then sets it through `Analyser::setGain()`/`setAudible()`, writing `Analyser/audible-0`. `setPlayback()` moves the control first under a `QSignalBlocker`. Without it both new session tests fail on the settings. +- Reference files `calibrate-audio-reference-N.wav`: every such file in the directory but the open session's main model file is removed, then the lowest free N is taken, so names alternate 1, 2. A timestamp per run would add a dead Recent Files entry per run (`RecentFiles` has no remove, and keeps 20). +- The check session keeps its playback after the run. The reference is made audible even where the user's settings mute it. +- `progress()` comes from `poll()` only, never from inside `start()`/`cancel()`: on each step change, and each whole second less while recording. `secondsLeft` counts recording to come (min(1 s, start) + range per take), not analyses. No metatype: direct connections only. +- "Afterwards" is tested after a cancelled run, against the same file opened before the check: the settings alone do not say how a session plays (below). +The next phase must know: +- Pre-existing, not fixed: (a) in the first file of a window the same feedback moves the pitch and notes gain from 0.5 to 0.562 and forces both audible, writing the settings; (b) `audible-0` is overridden at load by `audible-3`: the spectrogram is a layer on the reference's model, so it plays whenever `audible-3` is true. +- B4: add a few seconds per analysis to `secondsLeft` for a rough total. +Left open: the default reference path is exercised only through `nextReferencePath()`; the tests name their own file. + +### Lead — 2026-09-26, after B3 +- De-raced `TestSingingAnalysis::waitForRange()` (`2a306fb`): `initialAnalysisCompleted()` also fires from `layerCompletionChanged()` before a ranged merge; it failed once in a full run. +- For phase D's `open-points.md`, older bugs B3 found: (a) in a window's first file the toolbar level control's notches move the pitch/notes gain 0.5 → 0.562, force both audible and write that to the settings; (b) `audible-0` (Play Audio) is overridden on load by `audible-3` (the spectrogram layer on the same model, loaded last). Also B1's: closing a session during an ordinary take, then Stop, hangs. + +### Phase B4 — 2026-09-26 +Built: `main/CalibrateAudioDialog.{h,cpp}` (`tony_app`), a `QDialog`, not modal, with three pages (instructions, progress, result): `present()`, `startCheck()` (Start and Check Again), `cancelCheck()`, `useLatency()`, `showResult()`, `reject()`; for tests `page()`, `pageText()`, `canUseLatency()`, `setPlan()`; static `describeLatency(InUse)`. `AudioCheckRunner::calibrationPlan()` (4 × 3). `AudioCheckResult::key`: the Preferences' devices taken in `start()`, the rate set in `end()`; `storeMeasuredLatency()` stores under it. `MainWindow`: Playback ▸ Calibrate Audio..., a disabled line "Latency: ...", Forget Measured Latency, after the device submenus; `calibrateAudio()`, `updateLatencyMenuLine()`. `updateMenuStates()` shuts Calibrate Audio during any take or check, and both device menus during a check; the runner's `progress` and `finished` call it. `TestAudioCheck`: 5 tests `calibrate_audio_*`, one full run (13 s); `TestMainWindow` accessors. +Choices / deviations: +- The window owns the dialog, makes it on first use, and deletes it in `~MainWindow` before the runner. The dialog calls only `latencyInUse()` and `storeMeasuredLatency()`, and shows only runs it started itself. +- Closing it (title bar, Esc, Close) while its check runs cancels the check. Opened again, it shows the instructions. +- Menu line: "Latency: measured 281 ms, 26 Sep" (the year only when not this one), "Latency: driver's figure, 279 ms", plus " (the measured one is out of date)" when stale, or "not known yet" while the device reports 0. Forget is enabled while a figure is kept, stale or not. The line is refreshed on the menu's `aboutToShow`, in `updateMenuStates()`, and by store and forget. +- Result page: one sentence for the verdict, then its fix. A rate mismatch replaces the verdict's words, and an echo adds a paragraph. Then a table: the round trip measured (not for NoSignal or a rate mismatch) against the driver's (out + in), what the takes were placed with, each punch-in's offset, the spread, found of judged, both rates, the input peak, the echo, and the devices. The text is selectable, to copy. +- Progress: the runner's seconds left plus `kSecondsPerAnalysis` = 3 s for each analysis to come. The bar never goes back. +- Nothing new asks the user, so there is no new seam: Forget asks nothing. +The next phase must know: +- A run replacing a check session asks "Session modified: save?" (its takes mark it modified). Check Again always meets it; answer No. The runner could skip the question for its own reference's session. +- Not on the result page: §2's mic channel and noise floor. The runner measures neither. +Left open: Record stays enabled during a check. Pressing it there goes through the Stop path and ends the check's take early; what the run then makes of it was not tried. + +### Lead — 2026-09-26, after B4 +- The button is complete; the user's Windows run is the checkpoint (spec §7). C0 onwards goes on meanwhile. +- Left for C1 (small, in passing): Record stays enabled during a check, and pressing it ends the check's take early; grey it while a check runs. +- Left for D: spec §2 promises the mic channel and noise floor on the result page, which nothing measures yet (C2's item 5 measures the channel); §8 says "modal progress dialog", true only of the dev run; Check Again always asks to save the check's own session (answer No), a possible later nicety. + +### Phase C0 — 2026-09-26 +Built: `main/TakeDiff.{h,cpp}` (`tony_core`, namespace). Five comparisons, each returning a struct with `pass` and its numbers: `audioOutside()` → `AudioDiff {firstDifference, differences, largestDifference}`; `eventsOutside()` → `EventDiff {added, removed, changed (before, after), firstDifference, window}`; `pitchAcross()` → `PitchJoin {events, largestGap, largestGapFrom, doubled, firstDoubled, outOfOrder, firstOutOfOrder, window}`; `notesAcross()` → `NoteJoin {spanning, edgesNear, nearestEdge}`; `stepAt()` → `SampleStep {stepDb, largest, typical, largestAt, channel}`. Samples are interleaved (`const float *`, frames, channels); times are named constants in seconds, with the rate passed. Core class `TestTakeDiff`: 19 tests, 10 ms. +Choices / deviations: +- Fades: `weightAt()` in `TakeAudio.cpp` mixes only frames of [start, end), both edge frames included, and copies every other frame; take files are float WAV. So `audioOutside()` excuses nothing outside the range: pass the placed range (the coverage added). A test runs the real `splice()` and `erase()` through files: identical outside, and the range less one frame at either end differs exactly at that frame. Frames past a buffer's end are silence; bits are compared, not values. +- Events: "inside" is what `TakeEvents::eraseNotes()` of the window would touch (a crossing note is inside; a pitch event goes by its frame). Window [start − 0.25 s, end + 0.25 s), half-open. Multiset difference; what is left at one frame is "changed". A note that grew from outside into the window reads as removed. +- Pitch window ±0.5 s (`kPitchWindowSeconds`), not ±0.25: the second run's merge seam is 0.25 s before the join, and would sit on the edge of a ±0.25 s window. Gap limit 1 hop (`kMaxGapHops`). Gaps run to the neighbours beyond the window, or to its edge if there are none, so a hole reaching in, a track stopping inside, or an empty window all fail. +- Notes: X = 0.5 s (`kNoteClearanceSeconds`), closed. A note is [f, f + d), so one ending at the join does not hold it. +- Step: the largest |x[i] − x[i−1]| within ±2 ms, against the 95th percentile over the 50 ms around, in the worst channel. Measured: steady tone 0.03 dB, noise 2.7 dB (6.7 at most in 200 simulated trials), hard cut 36 dB, and the real splice 0.03 dB with its fade, 36 dB without. `kMaxStepDb` = 10 dB; about one random cut in ten reads under it (the two sides nearly meeting). +The next phase must know: +- **Two punch-ins that meet at J leave a 10 ms dip, not a crossfade.** Each fades against what the file held there, which is silence: [J − 5 ms, J) fades out and [J, J + 5 ms) fades in (read from `weightAt()`, not measured). `stepAt()` reads it as no step. +- The notes merge adds a new note only if its onset is in W (`Analyser.cpp`, "Notes go by their onset"). A second run's note that begins before J − 0.25 s is not added, and the dip may split the note at J. Either way, item 10's "one note across the join" may fail on today's code. C3 should measure it, not assume it. +Left open: every threshold untuned; nothing calls `TakeDiff` yet. + +### Lead — 2026-09-26, after C0 +- C1 is split into C1a (framework, items 1 and 2) and C1b (observer, items 7, 12, 13, 14); spec §7 says so. +- DevChecks runs as stages driven by the runner and a timer, not a nested event loop; scratch folders stay with the open session and the next run removes the old ones. Spec §5 and §8 changed to match. +- C0's warning about item 10 (a 10 ms dip at two punch-ins' join; notes merged by onset) stands for C3. + +### Phase C1a — 2026-09-26 +Built: `meson.build`: `dev_checks` = build type not starting with `release`; then `-DTONY_DEV_CHECKS` in `general_defines` and moc, `main/dev/DevChecks.cpp`, `TestDevChecks.h`. Runner: `Plan::ranges` (`punchInsOf()`), `keepSession`, `roundTrip` (window's `m_audioCheckRoundTrip`, read only in the `recordingStarted()` lambda), `abandon()`, public `analysing()` and `readTakeFile()`, no save question for a check's own session. `TakeLatency::startGap/startGapMeasured`. `MainWindow`: Record's action calls `recordPressed()`, which ignores presses while `audioCheckRunning()`; Record greyed then; `m_devChecks` (friend). `main/dev/DevChecks.{h,cpp}`: stages, `CheckResult`, `DevReport`, items 1 `latency_on_this_machine` and 2 `several_phrases_in_one_take`, `DevChecks.txt`, `nextScratchFolder()`. Dialog: checkbox, dev run, `setDevChecks()`, `setDevOptions()`. Tests: 6 in `TestAudioCheck` (~27 s), `TestDevChecks` 7 (~66 s). +Choices / deviations: +- Stage 1 ranges [6.3, 10.2] and [16.8, 21.2] s (sweeps 7.2/9.1, 17.7/20.1; 50 ms spare). Free: before 4.3 s (P = 1 s judging 3.1 s), 10.2–16.8 and 21.2–25 s, the held tones. A re-record starting at 17.9–19.2 s has its lead-in over B's first sweep and still judges its second. +- A stage that fails or times out ends the run: `failure` names it ("Stage 2 of 2, "Save and reopen", did not finish within 60 s."), and every check whose data is missing is Skipped with that text. Checks are worked out at the end: item 2 needs stage 1 only, item 1 both. +- Record goes through `recordPressed()`, not a guard in `record()`: the runner and `pollTakeProgress()` call `record()` during the check's take too, and it cannot tell them from a press. +- After the reopen pitch and notes are restored, not analysed: compared by value with values rounded as `Event::toXml()` writes them; offsets to the frame. Save: `saveSessionToPath()` (a dialog only on failure). +- A run's own round trip counts as `TakeLatency::measured`. The dev reference goes to `referenceDirectory()`, so a cancelled dev run leaves a session the next check replaces without asking. +- `TestAudioCheck`'s fixture deletes each window's `DevChecks`: in a dev build its B4 dialog tests would carry on into them (the checkbox is on) and write `DevChecks.txt` into the log directory. +The next phase must know: +- **B3 bug fixed:** `getLocalFilename()` is the decoded cache copy (spec §11), so `nextReferencePath()` never kept the open reference: always `-1`, written over while open. `mainModelFile()` now. +- On the fake with its true round trip every sweep lands at 0 frames; the fake's reported pair is 2.8 ms short, so a run ignoring its round trip fails item 1. +- `latencyInUse()`'s reported pair differs before any file is open: store test fingerprints after opening one. +- The watchdog answers "Session modified" with No; `QStandardPaths::setTestModeEnabled()` keeps references out of the test app's data directory. +Left open: the app suite now takes about 7m40s; the dev checks add about 66 s, over spec §6's minute. + +### Merge of default — 2026-09-26 +Merged `origin/default` at `92b8b5f` (15 commits); svgui at its new pin `a34646a`. +Conflicts: +- `TestRecordWorkflow.h`: `TestMainWindow` lives in default's `TestMainWindow.h` now. This branch's 17 accessors were merged into it three-way against the base's class (no overlap with default's `setUseRealDevice()` and the rest), and `dev/DevChecks.h` went with them, under `#ifdef TONY_DEV_CHECKS`. +- `TestSingingAnalysis.h`: both `waitForRange()` fixes are the same code; default's comment kept. +- `MainWindow.h` (includes), `meson.build`, `tony-core-test.cpp`, `tony-app-test.cpp`: both sides kept. `TestUiChecks` runs right after `TestRecordWorkflow`, as on default, then `TestAudioCheck` and `TestDevChecks`. +Beyond the conflicts: +- `TestAudioCheck.h` and `TestDevChecks.h` include `TestMainWindow.h` and what they use, not `TestRecordWorkflow.h`. +- `tony-app-test.cpp` draws text without sub-pixel anti-aliasing. Ubuntu's fontconfig asks for it (`10-sub-pixel-rgb.conf`) and Qt 6.4 follows it: the orange fringes of the waveform scale's labels at the pane's left edge were taken for the first live dot, and `TestUiChecks::live_dots_under_the_cursor` failed. Default ran on conda-forge's Qt 6.11, whose text was grey. +- `test-tony-device`'s moc needs no `dev_moc_args`: `TestMainWindow` has no `Q_OBJECT`, and `TestRealDevice.h` no `#ifdef`. +Checked: no string connect naming `ModelId` or `sv_frame_t` is left, and all of `e2cf7c0` survived. `liftPlaySelectionForTake()` (default) is called in the `recordingStarted()` lambda after the round trip, so a check's takes lift the constraint too. `closeSession()` tells the runner and the dev checks first, then restores the constraint. +For D: +- `testing.md` still calls `TestMainWindow` shared by three suites, and lists neither `TestAudioCheck` nor `TestDevChecks`. +- `building.md`'s "Qt 6.11, not Ubuntu's 6.4" is no longer needed for the connects. +- `README.md` says the manual checklist is "none of it tried yet"; `open-points.md` says the device check has been run in the cloud. +Seen, not fixed: every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()" from `svapp/audio/AudioCallbackRecordTarget.cpp:291`. The signal does not exist, so the warning is old and appears on any Qt. +App suite: 549 s, of which `TestUiChecks` takes 82 s. diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 69d01595..a1a7c1dc 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -12,7 +12,8 @@ comes first. ## 1. What to read, and what not to -1. This file, all of it. +1. This file, all of it. Not `docs/calibrate-audio-log.md` (the finished phases), unless + you are phase D. 2. `AGENTS.md` at the repository root. Its rules apply, except that the **build and test commands in section 2 below replace its Windows ones**. 3. `docs/calibrate-audio.md` (the spec, about 400 lines): always §1, §5, §10 and §11, @@ -53,7 +54,7 @@ MinGW setup. - Build: cd /home/user/tony - ninja -j 4 -C build_linux tony test-tony-core test-tony-app pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log + ninja -j 4 -C build_linux tony test-tony-core test-tony-app test-tony-dev test-tony-device pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log Search the log for `error:`; never read it whole. meson reconfigures by itself after `meson.build` changes. Targets have no `.exe`. @@ -64,18 +65,22 @@ MinGW setup. mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core > ../tmp/test.log 2>&1; echo "exit:$?" - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app > ../tmp/test.log 2>&1; echo "exit:$?" - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app some_test_name > ../tmp/test.log 2>&1 + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app > ../tmp/test-app.log 2>&1; echo "exit:$?" + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-dev > ../tmp/test-dev.log 2>&1; echo "exit:$?" + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-dev some_test_name > ../tmp/test-dev.log 2>&1 grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - Read results from the per-suite files, not stdout. - A test name on the command line goes to every suite in the executable, so the exit status of a run with names is meaningless. - - New test classes are registered in `main/test/tony-core-test.cpp` or - `tony-app-test.cpp` and added to `meson.build`. -- While working, run **only your tests**. The app suite takes several minutes of real - time: run both whole suites **once**, at the end, and again only if something failed. - Give the app suite a tool timeout of 10 minutes. + - New test classes are registered in `main/test/tony-core-test.cpp`, + `tony-app-test.cpp` or `tony-dev-test.cpp` and added to `meson.build`. + `TestDevChecks` is the only suite of `test-tony-dev`. +- While working, run **only your tests**. Run all three whole suites **once**, at the + end, and again only if something failed. `test-tony-app` takes about 8 minutes: run it + with the Bash tool's `run_in_background: true` and wait for the completion notice, as + a foreground call can hit the 10-minute tool limit. Never run two suites at once, nor a + suite while building: the app tests record in real time. - App tests run in real time against `FakeAudioIO`: keep them short (seconds, not tens of seconds). - Every behaviour gets a test that can fail. Show it for the two or three that matter @@ -84,7 +89,7 @@ MinGW setup. - Do not weaken or delete an existing test to get green. If one is wrong because the behaviour was meant to change, change it and say so. - **Linux traps:** - - Section 3 lists suite results known before this work started. + - Section 3 gives the suite results to expect. - Qt 6.4 does not match a `SIGNAL()`/`SLOT()` string naming `ModelId` or `sv_frame_t` with moc's `sv::` names: the connection fails at run time with "No such slot". The user's Qt happens to match them, so a string connect can pass there and fail here. @@ -106,474 +111,206 @@ push, amend, stash, or `git add -A`. **Report** (your final message, and all the lead sees; under 60 lines) - What was built, by file, briefly. -- The `Totals` lines of the final full runs of both suites, copied, not paraphrased. +- The `Totals` lines of the final full runs of all three suites, copied, not paraphrased. - Which tests you saw fail without the change. - Choices made, deviations, anything fragile or unfinished. Say it plainly: a problem reported is cheap, one found later is not. -## 3. State of the code (kept by the lead; as of 2026-09-25, before phase A1) - -- Nothing of the feature exists yet. The spec's §11 lists the facts about today's code - that the phases build on. -- **Linux baseline** (after the lead cherry-picked the member-pointer connect fix from - the lyrics branch, `e2cf7c0`): - - core suite: all green except 4 tests in `TestTakesFile` (`takes_folder`, - `relative_audio_path`, `resolve_audio_path`, `in_folder`). They use Windows paths - (`C:\Songs\...`, case-insensitive) and fail on Linux only. Not yours to fix; your - final runs must show exactly these 4 and nothing else. - - app suite: all green, `TestRecordWorkflow` 92 tests in about 4.5 minutes. The lead - also made `stale_pitch_event_ignored` queue its event as a functor (`e13fb9a`), - because invoking a slot by name with an `sv::` argument type fails under Qt 6.4. +## 3. State of the code + +As of 2026-09-26, after C1a and the merge of `default`. The work orders and log of the +finished phases are in `docs/calibrate-audio-log.md`: **do not read it**; what you need of +it is here. + +**What exists** + +- `tony_core`: `LatencyCheck` (the three layouts, the sweep finder, `judgeTake()` with its + verdicts and second-arrival `echo`, `punchInsFor()`), `LatencyCalibration` (the stored + round trip per device key and rate), `TakeDiff` (audio bit-identical outside a range, + events unchanged outside a range ± 0.25 s, pitch and notes across a join, sample step at + a join). Core tests: `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`. +- `AudioCheckRunner` (every build): a `Plan` (layout; `punchIns` × `eventsEach` or explicit + `ranges`; `keepSession`; a `roundTrip` for the run only), steps driven by a 50 ms timer, + never a nested event loop. Its takes go through `MainWindow::record()` with the + override `m_audioCheckTakes` (Record into Selection, Play Reference While Recording, + a 1 s pre-roll, whatever the toolbar says) and are judged from the take's own file. + `progress()` reports the step (`Recording`, `AnalysingTake`, …) and punch-in. + App tests: `TestAudioCheck` in `test-tony-app` (23 tests, about 2 minutes). +- `CalibrateAudioDialog` (every build): instructions, progress, result; in dev builds a + checkbox that carries on into the dev checks with the calibrated round trip. +- `MainWindow`: `m_takeLatency` (`TakeLatency`: round trip, reported pair, rate, start gap + and whether it was measured) for the last take; `roundTripAt()`; + `m_audioCheckRoundTrip`, used only in `recordingStarted()`'s deferred lambda; Record's + action goes to `recordPressed()`, which ignores presses while `audioCheckRunning()`. +- `main/dev/DevChecks` (dev builds, `TONY_DEV_CHECKS`): stages driven by a timer and the + runner's `finished()`. Stage 1 "Fresh punch-ins": the dev reference (`devLayout()`) opened + as a session of its own, punch-ins [6.3, 10.2] and [16.8, 21.2] s. Stage 2 "Save and + reopen" into a numbered scratch folder. Items 1 (`latency_on_this_machine`) and 2 + (`several_phrases_in_one_take`), ±2 ms per sweep. `CheckResult`, `DevReport`, the report + `DevChecks.txt`. Free for new punch-ins: before 4.3 s, 10.2–16.8 s, 21.2–25 s, and the + held tones from 26.9 s (C2's joins). + Tests: `TestDevChecks`, in its own executable `test-tony-dev` (9 tests, about 66 s). +- **From `default`:** `TestUiChecks` (in `test-tony-app`) automates the checklist's screen, + keyboard and dialog items on the fake device. `test-tony-device` (`TestRealDevice.h`, + run by hand) records a 4-minute click track through the air and checks latency, two + recordings in one take, echo, Stop time, input channel levels and live dots. **The user + decided that the dev run takes these over, and `test-tony-device` is retired (C3).** + `TestMainWindow` lives in `main/test/TestMainWindow.h`, with this work's accessors + (`audioCheck()`, `devChecks()`, `takeLatency()`, the menus and actions). + +**Facts found along the way** + +- A model's `getLocalFilename()` is the decoded copy in the temporary directory ("normalise + audio" is on); the file opened is `AudioCheckRunner::mainModelFile()`. +- On the loopback fake with its true round trip every sweep lands at 0 frames; its + reported pair is 2.8 ms short of the true round trip. +- After a reopen, pitch and notes are restored from the session, not analysed. +- `getOutputLevels()` / `getInputLevels()` give the peak since the previous call, and reset + it: one reader only, or each takes the other's peaks. +- `TestAudioCheck`'s fixture deletes each window's `DevChecks`, so that its dialog tests + do not carry on into them. Test references go to Qt's test location + (`QStandardPaths::setTestModeEnabled`); reports and scratch folders to the test's own + directory. +- Every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()" (from + svapp, old, harmless here); ignore it. + +**The user's first run on Windows** (MME, wired mic and headphones, one earcup to the mic): +round trip 301 and 295 ms, verdict Unsteady both times (the spread across or within +punch-ins between 5 and 15 ms); with the mic between both cups, Scattered (over 15 ms). +The detailed figures are awaited. So the ±2 ms of items 1 and 2 will fail on the user's +machine as things stand: thresholds get tuned from real report files, not now. Whether to +keep the stream running between takes (an svapp change) waits on those figures. + +**Linux baseline** (Qt 6.4.2): core all green but 4 `TestTakesFile` tests +(`takes_folder`, `relative_audio_path`, `resolve_audio_path`, `in_folder`: Windows paths); +your final runs must show exactly these 4. `test-tony-app` green in about 8 minutes +(`TestRecordWorkflow` 98, `TestUiChecks` 19, `TestAudioCheck` 23, …); `test-tony-dev` +green. `tony-app-test.cpp` draws text without sub-pixel anti-aliasing, or Qt 6.4's +coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`). -### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) - -- New `main/LatencyCheck.{h,cpp}` in `tony_core`, pure. -- **Generator.** - - A layout gives the length and the events. Make three: *calibration* (about 26 s), - *dev* (adds 3 s held tones) and *long* (4 minutes). - - Each event: a sweep of 1 → 8 kHz, 200 ms, 10 ms raised-cosine edges, −12 dBFS - peak; then a tone at a pitch with a whole number of samples per period at 44.1 kHz - (196, 220.5, 245, 294 Hz; see `docs/testing.md` on pYIN subharmonics). - - Gaps between events are irregular, 1.6–2.6 s, all different by at least 0.1 s. - - Deterministic. It returns the samples and each sweep's exact frame. - - Exponential or linear sweep: choose one and say why. The spec leans neither way. -- **Finder.** Given take audio (samples and rate) and an expected event time, search - ±0.8 s: - - FFT matched filter against the sweep (bqfft; see how `RealtimePitchTracker` uses - it), weighted to the sweep's band; - - the envelope of the result; - - the **earliest** peak within 6 dB of the largest; - - two confidences: the peak over the window's median in dB, and the peak over the - second-best peak outside ±10 ms in dB; - - the error in frames and in ms. - - The thresholds are named constants (15 dB and 6 dB to start). -- **Not in this phase:** aggregation over events and punch-ins, verdicts, second arrivals. -- **Tests,** in a new core test class `TestLatencyCheck`: - - generator: deterministic, events where it says, gaps all different, peak level; - - finder on synthetic takes, made by shifting the reference: - - shifts across ±0.75 s, plus one fractional shift (resample or interpolate); - - white noise at 0 and −10 dB SNR; - - high-pass at 1 kHz and low-pass at 4 kHz; - - polarity inverted; - - a reflection 6 dB **stronger** 7 ms after the direct sound: the direct sound - must be found; - - silence: no confident peak. - - **Show that the reflection test fails** when the finder takes the largest peak. - -### A2 — Verdicts and calibration arithmetic (spec §5 "tony_core") - -- In `LatencyCheck`: given the events inside a take's coverage ranges (one range per - punch-in), and the finder's results, compute: - - median and spread per punch-in and across punch-ins; - - the slope of offset over position; - - the input peak, for clipping; - - whether confidences fall steadily over time, for fading; - - a **second arrival**: a second confident peak at a consistent extra delay across - events, which means monitoring echo. -- `Verdict`: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent, with - the thresholds of spec §5 as named constants. -- The calibration arithmetic as a pure function: new round trip = used round trip + - median offset, in seconds. A take that lands late was spliced from too early a frame. -- **Refined by the lead after A1:** - - **Entry point for B1.** One function takes a layout, the take's samples and rate, - and the punch-ins. Each punch-in is its timeline range in seconds, in the order - recorded. The function returns a summary with the verdict, all flags that applied, - per-punch-in figures, and per-event results. - - **Which events are judged.** An event is *judged* in a punch-in when its sweep and - the finder's window sit inside the range with a margin, a named constant. The splice - cuts content at the range ends, and a sweep cut in half is not a failure of the - path. Count judged and found separately; NoSignal is about found out of judged. - - **Second arrivals.** `findSweep()` computes the second peak but does not return - its position; add it, and its level against the chosen one, to `Arrival`. Monitoring - echo is a second arrival at a consistent extra delay (a few ms of spread) across - most found events, not more than some named dB below the direct sound. - - **Verdict order.** When several apply, pick one and document it, with all flags kept. - Suggested: NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok. - - **PositionDependent against Scattered.** Fit offset over punch-in position. - PositionDependent is a slope above 0.5 % whose fit leaves little residual; large - residuals are Scattered. -- **Tests:** - - one event missing, the rest found; - - fading; - - clipped; - - misplacement that grows with position (the 48000/44100 case) → PositionDependent; - - two punch-ins 20 ms apart → Scattered or Unsteady by threshold; - - monitoring echo detected; - - an event cut by a range end is not judged; - - the arithmetic with both signs. **Show that the sign test fails** when flipped. - - A1 found that a take of 48 kHz frames read as 44.1 kHz finds nothing, because the - sweeps are stretched. Build the rate case as punch-ins displaced by - P·(1 − 44100/48000), not as a stretch. - -### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) - -Read also: `docs/recording.md` whole (154 lines), `docs/architecture.md` sections on -layers, models and commands (search the headings), `docs/testing.md` "What there is to -reuse". In `MainWindow.cpp`, read by range: `record()`, the deferred lambda in -`recordingStarted()`, `pollTakeProgress()`, `finishSingingTake()` and -`wantedPreRollFrames()`. - -- **Pure helper first**, in `LatencyCheck`, with a core test: - - `punchInsFor(layout, count, eventsEach)` returns `count` consecutive punch-in ranges - in seconds. Each holds `eventsEach` events that `judgeTake()` will judge, using its - margins. - - The calibration uses 4 × 3 on the calibration layout. Tests use a short layout of - 2 × 2 (≈ 8 s) to keep real time down. -- **New `main/AudioCheckRunner.{h,cpp}`** in `tony_app_files`. A QObject owned and wired - by `MainWindow`. It is driven by signals and a polling timer, like - `pollTakeProgress()`, **not by nested event loops**: it runs in every build, and the - user can close the window at any moment. -- **Steps:** - 1. Write the reference WAV to the app data directory, overwriting the old one. Mono, - 44.1 kHz. - 2. `checkSaveModified()`, then `openPath(path, ReplaceSession)`. Wait for the - reference's analysis, as `openReference()` in the tests does. - 3. For each punch-in: select its range and call `record()`; the take stops itself. - Wait for the take's analysis before the next punch-in. It is not strictly needed, - but it keeps pYIN's CPU load out of the next take's timing. - 4. Read the take's audio: the model `analyser2()->getMainModelId()`, mixed to mono, - at its own rate. Call `judgeTake()`. -- **Record into Selection, Play Reference While Recording and a 1 s pre-roll** apply to - the check's own takes through an override in `MainWindow`. `record()`, - `recordingStarted()` and `wantedPreRollFrames()` consult it. **Never through - `setChecked()`**: those actions write QSettings. -- **What each take used.** Keep, per take, the round trip it used: today - `computeRecordingLatency(out, in)`, which B2 will change. Also keep the reported - output and input latency, and the **recording's sample rate**. -- **The result** carries: - - the `TakeSummary`; - - the round trip used, and the reported pair; - - the recording's rate and the reference's; - - a **rate-mismatch flag**, set when the two rates differ, whatever the sweeps say - (A2's finding: a 48 kHz take runs off the finder's reach); - - the calibrated round trip from `calibratedRoundTrip()`, meaningful only on Ok or - Unsteady. - - Emitted as a signal when done. Failures (no device, recording refused, the session - closed mid-run) end the run with a reason. -- **Cancel** stops a take in progress through the normal Stop path and clears the - override. `closeSession()` and `~MainWindow()` cancel a running check. -- **Not in this phase:** storing or using the result (B2), any dialog or menu entry (B3). -- **App tests.** Choose between a new class and adding to `TestRecordWorkflow`; say - which. `TestMainWindow` and the fixtures live in `TestRecordWorkflow.h`. Use - `FakeAudioIO` `loopback = true` and the short layout. - - Wrong reported latencies (e.g. 2·4096 and 4096), `inputDelay` = 3·4096 + 123. The - median offset equals the difference, in seconds at 44.1 kHz, to within a few - frames. The verdict is Ok. The calibrated round trip equals `inputDelay`. - - Fake at 48 kHz: the rate mismatch is flagged and both rates are given. Assert only - what the runner reports, and that nothing crashes. What Tony does with 48 kHz takes - is a separate, known bug. - - After a check, the three toggles and their QSettings values are as before. - - Cancel during a take leaves no recording in progress and the override cleared. - - **Show failure** for the first test with the override's Play Reference half - removed: nothing is heard, so NoSignal. - -### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") - -Read also: `docs/recording.md` "Latency"; `main/LatencyUtils.h` whole; in -`MainWindow.cpp`, the deferred lambda in `recordingStarted()` (search -`m_takeLatency.roundTrip`) and `refineRecordingLatency()`; `AudioCheckRunner.h` -(`AudioCheckResult`, `calibrationUsable()`). - -- **New `main/LatencyCalibration.{h,cpp}`** in `tony_core`: - - **Key.** The three Preferences values `createAudioIO()` reads (`audio-target`, - then `audio-playback-device` and `audio-record-device`, each suffixed with the - implementation when one is pinned; see `audioDeviceSettingKey()` in - `MainWindow.cpp`) and the recording's rate. Device names can hold `/` and non-ASCII - characters: encode them, so that QSettings does not make subgroups. - - **Stored:** round trip and spread in seconds, the date, and the reported output - and input latency, each **in seconds**. The reported pair is the staleness - fingerprint: a stored figure is stale when either differs by more than a named - tolerance. - - `store`, `load`, `forget`, and a staleness test. QSettings group - `LatencyCalibration`. -- **Use in `recordingStarted()`.** The round trip is the stored one when there is one - for this key and it is not stale; otherwise the reported sum. Either way it is - **converted to frames at the recording's rate**, from seconds. - - Fix the reported sum's units while you are there. `getTargetPlayLatency()` counts - frames at the session's rate (`ResamplerWrapper` converts it), and - `getSystemRecordLatency()` at the device's. Today the two are added as they come; - B1 measured 242 ms instead of 256 at 48 kHz. - - The recording's model (`m_currentRecordingModelId`) exists by the time the lambda - runs, and gives the rate. - - Keep in `m_takeLatency` which source was used (reported or measured), and log it. - - The start gap and everything downstream stay as they are. -- **MainWindow API for B4.** Store a check's result, forget the stored figure, and - describe the figure in use (source, milliseconds, date). -- **Not in this phase:** any dialog, menu entry or playback change. -- **Tests:** - - **Core.** Store, load and forget round trip, with a device name holding `/` and - `ä`; the staleness tolerance on both sides; different keys stay apart. Use a - QSettings scope the tests own and clear. - - **App.** - - `latency_end_to_end`'s recipe with wrong reported latencies and a stored figure - equal to `inputDelay`: the sung step lands on the reference's. **The same test - without the stored figure must fail**; show it. - - A stale stored figure (fingerprint differs): the reported sum is used. - - 48 kHz fake: the reported sum, in recording frames, equals - `playbackLatency + recordLatency`, both device frames, to within a frame or two. - - Check, store, check again (short plan): the second check's median offset is - within a few frames of 0. This is the strongest test in the feature, and it - takes about 25 s. - -### B3 — The check's playback, and progress (spec §2, §8 "Loudness") - -Read also: `docs/architecture.md` on play parameters and on `Analyser::setAudible()` -(search "audible"); in `Analyser.cpp`, where the reference's pan and the -sonification's audibility are set (search `setPlayPan`, `PlayParameters`). - -- For the check's session only, the reference plays **centred** and at **−12 dBFS - peak** after normalisation: the model is normalised to full scale when it is read, so - the gain has to come off at playback. The pitch-track sonification is **silent**. - - Do it through the play parameters of the check session's models and layers, never - `Analyser::setAudible()`, which writes the shared settings (see `AGENTS.md`). - - Make sure nothing puts them back later in the run, for example the analysis - finishing, or the next take. - - Nothing of the user's own sessions changes. -- **A progress signal** on the runner: step, punch-in *k* of *n*, and the seconds left - where known, for B4's dialog. -- **Two runner fixes from B2's report:** - - **Reference file name.** The reference WAV gets a new name each run (a counter or a - timestamp), and old ones are removed when no session holds them. **Check Again** - otherwise rewrites the file that the check session it replaces still has open; on - Windows that write can fail. Linux cannot show it. - - **Reported output latency.** `AudioCheckRunner` divides the reported output latency - by the reference's rate. Have `TakeLatency` carry both reported figures **in - seconds**, as `MainWindow::roundTripAt()` works them out, and have the result use - those. Then the take path and the check can never disagree. -- **Tests:** - - During a check the reference's play parameters are centred at the planned gain, - and the sonification is not audible. - - Afterwards, a newly opened ordinary file plays as before: reference pan and - sonification as the settings say. - - The sweeps reach `FakeAudioIO`'s captured output at about −12 dBFS on the mixed - channel. - - Progress reports each punch-in in order. - -### B4 — Calibrate Audio dialog and menu (spec §2) - -Read also: how an existing Tony dialog is built and tested (search -`confirmRecordingOverTake` and `askForTakeName` in `MainWindow.cpp` and -`TestRecordWorkflow.h`). - -- **`main/CalibrateAudioDialog.{h,cpp}`**, non-modal and thin. Its pages: - 1. **Instructions:** the output and input device names, the latency in use with its - source, "hold one earcup against the mic, off your ears", moderate volume. - 2. **Progress,** from B3's signal, with Cancel. - 3. **Result:** - - the verdict in plain words, with the fix for each failure (spec §2 and §8); - - the measured round trip against the driver's figure; - - the spread; - - both rates, with a plain sentence when they differ; - - the input peak; - - the echo, if one was heard. - - **Use this latency** (only when `calibrationUsable()`), **Check Again** and - **Close**. -- **Menu, Playback:** - - **Calibrate Audio…**, disabled while recording; - - a disabled line saying the latency in use ("Latency: measured 187 ms, 25 Sep" or - "Latency: driver's figure, 400 ms"); - - **Forget Measured Latency**. -- The calibration plan is 4 punch-ins × 3 events on the calibration layout; it fits - since the lead's spacing change. -- **The device key is taken when the check starts.** Carry it in `AudioCheckResult`, - and have `storeMeasuredLatency()` use it, not the Preferences at the moment the - button is pressed. The dialog is non-modal, so the user could change device in - between (B2's report). Also disable both Audio Device menus while a check runs. -- Anything that asks the user goes through a virtual seam, as - `confirmRecordingOverTake()` does. The app tests drive the dialog's slots directly; - the dialog watchdog fails a test on any unexpected modal dialog. -- **Tests:** - - the menu starts a check; - - Use this latency stores (B2's API), and the menu line changes; - - Forget clears it; - - the dialog's result words for NoSignal and for a rate mismatch; - - Calibrate Audio is disabled during an ordinary take. -- **After B4 the user runs it on Windows** (the checkpoint in spec §7). - -### C0 — TakeDiff (spec §5 "tony_core") - -Read also: `docs/takes.md` on the splice's edge fades and on ranged analysis's merge -window (search "fade", "±", "W"); `main/TakeAudio.h` and `main/TakeEvents.h` for the -existing vocabulary. `TakeEvents` may already hold part of what is needed: reuse it and -do not duplicate it. - -- **New `main/TakeDiff.{h,cpp}`** in `tony_core`, pure. The comparisons the dev checks - (C1–C4) use on a real take. Each returns a plain result: pass or fail, and the numbers - behind it, so that a check can report them. The inputs are sample buffers, event - vectors and frame ranges; no models. -- **Audio unchanged outside a range.** Two sample buffers (before and after a - punch-in, as read from the take's file) are **bit-identical** outside - `[start, end)`, allowing for where the splice's fades fall. Find that in - `TakeAudio.cpp`; do not guess. Report the first differing frame. -- **Events unchanged outside a range ± margin.** Two event vectors (pitch, and notes - with durations) are identical outside `[start − margin, end + margin]`. A note that - crosses the boundary counts as inside. Report what was added, removed and changed. -- **Pitch continuous across a join.** Given pitch events and a join frame: - - no gap longer than N hops within ±W of the join; - - no two events at the same frame; - - frames strictly increasing. -- **One note across a join.** Exactly one note spans the join frame, and no note - begins or ends within ±X of it. X is a named constant. -- **No step at a join.** The largest first difference of the samples within ±2 ms of - the join, against the typical first difference over the 50 ms around it, in dB. It - should stay near 0 dB when the splice is clean, and a hard cut shows as a large - excess. The threshold is a named constant; justify it. -- **Tests,** in a new core class `TestTakeDiff`, on synthetic data: - - each comparison passing and failing on purpose: one sample changed just outside - the range; one pitch event dropped at the join; a doubled frame; a note split in - two at the join; a hard cut in a sine. - - **Show failure** for two of them by breaking the code. - -### C1a — Dev-check framework, items 1 and 2 (spec §2 point 5, §3, §4, §5 "development builds only") - -Read also: `main/AudioCheckRunner.{h,cpp}` whole (about 1000 lines together; you extend -it), `main/CalibrateAudioDialog.h`, `main/test/TestAudioCheck.h` for the fixture -(`loopback()`, `shortPlan()`, `makeWindow()`), `docs/takes.md` on where take files go -before and after a save, and spec §4's rows for items 1 and 2. - -- **Build flag** (spec §3). In `meson.build`, a build type not starting with `release` - adds `-DTONY_DEV_CHECKS` to `general_defines`, and only then are `main/dev/*.cpp` - compiled into `tony_app` and their headers moc'd. `build_linux` is `debugoptimized`, so - it has them. Everything under `main/dev/`, and every use of it elsewhere (a `friend` - line, a member, a dialog widget, the test class's registration), is inside - `#ifdef TONY_DEV_CHECKS`. A release build must compile with no `main/dev/` file; the - lead builds one later, so keep the `#ifdef`s tidy. -- **Runner extensions** (every build; small; each tested in `TestAudioCheck`): - - **`Plan` ranges.** Explicit punch-ins in seconds; when given, they replace - `punchInsFor()`. `start()` refuses ranges that overlap, are out of order or lie - outside the layout. - - **`Plan` keeps the session.** Record into the session open now, which the caller - says is a check reference of the plan's layout: no reference written or opened, no - save question; straight on to the reference's analysis wait and the punch-ins. The - take keeps what earlier runs recorded; only this plan's punch-ins are judged. - - **`Plan` round trip, for the run only** (spec §10 "a dev run uses the new figure - for itself only"). Seconds; unset means the window's own. The window uses it for the - check's takes only, where `recordingStarted()` takes `roundTripAt()` now. It never - touches the stored figure or the Playback menu's line. Log it as the check's own. - - **`TakeLatency` gets the start gap** each take used, and whether it was measured or - only estimated (item 2 reports it per punch-in). - - **No save question for the check's own session.** When the session open has never - been saved and its main file is in `referenceDirectory()`, replacing it asks - nothing: it is the check before. This also ends B4's Check Again prompt. - - **Record during a check** (from B4): greyed while a check runs, and `record()` - ignores the user's press then. Pressing it today ends the check's take early. -- **`main/dev/DevChecks.{h,cpp}`**, a `QObject`. - - **No nested event loop.** Spec §5 said `waitUntil()` with a `QEventLoop`; the lead - changed it: a list of stages driven by the runner's `finished()` and a polling timer, - as the runner is driven, for the runner's reason (the window can be closed at any - moment). Each stage starts something and says when it is done; a stage that times - out fails the run. - - `start(Options)`, `cancel()`, `isRunning()`, `sessionClosing()` (as the runner's: - ends the run unless the run itself is replacing the session). Signals `progress` - (stage name, n of m) and `finished(DevReport)`, once however the run ends. - - `Options`: the round trip for the run (seconds), the report directory ("" for - `TONY_TEST_LOG_DIR` if set, else `AppDataLocation`), the scratch directory ("" for - `AppDataLocation`). - - `CheckResult { item, name, verdict (Pass, Fail, Measured, Skipped), numbers (label and - value pairs, as text), message }`; `DevReport { checks, failure, reportPath, - sessionPath }`. A run that ends early marks the checks it did not reach Skipped, - with the reason. - - Owned by `MainWindow` in dev builds, like the runner; a `friend` of it under the - `#ifdef`. `~MainWindow` deletes it after the dialog and before the runner; - `closeSession()` calls its `sessionClosing()`. -- **The stages of C1a** (C1b inserts more before the last): - 1. **Fresh punch-ins.** A runner run on `devLayout()`, opening a new reference (no - save question: see above), with two explicit punch-ins in separate regions of the - calibration part, each holding two sweeps, and the run's round trip. Choose ranges - that leave the held tones (after 25 s) and the start (before 3 s) free for later - stages, and say which you chose. - 2. **Save and reopen.** Save the session into a scratch folder (below) with - `MainWindow`'s own save path, no dialog; reopen it; wait for the analyses; read the - take's file again. -- **Items:** - - **1** Pass when every judged sweep of every punch-in lands within ±2 ms (a named - constant), and after the reopen the take's audio judged again gives the same offsets - and its pitch and notes are the same events as before the save. Numbers: offsets - per punch-in, largest offset, the round trip used. - - **2** Pass when every punch-in was added to the take, each placed within ±2 ms, each - with a measured start gap of its own. Numbers: per punch-in its median offset and - start gap. -- **Scratch folders.** Not deleted at the end: the session open afterwards lives in it - (spec §2: the test session stays open). As `nextReferencePath()` does for references: - numbered folders, and at the start of a run every one that the open session does not - use is removed. The report names the folder. -- **Report.** - - Text file `DevChecks.txt` in the report directory: the run's date, devices and round - trip, then one block per check grouped by checklist item (verdict, message, - numbers), ending `Totals: N passed, N failed, N measured, N skipped`. - - Tests pass a report directory of their own: a failing run's report must not land - among the suites' own files, where the lead greps for `^FAIL`. -- **Dialog** (dev builds only). The instructions page gets a checkbox, "Run the dev - checks after calibrating", on by default and not remembered. With it on and the - calibration usable, the progress page goes straight on into the dev checks with - `calibratedRoundTrip`, Cancel cancels whichever is running, and the result page - shows the calibration as today plus the dev report: one line per check, then the - report file's path. With the calibration unusable, the dev checks do not run and the - page says so. -- **Tests,** new class `TestDevChecks` in the app suite, compiled and registered only - in dev builds; the fixture copied from `TestAudioCheck`, not shared by editing it: - - passing on a loopback fake with its true round trip: items 1 and 2 pass, the report - file ends with `Totals:`, and the session open afterwards is the one in the scratch - folder; - - failing with the round trip 20 ms off: items 1 and 2 fail, and the report shows the - offsets; - - cancelled mid-run, and the session closed mid-run: `finished` once, no take left - recording, the user's three toggles and stored latency untouched; - - the dialog with the checkbox on runs the dev checks after the calibration (the short - plan for the calibration). - - **Show failure** for item 1's tolerance and for the run round trip being ignored. - - Report the real time `TestDevChecks` adds. Spec §6 says more than about a minute - over all the dev phases moves them to a third executable, which the lead will ask - the user about; do not create one. - -### C1b — Observer, items 7, 12, 13, 14 (spec §4, §5 `TakeObserver`) - -To be refined by the lead after C1a. Outline: - -- `main/dev/TakeObserver`: polls every 20 ms while the runner reports a take recording - and records, with the time: playback frame, output and input levels (left and right, - as `getOutputLevels()` and `getInputLevels()` give them; find out and say exactly what - one reading covers), status text, frames received, any modal widget up, and when the - take stopped. Live dots, pane centre and action states wait for C2. -- Before and after each punch-in: the take's samples from its file, and its pitch and +Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). + +**Order from here:** C1b, C1c, C2, C3, then the lead's release build, then D. The spec's +old C2 (observer group) and C4 (smoke group) are gone: `TestUiChecks` covers the smoke +items and the screen, and what the dev run measures on the device is now in C1b–C2. + +**Budget.** C1a took over 500k tokens against a target of 200k. Read what you need, but +build only your phase, keep tests few and meaningful, and stop to report rather than +redesign. + +### C1b — Take observer; items 3, 4, 5 (spec §4 rows 3, 4, 5, 8) + +Read also: `main/dev/DevChecks.{h,cpp}` whole; `main/test/TestDevChecks.h` for the +fixture; in `main/test/TestRealDevice.h` the tests `record_the_reference_through_the_air`, +`nothing_of_the_take_comes_back_out` and `live_dots_were_drawn` (search `Checklist:`), +which these checks take over; `docs/recording.md` on the live tracker and the cursor +during a take; `main/test/FakeAudioIO.h`. + +- **`main/dev/TakeObserver.{h,cpp}`** (dev builds). DevChecks starts one when the + runner's `progress()` reports `Recording` for a punch-in, and stops it when that + punch-in's analysis is done. Every 20 ms it records, with the time: the playback frame + (the `ViewManager`'s), the output levels, the input levels, the frames received, the + status text, whether a modal widget is up; and each live dot as it first appears (the + realtime model, `m_realtimePitchModelId`, looked up each poll), with its frame, its + value and the playback frame at that moment; and when the take stopped recording. +- **Levels, a checked fact.** `getOutputLevels()` / `getInputLevels()` return the peak + since the previous call and reset it. `ViewManager::checkPlayStatus()` (svgui) already + reads them every 20 ms: **while recording it reads the input levels only**, and emits + `monitoringLevelsChanged(left, right)` when they change; while only playing, the + output levels. So during a take the observer takes the input levels from that signal, + and is the only reader of the output levels. Find out whether the lead-in counts as + recording there, and say. +- **Items**, from the punch-ins of stage 1 (they then hold for every later punch-in too): + - **3 Live dots** (and row 8's number). Pass when each punch-in drew more than 10 dots + (as `test-tony-device`), and the dots lie on the reference's tones: frames within the + tone's span ± 1 hop, value within 50 cents of the tone. Numbers: dots per punch-in, + and the dot-to-cursor offset (dot frame against the playback frame when it + appeared), median and spread in ms: the "Cursor versus dots" risk of spec §8. + - **4 Nothing of the take in the speakers.** Fail on a second arrival + (`TakeSummary::echo`, as the calibration already detects it), or on any output level + above zero in a poll that lies wholly in one of the reference's silent gaps, with a + margin you work out from the output latency and the poll interval; also Play Singing + Audio's state the same before and after the take. Numbers: echo delay and level, + the largest output level in the gaps. + - **5 Mic channel.** Per-channel level of each punch-in's raw recording (the newest + `recorded-*.wav`, as `newestRecording()` finds it in `TestRealDevice.h`, or better the + path the window itself knows, if it does). Measured: which input carries the mic, and + its level. When it is input 2, item 3's dots are the check (Pass/Fail); otherwise + "not applicable here". +- **The report header** gains what `test-tony-device`'s `device()` test logs: the audio + drivers built in, the reported playback and record latencies. +- **`FakeAudioIO`**: a second loopback tap (delay and gain), new fields that change no + existing meaning, for item 4's failing test. The input on one channel exists already + (`inputChannel`). +- **Tests** (`TestDevChecks`): passing on the loopback fake; item 4 failing with the echo + tap; item 5 with the input on channel 2 (dots still drawn, "input 2"). Show failure for + the dots-on-tones check and the silent-gap check by breaking them. + +### C1c — Re-record and pre-roll stages; items 7, 12, 13, 14 (spec §4 rows 7, 12, 13, 14) + +To be refined by the lead after C1b. Outline: + +- Before and after each punch-in: the take's samples from its file and its pitch and notes, compared with `TakeDiff`. -- New stages before the save and reopen: re-record over one of stage 1's punch-ins - starting inside it, so its lead-in plays over earlier material (items 7, 12, 14), with - no overwrite question for the check's takes; then a punch-in at P = 1 s with a 3 s - pre-roll for that plan (item 13: playback from 0, a shorter countdown, placement - right; checklist item 13 is "pre-roll less than 3 s from the start"). +- New stages before "Save and reopen": re-record over one of stage 1's punch-ins, starting + inside it, so its lead-in plays over earlier material (items 7, 12, 14), with no + overwrite question for the check's takes; then a punch-in at P = 1 s with a 3 s + pre-roll for that plan (item 13: playback from 0, a shorter countdown, placement right). +- Item 14 from the observer: the take stopped within 0.25 s plus one poll of the + selection's end; the coverage added is exactly the selection; no modal widget. - Items 1 and 2 then cover every punch-in of the run. -### C2 — Observer group (spec §4 items 3, 4, 5, 8, 15, 16) +### C2 — Long song and joins; items 9, 10 (spec §4 rows 9, 10) + +To be refined by the lead after C1c. Outline: -To be refined by the lead. +- **9** After "Save and reopen": the long reference (`longLayout()`, 4 minutes) as a new + session; the whole-song analysis time; two punch-ins far apart (as `test-tony-device`, + about 60 s and 150 s), each timed from Stop to its pitch merged. Pass when each is under + half the whole-song time and pitch outside the range is unchanged (the ranged path ran). + These punch-ins also count for items 1 and 2: placement far into a song is where a + rate mismatch shows. +- **10** Two punch-ins meeting in the middle of a held tone of the dev layout: `TakeDiff`'s + step, pitch and note checks at the join, and nothing moved outside ± 0.25 s. C0 found + (from the code) that the join is a 10 ms dip, and that the notes merge by onset may drop + or split the note: measure, report the result as it is, and if it fails on today's code + the app test is an expected failure with the reason, as `default` does for its defects. -### C3 — Joins and long song (spec §4 items 9, 10) +### C3 — Retire `test-tony-device` -To be refined by the lead. +To be refined by the lead after C2. Outline: remove `TestRealDevice.h`, +`tony-device-check.cpp` and its target once everything it checks is in the dev run (its +"no input does no harm" case included, as an app test if not already covered); update the +docs that name it in the same commit (`AGENTS.md`, `building.md`, `testing.md`, and +`manual-checklist.md` section 1, which becomes "Calibrate Audio with the dev checks"). -### C4 — Smoke group (spec §4 items 19–25, 27, 28) +### Lead — release build -To be refined by the lead. +A build directory of type `release`: it must compile and link with no `main/dev/` file +(spec §8). The lead does this, not an agent. ### D — Documentation pass -- Bring `docs/` up to date from the code, this log and the spec: +- Read the log in `docs/calibrate-audio-log.md` as well as this one: you are the one + phase that does. +- Bring `docs/` up to date from the code, the logs and the spec: - `recording.md`: the latency section; - - `testing.md`: a Dev checks section, and the loopback fake; - - `manual-checklist.md`: automated items marked with their check's name; + - `testing.md`: a Dev checks section, `TestAudioCheck`, the loopback fake, and who uses + `TestMainWindow`; + - `manual-checklist.md`: what the dev run settles, and what is left by hand; - `architecture.md`, if new classes change who owns what; - - `open-points.md`: remove the item, and add what is left open. + - `building.md`: whether "Qt 6.11, not Ubuntu's 6.4" still holds now that the connects + are member-pointer ones; + - `README.md` and `open-points.md`: agree on what of the checklist has been tried; remove + the Calibrate Audio "Not built" item and add what is left open, the svapp + `aboutToBeDeleted()` warning included. - Make `docs/calibrate-audio.md` describe what was built, with a "Known limitations and open points" section. -- Delete this work-orders file and remove the spec's link to it. +- Delete this work-orders file and the log file, and remove the spec's links to them. - No code. Suspected bugs go in the report. ## 5. Log (newest last; 25 lines at most per entry) @@ -585,161 +322,3 @@ Template: Choices / deviations: ... The next phase must know: ... Left open: ... - -### Phase A1 — 2026-09-25 -Built: `main/LatencyCheck.{h,cpp}` in `tony_core`: `calibrationLayout()`, `devLayout()`, `longLayout()` (rate defaults to 44100), `sweep(rate)`, `generate(layout)`, `findSweep(samples, count, rate, expectedSeconds)` → `Arrival {found, errorFrames, errorSeconds, peakOverMedianDb, peakOverSecondDb, levelDb, inputPeak}`. Thresholds are `k…` constants in the header. `main/test/TestLatencyCheck.h`: 16 tests, 1.4 s. -Choices / deviations: -- Linear sweep: flat spectrum, narrowest peak; an exponential sweep's harmonics match it 67/106 ms *early*, where the earliest-peak rule looks. -- Envelope = magnitude of the analytic signal (a second inverse FFT gives the Hilbert part). -- Earlier peak counts if it is a local maximum, within 6 dB, and ≥ 1 ms before the largest. No dip rule: one arrival with a hole in its band beats, with deep dips, so only time tells arrivals apart (`finder_takes_one_arrival_as_one`). -- No band mask beyond the matched filter itself: a 0 dBFS 100 Hz hum already comes through 100 dB down; a mask changed nothing measurable. -- Confidences are the chosen (earliest) peak's; "second" is the envelope's maximum more than 10 ms from it. -- A gap is sweep to sweep. Calibration sweeps at 1.0, 3.1, 4.7, 7.2, 9.1, 11.4, 13.1, 15.7, 17.7, 19.5, 21.9, 24.1 s; each tone starts 0.3 s after its sweep, 0.8 s long. Dev adds 3 s held tones after sweeps at 26.9, 30.9, 35.2 s (40 s). Long: 113 events, 240 s. -- Reflection test at 5.5 dB (direct found) and 7 dB (reflection taken): exactly 6 dB passes here only by rounding (6.1 dB flips). -The next phase must know: -- Pass the take's samples at their own rate, and the expected time in seconds (layout frame / layout rate). -- A take of 48 kHz samples read as 44.1 kHz (sweeps stretched 8.8%) finds nothing: level −22 dB, 0.1 dB over the second. `findSweep` at 48000 finds them. A2's "resampled by 48000/44100" case must be built as misplaced frames, or try both rates. -- Measured: noise alone 6–12 dB over the median (threshold 15); SNR 0 / −10 dB: 38 / 28 dB over the median, 26 / 16 dB over the second. In digital silence the median is ~0, so over-the-median reads up to the 200 dB clamp. -- About 20 ms per call. The second peak's position is computed but not returned; second arrivals (A2) need it. -Left open: every threshold untuned; nothing reads `inputPeak` yet. - -### Phase A2 — 2026-09-25 -Built: in `main/LatencyCheck.{h,cpp}`: `Arrival::secondDelaySeconds`, `secondLevelDb`; `judgeTake(layout, take, count, rate, punchIns)` → `TakeSummary {verdict, flags, punchIns[], events[], judged, found, medianOffset, spread, slope, slopeResidual, inputPeak, fadingDb, echo}`; `PunchIn {start, end}` in timeline seconds; `Verdict`, declared in precedence order (NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok); `verdictName()`; `calibratedRoundTrip(used, offset)` = used + offset. 9 tests in `TestLatencyCheck` itself (reusing its helpers); the class now takes 2.9 s. -Choices / deviations: -- Judged: the finder's window, plus a sweep's length past it, inside the range with 50 ms to spare (the splice crossfades 5 ms). An event under a later, overlapping punch-in is judged in that one only. Ranges stop at the take's end. -- Across = median of the punch-ins' medians (each stream start counts once); spread = their max − min. Unsteady/Scattered take the larger of that and any spread within one punch-in. PositionDependent: 3 punch-ins at least (a line through two always fits), |slope| > 0.5 %, and what the least-squares line leaves ≤ 5 ms. NoSignal also when nothing is found, or nothing judged. -- Fading reads `levelDb`, not a confidence: over the median of near silence a confidence runs up to the 200 dB clamp. Median of the first half of the judged events (as recorded) minus that of the second ≥ 10 dB; needs 6 events. -- Echo is judged over events *heard* (≥ 15 dB over the median), not found: an echo within 6 dB leaves nothing found. Added `kEchoMinDelaySeconds` = 20 ms: a reflection 9 ms after the direct sound and 5.5 dB stronger leaves the tail of its peak at 10.1 ms, 9–23 dB down, after every sweep, and was reported as an echo. Also ≤ 30 dB down, within 3 ms of the median delay, in more than half of the heard events and 3 at least. -- Clipped: the largest sample inside the ranges ≥ −0.2 dBFS. -The next phase must know: -- **At 48 kHz, §2's punch-ins at 14 and 20 s land 1.14 and 1.63 s early, beyond the finder's 0.8 s reach.** Measured: the finder takes the neighbouring sweeps, fully confident (+662, +575 ms), and the verdict is Scattered. PositionDependent needs punch-ins that start before about 9.8 s, so §6's "a 48 kHz fake reports PositionDependent" fails with §2's punch-ins. The rates themselves can name the rate. -- §2's punch-ins as [2,7], [8,13], [14,19], [20,25] s judge 7 events (2, 2, 2, 1). A punch-in shorter than 1.95 s judges none. -- An echo under 20 ms (an interface's direct monitor) is not seen. -Left open: every threshold untuned. The verdict thresholds came from the lead's brief; spec §5 has none of them. - -### Phase B1 — 2026-09-26 -Built: `LatencyCheck::punchInsFor()` (+ `kPunchInSlackSeconds`, 10 ms) and core test `punch_ins_hold_the_events_asked_for`. `TakeLatency` in `LatencyUtils.h`. `main/AudioCheckRunner.{h,cpp}` (`tony_app`): `Plan`, `start()`, `cancel()`, `sessionClosing()`, `finished(AudioCheckResult)`. `MainWindow`: `friend class AudioCheckRunner`, the override `m_audioCheckTakes` (read by `record()`, the `recordingStarted()` lambda, `wantedPreRollFrames()`), `m_takeLatency`, the runner made in the constructor, deleted first in `~MainWindow`, told by `closeSession()`. New app class `main/test/TestAudioCheck.h`: 6 tests, 32 s. -Choices / deviations: -- **4 × 3 does not fit the calibration layout** (spec §2 now says why). `punchInsFor()` returns nothing then; 4 × 2 and 3 × 3 fit. B3 needs the lead's choice. -- The take is read from its **file**, not its model: the model is peak-normalised as read (measured: every take Clipped) and resampled to 44.1 kHz. -- `friend` over accessors: the runner needs about nine internals. A new test class, not `TestRecordWorkflow` (5475 lines); its watchdog and init/cleanup are copied. -- Selection: `clearSelections()` + `addSelectionQuietly()`, so the reference is not re-analysed during a take; each punch-in adds one or two "Select" undo steps, as a user's selection does. -- Waits: poll every 50 ms for "nothing being analysed", not "analysed": with auto-analysis off the reference never gets layers (test `check_runs_without_automatic_analysis`). Limits 60 s reference, 30 s a take's analysis, take length + 10 s to stop (then the Stop path); each ends the run with a reason. -- `~MainWindow` deletes the runner: the run ends silently and a take in progress is left to the destructor (the Stop path would splice and start pYIN mid-teardown). `closeSession()` → Stop path + `finished()`. -- Reference: `AppDataLocation/calibrate-audio-reference.wav` unless the plan names a path (tests: their temp dir). Save question first, then write, then open. -- `calibrationUsable()`: Ok or Unsteady, and no rate mismatch. -- No loopback gain was needed (see below). -The next phase must know: -- `Analyser` pans the reference hard left, sonification hard right: only the **left earcup** carries sweeps. Normalised, the reference plays at 0 dBFS, not −12. The fake averages channels: sweeps loop back at half level (peak 0.905 with the synth). -- `getTargetPlayLatency()` counts session frames, `getSystemRecordLatency()` device frames; the result converts each at its own rate. At 48 kHz the round trip used was 242 ms, not 256. -- 48 kHz fake: Scattered, 3 of 3 found, offsets −200 ms median, as A2 foresaw. -- No progress signal yet; B3's dialog may want one (`m_punchIn`). -Left open: no test deletes the window mid-check. Seen while proving the session-close hook: closing a session during an **ordinary** take, then pressing Stop, hangs (pre-existing). - -### Lead — 2026-09-26, after B1 -- Reordered the calibration spacings to `{21,16,25,19,17,23,26,20,24,18,22}` (B1's suggestion) so that 4 × 3 punch-ins fit; `punch_ins_hold_the_events_asked_for` now asks for 4 × 3 and failed on the old order. `judge_only_events_inside_a_punch_in` names its events from the layout instead of 9.1 and 11.4 s. -- Split B3 into B3 (the check's playback and progress) and B4 (dialog and menu), after B1 needed 370k tokens. -- Calibration sweeps now at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, 17.7, 20.1, 21.9, 24.1 s. - -### Phase B2 — 2026-09-26 -Built: `main/LatencyCalibration.{h,cpp}` (`tony_core`, namespace): `Key`, `currentKey(settings, rate)`, `Figure`, `store`/`load`/`forget` (all take a `QSettings &`), `isStale`, `kStaleToleranceSeconds` = 1 ms, `Source`, `InUse {source, roundTrip, date, stale}`, `roundTripInUse()`, `reportedSeconds()`, `toFrames()`. `TakeLatency::measured`. `MainWindow`: `roundTripAt(rate)`, used by the `recordingStarted()` lambda; B4's API `storeMeasuredLatency(result)`, `forgetMeasuredLatency()`, `latencyInUse()`. Core class `TestLatencyCalibration` (6 tests); app tests `latency_measured_round_trip_used`, `latency_stale_round_trip_ignored`, `latency_reported_at_the_device_rate`, `latency_reported_with_device_opened_first` (TestRecordWorkflow), `check_stored_round_trip_is_used` (TestAudioCheck, 25 s). -Choices / deviations: -- Settings: `LatencyCalibration/||//`. In names only `%`, `/`, `\` and `|` are percent-encoded. Registry key names stop at 255 characters, and full encoding of long non-ASCII names could pass that. Values are stored as text (`'g'`, 17 digits) and the date as ISO UTC. -- The key follows `createAudioIO()`, not `audioDeviceSettingKey()`: they differ for `audio-target` = "auto", which Tony never writes. -- **The output latency's unit** (spec §11, corrected): frames at `m_playSource->getDeviceSampleRate()`, or at the recording's rate when that is 0. This is not always the session's rate. If a device is chosen before any file is opened (or a standalone take is the first action), `ResamplerWrapper` has no source rate yet. It passes the device's figure through unconverted and tells the play source 0. Using the session's rate there gave 13012 frames instead of 12288 at 48 kHz (`latency_reported_with_device_opened_first`). -- `storeMeasuredLatency()` stores only when `calibrationUsable()`. The key uses the current Preferences, the rate is the result's, and the fingerprint is the result's reported pair. -- `latencyInUse()`/`forget` use the rate of the last take placed with a round trip. Before any take they use the session's rate, the only rate at which a usable check stores. That rate is reset when a device is chosen from the menu. The device's rate cannot be known before a take: `AudioCallbackRecordTarget` has no getter for it. -- A stale figure is not deleted; it becomes valid again if the driver goes back to reporting the old pair. -The next phase must know: -- With no figure stored at 44.1 kHz, the round trip is exactly the old sum; this is tested in core across a grid of values. At 48 kHz the check now uses 256 ms, not 242. -- The runner's `reportedOutputLatency` still divides by the reference's rate. It matches the take path at 44.1 kHz, the only rate that is stored. -- `storeMeasuredLatency()` reads the device from the Preferences when "Use this latency" is pressed. If B4's non-modal dialog lets the device change in between, the figure is stored under the new device. -Left open: `computeRecordingLatency()` is unused outside `TestLatencyShift`. The svapp fork could add `AudioCallbackRecordTarget::getRecordSampleRate()` so that `latencyInUse()` knows the rate before the first take. - -### Phase B3 — 2026-09-26 -Built: `AudioCheckRunner::setPlayback()`: on the play parameters of the check session's own models, the reference audible, centred, gain 10^(kPeakDbfs/20) (1 if the normalise preference is off); its pitch and notes muted. Applied once the reference is open and again before every punch-in. Public `Step`, `Progress {step, punchIn, punchIns, secondsLeft}`, signal `progress()`. `referenceDirectory()`, `nextReferencePath(dir, inUse)`. `LatencyCalibration::InUse::reportedOutput/Input` (seconds, whichever source won); `TakeLatency::reportedOutput/Input` are now those seconds, and `end()` copies them. App tests `check_plays_the_reference_centred_and_quiet` (13 s), `check_leaves_the_next_session_alone` (4 s), `check_reference_gets_a_file_of_its_own`; core `round_trip_in_use` extended. Four existing tests now compare the reported pair in seconds. -Choices / deviations: -- The toolbar's reference level control (`m_audioLPW`) answers a gain between its notches (−12 dB lies between −11.25 and −20) by emitting the nearest; `audioGainChanged()` then sets it through `Analyser::setGain()`/`setAudible()`, writing `Analyser/audible-0`. `setPlayback()` moves the control first under a `QSignalBlocker`. Without it both new session tests fail on the settings. -- Reference files `calibrate-audio-reference-N.wav`: every such file in the directory but the open session's main model file is removed, then the lowest free N is taken, so names alternate 1, 2. A timestamp per run would add a dead Recent Files entry per run (`RecentFiles` has no remove, and keeps 20). -- The check session keeps its playback after the run. The reference is made audible even where the user's settings mute it. -- `progress()` comes from `poll()` only, never from inside `start()`/`cancel()`: on each step change, and each whole second less while recording. `secondsLeft` counts recording to come (min(1 s, start) + range per take), not analyses. No metatype: direct connections only. -- "Afterwards" is tested after a cancelled run, against the same file opened before the check: the settings alone do not say how a session plays (below). -The next phase must know: -- Pre-existing, not fixed: (a) in the first file of a window the same feedback moves the pitch and notes gain from 0.5 to 0.562 and forces both audible, writing the settings; (b) `audible-0` is overridden at load by `audible-3`: the spectrogram is a layer on the reference's model, so it plays whenever `audible-3` is true. -- B4: add a few seconds per analysis to `secondsLeft` for a rough total. -Left open: the default reference path is exercised only through `nextReferencePath()`; the tests name their own file. - -### Lead — 2026-09-26, after B3 -- De-raced `TestSingingAnalysis::waitForRange()` (`2a306fb`): `initialAnalysisCompleted()` also fires from `layerCompletionChanged()` before a ranged merge; it failed once in a full run. -- For phase D's `open-points.md`, older bugs B3 found: (a) in a window's first file the toolbar level control's notches move the pitch/notes gain 0.5 → 0.562, force both audible and write that to the settings; (b) `audible-0` (Play Audio) is overridden on load by `audible-3` (the spectrogram layer on the same model, loaded last). Also B1's: closing a session during an ordinary take, then Stop, hangs. - -### Phase B4 — 2026-09-26 -Built: `main/CalibrateAudioDialog.{h,cpp}` (`tony_app`), a `QDialog`, not modal, with three pages (instructions, progress, result): `present()`, `startCheck()` (Start and Check Again), `cancelCheck()`, `useLatency()`, `showResult()`, `reject()`; for tests `page()`, `pageText()`, `canUseLatency()`, `setPlan()`; static `describeLatency(InUse)`. `AudioCheckRunner::calibrationPlan()` (4 × 3). `AudioCheckResult::key`: the Preferences' devices taken in `start()`, the rate set in `end()`; `storeMeasuredLatency()` stores under it. `MainWindow`: Playback ▸ Calibrate Audio..., a disabled line "Latency: ...", Forget Measured Latency, after the device submenus; `calibrateAudio()`, `updateLatencyMenuLine()`. `updateMenuStates()` shuts Calibrate Audio during any take or check, and both device menus during a check; the runner's `progress` and `finished` call it. `TestAudioCheck`: 5 tests `calibrate_audio_*`, one full run (13 s); `TestMainWindow` accessors. -Choices / deviations: -- The window owns the dialog, makes it on first use, and deletes it in `~MainWindow` before the runner. The dialog calls only `latencyInUse()` and `storeMeasuredLatency()`, and shows only runs it started itself. -- Closing it (title bar, Esc, Close) while its check runs cancels the check. Opened again, it shows the instructions. -- Menu line: "Latency: measured 281 ms, 26 Sep" (the year only when not this one), "Latency: driver's figure, 279 ms", plus " (the measured one is out of date)" when stale, or "not known yet" while the device reports 0. Forget is enabled while a figure is kept, stale or not. The line is refreshed on the menu's `aboutToShow`, in `updateMenuStates()`, and by store and forget. -- Result page: one sentence for the verdict, then its fix. A rate mismatch replaces the verdict's words, and an echo adds a paragraph. Then a table: the round trip measured (not for NoSignal or a rate mismatch) against the driver's (out + in), what the takes were placed with, each punch-in's offset, the spread, found of judged, both rates, the input peak, the echo, and the devices. The text is selectable, to copy. -- Progress: the runner's seconds left plus `kSecondsPerAnalysis` = 3 s for each analysis to come. The bar never goes back. -- Nothing new asks the user, so there is no new seam: Forget asks nothing. -The next phase must know: -- A run replacing a check session asks "Session modified: save?" (its takes mark it modified). Check Again always meets it; answer No. The runner could skip the question for its own reference's session. -- Not on the result page: §2's mic channel and noise floor. The runner measures neither. -Left open: Record stays enabled during a check. Pressing it there goes through the Stop path and ends the check's take early; what the run then makes of it was not tried. - -### Lead — 2026-09-26, after B4 -- The button is complete; the user's Windows run is the checkpoint (spec §7). C0 onwards goes on meanwhile. -- Left for C1 (small, in passing): Record stays enabled during a check, and pressing it ends the check's take early; grey it while a check runs. -- Left for D: spec §2 promises the mic channel and noise floor on the result page, which nothing measures yet (C2's item 5 measures the channel); §8 says "modal progress dialog", true only of the dev run; Check Again always asks to save the check's own session (answer No), a possible later nicety. - -### Phase C0 — 2026-09-26 -Built: `main/TakeDiff.{h,cpp}` (`tony_core`, namespace). Five comparisons, each returning a struct with `pass` and its numbers: `audioOutside()` → `AudioDiff {firstDifference, differences, largestDifference}`; `eventsOutside()` → `EventDiff {added, removed, changed (before, after), firstDifference, window}`; `pitchAcross()` → `PitchJoin {events, largestGap, largestGapFrom, doubled, firstDoubled, outOfOrder, firstOutOfOrder, window}`; `notesAcross()` → `NoteJoin {spanning, edgesNear, nearestEdge}`; `stepAt()` → `SampleStep {stepDb, largest, typical, largestAt, channel}`. Samples are interleaved (`const float *`, frames, channels); times are named constants in seconds, with the rate passed. Core class `TestTakeDiff`: 19 tests, 10 ms. -Choices / deviations: -- Fades: `weightAt()` in `TakeAudio.cpp` mixes only frames of [start, end), both edge frames included, and copies every other frame; take files are float WAV. So `audioOutside()` excuses nothing outside the range: pass the placed range (the coverage added). A test runs the real `splice()` and `erase()` through files: identical outside, and the range less one frame at either end differs exactly at that frame. Frames past a buffer's end are silence; bits are compared, not values. -- Events: "inside" is what `TakeEvents::eraseNotes()` of the window would touch (a crossing note is inside; a pitch event goes by its frame). Window [start − 0.25 s, end + 0.25 s), half-open. Multiset difference; what is left at one frame is "changed". A note that grew from outside into the window reads as removed. -- Pitch window ±0.5 s (`kPitchWindowSeconds`), not ±0.25: the second run's merge seam is 0.25 s before the join, and would sit on the edge of a ±0.25 s window. Gap limit 1 hop (`kMaxGapHops`). Gaps run to the neighbours beyond the window, or to its edge if there are none, so a hole reaching in, a track stopping inside, or an empty window all fail. -- Notes: X = 0.5 s (`kNoteClearanceSeconds`), closed. A note is [f, f + d), so one ending at the join does not hold it. -- Step: the largest |x[i] − x[i−1]| within ±2 ms, against the 95th percentile over the 50 ms around, in the worst channel. Measured: steady tone 0.03 dB, noise 2.7 dB (6.7 at most in 200 simulated trials), hard cut 36 dB, and the real splice 0.03 dB with its fade, 36 dB without. `kMaxStepDb` = 10 dB; about one random cut in ten reads under it (the two sides nearly meeting). -The next phase must know: -- **Two punch-ins that meet at J leave a 10 ms dip, not a crossfade.** Each fades against what the file held there, which is silence: [J − 5 ms, J) fades out and [J, J + 5 ms) fades in (read from `weightAt()`, not measured). `stepAt()` reads it as no step. -- The notes merge adds a new note only if its onset is in W (`Analyser.cpp`, "Notes go by their onset"). A second run's note that begins before J − 0.25 s is not added, and the dip may split the note at J. Either way, item 10's "one note across the join" may fail on today's code. C3 should measure it, not assume it. -Left open: every threshold untuned; nothing calls `TakeDiff` yet. - -### Lead — 2026-09-26, after C0 -- C1 is split into C1a (framework, items 1 and 2) and C1b (observer, items 7, 12, 13, 14); spec §7 says so. -- DevChecks runs as stages driven by the runner and a timer, not a nested event loop; scratch folders stay with the open session and the next run removes the old ones. Spec §5 and §8 changed to match. -- C0's warning about item 10 (a 10 ms dip at two punch-ins' join; notes merged by onset) stands for C3. - -### Phase C1a — 2026-09-26 -Built: `meson.build`: `dev_checks` = build type not starting with `release`; then `-DTONY_DEV_CHECKS` in `general_defines` and moc, `main/dev/DevChecks.cpp`, `TestDevChecks.h`. Runner: `Plan::ranges` (`punchInsOf()`), `keepSession`, `roundTrip` (window's `m_audioCheckRoundTrip`, read only in the `recordingStarted()` lambda), `abandon()`, public `analysing()` and `readTakeFile()`, no save question for a check's own session. `TakeLatency::startGap/startGapMeasured`. `MainWindow`: Record's action calls `recordPressed()`, which ignores presses while `audioCheckRunning()`; Record greyed then; `m_devChecks` (friend). `main/dev/DevChecks.{h,cpp}`: stages, `CheckResult`, `DevReport`, items 1 `latency_on_this_machine` and 2 `several_phrases_in_one_take`, `DevChecks.txt`, `nextScratchFolder()`. Dialog: checkbox, dev run, `setDevChecks()`, `setDevOptions()`. Tests: 6 in `TestAudioCheck` (~27 s), `TestDevChecks` 7 (~66 s). -Choices / deviations: -- Stage 1 ranges [6.3, 10.2] and [16.8, 21.2] s (sweeps 7.2/9.1, 17.7/20.1; 50 ms spare). Free: before 4.3 s (P = 1 s judging 3.1 s), 10.2–16.8 and 21.2–25 s, the held tones. A re-record starting at 17.9–19.2 s has its lead-in over B's first sweep and still judges its second. -- A stage that fails or times out ends the run: `failure` names it ("Stage 2 of 2, "Save and reopen", did not finish within 60 s."), and every check whose data is missing is Skipped with that text. Checks are worked out at the end: item 2 needs stage 1 only, item 1 both. -- Record goes through `recordPressed()`, not a guard in `record()`: the runner and `pollTakeProgress()` call `record()` during the check's take too, and it cannot tell them from a press. -- After the reopen pitch and notes are restored, not analysed: compared by value with values rounded as `Event::toXml()` writes them; offsets to the frame. Save: `saveSessionToPath()` (a dialog only on failure). -- A run's own round trip counts as `TakeLatency::measured`. The dev reference goes to `referenceDirectory()`, so a cancelled dev run leaves a session the next check replaces without asking. -- `TestAudioCheck`'s fixture deletes each window's `DevChecks`: in a dev build its B4 dialog tests would carry on into them (the checkbox is on) and write `DevChecks.txt` into the log directory. -The next phase must know: -- **B3 bug fixed:** `getLocalFilename()` is the decoded cache copy (spec §11), so `nextReferencePath()` never kept the open reference: always `-1`, written over while open. `mainModelFile()` now. -- On the fake with its true round trip every sweep lands at 0 frames; the fake's reported pair is 2.8 ms short, so a run ignoring its round trip fails item 1. -- `latencyInUse()`'s reported pair differs before any file is open: store test fingerprints after opening one. -- The watchdog answers "Session modified" with No; `QStandardPaths::setTestModeEnabled()` keeps references out of the test app's data directory. -Left open: the app suite now takes about 7m40s; the dev checks add about 66 s, over spec §6's minute. - -### Merge of default — 2026-09-26 -Merged `origin/default` at `92b8b5f` (15 commits); svgui at its new pin `a34646a`. -Conflicts: -- `TestRecordWorkflow.h`: `TestMainWindow` lives in default's `TestMainWindow.h` now. This branch's 17 accessors were merged into it three-way against the base's class (no overlap with default's `setUseRealDevice()` and the rest), and `dev/DevChecks.h` went with them, under `#ifdef TONY_DEV_CHECKS`. -- `TestSingingAnalysis.h`: both `waitForRange()` fixes are the same code; default's comment kept. -- `MainWindow.h` (includes), `meson.build`, `tony-core-test.cpp`, `tony-app-test.cpp`: both sides kept. `TestUiChecks` runs right after `TestRecordWorkflow`, as on default, then `TestAudioCheck` and `TestDevChecks`. -Beyond the conflicts: -- `TestAudioCheck.h` and `TestDevChecks.h` include `TestMainWindow.h` and what they use, not `TestRecordWorkflow.h`. -- `tony-app-test.cpp` draws text without sub-pixel anti-aliasing. Ubuntu's fontconfig asks for it (`10-sub-pixel-rgb.conf`) and Qt 6.4 follows it: the orange fringes of the waveform scale's labels at the pane's left edge were taken for the first live dot, and `TestUiChecks::live_dots_under_the_cursor` failed. Default ran on conda-forge's Qt 6.11, whose text was grey. -- `test-tony-device`'s moc needs no `dev_moc_args`: `TestMainWindow` has no `Q_OBJECT`, and `TestRealDevice.h` no `#ifdef`. -Checked: no string connect naming `ModelId` or `sv_frame_t` is left, and all of `e2cf7c0` survived. `liftPlaySelectionForTake()` (default) is called in the `recordingStarted()` lambda after the round trip, so a check's takes lift the constraint too. `closeSession()` tells the runner and the dev checks first, then restores the constraint. -For D: -- `testing.md` still calls `TestMainWindow` shared by three suites, and lists neither `TestAudioCheck` nor `TestDevChecks`. -- `building.md`'s "Qt 6.11, not Ubuntu's 6.4" is no longer needed for the connects. -- `README.md` says the manual checklist is "none of it tried yet"; `open-points.md` says the device check has been run in the cloud. -Seen, not fixed: every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()" from `svapp/audio/AudioCallbackRecordTarget.cpp:291`. The signal does not exist, so the warning is old and appears on any Qt. -App suite: 549 s, of which `TestUiChecks` takes 82 s. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index e92c3490..e3e7a236 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -87,7 +87,15 @@ plan. ## 4. What the loopback run settles, item by item -The numbers are those of `docs/manual-checklist.md`. +*Since the merge of `default` (2026-09-26):* `default` automated much of the checklist on +its own. `TestUiChecks` covers the screen, keyboard and dialog items on the fake device, +the smoke items below among them. `test-tony-device`, run by hand, covers the device items +(1, 2, 4, 5, 6, 9). The user decided that the dev run takes the device items over and +`test-tony-device` is retired; the smoke group and items 15 and 16 are dropped from the dev +run. The table keeps the old numbering, which `default` has since rewritten; phase D +brings the two together. + +The numbers are those of `docs/manual-checklist.md` before that merge. - **Automated:** the dev checks pass or fail it. - **Measured:** the dev checks report numbers; a person still judges how it looks or @@ -231,17 +239,16 @@ goes in `tony_core`. Calibration first. The dev checks then use the new figure for the run only; your stored setting changes only through Use this latency. -| Step | What it does | Items | -| --- | --- | --- | -| 1 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 5, 8 | -| 2 | Re-record over an earlier punch-in, through its lead-in | 4, 7, 12, 14 | -| 3 | Punch-in at P = 1 s | 13 | -| 4 | Two adjacent punch-ins meeting inside a held tone | 10 | -| 5 | Constrain Playback to Selection on for one punch-in, then off | 16 | -| 6 | Play from P | 15 | -| 7 | Save to the temp `.ton`, reopen, analyse again | 1 | -| 8 | Long reference, punch-in near the end | 9 | -| 9 | Smoke group, optional | 19–25, 27, 28 | +| Step | What it does | Items | Phase | +| --- | --- | --- | --- | +| 1 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 4, 5, 8 | C1a, C1b | +| 2 | Re-record over an earlier punch-in, through its lead-in | 7, 12, 14 | C1c | +| 3 | Punch-in at P = 1 s with a 3 s pre-roll | 13 | C1c | +| 4 | Two adjacent punch-ins meeting inside a held tone | 10 | C2 | +| 5 | Save to the scratch `.ton` and reopen | 1 | C1a | +| 6 | Long reference, two punch-ins far apart | 9 (and 1, 2) | C2 | + +Items 15 and 16 and the smoke group are no longer in the dev run (§4). ## 6. Tests @@ -303,15 +310,22 @@ marked "Done" when it is committed. driver's figure is, whether the offset holds across stream restarts on MME, and whether your device's rate hits the takes. Work goes on meanwhile: only the thresholds and the restart-jitter remedy wait on those numbers. + *First run, 2026-09-26* (MME, wired mic and headphones): round trip 301 and 295 ms + with one earcup to the mic, verdict Unsteady both times (5–15 ms); Scattered with the + mic between both cups. The detailed figures are awaited. 3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). 4. **Dev-check framework:** - **C0** `TakeDiff`, pure. Done. - **C1a** Build flag, `DevChecks`, report, friend access, the dialog's dev run. Items 1 and 2. Done. - - **C1b** `TakeObserver`. Items 7, 12, 13, 14. -5. **C2** Observer group: items 3, 4, 5, 8, 15, 16. -6. **C3** Join and long-song group: items 9 and 10. -7. **C4** Smoke group. + - *After the merge of `default`:* `TestDevChecks` moved into an executable of its + own, `test-tony-dev`, run when a change touches what the checks drive (the user's + decision). The phases below were cut again (§4). + - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). + - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. +5. **C2** Long song and joins: items 9 and 10. +6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. +7. **Release build** by the lead (§8, "Release builds must stay clean"). 8. **D** Docs, from the code and the phase log: - `manual-checklist.md`: an automated item keeps its text and gets "*automated: dev check ``*"; a measured item keeps only the question for a person. The @@ -371,6 +385,8 @@ could convert. The button then shows the fix working on each device. | Form of a dev check | A function returning a plain `CheckResult`, not a QtTest function | | Checkpoint after B3 | The user runs it on Windows when they can; C0 onwards does not wait | | Commits | The lead commits each phase after review and pushes `feat/calibrateaudiotests` | +| `test-tony-device` (from `default`) | Its checks move into the dev run, and it is retired (C3) | +| Where the dev checks' tests run | `test-tony-dev`, a third executable in dev builds, run when a change touches the take path, the audio check or the dev checks | ## 11. Facts checked in the code From 04cd280fbb6e9c43bc7403a3992dc6816cba4638 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:36:20 +0000 Subject: [PATCH 146/275] docs: calibrate audio work orders name the Windows macro trap near and far are empty macros under Windows headers, so a variable of that name builds on Linux and breaks the user's MinGW build. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index a1a7c1dc..2921259d 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -94,6 +94,10 @@ MinGW setup. with moc's `sv::` names: the connection fails at run time with "No such slot". The user's Qt happens to match them, so a string connect can pass there and fail here. Use member-pointer `connect` only, and grep test output for "No such slot". +- **Windows traps** (the user builds with MinGW; Linux will not catch these): + - Windows headers define `near` and `far` as empty macros, so a variable named `near` + compiles here and breaks the user's build. Avoid those two names, and other + Windows macro names such as `min`, `max`, `ERROR`, `IN`, `OUT`. **Docs: almost none.** The documentation phase (D) brings `docs/` up to date. You write only: From 1b9c87971c5e33a3b7534caf34410448fcc13591 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 05:42:42 +0000 Subject: [PATCH 147/275] test: the dev checks' suite in an executable of its own, test-tony-dev Each TestDevChecks test records several takes in real time, and more checks are coming; test-tony-app already takes about eight minutes. test-tony-dev is built only where the checks are, registered with meson test, and run as well when a change touches the take path, the audio check or the dev checks, as AGENTS.md now says. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- AGENTS.md | 6 +++- docs/building.md | 4 +-- docs/testing.md | 10 ++++-- main/test/tony-app-test.cpp | 11 ------ main/test/tony-dev-test.cpp | 69 +++++++++++++++++++++++++++++++++++++ meson.build | 47 +++++++++++++++++++++---- 6 files changed, 124 insertions(+), 23 deletions(-) create mode 100644 main/test/tony-dev-test.cpp diff --git a/AGENTS.md b/AGENTS.md index 015351cd..365d0102 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,7 +27,7 @@ From **Git Bash** (the usual agent shell). `build.bat` does not run from sh. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-device.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe test-tony-device.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log ``` @@ -51,6 +51,10 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - A test name on the command line goes to **every** suite in the executable; the suites that lack it fail, so the exit status is only meaningful for a run with no names. - Run named tests while working; run **both whole suites** before calling anything done. +- `test-tony-dev.exe` (development builds only) holds the development checks' suite, + about a minute and more of real-time takes. Run it as well, whole, when a change touches + the take path (`record()`, Stop, latency, pre-roll), `AudioCheckRunner`, + `CalibrateAudioDialog` or `main/dev/`; "both whole suites" then means all three. - From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. - Give the app suite a tool timeout of 10 minutes. diff --git a/docs/building.md b/docs/building.md index b0e0de1e..15d082e8 100644 --- a/docs/building.md +++ b/docs/building.md @@ -11,7 +11,7 @@ described here. .\build.bat build Tony.exe .\build.bat run build, then launch .\build.bat launch launch without building -.\build.bat test meson test: tony-core, tony-app and four svcore suites +.\build.bat test meson test: tony-core, tony-app, tony-dev and four svcore suites .\build.bat clean wipe build_mingw and reconfigure (only for a broken build directory) ``` @@ -24,7 +24,7 @@ From PowerShell its output is safe to capture: `.\build.bat *> tmp\build.log`. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-device.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe test-tony-device.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log tail -20 tmp/build.log ``` diff --git a/docs/testing.md b/docs/testing.md index e21e9d06..21fb150b 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,17 +1,21 @@ # Testing QtTest suites in `main/test/`, in two executables that mirror the two libraries -(see [architecture.md](architecture.md)). The commands are in [AGENTS.md](../AGENTS.md). +(see [architecture.md](architecture.md)), plus one for the development checks and one for +the real device. The commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | | `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | +| `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about a minute and growing: each test records several takes in real time | | `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | -`meson test` / `build.bat test` runs the first two plus four svcore suites. `test-tony-device` +`meson test` / `build.bat test` runs the first three plus four svcore suites. `test-tony-device` is built with them and never run by `meson test`: it needs a microphone that hears the -speakers. +speakers. `test-tony-dev` is apart from `test-tony-app` so that the everyday runs stay +shorter: run it when a change touches what the development checks drive (see +[AGENTS.md](../AGENTS.md)). - The `tony-app` meson test has `timeout: 900`; the suite took about 277 s unloaded when that was set. Every workflow test adds real time, so if the suite comes near it, raise it diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index e3dd3344..fde84c7d 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -17,9 +17,6 @@ #include "TestRecordWorkflow.h" #include "TestUiChecks.h" #include "TestAudioCheck.h" -#ifdef TONY_DEV_CHECKS -#include "TestDevChecks.h" -#endif #include "RunSuite.h" @@ -104,14 +101,6 @@ int main(int argc, char *argv[]) else ++bad; } -#ifdef TONY_DEV_CHECKS - { - TestDevChecks t; - if (runSuite(&t, argc, argv)) ++good; - else ++bad; - } -#endif - (void)good; if (bad > 0) { diff --git a/main/test/tony-dev-test.cpp b/main/test/tony-dev-test.cpp new file mode 100644 index 00000000..033f81b3 --- /dev/null +++ b/main/test/tony-dev-test.cpp @@ -0,0 +1,69 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +// The development checks' own suite, built only where they are +// (TONY_DEV_CHECKS). Kept out of test-tony-app because each of its +// tests records several takes in real time: run it when a change +// touches what the checks drive (see AGENTS.md) + +#include "TestDevChecks.h" + +#include "RunSuite.h" + +#include "system/Init.h" + +#include +#include +#include +#include + +#include + +using namespace std; +using namespace sv; + +int main(int argc, char *argv[]) +{ + svSystemSpecificInitialisation(); + + // Offscreen, as test-tony-app + if (qEnvironmentVariableIsEmpty("QT_QPA_PLATFORM")) { + qputenv("QT_QPA_PLATFORM", "offscreen"); + } + + // Names distinct from the application's, so that nothing here reads + // or writes the user's real Tony settings + QApplication app(argc, argv); + app.setOrganizationName("tony-tests"); + app.setApplicationName("test-tony-dev"); + + // Text in shades of grey, as in test-tony-app + QFont font = QApplication::font(); + font.setStyleStrategy(QFont::NoSubpixelAntialias); + QApplication::setFont(font); + + // The real pYIN plugin, from next to this executable only + qputenv("VAMP_PATH", + QDir::toNativeSeparators(app.applicationDirPath()).toLocal8Bit()); + + TestDevChecks t; + bool ok = runSuite(&t, argc, argv); + + if (!ok) { + SVCERR << "\n********* the dev checks' suite failed!\n" << endl; + return 1; + } else { + SVCERR << "All tests passed" << endl; + return 0; + } +} diff --git a/meson.build b/meson.build index 37b3b0e2..102d948b 100644 --- a/meson.build +++ b/meson.build @@ -1432,12 +1432,6 @@ tony_app_test_moc_headers = [ 'main/test/TestAudioCheck.h', ] -if dev_checks - tony_app_test_moc_headers += [ - 'main/test/TestDevChecks.h', - ] -endif - tony_app_test_moc_files = qt.preprocess( moc_headers: tony_app_test_moc_headers, moc_extra_arguments: dev_moc_args, @@ -1504,6 +1498,47 @@ test('tony-app', tony_app_test_exe, timeout: 900, env: [ 'QT_QPA_PLATFORM=offscreen' ]) +# The development checks' suite, only where the checks are built. Its +# own executable: every test records takes in real time, and it is run +# when a change touches what the checks drive (AGENTS.md) +if dev_checks + tony_dev_test_moc_files = qt.preprocess( + moc_headers: [ + 'main/test/TestDevChecks.h', + ], + moc_extra_arguments: dev_moc_args, + ) + + tony_dev_test_exe = executable( + 'test-tony-dev', + tony_dev_test_moc_files, + 'main/test/tony-dev-test.cpp', + dependencies: [ + tony_app_dep, + tony_core_dep, + svcore_dep, + qt_dep, + feature_dependencies, + os_dep, + dl_dep, + ], + cpp_args: [ + feature_defines, + general_defines, + ], + link_args: [ + feature_additional_libs, + general_link_args, + ], + win_subsystem: 'console' + ) + + test('tony-dev', tony_dev_test_exe, + depends: pyin_plugin, + timeout: 900, + env: [ 'QT_QPA_PLATFORM=offscreen' ]) +endif + test('svcore-base', svcore_base_test_exe) test('svcore-system', svcore_system_test_exe) test('svcore-data-model', svcore_data_model_test_exe) From 276036e2201c00dbb9b7a296e3ec86fd4693b20e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 06:34:24 +0000 Subject: [PATCH 148/275] feat: dev checks for live dots, the take in the speakers, and the mic input A TakeObserver watches each take of the dev run every 20 ms, only looking: the cursor, the output levels (the one reader of them while recording, the view manager reading the input's), each live dot as it first appears, the status text and any dialog. From stage 1's punch-ins the dev run now judges item 3 (dots drawn, and on the reference's sounds; the dot-to-cursor offset reported), item 4 (no second arrival, and the output silent in the reference's gaps, from the frames the device was handed) and item 5 (which input carries the microphone, from each raw recording). The report header names the audio drivers and the latencies the device reports, as test-tony-device logs them. FakeAudioIO gains an echo tap and level reporting, both off unless asked for. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 26 ++ docs/calibrate-audio.md | 11 +- main/MainWindow.h | 3 + main/dev/DevChecks.cpp | 568 +++++++++++++++++++++++++++- main/dev/DevChecks.h | 56 ++- main/dev/TakeObserver.cpp | 210 ++++++++++ main/dev/TakeObserver.h | 193 ++++++++++ main/test/FakeAudioIO.h | 32 ++ main/test/TestDevChecks.h | 117 +++++- meson.build | 2 + 10 files changed, 1205 insertions(+), 13 deletions(-) create mode 100644 main/dev/TakeObserver.cpp create mode 100644 main/dev/TakeObserver.h diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 2921259d..1a877049 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -326,3 +326,29 @@ Template: Choices / deviations: ... The next phase must know: ... Left open: ... + +### Phase C1b — 2026-09-26 +Built: `main/dev/TakeObserver` (a sample every 20 ms: cursor frame, frames received just +before and after the output-level read, output levels while recording only, input levels +from `monitoringLevelsChanged`, status text, modal; each dot on first sight with the cursor +then; the raw recording's path; when it stopped; Play Singing Audio before and after). +`DevChecks` starts one on the runner's `Recording` progress, keeps it (`Watched`) when the +next punch-in records or the runner finishes; items 3 `live_dots`, 4 +`nothing_of_the_take_in_the_speakers`, 5 `mic_on_input_2` from stage 1; report header +with drivers and reported latencies. `FakeAudioIO`: `echoDelay`/`echoGain`, opt-in +`reportLevels`. `TestDevChecks`: 5 checks; `dev_checks_echo_and_the_mic_on_input_2`. +Choices / deviations: output is placed from frames received and the measured start gap +(input, then output, per callback), not from time or the output latency, which the levels +precede. Margin one block either side; the block bounded by the most frames received +between two looks not held up (35 ms on the fake). Item 3: dots from a sound's start +−1 hop to its end + half the tracker window +1 hop (±1 hop failed the clean fake: dots run +440 frames past a tone); sweep ends make 2–3 dots at 860–980 Hz, counted apart, pitch not +judged. Cursor = the ViewManager's frame (S + recorded), read only while recording: dots +trail it by the round trip + about 40 ms (+322 ms at 281 ms on the fake). Item 5 "not +applicable" is Measured; on input 2 alone it passes on more than 10 dots per punch-in. +Echo and input 2 share one run. +The next phase must know: an observer runs for every punch-in of a runner run DevChecks +starts (`m_watched`, cleared per stage); stage 1's kept in `m_freshWatched`. +Left open: at −20 ms item 3 passes or fails with the hop phase (the test allows both). +The gap check was seen failing on a non-silent reference, not a take played back out: +stage 1 has no take audio where it plays (C1c's re-record has). Dot spread 40–230 ms. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index e3e7a236..1110bdd0 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -108,7 +108,7 @@ The numbers are those of `docs/manual-checklist.md` before that merge. | --- | --- | --- | --- | | 1 | Latency on this machine, also after save and reopen | **Automated** | Every sweep of every punch-in lands within ±2 ms (§6). Save the test session to a temp `.ton`, reopen it, analyse again: unchanged. | | 2 | Several phrases in one take | **Automated** | Punch-ins at four positions of one take, each placed right. Each measures its own start gap. | -| 3 | Live dots | **Measured** | *Automated:* dots sit on the reference's tones on the timeline (±1 hop); they stay after Stop until the pitch track arrives, then go; the status bar stops changing; the take's own pitch and notes are hidden during the take and back after. *Reported:* how far behind the cursor a dot appears, in ms. *Eyes:* does it look right. | +| 3 | Live dots | **Measured** | *Automated:* dots sit on the reference's tones on the timeline (±1 hop; *found in C1b:* a dot trails the sound it heard by up to half the tracker's window, so from a tone's start to half a window past its end, ±1 hop); they stay after Stop until the pitch track arrives, then go; the status bar stops changing; the take's own pitch and notes are hidden during the take and back after. *Reported:* how far behind the cursor a dot appears, in ms. *Eyes:* does it look right. | | 4 | Nothing of the take in the speakers | **Automated** | Re-record over earlier material. Tony's output peak (`getOutputLevels()`) is exactly zero in the reference's silent gaps, so no take audio and no synth. The mic hears no second arrival of each sweep; one would mean the input is monitored somewhere (Windows "Listen to this device", or an interface's direct monitor). Play Singing Audio keeps its state. | | 5 | Mic on input 2 of a stereo interface | **Measured** | Per-channel input peaks show which input the mic is on. If it is input 2, dots appearing is the check; otherwise "not applicable here". | | 6 | No input device / device in use | **Manual** | Needs the device gone or busy. | @@ -321,7 +321,7 @@ marked "Done" when it is committed. - *After the merge of `default`:* `TestDevChecks` moved into an executable of its own, `test-tony-dev`, run when a change touches what the checks drive (the user's decision). The phases below were cut again (§4). - - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). + - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. 5. **C2** Long song and joins: items 9 and 10. 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. @@ -456,7 +456,12 @@ Checked on 2026-09-25, so that phases do not re-derive them. - Coverage is `m_takes->getCoverage().getRanges()`. - The take is analysed when `analysed(analyser2())` holds. - **Levels.** `getOutputLevels()` and `getInputLevels()`, on the play source and record - target, return per-channel peaks since the last call. + target, return per-channel peaks since the last call. *Found in C1b:* while recording, + lead-in included, `ViewManager::checkPlayStatus()` reads the input levels only, so the + observer is then the output levels' one reader. They are of what is handed to the + device, before its output latency. `FakeAudioIO` reports levels only with + `reportLevels`. The play source's `getTargetBlockSize()` is always its default 1024: + bqaudioio's `ResamplerWrapper` does not pass the device's block on. - **Fake device.** `FakeAudioIO::Config::loopback` adds the output to the input `inputDelay` frames late; `TestAudioCheck` uses it. The reported latencies are independent of the real delay. `TestMainWindow::createAudioIO()` installs the fake. diff --git a/main/MainWindow.h b/main/MainWindow.h index 67261c09..c9e450a3 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -63,6 +63,9 @@ class MainWindow : public sv::MainWindowBase // The development checks save and reopen the session, and read the // take's pitch and notes; see DevChecks friend class DevChecks; + // What the development checks see of a take, only looking; see + // TakeObserver + friend class TakeObserver; #endif public: diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index bcee21b3..7af008b2 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -17,13 +17,19 @@ #include "DevChecks.h" #include "../MainWindow.h" +#include "../RealtimePitchTracker.h" #include "../SingingTakes.h" +#include "audio/AudioCallbackPlaySource.h" #include "audio/AudioCallbackRecordTarget.h" +#include "data/fileio/FileSource.h" +#include "data/fileio/WavFileReader.h" #include "data/model/NoteModel.h" #include "data/model/SparseTimeValueModel.h" #include "layer/Layer.h" +#include + #include #include #include @@ -114,6 +120,63 @@ sameEvents(const EventVector &before, const EventVector &after, return false; } +double +median(vector values) +{ + if (values.empty()) return 0.0; + std::sort(values.begin(), values.end()); + const size_t n = values.size(); + return n % 2 ? values[n / 2] : 0.5 * (values[n / 2 - 1] + values[n / 2]); +} + +// A level, full scale 1, in dBFS; exact silence in words +QString +levelText(double level) +{ + if (level <= 0.0) return "silence"; + return QString("%1 dBFS").arg(20.0 * std::log10(level), 0, 'f', 1); +} + +// The peak of each channel of an audio file, full scale 1: the file as +// it is, not normalised as the session's models are read +vector +channelPeaks(QString path, QString &error) +{ + error = ""; + if (path == "") { + error = DevChecks::tr("its raw recording was not seen"); + return {}; + } + FileSource source(path); + WavFileReader reader(source); + const int channels = reader.getChannelCount(); + if (!reader.isOK() || channels < 1) { + error = DevChecks::tr("its raw recording \"%1\" could not be read: %2") + .arg(path).arg(reader.getError()); + return {}; + } + const floatvec_t data = reader.getInterleavedFrames + (0, reader.getFrameCount()); + vector peaks(channels, 0.f); + for (size_t i = 0; i < data.size(); ++i) { + float &p = peaks[i % size_t(channels)]; + p = std::max(p, std::fabs(data[i])); + } + return peaks; +} + +// The inputs, counting from 1, in words: "input 2", "inputs 1 and 2" +QString +inputsText(const vector &inputs) +{ + if (inputs.empty()) return DevChecks::tr("no input"); + QStringList names; + for (int i : inputs) names << QString::number(i + 1); + if (names.size() == 1) return DevChecks::tr("input %1").arg(names[0]); + const QString last = names.takeLast(); + return DevChecks::tr("inputs %1 and %2").arg(names.join(", ")).arg(last); +} + } // namespace QString @@ -149,6 +212,8 @@ DevChecks::DevChecks(MainWindow *window, AudioCheckRunner *runner) : m_runnerRunning(false), m_reopening(false), m_saved(false), + m_observer(new TakeObserver(window, this)), + m_observedPunchIn(0), m_haveFresh(false), m_reopened(false) { @@ -159,6 +224,11 @@ DevChecks::DevChecks(MainWindow *window, AudioCheckRunner *runner) : // is looked at on the next poll connect(runner, &AudioCheckRunner::finished, this, &DevChecks::runnerFinished); + + // Direct as well: a take is watched from the poll of the runner's + // that started it + connect(runner, &AudioCheckRunner::progress, + this, &DevChecks::runnerProgress); } DevChecks::~DevChecks() @@ -169,6 +239,7 @@ DevChecks::~DevChecks() // a take and start its analysis in the middle of the window's // teardown. The take in progress is left to the window m_timer->stop(); + m_observer->stop(); if (m_stage >= 0) { cerr << "DevChecks: the window is going; the run ends" << endl; if (m_runnerRunning && m_runner) m_runner->abandon(); @@ -247,9 +318,12 @@ DevChecks::start(const Options &options) } m_layout = LatencyCheck::devLayout(); + m_observedPunchIn = 0; + m_watched.clear(); m_haveFresh = false; m_fresh = AudioCheckResult(); m_coverageAfterFresh = Coverage(); + m_freshWatched.clear(); m_pitchBefore.clear(); m_notesBefore.clear(); m_pitchAfter.clear(); @@ -354,10 +428,51 @@ DevChecks::runnerFinished(const AudioCheckResult &result) { // A run the dialog started, or anyone else, is not ours if (!m_runnerRunning) return; + if (m_observer->isObserving()) finishObservation(); m_runnerRunning = false; m_runnerResult = result; } +void +DevChecks::runnerProgress(const AudioCheckRunner::Progress &state) +{ + if (!m_runnerRunning) return; + + // Reported once the take has started, from the same poll of the + // runner's. A punch-in's analysis is done when the next one starts + // recording, or when the runner finishes + using Step = AudioCheckRunner::Step; + const bool sameTake = + (state.step == Step::Recording || state.step == Step::AnalysingTake) && + state.punchIn == m_observedPunchIn; + if (m_observer->isObserving() && !sameTake) finishObservation(); + + if (state.step == Step::Recording && !m_observer->isObserving()) { + m_observedPunchIn = state.punchIn; + m_observer->start(); + } +} + +void +DevChecks::finishObservation() +{ + m_observer->stop(); + Watched w; + w.punchIn = m_observedPunchIn - 1; + w.seen = m_observer->observation(); + m_watched.push_back(w); + m_observedPunchIn = 0; +} + +const DevChecks::Watched * +DevChecks::freshWatched(int i) const +{ + for (const Watched &w : m_freshWatched) { + if (w.punchIn == i) return &w; + } + return nullptr; +} + void DevChecks::beginFreshPunchIns() { @@ -366,6 +481,8 @@ DevChecks::beginFreshPunchIns() plan.ranges = freshPunchIns(); plan.roundTrip = m_options.roundTrip; + m_watched.clear(); + m_observedPunchIn = 0; m_runnerResult = AudioCheckResult(); m_runnerRunning = m_runner && m_runner->start(plan); if (!m_runnerRunning) { @@ -387,6 +504,13 @@ DevChecks::freshPunchInsDone() m_fresh = m_runnerResult; m_haveFresh = true; m_coverageAfterFresh = m_window->m_takes->getCoverage(); + + // Each channel of what the device delivered, before anything was + // mixed or spliced, to see which input the mic is on + m_freshWatched = m_watched; + for (Watched &w : m_freshWatched) { + w.channelPeaks = channelPeaks(w.seen.recordingPath, w.channelError); + } return true; } @@ -482,6 +606,8 @@ DevChecks::end(QString failure) if (!isRunning()) return; m_timer->stop(); + m_observer->stop(); + m_observedPunchIn = 0; m_stage = -1; m_begun = false; @@ -512,7 +638,9 @@ DevChecks::end(QString failure) vector DevChecks::evaluate(QString reason) const { - return { latencyCheck(reason), phrasesCheck(reason) }; + return { latencyCheck(reason), phrasesCheck(reason), + liveDotsCheck(reason), speakersCheck(reason), + micChannelCheck(reason) }; } CheckResult @@ -701,6 +829,416 @@ DevChecks::phrasesCheck(QString reason) const return c; } +CheckResult +DevChecks::liveDotsCheck(QString reason) const +{ + CheckResult c; + c.item = 3; + c.name = "live_dots"; + + if (!m_haveFresh) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Every dot where the take's pitch track will have the sound it + // came from: on one of the reference's tones, and at its pitch. The + // loopback records the reference, so that is where the singing is. + // + // A dot is drawn at the middle of the tracker's window, but YIN + // hears mostly the window's first half, so a dot comes up to half a + // window after the sound that made it, and not before it: on the + // loopback fake a tone's dots begin about 540 frames into it and + // end up to 440 past it. So a sound's dots lie from its start to + // half a window past its end, give or take kDotHops. The end of + // each sweep, near 8 kHz, makes a dot or two at a subharmonic just + // under the tracker's 1 kHz ceiling: dots of the sweeps are counted + // apart, and their pitch means nothing + const sv_samplerate_t rate = m_fresh.referenceRate; + const LatencyCheck::Layout &layout = m_layout; + const double before = + double(kDotHops * RealtimePitchTracker::kHopSize) / rate; + const double after = before + + double(RealtimePitchTracker::kWindowSize / 2) / rate; + const double sweepSeconds = + double(LatencyCheck::sweep(layout.rate).size()) / layout.rate; + const LatencyCheck::TakeSummary &s = m_fresh.summary; + + QStringList problems; + vector behind; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const LatencyCheck::PunchInResult &p = s.punchIns[i]; + const Watched *w = freshWatched(i); + if (!w) { + problems << tr("punch-in %1 was not watched").arg(i + 1); + continue; + } + + const vector &dots = w->seen.dots; + int onSweeps = 0; + int off = 0; + QString firstOff; + for (const TakeObserver::Dot &d : dots) { + // Behind the cursor: the cursor had got this far past the + // dot's place when the dot was first there + behind.push_back(double(d.playbackFrame - d.frame) / rate); + + const double t = double(d.frame) / rate; + bool on = false; + bool sweep = false; + QString why = tr("on no tone"); + for (const LatencyCheck::Event &e : layout.events) { + const double sweepAt = double(e.sweepStart) / layout.rate; + if (t >= sweepAt - before && + t <= sweepAt + sweepSeconds + after) { + sweep = true; + break; + } + const double from = double(e.toneStart) / layout.rate; + const double to = + double(e.toneStart + e.toneLength) / layout.rate; + if (t < from - before || t > to + after) continue; + const double cents = + 1200.0 * std::log2(double(d.hz) / e.toneHz); + on = (std::fabs(cents) <= kDotCents); + why = tr("%1 cents from the tone of %2 Hz") + .arg(cents, 0, 'f', 0).arg(e.toneHz); + break; + } + if (sweep) ++onSweeps; + if (on || sweep) continue; + if (off++ == 0) { + firstOff = tr("the first at %1 s, %2 Hz, %3") + .arg(t, 0, 'f', 3).arg(d.hz, 0, 'f', 1).arg(why); + } + } + + if (int(dots.size()) <= kMinDots) { + problems << tr("punch-in %1 drew %2 live dots").arg(i + 1) + .arg(dots.size()); + } + if (off > 0) { + problems << tr("punch-in %1: %2 of its %3 dots are off the " + "reference's sounds, %4") + .arg(i + 1).arg(off).arg(dots.size()).arg(firstOff); + } + c.numbers.push_back + ({ tr("dots, punch-in %1 (%2 to %3 s)").arg(i + 1) + .arg(secondsText(p.range.start)) + .arg(secondsText(p.range.end)), + tr("%1: %2 on the tones, %3 on the sweeps, %4 elsewhere") + .arg(dots.size()).arg(int(dots.size()) - onSweeps - off) + .arg(onSweeps).arg(off) }); + } + if (s.punchIns.empty()) problems << tr("no punch-in was judged"); + + // Checklist item 8, and the "cursor versus dots" risk: the cursor + // runs with what has been recorded, the dots with the round trip + if (!behind.empty()) { + const auto range = std::minmax_element(behind.begin(), behind.end()); + c.numbers.push_back({ tr("dots behind the cursor, median"), + signedMs(median(behind)) }); + c.numbers.push_back({ tr("dots behind the cursor, spread"), + unsignedMs(*range.second - *range.first) }); + } + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Every punch-in drew more than %1 live dots, each on " + "one of the reference's sounds (from %2 before it " + "to %3 after it), and those on its tones within %4 " + "cents of their pitch.") + .arg(kMinDots).arg(unsignedMs(before)).arg(unsignedMs(after)) + .arg(kDotCents); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::speakersCheck(QString reason) const +{ + CheckResult c; + c.item = 4; + c.name = "nothing_of_the_take_in_the_speakers"; + + if (!m_haveFresh) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + QStringList problems; + + // The input played back out, by Tony or by the system, reaches the + // mic again a little later: every sweep arrives twice + const LatencyCheck::Echo &echo = m_fresh.summary.echo; + if (echo.heard) { + problems << tr("every sweep arrived a second time, %1 later at " + "%2 dB: the input is being played back out, by " + "Tony or by the system (\"Listen to this device\")") + .arg(unsignedMs(echo.delaySeconds)) + .arg(echo.levelDb, 0, 'f', 1); + } + c.numbers.push_back + ({ tr("second arrival"), + echo.heard ? tr("%1 after the sweep, %2 dB, in %3 sweeps") + .arg(unsignedMs(echo.delaySeconds)).arg(echo.levelDb, 0, 'f', 1) + .arg(echo.events) : tr("none heard") }); + + // What Tony played: exactly nothing where the reference is silent, + // so no take and no synth. The reference's sounds, in seconds + const LatencyCheck::Layout &layout = m_layout; + const double sweepSeconds = + double(LatencyCheck::sweep(layout.rate).size()) / layout.rate; + vector> sounds; + for (const LatencyCheck::Event &e : layout.events) { + const double sweepAt = double(e.sweepStart) / layout.rate; + sounds.push_back({ sweepAt, sweepAt + sweepSeconds }); + sounds.push_back({ double(e.toneStart) / layout.rate, + double(e.toneStart + e.toneLength) / layout.rate }); + } + auto silent = [&sounds](double from, double to) { + for (const auto &sound : sounds) { + if (from < sound.second && to > sound.first) return false; + } + return true; + }; + + const sv_samplerate_t rate = m_fresh.referenceRate; + const LatencyCheck::TakeSummary &s = m_fresh.summary; + double loudest = 0.0; + double loudestInGaps = 0.0; + double margin = 0.0; + int gapPolls = 0; + QString firstHeard; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const Watched *w = freshWatched(i); + if (!w) { + problems << tr("punch-in %1 was not watched").arg(i + 1); + continue; + } + const TakeObserver::Observation &o = w->seen; + + // Play Singing Audio: what it said before the take, it says + // after, and the take is heard or not as it says + if (!o.sawAfter) { + problems << tr("punch-in %1 was not seen after it stopped") + .arg(i + 1); + } else { + auto onOff = [](bool on) { return on ? tr("on") : tr("off"); }; + if (o.singingAudioAfter != o.singingAudioBefore) { + problems << tr("Play Singing Audio was %1 before punch-in " + "%2 and %3 after it") + .arg(onOff(o.singingAudioBefore)).arg(i + 1) + .arg(onOff(o.singingAudioAfter)); + } + if (o.takeAudibleAfter != o.singingAudioAfter) { + problems << tr("after punch-in %1 Play Singing Audio is %2, " + "but the take's audio is %3") + .arg(i + 1).arg(onOff(o.singingAudioAfter)) + .arg(o.takeAudibleAfter ? tr("heard") : tr("silent")); + } + c.numbers.push_back + ({ tr("Play Singing Audio, punch-in %1").arg(i + 1), + tr("%1 before, %2 after, the take %3") + .arg(onOff(o.singingAudioBefore)) + .arg(onOff(o.singingAudioAfter)) + .arg(o.takeAudibleAfter ? tr("heard") : tr("silent")) }); + } + + // An audio callback takes in a block of input, then hands out a + // block of output. The reference's first block went out from + // where playback started, after the start gap's input, so the + // frames received say which frames of the reference went out + const TakeLatency t = (i < int(m_fresh.takes.size()) ? + m_fresh.takes[i] : TakeLatency()); + if (!t.startGapMeasured || t.recordingRate <= 0) { + problems << tr("punch-in %1's start gap was not measured, so " + "what it played could not be placed").arg(i + 1); + continue; + } + const double start = double(o.playbackStart) / rate; + auto played = [&](sv_frame_t received) { + return start + double(received - t.startGap) / t.recordingRate; + }; + + // The levels read at a poll are of the blocks handed out since + // the read before. Those were taken in after the frames counted + // just before that read, but for the one block being handled + // then, which went out after it: one block early. And one late, + // for a resampler between the play source and a device at + // another rate. The output latency plays no part: the levels + // are of what was handed to the device, not of what was heard. + // How long a block is, nothing in the window says (the play + // source is never told the device's), and PortAudio may vary + // it; but each block's input is counted at once, so no block is + // longer than the most frames that came in between two looks. + // Not counting looks held up: the first ones of a take wait for + // the window to set it up, and would count several blocks + sv_frame_t blockFrames = 0; + sv_frame_t heldUp = 0; + const TakeObserver::Sample *previous = nullptr; + for (const TakeObserver::Sample &sample : o.samples) { + if (!sample.recording) continue; + blockFrames = std::max(blockFrames, sample.framesAfter - + sample.framesBefore); + if (previous) { + const sv_frame_t since = + sample.framesBefore - previous->framesAfter; + if (sample.ms - previous->ms <= 2 * TakeObserver::kPollMs) { + blockFrames = std::max(blockFrames, since); + } else { + heldUp = std::max(heldUp, since); + } + } + previous = &sample; + } + if (blockFrames <= 0) blockFrames = heldUp; + const double block = double(blockFrames) / t.recordingRate; + margin = std::max(margin, block); + for (size_t k = 1; k < o.samples.size(); ++k) { + const TakeObserver::Sample &a = o.samples[k - 1]; + const TakeObserver::Sample &b = o.samples[k]; + if (!a.outputRead || !b.outputRead) continue; + const double level = std::max(b.outputLeft, b.outputRight); + loudest = std::max(loudest, level); + const double from = played(a.framesBefore) - block; + const double to = played(b.framesAfter) + block; + if (from < start || !silent(from, to)) continue; + ++gapPolls; + if (level > loudestInGaps) loudestInGaps = level; + if (level > 0.0 && firstHeard == "") { + firstHeard = tr("%1 from %2 to %3 s, in punch-in %4") + .arg(levelText(level)).arg(from, 0, 'f', 3) + .arg(to, 0, 'f', 3).arg(i + 1); + } + } + } + if (s.punchIns.empty()) problems << tr("no punch-in was judged"); + + if (loudest <= 0.0) { + problems << tr("no output level was reported while the reference " + "played, so the silent gaps say nothing"); + } else if (gapPolls == 0) { + problems << tr("no look at the output fell wholly in one of the " + "reference's silent gaps"); + } + if (firstHeard != "") { + problems << tr("Tony played something where the reference is " + "silent: %1").arg(firstHeard); + } + c.numbers.push_back({ tr("largest output level in the silent gaps"), + tr("%1, over %2 looks").arg(levelText(loudestInGaps)) + .arg(gapPolls) }); + c.numbers.push_back({ tr("largest output level"), levelText(loudest) }); + c.numbers.push_back({ tr("margin either side of a look"), + unsignedMs(margin) }); + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("No sweep arrived twice, Tony played nothing where the " + "reference is silent, and Play Singing Audio was as " + "it had been after each take."); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::micChannelCheck(QString reason) const +{ + CheckResult c; + c.item = 5; + c.name = "mic_on_input_2"; + + if (!m_haveFresh) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Each channel the device delivered, as it delivered it + const LatencyCheck::TakeSummary &s = m_fresh.summary; + QStringList problems; + vector loudest; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const Watched *w = freshWatched(i); + if (!w) { + problems << tr("punch-in %1 was not watched").arg(i + 1); + continue; + } + if (w->channelError != "") { + problems << tr("punch-in %1: %2").arg(i + 1).arg(w->channelError); + continue; + } + QStringList peaks; + for (size_t ch = 0; ch < w->channelPeaks.size(); ++ch) { + if (loudest.size() <= ch) loudest.resize(ch + 1, 0.f); + loudest[ch] = std::max(loudest[ch], w->channelPeaks[ch]); + peaks << tr("input %1 %2").arg(ch + 1) + .arg(levelText(w->channelPeaks[ch])); + } + c.numbers.push_back({ tr("input peaks, punch-in %1").arg(i + 1), + peaks.join(", ") }); + } + + float top = 0.f; + for (float peak : loudest) top = std::max(top, peak); + vector carrying; + for (size_t ch = 0; ch < loudest.size(); ++ch) { + if (top > 0.f && loudest[ch] > 0.f && + 20.0 * std::log10(loudest[ch] / top) >= -kMicChannelDb) { + carrying.push_back(int(ch)); + } + } + c.numbers.push_back({ tr("the mic is on"), + tr("%1, peak %2").arg(inputsText(carrying)) + .arg(levelText(top)) }); + + if (problems.isEmpty() && carrying.empty()) { + problems << tr("nothing was recorded on any input"); + } + if (!problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + return c; + } + + if (carrying != vector{ 1 }) { + c.verdict = CheckResult::Verdict::Measured; + c.message = tr("Not applicable here: the mic is on %1, not on " + "input 2 alone.").arg(inputsText(carrying)); + return c; + } + + // On input 2 alone: the live tracker hears the mixdown, so the dots + // are drawn all the same + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const Watched *w = freshWatched(i); + const int dots = w ? int(w->seen.dots.size()) : 0; + if (dots <= kMinDots) { + problems << tr("punch-in %1 drew %2 live dots").arg(i + 1) + .arg(dots); + } + } + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("The mic is on input 2, and every punch-in drew more " + "than %1 live dots.").arg(kMinDots); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = tr("The mic is on input 2, and %1.") + .arg(problems.join("; ")); + } + return c; +} + QString DevChecks::writeReport(const DevReport &report) const { @@ -728,6 +1266,34 @@ DevChecks::writeReport(const DevReport &report) const out << "Run: " << m_startedAt.toString(Qt::ISODate) << "\n"; out << "Output device: " << device(m_devices.playbackDevice) << "\n"; out << "Input device: " << device(m_devices.recordDevice) << "\n"; + + // What the device says of itself, as the window has it now. The + // output latency counts frames at the rate the play source was told + // the device runs at, else the recording's (MainWindow::roundTripAt()) + QStringList drivers; + for (const std::string &name : + breakfastquay::AudioFactory::getImplementationNames()) { + drivers << QString::fromStdString(name); + } + out << "Audio drivers built in: " << drivers.join(", ") << "\n"; + const sv_samplerate_t recordingRate = m_fresh.recordingRate; + sv_samplerate_t outputRate = m_window->m_playSource ? + m_window->m_playSource->getDeviceSampleRate() : 0; + if (outputRate <= 0) outputRate = recordingRate; + auto latency = [](qint64 frames, sv_samplerate_t rate) { + if (rate <= 0) return QString("%1 frames").arg(frames); + return QString("%1 frames (%2)").arg(frames) + .arg(unsignedMs(double(frames) / rate)); + }; + out << "Playback latency reported: " + << (m_window->m_playSource ? + latency(m_window->m_playSource->getTargetPlayLatency(), + outputRate) : QString("no device")) << "\n"; + out << "Record latency reported: " + << (m_window->m_recordTarget ? + latency(m_window->m_recordTarget->getSystemRecordLatency(), + recordingRate > 0 ? recordingRate : outputRate) + : QString("no device")) << "\n"; out << "Round trip for the run: " << (m_options.roundTrip >= 0.0 ? unsignedMs(m_options.roundTrip) : QString("the one takes are placed with")) << "\n"; diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index 1d5815f9..6ca95e84 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -22,6 +22,7 @@ #include "../Coverage.h" #include "../LatencyCalibration.h" #include "../LatencyCheck.h" +#include "TakeObserver.h" #include "base/Event.h" @@ -107,8 +108,11 @@ struct DevReport * and its take's file judged again. * * The checks are worked out when the run ends, from what the stages - * kept: item 1 (latency, also after save and reopen) and item 2 - * (several phrases in one take). + * kept: item 1 (latency, also after save and reopen), item 2 (several + * phrases in one take), and from what a TakeObserver saw of each of + * stage 1's punch-ins, items 3 (live dots, and how far behind the + * cursor they appear), 4 (nothing of the take in the speakers) and 5 + * (the mic on input 2). * * The session saved stays open afterwards, so that the takes can be * looked at; its scratch folder stays with it, and the next run @@ -129,6 +133,21 @@ class DevChecks : public QObject /// may land, either way static constexpr double kPlacementSeconds = 0.002; + /// Item 3: more live dots than this in every punch-in, as + /// test-tony-device asked + static constexpr int kMinDots = 10; + + /// Item 3: a dot is on one of the reference's sounds when it lies + /// from the sound's start to half the live tracker's window past + /// its end, give or take this many of its hops (liveDotsCheck()), + /// and on a tone, within this many cents of its pitch + static constexpr int kDotHops = 1; + static constexpr double kDotCents = 50.0; + + /// Item 5: an input carries the mic when its peak is no more than + /// this far below the loudest input's + static constexpr double kMicChannelDb = 20.0; + /// How often a stage is looked at static constexpr int kPollMs = 50; @@ -254,12 +273,31 @@ class DevChecks : public QObject QString m_sessionPath; bool m_saved; - /// Stage 1: the layout, what the runner found, and the take's - /// coverage straight after + /// What the observer saw of one of the runner's punch-ins, and the + /// peak of each channel of its raw recording, full scale 1, or why + /// that could not be read + struct Watched { + int punchIn; ///< counting from 0 + TakeObserver::Observation seen; + std::vector channelPeaks; + QString channelError; + Watched() : punchIn(0) { } + }; + + /// Watches each punch-in of a run of the runner's that this run + /// started, from its Recording step until its analysis is done; + /// which punch-in, counting from 1, or 0; and what it saw + TakeObserver *m_observer; + int m_observedPunchIn; + std::vector m_watched; + + /// Stage 1: the layout, what the runner found, the take's coverage + /// straight after, and what was seen of each punch-in LatencyCheck::Layout m_layout; bool m_haveFresh; AudioCheckResult m_fresh; Coverage m_coverageAfterFresh; + std::vector m_freshWatched; /// Stage 2: the take's pitch and notes before the save and after /// the reopen, and its file judged again after it @@ -272,6 +310,13 @@ class DevChecks : public QObject void poll(); void runnerFinished(const AudioCheckResult &result); + void runnerProgress(const AudioCheckRunner::Progress &state); + + /// Stop the observer and keep what it saw + void finishObservation(); + + /// What was seen of stage 1's punch-in i, counting from 0, or null + const Watched *freshWatched(int i) const; void beginFreshPunchIns(); bool freshPunchInsDone(); @@ -289,6 +334,9 @@ class DevChecks : public QObject std::vector evaluate(QString reason) const; CheckResult latencyCheck(QString reason) const; CheckResult phrasesCheck(QString reason) const; + CheckResult liveDotsCheck(QString reason) const; + CheckResult speakersCheck(QString reason) const; + CheckResult micChannelCheck(QString reason) const; /// The report file written, or "" if it could not be QString writeReport(const DevReport &report) const; diff --git a/main/dev/TakeObserver.cpp b/main/dev/TakeObserver.cpp new file mode 100644 index 00000000..baa776ed --- /dev/null +++ b/main/dev/TakeObserver.cpp @@ -0,0 +1,210 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifdef TONY_DEV_CHECKS + +#include "TakeObserver.h" + +#include "../Analyser.h" +#include "../MainWindow.h" + +#include "audio/AudioCallbackPlaySource.h" +#include "audio/AudioCallbackRecordTarget.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/model/WritableWaveFileModel.h" +#include "view/ViewManager.h" + +#include +#include +#include +#include + +#include + +using namespace sv; + +TakeObserver::TakeObserver(MainWindow *window, QObject *parent) : + QObject(parent), + m_window(window), + m_timer(new QTimer(this)), + m_playbackFrame(0), + m_inputSince(false), + m_inputLeftSince(0.f), + m_inputRightSince(0.f), + m_inputLeft(0.f), + m_inputRight(0.f) +{ + m_timer->setInterval(kPollMs); + m_timer->setTimerType(Qt::PreciseTimer); + connect(m_timer, &QTimer::timeout, this, &TakeObserver::poll); + + // While recording, the meter is told the input levels the view + // manager reads; the observer must not read them itself + if (m_window->m_viewManager) { + connect(m_window->m_viewManager, &ViewManager::monitoringLevelsChanged, + this, &TakeObserver::inputLevels); + } +} + +TakeObserver::~TakeObserver() +{ +} + +void +TakeObserver::start() +{ + m_observation = Observation(); + m_seen.clear(); + m_inputSince = false; + m_inputLeftSince = m_inputRightSince = 0.f; + m_inputLeft = m_inputRight = 0.f; + m_playbackFrame = 0; + + // This take's dots come in a model made once the take is under way + m_earlierDots = m_window->m_realtimePitchModelId; + + if (m_window->m_viewManager) { + m_observation.playbackStart = + m_window->m_viewManager->getRecordStartFrame(); + } + if (m_window->m_playSingingAudio) { + m_observation.singingAudioBefore = + m_window->m_playSingingAudio->isChecked(); + } + + m_clock.start(); + poll(); + m_timer->start(); +} + +void +TakeObserver::stop() +{ + // No last look: the next take may have begun by now + m_timer->stop(); +} + +bool +TakeObserver::isObserving() const +{ + return m_timer->isActive(); +} + +void +TakeObserver::inputLevels(float left, float right) +{ + // While the take only plays, the view manager tells the meter the + // output levels instead + if (!isObserving() || !m_window->m_recordTarget || + !m_window->m_recordTarget->isRecording()) { + return; + } + if (!m_inputSince) { + m_inputSince = true; + m_inputLeftSince = left; + m_inputRightSince = right; + } else { + m_inputLeftSince = std::max(m_inputLeftSince, left); + m_inputRightSince = std::max(m_inputRightSince, right); + } +} + +void +TakeObserver::poll() +{ + AudioCallbackRecordTarget *target = m_window->m_recordTarget; + AudioCallbackPlaySource *source = m_window->m_playSource; + + Sample s; + s.ms = m_clock.elapsed(); + s.recording = target && target->isRecording(); + + // Only while recording: then getPlaybackFrame() works out what the + // view manager's own poll does. While the reference plays on after + // the take, it would take the play source's frame, and could do so + // between the take's putting the playhead back and stopping playback + if (s.recording && m_window->m_viewManager) { + m_playbackFrame = m_window->m_viewManager->getPlaybackFrame(); + } + s.playbackFrame = m_playbackFrame; + + if (target) s.framesBefore = target->getFramesReceived(); + if (s.recording && source) { + s.outputRead = true; + float left = 0.f, right = 0.f; + if (source->getOutputLevels(left, right)) { + s.outputLeft = left; + s.outputRight = right; + } + } + if (target) s.framesAfter = target->getFramesReceived(); + + if (s.recording) { + if (m_inputSince) { + m_inputLeft = m_inputLeftSince; + m_inputRight = m_inputRightSince; + m_inputSince = false; + } + s.inputLeft = m_inputLeft; + s.inputRight = m_inputRight; + } + + s.status = m_window->getStatusLabel()->text(); + s.modal = (QApplication::activeModalWidget() != nullptr); + + if (s.recording && m_observation.recordingPath == "") { + if (auto recording = ModelById::getAs + (m_window->m_currentRecordingModelId)) { + m_observation.recordingPath = recording->getLocation(); + } + } + + // Each dot the first time it is there. The window may throw the + // dots placed so far away once, when it learns the start gap; any + // placed again are new dots + const ModelId dotsId = m_window->m_realtimePitchModelId; + if (!dotsId.isNone() && dotsId != m_earlierDots) { + if (auto dots = ModelById::getAs(dotsId)) { + for (const Event &e : dots->getAllEvents()) { + const auto key = std::make_pair(e.getFrame(), e.getValue()); + if (!m_seen.insert(key).second) continue; + Dot d; + d.ms = s.ms; + d.frame = e.getFrame(); + d.hz = e.getValue(); + d.playbackFrame = s.playbackFrame; + m_observation.dots.push_back(d); + } + } + } + + if (!s.recording && m_observation.stoppedMs < 0 && + !m_observation.samples.empty() && + m_observation.samples.back().recording) { + m_observation.stoppedMs = s.ms; + } + if (!s.recording && m_observation.stoppedMs >= 0) { + m_observation.sawAfter = true; + m_observation.singingAudioAfter = + m_window->m_playSingingAudio && + m_window->m_playSingingAudio->isChecked(); + Analyser *take = m_window->m_analyser2; + m_observation.takeAudibleAfter = + take && take->isAudible(Analyser::Audio); + } + + m_observation.samples.push_back(s); +} + +#endif diff --git a/main/dev/TakeObserver.h b/main/dev/TakeObserver.h new file mode 100644 index 00000000..0096d72f --- /dev/null +++ b/main/dev/TakeObserver.h @@ -0,0 +1,193 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_TAKE_OBSERVER_H +#define TONY_TAKE_OBSERVER_H + +#ifdef TONY_DEV_CHECKS + +#include "base/BaseTypes.h" +#include "data/model/Model.h" + +#include +#include +#include + +#include +#include +#include + +class MainWindow; +class QTimer; + +/** + * Watches one take of the window's (development builds only), from + * just after it starts until it is let go, and keeps what it saw for + * the development checks to judge afterwards: DevChecks starts one for + * each punch-in of the audio check when it starts recording, and stops + * it when that punch-in's analysis is done. + * + * It only looks. It writes no model and no setting, and reads nothing + * that someone else would then miss, with one exception made on + * purpose: the play source's output levels. getOutputLevels() and + * getInputLevels() each give the peak since the previous call and + * reset it, so each can have one reader only. While a take is being + * recorded, lead-in included, ViewManager::checkPlayStatus() reads the + * input levels for the meter and never the output levels, so the + * observer reads the output levels then, and only then, and takes the + * input levels from the meter's own signal. + * + * A friend of MainWindow, as DevChecks is: it reads the take's state. + */ +class TakeObserver : public QObject +{ + Q_OBJECT + +public: + /// How often it looks: as often as ViewManager moves the cursor + static constexpr int kPollMs = 20; + + /// What one poll saw + struct Sample { + /// Since the observer started + qint64 ms; + + /// The record target was recording + bool recording; + + /// The ViewManager's playback frame, where the cursor is drawn: + /// where the take's playback started plus what has been + /// recorded (ViewManager::getPlaybackFrame()). Read only while + /// recording; after that, the last read + sv::sv_frame_t playbackFrame; + + /// Frames the record target had received, read just before and + /// just after the output levels. An audio callback takes in a + /// block of input and then hands out a block of output, so these + /// place the output blocks the levels were taken from + sv::sv_frame_t framesBefore; + sv::sv_frame_t framesAfter; + + /// The output levels were read at this poll: only while + /// recording. They are the loudest sample handed to the device + /// since the previous read, left and right, full scale 1; 0 if + /// the device reported none + bool outputRead; + float outputLeft; + float outputRight; + + /// The loudest input the level meter was told of since the + /// previous poll, or what it was last told if nothing since; + /// while recording only + float inputLeft; + float inputRight; + + /// The status bar's text, and whether a modal dialog was up + QString status; + bool modal; + + Sample() : ms(0), recording(false), playbackFrame(0), + framesBefore(0), framesAfter(0), outputRead(false), + outputLeft(0), outputRight(0), inputLeft(0), + inputRight(0), modal(false) { } + }; + + /// A live dot, as it first appeared + struct Dot { + qint64 ms; + + /// Where it was drawn, on the reference's timeline, and its + /// pitch + sv::sv_frame_t frame; + float hz; + + /// The playback frame at the poll that first saw it + sv::sv_frame_t playbackFrame; + + Dot() : ms(0), frame(0), hz(0), playbackFrame(0) { } + }; + + /// All it saw of one take + struct Observation { + std::vector samples; + std::vector dots; + + /// Where the take's playback started (the pre-roll's lead-in + /// included): the ViewManager's record start frame + sv::sv_frame_t playbackStart; + + /// The raw recording, with every channel the device delivered; + /// "" if none was seen + QString recordingPath; + + /// When a poll first found the take no longer recording, since + /// the observer started; -1 if none did + qint64 stoppedMs; + + /// Play Singing Audio as the button showed it when the take had + /// begun (it goes on showing what was asked for before the take, + /// while the take's audio is kept silent), and as it showed after + /// the take, with whether the take's audio was then audible; the + /// last two from the last poll after the take stopped + bool singingAudioBefore; + bool singingAudioAfter; + bool takeAudibleAfter; + bool sawAfter; + + Observation() : playbackStart(0), stoppedMs(-1), + singingAudioBefore(false), singingAudioAfter(false), + takeAudibleAfter(false), sawAfter(false) { } + }; + + explicit TakeObserver(MainWindow *window, QObject *parent = nullptr); + virtual ~TakeObserver(); + + /// Start watching the take just started, afresh; it is looked at at + /// once, and then every kPollMs + void start(); + + /// Stop watching. What was seen stays until the next start() + void stop(); + + bool isObserving() const; + + const Observation &observation() const { return m_observation; } + +private: + MainWindow *m_window; + QTimer *m_timer; + QElapsedTimer m_clock; + Observation m_observation; + sv::sv_frame_t m_playbackFrame; + + /// The dot model there was when the take began: the last take's, + /// if its dots were still on show. This take's is made later + sv::ModelId m_earlierDots; + + /// The dots seen so far, by frame and pitch + std::set> m_seen; + + /// What the level meter was told since the last poll, and last + bool m_inputSince; + float m_inputLeftSince; + float m_inputRightSince; + float m_inputLeft; + float m_inputRight; + + void poll(); + void inputLevels(float left, float right); +}; + +#endif +#endif diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h index cf91e70e..988b5004 100644 --- a/main/test/FakeAudioIO.h +++ b/main/test/FakeAudioIO.h @@ -27,6 +27,7 @@ #include #include +#include #include #include #include @@ -72,6 +73,17 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO // must be at least a block bool loopback = false; + // A second arrival of the loopback, echoDelay frames after the + // first, at echoGain times its level: the input played back out + // somewhere (Windows' "Listen to this device") and heard again. + // No second arrival while echoGain is 0 + int echoDelay = 0; + float echoGain = 0.f; + + // Tell the application the peak of each block's input and output, + // left and right, as PortAudioIO does for its level meters + bool reportLevels = false; + // Whether the application keeps the input it is given just // now. It discards input until its recording file is open, // which is some time after it resumes the device. If unset, @@ -205,6 +217,12 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO } } + static float peak(const float *samples, int count) { + float p = 0.f; + for (int i = 0; i < count; ++i) p = std::max(p, std::fabs(samples[i])); + return p; + } + // The input at a position in the captured output's timeline float inputAt(long frame) const { long origin = m_resumeFrame; @@ -228,6 +246,12 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO if (m_config.loopback) { long j = base + i - m_config.inputDelay; if (j >= 0 && j < base) in[i] += m_captured[size_t(j)]; + if (m_config.echoGain != 0.f) { + j -= m_config.echoDelay; + if (j >= 0 && j < base) { + in[i] += m_config.echoGain * m_captured[size_t(j)]; + } + } } } @@ -243,11 +267,19 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO } m_target->putSamples(inPtrs.data(), ch, n); if (kept) m_sinceResume += n; + if (m_config.reportLevels) { + m_target->setInputLevels(peak(inPtrs[0], n), + peak(inPtrs[ch > 1 ? 1 : 0], n)); + } std::vector> out(ch, std::vector(n, 0.f)); std::vector outPtrs; for (auto &v : out) outPtrs.push_back(v.data()); int got = m_source->getSourceSamples(outPtrs.data(), ch, n); + if (m_config.reportLevels) { + m_source->setOutputLevels(peak(outPtrs[0], got), + peak(outPtrs[ch > 1 ? 1 : 0], got)); + } for (int i = 0; i < n; ++i) { float mix = 0.f; diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index ec4c4c02..9b83ceaa 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -108,6 +108,7 @@ class TestDevChecks : public QObject config.recordLatency = reportedIn; config.inputDelay = roundTrip; config.loopback = true; + config.reportLevels = true; return config; } @@ -170,6 +171,14 @@ class TestDevChecks : public QObject return ok ? ms : std::nan(""); } + // Item 3's "279: 274 on the tones, 5 on the sweeps, 0 elsewhere" as + // 279; -1 for anything else + static int dotCount(QString text) { + bool ok = false; + int n = text.section(':', 0, 0).toInt(&ok); + return ok ? n : -1; + } + QString reportText() { QFile file(m_report.reportPath); if (!file.open(QIODevice::ReadOnly | QIODevice::Text)) return {}; @@ -257,13 +266,13 @@ class TestDevChecks : public QObject QVERIFY(!m_window->audioCheck()->isRunning()); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(!m_window->audioCheckTakes()); - QCOMPARE(int(m_report.checks.size()), 2); + QCOMPARE(int(m_report.checks.size()), 5); for (const CheckResult &c : m_report.checks) { QVERIFY2(c.verdict == CheckResult::Verdict::Skipped, describe()); QVERIFY2(c.message.contains(m_report.failure), describe()); } QCOMPARE(lastReportLine(), - QString("Totals: 0 passed, 0 failed, 0 measured, 2 skipped")); + QString("Totals: 0 passed, 0 failed, 0 measured, 5 skipped")); QTest::qWait(500); QCOMPARE(m_finished, 1); @@ -418,7 +427,7 @@ private slots: qDebug().noquote() << "report:" << line; } QVERIFY2(m_report.failure == "", describe()); - QCOMPARE(int(m_report.checks.size()), 2); + QCOMPARE(int(m_report.checks.size()), 5); const CheckResult *latency = check(1); const CheckResult *phrases = check(2); QVERIFY(latency && phrases); @@ -431,6 +440,52 @@ private slots: QVERIFY2(number(*latency, "pitch after reopening") .endsWith("pitch events, the same"), describe()); + // What was seen of each punch-in: the live dots on the + // reference's sounds, and behind the cursor by the round trip at + // least, since the cursor runs with what has been recorded; + // nothing played where the reference is silent, though the + // reference itself was heard; and the loopback on both inputs + const CheckResult *dots = check(3); + const CheckResult *speakers = check(4); + const CheckResult *mic = check(5); + QVERIFY(dots && speakers && mic); + QCOMPARE(dots->name, QString("live_dots")); + QCOMPARE(speakers->name, + QString("nothing_of_the_take_in_the_speakers")); + QCOMPARE(mic->name, QString("mic_on_input_2")); + QVERIFY2(dots->verdict == CheckResult::Verdict::Pass, describe()); + QVERIFY2(speakers->verdict == CheckResult::Verdict::Pass, describe()); + QVERIFY2(mic->verdict == CheckResult::Verdict::Measured, describe()); + for (QString label : { QString("dots, punch-in 1 (6.30 to 10.20 s)"), + QString("dots, punch-in 2 (16.80 to 21.20 " + "s)") }) { + QVERIFY2(dotCount(number(*dots, label)) > DevChecks::kMinDots, + describe()); + QVERIFY2(number(*dots, label).endsWith(", 0 elsewhere"), + describe()); + } + QVERIFY2(milliseconds(number(*dots, "dots behind the cursor, median")) + >= roundTrip * 1000.0 / rate, describe()); + QCOMPARE(number(*speakers, "second arrival"), QString("none heard")); + const QString gaps = + number(*speakers, "largest output level in the silent gaps"); + QVERIFY2(gaps.startsWith("silence, over ") && + !gaps.endsWith(" 0 looks"), describe()); + QCOMPARE(number(*speakers, "largest output level"), + QString("-12.0 dBFS")); + QVERIFY2(number(*mic, "the mic is on").startsWith("inputs 1 and 2, "), + describe()); + QVERIFY2(mic->message.startsWith("Not applicable here"), describe()); + + // What the device says of itself, at the head of the report + for (QString words : { QString("Audio drivers built in: "), + QString("Playback latency reported: 8192 " + "frames (185.8 ms)"), + QString("Record latency reported: 4096 " + "frames (92.9 ms)") }) { + QVERIFY2(reportText().contains(words), qPrintable(words)); + } + QCOMPARE(m_stages, QStringList() << "1 of 2: Fresh punch-ins" << "2 of 2: Save and reopen"); @@ -445,7 +500,7 @@ private slots: QVERIFY(QFileInfo(m_report.reportPath).fileName() == "DevChecks.txt"); QVERIFY(TakesFile::isInFolder(reportDirectory(), m_report.reportPath)); QCOMPARE(lastReportLine(), - QString("Totals: 2 passed, 0 failed, 0 measured, 0 skipped")); + QString("Totals: 4 passed, 0 failed, 1 measured, 0 skipped")); QVERIFY(m_report.sessionPath != ""); QCOMPARE(m_window->sessionFile(), m_report.sessionPath); @@ -501,8 +556,60 @@ private slots: QVERIFY2(text.contains(words), qPrintable(words + " not in:\n" + text)); } + + // Items 4 and 5 do not depend on where the take is placed. The + // dots are 20 ms early too, which is about as far as item 3 lets + // them be: whether they pass depends on where the tracker's hops + // fall + QVERIFY2(check(4) && check(4)->verdict == CheckResult::Verdict::Pass, + describe()); + QVERIFY2(check(5) && + check(5)->verdict == CheckResult::Verdict::Measured, + describe()); + const bool dotsPass = + check(3) && check(3)->verdict == CheckResult::Verdict::Pass; QCOMPARE(lastReportLine(), - QString("Totals: 0 passed, 2 failed, 0 measured, 0 skipped")); + QString("Totals: %1 passed, %2 failed, 1 measured, 0 skipped") + .arg(dotsPass ? 2 : 1).arg(dotsPass ? 2 : 3)); + } + + // The loopback heard a second time, 50 ms later at half the level, as + // a mic hears an input that the system plays back out; and the input + // on channel 2 only, as a mic on input 2 of an interface. Item 4 + // fails on the second arrival; item 5 names input 2, and passes + // because the live dots were drawn all the same + void dev_checks_echo_and_the_mic_on_input_2() { + FakeAudioIO::Config config = loopback(); + config.echoDelay = 2205; + config.echoGain = 0.5f; + config.inputChannel = 1; + makeWindow(config); + + runDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_report.failure == "", describe()); + const CheckResult *speakers = check(4); + const CheckResult *mic = check(5); + QVERIFY(speakers && mic); + QVERIFY2(speakers->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(speakers->message.contains("arrived a second time"), + describe()); + const QString echo = number(*speakers, "second arrival"); + const double delayMs = milliseconds(echo.section(" after", 0, 0)); + const double levelDb = echo.section(", ", 1, 1).chopped(3).toDouble(); + QVERIFY2(std::fabs(delayMs - 50.0) <= 1.0, describe()); + QVERIFY2(std::fabs(levelDb + 6.0) <= 1.0, describe()); + + // Tony played nothing more than without the echo + QVERIFY2(number(*speakers, "largest output level in the silent gaps") + .startsWith("silence, "), describe()); + + QVERIFY2(mic->verdict == CheckResult::Verdict::Pass, describe()); + QVERIFY2(number(*mic, "the mic is on").startsWith("input 2, "), + describe()); + QVERIFY2(number(*mic, "input peaks, punch-in 1") + .startsWith("input 1 silence, input 2 -"), describe()); } // Cancelled during a take: the take stops, the run ends once with diff --git a/meson.build b/meson.build index 102d948b..c98d491a 100644 --- a/meson.build +++ b/meson.build @@ -1153,9 +1153,11 @@ tony_app_moc_headers = [ if dev_checks tony_app_files += [ 'main/dev/DevChecks.cpp', + 'main/dev/TakeObserver.cpp', ] tony_app_moc_headers += [ 'main/dev/DevChecks.h', + 'main/dev/TakeObserver.h', ] endif From 68c9a915c4d14866b8a36740d42c5cb082187f59 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 06:35:03 +0000 Subject: [PATCH 149/275] docs: calibrate audio work orders, C1c refined Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 48 +++++++++++++++++++++-------- 1 file changed, 36 insertions(+), 12 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 1a877049..eacaf3ca 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,7 +194,7 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). @@ -256,17 +256,41 @@ during a take; `main/test/FakeAudioIO.h`. ### C1c — Re-record and pre-roll stages; items 7, 12, 13, 14 (spec §4 rows 7, 12, 13, 14) -To be refined by the lead after C1b. Outline: - -- Before and after each punch-in: the take's samples from its file and its pitch and - notes, compared with `TakeDiff`. -- New stages before "Save and reopen": re-record over one of stage 1's punch-ins, starting - inside it, so its lead-in plays over earlier material (items 7, 12, 14), with no - overwrite question for the check's takes; then a punch-in at P = 1 s with a 3 s - pre-roll for that plan (item 13: playback from 0, a shorter countdown, placement right). -- Item 14 from the observer: the take stopped within 0.25 s plus one poll of the - selection's end; the coverage added is exactly the selection; no modal widget. -- Items 1 and 2 then cover every punch-in of the run. +Read also: `main/dev/DevChecks.{h,cpp}` and `main/dev/TakeObserver.h`; `main/TakeDiff.h`; +`main/TakeTiming.h` (`shouldStopAt()`, `countdownText`); in `main/MainWindow.cpp` +`wantedPreRollFrames()`, `pollTakeProgress()` and the start of `record()`; +`docs/recording.md` on pre-roll and Record into Selection; spec §4 rows 7, 12, 13, 14. + +- **Snapshots.** Before and after each punch-in of the new stages: the take's samples from + its file (`AudioCheckRunner::readTakeFile()`), its pitch and notes (by value, looked up + afresh), and its coverage. Compared with `TakeDiff`. +- **Stage "Re-record"**, after "Fresh punch-ins": one runner run keeping the session, one + range starting inside stage 1's second punch-in ([16.8, 21.2] s) between 17.9 and 19.2 s + and ending by 21.2 s. Its 1 s lead-in then plays over the earlier punch-in's recorded + sweep at 17.7 s, and it still judges the sweep at 20.1 s (C1a's note). Items: + - **7** Placement as in items 1 and 2; outside the placed range the take's audio is + bit-identical (`audioOutside()`), and its pitch and notes are unchanged beyond + ± 0.25 s (`eventsOutside()`). + - **12** The lead-in: nothing before P changed (the same comparisons, their part before + P, reported apart), and C1b's output-in-the-gaps check over the lead-in's looks, which + now lie over take audio. **This is the case C1b could not show failing**: show it here. + - **14** The take stopped by itself: the frames recorded past what `shouldStopAt()` + needed, in seconds, within 0.25 s plus one poll of the take timer (100 ms); the + coverage added is exactly the selection; no modal widget seen during the take. The + check's takes always record into a selection, and `record()` asks the overwrite + question only when not (`end < 0`), so "no question" is watched, not arranged. +- **Stage "Pre-roll near the start"**: one range from P = 1 s judging the sweep at 3.1 s, + with a pre-roll of 3 s for this plan only: a new `Plan` field, read where + `wantedPreRollFrames()` reads `kPreRollSeconds` today. Item **13**: playback ran from + frame 0 and never before it (the observer's cursor), the countdown began at 1 and not + 3 (the observer's status text), placement right. +- Items 1 and 2 then cover every punch-in of the run; say how their numbers read now. +- **Tests** (`TestDevChecks`): the passing run gains the two stages (keep the whole run + under about 25 s on the fake). Failing cases, sharing runs where they can: + - the take audible during the re-record's lead-in (set its play parameters audible when + the runner reports `Recording` for that punch-in, from the test): items 4 and 12 fail; + - show failure by breaking the code for item 7 (e.g. the splice's fade beyond its range) + and item 13 (e.g. the pre-roll not clamped at 0), and undo by hand. ### C2 — Long song and joins; items 9, 10 (spec §4 rows 9, 10) From f627f962f4e638064804dbd5681fa9c62ec802b1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 06:58:59 +0000 Subject: [PATCH 150/275] build: container-setup.sh from feat/tonyandroid The script that makes a fresh ubuntu 24.04 cloud container able to build tony and run both suites: packages, qt 6.11 from conda-forge, the libraries at their pins (the mercurial ones from their github mirrors), meson setup build. Taken unchanged from feat/tonyandroid, so that the cloud session script on default can use it and a merge of the two branches adds nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/container-setup.sh | 289 ++++++++++++++++++++++++++++++++ 1 file changed, 289 insertions(+) create mode 100755 deploy/linux/container-setup.sh diff --git a/deploy/linux/container-setup.sh b/deploy/linux/container-setup.sh new file mode 100755 index 00000000..65b21c5a --- /dev/null +++ b/deploy/linux/container-setup.sh @@ -0,0 +1,289 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Makes a fresh Ubuntu 24.04 cloud container able to build Tony and run +# both test suites: installs the apt packages and Qt, checks out the +# library directories at the revisions pinned in repoint-lock.json, and +# configures build/ with meson. +# +# It exists because "./repoint install" cannot work in the cloud +# container: the Mercurial libraries are on hg.sr.ht, which cannot be +# reached from there. Their Git mirrors on GitHub are +# checked out instead, at the commits in the table below. The Git +# libraries are cloned from GitHub at their pins, as repoint would. +# sv-dependency-builds is skipped: only the macOS and Windows branches +# of meson.build use it. +# +# Qt is conda-forge's qt6-main, not Ubuntu's Qt 6.4. Tony builds with +# 6.4, but its analysis never completes there: Analyser connects by +# SIGNAL()/SLOT() strings naming ModelId and sv_frame_t to slots that +# moc records as sv::ModelId and sv::sv_frame_t, and only Qt 6.5 and +# later match those by their registered metatypes rather than by name. +# The development machine and the Android build use Qt 6.11. +# +# Safe to run again. Packages already installed, a Qt of the right +# version and libraries already at their pins are left alone, and a +# library with local changes or local commits is never moved. +# +# Usage, from anywhere: +# deploy/linux/container-setup.sh set up and configure build/ +# deploy/linux/container-setup.sh --build the same, then build Tony, +# the pYIN plugin and both +# test executables + +set -eu -o pipefail + +build=no +case "${1:-}" in + "") ;; + --build) build=yes ;; + *) echo "Usage: $0 [--build]" 1>&2; exit 2 ;; +esac + +cd "$(dirname "$0")/../.." +root=$(pwd) +echo "Setting up $root" + +sudo="" +if [ "$(id -u)" -ne 0 ]; then + sudo=sudo +fi + +qt_version=6.11.2 +qt_prefix=/opt/qt6-conda +micromamba_version=2.9.0-0 +micromamba=/opt/micromamba/bin/micromamba + +# Only Qt's own .pc files, so that pkg-config finds everything else +# (alsa, for one, which the conda prefix has too) on the system +qt_pkgconfig=$qt_prefix/tony-pkgconfig + +# 1. Packages. The names are those of .github/workflows/linux.yml where +# meson.build needs them, as spelled on Ubuntu 24.04, less Qt. Rubber +# Band is Ubuntu's librubberband-dev (3.3), which meets meson.build's +# ">= 3.0.0", so the CI's tarball from breakfastquay.com (blocked here +# too) is not needed. + +packages=" +build-essential pkg-config ninja-build meson git python3 curl ca-certificates +libboost-dev libbz2-dev libfftw3-dev libsndfile1-dev libsamplerate0-dev +librubberband-dev libsord-dev libserd-dev liboggz2-dev libfishsound1-dev +libmad0-dev libid3tag0-dev libopus-dev libopusfile-dev libopusenc-dev +libjack-jackd2-dev libpulse-dev libasound2-dev portaudio19-dev +fonts-dejavu-core +" + +missing="" +for p in $packages; do + if ! dpkg-query -W -f='${Status}' "$p" 2>/dev/null | grep -q "install ok installed"; then + missing="$missing $p" + fi +done + +if [ -n "$missing" ]; then + echo + echo "Installing packages:$missing" + # Some of the container's own apt sources (PPAs) are blocked too; + # apt-get update warns about them and carries on. + $sudo apt-get update -q + $sudo env DEBIAN_FRONTEND=noninteractive \ + apt-get install -y -q --no-install-recommends $missing +else + echo + echo "Packages: all installed" +fi + +# 2. Qt, from conda-forge through micromamba (both from hosts the +# container can reach: GitHub releases and conda.anaconda.org). + +echo +if [ -x "$micromamba" ]; then + echo "micromamba: installed already" +else + echo "Installing micromamba $micromamba_version in $(dirname "$micromamba")" + $sudo mkdir -p "$(dirname "$micromamba")" + # The container's proxy now and then answers 502 for a moment + $sudo curl -sSfL --retry 5 --retry-all-errors -o "$micromamba.part" \ + "https://github.com/mamba-org/micromamba-releases/releases/download/$micromamba_version/micromamba-linux-64" + $sudo chmod +x "$micromamba.part" + $sudo mv "$micromamba.part" "$micromamba" +fi + +installed_qt="" +if [ -f "$qt_prefix/lib/pkgconfig/Qt6Core.pc" ]; then + installed_qt=$(PKG_CONFIG_PATH="$qt_prefix/lib/pkgconfig" pkg-config --modversion Qt6Core) +fi + +if [ "$installed_qt" = "$qt_version" ]; then + echo "Qt: $qt_version in $qt_prefix already" +else + echo "Installing Qt $qt_version (conda-forge qt6-main) in $qt_prefix" + if [ -d "$qt_prefix/conda-meta" ]; then + action=install + else + action=create + fi + $sudo env MAMBA_ROOT_PREFIX="$(dirname "$(dirname "$micromamba")")" \ + "$micromamba" $action -y -q -p "$qt_prefix" -c conda-forge \ + "qt6-main=$qt_version" +fi + +$sudo mkdir -p "$qt_pkgconfig" +for pc in "$qt_prefix"/lib/pkgconfig/Qt6*.pc; do + $sudo ln -sf "$pc" "$qt_pkgconfig/" +done + +# 3. Libraries. +# +# repoint-lock.json pins the Mercurial libraries by Mercurial hash, +# which their Git mirrors (github.com/breakfastquay/) do not +# carry, and a conversion back to Mercurial does not reproduce it. Each +# pin was matched to a mirror commit by hand, by date and message: +# +# - bqaudioio 017ab3ed3a33 is the merge of toggle-record-in-io into +# default: the mirror's "Merge from branch toggle-record-in-io". +# - The other pins were taken by upstream Tony's "Update Repoint +# locations and revisions" (2024-06-25). For each, the commit is the +# last one on the mirror's master before that date, and Sonic +# Visualiser's repoint-lock.json took the same Mercurial pin shortly +# after that commit's date: +# dataquay "Fix warning", 2024-01-04 (SV pinned it the same +# day; the mirror's head is newer, from 2024-09) +# bqvec "Adjust local include policy", 2023-06-27 (head) +# bqfft "Update CI for SLEEF", 2022-08-09 (head) +# bqresample "Adjust local include policy", 2023-06-27 (head) +# bqthingfactory "Copyright dates", 2021-01-08 (head) +# +# When a pin in repoint-lock.json changes, this table must change with +# it; the script stops if they disagree. + +mirror_commit() { + case "$1 $2" in + "bqaudioio 017ab3ed3a33") echo 7ab6de96b44d2f0c8ce58a16ce1a9831724dd29f ;; + "dataquay 79623fb778da") echo 2dbf1bed112c1a7eaaf43335abbe5ddb5c03d0ff ;; + "bqvec 291cde50db9d") echo ddfcd1716576c6bb44218c5f5696bf24a248960a ;; + "bqfft d41a117b8cbe") echo 68dc4c5735c1e0da099e8473fc4562acf2895cb8 ;; + "bqresample 38c3e524416a") echo 6cef06961f15399f8cecc414f1364dac7d83c3db ;; + "bqthingfactory 2e4bd170f57f") echo 8468b3f98d9d1768561d6f1fc95d1689af193a3e ;; + *) echo "" ;; + esac +} + +warnings=0 + +checkout() { + local name="$1" url="$2" branch="$3" commit="$4" + local short="${commit:0:12}" + if [ -e "$name/.git" ]; then + local head + head=$(git -C "$name" rev-parse HEAD) + if [ "$head" = "$commit" ]; then + echo " $name: at $short already" + return + fi + if [ -n "$(git -C "$name" status --porcelain --untracked-files=no)" ]; then + echo " $name: WARNING: has local changes and is not at $short; left alone" + warnings=1 + return + fi + if ! git -C "$name" cat-file -e "$commit^{commit}" 2>/dev/null; then + echo " $name: fetching from $url" + git -C "$name" fetch -q origin + fi + # Local commits that are on no remote branch would be lost from + # sight by moving the checkout + if [ -z "$(git -C "$name" branch -r --contains HEAD)" ]; then + echo " $name: WARNING: has local commits and is not at $short; left alone" + warnings=1 + return + fi + # Detached, so that no local branch is moved + git -C "$name" checkout -q --detach "$commit" + elif [ -e "$name" ] && [ -n "$(ls -A "$name")" ]; then + echo "ERROR: $name exists but is not a Git checkout; move it away and run again" 1>&2 + exit 1 + else + echo " $name: cloning $url" + git clone -q ${branch:+--branch "$branch"} "$url" "$name" + # As repoint does: the local branch at the pin, where there is one + if [ -n "$branch" ]; then + git -C "$name" checkout -q -B "$branch" "$commit" + else + git -C "$name" checkout -q --detach "$commit" + fi + fi + echo " $name: checked out $short" +} + +echo +echo "Libraries:" + +# One line per library: name, vcs, owner, repository, branch, pin +libraries=$(python3 - <<'EOF' +import json +project = json.load(open("repoint-project.json"))["libraries"] +lock = json.load(open("repoint-lock.json"))["libraries"] +for name, lib in project.items(): + print(name, lib["vcs"], lib["owner"], lib.get("repository", name.split("/")[-1]), + lib.get("branch", "-"), lock[name]["pin"]) +EOF +) + +while read -r name vcs owner repository branch pin; do + [ "$branch" = "-" ] && branch="" + if [ "$name" = "sv-dependency-builds" ]; then + echo " $name: skipped, not used on Linux" + continue + fi + if [ "$vcs" = "git" ]; then + checkout "$name" "https://github.com/$owner/$repository.git" "$branch" "$pin" + else + commit=$(mirror_commit "$name" "$pin") + if [ -z "$commit" ]; then + echo "ERROR: repoint-lock.json pins $name at $pin, which this script does not map to a commit of its Git mirror; update the table in $0" 1>&2 + exit 1 + fi + checkout "$name" "https://github.com/$owner/$repository.git" "" "$commit" + fi +done <<< "$libraries" + +# 4. Configure. The same build type as the Windows build (build.bat): +# optimised, with asserts. meson keeps the pkg-config path in build/, so +# the reconfigures ninja runs by itself find the same Qt, and it puts +# Qt's library directory in the executables' RPATH. + +setup_args="--buildtype=debugoptimized --pkg-config-path=$qt_pkgconfig" + +echo +if [ -f build/build.ninja ] && + grep -q "$qt_pkgconfig" build/meson-info/intro-buildoptions.json; then + echo "build/: configured already" +elif [ -f build/build.ninja ]; then + echo "build/ is configured with another Qt: configuring it again from scratch" + meson setup --wipe build $setup_args +else + echo "Configuring build/" + meson setup build $setup_args +fi + +if [ "$build" = "yes" ]; then + echo + echo "Building" + ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app +fi + +echo +if [ "$warnings" -ne 0 ]; then + echo "Done, with warnings above" +else + echo "Done" +fi From bb29899304dcbaceeedf13d09f4f57e40784c73d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 06:58:59 +0000 Subject: [PATCH 151/275] test: suites run as shards in parallel processes With TONY_TEST_SHARD=i/n each suite runs every n-th test function from the i-th. The app suite nearly only waits on the fake device's real-time clock, so deploy/linux/run-tests.sh starts 2 x cores processes, each with XDG directories of its own for the suites' QSettings, and adds up what they report: 54 s instead of 364 s on the cloud container's 4 cores. TestRunSuite checks the selection. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/run-tests.sh | 113 +++++++++++++++++++++++++++++++++++ docs/testing.md | 15 ++++- main/test/RunSuite.h | 55 +++++++++++++++++ main/test/TestRunSuite.h | 98 ++++++++++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 +++ meson.build | 1 + 6 files changed, 287 insertions(+), 2 deletions(-) create mode 100755 deploy/linux/run-tests.sh create mode 100644 main/test/TestRunSuite.h diff --git a/deploy/linux/run-tests.sh b/deploy/linux/run-tests.sh new file mode 100755 index 00000000..c08d3134 --- /dev/null +++ b/deploy/linux/run-tests.sh @@ -0,0 +1,113 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Runs a test executable as several processes at once, each running its +# shard of every suite (TONY_TEST_SHARD, main/test/RunSuite.h), and adds +# up what they report. The app suite spends nearly all of its time +# waiting on FakeAudioIO, which plays in real time: on the cloud +# container's 4 cores, 8 processes run it in about a minute instead of +# six, and the load stays under 2. +# +# Each process has a log directory and XDG directories of its own. The +# suites keep QSettings per user, and processes sharing the file would +# read each other's settings. On Windows QSettings is the registry, which +# XDG_CONFIG_HOME does not move: this is for Linux. +# +# Usage, from anywhere: +# deploy/linux/run-tests.sh [-j N] [BUILD_DIR] EXECUTABLE +# +# deploy/linux/run-tests.sh test-tony-app +# +# N defaults to twice the number of cores, BUILD_DIR to build. The +# results are in tmp/tl/EXECUTABLE/SHARD/SUITE.txt; the summary gives +# each suite's counts, less initTestCase and cleanupTestCase, which +# every shard runs, and every failure with its location. The exit status +# is 0 only if every shard's was. + +set -u -o pipefail + +jobs=$(( $(nproc) * 2 )) +if [ "${1:-}" = "-j" ]; then + jobs=${2:-} + shift 2 +fi +root=$(cd "$(dirname "$0")/../.." && pwd) +case "$#" in + 1) build=$root/build; exe=$1 ;; + 2) build=$1; exe=$2 ;; + *) echo "Usage: $0 [-j N] [BUILD_DIR] EXECUTABLE" 1>&2; exit 2 ;; +esac +if ! [ "$jobs" -ge 1 ] 2>/dev/null; then + echo "Usage: $0 [-j N] [BUILD_DIR] EXECUTABLE" 1>&2 + exit 2 +fi + +# BUILD_DIR as given, from where this was run +if ! dir=$(cd "$build" 2>/dev/null && pwd); then + echo "No build directory $build" 1>&2 + exit 2 +fi +build=$dir +if [ ! -x "$build/$exe" ]; then + echo "No executable $build/$exe" 1>&2 + exit 2 +fi + +out=$root/tmp/tl/$exe +rm -rf "$out" +mkdir -p "$out" + +start=$SECONDS +for i in $(seq 0 $((jobs - 1))); do + dir=$out/$i + mkdir -p "$dir/xdg/config" "$dir/xdg/data" "$dir/xdg/cache" + ( + cd "$build" && + TONY_TEST_SHARD=$i/$jobs TONY_TEST_LOG_DIR=$dir \ + XDG_CONFIG_HOME=$dir/xdg/config XDG_DATA_HOME=$dir/xdg/data \ + XDG_CACHE_HOME=$dir/xdg/cache \ + "./$exe" > "$dir/stdout.log" 2>&1 + echo $? > "$dir/exit" + ) & +done +wait +echo "$exe: $jobs processes, $((SECONDS - start)) s" + +status=0 +for i in $(seq 0 $((jobs - 1))); do + code=$(cat "$out/$i/exit" 2>/dev/null || echo "none") + if [ "$code" != 0 ]; then + status=1 + fi + # 1 is failed tests, which the summary lists; anything else is a crash + if [ "$code" != 0 ] && [ "$code" != 1 ]; then + echo "Process $i ended with status $code: see $out/$i/stdout.log" + for f in "$out/$i"/*.txt; do + [ -f "$f" ] || continue + if ! grep -aq "^Totals" "$f"; then + echo " it was running $(basename "$f" .txt): last line $(grep -a '^[A-Z]' "$f" | tail -1)" + fi + done + fi +done + +for suite in $(ls "$out"/*/*.txt 2>/dev/null | xargs -n1 basename | sort -u); do + cat "$out"/*/"$suite" | awk -v suite="${suite%.txt}" ' + /^(PASS|XFAIL) / && !/::(initTestCase|cleanupTestCase)\(\)/ { passed++ } + /^(FAIL!|XPASS) / { failed++ } + /^SKIP / { skipped++ } + END { printf "%s: %d passed, %d failed, %d skipped\n", suite, passed, failed, skipped }' +done + +grep -a "^FAIL!\|^XPASS\|^ Loc" "$out"/*/*.txt | sed "s#^$out/##" + +exit $status diff --git a/docs/testing.md b/docs/testing.md index e21e9d06..13c29475 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,8 +5,8 @@ QtTest suites in `main/test/`, in two executables that mirror the two libraries | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle`, `TestRunSuite` | seconds | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 6 minutes in one process, under a minute in eight (measured 2026-09-26 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | | `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | `meson test` / `build.bat test` runs the first two plus four svcore suites. `test-tony-device` @@ -46,6 +46,17 @@ helpers must not be slots; connect to lambdas instead. For access to private sta the exit status of a run with names is always 1. Only a run with no names has a meaningful exit status. - `QT_QPA_PLATFORM=offscreen` is set by `main()` when not given. +- **Shards.** With `TONY_TEST_SHARD=i/n` each suite runs only every n-th of its test + functions, from the i-th, in declaration order, and a suite with none in the shard does + not run. The app suite nearly only waits on `FakeAudioIO`'s real-time clock, so n + processes at once take about 1/n of the time: on four cores the load stayed under 2 with + eight, and reached 3.5 with twelve. `deploy/linux/run-tests.sh` starts them and adds up + their results; each process needs XDG directories of its own, because the suites' + QSettings are per user and would be shared. On Windows QSettings is the registry, so the + script is for Linux. Do not combine shards with test names on the command line. +- A sharded run is a whole run of the suites, but the tests that share a process are other + ones. After a change to object lifetimes, threads or teardown (see "Timing and races"), + run the one-process run as well. ## Design principles diff --git a/main/test/RunSuite.h b/main/test/RunSuite.h index 06bf5d81..ad2e7b4e 100644 --- a/main/test/RunSuite.h +++ b/main/test/RunSuite.h @@ -16,12 +16,51 @@ #include #include +#include + +/** + * The test functions of a suite that shard \a shard of \a count runs: + * every count-th one in declaration order, starting at the shard's + * number. The test functions are what QTest takes them to be, the + * private slots without arguments, less initTestCase, cleanupTestCase, + * init, cleanup and the _data functions. + */ +inline QStringList +shardFunctions(const QObject *suite, int shard, int count) +{ + QStringList names; + const QMetaObject *mo = suite->metaObject(); + int index = 0; + for (int i = 0; i < mo->methodCount(); ++i) { + QMetaMethod m = mo->method(i); + if (m.methodType() != QMetaMethod::Slot || + m.access() != QMetaMethod::Private || + m.parameterCount() != 0) { + continue; + } + QString name = QString::fromLatin1(m.name()); + if (name == "initTestCase" || name == "cleanupTestCase" || + name == "init" || name == "cleanup" || name.endsWith("_data")) { + continue; + } + if (index++ % count == shard) { + names << name; + } + } + return names; +} /** * Run one suite with the command-line arguments given. If the * environment variable TONY_TEST_LOG_DIR is set, the suite's results * are also written to

/.txt. A single "-o file" * argument cannot do that, as each suite would overwrite the last. + * + * If TONY_TEST_SHARD is set to "i/n", only the suite's shard i of n + * runs (shardFunctions()), so that n processes started at once run the + * whole suite between them in about 1/n of the time: most tests wait + * on a fake device playing in real time. A suite with nothing in the + * shard is not run. Not for use with test names on the command line. */ inline bool runSuite(QObject *suite, int argc, char *argv[]) @@ -30,6 +69,22 @@ runSuite(QObject *suite, int argc, char *argv[]) for (int i = 0; i < argc; ++i) { args << QString::fromLocal8Bit(argv[i]); } + QString shard = qEnvironmentVariable("TONY_TEST_SHARD"); + if (shard != "") { + QStringList parts = shard.split('/'); + int n = (parts.size() == 2 ? parts[1].toInt() : 0); + int i = (parts.size() == 2 ? parts[0].toInt() : -1); + if (n < 1 || i < 0 || i >= n) { + qWarning("TONY_TEST_SHARD must be i/n with 0 <= i < n, not \"%s\"", + qPrintable(shard)); + return false; + } + QStringList names = shardFunctions(suite, i, n); + if (names.isEmpty()) { + return true; + } + args << names; + } QString logDir = qEnvironmentVariable("TONY_TEST_LOG_DIR"); if (logDir != "") { QString file = QDir(logDir).filePath diff --git a/main/test/TestRunSuite.h b/main/test/TestRunSuite.h new file mode 100644 index 00000000..5d2a5c65 --- /dev/null +++ b/main/test/TestRunSuite.h @@ -0,0 +1,98 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_RUN_SUITE_TEST_H +#define TEST_RUN_SUITE_TEST_H + +// Which test functions a shard of a suite runs (TONY_TEST_SHARD): each +// exactly once over the shards, and nothing QTest would not run as a +// test. + +#include "RunSuite.h" + +#include +#include +#include + +// A suite as QTest sees one: its test functions, and slots that are not +// test functions. Never run. +class ShardFixture : public QObject +{ + Q_OBJECT + +public slots: + void publicSlot() {} + +private slots: + void initTestCase() {} + void init() {} + void first() {} + void second_data() {} + void second() {} + void withArgument(int) {} + void third() {} + void fourth() {} + void fifth() {} + void cleanup() {} + void cleanupTestCase() {} +}; + +class TestRunSuite : public QObject +{ + Q_OBJECT + +private slots: + // One shard is the whole suite, in the order declared + void one_shard_is_every_test_function() { + ShardFixture fixture; + QCOMPARE(shardFunctions(&fixture, 0, 1), + QStringList({ "first", "second", "third", "fourth", "fifth" })); + } + + // Each count-th function from the shard's number, so that the + // long tests declared together are spread over the shards + void shards_take_every_nth() { + ShardFixture fixture; + QCOMPARE(shardFunctions(&fixture, 0, 2), + QStringList({ "first", "third", "fifth" })); + QCOMPARE(shardFunctions(&fixture, 1, 2), + QStringList({ "second", "fourth" })); + } + + // Over the shards of any count, every test function once + void shards_cover_each_function_once() { + ShardFixture fixture; + QStringList all = shardFunctions(&fixture, 0, 1); + for (int count = 1; count <= 7; ++count) { + QStringList seen; + for (int shard = 0; shard < count; ++shard) { + seen << shardFunctions(&fixture, shard, count); + } + QVERIFY2(seen.size() == all.size() && + QSet(seen.begin(), seen.end()) == + QSet(all.begin(), all.end()), + qPrintable(QString("%1 shards ran %2") + .arg(count).arg(seen.join(", ")))); + } + } + + // More shards than functions: the ones past the end have none, and + // runSuite() then does not run the suite at all + void shards_past_the_end_are_empty() { + ShardFixture fixture; + QCOMPARE(shardFunctions(&fixture, 4, 6), QStringList({ "fifth" })); + QVERIFY(shardFunctions(&fixture, 5, 6).isEmpty()); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 5beddc6d..a7a138cf 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -21,6 +21,7 @@ #include "TestTakesFile.h" #include "TestTakeTiming.h" #include "TestModelChangeThrottle.h" +#include "TestRunSuite.h" #include "RunSuite.h" @@ -105,6 +106,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestRunSuite t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index 611ffb73..e734fef1 100644 --- a/meson.build +++ b/meson.build @@ -1358,6 +1358,7 @@ tony_core_test_moc_files = qt.preprocess( 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', 'main/test/TestModelChangeThrottle.h', + 'main/test/TestRunSuite.h', ]) tony_core_test_exe = executable( From 75ed4e349f271d2fdbebf14e93c3a0f7becf73ff Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 06:58:59 +0000 Subject: [PATCH 152/275] build: the cloud environment's setup script and a session's background build cloud-environment.sh is pasted into the environment's settings: packages, qt, ccache and mold, the android sdk and ndk when dl.google.com is reachable, and ccache filled from a build of the libraries until four minutes have passed, so that the environment's snapshot is kept. cloud-session.sh start checks the libraries out and builds in the background at the start of a session: 3.7 minutes instead of about 7, while the session reads. building.md has the measurements and the environment's network and variables. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- AGENTS.md | 5 + deploy/linux/cloud-environment.sh | 232 ++++++++++++++++++++++++++++++ deploy/linux/cloud-session.sh | 102 +++++++++++++ docs/building.md | 90 ++++++++++-- 4 files changed, 414 insertions(+), 15 deletions(-) create mode 100755 deploy/linux/cloud-environment.sh create mode 100755 deploy/linux/cloud-session.sh diff --git a/AGENTS.md b/AGENTS.md index 015351cd..c698d12f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,6 +54,11 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. - Give the app suite a tool timeout of 10 minutes. +In a **Linux cloud session** the commands are others: run `deploy/linux/cloud-session.sh +start` first (it builds in the background) and `deploy/linux/cloud-session.sh wait` before +the first build or test; the rest is in +[docs/building.md](docs/building.md#on-linux-a-cloud-session). + ## Rules for working here ### Scope diff --git a/deploy/linux/cloud-environment.sh b/deploy/linux/cloud-environment.sh new file mode 100755 index 00000000..2356454f --- /dev/null +++ b/deploy/linux/cloud-environment.sh @@ -0,0 +1,232 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# The setup script of the Claude cloud environment Tony is developed in. +# It is pasted, whole, into the environment's "Setup script" field; this +# copy is where it is reviewed and versioned, and a change here reaches +# sessions only once it is pasted there too. +# +# The platform runs it as root after cloning the repository, the first +# time a session starts after the script or the network settings change, +# or after the environment's cache expires (about a week). It then keeps +# a snapshot of the disk, and later sessions start from that snapshot +# without running the script. The snapshot is kept only when the script +# finishes within about five minutes, so the script stops its open-ended +# steps at $deadline seconds. +# +# What it leaves in the snapshot: +# +# - The apt packages that container-setup.sh and +# deploy/android/setup-toolchain.sh install, and ccache and mold. +# - Qt 6.11.2 from conda-forge in /opt/qt6-conda, where +# container-setup.sh looks for it. +# - /etc/ccache.conf. meson uses ccache by itself when it is installed. +# - The Android SDK and NDK in /opt/android/sdk, where +# deploy/android/setup-toolchain.sh installs them, when dl.google.com is +# reachable. +# - ccache filled with as much of a build of the libraries as fits before +# the deadline, svcore first: the checkout is set up with its own +# container-setup.sh, built into build/, and left as it was cloned. +# The libraries' objects do not depend on main/, so they are found in +# ccache by every branch whose pins are the same. +# +# Only a failure to install the packages or Qt is worth a warning; the +# script ends with status 0 whatever happens, because a failing setup +# script stops the session from starting, and a session can run +# container-setup.sh itself. The logs are in /var/log/tony-environment. + +set -u -o pipefail + +# Seconds from the start after which nothing more is started, leaving a +# minute for the platform +deadline=240 + +logs=/var/log/tony-environment +mkdir -p "$logs" + +say() { + echo "[${SECONDS}s] $*" +} + +left() { + echo $((deadline - SECONDS)) +} + +# timeout, with what is left before the deadline; "timeout 0" would wait +# for ever +until_deadline() { + local t + t=$(left) + if [ "$t" -le 0 ]; then + return 124 + fi + timeout "$t" "$@" +} + +# 1. ccache. meson passes relative source paths, but -g puts the build +# directory into every result's key unless hash_dir is off, and then a +# second build directory or a worktree finds nothing. + +cat > /etc/ccache.conf <<'EOF' +# Written by Tony's cloud environment setup script +cache_dir = /root/.cache/ccache +max_size = 8G +hash_dir = false +base_dir = /home/user +EOF + +# 2. Packages, Qt and the Android SDK, side by side + +packages=" +build-essential pkg-config ninja-build meson git python3 curl ca-certificates +libboost-dev libbz2-dev libfftw3-dev libsndfile1-dev libsamplerate0-dev +librubberband-dev libsord-dev libserd-dev liboggz2-dev libfishsound1-dev +libmad0-dev libid3tag0-dev libopus-dev libopusfile-dev libopusenc-dev +libjack-jackd2-dev libpulse-dev libasound2-dev portaudio19-dev +fonts-dejavu-core ccache mold +openjdk-21-jdk-headless unzip xz-utils patch file cmake +" + +install_packages() { + # The image's PPAs are blocked; apt-get update warns and carries on + apt-get update -q + DEBIAN_FRONTEND=noninteractive apt-get install -y -q \ + --no-install-recommends -o Acquire::Retries=3 $packages +} + +install_qt() { + local micromamba=/opt/micromamba/bin/micromamba + if [ "$(PKG_CONFIG_PATH=/opt/qt6-conda/lib/pkgconfig pkg-config --modversion Qt6Core 2>/dev/null)" = 6.11.2 ]; then + echo "Qt 6.11.2 is installed already" + return 0 + fi + mkdir -p "$(dirname "$micromamba")" + curl -sSfL --retry 5 --retry-all-errors --max-time 60 -o "$micromamba" \ + https://github.com/mamba-org/micromamba-releases/releases/download/2.9.0-0/micromamba-linux-64 && + chmod +x "$micromamba" && + MAMBA_ROOT_PREFIX=/opt/micromamba "$micromamba" create -y -q \ + -p /opt/qt6-conda -c conda-forge qt6-main=6.11.2 || return 1 + # Only Qt's own .pc files, as container-setup.sh makes them + mkdir -p /opt/qt6-conda/tony-pkgconfig + ln -sf /opt/qt6-conda/lib/pkgconfig/Qt6*.pc /opt/qt6-conda/tony-pkgconfig/ +} + +# The versions and checks of deploy/android/setup-toolchain.sh, which +# finds these installed and installs anything it wants that is not +install_android() { + local sdk=/opt/android/sdk + local build=15859902 + local sha256=4e4c464f145a7512b57d088ac6c278c03c9eea610886b35a5e0804e74eedf583 + local zip=/opt/android/cmdline-tools.zip + if [ -f "$sdk/ndk/27.2.12479018/source.properties" ]; then + echo "The Android SDK and NDK are installed already" + return 0 + fi + if ! curl -sSf -o /dev/null --max-time 20 \ + https://dl.google.com/android/repository/repository2-3.xml; then + echo "dl.google.com is not reachable: no Android SDK" + return 0 + fi + mkdir -p "$sdk/cmdline-tools" + until_deadline curl -sSfL --retry 5 --retry-all-errors -o "$zip" \ + "https://dl.google.com/android/repository/commandlinetools-linux-${build}_latest.zip" && + echo "$sha256 $zip" | sha256sum -c --quiet && + rm -rf "$sdk/cmdline-tools/latest" /opt/android/cmdline-tools && + unzip -q "$zip" -d /opt/android && + mv /opt/android/cmdline-tools "$sdk/cmdline-tools/latest" || return 1 + rm -f "$zip" + # sdkmanager is Java: the proxy has to be named to it unless + # JAVA_TOOL_OPTIONS does it already + local proxy=() + local hostport=${HTTPS_PROXY:-} + hostport=${hostport#*://} + hostport=${hostport%%/*} + if [ -n "$hostport" ]; then + proxy=(--proxy=http "--proxy_host=${hostport%:*}" "--proxy_port=${hostport##*:}") + fi + local sdkmanager=$sdk/cmdline-tools/latest/bin/sdkmanager + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 + { yes || true; } | "$sdkmanager" --sdk_root="$sdk" "${proxy[@]}" --licenses > /dev/null + until_deadline "$sdkmanager" --sdk_root="$sdk" "${proxy[@]}" --install \ + "platforms;android-36" "build-tools;36.0.0" platform-tools "ndk;27.2.12479018" +} + +say "Installing packages, Qt and the Android SDK (logs in $logs)" +install_packages > "$logs/packages.log" 2>&1 & +packages_pid=$! +install_qt > "$logs/qt.log" 2>&1 & +qt_pid=$! +install_android > "$logs/android.log" 2>&1 & +android_pid=$! + +wait "$packages_pid" +packages_status=$? +say "Packages: exit $packages_status" +wait "$qt_pid" +qt_status=$? +say "Qt: exit $qt_status" + +# 3. ccache, from a build of the libraries in the checkout, until the +# deadline + +fill_ccache() { + local repo="" d + for d in "$PWD" /home/user/*; do + if [ -f "$d/repoint-project.json" ] && [ -x "$d/deploy/linux/container-setup.sh" ]; then + repo=$d + break + fi + done + if [ -z "$repo" ]; then + echo "No checkout with deploy/linux/container-setup.sh" + return 0 + fi + echo "Checkout: $repo" + + # What container-setup.sh adds to the checkout, to take away again + local added=() + for d in build $(python3 -c 'import json; print(" ".join(json.load(open("'"$repo"'/repoint-project.json"))["libraries"]))'); do + [ -e "$repo/$d" ] || added+=("$repo/$d") + done + + CC_LD=mold CXX_LD=mold until_deadline "$repo/deploy/linux/container-setup.sh" && + ( + cd "$repo/build" || exit 1 + # svcore, then svgui and svapp (in libtonyapp.a with main/), + # then the rest; each step until the deadline + svlibs=$(ninja -t targets all | sed -n 's/^\(libtonyapp\.a\.p\/sv\(gui\|app\)_[^:]*\.o\):.*/\1/p') + for step in libsvcore.a "$svlibs" "tony pyin.so test-tony-core test-tony-app test-tony-device"; do + [ -n "$step" ] || continue + [ "$(left)" -gt 10 ] || break + until_deadline nice ninja -j "$(nproc)" $step > /dev/null + echo "Built up to: ${step:0:40}...: exit $?" + done + ) + ccache -s | grep -iE 'hits|misses|cache size' || true + + rm -rf "${added[@]}" + git -C "$repo" status --short +} + +if [ "$packages_status" -eq 0 ] && [ "$qt_status" -eq 0 ]; then + say "Filling ccache until ${deadline}s" + fill_ccache > "$logs/ccache.log" 2>&1 + say "ccache: $(ccache -s | grep -a 'Cache size' | head -1 | sed 's/^ *//')" +else + say "WARNING: packages or Qt failed; see $logs. Sessions will install them with container-setup.sh." +fi + +wait "$android_pid" +say "Android SDK: exit $? ($(tail -1 "$logs/android.log"))" + +say "Done" +exit 0 diff --git a/deploy/linux/cloud-session.sh b/deploy/linux/cloud-session.sh new file mode 100755 index 00000000..ee04aa83 --- /dev/null +++ b/deploy/linux/cloud-session.sh @@ -0,0 +1,102 @@ +#!/bin/bash +# +# Tony +# An intonation analysis and annotation tool +# Centre for Digital Music, Queen Mary, University of London. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. See the file +# COPYING included with this distribution for more information. +# +# Builds Tony in the background at the start of a cloud session, so that +# the build runs while the session reads and edits instead of after: the +# library directories at their pins and build/ configured +# (container-setup.sh), then Tony, the pYIN plugin and the three test +# executables. +# +# The environment's setup script (cloud-environment.sh) has installed the +# packages and Qt already and filled ccache with the libraries' objects, +# so this takes a few minutes at most, and ninja carries on from there +# when the session builds again. The build runs at low priority, so that +# the session's own commands come first. +# +# build/ links with mold when it is installed: the four executables link +# in about a second instead of about twelve with GNU ld, which every +# change to main/ pays. +# +# Usage, from anywhere: +# deploy/linux/cloud-session.sh start start the build and return at +# once; nothing if it is running +# deploy/linux/cloud-session.sh wait wait for it to finish, show the +# end of its log, exit as it did +# +# "start --if-cloud" does nothing outside a cloud session, for a +# SessionStart hook. The log is tmp/cloud-session.log. + +set -u -o pipefail + +cd "$(dirname "$0")/../.." +root=$(pwd) +mkdir -p tmp +log=$root/tmp/cloud-session.log +lock=$root/tmp/cloud-session.lock +script=$root/deploy/linux/cloud-session.sh + +targets="tony pyin.so test-tony-core test-tony-app test-tony-device" + +case "${1:-} ${2:-}" in + "start "|"start --if-cloud") + if [ "${2:-}" = "--if-cloud" ] && [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then + exit 0 + fi + if ! flock -n "$lock" true; then + echo "The background build is running already (log: tmp/cloud-session.log)." + exit 0 + fi + # Detached, so that it outlives the command that started it + setsid flock -n "$lock" "$script" run < /dev/null > /dev/null 2>&1 & + # Until it holds the lock, so that a wait straight after this waits for it + for i in $(seq 50); do + flock -n "$lock" true || break + sleep 0.1 + done + echo "Started the background build: libraries, build/, then $targets (log: tmp/cloud-session.log)." + echo "Run deploy/linux/cloud-session.sh wait before building or testing: two ninjas must not share build/." + ;; + "wait ") + if [ ! -f "$log" ]; then + echo "No background build was started (deploy/linux/cloud-session.sh start)." + exit 1 + fi + flock "$lock" true + tail -5 "$log" + status=$(sed -n 's/^exit:\([0-9]*\)$/\1/p' "$log" | tail -1) + exit "${status:-1}" + ;; + "run ") + # Holding the lock, from start + exec > "$log" 2>&1 + echo "Started $(date -u '+%Y-%m-%d %H:%M:%S') UTC" + start=$SECONDS + if command -v mold > /dev/null; then + # Read by meson setup only: a build/ configured already keeps its linker + export CC_LD=mold CXX_LD=mold + fi + nice -n 10 deploy/linux/container-setup.sh + status=$? + if [ "$status" -eq 0 ]; then + echo + echo "Building $targets" + nice -n 10 ninja -j "$(nproc)" -C build $targets + status=$? + fi + echo "Finished in $((SECONDS - start)) s" + echo "exit:$status" + ;; + *) + echo "Usage: $0 start [--if-cloud] | wait" 1>&2 + exit 2 + ;; +esac diff --git a/docs/building.md b/docs/building.md index b0e0de1e..f0c41e82 100644 --- a/docs/building.md +++ b/docs/building.md @@ -68,28 +68,88 @@ echo "exit:$?" >> tmp/build.log ## On Linux (a cloud session) -Not how the project is developed, but it builds and both suites run; this is how it was done -on 2026-09-25 (Ubuntu 24.04, no sound card): +Not how the project is developed, but it builds and both suites run in Claude's cloud +container (Ubuntu 24.04, 4 cores, 16 GB, no sound card). Three scripts in `deploy/linux/` +do it: + +- **`cloud-environment.sh` is the cloud environment's setup script.** Its text is pasted + into the environment's settings, with the network access and variables below; the copy in + the repository does nothing by itself. The platform runs it once and keeps a snapshot of + the disk, which later sessions start from, until the script or the allowed hosts change + or about a week has passed. It installs the packages, Qt, ccache and mold, the Android SDK + and NDK when `dl.google.com` is reachable, and spends what is left of four minutes filling + ccache from a build of the libraries. The snapshot is kept only when the script ends + within about five minutes, so any change to it has to keep to that. Its logs are in + `/var/log/tony-environment/`. +- **`container-setup.sh`** makes any fresh Ubuntu 24.04 able to build: packages, Qt, the + library directories at their pins, `meson setup build`. Safe to run again. After the + environment's snapshot it only checks out the libraries and configures. +- **`cloud-session.sh start` is the first command of a cloud session.** It runs + `container-setup.sh` and then builds everything into `build/` in the background, at low + priority: about 4 minutes, while the session reads. The log is `tmp/cloud-session.log`. + `cloud-session.sh wait` waits for it and exits as it did. **Wait before the first build + or test**: two ninjas must not work in one build directory. + +The environment's settings: + +- Network access **Custom**, with the default list of package hosts, and `dl.google.com` + added for the Android branch (the SDK, the NDK, and Gradle's Google repository, which + `maven.google.com` redirects to). GitHub, conda-forge and Ubuntu's archive are in the + default list; hg.sr.ht, download.qt.io and Qt's mirrors are not. +- Variables `BASH_DEFAULT_TIMEOUT_MS=600000` and `BASH_MAX_TIMEOUT_MS=1800000`, so that a + build or a suite run is not moved to the background after the tool's default two minutes, + and a 30-minute timeout can be given at all. + +Then, from the repository root: + +```sh +ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app > tmp/build.log 2>&1 +echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log +deploy/linux/run-tests.sh test-tony-core # about a second +deploy/linux/run-tests.sh test-tony-app # about a minute +``` + +No `.exe` on Linux; the plugin target is `pyin.so`. `run-tests.sh` runs an executable as +several processes, each with a shard of every suite ([testing.md](testing.md#running)). +The one-process runs of AGENTS.md work too, from `build/`. + +Measured on 2026-09-26: + +| | | +| --- | --- | +| Full build, nothing in ccache | 6.6 minutes: 1570 CPU-seconds, nearly all compiling | +| Full build, everything in ccache | 4 to 6 seconds | +| A session's first build, with the setup script's ccache | 3.7 minutes, in the background | +| Linking Tony and the three test executables | 3 s with mold, 12 s with GNU ld | +| App suite | 356 s in one process, 54 s in eight | + +Why each part is as it is: -- Packages: the `apt-get install` list of `.github/workflows/linux.yml` (`smlnj` and - `mercurial` are not needed, and `libboost-dev` does for `libboost-all-dev`), plus - `librubberband-dev`, `libjack-jackd2-dev`, `libasound2-dev`, `libopusenc-dev`, `meson`. - **Qt 6.11 from conda-forge, not Ubuntu's 6.4.** Under 6.4 the string-based connects of `Analyser` with `sv::` types do not resolve ("No such slot Analyser::layerCompletionChanged(ModelId)"), so pYIN's completion never arrives and every analysing test times out. download.qt.io's mirrors are blocked by the session's proxy; - conda-forge is not: - `micromamba create -p /opt/qt611 -c conda-forge qt6-main=6.11.1`, then a directory with - links to only its `Qt6*.pc` files, so that nothing else of conda's is picked up: - `PKG_CONFIG_PATH= meson setup build_qt611`, and - `LD_LIBRARY_PATH=/opt/qt611/lib` to run. -- The libraries by `git clone` at the pins of `repoint-lock.json`. sourcehut (the `hg` - ones) was unreachable; their GitHub mirrors (`github.com/breakfastquay/...`) are at the - same tips. -- `-j 4` on four cores; the whole build takes about 20 minutes. Run the app suite with - nothing else building: it records in real time. + conda-forge is not. Only Qt's own `.pc` files are put on meson's pkg-config path + (`/opt/qt6-conda/tony-pkgconfig`), so that nothing else of conda's is picked up; meson + puts Qt's library directory in the executables' RPATH. +- **The Mercurial libraries come from their GitHub mirrors.** hg.sr.ht is blocked, and + `repoint-lock.json` pins them by Mercurial hash, which the mirrors do not carry. + `container-setup.sh` has a table from pin to mirror commit and stops at a pin it does not + know. +- **ccache**, which meson uses by itself when it is installed, with `hash_dir = false` + (`/etc/ccache.conf`). With `-g` every result's key otherwise holds the build directory, + and a second build directory or a worktree found 0.4 % of a full cache. The compiler's + name is part of the key too: a directory configured with `CC=gcc CXX=g++` finds nothing + that meson's own `cc` and `c++` put there. Configure through `container-setup.sh`. +- **mold**: `cloud-session.sh` has `meson setup` take it (`CC_LD=mold CXX_LD=mold`) for a + new `build/`; a build directory keeps the linker it was set up with. Every change to + `main/` relinks all four executables. +- **Run the app suite with nothing else building**: it records in real time. - Four tests of `TestTakesFile` fail on Linux and nowhere else: they are about Windows paths (backslashes, drive letters, case). +- Measured and left alone: `-g1` compiles svcore in 19 % less time than `-g`, but Windows + builds `debugoptimized`, with full debug information; clang is no faster than GCC; and a + unity build fails in the libraries, which define the same names in several files. ## What is particular about this `meson.build` From b1b8f080010112ff02183fc3167e79874acdfc35 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:28:50 +0000 Subject: [PATCH 153/275] feat: dev checks for recording from a position, the lead-in, pre-roll and auto-stop Two more stages of the dev run keep the session: a re-record starting inside an earlier punch-in, its lead-in over earlier material, and a punch-in at 1 s with a 3 s pre-roll. A snapshot of the take's file, pitch, notes and coverage before and after each is compared with TakeDiff. Items 7 (audio bit-identical and events unchanged outside the new range), 12 (nothing before P changed, and nothing of the take heard in the lead-in's silent gaps), 13 (playback from 0, a shorter countdown, placed right) and 14 (stopped by itself within the margin and a poll, coverage exactly the selection, no dialog) are judged; items 1, 2 and 4 now cover every punch-in. The runner's plan gets a pre-roll of its own. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 24 + docs/calibrate-audio.md | 6 +- main/AudioCheckRunner.cpp | 16 +- main/AudioCheckRunner.h | 23 +- main/MainWindow.cpp | 3 +- main/MainWindow.h | 10 +- main/dev/DevChecks.cpp | 1171 ++++++++++++++++++++++----- main/dev/DevChecks.h | 140 +++- main/test/TestAudioCheck.h | 28 + main/test/TestDevChecks.h | 233 +++++- 10 files changed, 1403 insertions(+), 251 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index eacaf3ca..efd3de30 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -376,3 +376,27 @@ starts (`m_watched`, cleared per stage); stage 1's kept in `m_freshWatched`. Left open: at −20 ms item 3 passes or fails with the hop phase (the test allows both). The gap check was seen failing on a non-silent reference, not a take played back out: stage 1 has no take audio where it plays (C1c's re-record has). Dot spread 40–230 ms. + +### Phase C1c — 2026-09-26 +Built: `AudioCheckRunner::Plan::preRoll` (default `kPreRollSeconds`, negative refused), +handed to `MainWindow::m_audioCheckPreRoll`, which `wantedPreRollFrames()` reads. +`DevChecks` stages 2 "Re-record" [19.2, 21.2] and 3 "Pre-roll near the start" [1.0, 4.2] +with 3 s (both `keepSession`), a `Snapshot` (take file, pitch, notes, coverage) before and +after each; items 7 `record_from_a_position`, 12 `nothing_heard_or_changed_in_the_lead_in`, +13 `pre_roll_near_the_start`, 14 `record_into_selection_stops_by_itself`. Items 1, 2, 4 now +cover every punch-in (`runs()`, numbered 1–4 along the run); item 1's reopen compares with +the file judged just before the save over all ranges. C1b's gap logic is `gapLooks(until)`. +Choices: P = 19.2 s: shortest range judging 20.1 s, a gap (18.8–19.2) in the lead-in +(16 looks), a whole note before P − 0.25. Item 7 excuses only the selection; item 12 is the +same before P, plus the looks that end by P. Item 14: raw recording frames minus (round +trip + start gap + R + E − P), within [0, 0.25 s + take timer interval + one block], not via +`TakeTiming`, whose margin it checks. Item 13: highest countdown shown ≤ ceil(min(3 s, P) ++ round trip + start gap + 50 ms), ending at 1. The fault test cancels at stage 3. +Found: on a noiseless loopback the take equals the reference, so one played out shows in +no gap: `loopbackInARoom()` adds −60 dBFS noise (pass and fault runs). The countdown reads +1, 2, 1: `record()` shows it before the round trip is known (harmless, reported). +The next phase must know: a new recording stage joins `runs()` for items 1, 2 and 4; a +cancelled run still works out the checks of the stages it finished; the runner repeats +`progress(Recording)` as the seconds left tick down. +Left open: an overwrite question would come inside `record()`, before the observer starts, +so item 14 cannot see it. Items 3 and 5 judge stage 1 only. `test-tony-dev` about 135 s. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 1110bdd0..1f086478 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -117,8 +117,8 @@ The numbers are those of `docs/manual-checklist.md` before that merge. | 9 | Stop on a 4-minute song | **Automated** | A generated 4-minute reference, punch-in near the end. Time from Stop to new pitch, against a threshold. Pitch outside the range unchanged, which proves the ranged path ran. | | 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. | | 11 | Is 3 s right, is the countdown readable | **Manual** | A judgement. | -| 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. | -| 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown starts at 1, placement is right. | +| 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. *Found in C1c:* on a noiseless loopback the earlier take is silent wherever the reference is, so a take played out shows in no gap; the app test gives the fake a −60 dBFS noise floor, as a room gives a real mic. | +| 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown counts only the 1 s there is (*found in C1c:* plus the round trip, by design, so it starts at 2; for the instant before the round trip is known it shows 1), placement is right. | | 14 | Record into Selection stops by itself | **Automated** | Stops within 0.25 s plus one poll of the end. The coverage added is exactly the selection. No dialog. | | 15 | The practice loop | **Measured** | *Automated:* it stops by itself, the playhead is back at P, and Play plays the take. *Yours:* "is anything else needed?". | | 16 | Constrain Playback to Selection with a pre-roll | **Measured** | Reports whether the lead-in was played in full. The decision is yours. | @@ -322,7 +322,7 @@ marked "Done" when it is committed. own, `test-tony-dev`, run when a change touches what the checks drive (the user's decision). The phases below were cut again (§4). - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. + - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. Done. 5. **C2** Long song and joins: items 9 and 10. 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. 7. **Release build** by the lead (§8, "Release builds must stay clean"). diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index a81842ff..49a984e1 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -189,6 +189,13 @@ AudioCheckRunner::start(const Plan &plan) return false; } + // Written so that a NaN fails too + if (!(plan.preRoll >= 0.0)) { + cerr << "AudioCheckRunner::start: a pre-roll of " << plan.preRoll + << " s" << endl; + return false; + } + if (plan.keepSession && !m_window->getMainModel()) { cerr << "AudioCheckRunner::start: there is no session to record into" << endl; @@ -219,6 +226,9 @@ AudioCheckRunner::start(const Plan &plan) cerr << ", placed with a round trip of its own, " << m_plan.roundTrip * 1000.0 << " ms"; } + if (m_plan.preRoll != kPreRollSeconds) { + cerr << ", with a pre-roll of " << m_plan.preRoll << " s"; + } cerr << endl; // Nothing is done before the first poll, so that however the run @@ -458,6 +468,7 @@ AudioCheckRunner::startPunchIn() m_window->m_audioCheckTakes = true; m_window->m_audioCheckRoundTrip = m_plan.roundTrip; + m_window->m_audioCheckPreRoll = m_plan.preRoll; m_window->record(); if (m_step != step) return; @@ -597,11 +608,11 @@ AudioCheckRunner::currentProgress() const p.punchIn = m_punchIn + 1; } - // The lead-in of a take to come is the whole of the check's, unless + // The lead-in of a take to come is the whole of the plan's, unless // its range starts sooner than that for (int i = m_punchIn; i < int(m_punchIns.size()); ++i) { const LatencyCheck::PunchIn &range = m_punchIns[i]; - double seconds = std::min(kPreRollSeconds, range.start) + + double seconds = std::min(m_plan.preRoll, range.start) + (range.end - range.start); if (i == m_punchIn) { if (m_step == Step::AnalysingTake) continue; @@ -705,6 +716,7 @@ AudioCheckRunner::clearOverride() { m_window->m_audioCheckTakes = false; m_window->m_audioCheckRoundTrip = -1.0; + m_window->m_audioCheckPreRoll = kPreRollSeconds; } bool diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index 4b5bb2c2..c3aadb09 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -93,10 +93,11 @@ struct AudioCheckResult * any take, which is what is being measured. * * The takes are recorded with Record into Selection, Play Reference - * While Recording and a pre-roll of kPreRollSeconds, whatever the - * toolbar says: MainWindow::record() and the rest consult an override - * the runner sets for each of its takes, since the toolbar's toggles - * write the user's settings. A punch-in's range is made the selection + * While Recording and the plan's pre-roll (kPreRollSeconds unless it + * says otherwise), whatever the toolbar says: MainWindow::record() and + * the rest consult an override the runner sets for each of its takes, + * since the toolbar's toggles write the user's settings. A punch-in's + * range is made the selection * (the previous one cleared, then this one selected: "Select" steps in * the history, as when the user selects), and record() is called; the * take stops itself at the end of the selection, through the same path @@ -127,7 +128,7 @@ class AudioCheckRunner : public QObject Q_OBJECT public: - /// The lead-in of the check's takes + /// The lead-in of the check's takes, unless the plan asks for another static constexpr double kPreRollSeconds = 1.0; /// How often the runner looks at how a step is going @@ -170,8 +171,13 @@ class AudioCheckRunner : public QObject /// stored, and the window goes on saying it uses its own double roundTrip; + /// The pre-roll this run's takes ask for, in seconds. As the + /// user's does, it gets shorter near the start of the song + /// (TakeTiming::preRollBefore()) + double preRoll; + Plan() : punchIns(0), eventsEach(0), keepSession(false), - roundTrip(-1.0) { } + roundTrip(-1.0), preRoll(kPreRollSeconds) { } }; /// The steps of a run, in order; the last two come once for each @@ -237,8 +243,9 @@ class AudioCheckRunner : public QObject /** * Begin a run. False, with nothing started, if one is running * already, if a take is being recorded, if the plan's punch-ins - * cannot be recorded (punchInsOf()), or if it keeps the session and - * there is none. Otherwise finished() comes once, at the end, + * cannot be recorded (punchInsOf()), if its pre-roll is negative, or + * if it keeps the session and there is none. Otherwise finished() + * comes once, at the end, * however the run ends. * * A run that replaces the session asks the user whether to save it diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 59f5c59f..5ab3511d 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -213,6 +213,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_audioCheck(nullptr), m_audioCheckTakes(false), m_audioCheckRoundTrip(-1.0), + m_audioCheckPreRoll(AudioCheckRunner::kPreRollSeconds), #ifdef TONY_DEV_CHECKS m_devChecks(nullptr), #endif @@ -4609,7 +4610,7 @@ MainWindow::wantedPreRollFrames() const double seconds = 0.0; if (m_audioCheckTakes) { // The audio check's takes have a lead-in of their own - seconds = AudioCheckRunner::kPreRollSeconds; + seconds = m_audioCheckPreRoll; } else { if (!m_preRoll || !m_preRoll->isChecked()) return 0; QSettings settings; diff --git a/main/MainWindow.h b/main/MainWindow.h index c9e450a3..9a57e1cc 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -914,9 +914,9 @@ protected slots: // The audio check, and the override it sets for each take of its // own: Record into Selection, Play Reference While Recording and a - // pre-roll of AudioCheckRunner::kPreRollSeconds, whatever the toolbar - // says. Not by setting the toggles, which write the user's settings. - // record(), recordingStarted() and wantedPreRollFrames() consult it + // pre-roll of its own, whatever the toolbar says. Not by setting the + // toggles, which write the user's settings. record(), + // recordingStarted() and wantedPreRollFrames() consult it AudioCheckRunner *m_audioCheck; bool m_audioCheckTakes; @@ -926,6 +926,10 @@ protected slots: // Read with m_audioCheckTakes, and never by latencyInUse() double m_audioCheckRoundTrip; + // The pre-roll the check's takes ask for, in seconds + // (AudioCheckRunner::Plan::preRoll). Read with m_audioCheckTakes + double m_audioCheckPreRoll; + #ifdef TONY_DEV_CHECKS // The development checks, which drive the audio check and the // session; made with the window, deleted in its destructor after the diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index 7af008b2..acd358e7 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -19,6 +19,8 @@ #include "../MainWindow.h" #include "../RealtimePitchTracker.h" #include "../SingingTakes.h" +#include "../TakeDiff.h" +#include "../TakeTiming.h" #include "audio/AudioCallbackPlaySource.h" #include "audio/AudioCallbackRecordTarget.h" @@ -41,6 +43,7 @@ #include #include +#include #include using std::cerr; @@ -177,6 +180,133 @@ inputsText(const vector &inputs) return DevChecks::tr("inputs %1 and %2").arg(names.join(", ")).arg(last); } +// How many frames an audio file holds, or -1 and why not +sv_frame_t +frameCount(QString path, QString &error) +{ + error = ""; + if (path == "") { + error = DevChecks::tr("its raw recording was not seen"); + return -1; + } + FileSource source(path); + WavFileReader reader(source); + if (!reader.isOK()) { + error = DevChecks::tr("its raw recording \"%1\" could not be read: %2") + .arg(path).arg(reader.getError()); + return -1; + } + return reader.getFrameCount(); +} + +// A time on the reference's timeline in whole frames, as the runner +// selected its punch-ins +sv_frame_t +frameAt(double seconds, sv_samplerate_t rate) +{ + return sv_frame_t(std::llround(seconds * rate)); +} + +QString +rangeText(double from, double to) +{ + return DevChecks::tr("%1 to %2 s").arg(secondsText(from)) + .arg(secondsText(to)); +} + +QString +coverageText(const Coverage &coverage, sv_samplerate_t rate) +{ + QStringList ranges; + for (const Coverage::Range &r : coverage.getRanges()) { + ranges << rangeText(double(r.start) / rate, double(r.end) / rate); + } + return ranges.isEmpty() ? DevChecks::tr("nothing") : ranges.join(", "); +} + +// Where a run's judged sweeps landed, in words, and a problem for each +// one not found or further off than items 1 and 2 allow +QString +offsetsOf(const LatencyCheck::TakeSummary &s, QStringList &problems) +{ + QStringList offsets; + for (const LatencyCheck::EventResult &e : s.events) { + if (!e.arrival.found) { + offsets << DevChecks::tr("not found"); + problems << DevChecks::tr("the sweep at %1 s was not found") + .arg(secondsText(e.expectedSeconds)); + continue; + } + offsets << signedMs(e.arrival.errorSeconds); + if (std::fabs(e.arrival.errorSeconds) > DevChecks::kPlacementSeconds) { + problems << DevChecks::tr("the sweep at %1 s landed %2 off") + .arg(secondsText(e.expectedSeconds)) + .arg(signedMs(e.arrival.errorSeconds)); + } + } + if (s.events.empty()) problems << DevChecks::tr("no sweep was judged"); + return offsets.isEmpty() ? DevChecks::tr("none judged") : offsets.join(", "); +} + +// What TakeDiff::audioOutside() found, in words +QString +audioText(const TakeDiff::AudioDiff &d, sv_samplerate_t rate) +{ + if (d.pass) return DevChecks::tr("the same, bit for bit"); + return DevChecks::tr("%1 frames differ, the first at %2 s, by up to %3") + .arg(d.differences) + .arg(double(d.firstDifference) / rate, 0, 'f', 3) + .arg(levelText(d.largestDifference)); +} + +// How many of the events lie wholly outside the window, as +// TakeDiff::eventsOutside() counts them: what it compared +int +countOutside(const EventVector &events, const Coverage::Range &window) +{ + int n = 0; + for (const Event &e : events) { + const sv_frame_t end = e.getFrame() + + std::max(e.getDuration(), sv_frame_t(1)); + if (end <= window.start || e.getFrame() >= window.end) ++n; + } + return n; +} + +// What TakeDiff::eventsOutside() found, in words: where it looked is +// "where", and the events before the change that lie there +QString +eventsText(const TakeDiff::EventDiff &d, const EventVector &before, + QString what, QString where, sv_samplerate_t rate) +{ + const int compared = countOutside(before, d.window); + if (d.pass) { + return DevChecks::tr("%1 %2 %3, unchanged").arg(compared).arg(what) + .arg(where); + } + return DevChecks::tr("%1 %2 %3: %4 added, %5 removed, %6 changed, the " + "first at %7 s") + .arg(compared).arg(what).arg(where).arg(d.added.size()) + .arg(d.removed.size()).arg(d.changed.size()) + .arg(double(d.firstDifference) / rate, 0, 'f', 3); +} + +// The number of seconds a status text counts down, if it is the lead-in's +// countdown as TakeTiming words it; 0 if it is anything else +int +countdownOf(QString status) +{ + const QRegularExpressionMatch m = + QRegularExpression("[0-9]+").match(status); + if (!m.hasMatch()) return 0; + const int n = m.captured(0).toInt(); + TakeTiming timing; + timing.rate = 1; + timing.position = n; + timing.preRoll = n; + return timing.countdownText(0) == status ? n : 0; +} + } // namespace QString @@ -255,6 +385,18 @@ DevChecks::freshPunchIns() LatencyCheck::PunchIn(16.8, 21.2) }; } +LatencyCheck::PunchIn +DevChecks::reRecording() +{ + return LatencyCheck::PunchIn(19.2, 21.2); +} + +LatencyCheck::PunchIn +DevChecks::nearTheStart() +{ + return LatencyCheck::PunchIn(1.0, 4.2); +} + QString DevChecks::nextScratchFolder(QString directory, QString inUse) { @@ -324,11 +466,14 @@ DevChecks::start(const Options &options) m_fresh = AudioCheckResult(); m_coverageAfterFresh = Coverage(); m_freshWatched.clear(); + m_reRecord = PunchInStage(); + m_nearStart = PunchInStage(); m_pitchBefore.clear(); m_notesBefore.clear(); m_pitchAfter.clear(); m_notesAfter.clear(); m_reopened = false; + m_beforeSave = LatencyCheck::TakeSummary(); m_rejudged = LatencyCheck::TakeSummary(); m_runnerRunning = false; m_runnerResult = AudioCheckResult(); @@ -338,6 +483,22 @@ DevChecks::start(const Options &options) [this]() { beginFreshPunchIns(); }, [this]() { return freshPunchInsDone(); }, kCheckStageTimeoutMs }); + m_stages.push_back({ tr("Re-record"), + [this]() { + beginPunchInStage + (m_reRecord, reRecording(), + AudioCheckRunner::kPreRollSeconds); + }, + [this]() { return punchInStageDone(m_reRecord); }, + kCheckStageTimeoutMs }); + m_stages.push_back({ tr("Pre-roll near the start"), + [this]() { + beginPunchInStage + (m_nearStart, nearTheStart(), + kNearStartPreRollSeconds); + }, + [this]() { return punchInStageDone(m_nearStart); }, + kCheckStageTimeoutMs }); m_stages.push_back({ tr("Save and reopen"), [this]() { beginReopen(); }, [this]() { return reopenDone(); }, @@ -514,6 +675,109 @@ DevChecks::freshPunchInsDone() return true; } +void +DevChecks::beginPunchInStage(PunchInStage &stage, LatencyCheck::PunchIn range, + double preRoll) +{ + // The take as the stages before left it, to compare with what this + // punch-in leaves + stage.before = snapshot(); + + AudioCheckRunner::Plan plan; + plan.layout = m_layout; + plan.ranges = { range }; + plan.keepSession = true; + plan.roundTrip = m_options.roundTrip; + plan.preRoll = preRoll; + + m_watched.clear(); + m_observedPunchIn = 0; + m_runnerResult = AudioCheckResult(); + m_runnerRunning = m_runner && m_runner->start(plan); + if (!m_runnerRunning) { + end(tr("The audio check could not start.")); + } +} + +bool +DevChecks::punchInStageDone(PunchInStage &stage) +{ + if (m_runnerRunning) return false; + + if (m_runnerResult.failure != "") { + end(tr("The punch-in could not be recorded: %1") + .arg(m_runnerResult.failure)); + return false; + } + + // The runner has waited for the take's analysis: its pitch and notes + // are those this punch-in leaves + stage.result = m_runnerResult; + stage.after = snapshot(); + stage.watched = m_watched; + + // The lead-in the window gave the take, which is the last so far, + // and all that the device delivered until the take stopped + stage.preRoll = m_window->m_takePreRoll; + stage.recorded = frameCount + (stage.watched.empty() ? QString() : + stage.watched.front().seen.recordingPath, stage.recordedError); + + stage.done = true; + return true; +} + +DevChecks::Snapshot +DevChecks::snapshot() const +{ + Snapshot s; + sv_samplerate_t rate = 0; + s.audioError = AudioCheckRunner::readTakeFile + (m_window->m_takes->getAudioPath(), s.audio, rate); + s.pitch = takeEvents(Analyser::PitchTrack); + s.notes = takeEvents(Analyser::Notes); + s.coverage = m_window->m_takes->getCoverage(); + return s; +} + +vector +DevChecks::runs() const +{ + vector all; + if (m_haveFresh) { + all.push_back({ &m_fresh, &m_coverageAfterFresh, &m_freshWatched }); + } + for (const PunchInStage *stage : { &m_reRecord, &m_nearStart }) { + if (stage->done) { + all.push_back({ &stage->result, &stage->after.coverage, + &stage->watched }); + } + } + return all; +} + +const DevChecks::Watched * +DevChecks::watchedOf(const Run &run, int i) +{ + for (const Watched &w : *run.watched) { + if (w.punchIn == i) return &w; + } + return nullptr; +} + +vector +DevChecks::punchInsSoFar() const +{ + vector ranges; + for (const Run &run : runs()) { + for (const LatencyCheck::PunchInResult &p : + run.result->summary.punchIns) { + ranges.push_back(p.range); + } + } + return ranges; +} + void DevChecks::beginReopen() { @@ -522,6 +786,23 @@ DevChecks::beginReopen() m_pitchBefore = takeEvents(Analyser::PitchTrack); m_notesBefore = takeEvents(Analyser::Notes); + // And the take's file judged over every punch-in so far, as it will + // be again after the reopen. Not the runs' own judgements: a later + // punch-in over an earlier one's sweep has replaced it + { + vector mono; + sv_samplerate_t rate = 0; + const QString error = AudioCheckRunner::readTakeFile + (m_window->m_takes->getAudioPath(), mono, rate); + if (error != "") { + end(error); + return; + } + m_beforeSave = LatencyCheck::judgeTake + (m_layout, mono.data(), sv_frame_t(mono.size()), rate, + punchInsSoFar()); + } + // The way Save As saves once it has a name, with no dialog but for a // failure: the take's audio is copied into the session's folder if (!m_window->saveSessionToPath(m_sessionPath)) { @@ -574,12 +855,9 @@ DevChecks::reopenDone() end(error); return false; } - vector ranges; - for (const LatencyCheck::PunchInResult &p : m_fresh.summary.punchIns) { - ranges.push_back(p.range); - } m_rejudged = LatencyCheck::judgeTake - (m_layout, mono.data(), sv_frame_t(mono.size()), rate, ranges); + (m_layout, mono.data(), sv_frame_t(mono.size()), rate, + punchInsSoFar()); m_reopened = true; return true; } @@ -640,7 +918,9 @@ DevChecks::evaluate(QString reason) const { return { latencyCheck(reason), phrasesCheck(reason), liveDotsCheck(reason), speakersCheck(reason), - micChannelCheck(reason) }; + micChannelCheck(reason), positionCheck(reason), + leadInCheck(reason), nearStartCheck(reason), + stopsItselfCheck(reason) }; } CheckResult @@ -656,41 +936,45 @@ DevChecks::latencyCheck(QString reason) const return c; } - // Every judged sweep of every punch-in, found and within the - // tolerance - const LatencyCheck::TakeSummary &s = m_fresh.summary; + // Every judged sweep of every punch-in of every run, found and within + // the tolerance; the punch-ins numbered along the run QStringList problems; double largest = 0.0; - for (int i = 0; i < int(s.punchIns.size()); ++i) { - const LatencyCheck::PunchInResult &p = s.punchIns[i]; - QStringList offsets; - for (const LatencyCheck::EventResult &e : s.events) { - if (e.punchIn != i) continue; - if (!e.arrival.found) { - offsets << tr("not found"); - problems << tr("the sweep at %1 s was not found") - .arg(secondsText(e.expectedSeconds)); - continue; + int n = 0; + for (const Run &run : runs()) { + const LatencyCheck::TakeSummary &s = run.result->summary; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const LatencyCheck::PunchInResult &p = s.punchIns[i]; + ++n; + QStringList offsets; + for (const LatencyCheck::EventResult &e : s.events) { + if (e.punchIn != i) continue; + if (!e.arrival.found) { + offsets << tr("not found"); + problems << tr("the sweep at %1 s was not found") + .arg(secondsText(e.expectedSeconds)); + continue; + } + const double offset = e.arrival.errorSeconds; + offsets << signedMs(offset); + if (std::fabs(offset) > std::fabs(largest)) largest = offset; + if (std::fabs(offset) > kPlacementSeconds) { + problems << tr("the sweep at %1 s landed %2 off") + .arg(secondsText(e.expectedSeconds)) + .arg(signedMs(offset)); + } } - const double offset = e.arrival.errorSeconds; - offsets << signedMs(offset); - if (std::fabs(offset) > std::fabs(largest)) largest = offset; - if (std::fabs(offset) > kPlacementSeconds) { - problems << tr("the sweep at %1 s landed %2 off") - .arg(secondsText(e.expectedSeconds)) - .arg(signedMs(offset)); + if (p.judged == 0) { + problems << tr("punch-in %1 judged no sweep").arg(n); } + c.numbers.push_back + ({ tr("offsets, punch-in %1 (%2)").arg(n) + .arg(rangeText(p.range.start, p.range.end)), + offsets.isEmpty() ? tr("none judged") : + offsets.join(", ") }); } - if (p.judged == 0) { - problems << tr("punch-in %1 judged no sweep").arg(i + 1); - } - c.numbers.push_back - ({ tr("offsets, punch-in %1 (%2 to %3 s)").arg(i + 1) - .arg(secondsText(p.range.start)) - .arg(secondsText(p.range.end)), - offsets.isEmpty() ? tr("none judged") : offsets.join(", ") }); } - if (s.punchIns.empty()) problems << tr("no punch-in was judged"); + if (n == 0) problems << tr("no punch-in was judged"); c.numbers.push_back({ tr("largest offset"), signedMs(largest) }); c.numbers.push_back({ tr("round trip used"), unsignedMs(m_fresh.usedRoundTrip) }); @@ -705,7 +989,8 @@ DevChecks::latencyCheck(QString reason) const } // The same file, copied into the session's folder by the save: every - // sweep where it was, to the frame + // sweep where it was just before the save, to the frame + const LatencyCheck::TakeSummary &s = m_beforeSave; const LatencyCheck::TakeSummary &r = m_rejudged; QString offsetsAfter = tr("the same"); bool same = (r.events.size() == s.events.size()); @@ -771,51 +1056,55 @@ DevChecks::phrasesCheck(QString reason) const return c; } - // Each punch-in in the take where it was recorded, placed right, and - // with the start gap of its own stream measured - const LatencyCheck::TakeSummary &s = m_fresh.summary; - const sv_samplerate_t rate = m_fresh.referenceRate; + // Each punch-in of every run in the take where it was recorded, as + // the take was straight after its run, placed right, and with the + // start gap of its own stream measured QStringList problems; - for (int i = 0; i < int(s.punchIns.size()); ++i) { - const LatencyCheck::PunchInResult &p = s.punchIns[i]; + int n = 0; + for (const Run &run : runs()) { + const LatencyCheck::TakeSummary &s = run.result->summary; + const sv_samplerate_t rate = run.result->referenceRate; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + const LatencyCheck::PunchInResult &p = s.punchIns[i]; + ++n; + + // As the runner selected it, in whole frames of the session + const sv_frame_t start = frameAt(p.range.start, rate); + const sv_frame_t end = frameAt(p.range.end, rate); + Coverage::Range held; + if (!run.coverage->getRangeAt(start, held) || + held.start > start || held.end < end) { + problems << tr("punch-in %1 is not all in the take").arg(n); + } - // As the runner selected it, in whole frames of the session - const sv_frame_t start = sv_frame_t(std::llround(p.range.start * rate)); - const sv_frame_t end = sv_frame_t(std::llround(p.range.end * rate)); - Coverage::Range held; - if (!m_coverageAfterFresh.getRangeAt(start, held) || - held.start > start || held.end < end) { - problems << tr("punch-in %1 is not all in the take").arg(i + 1); - } + if (p.found == 0) { + problems << tr("punch-in %1: no sweep found").arg(n); + } else if (std::fabs(p.medianOffset) > kPlacementSeconds) { + problems << tr("punch-in %1 landed %2 off").arg(n) + .arg(signedMs(p.medianOffset)); + } - if (p.found == 0) { - problems << tr("punch-in %1: no sweep found").arg(i + 1); - } else if (std::fabs(p.medianOffset) > kPlacementSeconds) { - problems << tr("punch-in %1 landed %2 off").arg(i + 1) - .arg(signedMs(p.medianOffset)); - } + const TakeLatency t = (i < int(run.result->takes.size()) ? + run.result->takes[i] : TakeLatency()); + if (!t.startGapMeasured) { + problems << tr("punch-in %1's start gap was not measured") + .arg(n); + } - const TakeLatency t = (i < int(m_fresh.takes.size()) ? - m_fresh.takes[i] : TakeLatency()); - if (!t.startGapMeasured) { - problems << tr("punch-in %1's start gap was not measured") - .arg(i + 1); + c.numbers.push_back + ({ tr("punch-in %1, median offset").arg(n), + p.found > 0 ? signedMs(p.medianOffset) : + tr("nothing found") }); + c.numbers.push_back + ({ tr("punch-in %1, start gap").arg(n), + tr("%1 (%2 frames), %3") + .arg(unsignedMs(t.recordingSeconds(t.startGap))) + .arg(t.startGap) + .arg(t.startGapMeasured ? tr("measured") : + tr("estimated only")) }); } - - c.numbers.push_back - ({ tr("punch-in %1, median offset").arg(i + 1), - p.found > 0 ? signedMs(p.medianOffset) : tr("nothing found") }); - c.numbers.push_back - ({ tr("punch-in %1, start gap").arg(i + 1), - tr("%1 (%2 frames), %3") - .arg(unsignedMs(t.recordingSeconds(t.startGap))) - .arg(t.startGap) - .arg(t.startGapMeasured ? tr("measured") : - tr("estimated only")) }); - } - if (s.punchIns.size() < 2) { - problems << tr("%1 punch-ins, not several").arg(s.punchIns.size()); } + if (n < 2) problems << tr("%1 punch-ins, not several").arg(n); if (problems.isEmpty()) { c.verdict = CheckResult::Verdict::Pass; @@ -958,6 +1247,98 @@ DevChecks::liveDotsCheck(QString reason) const return c; } +DevChecks::GapLooks +DevChecks::gapLooks(const TakeObserver::Observation &o, + const TakeLatency &t, sv_samplerate_t rate, + double until) const +{ + GapLooks g; + + // An audio callback takes in a block of input, then hands out a + // block of output. The reference's first block went out from + // where playback started, after the start gap's input, so the + // frames received say which frames of the reference went out + if (!t.startGapMeasured || t.recordingRate <= 0 || rate <= 0) return g; + g.placed = true; + + // The reference's sounds, in seconds + const LatencyCheck::Layout &layout = m_layout; + const double sweepSeconds = + double(LatencyCheck::sweep(layout.rate).size()) / layout.rate; + vector> sounds; + for (const LatencyCheck::Event &e : layout.events) { + const double sweepAt = double(e.sweepStart) / layout.rate; + sounds.push_back({ sweepAt, sweepAt + sweepSeconds }); + sounds.push_back({ double(e.toneStart) / layout.rate, + double(e.toneStart + e.toneLength) / layout.rate }); + } + auto silent = [&sounds](double from, double to) { + for (const auto &sound : sounds) { + if (from < sound.second && to > sound.first) return false; + } + return true; + }; + + const double start = double(o.playbackStart) / rate; + auto played = [&](sv_frame_t received) { + return start + double(received - t.startGap) / t.recordingRate; + }; + + // The levels read at a poll are of the blocks handed out since + // the read before. Those were taken in after the frames counted + // just before that read, but for the one block being handled + // then, which went out after it: one block early. And one late, + // for a resampler between the play source and a device at + // another rate. The output latency plays no part: the levels + // are of what was handed to the device, not of what was heard. + // How long a block is, nothing in the window says (the play + // source is never told the device's), and PortAudio may vary + // it; but each block's input is counted at once, so no block is + // longer than the most frames that came in between two looks. + // Not counting looks held up: the first ones of a take wait for + // the window to set it up, and would count several blocks + sv_frame_t blockFrames = 0; + sv_frame_t heldUp = 0; + const TakeObserver::Sample *previous = nullptr; + for (const TakeObserver::Sample &sample : o.samples) { + if (!sample.recording) continue; + blockFrames = std::max(blockFrames, sample.framesAfter - + sample.framesBefore); + if (previous) { + const sv_frame_t since = + sample.framesBefore - previous->framesAfter; + if (sample.ms - previous->ms <= 2 * TakeObserver::kPollMs) { + blockFrames = std::max(blockFrames, since); + } else { + heldUp = std::max(heldUp, since); + } + } + previous = &sample; + } + if (blockFrames <= 0) blockFrames = heldUp; + const double block = double(blockFrames) / t.recordingRate; + g.margin = block; + + for (size_t k = 1; k < o.samples.size(); ++k) { + const TakeObserver::Sample &a = o.samples[k - 1]; + const TakeObserver::Sample &b = o.samples[k]; + if (!a.outputRead || !b.outputRead) continue; + const double level = std::max(b.outputLeft, b.outputRight); + g.loudest = std::max(g.loudest, level); + const double from = played(a.framesBefore) - block; + const double to = played(b.framesAfter) + block; + if (from < start || to > until || !silent(from, to)) continue; + ++g.looks; + g.loudestInGaps = std::max(g.loudestInGaps, level); + if (level > 0.0 && g.heard <= 0.0) { + g.heard = level; + g.heardFrom = from; + g.heardTo = to; + } + } + return g; +} + CheckResult DevChecks::speakersCheck(QString reason) const { @@ -974,151 +1355,96 @@ DevChecks::speakersCheck(QString reason) const QStringList problems; // The input played back out, by Tony or by the system, reaches the - // mic again a little later: every sweep arrives twice - const LatencyCheck::Echo &echo = m_fresh.summary.echo; - if (echo.heard) { + // mic again a little later: every sweep arrives twice. In any run + const LatencyCheck::Echo *echo = &m_fresh.summary.echo; + for (const Run &run : runs()) { + if (run.result->summary.echo.heard) { + echo = &run.result->summary.echo; + break; + } + } + if (echo->heard) { problems << tr("every sweep arrived a second time, %1 later at " "%2 dB: the input is being played back out, by " "Tony or by the system (\"Listen to this device\")") - .arg(unsignedMs(echo.delaySeconds)) - .arg(echo.levelDb, 0, 'f', 1); + .arg(unsignedMs(echo->delaySeconds)) + .arg(echo->levelDb, 0, 'f', 1); } c.numbers.push_back ({ tr("second arrival"), - echo.heard ? tr("%1 after the sweep, %2 dB, in %3 sweeps") - .arg(unsignedMs(echo.delaySeconds)).arg(echo.levelDb, 0, 'f', 1) - .arg(echo.events) : tr("none heard") }); - - // What Tony played: exactly nothing where the reference is silent, - // so no take and no synth. The reference's sounds, in seconds - const LatencyCheck::Layout &layout = m_layout; - const double sweepSeconds = - double(LatencyCheck::sweep(layout.rate).size()) / layout.rate; - vector> sounds; - for (const LatencyCheck::Event &e : layout.events) { - const double sweepAt = double(e.sweepStart) / layout.rate; - sounds.push_back({ sweepAt, sweepAt + sweepSeconds }); - sounds.push_back({ double(e.toneStart) / layout.rate, - double(e.toneStart + e.toneLength) / layout.rate }); - } - auto silent = [&sounds](double from, double to) { - for (const auto &sound : sounds) { - if (from < sound.second && to > sound.first) return false; - } - return true; - }; + echo->heard ? tr("%1 after the sweep, %2 dB, in %3 sweeps") + .arg(unsignedMs(echo->delaySeconds)).arg(echo->levelDb, 0, 'f', 1) + .arg(echo->events) : tr("none heard") }); - const sv_samplerate_t rate = m_fresh.referenceRate; - const LatencyCheck::TakeSummary &s = m_fresh.summary; + // What Tony played, at every punch-in of every run: exactly nothing + // where the reference is silent, so no take and no synth double loudest = 0.0; double loudestInGaps = 0.0; double margin = 0.0; int gapPolls = 0; QString firstHeard; - for (int i = 0; i < int(s.punchIns.size()); ++i) { - const Watched *w = freshWatched(i); - if (!w) { - problems << tr("punch-in %1 was not watched").arg(i + 1); - continue; - } - const TakeObserver::Observation &o = w->seen; - - // Play Singing Audio: what it said before the take, it says - // after, and the take is heard or not as it says - if (!o.sawAfter) { - problems << tr("punch-in %1 was not seen after it stopped") - .arg(i + 1); - } else { - auto onOff = [](bool on) { return on ? tr("on") : tr("off"); }; - if (o.singingAudioAfter != o.singingAudioBefore) { - problems << tr("Play Singing Audio was %1 before punch-in " - "%2 and %3 after it") - .arg(onOff(o.singingAudioBefore)).arg(i + 1) - .arg(onOff(o.singingAudioAfter)); + int n = 0; + for (const Run &run : runs()) { + const LatencyCheck::TakeSummary &s = run.result->summary; + for (int i = 0; i < int(s.punchIns.size()); ++i) { + ++n; + const Watched *w = watchedOf(run, i); + if (!w) { + problems << tr("punch-in %1 was not watched").arg(n); + continue; } - if (o.takeAudibleAfter != o.singingAudioAfter) { - problems << tr("after punch-in %1 Play Singing Audio is %2, " - "but the take's audio is %3") - .arg(i + 1).arg(onOff(o.singingAudioAfter)) - .arg(o.takeAudibleAfter ? tr("heard") : tr("silent")); + const TakeObserver::Observation &o = w->seen; + + // Play Singing Audio: what it said before the take, it says + // after, and the take is heard or not as it says + if (!o.sawAfter) { + problems << tr("punch-in %1 was not seen after it stopped") + .arg(n); + } else { + auto onOff = [](bool on) { return on ? tr("on") : tr("off"); }; + if (o.singingAudioAfter != o.singingAudioBefore) { + problems << tr("Play Singing Audio was %1 before " + "punch-in %2 and %3 after it") + .arg(onOff(o.singingAudioBefore)).arg(n) + .arg(onOff(o.singingAudioAfter)); + } + if (o.takeAudibleAfter != o.singingAudioAfter) { + problems << tr("after punch-in %1 Play Singing Audio is " + "%2, but the take's audio is %3") + .arg(n).arg(onOff(o.singingAudioAfter)) + .arg(o.takeAudibleAfter ? tr("heard") : tr("silent")); + } + c.numbers.push_back + ({ tr("Play Singing Audio, punch-in %1").arg(n), + tr("%1 before, %2 after, the take %3") + .arg(onOff(o.singingAudioBefore)) + .arg(onOff(o.singingAudioAfter)) + .arg(o.takeAudibleAfter ? tr("heard") : + tr("silent")) }); } - c.numbers.push_back - ({ tr("Play Singing Audio, punch-in %1").arg(i + 1), - tr("%1 before, %2 after, the take %3") - .arg(onOff(o.singingAudioBefore)) - .arg(onOff(o.singingAudioAfter)) - .arg(o.takeAudibleAfter ? tr("heard") : tr("silent")) }); - } - // An audio callback takes in a block of input, then hands out a - // block of output. The reference's first block went out from - // where playback started, after the start gap's input, so the - // frames received say which frames of the reference went out - const TakeLatency t = (i < int(m_fresh.takes.size()) ? - m_fresh.takes[i] : TakeLatency()); - if (!t.startGapMeasured || t.recordingRate <= 0) { - problems << tr("punch-in %1's start gap was not measured, so " - "what it played could not be placed").arg(i + 1); - continue; - } - const double start = double(o.playbackStart) / rate; - auto played = [&](sv_frame_t received) { - return start + double(received - t.startGap) / t.recordingRate; - }; - - // The levels read at a poll are of the blocks handed out since - // the read before. Those were taken in after the frames counted - // just before that read, but for the one block being handled - // then, which went out after it: one block early. And one late, - // for a resampler between the play source and a device at - // another rate. The output latency plays no part: the levels - // are of what was handed to the device, not of what was heard. - // How long a block is, nothing in the window says (the play - // source is never told the device's), and PortAudio may vary - // it; but each block's input is counted at once, so no block is - // longer than the most frames that came in between two looks. - // Not counting looks held up: the first ones of a take wait for - // the window to set it up, and would count several blocks - sv_frame_t blockFrames = 0; - sv_frame_t heldUp = 0; - const TakeObserver::Sample *previous = nullptr; - for (const TakeObserver::Sample &sample : o.samples) { - if (!sample.recording) continue; - blockFrames = std::max(blockFrames, sample.framesAfter - - sample.framesBefore); - if (previous) { - const sv_frame_t since = - sample.framesBefore - previous->framesAfter; - if (sample.ms - previous->ms <= 2 * TakeObserver::kPollMs) { - blockFrames = std::max(blockFrames, since); - } else { - heldUp = std::max(heldUp, since); - } + const TakeLatency t = (i < int(run.result->takes.size()) ? + run.result->takes[i] : TakeLatency()); + const GapLooks g = gapLooks(o, t, run.result->referenceRate, + std::numeric_limits::max()); + if (!g.placed) { + problems << tr("punch-in %1's start gap was not measured, " + "so what it played could not be placed") + .arg(n); + continue; } - previous = &sample; - } - if (blockFrames <= 0) blockFrames = heldUp; - const double block = double(blockFrames) / t.recordingRate; - margin = std::max(margin, block); - for (size_t k = 1; k < o.samples.size(); ++k) { - const TakeObserver::Sample &a = o.samples[k - 1]; - const TakeObserver::Sample &b = o.samples[k]; - if (!a.outputRead || !b.outputRead) continue; - const double level = std::max(b.outputLeft, b.outputRight); - loudest = std::max(loudest, level); - const double from = played(a.framesBefore) - block; - const double to = played(b.framesAfter) + block; - if (from < start || !silent(from, to)) continue; - ++gapPolls; - if (level > loudestInGaps) loudestInGaps = level; - if (level > 0.0 && firstHeard == "") { + loudest = std::max(loudest, g.loudest); + loudestInGaps = std::max(loudestInGaps, g.loudestInGaps); + margin = std::max(margin, g.margin); + gapPolls += g.looks; + if (g.heard > 0.0 && firstHeard == "") { firstHeard = tr("%1 from %2 to %3 s, in punch-in %4") - .arg(levelText(level)).arg(from, 0, 'f', 3) - .arg(to, 0, 'f', 3).arg(i + 1); + .arg(levelText(g.heard)).arg(g.heardFrom, 0, 'f', 3) + .arg(g.heardTo, 0, 'f', 3).arg(n); } } } - if (s.punchIns.empty()) problems << tr("no punch-in was judged"); + if (n == 0) problems << tr("no punch-in was judged"); if (loudest <= 0.0) { problems << tr("no output level was reported while the reference " @@ -1239,6 +1565,451 @@ DevChecks::micChannelCheck(QString reason) const return c; } +CheckResult +DevChecks::positionCheck(QString reason) const +{ + CheckResult c; + c.item = 7; + c.name = "record_from_a_position"; + + const PunchInStage &stage = m_reRecord; + const LatencyCheck::TakeSummary &s = stage.result.summary; + if (!stage.done || s.punchIns.empty()) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Recorded over the end of an earlier punch-in: placed as items 1 and + // 2 ask, and outside the range selected the take's audio is what it + // was, to the bit, and its pitch and notes are, beyond the margin a + // ranged analysis may change. The range is the selection: what the + // splice placed, if the take ran to its end (item 14), and nothing + // the splice wrote past it is excused + const sv_samplerate_t rate = stage.result.referenceRate; + const LatencyCheck::PunchIn &p = s.punchIns[0].range; + const Coverage::Range range(frameAt(p.start, rate), frameAt(p.end, rate)); + QStringList problems; + c.numbers.push_back({ tr("range"), rangeText(p.start, p.end) }); + c.numbers.push_back({ tr("offsets"), offsetsOf(s, problems) }); + + const Snapshot &before = stage.before; + const Snapshot &after = stage.after; + if (before.audioError != "" || after.audioError != "") { + problems << (before.audioError != "" ? before.audioError : + after.audioError); + } else { + const TakeDiff::AudioDiff audio = TakeDiff::audioOutside + (before.audio.data(), sv_frame_t(before.audio.size()), + after.audio.data(), sv_frame_t(after.audio.size()), 1, range); + if (!audio.pass) { + problems << tr("the take's audio outside the range changed"); + } + c.numbers.push_back({ tr("audio outside the range"), + audioText(audio, rate) }); + } + + const TakeDiff::EventDiff pitch = + TakeDiff::eventsOutside(before.pitch, after.pitch, range, rate); + const TakeDiff::EventDiff notes = + TakeDiff::eventsOutside(before.notes, after.notes, range, rate); + const QString where = tr("outside %1") + .arg(rangeText(double(pitch.window.start) / rate, + double(pitch.window.end) / rate)); + if (!pitch.pass) problems << tr("the take's pitch outside the range " + "changed"); + if (!notes.pass) problems << tr("the take's notes outside the range " + "changed"); + if (countOutside(before.pitch, pitch.window) == 0) { + problems << tr("the take had no pitch outside the range to compare"); + } + c.numbers.push_back({ tr("pitch"), eventsText(pitch, before.pitch, + tr("pitch events"), where, + rate) }); + c.numbers.push_back({ tr("notes"), eventsText(notes, before.notes, + tr("notes"), where, rate) }); + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Recorded over part of an earlier punch-in, the take " + "was placed within %1, and outside the range its " + "audio is the same bit for bit, and its pitch and " + "notes beyond %2 s of it.") + .arg(unsignedMs(kPlacementSeconds)) + .arg(TakeDiff::kEventMarginSeconds); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::leadInCheck(QString reason) const +{ + CheckResult c; + c.item = 12; + c.name = "nothing_heard_or_changed_in_the_lead_in"; + + const PunchInStage &stage = m_reRecord; + const LatencyCheck::TakeSummary &s = stage.result.summary; + if (!stage.done || s.punchIns.empty()) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // The re-recording's lead-in played over what an earlier punch-in + // recorded before P. Nothing of that may change: item 7's + // comparisons, their part before P alone, so that a change there is + // told apart from one after the range + const sv_samplerate_t rate = stage.result.referenceRate; + const LatencyCheck::PunchIn &p = s.punchIns[0].range; + const Snapshot &before = stage.before; + const Snapshot &after = stage.after; + const sv_frame_t from = frameAt(p.start, rate); + const sv_frame_t beyond = std::max + (from + 1, sv_frame_t(std::max(before.audio.size(), + after.audio.size())) + + frameAt(1.0, rate)); + const Coverage::Range rest(from, beyond); + QStringList problems; + + if (before.audioError != "" || after.audioError != "") { + problems << (before.audioError != "" ? before.audioError : + after.audioError); + } else { + const TakeDiff::AudioDiff audio = TakeDiff::audioOutside + (before.audio.data(), sv_frame_t(before.audio.size()), + after.audio.data(), sv_frame_t(after.audio.size()), 1, rest); + if (!audio.pass) { + problems << tr("the take's audio before the punch-in changed"); + } + c.numbers.push_back({ tr("audio before %1 s").arg(secondsText(p.start)), + audioText(audio, rate) }); + } + + const TakeDiff::EventDiff pitch = + TakeDiff::eventsOutside(before.pitch, after.pitch, rest, rate); + const TakeDiff::EventDiff notes = + TakeDiff::eventsOutside(before.notes, after.notes, rest, rate); + const QString where = tr("before %1 s") + .arg(secondsText(double(pitch.window.start) / rate)); + if (!pitch.pass) problems << tr("the take's pitch before the punch-in " + "changed"); + if (!notes.pass) problems << tr("the take's notes before the punch-in " + "changed"); + if (countOutside(before.pitch, pitch.window) == 0) { + problems << tr("the take had no pitch before the punch-in to " + "compare"); + } + c.numbers.push_back({ tr("pitch"), eventsText(pitch, before.pitch, + tr("pitch events"), where, + rate) }); + c.numbers.push_back({ tr("notes"), eventsText(notes, before.notes, + tr("notes"), where, rate) }); + + // And while it played, Tony played the reference and nothing else: + // item 4's looks at the output, those that lie wholly before P. The + // take's audio is under them now, and kept silent as in any take + const Watched *w = stage.watched.empty() ? nullptr : &stage.watched[0]; + const TakeLatency t = stage.result.takes.empty() ? TakeLatency() : + stage.result.takes[0]; + if (!w) { + problems << tr("the punch-in was not watched"); + } else { + const GapLooks g = gapLooks(w->seen, t, rate, p.start); + if (!g.placed) { + problems << tr("the start gap was not measured, so what the " + "lead-in played could not be placed"); + } else { + if (g.looks == 0) { + problems << tr("no look at the output during the lead-in " + "fell wholly in one of the reference's " + "silent gaps"); + } + if (g.heard > 0.0) { + problems << tr("during the lead-in Tony played something " + "where the reference is silent: %1 from %2 " + "to %3 s") + .arg(levelText(g.heard)).arg(g.heardFrom, 0, 'f', 3) + .arg(g.heardTo, 0, 'f', 3); + } + c.numbers.push_back + ({ tr("output in the lead-in's silent gaps"), + tr("%1, over %2 looks").arg(levelText(g.loudestInGaps)) + .arg(g.looks) }); + } + } + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Before the punch-in the take's audio is the same bit " + "for bit, and its pitch and notes beyond %1 s of it; " + "and while the lead-in played over the take, Tony " + "played nothing where the reference is silent.") + .arg(TakeDiff::kEventMarginSeconds); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::nearStartCheck(QString reason) const +{ + CheckResult c; + c.item = 13; + c.name = "pre_roll_near_the_start"; + + const PunchInStage &stage = m_nearStart; + const LatencyCheck::TakeSummary &s = stage.result.summary; + if (!stage.done || s.punchIns.empty()) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Less song before the punch-in than the pre-roll asked for: the + // lead-in is all there is of it, and no more + const sv_samplerate_t rate = stage.result.referenceRate; + const LatencyCheck::PunchIn &p = s.punchIns[0].range; + const double room = std::min(kNearStartPreRollSeconds, p.start); + QStringList problems; + c.numbers.push_back({ tr("lead-in"), + tr("%1 s, of the %2 s asked for") + .arg(secondsText(double(stage.preRoll) / rate)) + .arg(secondsText(kNearStartPreRollSeconds)) }); + if (stage.preRoll > frameAt(room, rate)) { + problems << tr("the lead-in was %1 s, longer than the %2 s of song " + "before the punch-in") + .arg(secondsText(double(stage.preRoll) / rate)) + .arg(secondsText(room)); + } + + const Watched *w = stage.watched.empty() ? nullptr : &stage.watched[0]; + const TakeLatency t = stage.result.takes.empty() ? TakeLatency() : + stage.result.takes[0]; + if (!w) { + problems << tr("the punch-in was not watched"); + } else { + const TakeObserver::Observation &o = w->seen; + + // Playback from the start of the song, and the cursor never + // before it. The cursor is where playback started plus what has + // been recorded, so it is read while recording only + bool seen = false; + sv_frame_t lowest = 0; + for (const TakeObserver::Sample &sample : o.samples) { + if (!sample.recording) continue; + lowest = seen ? std::min(lowest, sample.playbackFrame) : + sample.playbackFrame; + seen = true; + } + c.numbers.push_back({ tr("playback from"), + tr("%1 s").arg(secondsText + (double(o.playbackStart) / + rate)) }); + c.numbers.push_back({ tr("cursor, lowest"), + seen ? tr("%1 s").arg(secondsText + (double(lowest) / rate)) + : tr("not seen") }); + if (o.playbackStart != 0) { + problems << tr("playback started at %1 s, not at the start of " + "the song") + .arg(secondsText(double(o.playbackStart) / rate)); + } + if (!seen) { + problems << tr("the take was not seen recording"); + } else if (lowest < 0) { + problems << tr("the cursor was at %1 s, before the start of the " + "song").arg(secondsText(double(lowest) / rate)); + } + + // The countdown as the status bar showed it: in whole seconds, + // the lead-in and the round trip still to come (the singing that + // answers the reference at P arrives a round trip later), down + // to 1. Before the start gap is measured the window counts with + // its estimate, which may differ by a block or two: hence the + // 50 ms + vector counted; + for (const TakeObserver::Sample &sample : o.samples) { + if (!sample.recording) continue; + const int n = countdownOf(sample.status); + if (n > 0 && (counted.empty() || counted.back() != n)) { + counted.push_back(n); + } + } + const double wait = room + (t.recordingRate > 0 ? + double(t.roundTrip + t.startGap) / + t.recordingRate : 0.0); + const int most = int(std::ceil(wait + 0.05)); + QStringList words; + for (int n : counted) words << QString::number(n); + c.numbers.push_back + ({ tr("countdown"), + counted.empty() ? tr("none shown") : + tr("%1; at most %2, for %3 s of lead-in and round trip") + .arg(words.join(", ")).arg(most).arg(secondsText(wait)) }); + if (counted.empty()) { + problems << tr("no countdown was shown"); + } else { + const int highest = + *std::max_element(counted.begin(), counted.end()); + if (highest > most) { + problems << tr("the countdown began at %1, not %2: it " + "counted more lead-in than there is room for") + .arg(highest).arg(most); + } + if (counted.back() != 1) { + problems << tr("the countdown ended at %1, not 1") + .arg(counted.back()); + } + } + } + + c.numbers.push_back({ tr("offsets"), offsetsOf(s, problems) }); + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("With a pre-roll of %1 s asked for at %2 s, playback " + "ran from the start of the song and not before it, " + "the countdown counted only the lead-in there is " + "room for, and the take was placed within %3.") + .arg(secondsText(kNearStartPreRollSeconds)) + .arg(secondsText(p.start)).arg(unsignedMs(kPlacementSeconds)); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::stopsItselfCheck(QString reason) const +{ + CheckResult c; + c.item = 14; + c.name = "record_into_selection_stops_by_itself"; + + vector stages; + for (const PunchInStage *stage : { &m_reRecord, &m_nearStart }) { + if (stage->done && !stage->result.summary.punchIns.empty()) { + stages.push_back(stage); + } + } + if (stages.empty()) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // The window looks this often whether a take into a selection has + // all it needs + const double poll = (m_window->m_takeTimer ? + m_window->m_takeTimer->interval() / 1000.0 : 0.0); + auto seconds = [](double s) { return QString("%1 s").arg(s, 0, 'f', 3); }; + + QStringList problems; + for (const PunchInStage *stage : stages) { + const LatencyCheck::TakeSummary &s = stage->result.summary; + const sv_samplerate_t rate = stage->result.referenceRate; + const LatencyCheck::PunchIn &p = s.punchIns[0].range; + const sv_frame_t start = frameAt(p.start, rate); + const sv_frame_t end = frameAt(p.end, rate); + const QString range = rangeText(p.start, p.end); + const Watched *w = stage->watched.empty() ? nullptr : + &stage->watched[0]; + const TakeLatency t = stage->result.takes.empty() ? TakeLatency() : + stage->result.takes[0]; + + // How far past the selection's end the take recorded: its raw + // recording against what reaches the end, the round trip and the + // lead-in before the selection. The take waits for a margin past + // the end (TakeTiming::shouldStopAt()), and is then stopped by + // the take timer's next look, with the block coming in as it + // looks. Not worked out with TakeTiming, whose margin is part of + // what is checked + if (!w) { + problems << tr("the take at %1 was not watched").arg(range); + } else if (stage->recorded < 0) { + problems << tr("the take at %1: %2").arg(range) + .arg(stage->recordedError); + } else if (t.recordingRate <= 0) { + problems << tr("the take at %1: its rate is not known") + .arg(range); + } else { + const sv_frame_t needed = + t.roundTrip + t.startGap + stage->preRoll + (end - start); + const double past = + double(stage->recorded - needed) / t.recordingRate; + const double allowed = kStopMarginSeconds + poll + gapLooks + (w->seen, t, rate, std::numeric_limits::max()).margin; + c.numbers.push_back + ({ tr("stopped, %1").arg(range), + tr("%1 past the end of the selection, at most %2 allowed") + .arg(seconds(past)).arg(seconds(allowed)) }); + if (past < 0.0) { + problems << tr("the take at %1 stopped %2 before the end of " + "its selection had been recorded") + .arg(range).arg(seconds(-past)); + } else if (past > allowed) { + problems << tr("the take at %1 went on %2 past the end of its " + "selection, more than %3 s, a look of the " + "take timer and a block").arg(range) + .arg(seconds(past)).arg(kStopMarginSeconds); + } + } + + // The take's coverage as it was, and the selection: no more, and + // no less + Coverage expected = stage->before.coverage; + expected.add(start, end); + c.numbers.push_back({ tr("coverage after, %1").arg(range), + coverageText(stage->after.coverage, rate) }); + if (stage->after.coverage != expected) { + problems << tr("after the take at %1 the take covers %2, not %3") + .arg(range).arg(coverageText(stage->after.coverage, rate)) + .arg(coverageText(expected, rate)); + } + + // No question asked, and no dialog of any kind, from the take's + // start to its analysis done + if (w) { + const TakeObserver::Sample *modal = nullptr; + for (const TakeObserver::Sample &sample : w->seen.samples) { + if (sample.modal) { + modal = &sample; + break; + } + } + c.numbers.push_back + ({ tr("dialogs, %1").arg(range), + modal ? tr("one up %1 into the take") + .arg(seconds(modal->ms / 1000.0)) : + tr("none, over %1 looks").arg(w->seen.samples.size()) }); + if (modal) { + problems << tr("a dialog was up %1 into the take at %2") + .arg(seconds(modal->ms / 1000.0)).arg(range); + } + } + } + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Each take into a selection stopped by itself within " + "%1 s of the selection's end, a look of the take " + "timer and a block, added the selection to the take " + "and nothing else, and no dialog came up.") + .arg(kStopMarginSeconds); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + QString DevChecks::writeReport(const DevReport &report) const { diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index 6ca95e84..c41852d3 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -103,16 +103,25 @@ struct DevReport * a reference of its own (replacing the calibration's session * without asking), with the two punch-ins of freshPunchIns() and * the round trip given. - * 2. Save and reopen: the session saved into a scratch folder of this + * 2. Re-record: a run into the same session and take, with the one + * punch-in of reRecording(), over part of stage 1's second. + * 3. Pre-roll near the start: the same with nearTheStart(), and a + * pre-roll longer than the song before it. + * 4. Save and reopen: the session saved into a scratch folder of this * run, the way Save As saves once it has a name, then opened again * and its take's file judged again. * * The checks are worked out when the run ends, from what the stages - * kept: item 1 (latency, also after save and reopen), item 2 (several - * phrases in one take), and from what a TakeObserver saw of each of - * stage 1's punch-ins, items 3 (live dots, and how far behind the - * cursor they appear), 4 (nothing of the take in the speakers) and 5 - * (the mic on input 2). + * kept: items 1 (latency, also after save and reopen) and 2 (several + * phrases in one take) over every punch-in of the run; from what a + * TakeObserver saw of each of stage 1's punch-ins, items 3 (live dots, + * and how far behind the cursor they appear) and 5 (the mic on input + * 2); from what it saw of every punch-in, item 4 (nothing of the take + * in the speakers); from the take before and after stage 2, items 7 + * (record from a position) and 12 (the lead-in); from stage 3, item 13 + * (pre-roll near the start); and from stages 2 and 3, item 14 (Record + * into Selection stops by itself). A run that ends early works out + * each check whose stages it got through, and has the rest Skipped. * * The session saved stays open afterwards, so that the takes can be * looked at; its scratch folder stays with it, and the next run @@ -148,6 +157,15 @@ class DevChecks : public QObject /// this far below the loudest input's static constexpr double kMicChannelDb = 20.0; + /// Stage 3's pre-roll, in seconds: more than there is room for + /// before nearTheStart() + static constexpr double kNearStartPreRollSeconds = 3.0; + + /// Item 14: how far past the end of its selection a take may record + /// before it stops itself, besides a look of the window's take timer + /// and a block of the device, as the checklist asks + static constexpr double kStopMarginSeconds = 0.25; + /// How often a stage is looked at static constexpr int kPollMs = 50; @@ -198,6 +216,25 @@ class DevChecks : public QObject */ static std::vector freshPunchIns(); + /** + * Stage 2's punch-in, in seconds: [19.2, 21.2], over the end of + * stage 1's second and past its first sweep, judging the sweep at + * 20.1 s. Its 1 s lead-in plays over what stage 1 recorded there: + * the last of the tone from 18 s, and from 18.8 s a gap where the + * reference is silent and the take holds only what the mic heard + * besides. Starting this late keeps the range short, gives the + * lead-in a gap to be looked at in, and leaves a whole note of the + * take (18 to 18.8 s) before it for the ranged analysis to leave + * alone. + */ + static LatencyCheck::PunchIn reRecording(); + + /** + * Stage 3's punch-in, in seconds: [1.0, 4.2], judging the sweep at + * 3.1 s, recorded with a pre-roll of kNearStartPreRollSeconds. + */ + static LatencyCheck::PunchIn nearTheStart(); + /** * A new scratch folder in the directory, made: dev-checks-1, * dev-checks-2 and so on, the lowest number free. Every such @@ -299,15 +336,89 @@ class DevChecks : public QObject Coverage m_coverageAfterFresh; std::vector m_freshWatched; - /// Stage 2: the take's pitch and notes before the save and after - /// the reopen, and its file judged again after it + /// The take as it was at one moment: its audio as its file holds it, + /// mixed to one channel (AudioCheckRunner::readTakeFile()), or why + /// it could not be read; its pitch and notes; and its coverage + struct Snapshot { + std::vector audio; + QString audioError; + sv::EventVector pitch; + sv::EventVector notes; + Coverage coverage; + }; + + /// Stages 2 and 3, each a run of the runner's with one punch-in into + /// the take there is: whether it got through, what the runner found, + /// the take before and after, what was seen of the punch-in, the + /// lead-in the window gave it, and how many frames its raw recording + /// holds (-1, and why, if that could not be read) + struct PunchInStage { + bool done; + AudioCheckResult result; + Snapshot before; + Snapshot after; + std::vector watched; + sv::sv_frame_t preRoll; + sv::sv_frame_t recorded; + QString recordedError; + PunchInStage() : done(false), preRoll(0), recorded(-1) { } + }; + PunchInStage m_reRecord; + PunchInStage m_nearStart; + + /// Stage 4: the take's pitch and notes before the save and after + /// the reopen, and its file judged, over every punch-in of the run, + /// just before the save and again after the reopen sv::EventVector m_pitchBefore; sv::EventVector m_notesBefore; sv::EventVector m_pitchAfter; sv::EventVector m_notesAfter; bool m_reopened; + LatencyCheck::TakeSummary m_beforeSave; LatencyCheck::TakeSummary m_rejudged; + /// One of the runs of the runner's that the run got through, in + /// order: what it found, the take's coverage after it, and what was + /// seen of its punch-ins + struct Run { + const AudioCheckResult *result; + const Coverage *coverage; + const std::vector *watched; + }; + std::vector runs() const; + + /// What was seen of a run's punch-in i, counting from 0, or null + static const Watched *watchedOf(const Run &run, int i); + + /// What the output levels read at a punch-in's looks say of the + /// reference's silent gaps (speakersCheck()). Looks that reach past + /// "until", in seconds on the reference's timeline, are left out + struct GapLooks { + /// The start gap was measured, so that the looks could be placed + bool placed; + + /// How far either side of a look what it read may lie, seconds + double margin; + + /// The loudest level read at any look, and at any look lying + /// wholly in a gap, full scale 1; and how many looks did + double loudest; + double loudestInGaps; + int looks; + + /// The first look in a gap that read anything: its level (0 if + /// none did) and where it lay, in seconds + double heard; + double heardFrom; + double heardTo; + + GapLooks() : placed(false), margin(0), loudest(0), loudestInGaps(0), + looks(0), heard(0), heardFrom(0), heardTo(0) { } + }; + GapLooks gapLooks(const TakeObserver::Observation &seen, + const TakeLatency &latency, sv::sv_samplerate_t rate, + double until) const; + void poll(); void runnerFinished(const AudioCheckResult &result); void runnerProgress(const AudioCheckRunner::Progress &state); @@ -320,9 +431,18 @@ class DevChecks : public QObject void beginFreshPunchIns(); bool freshPunchInsDone(); + void beginPunchInStage(PunchInStage &stage, LatencyCheck::PunchIn range, + double preRoll); + bool punchInStageDone(PunchInStage &stage); void beginReopen(); bool reopenDone(); + /// The take as it is now, its models looked up afresh + Snapshot snapshot() const; + + /// Every punch-in of the run so far, in the order recorded + std::vector punchInsSoFar() const; + /// The events of the take's pitch track or notes, from its model /// as it is now sv::EventVector takeEvents(Analyser::Component component) const; @@ -337,6 +457,10 @@ class DevChecks : public QObject CheckResult liveDotsCheck(QString reason) const; CheckResult speakersCheck(QString reason) const; CheckResult micChannelCheck(QString reason) const; + CheckResult positionCheck(QString reason) const; + CheckResult leadInCheck(QString reason) const; + CheckResult nearStartCheck(QString reason) const; + CheckResult stopsItselfCheck(QString reason) const; /// The report file written, or "" if it could not be QString writeReport(const DevReport &report) const; diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 3d23678d..4ddecc3f 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -1037,6 +1037,34 @@ private slots: QCOMPARE(inUse.roundTrip, 0.3); } + // A plan asks for a pre-roll of its own, the check's unless it says + // otherwise: its take has that lead-in, and is placed right with it. + // A negative one is refused + void check_uses_the_pre_roll_it_is_given() { + makeWindow(loopback()); + QCOMPARE(AudioCheckRunner::Plan().preRoll, + AudioCheckRunner::kPreRollSeconds); + + AudioCheckRunner::Plan refused = onePunchIn(); + refused.preRoll = -0.5; + QVERIFY(!m_window->audioCheck()->start(refused)); + QVERIFY(!m_window->audioCheck()->isRunning()); + + AudioCheckRunner::Plan plan = onePunchIn(); + plan.roundTrip = roundTrip / rate; + plan.preRoll = 0.5; + runCheck(plan); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QCOMPARE(r.summary.found, 1); + QVERIFY2(std::fabs(r.summary.medianOffset * rate) <= 4.0, + describe(r).constData()); + QCOMPARE(m_window->takePreRoll(), sv::sv_frame_t(0.5 * rate)); + QVERIFY(!m_window->audioCheckTakes()); + } + // A run that keeps the session records into the one open, the // reference and take of the run before: nothing is written, opened // or asked, though the session is modified. The take keeps the diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index 9b83ceaa..3e629416 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -19,8 +19,9 @@ // Tier 5, as TestAudioCheck: the development checks (DevChecks) on the // real MainWindow, recording from the fake device with its output // looped back into its input. A run records two punch-ins against the -// 40 s dev reference, then saves the session and opens it again: about -// 20 s of real time. +// 40 s dev reference, one over the end of the second, one near the +// start of the song, then saves the session and opens it again: about +// 30 s of real time. // // The fixture is TestAudioCheck's, copied rather than shared. The // application's data directory, where the check writes its references, @@ -29,6 +30,7 @@ // failing run's report never lands among the suites' results. #include "TestMainWindow.h" +#include "TestSignals.h" #include "../AudioCheckRunner.h" #include "../CalibrateAudioDialog.h" @@ -38,7 +40,9 @@ #include "version.h" +#include "base/PlayParameters.h" #include "base/RecordDirectory.h" +#include "layer/Layer.h" #include "transform/ModelTransformerFactory.h" #include "widgets/InteractiveFileFinder.h" @@ -47,6 +51,7 @@ #include #include #include +#include #include #include #include @@ -79,6 +84,9 @@ class TestDevChecks : public QObject QStringList m_stages; std::vector m_checks; + // How often a test's fault was put in + int m_faults = 0; + void makeWindow(FakeAudioIO::Config config) { delete m_window; m_window = new TestMainWindow(config); @@ -86,6 +94,7 @@ class TestDevChecks : public QObject m_finished = 0; m_stages.clear(); m_checks.clear(); + m_faults = 0; connect(m_window->devChecks(), &DevChecks::finished, this, [this](const DevReport &report) { m_report = report; @@ -112,6 +121,17 @@ class TestDevChecks : public QObject return config; } + // The loopback with a room's noise on the input, at -60 dBFS: the + // takes then hold something where the reference is silent, as they + // do on a real device, so that a take played back out shows in the + // output there. Too quiet for the sweep finder, and no pitch for the + // live tracker or pYIN + static FakeAudioIO::Config loopbackInARoom() { + FakeAudioIO::Config config = loopback(); + config.input = TestSignals::whiteNoise(int(10 * rate), 1, 0.001); + return config; + } + QString reportDirectory() { return m_dir.filePath("report"); } QString scratchDirectory() { return m_dir.filePath("scratch"); } @@ -266,13 +286,13 @@ class TestDevChecks : public QObject QVERIFY(!m_window->audioCheck()->isRunning()); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(!m_window->audioCheckTakes()); - QCOMPARE(int(m_report.checks.size()), 5); + QCOMPARE(int(m_report.checks.size()), 9); for (const CheckResult &c : m_report.checks) { QVERIFY2(c.verdict == CheckResult::Verdict::Skipped, describe()); QVERIFY2(c.message.contains(m_report.failure), describe()); } QCOMPARE(lastReportLine(), - QString("Totals: 0 passed, 0 failed, 0 measured, 5 skipped")); + QString("Totals: 0 passed, 0 failed, 0 measured, 9 skipped")); QTest::qWait(500); QCOMPARE(m_finished, 1); @@ -410,13 +430,17 @@ private slots: << "dev-checks-x" << "mine"); } - // The loopback fake, placed with its true round trip: every sweep of - // both punch-ins lands within 2 ms, the session saved and opened - // again holds the same take, and each punch-in measured its own start - // gap. The report ends with its totals, and the session open - // afterwards is the one saved in the scratch folder + // The loopback fake in a room, placed with its true round trip: every + // sweep of every punch-in lands within 2 ms, the session saved and + // opened again holds the same take, and each punch-in measured its own + // start gap. The re-recording changed nothing outside its range and + // nothing before it, and nothing of the take was heard during its + // lead-in; the punch-in near the start played from the start of the + // song; both stopped by themselves. The report ends with its totals, + // and the session open afterwards is the one saved in the scratch + // folder void dev_checks_pass_with_the_true_round_trip() { - makeWindow(loopback()); + makeWindow(loopbackInARoom()); runDevChecks(roundTrip / rate); if (QTest::currentTestFailed()) return; @@ -427,7 +451,7 @@ private slots: qDebug().noquote() << "report:" << line; } QVERIFY2(m_report.failure == "", describe()); - QCOMPARE(int(m_report.checks.size()), 5); + QCOMPARE(int(m_report.checks.size()), 9); const CheckResult *latency = check(1); const CheckResult *phrases = check(2); QVERIFY(latency && phrases); @@ -486,21 +510,97 @@ private slots: QVERIFY2(reportText().contains(words), qPrintable(words)); } - QCOMPARE(m_stages, QStringList() << "1 of 2: Fresh punch-ins" - << "2 of 2: Save and reopen"); + QCOMPARE(m_stages, QStringList() << "1 of 4: Fresh punch-ins" + << "2 of 4: Re-record" << "3 of 4: Pre-roll near the start" + << "4 of 4: Save and reopen"); // Two punch-ins into a reference of the dev layout, recorded in - // the order given - QCOMPARE(int(m_checks.size()), 1); + // the order given; then one over the end of the second, which + // judges the sweep at 20.1 s; then one near the start, which + // judges the sweep at 3.1 s. Items 1 and 2 count all four + QCOMPARE(int(m_checks.size()), 3); const LatencyCheck::TakeSummary &s = m_checks[0].summary; QCOMPARE(int(s.punchIns.size()), 2); QCOMPARE(s.judged, 4); QCOMPARE(s.found, 4); + for (int i : { 1, 2 }) { + QCOMPARE(int(m_checks[i].summary.punchIns.size()), 1); + QCOMPARE(m_checks[i].summary.judged, 1); + QCOMPARE(m_checks[i].summary.found, 1); + } + QVERIFY(std::fabs(m_checks[1].summary.events[0].expectedSeconds - + 20.1) < 1e-4); + QVERIFY(std::fabs(m_checks[2].summary.events[0].expectedSeconds - + 3.1) < 1e-4); + for (QString label : { QString("offsets, punch-in 3 (19.20 to 21.20 " + "s)"), + QString("offsets, punch-in 4 (1.00 to 4.20 " + "s)") }) { + QVERIFY2(number(*latency, label) != "", describe()); + } + QVERIFY2(number(*phrases, "punch-in 4, start gap").endsWith("measured"), + describe()); + + const CheckResult *position = check(7); + const CheckResult *leadIn = check(12); + const CheckResult *nearStart = check(13); + const CheckResult *stops = check(14); + QVERIFY(position && leadIn && nearStart && stops); + QCOMPARE(position->name, QString("record_from_a_position")); + QCOMPARE(leadIn->name, + QString("nothing_heard_or_changed_in_the_lead_in")); + QCOMPARE(nearStart->name, QString("pre_roll_near_the_start")); + QCOMPARE(stops->name, + QString("record_into_selection_stops_by_itself")); + for (const CheckResult *c : { position, leadIn, nearStart, stops }) { + QVERIFY2(c->verdict == CheckResult::Verdict::Pass, describe()); + } + + // Outside the re-recording's range, and before it, the take as it + // was, and something there to compare + QCOMPARE(number(*position, "audio outside the range"), + QString("the same, bit for bit")); + QCOMPARE(number(*leadIn, "audio before 19.20 s"), + QString("the same, bit for bit")); + for (const CheckResult *c : { position, leadIn }) { + for (QString label : { QString("pitch"), QString("notes") }) { + const QString words = number(*c, label); + QVERIFY2(words.endsWith(", unchanged") && + words.section(' ', 0, 0).toInt() > 0, describe()); + } + } + + // The take under the lead-in holds the room's noise where the + // reference is silent, and none of it was heard + const QString leadInGaps = + number(*leadIn, "output in the lead-in's silent gaps"); + QVERIFY2(leadInGaps.startsWith("silence, over ") && + !leadInGaps.endsWith(" 0 looks"), describe()); + + // A lead-in of all the 1 s there is before the punch-in, from the + // start of the song, counted down from 2: 1 s and the round trip + // of 0.28 s, with the start gap. For the moment before the round + // trip is known the count is of the lead-in alone, 1 + QCOMPARE(number(*nearStart, "lead-in"), + QString("1.00 s, of the 3.00 s asked for")); + QCOMPARE(number(*nearStart, "playback from"), QString("0.00 s")); + QCOMPARE(number(*nearStart, "cursor, lowest"), QString("0.00 s")); + QVERIFY2(QRegularExpression("^(1, )?2, 1; at most 2, ") + .match(number(*nearStart, "countdown")).hasMatch(), + describe()); + + // Each stopped within a look of the take timer of when it could, + // and the one near the start added its range to the take + QVERIFY2(number(*stops, "coverage after, 1.00 to 4.20 s") == + "1.00 to 4.20 s, 6.30 to 10.20 s, 16.80 to 21.20 s", + describe()); + QVERIFY2(number(*stops, "dialogs, 19.20 to 21.20 s") + .startsWith("none, over "), describe()); QVERIFY(QFileInfo(m_report.reportPath).fileName() == "DevChecks.txt"); QVERIFY(TakesFile::isInFolder(reportDirectory(), m_report.reportPath)); QCOMPARE(lastReportLine(), - QString("Totals: 4 passed, 0 failed, 1 measured, 0 skipped")); + QString("Totals: 8 passed, 0 failed, 1 measured, 0 skipped")); QVERIFY(m_report.sessionPath != ""); QCOMPARE(m_window->sessionFile(), m_report.sessionPath); @@ -516,8 +616,9 @@ private slots: } // The same with a round trip 20 ms too long: every take is spliced - // from 20 ms too late, and lands 20 ms early. Both items fail, and - // the report shows where the sweeps landed + // from 20 ms too late, and lands 20 ms early. Items 1, 2, 7 and 13, + // which ask where it landed, fail, and the report shows where the + // sweeps landed void dev_checks_fail_with_the_round_trip_off() { makeWindow(loopback()); @@ -557,12 +658,28 @@ private slots: text)); } - // Items 4 and 5 do not depend on where the take is placed. The - // dots are 20 ms early too, which is about as far as item 3 lets - // them be: whether they pass depends on where the tracker's hops - // fall - QVERIFY2(check(4) && check(4)->verdict == CheckResult::Verdict::Pass, - describe()); + // The re-recording and the punch-in near the start landed 20 ms + // early as well, and fail for that alone + for (int item : { 7, 13 }) { + const CheckResult *c = check(item); + QVERIFY2(c && c->verdict == CheckResult::Verdict::Fail, + describe()); + QVERIFY2(c->message.startsWith("the sweep at ") && + c->message.contains(" landed -") && + !c->message.contains(";"), describe()); + QVERIFY2(std::fabs(milliseconds(number(*c, "offsets")) + 20.0) + <= 1.0, describe()); + } + + // Items 4, 5, 12 and 14 do not depend on where the take is + // placed. The dots are 20 ms early too, which is about as far as + // item 3 lets them be: whether they pass depends on where the + // tracker's hops fall + for (int item : { 4, 12, 14 }) { + QVERIFY2(check(item) && + check(item)->verdict == CheckResult::Verdict::Pass, + describe()); + } QVERIFY2(check(5) && check(5)->verdict == CheckResult::Verdict::Measured, describe()); @@ -570,7 +687,7 @@ private slots: check(3) && check(3)->verdict == CheckResult::Verdict::Pass; QCOMPARE(lastReportLine(), QString("Totals: %1 passed, %2 failed, 1 measured, 0 skipped") - .arg(dotsPass ? 2 : 1).arg(dotsPass ? 2 : 3)); + .arg(dotsPass ? 4 : 3).arg(dotsPass ? 4 : 5)); } // The loopback heard a second time, 50 ms later at half the level, as @@ -612,6 +729,70 @@ private slots: .startsWith("input 1 silence, input 2 -"), describe()); } + // The take heard during the re-recording's lead-in, as if Tony did + // not keep it silent: its audio made audible as the runner reports + // that punch-in recording, before the reference starts to play. The + // lead-in plays over what stage 1 recorded, which in a room holds + // noise where the reference is silent, and items 4 and 12 fail on + // the output there; nothing before the punch-in changed. The run is + // cancelled as the stage after the re-recording begins: the checks + // of the stages it got through are worked out all the same + void dev_checks_take_heard_during_the_lead_in() { + makeWindow(loopbackInARoom()); + connect(m_window->audioCheck(), &AudioCheckRunner::progress, + this, [this](const AudioCheckRunner::Progress &state) { + // Reported again as the seconds left go down + if (state.step != AudioCheckRunner::Step::Recording || + m_faults > 0 || m_stages.isEmpty() || + !m_stages.last().endsWith(": Re-record")) { + return; + } + Analyser *take = m_window->analyser2(); + sv::Layer *audio = + take ? take->getLayer(Analyser::Audio) : nullptr; + auto params = audio ? audio->getPlayParameters() : nullptr; + if (params) { + params->setPlayAudible(true); + ++m_faults; + } + }); + connect(m_window->devChecks(), &DevChecks::progress, + this, [this](QString, int n, int) { + if (n == 3) m_window->devChecks()->cancel(); + }); + + runDevChecks(roundTrip / rate); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_faults, 1); + QCOMPARE(m_report.failure, QString("The dev checks were cancelled.")); + QCOMPARE(int(m_checks.size()), 2); + + const CheckResult *speakers = check(4); + const CheckResult *leadIn = check(12); + QVERIFY(speakers && leadIn); + QVERIFY2(speakers->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(speakers->message.startsWith("Tony played something where " + "the reference is silent: -"), + describe()); + QVERIFY2(speakers->message.contains(", in punch-in 3."), describe()); + QCOMPARE(number(*speakers, "second arrival"), QString("none heard")); + + QVERIFY2(leadIn->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(leadIn->message.startsWith("during the lead-in Tony played " + "something where the reference " + "is silent: -"), describe()); + QVERIFY2(!leadIn->message.contains(";"), describe()); + QCOMPARE(number(*leadIn, "audio before 19.20 s"), + QString("the same, bit for bit")); + + // Not reached: the save, and the punch-in near the start + for (int item : { 1, 13 }) { + QVERIFY2(check(item) && + check(item)->verdict == CheckResult::Verdict::Skipped, + describe()); + } + } + // Cancelled during a take: the take stops, the run ends once with // every check skipped, and the user's toggles and stored round trip // are as they were @@ -720,7 +901,7 @@ private slots: QVERIFY(calibration.calibrationUsable()); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); QTRY_VERIFY_WITH_TIMEOUT - (dialog->pageText().contains("Dev checks, stage 1 of 2"), 10000); + (dialog->pageText().contains("Dev checks, stage 1 of 4"), 10000); QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); From 668057fc89da977de3022a00b0e9b44998507b31 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:29:53 +0000 Subject: [PATCH 154/275] docs: calibrate audio, the user's dev run and the driver project; C2 refined The user's dev run on MME: 0.3 ms within a take, 13 ms between two takes, the start gap blind to it. The user chose a lower-latency driver as the next project (rate fix, a bqaudioio fork, a driver type in Tony, measuring), with MME the default until a run shows better; items 1 and 2 keep their tolerance meanwhile. C2's work order puts the long song first in the run, so no saved session is ever replaced. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 55 +++++++++++++++++++++-------- docs/calibrate-audio.md | 38 ++++++++++++++++---- 2 files changed, 72 insertions(+), 21 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index efd3de30..a6e188b5 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,7 +194,7 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). @@ -294,19 +294,46 @@ Read also: `main/dev/DevChecks.{h,cpp}` and `main/dev/TakeObserver.h`; `main/Tak ### C2 — Long song and joins; items 9, 10 (spec §4 rows 9, 10) -To be refined by the lead after C1c. Outline: - -- **9** After "Save and reopen": the long reference (`longLayout()`, 4 minutes) as a new - session; the whole-song analysis time; two punch-ins far apart (as `test-tony-device`, - about 60 s and 150 s), each timed from Stop to its pitch merged. Pass when each is under - half the whole-song time and pitch outside the range is unchanged (the ranged path ran). - These punch-ins also count for items 1 and 2: placement far into a song is where a - rate mismatch shows. -- **10** Two punch-ins meeting in the middle of a held tone of the dev layout: `TakeDiff`'s - step, pitch and note checks at the join, and nothing moved outside ± 0.25 s. C0 found - (from the code) that the join is a 10 ms dip, and that the notes merge by onset may drop - or split the note: measure, report the result as it is, and if it fails on today's code - the app test is an expected failure with the reason, as `default` does for its defects. +Read also: `main/dev/DevChecks.{h,cpp}` (its stages, `runs()`, `Snapshot`); `main/TakeDiff.h` +(`stepAt()`, `pitchAcross()`, `notesAcross()`, `eventsOutside()`); `main/LatencyCheck.{h,cpp}` +for `longLayout()` and the dev layout's held tones; `docs/takes.md` on ranged analysis and +the merge window W; in `main/test/TestRealDevice.h`, `stop_is_quicker_than_a_whole_song` +and how it times the whole-song analysis; spec §4 rows 9 and 10. + +- **Stage "Long song", the first of the run**, before "Fresh punch-ins": the long + reference as a session of its own (a new reference, not `keepSession`), timed from the + session opening to the reference analysed; then two punch-ins far apart in one runner + run (as `test-tony-device`, near 60 s and 150 s), each timed from its Stop to its pitch + merged (the runner's `AnalysingTake` step). First, so that the dev reference then + replaces it as a check's own unsaved session, and the run still ends on the saved dev + session; no saved session is ever replaced. + - **9** Pass when each punch-in's Stop-to-merged time is under half the whole-song + analysis time, and the take's pitch outside the range ± 0.25 s is unchanged + (`eventsOutside()`), which shows the ranged path ran. Numbers: both times. + - These punch-ins join `runs()` for items 1, 2 and 4: placement far into a song is where + a rate mismatch shows. + - **Test time**: a 4-minute reference in every passing test is too slow. Give + `longLayout()` a length parameter (default 240 s, core test for it) and `Options` the + long reference's length; tests use about 60 s with punch-ins to match. Report what + that makes the whole-song and ranged times on the fake. +- **Stage "Joins"**, after "Pre-roll near the start", keeping the session: two punch-ins + in one runner run that meet at J in the middle of one of the dev layout's held tones + (3 s tones after the sweeps at 26.9, 30.9 and 35.2 s). The first holds that tone's + sweep; the second runs on past the next event's sweep so that both are judged, and its + lead-in plays over the first's recording. Snapshots before and after. + - **10** At J: `stepAt()` on the take's file, `pitchAcross()` and `notesAcross()` on the + take's pitch and notes; and `eventsOutside()` over the two ranges together. Numbers: + the step in dB, the largest pitch gap, the notes at J and the nearest note edge. + - C0 found (from the code) that the join is a 10 ms dip, and that the notes merge by + onset may drop or split the note. **Measure and report the result as it is.** If it + fails on today's code, do not fix `Analyser` or the splice: the app test expects that + failure with `QEXPECT_FAIL` and the reason, as `default` does for its defects, and the + report says so. + - On the user's MME, two punch-ins carry different restart offsets (spec §8), so the + join may fail there for that reason too: the message should say which part failed. +- **Tests**: the passing run gains both stages; keep `test-tony-dev` under about 3 minutes + in all (135 s now). Show failure for item 9 (e.g. Stop analysing the whole song) and one + part of item 10 by breaking the code, and undo by hand. ### C3 — Retire `test-tony-device` diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 1f086478..e6d6253a 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -312,7 +312,11 @@ marked "Done" when it is committed. thresholds and the restart-jitter remedy wait on those numbers. *First run, 2026-09-26* (MME, wired mic and headphones): round trip 301 and 295 ms with one earcup to the mic, verdict Unsteady both times (5–15 ms); Scattered with the - mic between both cups. The detailed figures are awaited. + mic between both cups. *A dev run (C1a's):* round trip 303.5 ms; the two sweeps of + each punch-in agree to 0.2–0.3 ms, but punch-in 1 landed at +0.6 ms and punch-in 2 at + −12.9 ms; the measured start gap was 0 frames both times. So the finder is precise, and + what moves is the stream's input-to-output offset at each restart, which the start gap + does not see. The decision is in §8, "Restart jitter". 3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). 4. **Dev-check framework:** - **C0** `TakeDiff`, pure. Done. @@ -334,16 +338,35 @@ marked "Done" when it is committed. - `recording.md`: the latency section. - `open-points.md`. -**Separate task:** fix the device-rate mismatch. For example the record target could ask -for the session's rate, a one-line change to -`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork; or the splice -could convert. The button then shows the fix working on each device. +**Next project, after this branch: a lower-latency driver** (the user's decision, +2026-09-26, from the restart jitter in §8). In order: + +1. **The device-rate mismatch.** A take recorded at 48 kHz is placed frame for frame + into a 44.1 kHz session. The robust fix converts when the take is spliced, whatever + the device's rate; the other way, the record target asking for the session's rate + (`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork), fails + where the device only runs at its mixer's rate. It comes first, because WASAPI opens + at the Windows mixer's rate, usually 48 kHz. The button then shows the fix working. +2. **A `bqaudioio` fork**, `jhhr/bqaudioio` (created 2026-09-26, the remote `jhhr` in + `bqaudioio/`; `repoint-project.json` still takes sourcehut until this phase): + choosing the host API, WASAPI's automatic rate conversion, and the 0.2 s + `suggestedLatency` made settable. +3. **Driver type in Tony:** MME, DirectSound or WASAPI; the device menus list that + type's devices only (today every host API's are listed, and `getDeviceIndex()` takes + the first name that matches, which is MME's); the stored round trip kept per type. +4. **Measure** with Calibrate Audio and a dev run on each type, on the user's PC. + +**MME stays the default** until such a run shows WASAPI (or another type) better. ## 8. Risks - **Restart jitter.** If the spread across punch-ins is ≥ 10 ms, no stored figure fits - every take. The remedy would be to keep the stream running between takes instead of - suspending it in `stop()`, an svapp change. Decide on step 2's numbers. + every take. *Measured on the user's PC (MME):* about 13 ms between two takes, 5–20 ms + over three calibrations, 0.3 ms within a take (§7). Considered: tuning items 1 and 2 + to ±15 ms, and keeping the stream running between takes (an svapp change, which would + make one session's takes agree with each other but not with the reference). Chosen: a + lower-latency driver, the next project (§7). Until then items 1 and 2 keep ±2 ms and + fail on MME, which is the true reading. - **Windows enhancements or echo cancellation** can remove the sweeps. Tony cannot ask for raw capture: bqaudioio is upstream and MME has no raw mode. The NoSignal and Fading verdicts point to *Sound settings ▸ device ▸ Audio enhancements: Off*. @@ -386,6 +409,7 @@ could convert. The button then shows the fix working on each device. | Checkpoint after B3 | The user runs it on Windows when they can; C0 onwards does not wait | | Commits | The lead commits each phase after review and pushes `feat/calibrateaudiotests` | | `test-tony-device` (from `default`) | Its checks move into the dev run, and it is retired (C3) | +| Restart jitter on MME (about 13 ms) | Not tuned away: a lower-latency driver project after this branch; MME the default until a run shows another type better | | Where the dev checks' tests run | `test-tony-dev`, a third executable in dev builds, run when a change touches the take path, the audio check or the dev checks | ## 11. Facts checked in the code From 8ba73a42b657424fa92f7eaa2c4d55c521fd0a94 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:55:53 +0000 Subject: [PATCH 155/275] docs: work orders for the second phone test's fixes and vertical zoom Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index a2fadeb9..34f36661 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -160,6 +160,8 @@ builds happen in the container.) - A5 — Compact touch mode. Done. - A6 — Oboe audio backend. Done. - A7 — Sessions in place on the phone, and fixes from the first phone test. Done. +- A7b — Fixes from the second phone test: menus, the picker, Downloads. +- A4b — Vertical zoom and scroll by touch. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -343,6 +345,41 @@ lifecycle"; [mobile-port.md](mobile-port.md) "Files and sessions"; [takes.md](ta does; save the session if it has a path and is modified. - The 48 kHz play cursor (A1) is not this phase's: the user expects a fix on `default`. +### A7b — Fixes from the second phone test: menus, the picker, Downloads + +The user's second phone test (APK at 3062d25), 2026-09-26. Worked: menu scrolling; the +All files access prompt; Save Session As with a suggested name, saving once access was +granted; playback stopping in the background. Did not: + +- The topmost menu item (Open, in the File menu) is hard to tap: a tap at the very top + edge of the screen hardly registers. Menus must keep clear of the screen's edges and of + the system bars (`QScreen::availableGeometry()`, the window's safe area margins in Qt + 6.9+), with a margin, wherever Qt places them. +- Files in Google Drive cannot be picked: tapping one in the picker does nothing. Suspect + the picker's MIME type filter (Qt turns name filters into `EXTRA_MIME_TYPES`; a provider + disables files whose type is not listed): check `qandroidplatformfiledialoghelper.cpp` + and what Tony's filters become; on Android offer all files and check the type after. +- A file downloaded from Drive into the phone's storage then failed to open, audio as + well as `.ton`, "with the same error as before" (the user suspected the access grant + had been lost). Likely: picks from Downloads (`msf:` ids) and from the picker's Recent + and Audio roots (the media provider's `audio:` ids) have no path in A7's mapping. With + All files access, MediaStore gives their path (the `_data` column, through the + `ContentResolver`); map those too. The refusal message names the provider, so that a + case still unmapped can be reported; and log the URI. + +### A4b — Vertical zoom and scroll by touch + +The same phone test: the pitch track is tiny, because the pane's frequency range is far +wider than the singing, and pinch zooms only the time axis. + +- A pinch zooms each axis by the spread of the fingers along it: horizontal spread the + time axis as now, vertical spread the frequency range, about the frequency under the + pinch centre; a diagonal spread both. A two-finger drag scrolls vertically too. +- The frequency range is the analyser's: `Analyser::getDisplayFrequencyExtents()` / + `setDisplayFrequencyExtents()`, which View > Edit Display Extents + (`MainWindow::editDisplayExtents()`) uses. Check the scale (log for pitch), the limits, + and what the other layers in the pane do when it changes. + ### A8 — Documentation pass - Bring the docs pages up to date from the code and the log: building.md (the container From 6157d2eeb30719d19abc17d0b069acd1c3d64061 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:00:28 +0000 Subject: [PATCH 156/275] build: a sessionstart hook starts the cloud session's build .claude/settings.json runs cloud-session.sh start --if-cloud at the start and resumption of every session. In the cloud it starts the background build and returns at once, telling the session to wait for it before building; elsewhere, the windows machine included, it does nothing. The rest of .claude/ stays out of the repository through .gitignore, and AGENTS.md says so. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- .claude/settings.json | 15 +++++++++++++++ .gitignore | 4 ++++ AGENTS.md | 11 ++++++----- deploy/linux/cloud-session.sh | 13 ++++++++----- docs/building.md | 13 ++++++++----- 5 files changed, 41 insertions(+), 15 deletions(-) create mode 100644 .claude/settings.json diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 00000000..8b170e36 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,15 @@ +{ + "hooks": { + "SessionStart": [ + { + "matcher": "startup|resume", + "hooks": [ + { + "type": "command", + "command": "bash \"$CLAUDE_PROJECT_DIR\"/deploy/linux/cloud-session.sh start --if-cloud" + } + ] + } + ] + } +} diff --git a/.gitignore b/.gitignore index 96440733..451bf503 100644 --- a/.gitignore +++ b/.gitignore @@ -50,3 +50,7 @@ test-svcore-* .notarization-uuid Dockerfile*.gen tmp +# Claude Code's local files (settings.local.json, worktrees); only the +# shared settings, with the cloud session's hook, are tracked +.claude/* +!.claude/settings.json diff --git a/AGENTS.md b/AGENTS.md index c698d12f..06701d3a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,10 +54,10 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. - Give the app suite a tool timeout of 10 minutes. -In a **Linux cloud session** the commands are others: run `deploy/linux/cloud-session.sh -start` first (it builds in the background) and `deploy/linux/cloud-session.sh wait` before -the first build or test; the rest is in -[docs/building.md](docs/building.md#on-linux-a-cloud-session). +In a **Linux cloud session** the commands are others. The session's hook has started a +build in the background; run `deploy/linux/cloud-session.sh wait` before the first build or +test (if it says none was started, run `deploy/linux/cloud-session.sh start` first). The +rest is in [docs/building.md](docs/building.md#on-linux-a-cloud-session). ## Rules for working here @@ -107,7 +107,8 @@ the first build or test; the rest is in ### Git - Commit only when asked, and only with both whole suites green. One commit per coherent - step, staged by file name (never `git add -A`; `.claude/` and `tmp/` stay out). + step, staged by file name (never `git add -A`; `tmp/` and all of `.claude/` but + `settings.json` stay out). - Messages: `feat:` / `fix:` / `test:` / `docs:`, lower case, then a short what-and-why body. End with a `Co-Authored-By` trailer naming the model that wrote the code. - Use the `gh` CLI for anything on GitHub. diff --git a/deploy/linux/cloud-session.sh b/deploy/linux/cloud-session.sh index ee04aa83..cad75c91 100755 --- a/deploy/linux/cloud-session.sh +++ b/deploy/linux/cloud-session.sh @@ -32,11 +32,17 @@ # deploy/linux/cloud-session.sh wait wait for it to finish, show the # end of its log, exit as it did # -# "start --if-cloud" does nothing outside a cloud session, for a -# SessionStart hook. The log is tmp/cloud-session.log. +# "start --if-cloud" does nothing outside a cloud session: it is what the +# SessionStart hook in .claude/settings.json runs, at the start and the +# resumption of every session, including those on the Windows machine. +# The log is tmp/cloud-session.log. set -u -o pipefail +if [ "${1:-} ${2:-}" = "start --if-cloud" ] && [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then + exit 0 +fi + cd "$(dirname "$0")/../.." root=$(pwd) mkdir -p tmp @@ -48,9 +54,6 @@ targets="tony pyin.so test-tony-core test-tony-app test-tony-device" case "${1:-} ${2:-}" in "start "|"start --if-cloud") - if [ "${2:-}" = "--if-cloud" ] && [ "${CLAUDE_CODE_REMOTE:-}" != "true" ]; then - exit 0 - fi if ! flock -n "$lock" true; then echo "The background build is running already (log: tmp/cloud-session.log)." exit 0 diff --git a/docs/building.md b/docs/building.md index f0c41e82..26828e96 100644 --- a/docs/building.md +++ b/docs/building.md @@ -84,11 +84,14 @@ do it: - **`container-setup.sh`** makes any fresh Ubuntu 24.04 able to build: packages, Qt, the library directories at their pins, `meson setup build`. Safe to run again. After the environment's snapshot it only checks out the libraries and configures. -- **`cloud-session.sh start` is the first command of a cloud session.** It runs - `container-setup.sh` and then builds everything into `build/` in the background, at low - priority: about 4 minutes, while the session reads. The log is `tmp/cloud-session.log`. - `cloud-session.sh wait` waits for it and exits as it did. **Wait before the first build - or test**: two ninjas must not work in one build directory. +- **`cloud-session.sh start` runs at the start of every cloud session**, from the + SessionStart hook in `.claude/settings.json` (`start --if-cloud`, which does nothing + outside the cloud). It runs `container-setup.sh` and then builds everything into `build/` + in the background, at low priority: about 4 minutes, while the session reads. The hook + itself returns at once. The log is `tmp/cloud-session.log`. `cloud-session.sh wait` + waits for it and exits as it did. **Wait before the first build or test**: two ninjas + must not work in one build directory. A session with several repositories runs no + repository's hooks; there, run `cloud-session.sh start` by hand. The environment's settings: From de4cb958c9a915d45e0bd0134a81ec76455412fd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:03:52 +0000 Subject: [PATCH 157/275] feat: shift the whole lyrics, by dragging or by a number of seconds Every word moves by the same amount, its length, text and line kept, and never before the start of the reference. With Edit Lyrics on, Shift and a drag anywhere in the row of words moves them all with the pointer, to be lined up by ear; Edit > Shift Lyrics... takes an offset in seconds, such as the exporter's 0.2 s. Either is one step to undo, made of the words before and after, however long the drag. TestUiChecks' stop_then_close_the_window_at_once joins the Linux list of tests that race the analysis: it fails there without this change too. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012YZZce5FzUjNEE1ZPgVtJj --- README.md | 6 +- docs/architecture.md | 20 +- docs/manual-checklist.md | 13 +- docs/open-points.md | 18 +- docs/testing.md | 14 +- main/LyricsEdit.cpp | 42 ++- main/LyricsEdit.h | 35 +- main/LyricsEditor.cpp | 144 +++++++- main/LyricsEditor.h | 56 ++- main/MainWindow.cpp | 62 +++- main/MainWindow.h | 10 + main/test/TestLyricsEdit.h | 92 ++++- main/test/TestMainWindow.h | 28 +- main/test/TestRecordWorkflow.h | 630 ++++++++++++++++++++++++++++++++- 14 files changed, 1116 insertions(+), 54 deletions(-) diff --git a/README.md b/README.md index 2ff60ccc..c895027f 100644 --- a/README.md +++ b/README.md @@ -60,8 +60,10 @@ orange pitch track to compare with it. * the lyrics can be corrected by ear, with the song there to listen to: with Edit -> Edit Lyrics on, drag the start or end of a word in its box, double-click a word to change its text, or right-click to delete a word or - add one between words. Each edit can be undone; the edits are saved with - the session, and Export Lyrics writes them + add one between words. Lyrics that are all early or late move together: + Shift-drag in the row, or Edit -> Shift Lyrics... by a number of seconds. + Each edit can be undone; the edits are saved with the session, and Export + Lyrics writes them * lyrics can be exported from Moises with the Moises-Lyric-Exporter browser extension. Set it to TTML, word by word, with the offset at 0 (otherwise every line is 0.2 s early, and Tony cannot tell): TTML gives every word diff --git a/docs/architecture.md b/docs/architecture.md index d2272236..1ee78b23 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -290,6 +290,7 @@ tools act only on the top layer (`PlotLyrics` also calls itself not editable), s mode can reach the words. The editor is therefore an **event filter on the lyrics' pane**, installed only while Edit > Edit Lyrics is on. In the box row it keeps from the pane only what it acts on: a left press on an edge and the drag it starts, up to the release; a +left press with Shift held anywhere in the row and the drag it starts; a double-click inside a word; a right press, for the words' menu; and moves with no button held, where the cursor and the context help are its own. Everything else reaches the pane as it would without edit mode, so a click still moves the playback cursor. Editing needs @@ -307,7 +308,7 @@ would take the clicks meant for the cursor. command running, so nothing an undo or redo does may make `lyricsEditAllowed()` false: the lyrics' presence and visibility stay out of commands. - **One `ChangeEventsCommand` per edit** ("Move Word Start", "Move Word End", "Change Word - Text", "Add Word", "Delete Word"), holding the model's id and `Event` values, pushed + Text", "Add Word", "Delete Word", "Shift Lyrics"), holding the model's id and `Event` values, pushed done with `addCommand(command, false)`; `CommandHistory` marks the session modified, and the session saves the model as it is. A drag changes the model at every move, so that the word follows the pointer and the layer lays the words out again (svgui fork), and @@ -343,6 +344,23 @@ would take the clicks meant for the cursor. when chosen. - Add Word goes at the **last** frame of the column clicked: the first can be inside the word before, whose end lies in the column after its box. +- **Shifting all the words** (a Shift-drag in edit mode, or Edit > Shift Lyrics..., which + is to be had whenever `lyricsEditAllowed()` is, edit mode on or not): every start and + end moves by one amount, clamped by `LyricsEdit::clampShift()` so that the first word + does not go before frame 0; a shift that comes to 0 pushes nothing. What a press is, + an edge's drag or all the words', is decided at the press, whatever Shift does after. + The command takes **all the words out, then puts all the shifted ones in**: one out + and one in at a time, a shifted word could meet an original still in the model. + A Shift-drag must **not fill its command as it goes**, as an edge drag does: + `ChangeEventsCommand` folds an add and the remove of the same event only when the two + are next to each other, which they never are when all the words go out and come back + at each move, so the command would keep every word's every step and its undo would + replay them all. The drag moves the model itself, with no command, checking at each + move that the model holds exactly the words it last put there (anything else, and the + drag is dropped, as an edge drag is); at the release it puts the words back as they + were and pushes one command from the words at the press to the words at the release. + The dialog's shift, like the text question, looks the lyrics up again after the + dialog (the same model, editing still allowed) and finishes a drag in progress first. - Pane 0 of a reference has no context-help connection (only `newSession()` makes one), so the editor's help goes straight to the status bar, and the editor clears it itself when the pointer leaves the row. diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index dc6b02f6..e7a5034b 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -117,8 +117,8 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w eye and while playing, each box ends where the word does, and the highlight moves with the voice and goes off in the gaps. A word Moises split into syllables is one box; punctuation is on its word. All early or late by the same amount means the reference - is not the recording Moises timed (Tony cannot shift the whole song; an `[offset:]` - line added to an LRC file can). Compare with an LRC export of the same song, gap + is not the recording Moises timed, or the exporter's offset was not 0: Shift Lyrics + (section 4) moves them all. Compare with an LRC export of the same song, gap threshold low. 9. **Finnish text**: ä and ö come out right in the pane, and again after save and reopen. 10. **Show Lyrics and Remove Lyrics**: hiding keeps the words for later, Remove takes them @@ -177,6 +177,15 @@ harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, w 7. **Drag smoothness**: with a whole song's words (hundreds) in view, and again while the reference plays, a drag follows the pointer without stutter and playback does not break up. +8. **Shifting the whole song**: on a real Moises file that is all early or late, with + Edit Lyrics on, Shift-drag anywhere in the row (on a word, an edge or a gap): the + cursor is a closed hand, all the words and the highlight follow the pointer, none + goes before the start of the song, and one Ctrl+Z takes the whole drag back. Is + lining them up by eye and ear while playing easy enough, and does a whole song's worth + move without stutter? Then Edit > Shift Lyrics... with a known offset (0.2 for an + export made with the exporter's default offset, -0.2 the other way): the words move + by that much, the status bar says how far and which way, and an Export writes the + shifted times. Is the dialog's wording clear about which sign is earlier? The questions the automated checks raised, and the facts they established for the decisions above, are in [open-points.md](open-points.md). diff --git a/docs/open-points.md b/docs/open-points.md index e40d9850..60fe6010 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -31,12 +31,16 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - Background music is not saved in the session; it is reloaded by hand. - An old session (before takes) loses its singing track without telling the user why. - Recording that starts before frame 0 of the reference. -- **Lyrics are edited a word at a time**: no shifting of a line or of the whole song, no +- **Lyrics are edited a word at a time, or all together**: the whole song can be shifted + (Shift-drag, Edit > Shift Lyrics...), but not one line or the words from one on; no splitting or merging of words, no editing of line breaks, no syllables (a file's - syllables are joined into their word). Not wanted for now. Lyrics that are all off by - the same amount, because the reference is not the recording they were timed to, can - only be moved by an LRC file's `[offset:]` tag. Import and Remove are not undoable, and - a new import replaces the lyrics, edits made in Tony included, without asking. + syllables are joined into their word). Not wanted for now. Import and Remove are not + undoable, and a new import replaces the lyrics, edits made in Tony included, without + asking. +- **Shift-drag in the lyrics' box row is the editor's while edit mode is on**: in + Navigate mode it is also the pane's "Re-Analyse Area" gesture, which then works only + above the row. A shift is clamped at the first word reaching 0 and says so only by + the amount in the status bar (the dialog's) or by the words stopping (the drag's). - **TTML and LRC only**: no SRT or Moises JSON. Another format is another parser that `parseLyrics()` chooses. @@ -87,7 +91,9 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. mouse button still held) removes the word as that step left it, which is not in the model, and adds the old one back beside the moved one. The word the drag made is still there, so the drag's own check does not see the change. There is no hook before an - undo to end the drag first. + undo to end the drag first. A Shift-drag sees any change and is dropped, but an undo + in the middle of one acts on words the drag has moved and the history does not know + about, so it can leave a word twice in the same way. - **The first click of a double-click on a word moves the playback cursor** there, as any click does; only the second is the editor's. Accepted for now. - **A double-click the pane handles itself** (edit mode off, or between words in edit diff --git a/docs/testing.md b/docs/testing.md index 6c37e01d..03b257a7 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -60,9 +60,10 @@ Windows path would start an escape in the C string. `TestRecordWorkflow`'s `take_analysis_covers_the_range_it_lost`, `range_analysis_torn_down_while_running`, `save_during_ranged_analysis`, `undo_during_analysis_then_redo` and `analyse_now_reanalyses_the_take`, and - `TestUiChecks`' `menus_follow_the_take_by_themselves`, where the analysis finishes - before the race they need can be set up. Which of those six fail changes from run to - run, and so can where: `undo_during_analysis_then_redo` fails + `TestUiChecks`' `menus_follow_the_take_by_themselves` and + `stop_then_close_the_window_at_once` (the latter nearly every time), where the analysis + finishes before the race they need can be set up. Which of those seven fail changes from + run to run, and so can where: `undo_during_analysis_then_redo` fails either before the undo, with no ranged analysis left running, or after the redo, with `analysedRangeStart()` already 0 — the same race. @@ -105,6 +106,8 @@ Windows path would start an escape in the C string. `askForLyricsWordText()` takes a queue of answers (`answerWordText()`, `cancelWordText()`; none left is Cancel) and can run something while the question is open (`whileAskingWordText()`), as a real dialog's event loop lets anything happen. + `askForLyricsShift()` is answered the same way (`answerLyricsShift()`, + `cancelLyricsShift()`, `whileAskingLyricsShift()`). Anything new that asks the user needs such a virtual. `setRecordOverAskedInDialog()` lets the real dialog through instead, for a test that presses its buttons. - Fixture helpers: `makeWindow(config)`, `writeWav()`, `openReference()`, `startTake()` / @@ -149,7 +152,8 @@ The rules of the edits themselves are tested without a window, in `TestLyricsEdi - **`QApplication::sendEvent()` to the pane** (`sendMouse()`, and `hoverAt()`, `pressAt()`, `moveHeldTo()`, `releaseAt()`, `dragFromTo()`, `doubleClickAt()`, - `rightPressAt()` on top of it): an event sent so goes to the pane's event filters first + `rightPressAt()` on top of it, and `shiftPressAt()` ... `shiftDragFromTo()` with Shift + held): an event sent so goes to the pane's event filters first and then to the pane, as real input does. Not `QTest::mouseMove`, which does not carry the buttons held. A double-click is sent as Qt makes one: a press and a release, then `MouseButtonDblClick` in place of the second press, and its release. @@ -171,6 +175,8 @@ The rules of the edits themselves are tested without a window, in `TestLyricsEdi `cleanup()` fails it. - `lyricsModel()` changes the words straight in the model, as setup: no command, nothing marked modified. +- A spy on `CommandHistory::commandExecuted()` counts undos and redos as well as pushes: + count pushes before any undo, or from a count taken after the last one. ### Setup that fails confusingly when it is missing diff --git a/main/LyricsEdit.cpp b/main/LyricsEdit.cpp index a1b53c24..dacb2829 100644 --- a/main/LyricsEdit.cpp +++ b/main/LyricsEdit.cpp @@ -23,12 +23,6 @@ using namespace sv; namespace { -sv_frame_t framesFor(double seconds, sv_samplerate_t rate) -{ - if (rate <= 0) return 0; - return sv_frame_t(std::llround(seconds * rate)); -} - // One past the box's last column. A word too short to reach the next // column is still painted one column wide, and can be hit there int columnAfter(const LyricsEdit::Box &box) @@ -48,6 +42,13 @@ bool isWord(const EventVector &words, int word) } // namespace +sv_frame_t +LyricsEdit::framesFor(double seconds, sv_samplerate_t rate) +{ + if (rate <= 0) return 0; + return sv_frame_t(std::llround(seconds * rate)); +} + sv_frame_t LyricsEdit::minWordFrames(sv_samplerate_t rate) { @@ -300,3 +301,32 @@ LyricsEdit::cleanText(const QString &typed) { return lyricsLabel(typed); } + +sv_frame_t +LyricsEdit::clampShift(const EventVector &words, sv_frame_t wanted) +{ + if (words.empty()) return 0; + + // The earliest start, wherever it is in the vector: the order of the + // words is not relied on + sv_frame_t earliest = words[0].getFrame(); + for (const Event &w : words) earliest = std::min(earliest, w.getFrame()); + + // Back as far as frame 0, and not back at all for words that are + // before it already, so the answer is between 0 and wanted + const sv_frame_t lowest = std::min(sv_frame_t(0), -earliest); + return std::max(wanted, lowest); +} + +EventVector +LyricsEdit::shifted(const EventVector &words, sv_frame_t wanted) +{ + const sv_frame_t by = clampShift(words, wanted); + + // One amount for all keeps every word's place among the others, and + // so the order the model sorts them in + EventVector moved; + moved.reserve(words.size()); + for (const Event &w : words) moved.push_back(w.withFrame(w.getFrame() + by)); + return moved; +} diff --git a/main/LyricsEdit.h b/main/LyricsEdit.h index 833b83d4..acf8ece2 100644 --- a/main/LyricsEdit.h +++ b/main/LyricsEdit.h @@ -26,7 +26,8 @@ /** * What editing a word of the lyrics may do: which edge the pointer is * on, how far an edge can be dragged, where a new word goes and which - * line it joins, and what a typed text becomes. + * line it joins, what a typed text becomes, and how far all the words + * can be shifted together. * * The words are the events of the lyrics' region model, in the order * the model gives them (by start): a word's start is its frame and its @@ -45,9 +46,15 @@ namespace LyricsEdit constexpr double newWordSeconds = 0.5; /** - * Those two in frames at this rate, to the nearest frame, as - * lyricsToEvents() rounds times: 882 and 22050 frames at 44.1 kHz. - * 0 for a rate that is not positive. + * Seconds in frames at this rate, to the nearest frame, as + * lyricsToEvents() rounds times; negative seconds give negative + * frames. 0 for a rate that is not positive. + */ + sv::sv_frame_t framesFor(double seconds, sv::sv_samplerate_t rate); + + /** + * The two above in frames at this rate, as framesFor() gives them: + * 882 and 22050 frames at 44.1 kHz. */ sv::sv_frame_t minWordFrames(sv::sv_samplerate_t rate); sv::sv_frame_t newWordFrames(sv::sv_samplerate_t rate); @@ -208,6 +215,26 @@ namespace LyricsEdit * left, which the edit must refuse. */ QString cleanText(const QString &typed); + + /** + * How far all the words may move together, for a shift of wanted + * frames (negative: earlier): as far as wanted, except that the + * word starting earliest does not go back past frame 0. Words that + * already start before frame 0, which no parser makes, do not go + * back at all. No limit later. 0 with no words. + * + * The answer is always between 0 and wanted. + */ + sv::sv_frame_t clampShift(const sv::EventVector &words, + sv::sv_frame_t wanted); + + /** + * The words, every one moved by clampShift(words, wanted), start + * and end alike: lengths, texts and lines are kept, and so is their + * order. + */ + sv::EventVector shifted(const sv::EventVector &words, + sv::sv_frame_t wanted); } #endif diff --git a/main/LyricsEditor.cpp b/main/LyricsEditor.cpp index c37cdef6..432f4856 100644 --- a/main/LyricsEditor.cpp +++ b/main/LyricsEditor.cpp @@ -44,6 +44,16 @@ pushDone(ChangeEventsCommand *command) } } +// Straight in the model, with no command: all out, then all in, so that +// no word put in meets one of those still to come out +void +replaceWords(RegionModel *model, const EventVector &from, + const EventVector &to) +{ + for (const Event &e : from) model->remove(e); + for (const Event &e : to) model->add(e); +} + } LyricsEditor::LyricsEditor(LyricsTrack *lyrics, QObject *parent) : @@ -51,12 +61,14 @@ LyricsEditor::LyricsEditor(LyricsTrack *lyrics, QObject *parent) : m_lyrics(lyrics), m_enabled(false), m_dragging(false), + m_shifting(false), m_dragWord(-1), m_dragPart(LyricsEdit::Part::Nothing), m_dragEdgeFrame(0), m_dragPressFrame(0), m_dragCommand(nullptr), m_cursorSet(false), + m_cursorShape(Qt::ArrowCursor), m_helpShown(false) { } @@ -66,6 +78,7 @@ LyricsEditor::~LyricsEditor() // Nothing is pushed from here: the history may be going as well abandonDrag(); m_dragging = false; + m_shifting = false; restoreCursor(); if (m_pane) m_pane->removeEventFilter(this); } @@ -383,6 +396,21 @@ LyricsEditor::mousePressed(QMouseEvent *e) RegionLayer *layer = currentLayer(); if (!layer || layer->getModel().isNone()) return false; + // Shift held: all the words, from anywhere in the row, an edge or a + // word or the space between. A double-click with Shift held is a + // press here as well, as on an edge + if ((e->modifiers() & Qt::ShiftModifier) && !words.empty()) { + m_dragging = true; + m_shifting = true; + m_dragModel = layer->getModel(); + m_dragWords = words; + m_shiftCurrent = words; + m_dragPressFrame = m_pane->getFrameForX(pos.x()); + m_dragCommand = nullptr; + setCursorShape(Qt::ClosedHandCursor); + return true; + } + if (!hit.isEdge()) { // A double-click in a word, away from its edges, is for its @@ -398,6 +426,7 @@ LyricsEditor::mousePressed(QMouseEvent *e) } m_dragging = true; + m_shifting = false; m_dragModel = layer->getModel(); m_dragWords = words; m_dragWord = hit.word; @@ -414,7 +443,7 @@ LyricsEditor::mousePressed(QMouseEvent *e) // edit at all m_dragCommand = nullptr; - setEdgeCursor(); + setCursorShape(Qt::SizeHorCursor); return true; } @@ -465,23 +494,29 @@ LyricsEditor::hover(QPoint pos) return false; } + // Shift-drag moves all the words from anywhere in the row, which the + // help says wherever the pointer is if (hit.isEdge()) { - setEdgeCursor(); + setCursorShape(Qt::SizeHorCursor); QString word = words[hit.word].getLabel(); if (hit.part == LyricsEdit::Part::Start) { - showHelp(tr("Drag to move the start of \"%1\"").arg(word)); + showHelp(tr("Drag to move the start of \"%1\", " + "Shift-drag to move all the words").arg(word)); } else { - showHelp(tr("Drag to move the end of \"%1\"").arg(word)); + showHelp(tr("Drag to move the end of \"%1\", " + "Shift-drag to move all the words").arg(word)); } } else if (hit.part == LyricsEdit::Part::Inside) { restoreCursor(); showHelp(tr("Double-click to change the text of \"%1\", " - "right-click to delete it") + "right-click to delete it, " + "Shift-drag to move all the words") .arg(words[hit.word].getLabel())); } else { restoreCursor(); showHelp(tr("Right-click to add a word, " - "drag a word's start or end to move it")); + "drag a word's start or end to move it, " + "Shift-drag to move all the words")); } // The row is ours while edit mode is on: the pane would only put its @@ -504,6 +539,11 @@ LyricsEditor::dragTo(int x) { if (!m_dragging || !m_pane || m_dragModel.isNone()) return; + if (m_shifting) { + shiftTo(x); + return; + } + // The model can change under a drag: a keyboard undo that takes the // word away, or the lyrics removed or replaced. The word being // dragged is then not what the drag last made it, and nothing it @@ -543,12 +583,96 @@ LyricsEditor::dragTo(int x) m_dragCommand->add(m_dragCurrent); } +void +LyricsEditor::shiftTo(int x) +{ + // As for an edge: if anything but this drag has changed the words, + // nothing it would do now is right + auto model = dragModel(); + if (!model || model->getAllEvents() != m_shiftCurrent) { + abandonDrag(); + return; + } + + // As far as the pointer has moved, in time, from the words as they + // were at the press + sv_frame_t wanted = m_pane->getFrameForX(x) - m_dragPressFrame; + EventVector moved = LyricsEdit::shifted(m_dragWords, wanted); + if (moved == m_shiftCurrent) return; + + // Straight in the model, so that the words follow the pointer. A + // command filled at every move would keep every word's every step: + // ChangeEventsCommand folds an add and the remove of the same event + // only when they are next to each other, and all the words out and + // all in never are. The one command is made at the release + replaceWords(model.get(), m_shiftCurrent, moved); + m_shiftCurrent = moved; +} + +void +LyricsEditor::pushShift(ModelId modelId, const EventVector &from, + const EventVector &to) +{ + auto command = new ChangeEventsCommand + (modelId.untyped, tr("Shift Lyrics")); + for (const Event &e : from) command->remove(e); + for (const Event &e : to) command->add(e); + pushDone(command); +} + +double +LyricsEditor::shiftLyrics(ModelId modelId, double seconds) +{ + // A drag goes on the history before the shift, as it would if the + // button had been let go first + finishDrag(); + + RegionLayer *layer = m_lyrics ? m_lyrics->getLayer() : nullptr; + if (!layer || modelId.isNone() || layer->getModel() != modelId) { + return 0.0; + } + auto model = ModelById::getAs(modelId); + if (!model) return 0.0; + + sv_samplerate_t rate = model->getSampleRate(); + EventVector words = model->getAllEvents(); + sv_frame_t by = LyricsEdit::clampShift + (words, LyricsEdit::framesFor(seconds, rate)); + if (by == 0) return 0.0; + + pushShift(modelId, words, LyricsEdit::shifted(words, by)); + return double(by) / rate; +} + void LyricsEditor::finishDrag() { if (!m_dragging) return; m_dragging = false; + if (m_shifting) { + m_shifting = false; + EventVector original = m_dragWords; + EventVector current = m_shiftCurrent; + m_dragWords.clear(); + m_shiftCurrent.clear(); + + // Changed under the drag, or dragged away and back: nothing to + // put on the history + auto model = dragModel(); + if (!model || model->getAllEvents() != current || + current == original) { + return; + } + + // The words back as they were, and then the one command that + // moves them, which holds the words at the press and the words + // at the release and nothing of the moves between + replaceWords(model.get(), current, original); + pushShift(m_dragModel, original, current); + return; + } + ChangeEventsCommand *command = m_dragCommand; m_dragCommand = nullptr; m_dragWords.clear(); @@ -585,6 +709,7 @@ LyricsEditor::abandonDrag() delete m_dragCommand; m_dragCommand = nullptr; m_dragWords.clear(); + m_shiftCurrent.clear(); // The rest of the drag, up to the release, moves nothing, and is // still not the pane's: it never saw the press @@ -592,14 +717,15 @@ LyricsEditor::abandonDrag() } void -LyricsEditor::setEdgeCursor() +LyricsEditor::setCursorShape(Qt::CursorShape shape) { if (!m_pane) return; if (!m_cursorSet) { m_savedCursor = m_pane->cursor(); m_cursorSet = true; } - m_pane->setCursor(Qt::SizeHorCursor); + m_cursorShape = shape; + m_pane->setCursor(shape); } void @@ -610,7 +736,7 @@ LyricsEditor::restoreCursor() // Unless the pane has put another of its own in the meantime, for a // change of tool say: that one stays - if (m_pane && m_pane->cursor().shape() == Qt::SizeHorCursor) { + if (m_pane && m_pane->cursor().shape() == m_cursorShape) { m_pane->setCursor(m_savedCursor); } } diff --git a/main/LyricsEditor.h b/main/LyricsEditor.h index 4585ceef..9e681638 100644 --- a/main/LyricsEditor.h +++ b/main/LyricsEditor.h @@ -44,17 +44,19 @@ class ChangeEventsCommand; /** * Edit mode for the lyrics (Edit > Edit Lyrics): the mouse in the box * row of the lyrics, along the bottom of their pane, moves a word's - * start or end, changes its text on a double-click, and on a right - * click offers a small menu to change or delete the word there, or to - * add one in the space between words. + * start or end, with Shift held moves all the words together, changes + * a word's text on a double-click, and on a right click offers a small + * menu to change or delete the word there, or to add one in the space + * between words. * * The lyrics layer is never the pane's top layer, so the pane's tools * never reach it: this watches the pane's mouse events through an * event filter, which is on only while edit mode is. Only what it acts * on is kept from the pane: a left press on an edge and the drag it - * starts, a double-click on a word, a right press anywhere in the box - * row, and moves with no button held in the box row, where the cursor - * and the context help are this one's. Everything else, a click + * starts, a left press with Shift held anywhere in the box row and the + * drag it starts, a double-click on a word, a right press anywhere in + * the box row, and moves with no button held in the box row, where the + * cursor and the context help are this one's. Everything else, a click * anywhere to move the playback cursor included, goes to the pane as it * would without edit mode. * @@ -62,7 +64,8 @@ class ChangeEventsCommand; * A drag edits the model as it goes, so the words move under the * pointer, and is one command on the undo history when the button is * let go, or nothing at all if the word is where it was. A change of - * text, an added word and a deleted one are a command each. + * text, an added word and a deleted one are a command each, and so is + * a shift of all the words, by a drag or by shiftLyrics(). * * The layer and the model are found through LyricsTrack at every * event, and nothing of them is kept between drags: an import, a @@ -90,9 +93,24 @@ class LyricsEditor : public QObject void setEnabled(bool enabled); bool isEnabled() const { return m_enabled; } - /// True from the press on an edge to the release + /// True from the press that starts a drag to the release bool isDragging() const { return m_dragging; } + /// True while the drag is one of all the words (Shift was held at + /// its press), not of an edge + bool isShifting() const { return m_dragging && m_shifting; } + + /** + * Move all the words of the lyrics in this model by this many + * seconds, negative for earlier, as far as LyricsEdit::clampShift() + * lets them go: one command, "Shift Lyrics". Edit mode need not be + * on; a drag in progress is finished first. The seconds they moved + * by, 0 if they did not move, the shift being 0 or clamped to 0, or + * if the model is no longer that of the lyrics: nothing is pushed + * then. Not to be called from an undo or a redo. + */ + double shiftLyrics(sv::ModelId model, double seconds); + /// How near an edge, in logical pixels on either side, grabs it static constexpr int grabPixels = 6; @@ -159,8 +177,11 @@ class LyricsEditor : public QObject // The drag. The words are those of the model at the press, and the // same ones go to LyricsEdit at every move: the limits of the edge - // come from where the word was then, so a drag back puts it back + // come from where the word was then, so a drag back puts it back. + // Whether it moves an edge or all the words is decided at the press, + // whatever the Shift key does after bool m_dragging; + bool m_shifting; sv::ModelId m_dragModel; sv::EventVector m_dragWords; int m_dragWord; @@ -171,9 +192,15 @@ class LyricsEditor : public QObject sv::Event m_dragCurrent; sv::ChangeEventsCommand *m_dragCommand; - // The pane's own cursor, while ours is shown over an edge + // A drag of all the words: they are as they were at the press, moved + // together, and the model holds them so. They go into the model + // straight, with no command; the command is made at the release + sv::EventVector m_shiftCurrent; + + // The pane's own cursor, while one of ours is shown, and ours bool m_cursorSet; QCursor m_savedCursor; + Qt::CursorShape m_cursorShape; // Whether the context help is ours bool m_helpShown; @@ -219,6 +246,13 @@ class LyricsEditor : public QObject void leaveRow(); void dragTo(int x); + void shiftTo(int x); + + // All the words out, then all the moved ones in, as one command: + // never one out and one in at a time, or a moved word could meet an + // original that is still there + void pushShift(sv::ModelId model, const sv::EventVector &from, + const sv::EventVector &to); // Push the drag's command, if the word has moved void finishDrag(); @@ -227,7 +261,7 @@ class LyricsEditor : public QObject // model has changed under the drag void abandonDrag(); - void setEdgeCursor(); + void setCursorShape(Qt::CursorShape); void restoreCursor(); void showHelp(const QString &); void clearHelp(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index a4b0d047..cd4fdb82 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -169,6 +169,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_showLyrics(nullptr), m_lyricsEditor(nullptr), m_editLyricsAction(nullptr), + m_shiftLyricsAction(nullptr), m_takesMenu(nullptr), m_takeCombo(nullptr), m_newTakeAction(nullptr), @@ -993,11 +994,20 @@ MainWindow::setupEditMenu() // No shortcut: it is not switched on and off in the middle of things m_editLyricsAction = new QAction(tr("Edit L&yrics"), this); m_editLyricsAction->setCheckable(true); - m_editLyricsAction->setStatusTip(tr("Edit the words of the lyrics along the bottom of the pane: drag a start or end, double-click a word to change its text, right-click to add or delete one")); + m_editLyricsAction->setStatusTip(tr("Edit the words of the lyrics along the bottom of the pane: drag a start or end, Shift-drag to move all the words, double-click a word to change its text, right-click to add or delete one")); m_editLyricsAction->setEnabled(false); connect(m_editLyricsAction, &QAction::triggered, this, &MainWindow::editLyricsToggled); menu->addAction(m_editLyricsAction); + + // The same shift as a Shift-drag in edit mode, by a number: for an + // offset known beforehand, such as the lyrics exporter's + m_shiftLyricsAction = new QAction(tr("S&hift Lyrics..."), this); + m_shiftLyricsAction->setStatusTip(tr("Move all the words of the lyrics earlier or later by a number of seconds")); + m_shiftLyricsAction->setEnabled(false); + connect(m_shiftLyricsAction, &QAction::triggered, + this, &MainWindow::shiftLyrics); + menu->addAction(m_shiftLyricsAction); } void @@ -2386,6 +2396,9 @@ MainWindow::updateMenuStates() m_editLyricsAction->setChecked (m_lyricsEditor && m_lyricsEditor->isEnabled()); } + if (m_shiftLyricsAction) { + m_shiftLyricsAction->setEnabled(lyricsEditable); + } // The audio check records takes of its own, and keeps what it // measures for the devices it started on. Record is shut after the @@ -3691,6 +3704,22 @@ MainWindow::askForLyricsWordText(QString &text, bool isNew) return true; } +bool +MainWindow::askForLyricsShift(double &seconds) +{ + // Milliseconds are as fine as anyone can hear, and an hour is longer + // than any song + bool ok = false; + double typed = QInputDialog::getDouble + (this, tr("Shift Lyrics"), + tr("Move all the words by this many seconds\n" + "(negative: earlier, positive: later):"), + seconds, -3600.0, 3600.0, 3, &ok); + if (!ok) return false; + seconds = typed; + return true; +} + void MainWindow::exportLyrics() { @@ -3816,6 +3845,37 @@ MainWindow::editLyricsToggled() setLyricsEditing(m_editLyricsAction->isChecked()); } +void +MainWindow::shiftLyrics() +{ + if (!m_lyricsEditor || !lyricsEditAllowed()) return; + ModelId lyricsModel = m_lyrics->getModelId(); + + double seconds = 0.0; + if (!askForLyricsShift(seconds)) return; + + // The dialog ran an event loop of its own, in which the lyrics may + // have gone, been hidden or replaced, or a take begun: then the + // offset typed is not for what is there now + if (!lyricsEditAllowed() || m_lyrics->getModelId() != lyricsModel) { + return; + } + + // One command; a shift of 0, or one the start of the song leaves no + // room for, is none + double shifted = m_lyricsEditor->shiftLyrics(lyricsModel, seconds); + if (shifted == 0.0) return; + + // Kept as the status message, as an import's is. The amount is the + // one the words moved by, which is less than asked if the first word + // reached the start + QString amount = QString::number(std::abs(shifted), 'f', 3); + m_myStatusMessage = (shifted < 0.0 ? + tr("Shifted the lyrics %1 s earlier.").arg(amount) : + tr("Shifted the lyrics %1 s later.").arg(amount)); + getStatusLabel()->setText(m_myStatusMessage); +} + void MainWindow::updateWaveformFade() { diff --git a/main/MainWindow.h b/main/MainWindow.h index 253fc344..d49f621e 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -229,6 +229,7 @@ protected slots: virtual void removeLyrics(); virtual void showLyricsToggled(); virtual void editLyricsToggled(); + virtual void shiftLyrics(); virtual void editDisplayExtents(); @@ -434,6 +435,10 @@ protected slots: LyricsEditor *m_lyricsEditor; QAction *m_editLyricsAction; + // Edit > Shift Lyrics...: all the words earlier or later by a number + // of seconds. To be had whenever editing is, edit mode on or not + QAction *m_shiftLyricsAction; + // The lyrics are there to be edited: shown, visible, and no take // being recorded (the singer is reading them) bool lyricsEditAllowed() const; @@ -482,6 +487,11 @@ protected slots: // Overridden by the tests virtual bool askForLyricsWordText(QString &text, bool isNew); + // Ask how far to shift all the words of the lyrics, in seconds, + // negative for earlier: true with the seconds, false if the user + // cancelled. Overridden by the tests + virtual bool askForLyricsShift(double &seconds); + // --- The audio folder of the session (spec 6.4) --- // Where the next combined audio file of a take is to be written: the diff --git a/main/test/TestLyricsEdit.h b/main/test/TestLyricsEdit.h index 284f706d..a5f56b77 100644 --- a/main/test/TestLyricsEdit.h +++ b/main/test/TestLyricsEdit.h @@ -16,7 +16,8 @@ // Tier 2: what an edit of the lyrics may do, as numbers: which edge // the pointer is on, how far a dragged edge goes, where a new word -// goes and which line it joins, and what typed text becomes. No window +// goes and which line it joins, what typed text becomes, and how far +// all the words shift together. No window // and no model: what the editor does with the answers is the app // suite's business. @@ -115,6 +116,14 @@ private slots: QCOMPARE(LyricsEdit::newWordFrames(48000), frame_t(24000)); QCOMPARE(LyricsEdit::minWordFrames(0), frame_t(0)); QCOMPARE(LyricsEdit::newWordFrames(0), frame_t(0)); + + // A shift typed in seconds, earlier or later, three decimals + QCOMPARE(LyricsEdit::framesFor(0.2, 44100), frame_t(8820)); + QCOMPARE(LyricsEdit::framesFor(-0.2, 44100), frame_t(-8820)); + QCOMPARE(LyricsEdit::framesFor(-1.234, 48000), frame_t(-59232)); + QCOMPARE(LyricsEdit::framesFor(0.001, 44100), frame_t(44)); + QCOMPARE(LyricsEdit::framesFor(0.0, 44100), frame_t(0)); + QCOMPARE(LyricsEdit::framesFor(1.0, -44100), frame_t(0)); } // A box runs from the x of the word's start to the x of its end @@ -720,6 +729,87 @@ private slots: QVERIFY(LyricsEdit::cleanText("\t\n").isEmpty()); QVERIFY(LyricsEdit::cleanText(QString("\x01\x02")).isEmpty()); } + + // Decision 17: all the words together, later as far as asked, and + // earlier only until the first word starts at 0 + void shift_clamp() { + const frame_t S = kSecond; + sv::EventVector words = { word(S, 2 * S), word(2 * S, 3 * S, 1.f), + word(5 * S, 6 * S, 2.f) }; + + QCOMPARE(LyricsEdit::clampShift(words, 0), frame_t(0)); + QCOMPARE(LyricsEdit::clampShift(words, 8820), frame_t(8820)); + QCOMPARE(LyricsEdit::clampShift(words, 100 * S), 100 * S); + QCOMPARE(LyricsEdit::clampShift(words, -8820), frame_t(-8820)); + QCOMPARE(LyricsEdit::clampShift(words, -S), -S); + QCOMPARE(LyricsEdit::clampShift(words, -S - 1), -S); + QCOMPARE(LyricsEdit::clampShift(words, -10 * S), -S); + + // The earliest start wherever it is in the vector, and not the + // first word's if another starts before it (one inside another) + sv::EventVector unordered = { word(3 * S, 4 * S), word(S / 2, S), + word(2 * S, 3 * S) }; + QCOMPARE(LyricsEdit::clampShift(unordered, -S), -S / 2); + sv::EventVector inside = { word(S, 5 * S), word(S / 4, S / 2) }; + QCOMPARE(LyricsEdit::clampShift(inside, -S), -S / 4); + + // A first word at 0 goes nowhere earlier, but anywhere later + sv::EventVector atZero = { word(0, S), word(S, 2 * S) }; + QCOMPARE(LyricsEdit::clampShift(atZero, -1), frame_t(0)); + QCOMPARE(LyricsEdit::clampShift(atZero, -S), frame_t(0)); + QCOMPARE(LyricsEdit::clampShift(atZero, 1), frame_t(1)); + + // A word before 0 already, which no parser makes, is not taken + // further back; later is still allowed + sv::EventVector before = { word(-100, S), word(S, 2 * S) }; + QCOMPARE(LyricsEdit::clampShift(before, -S), frame_t(0)); + QCOMPARE(LyricsEdit::clampShift(before, S), S); + + // Nothing to move + QCOMPARE(LyricsEdit::clampShift({}, -S), frame_t(0)); + QCOMPARE(LyricsEdit::clampShift({}, S), frame_t(0)); + } + + // Every word moved by the same amount, start and end: lengths, texts, + // lines and order kept, overlaps and gaps from the file as they were + void shifted_words() { + const frame_t S = kSecond; + sv::EventVector words = { word(S, 2 * S, 0.f, "Yksi"), + word(2 * S, 3 * S, 0.f, "kaksi"), + word(2 * S + 100, 4 * S, 1.f, "kolme"), + word(6 * S, 6 * S + 10, 2.f, "nelja") }; + + auto verifyMovedBy = [&](const sv::EventVector &moved, frame_t by) { + QCOMPARE(int(moved.size()), int(words.size())); + for (int i = 0; i < int(words.size()); ++i) { + QCOMPARE(moved[i].getFrame(), words[i].getFrame() + by); + QCOMPARE(endOf(moved[i]), endOf(words[i]) + by); + QCOMPARE(moved[i].getDuration(), words[i].getDuration()); + QCOMPARE(moved[i].getLabel(), words[i].getLabel()); + QCOMPARE(moved[i].getValue(), words[i].getValue()); + QVERIFY2(i == 0 || !(moved[i] < moved[i - 1]), + qPrintable(describe(moved[i]))); + } + }; + + verifyMovedBy(LyricsEdit::shifted(words, 8820), 8820); + verifyMovedBy(LyricsEdit::shifted(words, -8820), -8820); + verifyMovedBy(LyricsEdit::shifted(words, 0), 0); + QCOMPARE(LyricsEdit::shifted(words, 0), words); + + // Clamped: the first word lands on 0, the rest keep their places + // relative to it + sv::EventVector clamped = LyricsEdit::shifted(words, -5 * S); + verifyMovedBy(clamped, -S); + QCOMPARE(clamped[0].getFrame(), frame_t(0)); + QCOMPARE(LyricsEdit::shifted(clamped, -S), clamped); + + // There and back + QCOMPARE(LyricsEdit::shifted(LyricsEdit::shifted(words, 12345), + -12345), words); + + QVERIFY(LyricsEdit::shifted({}, S).empty()); + } }; #endif diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index e5cd811f..a1be511a 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -241,8 +241,10 @@ class TestMainWindow : public MainWindow QAction *removeLyricsAction() { return m_removeLyricsAction; } QAction *showLyricsAction() { return m_showLyrics; } - // Edit > Edit Lyrics, and the editor it switches on + // Edit > Edit Lyrics, and the editor it switches on; Edit > Shift + // Lyrics... QAction *editLyricsAction() { return m_editLyricsAction; } + QAction *shiftLyricsAction() { return m_shiftLyricsAction; } LyricsEditor *lyricsEditor() { return m_lyricsEditor; } // The file Import Lyrics asks for, answered from here: "" is Cancel @@ -267,6 +269,13 @@ class TestMainWindow : public MainWindow // dialog runs an event loop void whileAskingWordText(std::function f) { m_whileAskingWordText = f; } + // The seconds Shift Lyrics asks for, answered from here in turn, as + // the texts are. A question with no answer left is cancelled + void answerLyricsShift(double seconds) { m_shiftAnswers.push_back({true, seconds}); } + void cancelLyricsShift() { m_shiftAnswers.push_back({false, 0.0}); } + int lyricsShiftQuestions() const { return m_shiftQuestions; } + void whileAskingLyricsShift(std::function f) { m_whileAskingShift = f; } + void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { onRealtimePitchDetected(frame, hz); } @@ -334,6 +343,20 @@ class TestMainWindow : public MainWindow return true; } + bool askForLyricsShift(double &seconds) override { + ++m_shiftQuestions; + if (m_whileAskingShift) { + auto during = m_whileAskingShift; + m_whileAskingShift = nullptr; + during(); + } + if (m_shiftAnswers.isEmpty()) return false; + auto answer = m_shiftAnswers.takeFirst(); + if (!answer.first) return false; + seconds = answer.second; + return true; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -357,6 +380,9 @@ class TestMainWindow : public MainWindow QString m_wordTextOffered; bool m_wordTextWasNew = false; std::function m_whileAskingWordText; + QList> m_shiftAnswers; + int m_shiftQuestions = 0; + std::function m_whileAskingShift; }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 14ee2971..8653f5d3 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -840,11 +840,12 @@ class TestRecordWorkflow : public QObject // first, then to the pane. Not QTest::mouseMove, which does not // carry the buttons held void sendMouse(QEvent::Type type, QPoint pos, Qt::MouseButton button, - Qt::MouseButtons buttons) { + Qt::MouseButtons buttons, + Qt::KeyboardModifiers modifiers = Qt::NoModifier) { sv::Pane *pane = pane0(); QVERIFY(pane); QMouseEvent event(type, QPointF(pos), QPointF(pane->mapToGlobal(pos)), - button, buttons, Qt::NoModifier); + button, buttons, modifiers); QApplication::sendEvent(pane, &event); } @@ -861,6 +862,20 @@ class TestRecordWorkflow : public QObject sendMouse(QEvent::MouseButtonRelease, pos, Qt::LeftButton, Qt::NoButton); } + // The same with Shift held + void shiftPressAt(QPoint pos) { + sendMouse(QEvent::MouseButtonPress, pos, Qt::LeftButton, Qt::LeftButton, + Qt::ShiftModifier); + } + void shiftMoveHeldTo(QPoint pos) { + sendMouse(QEvent::MouseMove, pos, Qt::NoButton, Qt::LeftButton, + Qt::ShiftModifier); + } + void shiftReleaseAt(QPoint pos) { + sendMouse(QEvent::MouseButtonRelease, pos, Qt::LeftButton, + Qt::NoButton, Qt::ShiftModifier); + } + // Press at one point, move to the other a few pixels at a time with // the button held, and let go there void dragFromTo(QPoint from, QPoint to) { @@ -873,6 +888,27 @@ class TestRecordWorkflow : public QObject releaseAt(to); } + // The same with Shift held throughout + void shiftDragFromTo(QPoint from, QPoint to) { + shiftPressAt(from); + int steps = std::max(1, std::abs(to.x() - from.x()) / 3); + for (int i = 1; i <= steps; ++i) { + shiftMoveHeldTo(QPoint(from.x() + (to.x() - from.x()) * i / steps, + from.y() + (to.y() - from.y()) * i / steps)); + } + shiftReleaseAt(to); + } + + // The words, every one moved by the same number of frames + static sv::EventVector shiftedBy(const sv::EventVector &words, + sv::sv_frame_t by) { + sv::EventVector moved; + for (const sv::Event &e : words) { + moved.push_back(e.withFrame(e.getFrame() + by)); + } + return moved; + } + // A reference with the gapped lyrics on it, painted. Yksi and kaksi // share an edge at 0.6 s; kolme comes after a gap. The row's middle // is where the tests point, at the columns the words are in. @@ -7447,23 +7483,45 @@ private slots: hoverAt(inRow(edge - 1)); QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); QCOMPARE(m_window->statusText(), - QString("Drag to move the end of \"Yksi\"")); + QString("Drag to move the end of \"Yksi\", " + "Shift-drag to move all the words")); hoverAt(inRow(edge + 2)); QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); QCOMPARE(m_window->statusText(), - QString("Drag to move the start of \"kaksi\"")); + QString("Drag to move the start of \"kaksi\", " + "Shift-drag to move all the words")); hoverAt(inRow(edge + 40)); QCOMPARE(pane->cursor().shape(), own); QCOMPARE(m_window->statusText(), QString("Double-click to change the text of \"kaksi\", " - "right-click to delete it")); + "right-click to delete it, " + "Shift-drag to move all the words")); int gap = columnOf(endOf(lyricsWord("kaksi"))) + 20; hoverAt(inRow(gap)); QCOMPARE(pane->cursor().shape(), own); QCOMPARE(m_window->statusText(), QString("Right-click to add a word, " - "drag a word's start or end to move it")); + "drag a word's start or end to move it, " + "Shift-drag to move all the words")); + + // A closed hand through a drag of all the words (decision 19), + // from a word or from an edge, and the pane's own or the edge's + // back after + shiftPressAt(inRow(edge + 40)); + QCOMPARE(pane->cursor().shape(), Qt::ClosedHandCursor); + shiftMoveHeldTo(inRow(edge + 50)); + QCOMPARE(pane->cursor().shape(), Qt::ClosedHandCursor); + shiftReleaseAt(inRow(edge + 50)); + QCOMPARE(pane->cursor().shape(), own); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + edge = columnOf(lyricsWord("kaksi").getFrame()); + shiftPressAt(inRow(edge)); + QCOMPARE(pane->cursor().shape(), Qt::ClosedHandCursor); + shiftReleaseAt(inRow(edge)); + QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); + hoverAt(inRow(gap)); + QCOMPARE(pane->cursor().shape(), own); hoverAt(inRow(edge)); QCOMPARE(pane->cursor().shape(), Qt::SizeHorCursor); @@ -8083,6 +8141,566 @@ private slots: QCOMPARE(lyricsEvents(), events); } + // Shift held at the press, anywhere in the box row (in a word, in a + // gap, on an edge), a drag moves all the words together as it goes, + // starts and ends alike, and is one "Shift Lyrics" step when let go + // (decisions 17, 18a) + void lyrics_edit_shift_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + QCOMPARE(int(before.size()), 3); + + // Later, from inside a word, each move seen in the model at once + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + shiftPressAt(inRow(inside)); + QVERIFY(editor->isShifting()); + for (int x = inside + 3; x <= inside + 30; x += 3) { + shiftMoveHeldTo(inRow(x)); + QCOMPARE(lyricsEvents(), shiftedBy(before, framesBetween(inside, x))); + } + QCOMPARE(int(commands.count()), 0); + shiftReleaseAt(inRow(inside + 30)); + QVERIFY(!editor->isDragging()); + sv::EventVector later = + shiftedBy(before, framesBetween(inside, inside + 30)); + QCOMPARE(lyricsEvents(), later); + QCOMPARE(int(commands.count()), 1); + QVERIFY(m_window->isDocumentModified()); + + // Earlier, from the gap between kaksi and kolme + int gap = columnOf(endOf(lyricsWord("kaksi"))) + 20; + shiftDragFromTo(inRow(gap), inRow(gap - 20)); + sv::EventVector earlier = shiftedBy(later, framesBetween(gap, gap - 20)); + QCOMPARE(lyricsEvents(), earlier); + + // From the shared edge: all the words, not the one edge + int edge = columnOf(lyricsWord("kaksi").getFrame()); + shiftDragFromTo(inRow(edge), inRow(edge + 25)); + sv::EventVector fromEdge = + shiftedBy(earlier, framesBetween(edge, edge + 25)); + QCOMPARE(lyricsEvents(), fromEdge); + QCOMPARE(int(commands.count()), 3); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), earlier); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), later); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), later); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), fromEdge); + QCOMPARE(redoOnce(), QString()); + + QVERIFY(m_window->playSource()->getModels().count + (m_window->lyrics()->getModelId()) == 0); + verifyPlaySourceClean(); + } + + // The first word stops at 0, however far the pointer goes, and the + // rest keep their places behind it; a drag past and back puts the + // words where the pointer is. Already at 0, a drag earlier is no + // edit at all (decision 17) + void lyrics_edit_shift_drag_stops_at_zero() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::sv_frame_t first = lyricsWord("Yksi").getFrame(); + QVERIFY(first > 0); + int inside = columnOf(first) + 40; + QVERIFY2(framesBetween(inside, 20) < -first - 1000, + "the pointer cannot go far enough left"); + + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(20)); + QCOMPARE(lyricsEvents(), shiftedBy(before, -first)); + shiftMoveHeldTo(inRow(inside - 10)); + shiftReleaseAt(inRow(inside - 10)); + QCOMPARE(lyricsEvents(), + shiftedBy(before, framesBetween(inside, inside - 10))); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + + shiftDragFromTo(inRow(inside), inRow(20)); + sv::EventVector atZero = shiftedBy(before, -first); + QCOMPARE(lyricsEvents(), atZero); + QCOMPARE(lyricsWord("Yksi").getFrame(), sv::sv_frame_t(0)); + + // (An undo and a redo are commands executed as well) + m_window->discardModifications(); + int pushed = int(commands.count()); + int again = columnOf(0) + 40; + shiftDragFromTo(inRow(again), inRow(again - 30)); + QCOMPARE(lyricsEvents(), atZero); + QCOMPARE(int(commands.count()), pushed); + QVERIFY(!m_window->isDocumentModified()); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + } + + // The one command of a long drag holds the words as they were at the + // press and as they are at the release, and nothing of the moves + // between. CommandHistory does not show what a command holds, but + // the model says what is done to it: a word taken out is one signal, + // a word put in two, so the undo and the redo of a command of N words + // out and N in each give 3N at most, where a command that kept every + // move would replay all of them + void lyrics_edit_shift_drag_one_command() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + auto model = lyricsModel(); + QVERIFY(model); + sv::EventVector before = lyricsEvents(); + int words = int(before.size()); + + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + shiftPressAt(inRow(inside)); + for (int i = 0; i < 40; ++i) { + shiftMoveHeldTo(inRow(inside + (i % 2 ? 30 : -30) + i / 4)); + } + shiftMoveHeldTo(inRow(inside + 17)); + shiftReleaseAt(inRow(inside + 17)); + sv::EventVector after = + shiftedBy(before, framesBetween(inside, inside + 17)); + QCOMPARE(lyricsEvents(), after); + + int changes = 0; + auto counted = [&changes]() { ++changes; }; + auto within = connect(model.get(), &sv::Model::modelChangedWithin, + this, counted); + auto whole = connect(model.get(), &sv::Model::modelChanged, + this, counted); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + QVERIFY2(changes >= words && changes <= 3 * words, + qPrintable(QString("the undo changed the model %1 times " + "for %2 words").arg(changes).arg(words))); + changes = 0; + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), after); + QVERIFY2(changes >= words && changes <= 3 * words, + qPrintable(QString("the redo changed the model %1 times " + "for %2 words").arg(changes).arg(words))); + disconnect(within); + disconnect(whole); + } + + // What a press is, an edge's or all the words', is decided at the + // press: a plain press on an edge moves that edge alone, Shift held + // or not after, and a Shift press moves all the words, Shift let go + // or not. Pressed and let go, or dragged away and back: no edit + void lyrics_edit_shift_decided_at_the_press() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + sv::Event yksi = lyricsWord("Yksi"); + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + int edge = columnOf(kaksi.getFrame()); + + pressAt(inRow(edge)); + QVERIFY(editor->isDragging()); + QVERIFY(!editor->isShifting()); + shiftMoveHeldTo(inRow(edge + 20)); + shiftReleaseAt(inRow(edge + 20)); + QCOMPARE(lyricsWord("kaksi").getFrame(), + kaksi.getFrame() + framesBetween(edge, edge + 20)); + QCOMPARE(endOf(lyricsWord("kaksi")), endOf(kaksi)); + QCOMPARE(lyricsWord("Yksi"), yksi); + QCOMPARE(lyricsWord("kolme"), kolme); + QCOMPARE(undoOnce(), QString("Move Word Start")); + QCOMPARE(lyricsEvents(), before); + + shiftPressAt(inRow(edge)); + QVERIFY(editor->isShifting()); + moveHeldTo(inRow(edge + 20)); + releaseAt(inRow(edge + 20)); + QCOMPARE(lyricsEvents(), + shiftedBy(before, framesBetween(edge, edge + 20))); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + + // (An undo is a command executed as well) + m_window->discardModifications(); + int pushed = int(commands.count()); + int inside = edge + 40; + shiftPressAt(inRow(inside)); + shiftReleaseAt(inRow(inside)); + shiftPressAt(inRow(inside)); + for (int x = inside; x <= inside + 30; x += 5) shiftMoveHeldTo(inRow(x)); + QVERIFY(lyricsEvents() != before); + for (int x = inside + 30; x >= inside; x -= 5) shiftMoveHeldTo(inRow(x)); + shiftReleaseAt(inRow(inside)); + QVERIFY(!editor->isDragging()); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(int(commands.count()), pushed); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(undoOnce(), QString()); + } + + // Edit mode off, a Shift press is the pane's as any press is + // (decision 19); edit mode on, so is one outside the box row + void lyrics_edit_shift_press_elsewhere_is_the_panes() { + makeWindow(FakeAudioIO::Config()); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + + shiftPressAt(inRow(inside)); + QVERIFY(!editor->isDragging()); + shiftMoveHeldTo(inRow(inside + 30)); + shiftReleaseAt(inRow(inside + 30)); + QCOMPARE(lyricsEvents(), before); + + switchLyricsEditingOn(); + if (QTest::currentTestFailed()) return; + QPoint above(inside, m_row.top() - 8); + shiftPressAt(above); + QVERIFY(!editor->isDragging()); + shiftMoveHeldTo(QPoint(inside + 30, above.y())); + shiftReleaseAt(QPoint(inside + 30, above.y())); + QCOMPARE(lyricsEvents(), before); + + // The pane's Shift-drag outlines a region to analyse again + QTRY_VERIFY_WITH_TIMEOUT(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + QVERIFY(undoOnce() != QString("Shift Lyrics")); + } + + // Edit mode switched off in the middle of a drag of all the words: + // the drag ends as a release would, one step, and the rest of it is + // not an edit + void lyrics_edit_shift_off_in_the_middle_of_a_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(inside + 30)); + sv::EventVector dragged = lyricsEvents(); + QCOMPARE(dragged, shiftedBy(before, framesBetween(inside, inside + 30))); + + m_window->editLyricsAction()->trigger(); + QVERIFY(!editor->isEnabled()); + QVERIFY(!editor->isDragging()); + QVERIFY(m_window->isDocumentModified()); + + shiftMoveHeldTo(inRow(inside + 60)); + shiftReleaseAt(inRow(inside + 60)); + QCOMPARE(lyricsEvents(), dragged); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(undoOnce(), QString()); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), dragged); + } + + // The words changed under a drag of them all, by something other + // than the drag: the drag ends, the model is left as that made it, + // and nothing goes on the history. Removed in the middle likewise + void lyrics_edit_shift_words_change_under_a_drag() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(inside + 20)); + sv::Event kolme = lyricsWord("kolme"); + replaceWord(kolme, kolme.withLabel("kolmas")); + sv::EventVector changed = lyricsEvents(); + shiftMoveHeldTo(inRow(inside + 40)); + QCOMPARE(lyricsEvents(), changed); + shiftReleaseAt(inRow(inside + 40)); + QVERIFY(!editor->isDragging()); + QCOMPARE(lyricsEvents(), changed); + QCOMPARE(undoOnce(), QString()); + + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(inside + 20)); + m_window->removeLyricsAction()->trigger(); + QVERIFY(!editor->isDragging()); + QVERIFY(!editor->isEnabled()); + shiftMoveHeldTo(inRow(inside + 40)); + shiftReleaseAt(inRow(inside + 40)); + QVERIFY(!m_window->lyrics()->isShown()); + QCOMPARE(undoOnce(), QString()); + verifyPlaySourceClean(); + } + + // Edit > Shift Lyrics...: all the words by the seconds typed, later or + // earlier, one step each, the status bar saying how far they went; + // the first word stops at 0, and a shift of 0, or one clamped to 0, + // or a cancel, is no edit at all (decisions 18b, 20). Edit mode need + // not be on + void lyrics_edit_shift_by_number() { + makeWindow(FakeAudioIO::Config()); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + QAction *shift = m_window->shiftLyricsAction(); + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + sv::EventVector before = lyricsEvents(); + QVERIFY(!m_window->lyricsEditor()->isEnabled()); + QVERIFY(shift->isEnabled()); + + m_window->answerLyricsShift(0.2); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 1); + sv::EventVector later = shiftedBy(before, 8820); + QCOMPARE(lyricsEvents(), later); + QCOMPARE(m_window->statusText(), + QString("Shifted the lyrics 0.200 s later.")); + QVERIFY(m_window->isDocumentModified()); + + m_window->answerLyricsShift(-0.05); + shift->trigger(); + sv::EventVector earlier = shiftedBy(later, -2205); + QCOMPARE(lyricsEvents(), earlier); + QCOMPARE(m_window->statusText(), + QString("Shifted the lyrics 0.050 s earlier.")); + QCOMPARE(int(commands.count()), 2); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), later); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), earlier); + + // Yksi starts at 0.35 s now: that far and no further + QCOMPARE(lyricsWord("Yksi").getFrame(), sv::sv_frame_t(15435)); + m_window->answerLyricsShift(-5.0); + shift->trigger(); + sv::EventVector atZero = shiftedBy(earlier, -15435); + QCOMPARE(lyricsEvents(), atZero); + QCOMPARE(m_window->statusText(), + QString("Shifted the lyrics 0.350 s earlier.")); + + // Nothing to do: no step, and nothing said. (An undo and a redo + // are commands executed as well) + int pushed = int(commands.count()); + m_window->discardModifications(); + m_window->setStatusText("before"); + m_window->answerLyricsShift(-1.0); + m_window->answerLyricsShift(0.0); + m_window->answerLyricsShift(0.00001); + m_window->cancelLyricsShift(); + for (int i = 0; i < 4; ++i) shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 7); + QCOMPARE(lyricsEvents(), atZero); + QCOMPARE(int(commands.count()), pushed); + QVERIFY(!m_window->isDocumentModified()); + QCOMPARE(m_window->statusText(), QString("before")); + + // With edit mode on as well, which stays on + switchLyricsEditingOn(); + if (QTest::currentTestFailed()) return; + m_window->answerLyricsShift(1.5); + shift->trigger(); + QCOMPARE(lyricsEvents(), shiftedBy(atZero, 66150)); + QVERIFY(m_window->lyricsEditor()->isEnabled()); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), atZero); + } + + // To be had when editing is (lyricsEditAllowed()), edit mode on or + // not: not without lyrics, hidden ones, or once they are removed; and + // a trigger then asks nothing + void lyrics_edit_shift_needs_lyrics() { + makeWindow(FakeAudioIO::Config()); + QAction *shift = m_window->shiftLyricsAction(); + QVERIFY(shift); + QVERIFY(!shift->isEnabled()); + openReference(writeWav(tone(lowHz, 2.0))); + if (QTest::currentTestFailed()) return; + QVERIFY(!shift->isEnabled()); + m_window->answerLyricsShift(0.5); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 0); + + QVERIFY(m_window->doImportLyricsFrom(writeLrc(gappedLyrics()))); + QVERIFY(shift->isEnabled()); + m_window->showLyricsAction()->trigger(); + QVERIFY(!shift->isEnabled()); + sv::EventVector before = lyricsEvents(); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 0); + QCOMPARE(lyricsEvents(), before); + m_window->showLyricsAction()->trigger(); + QVERIFY(shift->isEnabled()); + + m_window->removeLyricsAction()->trigger(); + QVERIFY(!shift->isEnabled()); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 0); + } + + // Not while a take is recorded. A drag of all the words going on when + // the take starts is finished first, and so is on the history before + // the take + void lyrics_edit_shift_off_while_recording() { + FakeAudioIO::Config config; + config.input = tone(highHz, 3.0); + lyricsEditFixture(config); + if (QTest::currentTestFailed()) return; + QAction *shift = m_window->shiftLyricsAction(); + LyricsEditor *editor = m_window->lyricsEditor(); + sv::EventVector before = lyricsEvents(); + + int inside = columnOf(lyricsWord("kaksi").getFrame()) + 40; + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(inside + 30)); + sv::EventVector dragged = lyricsEvents(); + QVERIFY(dragged != before); + + startTake(); + if (QTest::currentTestFailed()) return; + QVERIFY(!editor->isDragging()); + QVERIFY(!shift->isEnabled()); + m_window->answerLyricsShift(0.5); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 0); + + shiftMoveHeldTo(inRow(inside + 60)); + shiftReleaseAt(inRow(inside + 60)); + QCOMPARE(lyricsEvents(), dragged); + + QTest::qWait(300); + stopTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT(shift->isEnabled(), 2000); + + QCOMPARE(undoOnce(), QString("Record Singing")); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(lyricsEvents(), before); + } + + // The question has an event loop of its own: lyrics removed, hidden or + // imported again while it is open are not what the seconds were typed + // for, and nothing is shifted + void lyrics_edit_shift_lyrics_change_during_question() { + makeWindow(FakeAudioIO::Config()); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + QAction *shift = m_window->shiftLyricsAction(); + QSignalSpy commands(sv::CommandHistory::getInstance(), qOverload<> + (&sv::CommandHistory::commandExecuted)); + QString file = writeLrc(gappedLyrics()); + sv::EventVector before = lyricsEvents(); + + m_window->whileAskingLyricsShift([this, file]() { + QVERIFY(m_window->doImportLyricsFrom(file)); + }); + m_window->answerLyricsShift(0.3); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 1); + QCOMPARE(lyricsEvents(), before); + + m_window->whileAskingLyricsShift([this]() { + m_window->showLyricsAction()->trigger(); + }); + m_window->answerLyricsShift(0.3); + shift->trigger(); + QCOMPARE(lyricsEvents(), before); + m_window->showLyricsAction()->trigger(); + + m_window->whileAskingLyricsShift([this]() { + m_window->removeLyricsAction()->trigger(); + }); + m_window->answerLyricsShift(0.3); + shift->trigger(); + QCOMPARE(m_window->lyricsShiftQuestions(), 3); + QVERIFY(!m_window->lyrics()->isShown()); + QCOMPARE(int(commands.count()), 0); + QCOMPARE(undoOnce(), QString()); + } + + // The word at the cursor is found again as all the words move, by a + // drag and by a number + void lyrics_edit_shift_highlight_follows() { + lyricsEditFixture(); + if (QTest::currentTestFailed()) return; + sv::Event kaksi = lyricsWord("kaksi"); + sv::Event kolme = lyricsWord("kolme"); + sv::sv_frame_t gap = (endOf(kaksi) + kolme.getFrame()) / 2; + m_window->seekTo(gap); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + // About 0.29 s later: kaksi, 0.6 to 0.9 s, then covers 1.15 s + int inside = columnOf(kaksi.getFrame()) + 40; + shiftPressAt(inRow(inside)); + shiftMoveHeldTo(inRow(inside + 100)); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("kaksi"), 1000); + shiftReleaseAt(inRow(inside + 100)); + QCOMPARE(highlightedWord(), QString("kaksi")); + + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString(), 1000); + + m_window->answerLyricsShift(0.3); + m_window->shiftLyricsAction()->trigger(); + QTRY_COMPARE_WITH_TIMEOUT(highlightedWord(), QString("kaksi"), 1000); + } + + // Export Lyrics after a shift writes the shifted times + void lyrics_edit_shift_then_export() { + makeWindow(FakeAudioIO::Config()); + showEditableLyrics(); + if (QTest::currentTestFailed()) return; + Lyrics was = lyricsFromEvents(lyricsEvents(), rate); + + m_window->answerLyricsShift(0.25); + m_window->shiftLyricsAction()->trigger(); + QCOMPARE(undoOnce(), QString("Shift Lyrics")); + QCOMPARE(redoOnce(), QString("Shift Lyrics")); + + QString path = m_dir.filePath("shifted-lyrics.ttml"); + m_window->setLyricsExportAnswer(path); + m_window->exportLyricsAction()->trigger(); + QFile file(path); + QVERIFY2(file.open(QIODevice::ReadOnly), "nothing was written"); + LyricsParseResult parsed = parseTtml(file.readAll()); + QVERIFY2(parsed.error == "", qPrintable(parsed.error)); + + const QVector &back = parsed.lyrics.words; + QCOMPARE(back.size(), was.words.size()); + for (int i = 0; i < back.size(); ++i) { + QString what = QString("word %1, \"%2\"").arg(i) + .arg(was.words[i].text); + QVERIFY2(back[i].text == was.words[i].text, + qPrintable(what + " came back as " + back[i].text)); + QVERIFY2(back[i].line == was.words[i].line, + qPrintable(what + ": another line")); + QVERIFY2(std::fabs(back[i].start - (was.words[i].start + 0.25)) + < 0.0005001, qPrintable(what + ": another start")); + QVERIFY2(std::fabs(back[i].end - (was.words[i].end + 0.25)) + < 0.0005001, qPrintable(what + ": another end")); + } + } + // Closing while pYIN is still running on the take (review finding // 15). Unless the analysis is cancelled first, about one run in // three under CPU load destroys the take's model on the transform From 0f1688ab63e34b2ea84b04160c39ffb117c0b052 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:13:40 +0000 Subject: [PATCH 158/275] test: the takes file suite's windows path assertions run on windows only back slashes as separators and names that ignore case are windows behaviour by design; on other systems the suite checks that names are case-sensitive and uses a posix absolute path. taken from bcbb5c9 on feat/tonyandroid: without it the core suite fails on linux and macos, in ci as well. the docs no longer list the four tests as failing there. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- docs/building.md | 4 ++-- docs/calibrate-audio-work-orders.md | 5 ++--- main/test/TestTakesFile.h | 26 +++++++++++++++++++++++--- 3 files changed, 27 insertions(+), 8 deletions(-) diff --git a/docs/building.md b/docs/building.md index 15d082e8..77a5dc6f 100644 --- a/docs/building.md +++ b/docs/building.md @@ -88,8 +88,8 @@ on 2026-09-25 (Ubuntu 24.04, no sound card): same tips. - `-j 4` on four cores; the whole build takes about 20 minutes. Run the app suite with nothing else building: it records in real time. -- Four tests of `TestTakesFile` fail on Linux and nowhere else: they are about Windows - paths (backslashes, drive letters, case). +- `TestTakesFile` checks Windows paths (backslashes, drive letters, case) on Windows only; + elsewhere it checks that names are case-sensitive and uses a POSIX absolute path. ## What is particular about this `meson.build` diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 2921259d..bfd34234 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -185,9 +185,8 @@ The detailed figures are awaited. So the ±2 ms of items 1 and 2 will fail on th machine as things stand: thresholds get tuned from real report files, not now. Whether to keep the stream running between takes (an svapp change) waits on those figures. -**Linux baseline** (Qt 6.4.2): core all green but 4 `TestTakesFile` tests -(`takes_folder`, `relative_audio_path`, `resolve_audio_path`, `in_folder`: Windows paths); -your final runs must show exactly these 4. `test-tony-app` green in about 8 minutes +**Linux baseline** (Qt 6.4.2): core all green (`TestTakesFile` checks Windows paths on +Windows only). `test-tony-app` green in about 8 minutes (`TestRecordWorkflow` 98, `TestUiChecks` 19, `TestAudioCheck` 23, …); `test-tony-dev` green. `tony-app-test.cpp` draws text without sub-pixel anti-aliasing, or Qt 6.4's coloured fringes on the scale's labels read as live dots in `TestUiChecks`. diff --git a/main/test/TestTakesFile.h b/main/test/TestTakesFile.h index 223466cd..c968a3d0 100644 --- a/main/test/TestTakesFile.h +++ b/main/test/TestTakesFile.h @@ -211,8 +211,12 @@ private slots: void takes_folder() { QCOMPARE(TakesFile::takesFolder("C:/songs/My Song.ton"), QString("C:/songs/My Song.takes")); +#ifdef Q_OS_WIN + // Back slashes separate the parts of a path on Windows only; + // elsewhere they are part of a name QCOMPARE(TakesFile::takesFolder("C:\\songs\\My Song.ton"), QString("C:/songs/My Song.takes")); +#endif QCOMPARE(TakesFile::takesFolder("C:/Käännös/Säkeistö 2.ton"), QString("C:/Käännös/Säkeistö 2.takes")); // Only the extension goes, so a name with dots of its own keeps them @@ -232,10 +236,12 @@ private slots: ("C:/songs/My Song.ton", "C:/songs/My Song.takes/take-1.wav"), QString("My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN QCOMPARE(TakesFile::relativeAudioPath ("C:/songs/My Song.ton", "C:\\songs\\My Song.takes\\take-1.wav"), QString("My Song.takes/take-1.wav")); +#endif QCOMPARE(TakesFile::relativeAudioPath ("C:/Käännös/Säkeistö.ton", "C:/Käännös/Säkeistö.takes/take-1.wav"), @@ -268,15 +274,23 @@ private slots: QCOMPARE(TakesFile::resolveAudioPath ("C:/songs/My Song.ton", "My Song.takes/take-1.wav"), QString("C:/songs/My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN QCOMPARE(TakesFile::resolveAudioPath ("C:\\songs\\My Song.ton", "My Song.takes\\take-1.wav"), QString("C:/songs/My Song.takes/take-1.wav")); +#endif QCOMPARE(TakesFile::resolveAudioPath ("C:/Käännös/Säkeistö.ton", "Säkeistö.takes/take-1.wav"), QString("C:/Käännös/Säkeistö.takes/take-1.wav")); - QCOMPARE(TakesFile::resolveAudioPath - ("C:/songs/My Song.ton", "C:/recorded/take-1.wav"), - QString("C:/recorded/take-1.wav")); + // An absolute path is one in the form of the system the session + // is read on: a drive letter makes one only on Windows +#ifdef Q_OS_WIN + QString elsewhere = "C:/recorded/take-1.wav"; +#else + QString elsewhere = "/recorded/take-1.wav"; +#endif + QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", elsewhere), + elsewhere); QCOMPARE(TakesFile::resolveAudioPath("C:/songs/My Song.ton", ""), QString()); @@ -302,9 +316,15 @@ private slots: void in_folder() { QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes", "C:/songs/My Song.takes/take-1.wav")); +#ifdef Q_OS_WIN // Windows tells no two names apart by their case QVERIFY(TakesFile::isInFolder("C:/songs/my song.takes", "C:\\Songs\\My Song.takes\\take-1.wav")); +#else + // Other systems do + QVERIFY(!TakesFile::isInFolder("/songs/my song.takes", + "/Songs/My Song.takes/take-1.wav")); +#endif QVERIFY(TakesFile::isInFolder("C:/songs/My Song.takes/", "C:/songs/My Song.takes/in/take-1.wav")); QVERIFY(!TakesFile::isInFolder("C:/songs/My Song.takes", From f5b4ed296bfa73894acaf661d6abcd0659f0fb01 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:13:40 +0000 Subject: [PATCH 159/275] fix: no helper named bold, which the macos sdk declares globally MacTypes.h declares bold, italic and six other style enumerators in the global namespace, so every call of the calibrate audio dialog's bold() helper was ambiguous and the macos build stopped there. building.md names the trap beside windows' near and far. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- docs/building.md | 4 ++++ main/CalibrateAudioDialog.cpp | 30 ++++++++++++++++-------------- 2 files changed, 20 insertions(+), 14 deletions(-) diff --git a/docs/building.md b/docs/building.md index 77a5dc6f..d0e65f69 100644 --- a/docs/building.md +++ b/docs/building.md @@ -104,3 +104,7 @@ on 2026-09-25 (Ubuntu 24.04, no sound card): `tony_core_files` or `tony_app_files`, and its header into the matching `*_moc_files` only if it declares `Q_OBJECT`. - Windows headers define `near` and `far` as macros. Do not use them as identifiers. +- The macOS SDK's `MacTypes.h` declares `normal`, `bold`, `italic`, `underline`, + `outline`, `shadow`, `condense` and `extend` in the global namespace. A function of one + of those names, even in an anonymous namespace, makes each unqualified call to it + ambiguous on macOS, and nothing else catches it. diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index daabd8b7..956ccdcd 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -78,8 +78,10 @@ paragraph(QString html) return "

" + html + "

"; } +// Not "bold": the macOS SDK declares an enumerator of that name in the +// global namespace, which makes every unqualified call ambiguous there QString -bold(QString html) +boldHtml(QString html) { return "" + html + ""; } @@ -289,7 +291,7 @@ CalibrateAudioDialog::devHtml() const if (m_devNote != "") html += paragraph(m_devNote.toHtmlEscaped()); if (!m_haveDevReport) return html; - html += paragraph(bold(tr("Dev checks"))); + html += paragraph(boldHtml(tr("Dev checks"))); if (m_devReport.failure != "") { html += paragraph(tr("They ended early: %1") .arg(m_devReport.failure.toHtmlEscaped())); @@ -559,10 +561,10 @@ CalibrateAudioDialog::instructionsHtml() const (tr("Tony plays short chirps and records them, to measure how late " "recordings arrive through your devices. What it measures is " "used to place your takes on the reference.")); - html += paragraph(bold(tr("Before you start:"))); + html += paragraph(boldHtml(tr("Before you start:"))); html += "
  • " + tr("Hold one earcup of your headphones against the microphone, " - "%1: the chirps are sharp.").arg(bold(tr("off your ears"))) + + "%1: the chirps are sharp.").arg(boldHtml(tr("off your ears"))) + "
  • " + tr("Set a moderate volume, and keep the room quiet.") + "
"; @@ -599,7 +601,7 @@ CalibrateAudioDialog::calibrationHtml() const const AudioCheckResult &r = m_result; if (r.failure != "") { - return paragraph(bold(tr("The check did not finish."))) + + return paragraph(boldHtml(tr("The check did not finish."))) + paragraph(r.failure.toHtmlEscaped()); } @@ -613,7 +615,7 @@ CalibrateAudioDialog::calibrationHtml() const // by it, further the later they come QString html; if (r.rateMismatch) { - html += paragraph(bold(tr("The recording device runs at %1 Hz; takes " + html += paragraph(boldHtml(tr("The recording device runs at %1 Hz; takes " "cannot line up until that is fixed.") .arg(hertz(r.recordingRate)))); html += paragraph @@ -624,14 +626,14 @@ CalibrateAudioDialog::calibrationHtml() const } else { switch (s.verdict) { case Verdict::Ok: - html += paragraph(bold(tr("The test sounds came back steadily, " + html += paragraph(boldHtml(tr("The test sounds came back steadily, " "%1 after they were played.") .arg(measured))); html += paragraph (tr("Press Use this latency to place your takes with it.")); break; case Verdict::NoSignal: - html += paragraph(bold(tr("Tony could not hear the test sounds: " + html += paragraph(boldHtml(tr("Tony could not hear the test sounds: " "it found %1 of %2.") .arg(s.found).arg(s.judged))); html += "
  • " + @@ -647,14 +649,14 @@ CalibrateAudioDialog::calibrationHtml() const "
"; break; case Verdict::Clipped: - html += paragraph(bold(tr("The test sounds were too loud: the " + html += paragraph(boldHtml(tr("The test sounds were too loud: the " "recording reached full scale."))); html += paragraph(tr("Turn the volume down, or hold the earcup a " "little away from the microphone, and " "check again.")); break; case Verdict::Fading: - html += paragraph(bold(tr("The test sounds got quieter as the " + html += paragraph(boldHtml(tr("The test sounds got quieter as the " "check went on, by %1 dB.") .arg(QLocale().toString (s.fadingDb, 'f', 0)))); @@ -665,14 +667,14 @@ CalibrateAudioDialog::calibrationHtml() const "enhancements, and check again.")); break; case Verdict::PositionDependent: - html += paragraph(bold(tr("The delay grew from one punch-in to " + html += paragraph(boldHtml(tr("The delay grew from one punch-in to " "the next."))); html += paragraph (tr("The recording seems to run at another speed than the " "playback, so no one latency places every take right.")); break; case Verdict::Scattered: - html += paragraph(bold(tr("The driver's timing varies from take " + html += paragraph(boldHtml(tr("The driver's timing varies from take " "to take by %1.").arg(timing))); html += paragraph (tr("No one latency places every take right when it varies " @@ -680,7 +682,7 @@ CalibrateAudioDialog::calibrationHtml() const "check again.")); break; case Verdict::Unsteady: - html += paragraph(bold(tr("The driver's timing varies from take " + html += paragraph(boldHtml(tr("The driver's timing varies from take " "to take by %1.").arg(timing))); html += paragraph (tr("That is small enough to use: the measured round trip, " @@ -700,7 +702,7 @@ CalibrateAudioDialog::calibrationHtml() const } if (m_latencyKept) { - html += paragraph(bold(tr("Kept.")) + " " + + html += paragraph(boldHtml(tr("Kept.")) + " " + tr("Takes on these devices are now placed with %1.") .arg(measured)); } From cf7d2fc19af532f65bf9f2a39b6ec92ace4a105b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:13:41 +0000 Subject: [PATCH 160/275] test: ci names each failed test and runs the suites one at a time meson test said only which executable failed, and --print-errorlogs shows the last 100 lines of a log, while one executable runs several qtest suites. a step after a failed test run prints each failed test, where it failed and the totals of its suite, from meson's full log. the app suite records in real time, so no suite runs beside another. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/linux.yml | 14 +++++++++++++- .github/workflows/macos.yml | 14 +++++++++++++- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index 9c2de2d5..d6b9dadf 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -39,4 +39,16 @@ jobs: - name: make run: ninja -C build - name: test - run: meson test -C build + id: test + run: meson test -C build --print-errorlogs --num-processes 1 + # meson shows at most the last 100 lines of a failing log, and one + # test executable runs several QTest suites: name every failed test, + # with where it failed and the totals of each suite that failed + - name: test-failures + if: failure() && steps.test.outcome == 'failure' + run: | + awk '/^test:/ { print; next } + /^Totals:/ { if ($0 !~ /, 0 failed/) print; next } + /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } + fail && /^ Loc:/ { print } + { fail = 0 }' build/meson-logs/testlog.txt diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml index 89f5318c..8c4e8a79 100644 --- a/.github/workflows/macos.yml +++ b/.github/workflows/macos.yml @@ -18,7 +18,19 @@ jobs: - name: make run: ninja -C build - name: test - run: meson test -C build + id: test + run: meson test -C build --print-errorlogs --num-processes 1 + # meson shows at most the last 100 lines of a failing log, and one + # test executable runs several QTest suites: name every failed test, + # with where it failed and the totals of each suite that failed + - name: test-failures + if: failure() && steps.test.outcome == 'failure' + run: | + awk '/^test:/ { print; next } + /^Totals:/ { if ($0 !~ /, 0 failed/) print; next } + /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } + fail && /^ Loc:/ { print } + { fail = 0 }' build/meson-logs/testlog.txt - name: deploy-app run: | ls -lR /usr/local/opt From fbdce6c0ef2ceee2b5209cfefb3477eb029d8d9b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:30:01 +0000 Subject: [PATCH 161/275] feat: dev checks for Stop on a long song, and the joins The dev run begins with a long reference of its own (longLayout() now takes a length, 240 s unless asked): two punch-ins far apart, each timed from Stop to its pitch merged against the whole song's analysis, and the pitch outside each range unchanged, which shows the ranged path ran (item 9). After the pre-roll stage, two punch-ins meet in the middle of a held tone; the join is judged for a step in the samples, the pitch track running through, one note across it, and nothing moved outside (item 10), each failure naming its part. Item 10's note part fails on today's code: the notes merge by onset drops the second punch-in's note. The passing test expects that failure until the merge is fixed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 26 ++ docs/calibrate-audio.md | 19 +- main/LatencyCheck.cpp | 7 +- main/LatencyCheck.h | 14 +- main/dev/DevChecks.cpp | 483 ++++++++++++++++++++++++++-- main/dev/DevChecks.h | 158 ++++++--- main/test/TestDevChecks.h | 175 +++++++--- main/test/TestLatencyCheck.h | 38 +++ 8 files changed, 802 insertions(+), 118 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index a6e188b5..f6e9146c 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -427,3 +427,29 @@ cancelled run still works out the checks of the stages it finished; the runner r `progress(Recording)` as the seconds left tick down. Left open: an overwrite question would come inside `record()`, before the observer starts, so item 14 cannot see it. Items 3 and 5 judge stage 1 only. `test-tony-dev` about 135 s. + +### Phase C2 — 2026-09-26 +Built: `LatencyCheck::longLayout(rate, seconds)`, default `kLongSeconds` (240), core test +`long_layout_of_another_length`. `DevChecks` stage 1 "Long song" (`Options::longSeconds`, +0 leaves it out and item 9 Skipped; `longPunchIns()`: the shortest range judging the first +sweep from 1/4 and from 5/8 of the song, 61.04–62.96 and 150.84–152.76 s at 240 s) and +stage 5 "Joins" (`joinPunchIns()`: [26.0, 28.7] and [28.7, 32.0], J = 28.7 s, the middle of +the tone 27.2–30.2 s; they judge 26.9 and 30.9 s). Items 9 `stop_on_a_long_song`, 10 +`the_joins`. `Run` carries its layout (`gapLooks()` takes it); items 1, 2, 4 count the long +song's punch-ins first (1–8 now); the reopen's `punchInsSoFar()` keeps to the dev take. +Choices: timed by `longSongStep()` from the runner's reports: whole song = its +AnalysingReference step (writing and opening before it, 0.14 s at 60 s, not counted); a +punch-in = its AnalysingTake step, ending at the next Recording report (after `record()`) +or at `finished()` (after the judging). Item 9's pitch before a punch-in is read at its first +Recording report. Item 10's messages name the part: "step:", "pitch:", "note:", "outside:". +On the fake: 60 s song 2.9–3.0 s, punch-ins 0.59–0.70 s (20–24 %); 240 s: 10.4 s, 0.63 and +0.75 s (6–7 %). At J the step reads −8.3 dB (the two 5 ms fades, a dip), pitch gap 1 hop. +Found: item 10 fails, the note only: the first punch-in's note ends 5.7 ms past J, the tone +after J has none (the ranged run starts 0.5 s before J, so its note begins before W and is +dropped). Analysing the whole take gives one note, 27.21–30.20 s. `QEXPECT_FAIL` in the test. +Seen failing: item 9 (Stop analysing the whole take: 40 % and 72 %, pitch changed), item +10's step (no fade-in: 29.4 dB). Tests: the passing run and the cancel, close and dialog tests +have a 60 s long song; the fault runs and the deletion test leave it out. +Left open: item 12 (C1c) once found 0 looks in its 0.4 s lead-in gap (15–16 usual, 9 once): +a silent GUI stall of 340 ms or more there, likely this VM's disk; a real run could show it. +The runner's 60 s limit on the reference's analysis is 6× 240 s's 10 s. `test-tony-dev` 180 s. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index e6d6253a..aabc682d 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -115,7 +115,7 @@ The numbers are those of `docs/manual-checklist.md` before that merge. | 7 | Record from a position; overwrite question | **Automated** (placement) | Placement as in 1. Outside the new range, the take's audio is bit-identical and pitch and notes are unchanged beyond ±0.25 s. The question, No, and "Don't ask again" stay with the app suite (`record_over_existing_question`) and a glance. | | 8 | Cursor from P, pane follows, all in one place | **Measured** | *Automated:* the cursor is at S when the take starts, and inside the visible range throughout. *Reported:* the dot-to-cursor offset. *Eyes:* the rest. | | 9 | Stop on a 4-minute song | **Automated** | A generated 4-minute reference, punch-in near the end. Time from Stop to new pitch, against a threshold. Pitch outside the range unchanged, which proves the ranged path ran. | -| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. | +| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. *Found in C2:* the note fails on today's code. The second punch-in's analysis starts 0.5 s before the join, inside the tone, so its note begins before the merge window and is dropped; the first's note ends at the join. A whole-take analysis gives one note. | | 11 | Is 3 s right, is the countdown readable | **Manual** | A judgement. | | 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. *Found in C1c:* on a noiseless loopback the earlier take is silent wherever the reference is, so a take played out shows in no gap; the app test gives the fake a −60 dBFS noise floor, as a room gives a real mic. | | 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown counts only the 1 s there is (*found in C1c:* plus the round trip, by design, so it starts at 2; for the instant before the round trip is known it shows 1), placement is right. | @@ -241,12 +241,15 @@ setting changes only through Use this latency. | Step | What it does | Items | Phase | | --- | --- | --- | --- | -| 1 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 4, 5, 8 | C1a, C1b | -| 2 | Re-record over an earlier punch-in, through its lead-in | 7, 12, 14 | C1c | -| 3 | Punch-in at P = 1 s with a 3 s pre-roll | 13 | C1c | -| 4 | Two adjacent punch-ins meeting inside a held tone | 10 | C2 | -| 5 | Save to the scratch `.ton` and reopen | 1 | C1a | -| 6 | Long reference, two punch-ins far apart | 9 (and 1, 2) | C2 | +| 1 | Long reference, two punch-ins far apart | 9 (and 1, 2, 4) | C2 | +| 2 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 4, 5, 8 | C1a, C1b | +| 3 | Re-record over an earlier punch-in, through its lead-in | 7, 12, 14 | C1c | +| 4 | Punch-in at P = 1 s with a 3 s pre-roll | 13 | C1c | +| 5 | Two adjacent punch-ins meeting inside a held tone | 10 | C2 | +| 6 | Save to the scratch `.ton` and reopen | 1 | C1a | + +*Since C2* the long reference comes first: the dev reference then replaces its session +as a check's own, unsaved, without asking, and the run still ends on the saved session. Items 15 and 16 and the smoke group are no longer in the dev run (§4). @@ -327,7 +330,7 @@ marked "Done" when it is committed. decision). The phases below were cut again (§4). - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. Done. -5. **C2** Long song and joins: items 9 and 10. +5. **C2** Long song and joins: items 9 and 10. Done. 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. 7. **Release build** by the lead (§8, "Release builds must stay clean"). 8. **D** Docs, from the code and the phase log: diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp index 779752df..ba3069c3 100644 --- a/main/LatencyCheck.cpp +++ b/main/LatencyCheck.cpp @@ -45,7 +45,6 @@ const int firstSweepTenths = 10; const double calibrationSeconds = 26.0; const double devSeconds = 40.0; -const double longSeconds = 240.0; // The silence after the last event, as the calibration layout has it const double tailSeconds = 0.8; @@ -141,7 +140,7 @@ LatencyCheck::devLayout(sv_samplerate_t rate) } LatencyCheck::Layout -LatencyCheck::longLayout(sv_samplerate_t rate) +LatencyCheck::longLayout(sv_samplerate_t rate, double seconds) { Layout layout; layout.rate = rate; @@ -149,13 +148,13 @@ LatencyCheck::longLayout(sv_samplerate_t rate) const double eventSeconds = kSweepSeconds + kPauseSeconds + kToneSeconds; int at = firstSweepTenths; - for (int i = 0; at / 10.0 + eventSeconds + tailSeconds <= longSeconds; + for (int i = 0; at / 10.0 + eventSeconds + tailSeconds <= seconds; ++i) { addEvent(layout, at, kToneSeconds); at += calibrationSpacings[i % calibrationSpacingCount]; } - layout.length = framesAt(longSeconds, rate); + layout.length = framesAt(seconds, rate); return layout; } diff --git a/main/LatencyCheck.h b/main/LatencyCheck.h index c3e81712..8c65b5f3 100644 --- a/main/LatencyCheck.h +++ b/main/LatencyCheck.h @@ -149,10 +149,16 @@ namespace LatencyCheck /// ends before the next sweep Layout devLayout(sv::sv_samplerate_t rate = kReferenceRate); - /// 4 minutes of events at the calibration spacings, over and over. - /// No more than eleven spacings can differ by 0.1 s inside 1.6 to - /// 2.6 s, so here any eleven in a row do - Layout longLayout(sv::sv_samplerate_t rate = kReferenceRate); + /// How long the long layout is unless asked otherwise: a song + constexpr double kLongSeconds = 240.0; + + /// 4 minutes, or the length given, of events at the calibration + /// spacings, over and over, as many as end in time for the silence + /// the calibration layout ends with. No more than eleven spacings + /// can differ by 0.1 s inside 1.6 to 2.6 s, so here any eleven in a + /// row do. A shorter one is the start of the 4-minute one + Layout longLayout(sv::sv_samplerate_t rate = kReferenceRate, + double seconds = kLongSeconds); /// One sweep at the given rate, as the reference has it std::vector sweep(sv::sv_samplerate_t rate); diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index acd358e7..310da864 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -378,6 +378,35 @@ DevChecks::~DevChecks() } } +vector +DevChecks::longPunchIns(const LatencyCheck::Layout &layout) +{ + // All that judgeTake() reads for a sweep, and the little more that + // LatencyCheck::punchInsFor() leaves + const double before = LatencyCheck::kSearchSeconds + + LatencyCheck::kJudgeMarginSeconds + LatencyCheck::kPunchInSlackSeconds; + const double after = LatencyCheck::kSearchSeconds + + LatencyCheck::kSweepSeconds + LatencyCheck::kJudgeMarginSeconds + + LatencyCheck::kPunchInSlackSeconds; + if (layout.rate <= 0) return {}; + const double length = double(layout.length) / layout.rate; + + vector punchIns; + for (double share : { 0.25, 0.625 }) { + for (const LatencyCheck::Event &e : layout.events) { + const double at = double(e.sweepStart) / layout.rate; + if (at < share * length) continue; + punchIns.push_back(LatencyCheck::PunchIn(at - before, at + after)); + break; + } + } + if (punchIns.size() != 2 || punchIns[0].end > punchIns[1].start || + punchIns[1].end > length) { + return {}; + } + return punchIns; +} + vector DevChecks::freshPunchIns() { @@ -385,6 +414,13 @@ DevChecks::freshPunchIns() LatencyCheck::PunchIn(16.8, 21.2) }; } +vector +DevChecks::joinPunchIns() +{ + return { LatencyCheck::PunchIn(26.0, 28.7), + LatencyCheck::PunchIn(28.7, 32.0) }; +} + LatencyCheck::PunchIn DevChecks::reRecording() { @@ -462,12 +498,16 @@ DevChecks::start(const Options &options) m_layout = LatencyCheck::devLayout(); m_observedPunchIn = 0; m_watched.clear(); + m_long = LongSong(); + m_long.layout = LatencyCheck::longLayout(m_layout.rate, + m_options.longSeconds); m_haveFresh = false; m_fresh = AudioCheckResult(); m_coverageAfterFresh = Coverage(); m_freshWatched.clear(); m_reRecord = PunchInStage(); m_nearStart = PunchInStage(); + m_joins = PunchInStage(); m_pitchBefore.clear(); m_notesBefore.clear(); m_pitchAfter.clear(); @@ -479,6 +519,12 @@ DevChecks::start(const Options &options) m_runnerResult = AudioCheckResult(); m_stages.clear(); + if (m_options.longSeconds > 0.0) { + m_stages.push_back({ tr("Long song"), + [this]() { beginLongSong(); }, + [this]() { return longSongDone(); }, + kCheckStageTimeoutMs }); + } m_stages.push_back({ tr("Fresh punch-ins"), [this]() { beginFreshPunchIns(); }, [this]() { return freshPunchInsDone(); }, @@ -486,7 +532,7 @@ DevChecks::start(const Options &options) m_stages.push_back({ tr("Re-record"), [this]() { beginPunchInStage - (m_reRecord, reRecording(), + (m_reRecord, { reRecording() }, AudioCheckRunner::kPreRollSeconds); }, [this]() { return punchInStageDone(m_reRecord); }, @@ -494,11 +540,19 @@ DevChecks::start(const Options &options) m_stages.push_back({ tr("Pre-roll near the start"), [this]() { beginPunchInStage - (m_nearStart, nearTheStart(), + (m_nearStart, { nearTheStart() }, kNearStartPreRollSeconds); }, [this]() { return punchInStageDone(m_nearStart); }, kCheckStageTimeoutMs }); + m_stages.push_back({ tr("Joins"), + [this]() { + beginPunchInStage + (m_joins, joinPunchIns(), + AudioCheckRunner::kPreRollSeconds); + }, + [this]() { return punchInStageDone(m_joins); }, + kCheckStageTimeoutMs }); m_stages.push_back({ tr("Save and reopen"), [this]() { beginReopen(); }, [this]() { return reopenDone(); }, @@ -589,6 +643,7 @@ DevChecks::runnerFinished(const AudioCheckResult &result) { // A run the dialog started, or anyone else, is not ours if (!m_runnerRunning) return; + if (m_long.timing) longSongStep(AudioCheckRunner::Step::Idle, 0); if (m_observer->isObserving()) finishObservation(); m_runnerRunning = false; m_runnerResult = result; @@ -598,6 +653,7 @@ void DevChecks::runnerProgress(const AudioCheckRunner::Progress &state) { if (!m_runnerRunning) return; + if (m_long.timing) longSongStep(state.step, state.punchIn); // Reported once the take has started, from the same poll of the // runner's. A punch-in's analysis is done when the next one starts @@ -625,6 +681,46 @@ DevChecks::finishObservation() m_observedPunchIn = 0; } +void +DevChecks::longSongStep(AudioCheckRunner::Step step, int punchIn) +{ + // Reported again as the seconds left go down + LongSong &s = m_long; + if (step == s.step && punchIn == s.punchIn) return; + + // Each step ends as the next begins: the whole song's analysis as + // the first punch-in starts to record, and a punch-in's analysis as + // the next starts or the runner ends. The whole song's is timed from + // its session open, as a song opened by the user is analysed, and a + // punch-in's from the take stopped: the steps the audio check waits + using Step = AudioCheckRunner::Step; + const qint64 now = s.clock.elapsed(); + const double seconds = double(now - s.stepFrom) / 1000.0; + if (s.step == Step::AnalysingReference) s.wholeSongSeconds = seconds; + if (s.step == Step::AnalysingTake) s.stopSeconds.push_back(seconds); + + // The take's pitch before each punch-in, which is the pitch after the + // one before it. Starting a take hides the pitch but leaves its model + if (step == Step::Recording) { + s.pitch.push_back(takeEvents(Analyser::PitchTrack)); + } + + if (s.step == Step::OpeningReference) { + cerr << "DevChecks: the long song was written and opened in " + << seconds << " s" << endl; + } else if (s.step == Step::AnalysingReference) { + cerr << "DevChecks: the long song was analysed in " << seconds + << " s" << endl; + } else if (s.step == Step::AnalysingTake) { + cerr << "DevChecks: punch-in " << s.punchIn << " into the long song " + << "was analysed in " << seconds << " s" << endl; + } + + s.step = step; + s.punchIn = punchIn; + s.stepFrom = now; +} + const DevChecks::Watched * DevChecks::freshWatched(int i) const { @@ -634,6 +730,48 @@ DevChecks::freshWatched(int i) const return nullptr; } +void +DevChecks::beginLongSong() +{ + AudioCheckRunner::Plan plan; + plan.layout = m_long.layout; + plan.ranges = longPunchIns(m_long.layout); + plan.roundTrip = m_options.roundTrip; + + m_watched.clear(); + m_observedPunchIn = 0; + m_runnerResult = AudioCheckResult(); + m_long.timing = true; + m_long.clock.start(); + m_runnerRunning = m_runner && m_runner->start(plan); + if (!m_runnerRunning) { + m_long.timing = false; + end(tr("The audio check could not start on a long song of %1 s.") + .arg(m_options.longSeconds)); + } +} + +bool +DevChecks::longSongDone() +{ + if (m_runnerRunning) return false; + m_long.timing = false; + + if (m_runnerResult.failure != "") { + end(tr("The punch-ins into the long song could not be recorded: %1") + .arg(m_runnerResult.failure)); + return false; + } + + // The runner has waited for the last punch-in's analysis + m_long.result = m_runnerResult; + m_long.coverage = m_window->m_takes->getCoverage(); + m_long.watched = m_watched; + m_long.pitch.push_back(takeEvents(Analyser::PitchTrack)); + m_long.done = true; + return true; +} + void DevChecks::beginFreshPunchIns() { @@ -676,16 +814,17 @@ DevChecks::freshPunchInsDone() } void -DevChecks::beginPunchInStage(PunchInStage &stage, LatencyCheck::PunchIn range, +DevChecks::beginPunchInStage(PunchInStage &stage, + vector ranges, double preRoll) { - // The take as the stages before left it, to compare with what this - // punch-in leaves + // The take as the stages before left it, to compare with what these + // punch-ins leave stage.before = snapshot(); AudioCheckRunner::Plan plan; plan.layout = m_layout; - plan.ranges = { range }; + plan.ranges = ranges; plan.keepSession = true; plan.roundTrip = m_options.roundTrip; plan.preRoll = preRoll; @@ -744,12 +883,17 @@ vector DevChecks::runs() const { vector all; + if (m_long.done) { + all.push_back({ &m_long.layout, &m_long.result, &m_long.coverage, + &m_long.watched }); + } if (m_haveFresh) { - all.push_back({ &m_fresh, &m_coverageAfterFresh, &m_freshWatched }); + all.push_back({ &m_layout, &m_fresh, &m_coverageAfterFresh, + &m_freshWatched }); } - for (const PunchInStage *stage : { &m_reRecord, &m_nearStart }) { + for (const PunchInStage *stage : { &m_reRecord, &m_nearStart, &m_joins }) { if (stage->done) { - all.push_back({ &stage->result, &stage->after.coverage, + all.push_back({ &m_layout, &stage->result, &stage->after.coverage, &stage->watched }); } } @@ -768,8 +912,10 @@ DevChecks::watchedOf(const Run &run, int i) vector DevChecks::punchInsSoFar() const { + // Not the long song's: that was another session, and another take vector ranges; for (const Run &run : runs()) { + if (run.layout != &m_layout) continue; for (const LatencyCheck::PunchInResult &p : run.result->summary.punchIns) { ranges.push_back(p.range); @@ -886,6 +1032,7 @@ DevChecks::end(QString failure) m_timer->stop(); m_observer->stop(); m_observedPunchIn = 0; + m_long.timing = false; m_stage = -1; m_begun = false; @@ -919,6 +1066,7 @@ DevChecks::evaluate(QString reason) const return { latencyCheck(reason), phrasesCheck(reason), liveDotsCheck(reason), speakersCheck(reason), micChannelCheck(reason), positionCheck(reason), + longSongCheck(reason), joinsCheck(reason), leadInCheck(reason), nearStartCheck(reason), stopsItselfCheck(reason) }; } @@ -930,7 +1078,8 @@ DevChecks::latencyCheck(QString reason) const c.item = 1; c.name = "latency_on_this_machine"; - if (!m_haveFresh) { + const vector all = runs(); + if (all.empty()) { c.verdict = CheckResult::Verdict::Skipped; c.message = reason; return c; @@ -941,7 +1090,7 @@ DevChecks::latencyCheck(QString reason) const QStringList problems; double largest = 0.0; int n = 0; - for (const Run &run : runs()) { + for (const Run &run : all) { const LatencyCheck::TakeSummary &s = run.result->summary; for (int i = 0; i < int(s.punchIns.size()); ++i) { const LatencyCheck::PunchInResult &p = s.punchIns[i]; @@ -977,7 +1126,7 @@ DevChecks::latencyCheck(QString reason) const if (n == 0) problems << tr("no punch-in was judged"); c.numbers.push_back({ tr("largest offset"), signedMs(largest) }); c.numbers.push_back({ tr("round trip used"), - unsignedMs(m_fresh.usedRoundTrip) }); + unsignedMs(all.front().result->usedRoundTrip) }); if (!m_reopened) { c.verdict = CheckResult::Verdict::Skipped; @@ -1050,7 +1199,7 @@ DevChecks::phrasesCheck(QString reason) const c.item = 2; c.name = "several_phrases_in_one_take"; - if (!m_haveFresh) { + if (runs().empty()) { c.verdict = CheckResult::Verdict::Skipped; c.message = reason; return c; @@ -1248,7 +1397,8 @@ DevChecks::liveDotsCheck(QString reason) const } DevChecks::GapLooks -DevChecks::gapLooks(const TakeObserver::Observation &o, +DevChecks::gapLooks(const LatencyCheck::Layout &layout, + const TakeObserver::Observation &o, const TakeLatency &t, sv_samplerate_t rate, double until) const { @@ -1262,7 +1412,6 @@ DevChecks::gapLooks(const TakeObserver::Observation &o, g.placed = true; // The reference's sounds, in seconds - const LatencyCheck::Layout &layout = m_layout; const double sweepSeconds = double(LatencyCheck::sweep(layout.rate).size()) / layout.rate; vector> sounds; @@ -1346,7 +1495,8 @@ DevChecks::speakersCheck(QString reason) const c.item = 4; c.name = "nothing_of_the_take_in_the_speakers"; - if (!m_haveFresh) { + const vector all = runs(); + if (all.empty()) { c.verdict = CheckResult::Verdict::Skipped; c.message = reason; return c; @@ -1356,8 +1506,8 @@ DevChecks::speakersCheck(QString reason) const // The input played back out, by Tony or by the system, reaches the // mic again a little later: every sweep arrives twice. In any run - const LatencyCheck::Echo *echo = &m_fresh.summary.echo; - for (const Run &run : runs()) { + const LatencyCheck::Echo *echo = &all.front().result->summary.echo; + for (const Run &run : all) { if (run.result->summary.echo.heard) { echo = &run.result->summary.echo; break; @@ -1384,7 +1534,7 @@ DevChecks::speakersCheck(QString reason) const int gapPolls = 0; QString firstHeard; int n = 0; - for (const Run &run : runs()) { + for (const Run &run : all) { const LatencyCheck::TakeSummary &s = run.result->summary; for (int i = 0; i < int(s.punchIns.size()); ++i) { ++n; @@ -1425,7 +1575,8 @@ DevChecks::speakersCheck(QString reason) const const TakeLatency t = (i < int(run.result->takes.size()) ? run.result->takes[i] : TakeLatency()); - const GapLooks g = gapLooks(o, t, run.result->referenceRate, + const GapLooks g = gapLooks(*run.layout, o, t, + run.result->referenceRate, std::numeric_limits::max()); if (!g.placed) { problems << tr("punch-in %1's start gap was not measured, " @@ -1644,6 +1795,289 @@ DevChecks::positionCheck(QString reason) const return c; } +CheckResult +DevChecks::longSongCheck(QString reason) const +{ + CheckResult c; + c.item = 9; + c.name = "stop_on_a_long_song"; + + const LongSong &s = m_long; + const LatencyCheck::TakeSummary &summary = s.result.summary; + if (!s.done || summary.punchIns.empty()) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = (m_options.longSeconds > 0.0 ? reason : + tr("The long song was left out of this run.")); + return c; + } + + // Stop analyses only the range recorded, and merges it in: far + // quicker than the whole song's analysis, and nothing of the take's + // pitch outside the range changes, beyond the margin the merge may + // touch. The second punch-in has the first's pitch to leave alone, + // which a whole-take analysis would make again + const sv_samplerate_t rate = s.result.referenceRate; + const double whole = s.wholeSongSeconds; + QStringList problems; + c.numbers.push_back + ({ tr("whole-song analysis"), + whole < 0.0 ? tr("not timed") : + tr("%1 s, of a song of %2 s, from its session open") + .arg(secondsText(whole)) + .arg(secondsText(double(s.layout.length) / s.layout.rate)) }); + if (whole < 0.0) problems << tr("the whole song's analysis was not timed"); + + int compared = 0; + for (int i = 0; i < int(summary.punchIns.size()); ++i) { + const LatencyCheck::PunchIn &p = summary.punchIns[i].range; + const QString range = rangeText(p.start, p.end); + if (i >= int(s.stopSeconds.size())) { + problems << tr("punch-in %1's analysis was not timed").arg(i + 1); + } else { + const double stop = s.stopSeconds[i]; + c.numbers.push_back + ({ tr("Stop to pitch merged, punch-in %1 (%2)").arg(i + 1) + .arg(range), + whole > 0.0 ? + tr("%1 s, %2 per cent of the whole song's") + .arg(secondsText(stop)).arg(100.0 * stop / whole, 0, 'f', 0) + : tr("%1 s").arg(secondsText(stop)) }); + if (whole >= 0.0 && !(stop < kStopShare * whole)) { + problems << tr("punch-in %1 took %2 s from Stop to its pitch " + "merged, not under %3 of the whole song's " + "analysis, %4 s") + .arg(i + 1).arg(secondsText(stop)).arg(kStopShare) + .arg(secondsText(whole)); + } + } + + if (i + 1 >= int(s.pitch.size())) { + problems << tr("the take's pitch around punch-in %1 was not seen") + .arg(i + 1); + continue; + } + const EventVector &before = s.pitch[i]; + const EventVector &after = s.pitch[i + 1]; + const Coverage::Range r(frameAt(p.start, rate), frameAt(p.end, rate)); + const TakeDiff::EventDiff pitch = + TakeDiff::eventsOutside(before, after, r, rate); + compared += countOutside(before, pitch.window); + if (!pitch.pass) { + problems << tr("punch-in %1 changed the take's pitch outside its " + "range").arg(i + 1); + } + c.numbers.push_back + ({ tr("pitch, punch-in %1").arg(i + 1), + eventsText(pitch, before, tr("pitch events"), + tr("outside %1") + .arg(rangeText(double(pitch.window.start) / rate, + double(pitch.window.end) / rate)), + rate) }); + } + if (compared == 0) { + problems << tr("the take had no pitch outside the punch-ins to " + "compare"); + } + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("On a song of %1 s, each punch-in had its pitch merged " + "in under %2 of the time the whole song's analysis " + "took, and left the take's pitch beyond %3 s of its " + "range as it was: only the range was analysed.") + .arg(secondsText(double(s.layout.length) / s.layout.rate)) + .arg(kStopShare).arg(TakeDiff::kEventMarginSeconds); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + +CheckResult +DevChecks::joinsCheck(QString reason) const +{ + CheckResult c; + c.item = 10; + c.name = "the_joins"; + + const PunchInStage &stage = m_joins; + const LatencyCheck::TakeSummary &s = stage.result.summary; + if (!stage.done || s.punchIns.size() < 2) { + c.verdict = CheckResult::Verdict::Skipped; + c.message = reason; + return c; + } + + // Two punch-ins that meet in the middle of a held tone, as a singer + // punching in twice within one note: at the join the samples run on + // without a step, the pitch track without a hole or a frame twice, + // one note runs through it, and outside the two ranges the take's + // pitch and notes are as they were. Each part is named when it + // fails: two punch-ins placed differently, as on a device whose + // offset moves when its stream restarts, show in the step and the + // pitch, and the note merge may fail on its own + const sv_samplerate_t rate = stage.result.referenceRate; + const LatencyCheck::PunchIn &first = s.punchIns[0].range; + const LatencyCheck::PunchIn &second = s.punchIns[1].range; + const sv_frame_t join = frameAt(first.end, rate); + auto at = [rate](sv_frame_t frame) { + return QString("%1 s").arg(double(frame) / rate, 0, 'f', 3); + }; + QStringList problems; + c.numbers.push_back({ tr("join"), + tr("%1 s, between %2 and %3") + .arg(secondsText(first.end)) + .arg(rangeText(first.start, first.end)) + .arg(rangeText(second.start, second.end)) }); + + // Where each landed, for reading the rest: items 1 and 2 judge it + QStringList placing; + c.numbers.push_back({ tr("offsets"), offsetsOf(s, placing) }); + if (s.punchIns[0].found > 0 && s.punchIns[1].found > 0) { + c.numbers.push_back({ tr("second punch-in against the first"), + signedMs(s.punchIns[1].medianOffset - + s.punchIns[0].medianOffset) }); + } + + const Snapshot &before = stage.before; + const Snapshot &after = stage.after; + if (after.audioError != "") { + problems << tr("step: %1").arg(after.audioError); + } else { + const TakeDiff::SampleStep step = TakeDiff::stepAt + (after.audio.data(), sv_frame_t(after.audio.size()), 1, rate, + join); + c.numbers.push_back + ({ tr("step at the join"), + tr("%1 dB, at most %2: the largest first difference %3 at %4, " + "the typical one %5") + .arg(step.stepDb, 0, 'f', 1).arg(TakeDiff::kMaxStepDb) + .arg(levelText(step.largest)).arg(at(step.largestAt)) + .arg(levelText(step.typical)) }); + if (!step.pass) { + problems << tr("step: the samples jump at the join, %1 dB over " + "their typical step, more than %2 dB") + .arg(step.stepDb, 0, 'f', 1).arg(TakeDiff::kMaxStepDb); + } + } + + const TakeDiff::PitchJoin pitch = + TakeDiff::pitchAcross(after.pitch, join, rate); + c.numbers.push_back + ({ tr("pitch across the join"), + tr("%1 events within %2 s of it; the largest gap %3 (%4 hops) " + "from %5; %6 frames twice, %7 out of order") + .arg(pitch.events).arg(TakeDiff::kPitchWindowSeconds) + .arg(unsignedMs(double(pitch.largestGap) / rate)) + .arg(double(pitch.largestGap) / double(TakeDiff::kHopFrames), 0, + 'f', 1) + .arg(at(pitch.largestGapFrom)).arg(pitch.doubled) + .arg(pitch.outOfOrder) }); + if (!pitch.pass) { + QStringList why; + if (pitch.largestGap > TakeDiff::kMaxGapHops * TakeDiff::kHopFrames) { + why << tr("a gap of %1 from %2") + .arg(unsignedMs(double(pitch.largestGap) / rate)) + .arg(at(pitch.largestGapFrom)); + } + if (pitch.doubled > 0) { + why << tr("%1 frames twice, the first at %2").arg(pitch.doubled) + .arg(at(pitch.firstDoubled)); + } + if (pitch.outOfOrder > 0) { + why << tr("%1 events out of order, the first at %2") + .arg(pitch.outOfOrder).arg(at(pitch.firstOutOfOrder)); + } + problems << tr("pitch: the track does not run through the join: %1") + .arg(why.join(", ")); + } + + // And every note within a second of it, to show how a note that does + // not run through came apart + const TakeDiff::NoteJoin notes = + TakeDiff::notesAcross(after.notes, join, rate); + auto noteText = [rate](const Event &e) { + return rangeText(double(e.getFrame()) / rate, + double(e.getFrame() + e.getDuration()) / rate); + }; + QStringList spanning, close, around; + for (const Event &e : notes.spanning) spanning << noteText(e); + for (const Event &e : notes.edgesNear) close << noteText(e); + const sv_frame_t oneSecond = frameAt(1.0, rate); + for (const Event &e : after.notes) { + if (e.getFrame() < join + oneSecond && + e.getFrame() + e.getDuration() > join - oneSecond) { + around << noteText(e); + } + } + c.numbers.push_back({ tr("notes at the join"), + spanning.isEmpty() ? tr("none") : + spanning.join(", ") }); + c.numbers.push_back({ tr("nearest note edge"), + after.notes.empty() ? tr("no notes") : + signedMs(double(notes.nearestEdge) / rate) }); + c.numbers.push_back({ tr("notes within 1 s of the join"), + around.isEmpty() ? tr("none") : around.join(", ") }); + if (!notes.pass) { + QStringList why; + if (notes.spanning.size() != 1) { + why << tr("%1 notes hold the join, not one") + .arg(notes.spanning.size()); + } + if (!close.isEmpty()) { + why << tr("a note begins or ends within %1 s of it: %2") + .arg(TakeDiff::kNoteClearanceSeconds).arg(close.join(", ")); + } + problems << tr("note: one note does not run through the join: %1") + .arg(why.join(", and ")); + } + + // Nothing moved outside the two ranges together, beyond the margin + const Coverage::Range both(frameAt(first.start, rate), + frameAt(second.end, rate)); + const TakeDiff::EventDiff pitchOutside = + TakeDiff::eventsOutside(before.pitch, after.pitch, both, rate); + const TakeDiff::EventDiff notesOutside = + TakeDiff::eventsOutside(before.notes, after.notes, both, rate); + const QString where = tr("outside %1") + .arg(rangeText(double(pitchOutside.window.start) / rate, + double(pitchOutside.window.end) / rate)); + if (!pitchOutside.pass) { + problems << tr("outside: the take's pitch outside the ranges changed"); + } + if (!notesOutside.pass) { + problems << tr("outside: the take's notes outside the ranges changed"); + } + if (countOutside(before.pitch, pitchOutside.window) == 0) { + problems << tr("outside: the take had no pitch outside the ranges to " + "compare"); + } + c.numbers.push_back({ tr("pitch outside"), + eventsText(pitchOutside, before.pitch, + tr("pitch events"), where, rate) }); + c.numbers.push_back({ tr("notes outside"), + eventsText(notesOutside, before.notes, tr("notes"), + where, rate) }); + + if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Two punch-ins meeting at %1 s, in the middle of a " + "held tone, left no step in the samples there, the " + "pitch track running through it, one note across it " + "and no note beginning or ending within %2 s of it, " + "and the take's pitch and notes beyond %3 s of the " + "two ranges as they were.") + .arg(secondsText(first.end)) + .arg(TakeDiff::kNoteClearanceSeconds) + .arg(TakeDiff::kEventMarginSeconds); + } else { + c.verdict = CheckResult::Verdict::Fail; + c.message = problems.join("; ") + "."; + } + return c; +} + CheckResult DevChecks::leadInCheck(QString reason) const { @@ -1718,7 +2152,7 @@ DevChecks::leadInCheck(QString reason) const if (!w) { problems << tr("the punch-in was not watched"); } else { - const GapLooks g = gapLooks(w->seen, t, rate, p.start); + const GapLooks g = gapLooks(m_layout, w->seen, t, rate, p.start); if (!g.placed) { problems << tr("the start gap was not measured, so what the " "lead-in played could not be placed"); @@ -1945,7 +2379,8 @@ DevChecks::stopsItselfCheck(QString reason) const const double past = double(stage->recorded - needed) / t.recordingRate; const double allowed = kStopMarginSeconds + poll + gapLooks - (w->seen, t, rate, std::numeric_limits::max()).margin; + (m_layout, w->seen, t, rate, + std::numeric_limits::max()).margin; c.numbers.push_back ({ tr("stopped, %1").arg(range), tr("%1 past the end of the selection, at most %2 allowed") @@ -2047,7 +2482,9 @@ DevChecks::writeReport(const DevReport &report) const drivers << QString::fromStdString(name); } out << "Audio drivers built in: " << drivers.join(", ") << "\n"; - const sv_samplerate_t recordingRate = m_fresh.recordingRate; + const vector all = runs(); + const sv_samplerate_t recordingRate = + all.empty() ? 0 : all.front().result->recordingRate; sv_samplerate_t outputRate = m_window->m_playSource ? m_window->m_playSource->getDeviceSampleRate() : 0; if (outputRate <= 0) outputRate = recordingRate; diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index c41852d3..a9164dcf 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -99,29 +99,38 @@ struct DevReport * replaced, at any moment, and a run ends cleanly then. * * The stages: - * 1. Fresh punch-ins: a run of the audio check on the dev layout, in - * a reference of its own (replacing the calibration's session - * without asking), with the two punch-ins of freshPunchIns() and - * the round trip given. - * 2. Re-record: a run into the same session and take, with the one - * punch-in of reRecording(), over part of stage 1's second. - * 3. Pre-roll near the start: the same with nearTheStart(), and a + * 1. Long song: a run of the audio check on a long reference of its + * own (Options::longSeconds; replacing the calibration's session + * without asking), with the two punch-ins far apart of + * longPunchIns(). First, so that the next stage replaces this + * session as a check's own, unsaved, and no saved one is ever + * replaced. + * 2. Fresh punch-ins: a run on the dev layout, in a reference of its + * own, with the two punch-ins of freshPunchIns() and the round trip + * given. + * 3. Re-record: a run into the same session and take, with the one + * punch-in of reRecording(), over part of stage 2's second. + * 4. Pre-roll near the start: the same with nearTheStart(), and a * pre-roll longer than the song before it. - * 4. Save and reopen: the session saved into a scratch folder of this + * 5. Joins: the same with the two punch-ins of joinPunchIns(), which + * meet in the middle of a held tone. + * 6. Save and reopen: the session saved into a scratch folder of this * run, the way Save As saves once it has a name, then opened again * and its take's file judged again. * * The checks are worked out when the run ends, from what the stages * kept: items 1 (latency, also after save and reopen) and 2 (several * phrases in one take) over every punch-in of the run; from what a - * TakeObserver saw of each of stage 1's punch-ins, items 3 (live dots, + * TakeObserver saw of each of stage 2's punch-ins, items 3 (live dots, * and how far behind the cursor they appear) and 5 (the mic on input * 2); from what it saw of every punch-in, item 4 (nothing of the take - * in the speakers); from the take before and after stage 2, items 7 - * (record from a position) and 12 (the lead-in); from stage 3, item 13 - * (pre-roll near the start); and from stages 2 and 3, item 14 (Record - * into Selection stops by itself). A run that ends early works out - * each check whose stages it got through, and has the rest Skipped. + * in the speakers); from stage 1, item 9 (Stop on a long song); from + * the take before and after stage 3, items 7 (record from a position) + * and 12 (the lead-in); from stage 4, item 13 (pre-roll near the + * start); from stages 3 and 4, item 14 (Record into Selection stops by + * itself); and from stage 5, item 10 (the joins). A run that ends + * early works out each check whose stages it got through, and has the + * rest Skipped. * * The session saved stays open afterwards, so that the takes can be * looked at; its scratch folder stays with it, and the next run @@ -157,7 +166,7 @@ class DevChecks : public QObject /// this far below the loudest input's static constexpr double kMicChannelDb = 20.0; - /// Stage 3's pre-roll, in seconds: more than there is room for + /// Stage 4's pre-roll, in seconds: more than there is room for /// before nearTheStart() static constexpr double kNearStartPreRollSeconds = 3.0; @@ -166,6 +175,11 @@ class DevChecks : public QObject /// and a block of the device, as the checklist asks static constexpr double kStopMarginSeconds = 0.25; + /// Item 9: each punch-in into the long song has its pitch merged + /// within this share of the time the whole song's analysis took, as + /// test-tony-device asked + static constexpr double kStopShare = 0.5; + /// How often a stage is looked at static constexpr int kPollMs = 50; @@ -196,14 +210,30 @@ class DevChecks : public QObject /// directory QString scratchDirectory; - Options() : roundTrip(-1.0) { } + /// How long the long song of stage 1 is, in seconds; 0 leaves + /// the stage out, and item 9 is Skipped (for tests that look at + /// the other stages, which it would only make longer) + double longSeconds; + + Options() : roundTrip(-1.0), + longSeconds(LatencyCheck::kLongSeconds) { } }; DevChecks(MainWindow *window, AudioCheckRunner *runner); virtual ~DevChecks(); /** - * Stage 1's punch-ins on the dev layout, in seconds: [6.3, 10.2] + * Stage 1's punch-ins into a long layout, in seconds: each the + * shortest range that judges one sweep, the first sweep from a + * quarter of the way into the song, and the first from five eighths + * (near 60 and 150 s of 4 minutes), so that each is far from the + * start and from the other. Empty if the layout is too short. + */ + static std::vector + longPunchIns(const LatencyCheck::Layout &layout); + + /** + * Stage 2's punch-ins on the dev layout, in seconds: [6.3, 10.2] * and [16.8, 21.2], each judging two sweeps (7.2 and 9.1 s; 17.7 * and 20.1 s) with 50 ms to spare. In two separate regions of the * calibration part, clear of what later stages need: the start @@ -217,9 +247,21 @@ class DevChecks : public QObject static std::vector freshPunchIns(); /** - * Stage 2's punch-in, in seconds: [19.2, 21.2], over the end of - * stage 1's second and past its first sweep, judging the sweep at - * 20.1 s. Its 1 s lead-in plays over what stage 1 recorded there: + * Stage 5's punch-ins, in seconds: [26.0, 28.7] and [28.7, 32.0], + * meeting at 28.7 s, the middle of the held tone from 27.2 to 30.2 + * s, which reaches 1.5 s either side of the join: further than + * TakeDiff looks at the pitch and notes around it. The first + * judges that tone's sweep at 26.9 s, and the second the next + * sweep, at 30.9 s; the second's 1 s lead-in plays over what the + * first recorded. The shortest pair that does that, and clear of + * every other stage's range. + */ + static std::vector joinPunchIns(); + + /** + * Stage 3's punch-in, in seconds: [19.2, 21.2], over the end of + * stage 2's second and past its first sweep, judging the sweep at + * 20.1 s. Its 1 s lead-in plays over what stage 2 recorded there: * the last of the tone from 18 s, and from 18.8 s a gap where the * reference is silent and the take holds only what the mic heard * besides. Starting this late keeps the range short, gives the @@ -230,7 +272,7 @@ class DevChecks : public QObject static LatencyCheck::PunchIn reRecording(); /** - * Stage 3's punch-in, in seconds: [1.0, 4.2], judging the sweep at + * Stage 4's punch-in, in seconds: [1.0, 4.2], judging the sweep at * 3.1 s, recorded with a pre-roll of kNearStartPreRollSeconds. */ static LatencyCheck::PunchIn nearTheStart(); @@ -328,8 +370,38 @@ class DevChecks : public QObject int m_observedPunchIn; std::vector m_watched; - /// Stage 1: the layout, what the runner found, the take's coverage - /// straight after, and what was seen of each punch-in + /// Stage 1, on a long song of its own: its layout, whether it got + /// through, what the runner found, the take's coverage straight + /// after, and what was seen of each punch-in. The runner's steps + /// timed: the whole song's analysis, from the session open to the + /// analysis done (-1 if it was not seen through), and each + /// punch-in's, from the take stopped to its pitch merged, in the + /// order recorded; and the take's pitch as each punch-in began to + /// record, and after the last. While the stage runs, the step and + /// punch-in the runner last reported, and when that began + struct LongSong { + LatencyCheck::Layout layout; + bool done; + AudioCheckResult result; + Coverage coverage; + std::vector watched; + double wholeSongSeconds; + std::vector stopSeconds; + std::vector pitch; + bool timing; + QElapsedTimer clock; + AudioCheckRunner::Step step; + int punchIn; + qint64 stepFrom; + LongSong() : done(false), wholeSongSeconds(-1.0), timing(false), + step(AudioCheckRunner::Step::Idle), punchIn(0), + stepFrom(0) { } + }; + LongSong m_long; + + /// Stage 2: the dev layout, which later stages record into too, what + /// the runner found, the take's coverage straight after, and what was + /// seen of each punch-in LatencyCheck::Layout m_layout; bool m_haveFresh; AudioCheckResult m_fresh; @@ -347,11 +419,11 @@ class DevChecks : public QObject Coverage coverage; }; - /// Stages 2 and 3, each a run of the runner's with one punch-in into - /// the take there is: whether it got through, what the runner found, - /// the take before and after, what was seen of the punch-in, the - /// lead-in the window gave it, and how many frames its raw recording - /// holds (-1, and why, if that could not be read) + /// Stages 3, 4 and 5, each a run of the runner's into the take there + /// is: whether it got through, what the runner found, the take + /// before and after, what was seen of the punch-ins, the lead-in the + /// window gave the last, and how many frames the first one's raw + /// recording holds (-1, and why, if that could not be read) struct PunchInStage { bool done; AudioCheckResult result; @@ -365,9 +437,10 @@ class DevChecks : public QObject }; PunchInStage m_reRecord; PunchInStage m_nearStart; + PunchInStage m_joins; - /// Stage 4: the take's pitch and notes before the save and after - /// the reopen, and its file judged, over every punch-in of the run, + /// Stage 6: the take's pitch and notes before the save and after + /// the reopen, and its file judged, over every punch-in into it, /// just before the save and again after the reopen sv::EventVector m_pitchBefore; sv::EventVector m_notesBefore; @@ -378,9 +451,11 @@ class DevChecks : public QObject LatencyCheck::TakeSummary m_rejudged; /// One of the runs of the runner's that the run got through, in - /// order: what it found, the take's coverage after it, and what was - /// seen of its punch-ins + /// order: the layout of the reference it recorded against, what it + /// found, the take's coverage after it, and what was seen of its + /// punch-ins struct Run { + const LatencyCheck::Layout *layout; const AudioCheckResult *result; const Coverage *coverage; const std::vector *watched; @@ -415,7 +490,8 @@ class DevChecks : public QObject GapLooks() : placed(false), margin(0), loudest(0), loudestInGaps(0), looks(0), heard(0), heardFrom(0), heardTo(0) { } }; - GapLooks gapLooks(const TakeObserver::Observation &seen, + GapLooks gapLooks(const LatencyCheck::Layout &layout, + const TakeObserver::Observation &seen, const TakeLatency &latency, sv::sv_samplerate_t rate, double until) const; @@ -426,12 +502,19 @@ class DevChecks : public QObject /// Stop the observer and keep what it saw void finishObservation(); - /// What was seen of stage 1's punch-in i, counting from 0, or null + /// Stage 1: the runner has reported a step and punch-in (Idle and 0 + /// when it finished); a step it had reported before ends + void longSongStep(AudioCheckRunner::Step step, int punchIn); + + /// What was seen of stage 2's punch-in i, counting from 0, or null const Watched *freshWatched(int i) const; + void beginLongSong(); + bool longSongDone(); void beginFreshPunchIns(); bool freshPunchInsDone(); - void beginPunchInStage(PunchInStage &stage, LatencyCheck::PunchIn range, + void beginPunchInStage(PunchInStage &stage, + std::vector ranges, double preRoll); bool punchInStageDone(PunchInStage &stage); void beginReopen(); @@ -440,7 +523,8 @@ class DevChecks : public QObject /// The take as it is now, its models looked up afresh Snapshot snapshot() const; - /// Every punch-in of the run so far, in the order recorded + /// Every punch-in into the dev reference's take so far, in the order + /// recorded std::vector punchInsSoFar() const; /// The events of the take's pitch track or notes, from its model @@ -458,6 +542,8 @@ class DevChecks : public QObject CheckResult speakersCheck(QString reason) const; CheckResult micChannelCheck(QString reason) const; CheckResult positionCheck(QString reason) const; + CheckResult longSongCheck(QString reason) const; + CheckResult joinsCheck(QString reason) const; CheckResult leadInCheck(QString reason) const; CheckResult nearStartCheck(QString reason) const; CheckResult stopsItselfCheck(QString reason) const; diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index 3e629416..cd297682 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -18,10 +18,12 @@ // Tier 5, as TestAudioCheck: the development checks (DevChecks) on the // real MainWindow, recording from the fake device with its output -// looped back into its input. A run records two punch-ins against the -// 40 s dev reference, one over the end of the second, one near the -// start of the song, then saves the session and opens it again: about -// 30 s of real time. +// looped back into its input. A run records two punch-ins far apart +// into a long song of 60 s here, then two against the 40 s dev +// reference, one over the end of the second, one near the start of the +// song, two that meet inside a held tone, then saves the session and +// opens it again: about 50 s of real time. The runs that look at other +// things leave the long song out. // // The fixture is TestAudioCheck's, copied rather than shared. The // application's data directory, where the check writes its references, @@ -72,6 +74,10 @@ class TestDevChecks : public QObject static constexpr int reportedIn = 4096; static constexpr int roundTrip = 3 * 4096 + 123; + // The long song of the passing run: a quarter of the real one, long + // enough that its analysis takes well over twice a punch-in's + static constexpr double longSeconds = 60.0; + QTemporaryDir m_dir; TestMainWindow *m_window = nullptr; QTimer m_watchdog; @@ -135,9 +141,12 @@ class TestDevChecks : public QObject QString reportDirectory() { return m_dir.filePath("report"); } QString scratchDirectory() { return m_dir.filePath("scratch"); } - DevChecks::Options options(double roundTripSeconds) { + // With the long song, unless a length of 0 leaves it out + DevChecks::Options options(double roundTripSeconds, + double longSongSeconds = longSeconds) { DevChecks::Options o; o.roundTrip = roundTripSeconds; + o.longSeconds = longSongSeconds; o.reportDirectory = reportDirectory(); o.scratchDirectory = scratchDirectory(); return o; @@ -155,14 +164,17 @@ class TestDevChecks : public QObject return plan; } - void startDevChecks(double roundTripSeconds) { + void startDevChecks(double roundTripSeconds, + double longSongSeconds = longSeconds) { m_window->discardModifications(); - QVERIFY(m_window->devChecks()->start(options(roundTripSeconds))); + QVERIFY(m_window->devChecks()->start(options(roundTripSeconds, + longSongSeconds))); QVERIFY(m_window->devChecks()->isRunning()); } - void runDevChecks(double roundTripSeconds) { - startDevChecks(roundTripSeconds); + void runDevChecks(double roundTripSeconds, + double longSongSeconds = longSeconds) { + startDevChecks(roundTripSeconds, longSongSeconds); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0, 120000); QCOMPARE(m_finished, 1); @@ -286,13 +298,13 @@ class TestDevChecks : public QObject QVERIFY(!m_window->audioCheck()->isRunning()); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(!m_window->audioCheckTakes()); - QCOMPARE(int(m_report.checks.size()), 9); + QCOMPARE(int(m_report.checks.size()), 11); for (const CheckResult &c : m_report.checks) { QVERIFY2(c.verdict == CheckResult::Verdict::Skipped, describe()); QVERIFY2(c.message.contains(m_report.failure), describe()); } QCOMPARE(lastReportLine(), - QString("Totals: 0 passed, 0 failed, 0 measured, 9 skipped")); + QString("Totals: 0 passed, 0 failed, 0 measured, 11 skipped")); QTest::qWait(500); QCOMPARE(m_finished, 1); @@ -436,7 +448,10 @@ private slots: // start gap. The re-recording changed nothing outside its range and // nothing before it, and nothing of the take was heard during its // lead-in; the punch-in near the start played from the start of the - // song; both stopped by themselves. The report ends with its totals, + // song; both stopped by themselves. Stop on the long song analysed + // only the ranges; the joins have no step and the pitch runs through + // them, but the note does not (expected to fail, a defect of the + // notes merge). The report ends with its totals, // and the session open afterwards is the one saved in the scratch // folder void dev_checks_pass_with_the_true_round_trip() { @@ -451,7 +466,7 @@ private slots: qDebug().noquote() << "report:" << line; } QVERIFY2(m_report.failure == "", describe()); - QCOMPARE(int(m_report.checks.size()), 9); + QCOMPARE(int(m_report.checks.size()), 11); const CheckResult *latency = check(1); const CheckResult *phrases = check(2); QVERIFY(latency && phrases); @@ -510,35 +525,59 @@ private slots: QVERIFY2(reportText().contains(words), qPrintable(words)); } - QCOMPARE(m_stages, QStringList() << "1 of 4: Fresh punch-ins" - << "2 of 4: Re-record" << "3 of 4: Pre-roll near the start" - << "4 of 4: Save and reopen"); - - // Two punch-ins into a reference of the dev layout, recorded in - // the order given; then one over the end of the second, which - // judges the sweep at 20.1 s; then one near the start, which - // judges the sweep at 3.1 s. Items 1 and 2 count all four - QCOMPARE(int(m_checks.size()), 3); - const LatencyCheck::TakeSummary &s = m_checks[0].summary; - QCOMPARE(int(s.punchIns.size()), 2); - QCOMPARE(s.judged, 4); - QCOMPARE(s.found, 4); - for (int i : { 1, 2 }) { + QCOMPARE(m_stages, QStringList() << "1 of 6: Long song" + << "2 of 6: Fresh punch-ins" << "3 of 6: Re-record" + << "4 of 6: Pre-roll near the start" << "5 of 6: Joins" + << "6 of 6: Save and reopen"); + + // Two punch-ins far apart into the long song, each judging one + // sweep; two into a reference of the dev layout, recorded in the + // order given; then one over the end of the second, which judges + // the sweep at 20.1 s; then one near the start, which judges the + // sweep at 3.1 s; then two that meet at 28.7 s, judging the sweeps + // at 26.9 and 30.9 s. Items 1 and 2 count all eight, numbered + // along the run + QCOMPARE(int(m_checks.size()), 5); + for (int i : { 0, 1, 4 }) { + const LatencyCheck::TakeSummary &s = m_checks[i].summary; + QCOMPARE(int(s.punchIns.size()), 2); + QCOMPARE(s.judged, i == 1 ? 4 : 2); + QCOMPARE(s.found, s.judged); + } + for (int i : { 2, 3 }) { QCOMPARE(int(m_checks[i].summary.punchIns.size()), 1); QCOMPARE(m_checks[i].summary.judged, 1); QCOMPARE(m_checks[i].summary.found, 1); } - QVERIFY(std::fabs(m_checks[1].summary.events[0].expectedSeconds - - 20.1) < 1e-4); QVERIFY(std::fabs(m_checks[2].summary.events[0].expectedSeconds - + 20.1) < 1e-4); + QVERIFY(std::fabs(m_checks[3].summary.events[0].expectedSeconds - 3.1) < 1e-4); - for (QString label : { QString("offsets, punch-in 3 (19.20 to 21.20 " + QVERIFY(std::fabs(m_checks[4].summary.events[1].expectedSeconds - + 30.9) < 1e-4); + + // The long song's sweeps: the first from a quarter of the way in, + // and the first from five eighths + const LatencyCheck::TakeSummary &song = m_checks[0].summary; + for (int i : { 0, 1 }) { + const double from = (i == 0 ? 0.25 : 0.625) * longSeconds; + const double at = song.events[i].expectedSeconds; + QVERIFY2(at >= from && at < from + 2.6, + qPrintable(QString::number(at))); + } + const LatencyCheck::PunchIn first = song.punchIns[0].range; + for (QString label : { QString("offsets, punch-in 1 (%1 to %2 s)") + .arg(first.start, 0, 'f', 2) + .arg(first.end, 0, 'f', 2), + QString("offsets, punch-in 5 (19.20 to 21.20 " "s)"), - QString("offsets, punch-in 4 (1.00 to 4.20 " + QString("offsets, punch-in 6 (1.00 to 4.20 " + "s)"), + QString("offsets, punch-in 8 (28.70 to 32.00 " "s)") }) { QVERIFY2(number(*latency, label) != "", describe()); } - QVERIFY2(number(*phrases, "punch-in 4, start gap").endsWith("measured"), + QVERIFY2(number(*phrases, "punch-in 8, start gap").endsWith("measured"), describe()); const CheckResult *position = check(7); @@ -597,10 +636,48 @@ private slots: QVERIFY2(number(*stops, "dialogs, 19.20 to 21.20 s") .startsWith("none, over "), describe()); + // Stop on the long song analysed each punch-in's range alone, in + // under half the time the whole song's analysis took, and the + // second left the first's pitch as it was + const CheckResult *longSong = check(9); + QVERIFY(longSong); + QCOMPARE(longSong->name, QString("stop_on_a_long_song")); + QVERIFY2(longSong->verdict == CheckResult::Verdict::Pass, describe()); + const QString kept = number(*longSong, "pitch, punch-in 2"); + QVERIFY2(kept.endsWith(", unchanged") && + kept.section(' ', 0, 0).toInt() > 0, describe()); + + // The punch-ins that meet inside the held tone, placed alike: no + // step in the samples there, the pitch running through, and + // nothing moved outside the two + const CheckResult *joins = check(10); + QVERIFY(joins); + QCOMPARE(joins->name, QString("the_joins")); + QCOMPARE(number(*joins, "second punch-in against the first"), + QString("0.0 ms")); + for (QString part : { QString("step: "), QString("pitch: "), + QString("outside: ") }) { + QVERIFY2(!joins->message.contains(part), describe()); + } + + // And one note through the join, which the take does not have: + // the second punch-in's analysis starts 0.5 s before the join, + // inside the tone, so its note begins before the merge window and + // is not merged in, while the first's note, which ends at the + // join, stays. The tone after the join has no note + const bool joinsPass = joins->verdict == CheckResult::Verdict::Pass; + QEXPECT_FAIL("", "one note should run through a join inside a held " + "tone, but the notes merge by onset keeps the first " + "punch-in's note, ending at the join, and drops the " + "second's, which begins before the merge window " + "(docs/takes.md, \"Notes, by onset\")", Continue); + QVERIFY2(joinsPass, describe()); + QVERIFY(QFileInfo(m_report.reportPath).fileName() == "DevChecks.txt"); QVERIFY(TakesFile::isInFolder(reportDirectory(), m_report.reportPath)); QCOMPARE(lastReportLine(), - QString("Totals: 8 passed, 0 failed, 1 measured, 0 skipped")); + QString("Totals: %1 passed, %2 failed, 1 measured, 0 skipped") + .arg(joinsPass ? 10 : 9).arg(joinsPass ? 0 : 1)); QVERIFY(m_report.sessionPath != ""); QCOMPARE(m_window->sessionFile(), m_report.sessionPath); @@ -622,7 +699,7 @@ private slots: void dev_checks_fail_with_the_round_trip_off() { makeWindow(loopback()); - runDevChecks(roundTrip / rate + 0.020); + runDevChecks(roundTrip / rate + 0.020, 0.0); if (QTest::currentTestFailed()) return; QVERIFY2(m_report.failure == "", describe()); @@ -685,9 +762,19 @@ private slots: describe()); const bool dotsPass = check(3) && check(3)->verdict == CheckResult::Verdict::Pass; + + // Without the long song, which this run leaves out; the joins as + // in any run, placed alike + QVERIFY2(check(9) && + check(9)->verdict == CheckResult::Verdict::Skipped && + check(9)->message == "The long song was left out of this " + "run.", describe()); + const bool joinsPass = + check(10) && check(10)->verdict == CheckResult::Verdict::Pass; QCOMPARE(lastReportLine(), - QString("Totals: %1 passed, %2 failed, 1 measured, 0 skipped") - .arg(dotsPass ? 4 : 3).arg(dotsPass ? 4 : 5)); + QString("Totals: %1 passed, %2 failed, 1 measured, 1 skipped") + .arg((dotsPass ? 4 : 3) + (joinsPass ? 1 : 0)) + .arg((dotsPass ? 4 : 5) + (joinsPass ? 0 : 1))); } // The loopback heard a second time, 50 ms later at half the level, as @@ -702,7 +789,7 @@ private slots: config.inputChannel = 1; makeWindow(config); - runDevChecks(roundTrip / rate); + runDevChecks(roundTrip / rate, 0.0); if (QTest::currentTestFailed()) return; QVERIFY2(m_report.failure == "", describe()); @@ -757,11 +844,13 @@ private slots: } }); connect(m_window->devChecks(), &DevChecks::progress, - this, [this](QString, int n, int) { - if (n == 3) m_window->devChecks()->cancel(); + this, [this](QString stage, int, int) { + if (stage == "Pre-roll near the start") { + m_window->devChecks()->cancel(); + } }); - runDevChecks(roundTrip / rate); + runDevChecks(roundTrip / rate, 0.0); if (QTest::currentTestFailed()) return; QCOMPARE(m_faults, 1); QCOMPARE(m_report.failure, QString("The dev checks were cancelled.")); @@ -841,7 +930,7 @@ private slots: // itself, deleted during a take of theirs void dev_checks_deleted_during_a_run() { makeWindow(loopback()); - startDevChecks(roundTrip / rate); + startDevChecks(roundTrip / rate, 0.0); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); @@ -859,7 +948,7 @@ private slots: ->haveRunningTransformers(), 30000); makeWindow(loopback()); - startDevChecks(roundTrip / rate); + startDevChecks(roundTrip / rate, 0.0); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); @@ -901,7 +990,7 @@ private slots: QVERIFY(calibration.calibrationUsable()); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); QTRY_VERIFY_WITH_TIMEOUT - (dialog->pageText().contains("Dev checks, stage 1 of 4"), 10000); + (dialog->pageText().contains("Dev checks, stage 1 of 6"), 10000); QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h index 87acf0a5..c07c15dd 100644 --- a/main/test/TestLatencyCheck.h +++ b/main/test/TestLatencyCheck.h @@ -354,6 +354,44 @@ private slots: QVERIFY(LatencyCheck::longLayout().events.size() > 100); } + // A long layout of another length is the start of the 4-minute one: + // its events, as many as end in time for the calibration's 0.8 s of + // silence, and the length asked for + void long_layout_of_another_length() { + const LatencyCheck::Layout whole = LatencyCheck::longLayout(); + QCOMPARE(LatencyCheck::kLongSeconds, 240.0); + QCOMPARE(whole.length, framesOf(240.0)); + + for (double seconds : { 60.0, 61.3 }) { + const LatencyCheck::Layout part = + LatencyCheck::longLayout(kRate, seconds); + QCOMPARE(part.rate, kRate); + QCOMPARE(part.length, framesOf(seconds)); + + const int n = int(part.events.size()); + QVERIFY2(n > 20 && n < int(whole.events.size()), + qPrintable(QString::number(n))); + for (int i = 0; i < n; ++i) { + const LatencyCheck::Event &a = part.events[i]; + const LatencyCheck::Event &b = whole.events[i]; + QCOMPARE(a.sweepStart, b.sweepStart); + QCOMPARE(a.toneStart, b.toneStart); + QCOMPARE(a.toneLength, b.toneLength); + QCOMPARE(a.toneHz, b.toneHz); + } + + auto endOf = [](const LatencyCheck::Event &e) { + return e.toneStart + e.toneLength; + }; + QVERIFY(endOf(part.events[n - 1]) + framesOf(0.8) <= part.length); + QVERIFY(endOf(whole.events[n]) + framesOf(0.8) > part.length); + } + + // 61.3 s has room for one event more than 60 s + QCOMPARE(LatencyCheck::longLayout(kRate, 61.3).events.size(), + LatencyCheck::longLayout(kRate, 60.0).events.size() + 1); + } + // Every sweep and every tone peaks at -12 dBFS, and nothing is louder void generator_peaks_at_minus_12_dbfs() { const LatencyCheck::Layout layout = LatencyCheck::devLayout(); From 7de2e80b2ff0ba8c3cbb04afa4b960a42ff735f4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:30:45 +0000 Subject: [PATCH 162/275] docs: calibrate audio work orders, C2b and C2c C2b de-races item 12, whose only lead-in gap a stalled event loop could empty. C2c fixes the notes merge that item 10 found dropping the note after a join inside a held tone, which the user chose to fix here. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 55 +++++++++++++++++++++++++++-- docs/calibrate-audio.md | 3 ++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index f6e9146c..b684f498 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,11 +194,11 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). -**Order from here:** C1b, C1c, C2, C3, then the lead's release build, then D. The spec's +**Order from here:** C1b, C1c, C2, C2b, C2c, C3, then the lead's release build, then D. The spec's old C2 (observer group) and C4 (smoke group) are gone: `TestUiChecks` covers the smoke items and the screen, and what the dev run measures on the device is now in C1b–C2. @@ -335,6 +335,57 @@ and how it times the whole-song analysis; spec §4 rows 9 and 10. in all (135 s now). Show failure for item 9 (e.g. Stop analysing the whole song) and one part of item 10 by breaking the code, and undo by hand. +### C2b — Item 12 robust against a stalled event loop (a de-race, its own commit) + +Read also: `main/dev/DevChecks.cpp` (`gapLooks()`, the "Re-record" stage, item 12) and +`main/dev/TakeObserver.{h,cpp}`; C1b's and C1c's log entries. + +- **The flake (found in C2):** item 12 judges the output levels in the silent gaps of the + re-record's lead-in. Its only gap there, 18.8–19.2 s, holds 15–16 looks; a stall of the + GUI thread of about 340 ms (seen on this VM, nothing in the log) left 9, once 0, and + item 12 failed in a clean run. A real device can stall too, and MME's larger blocks + widen the margin and shrink the gap further. +- **The fix, in the design, not the tolerance:** + - Give the re-record stage's plan a longer pre-roll (`Plan::preRoll`), so that its + lead-in also spans the 0.9 s gap at 16.8–17.7 s inside the earlier punch-in + [16.8, 21.2]: a pre-roll of 2.4 s starts it at 16.8 s. Check that items 7, 12 and 14 + still read as before, and say what the lead-in now covers. + - Look at what a stall does to a look (the frames received jump; the look's window then + spans a sweep and is left out). Where a gap check ends with no look in any gap, the + part is "not judged" with the reason, not a Fail: it has not seen the take played + out. Item 4 the same. +- **Tests:** a test that stalls the GUI thread for about 0.4 s during the re-record's + lead-in (a single-shot timer that busy-waits, started when the runner reports that + punch-in recording): item 12 still judges and passes; and the take-heard fault still + fails with such a stall. Show the stall test failing on the code before the fix. + +### C2c — The notes merge keeps one note across a join (`Analyser`, its own commit) + +Read also: `docs/takes.md` "Ranged analysis and merge" and its known limits (search "by +onset", "deliberate trade"); `Analyser::analyseRange()`'s merge in `main/Analyser.cpp` +(search "Notes go by their onset"); the existing merge tests (search `TestRecordWorkflow.h` +and `TestSingingAnalysis.h` for "note"); C2's log entry. + +- **The defect (found by item 10):** two punch-ins meeting at J inside a held tone. The + first's note ends at J (the audio after J was silence when it was analysed). The second + punch-in's run starts 0.5 s before J, inside the tone, so its note begins before W (the + range ± 0.25 s): the merge adds only notes with their onset in W, and the tone after J + has no note. Analysing the whole take gives one note, 27.21–30.20 s. +- **Wanted:** one note across the join. When an old note runs into W from before it and + a new note that begins before W overlaps it there, the old note keeps its onset (the + audio before W has not changed) and takes the new note's end, cut back as today at an + old onset after W. Decide, and justify, what happens when a new note begins before W + with no old note there. Keep everything else the merge promises: notes in unchanged + audio not split, the change record (`m_rangedNotesChange`) that undo reverses, and the + "deliberate trade" of `docs/takes.md` unless the fix removes it (then say so). +- **Tests:** + - an app test on the fake: two punch-ins meeting inside a held tone leave one note + across J; seen failing before the fix; + - undo of the second punch-in gives back the first's note exactly; + - every existing merge test green and unchanged; + - `TestDevChecks`: item 10's `QEXPECT_FAIL` removed; the check passes. +- **Docs:** `docs/takes.md`'s "Notes, by onset" and its known limit, in the same commit. + ### C3 — Retire `test-tony-device` To be refined by the lead after C2. Outline: remove `TestRealDevice.h`, diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index aabc682d..1de261c6 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -331,6 +331,9 @@ marked "Done" when it is committed. - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. Done. 5. **C2** Long song and joins: items 9 and 10. Done. + - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). + - **C2c** The notes merge keeps one note across a join: the defect item 10 found (the + user chose to fix it on this branch). 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. 7. **Release build** by the lead (§8, "Release builds must stay clean"). 8. **D** Docs, from the code and the phase log: From 18a12e8819c55454d404f463ad55dac994c41acf Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:41:01 +0000 Subject: [PATCH 163/275] test: ci's list of failed tests keeps each one's values and location a qcompare failure puts its actual and expected lines between the fail line and its location, and the list stopped at the first of them, so macos showed no locations. every line qtest indents under a failure is printed now, up to the next record. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/linux.yml | 7 ++++--- .github/workflows/macos.yml | 7 ++++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index d6b9dadf..cd1f6626 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -43,12 +43,13 @@ jobs: run: meson test -C build --print-errorlogs --num-processes 1 # meson shows at most the last 100 lines of a failing log, and one # test executable runs several QTest suites: name every failed test, - # with where it failed and the totals of each suite that failed + # with the lines QTest indents under it (compared values, where it + # failed) and the totals of each suite that failed - name: test-failures if: failure() && steps.test.outcome == 'failure' run: | awk '/^test:/ { print; next } /^Totals:/ { if ($0 !~ /, 0 failed/) print; next } /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } - fail && /^ Loc:/ { print } - { fail = 0 }' build/meson-logs/testlog.txt + /^[A-Z!]+ *: / { fail = 0 } + fail && /^ / { print }' build/meson-logs/testlog.txt diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml index 8c4e8a79..03d0c25a 100644 --- a/.github/workflows/macos.yml +++ b/.github/workflows/macos.yml @@ -22,15 +22,16 @@ jobs: run: meson test -C build --print-errorlogs --num-processes 1 # meson shows at most the last 100 lines of a failing log, and one # test executable runs several QTest suites: name every failed test, - # with where it failed and the totals of each suite that failed + # with the lines QTest indents under it (compared values, where it + # failed) and the totals of each suite that failed - name: test-failures if: failure() && steps.test.outcome == 'failure' run: | awk '/^test:/ { print; next } /^Totals:/ { if ($0 !~ /, 0 failed/) print; next } /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } - fail && /^ Loc:/ { print } - { fail = 0 }' build/meson-logs/testlog.txt + /^[A-Z!]+ *: / { fail = 0 } + fail && /^ / { print }' build/meson-logs/testlog.txt - name: deploy-app run: | ls -lR /usr/local/opt From a654b38819fe96c56ffd49615b0b085060b2f2e9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:44:41 +0000 Subject: [PATCH 164/275] fix: each test shard has a home of its own TestDevChecks turns on QStandardPaths' test mode, which keeps QSettings in $HOME/.qttest whatever XDG_CONFIG_HOME says, so the shards of test-tony-dev shared one settings file and cleared each other's: dev_checks_cancelled and dev_checks_end_when_the_session_closes failed sharded and passed alone. With a HOME per shard all seven pass, in 18 s instead of 68. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/run-tests.sh | 18 ++++++++++-------- docs/testing.md | 8 +++++--- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/deploy/linux/run-tests.sh b/deploy/linux/run-tests.sh index c08d3134..30b03f4d 100755 --- a/deploy/linux/run-tests.sh +++ b/deploy/linux/run-tests.sh @@ -14,13 +14,15 @@ # shard of every suite (TONY_TEST_SHARD, main/test/RunSuite.h), and adds # up what they report. The app suite spends nearly all of its time # waiting on FakeAudioIO, which plays in real time: on the cloud -# container's 4 cores, 8 processes run it in about a minute instead of -# six, and the load stays under 2. +# container's 4 cores, 8 processes run it in a minute and a half instead +# of eight, and the load stays under 2. # -# Each process has a log directory and XDG directories of its own. The -# suites keep QSettings per user, and processes sharing the file would -# read each other's settings. On Windows QSettings is the registry, which -# XDG_CONFIG_HOME does not move: this is for Linux. +# Each process has a log directory, a HOME and XDG directories of its own. +# The suites keep QSettings per user, and processes sharing the file +# would read and clear each other's settings; a suite that turns on +# QStandardPaths' test mode keeps them in $HOME/.qttest, which the XDG +# variables do not move. On Windows QSettings is the registry, which +# neither moves: this is for Linux. # # Usage, from anywhere: # deploy/linux/run-tests.sh [-j N] [BUILD_DIR] EXECUTABLE @@ -69,10 +71,10 @@ mkdir -p "$out" start=$SECONDS for i in $(seq 0 $((jobs - 1))); do dir=$out/$i - mkdir -p "$dir/xdg/config" "$dir/xdg/data" "$dir/xdg/cache" + mkdir -p "$dir/home" "$dir/xdg/config" "$dir/xdg/data" "$dir/xdg/cache" ( cd "$build" && - TONY_TEST_SHARD=$i/$jobs TONY_TEST_LOG_DIR=$dir \ + TONY_TEST_SHARD=$i/$jobs TONY_TEST_LOG_DIR=$dir HOME=$dir/home \ XDG_CONFIG_HOME=$dir/xdg/config XDG_DATA_HOME=$dir/xdg/data \ XDG_CACHE_HOME=$dir/xdg/cache \ "./$exe" > "$dir/stdout.log" 2>&1 diff --git a/docs/testing.md b/docs/testing.md index b882c24e..4324b35e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -55,9 +55,11 @@ helpers must not be slots; connect to lambdas instead. For access to private sta not run. The app suite nearly only waits on `FakeAudioIO`'s real-time clock, so n processes at once take about 1/n of the time: on four cores the load stayed under 2 with eight, and reached 3.5 with twelve. `deploy/linux/run-tests.sh` starts them and adds up - their results; each process needs XDG directories of its own, because the suites' - QSettings are per user and would be shared. On Windows QSettings is the registry, so the - script is for Linux. Do not combine shards with test names on the command line. + their results. Each process needs a `HOME` and XDG directories of its own: the suites' + QSettings are per user, and processes sharing them clear each other's settings. + `TestDevChecks` turns on `QStandardPaths`' test mode, which keeps them in `~/.qttest` + whatever the XDG variables say. On Windows QSettings is the registry, so the script is + for Linux. Do not combine shards with test names on the command line. - A sharded run is a whole run of the suites, but the tests that share a process are other ones. After a change to object lifetimes, threads or teardown (see "Timing and races"), run the one-process run as well. From 0bb8a21bb9e26a9c45ce847e3ae97f1ae652be1c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:44:41 +0000 Subject: [PATCH 165/275] build: the cloud build covers test-tony-dev The background build and the setup script's last ccache step build test-tony-dev with the rest, as AGENTS.md's build does; container-setup.sh configures debugoptimized, where it always exists. building.md gives the commands and the suites' times after the merge of default. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/cloud-environment.sh | 2 +- deploy/linux/cloud-session.sh | 12 +++++++----- docs/building.md | 10 ++++++---- 3 files changed, 14 insertions(+), 10 deletions(-) diff --git a/deploy/linux/cloud-environment.sh b/deploy/linux/cloud-environment.sh index 2356454f..5c49ba7f 100755 --- a/deploy/linux/cloud-environment.sh +++ b/deploy/linux/cloud-environment.sh @@ -204,7 +204,7 @@ fill_ccache() { # svcore, then svgui and svapp (in libtonyapp.a with main/), # then the rest; each step until the deadline svlibs=$(ninja -t targets all | sed -n 's/^\(libtonyapp\.a\.p\/sv\(gui\|app\)_[^:]*\.o\):.*/\1/p') - for step in libsvcore.a "$svlibs" "tony pyin.so test-tony-core test-tony-app test-tony-device"; do + for step in libsvcore.a "$svlibs" "tony pyin.so test-tony-core test-tony-app test-tony-dev test-tony-device"; do [ -n "$step" ] || continue [ "$(left)" -gt 10 ] || break until_deadline nice ninja -j "$(nproc)" $step > /dev/null diff --git a/deploy/linux/cloud-session.sh b/deploy/linux/cloud-session.sh index cad75c91..2e612187 100755 --- a/deploy/linux/cloud-session.sh +++ b/deploy/linux/cloud-session.sh @@ -13,7 +13,7 @@ # Builds Tony in the background at the start of a cloud session, so that # the build runs while the session reads and edits instead of after: the # library directories at their pins and build/ configured -# (container-setup.sh), then Tony, the pYIN plugin and the three test +# (container-setup.sh), then Tony, the pYIN plugin and the four test # executables. # # The environment's setup script (cloud-environment.sh) has installed the @@ -22,9 +22,9 @@ # when the session builds again. The build runs at low priority, so that # the session's own commands come first. # -# build/ links with mold when it is installed: the four executables link -# in about a second instead of about twelve with GNU ld, which every -# change to main/ pays. +# build/ links with mold when it is installed: Tony and the test +# executables link in a few seconds instead of about twelve with GNU ld, +# which every change to main/ pays. # # Usage, from anywhere: # deploy/linux/cloud-session.sh start start the build and return at @@ -50,7 +50,9 @@ log=$root/tmp/cloud-session.log lock=$root/tmp/cloud-session.lock script=$root/deploy/linux/cloud-session.sh -targets="tony pyin.so test-tony-core test-tony-app test-tony-device" +# As AGENTS.md's. test-tony-dev is in every build but a release one, and +# container-setup.sh configures debugoptimized +targets="tony pyin.so test-tony-core test-tony-app test-tony-dev test-tony-device" case "${1:-} ${2:-}" in "start "|"start --if-cloud") diff --git a/docs/building.md b/docs/building.md index b3b0b530..5c4bcbca 100644 --- a/docs/building.md +++ b/docs/building.md @@ -106,10 +106,11 @@ The environment's settings: Then, from the repository root: ```sh -ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app > tmp/build.log 2>&1 +ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app test-tony-dev > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log deploy/linux/run-tests.sh test-tony-core # about a second -deploy/linux/run-tests.sh test-tony-app # about a minute +deploy/linux/run-tests.sh test-tony-app # a minute and a half +deploy/linux/run-tests.sh test-tony-dev # when AGENTS.md says to run it ``` No `.exe` on Linux; the plugin target is `pyin.so`. `run-tests.sh` runs an executable as @@ -123,8 +124,9 @@ Measured on 2026-09-26: | Full build, nothing in ccache | 6.6 minutes: 1570 CPU-seconds, nearly all compiling | | Full build, everything in ccache | 4 to 6 seconds | | A session's first build, with the setup script's ccache | 3.7 minutes, in the background | -| Linking Tony and the three test executables | 3 s with mold, 12 s with GNU ld | -| App suite | 356 s in one process, 54 s in eight | +| Linking `tony`, `test-tony-core`, `test-tony-app` and `test-tony-device` | 3 s with mold, 12 s with GNU ld | +| App suite | 488 s in one process, 87 s in eight | +| Development checks' suite | 68 s in one process, 18 s in eight | Why each part is as it is: From 50dea8ee294a9a8edc28c3f3265791b101ebe1c0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:44:41 +0000 Subject: [PATCH 166/275] docs: a cloud session pushes to a fork only once it is attached The session's git proxy refuses a push to a repository not attached to the session, a new branch included (HTTP 403), whatever the network setting; attached with push access, the same push goes through. A session started with the forks selected runs no repository's hooks. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- docs/forks.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/forks.md b/docs/forks.md index 0f60d1c8..0d0fbdc2 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -25,7 +25,13 @@ library over a workaround in `main/`. which reaches the fork branch when the Tony branch is merged. Commit messages there follow that repository's style: `area: what`. 2. Push to the remote named **`jhhr`**. In `svcore`, `svgui` and `svapp`, `origin` is - upstream sonic-visualiser — do not push there. + upstream sonic-visualiser — do not push there. In a cloud session the checkouts are + `container-setup.sh`'s, whose `origin` is the fork. There a push, even of a new branch, + is refused (HTTP 403) unless the fork is attached to the session: attach it with the + session's add-repository tool, with push access, and push from the checkout that is + there. Do not start the session with the forks selected instead: a session with several + repositories runs no repository's SessionStart hook, so the background build does not + start. 3. Put the new commit hash in `repoint-lock.json` as that library's `pin`, and commit that in Tony together with the code that needs it. 4. A sub-agent that was told to work only in `main/` does not edit a fork: it reports From 6fa6b2de6dba11579cdb140676fcbd19e0c03294 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:54:20 +0000 Subject: [PATCH 167/275] test: the save prompt is recognised by its text, which macos keeps qt on macos keeps no title on a message box, so three tests that looked for "Session modified" in the prompt's title read an empty one there. they look for the prompt's text now. answerWith() records the text, as its comment already said it did. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- main/test/TestAudioCheck.h | 8 ++++++-- main/test/TestUiChecks.h | 12 +++++++++--- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 3d23678d..8f3152c6 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -1130,7 +1130,10 @@ private slots: startCheck(onePunchIn(), false); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(!m_dialogs.isEmpty(), 10000); - QVERIFY2(m_dialogs.first().startsWith("Session modified"), + // By its text: macOS shows no title on a message box, and Qt + // keeps none there + QVERIFY2(m_dialogs.first().contains + ("The current session has been modified."), qPrintable(m_dialogs.join(" | "))); QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0 || m_window->recordTarget()->isRecording(), @@ -1150,7 +1153,8 @@ private slots: startCheck(onePunchIn(), false); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(!m_dialogs.isEmpty(), 10000); - QVERIFY2(m_dialogs.first().startsWith("Session modified"), + QVERIFY2(m_dialogs.first().contains + ("The current session has been modified."), qPrintable(m_dialogs.join(" | "))); QTRY_VERIFY_WITH_TIMEOUT(m_finished > 0 || m_window->recordTarget()->isRecording(), diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 64d6b925..b6968bb3 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -461,7 +461,9 @@ class TestUiChecks : public QObject return [=](QWidget *modal) { auto box = qobject_cast(modal); if (!box || !box->button(button)) return false; - if (asked) asked->push_back(box->windowTitle()); + // The text, not the title: macOS shows no title on a message + // box, and Qt keeps none there + if (asked) asked->push_back(box->text()); if (tick && box->checkBox()) box->checkBox()->setChecked(true); box->button(button)->click(); return true; @@ -1407,7 +1409,9 @@ private slots: QStringList asked; m_answerDialog = answerWith(QMessageBox::Cancel, false, &asked); QVERIFY(!m_window->close()); - QCOMPARE(asked, QStringList({ tr("Session modified") })); + QCOMPARE(asked.size(), 1); + QVERIFY2(asked[0].startsWith(tr("The current session has been modified.")), + qPrintable(asked[0])); QVERIFY2(m_window->isVisible(), "Cancel did not keep the window open"); QVERIFY(m_window->takes()->haveTake()); } @@ -1431,7 +1435,9 @@ private slots: QStringList asked; m_answerDialog = answerWith(QMessageBox::No, false, &asked); QVERIFY(m_window->close()); - QCOMPARE(asked, QStringList({ tr("Session modified") })); + QCOMPARE(asked.size(), 1); + QVERIFY2(asked[0].startsWith(tr("The current session has been modified.")), + qPrintable(asked[0])); delete m_window; m_window = nullptr; From 6ba885fe13e9f1f53d25a7b38649094f5b00ceb4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:03:02 +0000 Subject: [PATCH 168/275] test: the throttle's last notice is waited for, not given a fixed time the last notice comes an interval after the changes stop, when the timer fires, and a loaded ci runner's timers can overrun the fixed two intervals the test allowed: a_stream_is_told_once_an_interval failed once on macos. it waits for the notice now, up to 40 intervals. seen failing with a throttle that drops the end of a batch. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- main/test/TestModelChangeThrottle.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/main/test/TestModelChangeThrottle.h b/main/test/TestModelChangeThrottle.h index 5a51a2d2..5f284072 100644 --- a/main/test/TestModelChangeThrottle.h +++ b/main/test/TestModelChangeThrottle.h @@ -118,10 +118,13 @@ private slots: qPrintable(QString("%1 changes over ten intervals were told " "%2 times") .arg(changes).arg(m_told.size()))); - // Between them the notices cover every change - QTest::qWait(kInterval * 2); + // Between them the notices cover every change. The last one comes + // an interval after the changes stop, as late as the timer fires: + // wait for it rather than for a fixed time, which a loaded + // machine's timers can overrun QCOMPARE(m_told.front().first, frame_t(0)); - QCOMPARE(m_told.back().second, frame_t(changes * 256)); + QTRY_COMPARE_WITH_TIMEOUT(m_told.back().second, frame_t(changes * 256), + kInterval * 40); for (size_t i = 1; i < m_told.size(); ++i) { QVERIFY(m_told[i].first <= m_told[i-1].second); } From aa0968170794a736ab5f931d1c303c7369255aab Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:04:35 +0000 Subject: [PATCH 169/275] test: linux ci on ubuntu 24.04, whose qt is 6.4 ubuntu 22.04's qt is 6.2.4, older than anything this fork is tested with, and a drag in edited_take_notes_keep_the_take_pitch did not move the note under it. 24.04 has qt 6.4.2, the linux baseline of the docs. three packages are named as 24.04 names them, and meson finds qt by pkg-config there, so qtchooser, which 24.04 does not install, goes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/linux.yml | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml index cd1f6626..3a926e02 100644 --- a/.github/workflows/linux.yml +++ b/.github/workflows/linux.yml @@ -5,21 +5,21 @@ on: [push, pull_request] jobs: build: - runs-on: ubuntu-22.04 + runs-on: ubuntu-24.04 steps: - uses: actions/checkout@v2 - name: install-packages run: | sudo apt-get update - sudo apt-get install build-essential libbz2-dev libfftw3-dev libfishsound1-dev libid3tag0-dev liblo-dev liblrdf0-dev libmad0-dev liboggz2-dev libopus-dev libopusfile-dev libpulse-dev libsamplerate-dev libsndfile-dev libsord-dev libxml2-utils portaudio19-dev qt6-base-dev qt6-pdf-dev qt6-base-dev-tools libqt6svg6-dev raptor2-utils git mercurial autoconf automake libtool smlnj capnproto libcapnp-dev ninja-build libglib2.0-dev libboost-all-dev + sudo apt-get install build-essential libbz2-dev libfftw3-dev libfishsound1-dev libid3tag0-dev liblo-dev liblrdf0-dev libmad0-dev liboggz2-dev libopus-dev libopusfile-dev libpulse-dev libsamplerate0-dev libsndfile1-dev libsord-dev libxml2-utils portaudio19-dev qt6-base-dev qt6-pdf-dev qt6-base-dev-tools qt6-svg-dev raptor2-utils git mercurial autoconf automake libtool smlnj capnproto libcapnp-dev ninja-build libglib2.0-dev libboost-all-dev - name: install-meson run: | mkdir -p tmp/meson cd tmp/meson wget https://github.com/mesonbuild/meson/releases/download/1.3.1/meson-1.3.1.tar.gz tar xvf meson-1.3.1.tar.gz - sudo ln -s $(pwd)/meson-1.3.1/meson.py /usr/bin/meson + sudo ln -sf $(pwd)/meson-1.3.1/meson.py /usr/bin/meson - name: install-rubberband run: | mkdir -p tmp/rubberband @@ -33,9 +33,7 @@ jobs: - name: repoint run: ./repoint install - name: configure - run: | - qtchooser -install qt6 $(which qmake6) - QT_SELECT=qt6 meson setup build --buildtype release + run: meson setup build --buildtype release - name: make run: ninja -C build - name: test From ab3b5965fab0277ea9a27fcd994f7c824cc8983b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:06:27 +0000 Subject: [PATCH 170/275] test: windows ci builds with msys2's mingw64, as the fork is built the workflow asked for windows-2019, a runner github has retired, so its jobs were never started. it now runs on windows-2022 and builds as docs/building.md describes, with msys2's gcc, qt and libraries, not msvc and upstream's prebuilt libraries. repoint still runs from powershell, and meson gets MINGW_PREFIX in the windows form it needs. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/windows.yml | 97 +++++++++++++++++++++++------------ docs/building.md | 6 +-- 2 files changed, 68 insertions(+), 35 deletions(-) diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index 50ec1d9a..f38c0640 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -2,54 +2,87 @@ name: Windows CI on: [push, pull_request] +# Built as it is on the development machine (docs/building.md): MSYS2's +# mingw64 toolchain, Qt and libraries, with meson and ninja + jobs: build: - runs-on: windows-2019 + runs-on: windows-2022 steps: - uses: actions/checkout@v3 - uses: ProjectSavanna/setup-sml@v1.2.0 with: smlnj-version: '110.99.4' - - uses: actions/setup-python@v1 - with: - python-version: '3.x' - - uses: jurplel/install-qt-action@v3 - with: - version: '6.6.1' - host: 'windows' - target: 'desktop' - arch: 'win64_msvc2019_64' - - uses: MarkusJx/install-boost@v2.4.5 - id: install-boost - with: - boost_version: 1.84.0 - platform_version: 2019 - toolset: msvc - - uses: ilammy/msvc-dev-cmd@v1 - with: - vsversion: '2019' - - name: install-meson - run: pip install meson ninja + # repoint checks six of the libraries out with Mercurial + - name: mercurial + run: if (-not (Get-Command hg -ErrorAction SilentlyContinue)) { pip install mercurial } - name: repoint run: ./repoint install - - name: configure - run: meson setup build --buildtype release - env: - BOOST_ROOT: ${{ steps.install-boost.outputs.BOOST_ROOT }} + - uses: msys2/setup-msys2@v2 + with: + msystem: MINGW64 + update: true + install: >- + mingw-w64-x86_64-gcc + mingw-w64-x86_64-meson + mingw-w64-x86_64-ninja + mingw-w64-x86_64-pkgconf + mingw-w64-x86_64-qt6-base + mingw-w64-x86_64-qt6-svg + mingw-w64-x86_64-boost + mingw-w64-x86_64-bzip2 + mingw-w64-x86_64-zlib + mingw-w64-x86_64-fftw + mingw-w64-x86_64-libsamplerate + mingw-w64-x86_64-rubberband + mingw-w64-x86_64-libsndfile + mingw-w64-x86_64-flac + mingw-w64-x86_64-libogg + mingw-w64-x86_64-libvorbis + mingw-w64-x86_64-opus + mingw-w64-x86_64-opusfile + mingw-w64-x86_64-libmad + mingw-w64-x86_64-libid3tag + mingw-w64-x86_64-portaudio + mingw-w64-x86_64-serd + mingw-w64-x86_64-sord - - name: copy-qt-etc + # meson.build reads MINGW_PREFIX, which an MSYS2 shell sets to the + # POSIX path /mingw64: gcc wants the Windows one. Every step sets it, + # as ninja may run meson again + - name: configure + shell: msys2 {0} run: | - foreach ($lib in 'Core','Gui','Widgets','Network','Xml','Svg','Test') { copy "..\Qt\6.6.1\msvc2019_64\bin\Qt6$lib.dll" build } - cp sv-dependency-builds\win64-msvc\lib\libsndfile-1.dll build + export MINGW_PREFIX=$(cygpath -m /mingw64) + meson setup build --buildtype release - - name: build - run: ninja -C build + - name: make + shell: msys2 {0} + run: | + export MINGW_PREFIX=$(cygpath -m /mingw64) + ninja -C build - name: test - run: meson test -C build - + id: test + shell: msys2 {0} + run: | + export MINGW_PREFIX=$(cygpath -m /mingw64) + meson test -C build --print-errorlogs --num-processes 1 + # meson shows at most the last 100 lines of a failing log, and one + # test executable runs several QTest suites: name every failed test, + # with the lines QTest indents under it (compared values, where it + # failed) and the totals of each suite that failed + - name: test-failures + if: failure() && steps.test.outcome == 'failure' + shell: msys2 {0} + run: | + awk '/^test:/ { print; next } + /^Totals:/ { if ($0 !~ /, 0 failed/) print; next } + /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } + /^[A-Z!]+ *: / { fail = 0 } + fail && /^ / { print }' build/meson-logs/testlog.txt diff --git a/docs/building.md b/docs/building.md index d0e65f69..34dcca66 100644 --- a/docs/building.md +++ b/docs/building.md @@ -1,9 +1,9 @@ # Building on Windows (MSYS2 MinGW-w64) The development machine builds with meson + ninja under MSYS2's `mingw64` toolchain -(default prefix `C:\msys64\mingw64`) into `build_mingw/`. The CI workflows in -`.github/workflows/` build the upstream way on Linux, macOS and MSVC and are not what is -described here. +(default prefix `C:\msys64\mingw64`) into `build_mingw/`. The Windows CI workflow in +`.github/workflows/` builds the same way, from MSYS2's packages; the Linux and macOS ones +build the upstream way. ## From cmd or PowerShell: `build.bat` From aee97cd28c3b37da8c69bc38620b709d540f645d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:10:21 +0000 Subject: [PATCH 171/275] docs: the fork push was refused under custom network access The session that found a push to an unattached fork refused, and the same push going through once the fork was attached, had Custom network access, not Trusted; what Trusted does was not tried. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- docs/building.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/building.md b/docs/building.md index 919dbe42..f2a0d324 100644 --- a/docs/building.md +++ b/docs/building.md @@ -117,8 +117,9 @@ The environment's settings: added for the Android branch (the SDK, the NDK, and Gradle's Google repository, which `maven.google.com` redirects to). GitHub, conda-forge and Ubuntu's archive are in the default list; hg.sr.ht, download.qt.io and Qt's mirrors are not. A push to one of the - forks is another matter, which the level does not decide: with Trusted access too, it is - refused until the fork is attached to the session ([forks.md](forks.md#changing-a-fork)). + forks is another matter: with this Custom access it is refused (HTTP 403) until the fork + is attached to the session, and goes through once it is + ([forks.md](forks.md#changing-a-fork)). - Variables `BASH_DEFAULT_TIMEOUT_MS=600000` and `BASH_MAX_TIMEOUT_MS=1800000`, so that a build or a suite run is not moved to the background after the tool's default two minutes, and a 30-minute timeout can be given at all. From a4149363a5f26aa696d627c271f930c4b0fcc22f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:12:12 +0000 Subject: [PATCH 172/275] fix: the setup script says where the android sdk went Its last line showed the end of sdkmanager's progress bar, which is empty. Tried with dl.google.com allowed: the SDK and NDK arrive in 50 s, and deploy/android/setup-toolchain.sh then finds all of them installed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/cloud-environment.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/deploy/linux/cloud-environment.sh b/deploy/linux/cloud-environment.sh index 5c49ba7f..9d7c79d4 100755 --- a/deploy/linux/cloud-environment.sh +++ b/deploy/linux/cloud-environment.sh @@ -226,7 +226,12 @@ else fi wait "$android_pid" -say "Android SDK: exit $? ($(tail -1 "$logs/android.log"))" +android_status=$? +if [ -f /opt/android/sdk/ndk/27.2.12479018/source.properties ]; then + say "Android SDK: exit $android_status, the SDK and NDK are in /opt/android/sdk" +else + say "Android SDK: exit $android_status ($(tail -1 "$logs/android.log"))" +fi say "Done" exit 0 From 4fb73ac8dd86c5e0e2b6a7fdc40a99d1eaba6d96 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:15:25 +0000 Subject: [PATCH 173/275] fix: the lead-in check survives a stalled event loop Item 12 read the output levels only in the re-record's one lead-in gap, 0.4 s long; a stall of the GUI thread of about 340 ms could leave no look wholly inside it, and the check failed a clean run. The re-record now has a 2.4 s pre-roll, so its lead-in also takes in the 0.9 s gap inside the earlier punch-in (57 looks instead of 16). Where no look falls wholly in any gap, items 4 and 12 now say that part was not judged, and why, instead of failing. A test stalls the event loop during the lead-in, and the take-heard fault still fails through it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 22 ++++ docs/calibrate-audio.md | 2 +- main/dev/DevChecks.cpp | 53 +++++++-- main/dev/DevChecks.h | 31 +++-- main/test/TestDevChecks.h | 172 ++++++++++++++++++++++++++-- 5 files changed, 252 insertions(+), 28 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index b684f498..1287ddc3 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -504,3 +504,25 @@ have a 60 s long song; the fault runs and the deletion test leave it out. Left open: item 12 (C1c) once found 0 looks in its 0.4 s lead-in gap (15–16 usual, 9 once): a silent GUI stall of 340 ms or more there, likely this VM's disk; a real run could show it. The runner's 60 s limit on the reference's analysis is 6× 240 s's 10 s. `test-tony-dev` 180 s. + +### Phase C2b — 2026-09-26 +Built: `DevChecks::kReRecordPreRollSeconds` (2.4 s) for stage 3: its lead-in runs from 16.8 s, +where stage 2's second punch-in begins, over two silent gaps (16.8–17.7 and 18.8–19.2 s), the +sweep at 17.7 s and the tone from 18 s; item 12's looks there went from 15–16 to 57. +`GapLooks::longestWait` (the longest wait between two looks begun before `until`), a number +of item 12. Items 4 and 12: no look in any gap makes that part "not judged", with the reason +(longest wait, margin) in the message, not a Fail. `TestDevChecks`: +`dev_checks_lead_in_through_a_stall`, two rows: 0.45 s over the gap before P (judged, Pass) +and over the whole lead-in (not judged, Pass); `dev_checks_take_heard_during_the_lead_in` +gains the 0.45 s stall and still fails items 4 and 12 on the take heard; `describe(item)`. +Choices: the stall is a busy-wait in a 5 ms timer's slot, due by frames received since the +record start (the record duration is counted on the GUI thread and stands still in a stall). +A part not judged leaves the verdict to the other parts: Pass if they pass (Measured and +Skipped mean other things), the message saying what was not judged. +Items 7 and 14 read as before (14: 0.299 s past, earlier runs 0.31–0.35 s); item 13 unchanged. +Seen failing: before the fix, both tests (0 looks, item 12 Fail: the flake); the whole-lead-in +row with "not judged" put back among the problems. +The next phase must know: the margin is the most frames received across one look's two reads; +an OS stall between those reads (not the event loop) widens it for the whole take and can +leave no look in any gap: now "not judged", not a Fail. +Left open: `test-tony-dev` 13 tests, about 228 s (two runs of about 22 s added). diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 1de261c6..de5c2cb2 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -331,7 +331,7 @@ marked "Done" when it is committed. - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. Done. 5. **C2** Long song and joins: items 9 and 10. Done. - - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). + - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). Done. - **C2c** The notes merge keeps one note across a join: the defect item 10 found (the user chose to fix it on this branch). 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index 310da864..67b38f9b 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -533,7 +533,7 @@ DevChecks::start(const Options &options) [this]() { beginPunchInStage (m_reRecord, { reRecording() }, - AudioCheckRunner::kPreRollSeconds); + kReRecordPreRollSeconds); }, [this]() { return punchInStageDone(m_reRecord); }, kCheckStageTimeoutMs }); @@ -1476,6 +1476,9 @@ DevChecks::gapLooks(const LatencyCheck::Layout &layout, g.loudest = std::max(g.loudest, level); const double from = played(a.framesBefore) - block; const double to = played(b.framesAfter) + block; + if (from < until) { + g.longestWait = std::max(g.longestWait, (b.ms - a.ms) / 1000.0); + } if (from < start || to > until || !silent(from, to)) continue; ++g.looks; g.loudestInGaps = std::max(g.loudestInGaps, level); @@ -1531,6 +1534,7 @@ DevChecks::speakersCheck(QString reason) const double loudest = 0.0; double loudestInGaps = 0.0; double margin = 0.0; + double longestWait = 0.0; int gapPolls = 0; QString firstHeard; int n = 0; @@ -1587,6 +1591,7 @@ DevChecks::speakersCheck(QString reason) const loudest = std::max(loudest, g.loudest); loudestInGaps = std::max(loudestInGaps, g.loudestInGaps); margin = std::max(margin, g.margin); + longestWait = std::max(longestWait, g.longestWait); gapPolls += g.looks; if (g.heard > 0.0 && firstHeard == "") { firstHeard = tr("%1 from %2 to %3 s, in punch-in %4") @@ -1597,12 +1602,19 @@ DevChecks::speakersCheck(QString reason) const } if (n == 0) problems << tr("no punch-in was judged"); + // With no look in any gap, the take played out would not have shown: + // that part is not judged, rather than failed + QString notJudged; if (loudest <= 0.0) { problems << tr("no output level was reported while the reference " "played, so the silent gaps say nothing"); } else if (gapPolls == 0) { - problems << tr("no look at the output fell wholly in one of the " - "reference's silent gaps"); + notJudged = tr("What Tony played was not judged: no look at the " + "output lay wholly in one of the reference's silent " + "gaps, where the take played out would show (the " + "longest wait between two looks was %1, and a look " + "reaches %2 either side).") + .arg(unsignedMs(longestWait)).arg(unsignedMs(margin)); } if (firstHeard != "") { problems << tr("Tony played something where the reference is " @@ -1615,14 +1627,19 @@ DevChecks::speakersCheck(QString reason) const c.numbers.push_back({ tr("margin either side of a look"), unsignedMs(margin) }); - if (problems.isEmpty()) { + if (problems.isEmpty() && notJudged == "") { c.verdict = CheckResult::Verdict::Pass; c.message = tr("No sweep arrived twice, Tony played nothing where the " "reference is silent, and Play Singing Audio was as " "it had been after each take."); + } else if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("No sweep arrived twice, and Play Singing Audio was as " + "it had been after each take. %1").arg(notJudged); } else { c.verdict = CheckResult::Verdict::Fail; c.message = problems.join("; ") + "."; + if (notJudged != "") c.message += " " + notJudged; } return c; } @@ -2145,10 +2162,13 @@ DevChecks::leadInCheck(QString reason) const // And while it played, Tony played the reference and nothing else: // item 4's looks at the output, those that lie wholly before P. The - // take's audio is under them now, and kept silent as in any take + // take's audio is under them now, and kept silent as in any take. + // With no look in any gap, the take played out would not have shown: + // that part is not judged, rather than failed const Watched *w = stage.watched.empty() ? nullptr : &stage.watched[0]; const TakeLatency t = stage.result.takes.empty() ? TakeLatency() : stage.result.takes[0]; + QString notJudged; if (!w) { problems << tr("the punch-in was not watched"); } else { @@ -2158,9 +2178,14 @@ DevChecks::leadInCheck(QString reason) const "lead-in played could not be placed"); } else { if (g.looks == 0) { - problems << tr("no look at the output during the lead-in " - "fell wholly in one of the reference's " - "silent gaps"); + notJudged = tr("What Tony played during the lead-in was " + "not judged: no look at the output lay " + "wholly in one of the reference's silent " + "gaps, where the take played out would " + "show (the longest wait between two looks " + "was %1, and a look reaches %2 either " + "side).") + .arg(unsignedMs(g.longestWait)).arg(unsignedMs(g.margin)); } if (g.heard > 0.0) { problems << tr("during the lead-in Tony played something " @@ -2173,19 +2198,29 @@ DevChecks::leadInCheck(QString reason) const ({ tr("output in the lead-in's silent gaps"), tr("%1, over %2 looks").arg(levelText(g.loudestInGaps)) .arg(g.looks) }); + c.numbers.push_back + ({ tr("longest wait between two looks in the lead-in"), + unsignedMs(g.longestWait) }); } } - if (problems.isEmpty()) { + if (problems.isEmpty() && notJudged == "") { c.verdict = CheckResult::Verdict::Pass; c.message = tr("Before the punch-in the take's audio is the same bit " "for bit, and its pitch and notes beyond %1 s of it; " "and while the lead-in played over the take, Tony " "played nothing where the reference is silent.") .arg(TakeDiff::kEventMarginSeconds); + } else if (problems.isEmpty()) { + c.verdict = CheckResult::Verdict::Pass; + c.message = tr("Before the punch-in the take's audio is the same bit " + "for bit, and its pitch and notes beyond %1 s of it. " + "%2").arg(TakeDiff::kEventMarginSeconds) + .arg(notJudged); } else { c.verdict = CheckResult::Verdict::Fail; c.message = problems.join("; ") + "."; + if (notJudged != "") c.message += " " + notJudged; } return c; } diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index a9164dcf..9da67a7f 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -166,6 +166,14 @@ class DevChecks : public QObject /// this far below the loudest input's static constexpr double kMicChannelDb = 20.0; + /// Stage 3's pre-roll, in seconds: a lead-in from 16.8 s, where + /// stage 2's second punch-in begins, so that it plays over two of + /// the reference's silent gaps in what that punch-in recorded (16.8 + /// to 17.7 s and 18.8 to 19.2 s). Item 12 looks at the output in + /// both: a window held up over one still has the other to be judged + /// in + static constexpr double kReRecordPreRollSeconds = 2.4; + /// Stage 4's pre-roll, in seconds: more than there is room for /// before nearTheStart() static constexpr double kNearStartPreRollSeconds = 3.0; @@ -261,13 +269,13 @@ class DevChecks : public QObject /** * Stage 3's punch-in, in seconds: [19.2, 21.2], over the end of * stage 2's second and past its first sweep, judging the sweep at - * 20.1 s. Its 1 s lead-in plays over what stage 2 recorded there: - * the last of the tone from 18 s, and from 18.8 s a gap where the - * reference is silent and the take holds only what the mic heard - * besides. Starting this late keeps the range short, gives the - * lead-in a gap to be looked at in, and leaves a whole note of the - * take (18 to 18.8 s) before it for the ranged analysis to leave - * alone. + * 20.1 s. Its lead-in (kReRecordPreRollSeconds) plays over what + * stage 2 recorded from its start: a gap where the reference is + * silent and the take holds only what the mic heard besides (16.8 + * to 17.7 s), the sweep at 17.7 s, the tone from 18 s, and another + * such gap from 18.8 s. Starting this late keeps the range short + * and leaves a whole note of the take (18 to 18.8 s) before it for + * the ranged analysis to leave alone. */ static LatencyCheck::PunchIn reRecording(); @@ -487,8 +495,15 @@ class DevChecks : public QObject double heardFrom; double heardTo; + /// The longest wait between two looks while recording, in + /// seconds, of those beginning before "until": the window held + /// up. The frames received jump meanwhile, so the look across + /// the wait reaches over a sound, and is left out + double longestWait; + GapLooks() : placed(false), margin(0), loudest(0), loudestInGaps(0), - looks(0), heard(0), heardFrom(0), heardTo(0) { } + looks(0), heard(0), heardFrom(0), heardTo(0), + longestWait(0) { } }; GapLooks gapLooks(const LatencyCheck::Layout &layout, const TakeObserver::Observation &seen, diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index cd297682..6b69923d 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -93,6 +94,18 @@ class TestDevChecks : public QObject // How often a test's fault was put in int m_faults = 0; + // A stall of the GUI thread during the re-recording's lead-in + // (stallTheReRecording()): watched for from the runner's first + // report that it records; from where in the reference it is due, in + // seconds (negative for none), and how long; how often it came, and + // where playback was as it began and as it ended + QTimer m_stallWatch; + double m_stallAt = -1.0; + int m_stallMs = 0; + int m_stalls = 0; + double m_stalledFrom = 0.0; + double m_stalledTo = 0.0; + void makeWindow(FakeAudioIO::Config config) { delete m_window; m_window = new TestMainWindow(config); @@ -101,6 +114,10 @@ class TestDevChecks : public QObject m_stages.clear(); m_checks.clear(); m_faults = 0; + m_stallWatch.stop(); + m_stallAt = -1.0; + m_stallMs = 0; + m_stalls = 0; connect(m_window->devChecks(), &DevChecks::finished, this, [this](const DevReport &report) { m_report = report; @@ -222,10 +239,12 @@ class TestDevChecks : public QObject return lines.isEmpty() ? QString() : lines.last(); } - QByteArray describe() { + // Every check, or only the item given: QtTest cuts a long message + QByteArray describe(int item = 0) { QStringList words; words << "failure: " + m_report.failure; for (const CheckResult &c : m_report.checks) { + if (item > 0 && c.item != item) continue; QStringList numbers; for (const auto &n : c.numbers) numbers << n.first + ": " + n.second; words << QString("item %1 %2 %3 (%4) [%5]").arg(c.item).arg(c.name) @@ -313,6 +332,54 @@ class TestDevChecks : public QObject QCOMPARE(storedLatency(), storedBefore); } + // The GUI thread held up, busy, for ms, as a busy system can hold a + // window up: no timer fires meanwhile, so the observer takes no + // look. It begins once the re-recording has played the reference up + // to "at", in seconds. By default over the reference's silent gap + // just before the punch-in (18.8 to 19.2 s): no look there lies + // wholly in the gap + void stallTheReRecording(double at = DevChecks::reRecording().start - 0.45, + int ms = 450) { + m_stallAt = at; + m_stallMs = ms; + connect(m_window->audioCheck(), &AudioCheckRunner::progress, + this, [this](const AudioCheckRunner::Progress &state) { + // Reported again as the seconds left go down + if (state.step == AudioCheckRunner::Step::Recording && + m_stalls == 0 && !m_stallWatch.isActive() && + !m_stages.isEmpty() && + m_stages.last().endsWith(": Re-record")) { + m_stallWatch.start(); + } + }); + } + + // Not a slot: QtTest would run it as a test + void stallIfDue() { + sv::AudioCallbackRecordTarget *target = + m_window ? m_window->recordTarget() : nullptr; + if (m_stallAt < 0.0 || !target || !target->isRecording()) return; + + // Where the reference being handed out is: where playback + // started, and the frames come in since, which the audio thread + // counts. The record duration, which the cursor goes by, is + // counted on this thread, and would stand still during the stall + const sv::sv_frame_t start = + m_window->playbackFrame() - target->getRecordDuration(); + auto played = [&]() { + return double(start + target->getFramesReceived()) / rate; + }; + const double at = played(); + if (at < m_stallAt) return; + m_stallWatch.stop(); + QElapsedTimer held; + held.start(); + while (held.elapsed() < m_stallMs) { } + m_stalledFrom = at; + m_stalledTo = played(); + ++m_stalls; + } + // Not a slot: QtTest would run it as a test. As TestAudioCheck's void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -362,6 +429,10 @@ private slots: connect(&m_watchdog, &QTimer::timeout, this, [this]() { dismissDialog(); }); m_watchdog.start(50); + + m_stallWatch.setInterval(5); + connect(&m_stallWatch, &QTimer::timeout, + this, [this]() { stallIfDue(); }); } void init() { @@ -385,6 +456,7 @@ private slots: void cleanup() { QSettings().remove("LatencyCalibration"); + m_stallWatch.stop(); if (m_window) { if (m_window->recordTarget()->isRecording()) { @@ -821,11 +893,15 @@ private slots: // that punch-in recording, before the reference starts to play. The // lead-in plays over what stage 1 recorded, which in a room holds // noise where the reference is silent, and items 4 and 12 fail on - // the output there; nothing before the punch-in changed. The run is - // cancelled as the stage after the re-recording begins: the checks - // of the stages it got through are worked out all the same + // the output there; nothing before the punch-in changed. The window + // is held up over the lead-in's last silent gap, as in + // dev_checks_lead_in_through_a_stall(): the take is heard in the gap + // before it all the same. The run is cancelled as the stage after + // the re-recording begins: the checks of the stages it got through + // are worked out all the same void dev_checks_take_heard_during_the_lead_in() { makeWindow(loopbackInARoom()); + stallTheReRecording(); connect(m_window->audioCheck(), &AudioCheckRunner::progress, this, [this](const AudioCheckRunner::Progress &state) { // Reported again as the seconds left go down @@ -853,24 +929,25 @@ private slots: runDevChecks(roundTrip / rate, 0.0); if (QTest::currentTestFailed()) return; QCOMPARE(m_faults, 1); + QCOMPARE(m_stalls, 1); QCOMPARE(m_report.failure, QString("The dev checks were cancelled.")); QCOMPARE(int(m_checks.size()), 2); const CheckResult *speakers = check(4); const CheckResult *leadIn = check(12); QVERIFY(speakers && leadIn); - QVERIFY2(speakers->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(speakers->verdict == CheckResult::Verdict::Fail, describe(4)); QVERIFY2(speakers->message.startsWith("Tony played something where " "the reference is silent: -"), - describe()); - QVERIFY2(speakers->message.contains(", in punch-in 3."), describe()); + describe(4)); + QVERIFY2(speakers->message.contains(", in punch-in 3."), describe(4)); QCOMPARE(number(*speakers, "second arrival"), QString("none heard")); - QVERIFY2(leadIn->verdict == CheckResult::Verdict::Fail, describe()); + QVERIFY2(leadIn->verdict == CheckResult::Verdict::Fail, describe(12)); QVERIFY2(leadIn->message.startsWith("during the lead-in Tony played " "something where the reference " - "is silent: -"), describe()); - QVERIFY2(!leadIn->message.contains(";"), describe()); + "is silent: -"), describe(12)); + QVERIFY2(!leadIn->message.contains(";"), describe(12)); QCOMPARE(number(*leadIn, "audio before 19.20 s"), QString("the same, bit for bit")); @@ -882,6 +959,81 @@ private slots: } } + // The window held up during the re-recording's lead-in, as a busy + // system may hold it up: the observer takes no look meanwhile, and + // the look across the stall reaches over the sounds either side, so + // it is left out. Held up for 0.45 s over the silent gap just before + // the punch-in, the lead-in still has the gap from 16.8 to 17.7 s + // looked at, so item 12 judges what was played, and passes, as do + // the re-recording's other checks. Held up over the whole lead-in, + // no look lies in a gap, and that part of item 12 is not judged, + // with the reason, rather than failed. Cancelled as the stage after + // the re-recording begins + void dev_checks_lead_in_through_a_stall_data() { + QTest::addColumn("at"); + QTest::addColumn("ms"); + QTest::addColumn("judged"); + const double start = DevChecks::reRecording().start; + QTest::newRow("over the last gap") << start - 0.45 << 450 << true; + QTest::newRow("over the whole lead-in") + << start - DevChecks::kReRecordPreRollSeconds + << int(1000 * DevChecks::kReRecordPreRollSeconds) + 50 << false; + } + + void dev_checks_lead_in_through_a_stall() { + QFETCH(double, at); + QFETCH(int, ms); + QFETCH(bool, judged); + makeWindow(loopbackInARoom()); + stallTheReRecording(at, ms); + connect(m_window->devChecks(), &DevChecks::progress, + this, [this](QString stage, int, int) { + if (stage == "Pre-roll near the start") { + m_window->devChecks()->cancel(); + } + }); + + runDevChecks(roundTrip / rate, 0.0); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_report.failure, QString("The dev checks were cancelled.")); + QCOMPARE(int(m_checks.size()), 2); + + // Held up from where it was due until the punch-in, and the + // observer saw it + QCOMPARE(m_stalls, 1); + const double start = DevChecks::reRecording().start; + QVERIFY2(m_stalledFrom < at + 0.05 && m_stalledTo > start - 0.05, + qPrintable(QString("stalled from %1 to %2 s") + .arg(m_stalledFrom).arg(m_stalledTo))); + const CheckResult *leadIn = check(12); + QVERIFY(leadIn); + QVERIFY2(milliseconds(number(*leadIn, "longest wait between two " + "looks in the lead-in")) >= ms, + describe(12)); + + QVERIFY2(leadIn->verdict == CheckResult::Verdict::Pass, describe(12)); + const QString gaps = + number(*leadIn, "output in the lead-in's silent gaps"); + if (judged) { + QVERIFY2(gaps.startsWith("silence, over ") && + !gaps.endsWith(" 0 looks"), describe(12)); + QVERIFY2(!leadIn->message.contains("not judged"), describe(12)); + } else { + QCOMPARE(gaps, QString("silence, over 0 looks")); + QVERIFY2(leadIn->message.contains + ("What Tony played during the lead-in was not judged: " + "no look at the output lay wholly in one of the " + "reference's silent gaps"), describe(12)); + QVERIFY2(!leadIn->message.contains("Tony played nothing"), + describe(12)); + } + for (int item : { 4, 7, 14 }) { + QVERIFY2(check(item) && + check(item)->verdict == CheckResult::Verdict::Pass, + describe(item)); + } + } + // Cancelled during a take: the take stops, the run ends once with // every check skipped, and the user's toggles and stored round trip // are as they were From c4c0807a5f9d153a9a44ea82f831ce3363ad8b27 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:15:49 +0000 Subject: [PATCH 174/275] docs: calibrate audio work orders, C2b done Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 1287ddc3..ec2e9003 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,7 +194,7 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). From 082e7c9666feccb3172df64cac6519d643d7432a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:23:23 +0000 Subject: [PATCH 175/275] docs: auto mode blocks fork work unless the user names it A cloud session that followed the fork steps was denied at each one: auto mode trusts only the session's own repository and its remotes, so committing in a fork's checkout, attaching the fork and pushing to it need the user's own message to ask for that action, and a retry after a denial counts as getting round the check. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- docs/forks.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/forks.md b/docs/forks.md index 2108643c..db21cea0 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -26,10 +26,17 @@ library over a workaround in `main/`. follow that repository's style: `area: what`. 2. Push to the remote named **`jhhr`**. In `svcore`, `svgui` and `svapp`, `origin` is upstream sonic-visualiser — do not push there. In a cloud session the checkouts are - `container-setup.sh`'s, whose `origin` is the fork. There a push, even of a new branch, - is refused (HTTP 403) unless the fork is attached to the session: attach it with the - session's add-repository tool, with push access, and push from the checkout that is - there. Do not start the session with the forks selected instead: a session with several + `container-setup.sh`'s, whose `origin` is the fork, and two checks stand in the way: + - The session's git proxy refuses a push to a repository not attached to the session, + a new branch included (HTTP 403). The session's add-repository tool attaches it, with + push access. + - Auto mode trusts only the repository the session started in and its remotes. It + blocks committing in a fork's checkout, attaching the fork and pushing to it, unless + the user's own message asks for that action, naming the fork and the branch. After a + denial, stop and tell the user what is blocked: trying again another way counts as + getting round the check, and is blocked too. The user can instead push the change. + + Do not start the session with the forks selected instead: a session with several repositories runs no repository's SessionStart hook, so the background build does not start. 3. Put the new commit hash in `repoint-lock.json` as that library's `pin`, and commit that From 3b94043da6704bc9a7e55c22f782c4f167fc289c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:26:29 +0000 Subject: [PATCH 176/275] build: the cloud environment trusts the library forks in auto mode As the user chose: the setup script writes an autoMode.environment entry to /root/.claude/settings.json naming jhhr/svcore, svgui, svapp and bqaudiostream as trusted repositories, so a session may commit in their checkouts, attach them and push branches to them. Auto mode does not read autoMode from the repository's .claude/settings.json; claude auto-mode config combines this entry with the platform's own. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01K5Hto1vETpg1RaQSaQUvim --- deploy/linux/cloud-environment.sh | 43 +++++++++++++++++++++++++++++-- docs/building.md | 5 +++- docs/forks.md | 13 ++++++---- 3 files changed, 53 insertions(+), 8 deletions(-) diff --git a/deploy/linux/cloud-environment.sh b/deploy/linux/cloud-environment.sh index 9d7c79d4..a10b4a56 100755 --- a/deploy/linux/cloud-environment.sh +++ b/deploy/linux/cloud-environment.sh @@ -30,6 +30,8 @@ # - Qt 6.11.2 from conda-forge in /opt/qt6-conda, where # container-setup.sh looks for it. # - /etc/ccache.conf. meson uses ccache by itself when it is installed. +# - An autoMode entry in /root/.claude/settings.json by which auto mode +# trusts the library forks as it does the session's own repository. # - The Android SDK and NDK in /opt/android/sdk, where # deploy/android/setup-toolchain.sh installs them, when dl.google.com is # reachable. @@ -84,7 +86,44 @@ hash_dir = false base_dir = /home/user EOF -# 2. Packages, Qt and the Android SDK, side by side +# 2. Auto mode's trust, as the user chose it: the library forks are the +# user's own repositories. Out of the box auto mode trusts only the +# repository a session started in and its remotes, and blocks committing +# in a fork's checkout, attaching the fork and pushing to it +# (docs/forks.md). It reads autoMode from the user's settings, never +# from the repository's .claude/settings.json, and combines them with +# the platform's own (--settings); `claude auto-mode config` shows the +# result. + +TONY_AUTOMODE_SETTINGS=${TONY_AUTOMODE_SETTINGS:-/root/.claude/settings.json} python3 - <<'EOF' +import json, os +path = os.environ["TONY_AUTOMODE_SETTINGS"] +entries = [ + "Trusted repo: besides the working repository github.com/jhhr/tony, the user's own " + "forks of its libraries, github.com/jhhr/svcore, github.com/jhhr/svgui, " + "github.com/jhhr/svapp and github.com/jhhr/bqaudiostream, checked out inside the " + "working directory as svcore/, svgui/, svapp/ and bqaudiostream/", + "Source control: github.com/jhhr/tony and those four forks. Committing in the fork " + "checkouts, attaching the forks to the session with push access and pushing branches " + "to them is routine work on Tony (docs/forks.md)", +] +settings = {} +if os.path.exists(path): + with open(path) as f: + settings = json.load(f) +environment = settings.setdefault("autoMode", {}).setdefault("environment", []) +if not environment: + environment.append("$defaults") +for entry in entries: + if entry not in environment: + environment.append(entry) +os.makedirs(os.path.dirname(path), exist_ok=True) +with open(path, "w") as f: + json.dump(settings, f, indent=2) + f.write("\n") +EOF + +# 3. Packages, Qt and the Android SDK, side by side packages=" build-essential pkg-config ninja-build meson git python3 curl ca-certificates @@ -175,7 +214,7 @@ wait "$qt_pid" qt_status=$? say "Qt: exit $qt_status" -# 3. ccache, from a build of the libraries in the checkout, until the +# 4. ccache, from a build of the libraries in the checkout, until the # deadline fill_ccache() { diff --git a/docs/building.md b/docs/building.md index f2a0d324..41a905b0 100644 --- a/docs/building.md +++ b/docs/building.md @@ -96,7 +96,10 @@ reached. Three scripts in `deploy/linux/` do the work: the disk, which later sessions start from, until the script or the allowed hosts change or about a week has passed. It installs the packages, Qt, ccache and mold, the Android SDK and NDK when `dl.google.com` is reachable, and spends what is left of four minutes filling - ccache from a build of the libraries. The snapshot is kept only when the script ends + ccache from a build of the libraries. It also writes an `autoMode` entry to + `/root/.claude/settings.json` by which auto mode trusts the four library forks as it does + Tony's own repository ([forks.md](forks.md#changing-a-fork)): auto mode reads that from + the user's settings, never from the repository's `.claude/settings.json`. The snapshot is kept only when the script ends within about five minutes, so any change to it has to keep to that. Its logs are in `/var/log/tony-environment/`. - **`container-setup.sh`** makes any fresh Ubuntu 24.04 able to build: packages, Qt, the diff --git a/docs/forks.md b/docs/forks.md index db21cea0..d214a5c0 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -30,11 +30,14 @@ library over a workaround in `main/`. - The session's git proxy refuses a push to a repository not attached to the session, a new branch included (HTTP 403). The session's add-repository tool attaches it, with push access. - - Auto mode trusts only the repository the session started in and its remotes. It - blocks committing in a fork's checkout, attaching the fork and pushing to it, unless - the user's own message asks for that action, naming the fork and the branch. After a - denial, stop and tell the user what is blocked: trying again another way counts as - getting round the check, and is blocked too. The user can instead push the change. + - Auto mode trusts only the repository the session started in and its remotes, and so + blocks committing in a fork's checkout, attaching the fork and pushing to it. The + environment's setup script names the four forks as trusted as well, which the user + chose ([building.md](building.md#building-on-linux)); `claude auto-mode config` + shows whether a session has that entry. Without it, the user's own message has to + ask for the action, naming the fork and the branch. After a denial, stop and tell the + user what is blocked: trying again another way counts as getting round the check, and + is blocked too. The user can instead push the change. Do not start the session with the forks selected instead: a session with several repositories runs no repository's SessionStart hook, so the background build does not From 2024f9da399a1642eb195d2356a186de3f779cf5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:40:04 +0000 Subject: [PATCH 177/275] test: tests of what happens during a take's analysis hold that analysis on a fast machine pyin over a take under a second long finishes and is merged inside analyseRange(), before stop returns, and eleven tests of undo, save, a second take or a teardown during the analysis failed there with "the race was not set up". a test hold on the analyser (setHoldRangedMerge(), off unless a test sets it) leaves the finished run unmerged until the test releases it; mainwindow passes it on to every analyser made while it is set, as stop, undo and redo make new ones. the teardown tests' comments say that pyin's thread itself may still have finished. seen failing with the hold not applied, the save not waiting, undo not abandoning the run, and the hold not passed on. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- docs/testing.md | 12 ++++++ main/Analyser.cpp | 17 +++++++- main/Analyser.h | 14 ++++++ main/MainWindow.cpp | 2 + main/MainWindow.h | 7 +++ main/test/TestMainWindow.h | 15 +++++++ main/test/TestRecordWorkflow.h | 79 +++++++++++++++++++++++++--------- main/test/TestUiChecks.h | 14 ++++-- 8 files changed, 136 insertions(+), 24 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index 21fb150b..b4b457d1 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -161,6 +161,18 @@ it; the marker goes in the commit that fixes it. There are none at present. - A take's analysis lands in two steps, `rangedAnalysisMerged()` then `initialAnalysisCompleted()`. Read results after the merge (`analysingRange()` false), not after some other signal that happens to come at about the same time. +- **A test of something done while a take is being analysed holds that analysis**: + `TestMainWindow::holdTakeAnalysis()` before Stop (or Analyse Now), and + `releaseTakeAnalysis()` where the race is over. On a fast machine pYIN over a take + under a second long is finished and merged before Stop returns, and such a test fails + there with "the race was not set up" while passing on a slower one. Held, the finished + result waits unmerged and `analysingRange()` stays true through turns of the event + loop; the hold stays on for every run started until the release, through undo, redo + and the analyser being made again (`Analyser::setHoldRangedMerge()`). Release before + anything that waits for the analysis (`analysed()`), or the wait times out; a save + waits by itself, so a test of that releases from a zero-time timer set just before + it, which runs inside the save's wait. The hold does not hold pYIN's thread: whether + that is still running at a teardown depends on the machine and its load, as below. - The status bar is written by three base-class timers; a test that reads it must go through what `showTakeCountdown()` controls. - Deleting a derived layer does not stop its transform; only diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 15faf8d1..e916e108 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -67,7 +67,8 @@ Analyser::Analyser(ColorScheme colorScheme) : m_rangedEnd(0), m_rangedMergeStart(0), m_rangedMergeEnd(0), - m_rangedClippedEnd(false) + m_rangedClippedEnd(false), + m_holdRangedMerge(false) { QSettings settings; settings.beginGroup("LayerDefaults"); @@ -1204,9 +1205,23 @@ Analyser::rangedAnalysisCompletionChanged(ModelId) // at 100 in both means a result if (!newPitch->isReady() || !newNotes->isReady()) return; + // A test holding the result (setHoldRangedMerge()): no more signals + // are coming, and the release calls this again + if (m_holdRangedMerge) return; + mergeRangedAnalysis(); } +void +Analyser::setHoldRangedMerge(bool hold) +{ + m_holdRangedMerge = hold; + + // A run that finished while held has had the last of its completion + // signals, so nothing else would merge it + if (!hold) rangedAnalysisCompletionChanged({}); +} + void Analyser::mergeRangedAnalysis() { diff --git a/main/Analyser.h b/main/Analyser.h index 2f90469c..f70e6cd1 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -236,6 +236,17 @@ class Analyser : public QObject, */ void cancelRangedAnalysis() { discardRangedAnalysis(); } + /** + * For the tests: while held, a ranged analysis that has finished is + * left unmerged, and isAnalysingRange() stays true, just as while + * pYIN is still running. Releasing merges a run that finished + * while held. A test that does something during the analysis of a + * take needs the run still there when it does it, and pYIN over + * less than a second of audio can finish, on a fast machine, before + * analyseRange() has even returned. Off unless a test sets it. + */ + void setHoldRangedMerge(bool hold); + /** * What the last ranged merge took out of and put into the pitch * track and the notes. Reversing these two changes undoes the @@ -400,6 +411,9 @@ protected slots: // cannot stamp anything before its own first two hops anyway bool m_rangedClippedEnd; + // See setHoldRangedMerge() + bool m_holdRangedMerge; + // What the last merge did, for the undo command of the recording // that asked for the analysis (see getRangedPitchChange()) TakeEvents::Change m_rangedPitchChange; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 59f5c59f..5a7f11de 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -205,6 +205,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_currentRecordingModelId(), m_recordingLayer(nullptr), m_rebuildingTakeAudio(false), + m_holdTakeAnalysis(false), m_recordingLatencyFrames(0), m_recordingStartGapEstimate(0), m_recordingStartGapMeasured(-1), @@ -3439,6 +3440,7 @@ MainWindow::setupSingingTrackAnalyser(sv::ModelId singingModelId, bool deferAnal // Create the secondary analyser with the singing-track colour scheme m_analyser2 = new Analyser(Analyser::SecondaryColors); + m_analyser2->setHoldRangedMerge(m_holdTakeAnalysis); connect(m_analyser2, SIGNAL(layersChanged()), this, SLOT(updateLayerStatuses())); diff --git a/main/MainWindow.h b/main/MainWindow.h index 67261c09..94cee819 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -879,6 +879,13 @@ protected slots: // what the singer sang would be left unanalysed. Coverage::Range m_takeAnalysisRange; + // For the tests (TestMainWindow::holdTakeAnalysis()): every singing + // analyser made while this is set holds the result of its ranged + // analysis unmerged (Analyser::setHoldRangedMerge()). Here and not + // only in the Analyser because a recording, an undo or a redo makes + // the analyser again before it starts the analysis to be held. + bool m_holdTakeAnalysis; + // Round-trip hardware latency (the figure the audio check measured, // or else output + input as the device reports them, in frames of the // recording; see roundTripAt()) stored when a singing-track recording diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index b64a54aa..7311a551 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -109,6 +109,21 @@ class TestMainWindow : public MainWindow sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } + // A test of something done while the take is being analysed holds + // the analysis from before Stop: on a fast machine pYIN over a short + // take is finished and merged before Stop returns. Held, the result + // waits unmerged, and analysingRange() stays true, through any number + // of turns of the event loop and of analysers made again, until it is + // released. Release before waiting for the analysis to finish + void holdTakeAnalysis() { + m_holdTakeAnalysis = true; + if (m_analyser2) m_analyser2->setHoldRangedMerge(true); + } + void releaseTakeAnalysis() { + m_holdTakeAnalysis = false; + if (m_analyser2) m_analyser2->setHoldRangedMerge(false); + } + // Save As, with the file name given here instead of by a dialog: the // session's own file is set, so that what is recorded next goes into // its takes folder diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 741d465d..9fc1e838 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1661,13 +1661,13 @@ private slots: // Stop splices the recording in and asks for the analysis of the // range it went into. The range is read here, while that run is - // still going: it is remembered only until the merge, so a run that - // finishes before the splice call returns never records one at all + // held: it is remembered only until the merge + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); - if (m_window->analysingRange()) { - QCOMPARE(m_window->analysedRangeStart(), P); - } + QVERIFY(m_window->analysingRange()); + QCOMPARE(m_window->analysedRangeStart(), P); + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); // The analysis of the take was not thrown away and run again: the @@ -1725,7 +1725,9 @@ private slots: QTest::qWait(900); // Stop splices the recording in and starts the analysis of the - // range it went into there and then + // range it went into there and then. Held, so that it is still + // unmerged when the second take stops, however quick the machine + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(m_window->analysingRange(), @@ -1735,20 +1737,16 @@ private slots: sv::sv_frame_t firstEnd = m_window->analysedRangeEnd(); QVERIFY(firstEnd > sv::sv_frame_t(0.7 * rate)); - // A second take in a gap, recorded without letting the event - // loop run: the result of a ranged analysis is merged from a - // queued call, so the first one cannot have finished by the time - // this one stops, however quick the machine is. (The device - // records from a thread of its own, and the record target's ring - // buffer holds ten seconds.) + // A second take in a gap, with the first range's analysis still + // held through it const sv::sv_frame_t P = sv::sv_frame_t(3.0 * rate); m_window->seekTo(P); startTake(); if (QTest::currentTestFailed()) return; - QThread::msleep(250); + QTest::qWait(250); QVERIFY2(m_window->analysingRange(), - "the first range's analysis finished before the second take " - "stopped: something ran the event loop"); + "the first range's analysis was merged before the second " + "take stopped, although it was held"); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); @@ -1757,6 +1755,7 @@ private slots: QCOMPARE(m_window->analysedRangeStart(), sv::sv_frame_t(0)); QVERIFY(m_window->analysedRangeEnd() > P); + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); auto events = pitchEvents(m_window->analyser2()); QVERIFY2(!eventsBetween(events, 0, firstEnd).empty(), @@ -1770,7 +1769,10 @@ private slots: // The models the analysis of a recorded range is to be merged into, // torn down while it is still running: by another singing track, and // by the session going. A regression guard for the area this fork has - // crashed in before -- a crash is the failure. + // crashed in before -- a crash is the failure. The analysis is held, + // so its run is there, unmerged, when they go, however quick the + // machine. Whether pYIN's thread is still going then as well depends + // on the machine and its load: run it under load to check. void range_analysis_torn_down_while_running() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); @@ -1781,6 +1783,7 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(900); + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY2(m_window->analysingRange(), "the race was not set up: nothing was being analysed after " @@ -1788,6 +1791,7 @@ private slots: // Another singing track over it m_window->loadSingingTrack(writeWav(tone(lowHz, 1.0))); + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); QVERIFY(!m_window->analysingRange()); verifyPlaySourceClean(); @@ -1798,10 +1802,12 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(700); + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(m_window->analysingRange()); m_window->doCloseSession(); + m_window->releaseTakeAnalysis(); QVERIFY(!m_window->analyser2()); QVERIFY(!m_window->takes()->haveTake()); QCOMPARE(m_window->paneStack()->getPaneCount(), 0); @@ -2405,6 +2411,9 @@ private slots: // middle of its pYIN, which is the area this fork has crashed in // before; cancelAnalyses() is what keeps it safe. A regression guard, // not a new behaviour: run it under load, a crash is the failure. + // The analysis is held, so that it is unmerged when the second take + // stops however quick the machine; whether its thread is still going + // then as well depends on the machine and its load. void rerecord_during_analysis() { FakeAudioIO::Config config; config.input = tone(highHz, 4.0); @@ -2418,19 +2427,29 @@ private slots: // Stop splices the recording in and starts the analysis of the // result there and then + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(sv::ModelTransformerFactory::getInstance() ->haveRunningTransformers(), "the race was not set up: no analysis was running when the " "second take started"); + QVERIFY(m_window->analysingRange()); QString first = m_window->takes()->getAudioPath(); QVERIFY(!first.isEmpty()); // In a gap, so nothing is asked m_window->seekTo(sv::sv_frame_t(1.5 * rate)); - take(500); + startTake(); if (QTest::currentTestFailed()) return; + QTest::qWait(500); + QVERIFY2(m_window->analysingRange(), + "the first take's analysis was over before the second one " + "stopped"); + m_window->doRecord(); + QVERIFY(!m_window->recordTarget()->isRecording()); + m_window->releaseTakeAnalysis(); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); QCOMPARE(m_window->recordOverQuestions(), 0); QVERIFY(m_window->analyser2()); @@ -2702,6 +2721,9 @@ private slots: for (const auto &e : model->getAllEvents()) model->remove(e); QVERIFY(pitchEvents(pitch).empty()); + // Held, so that the range is still there to be read however quick + // the run: it is remembered only until the merge + m_window->holdTakeAnalysis(); m_window->doAnalyseNow(); // One run over the span of the coverage, not one per range @@ -2709,6 +2731,7 @@ private slots: QCOMPARE(m_window->analysedRangeStart(), ranges[0].start); QCOMPARE(m_window->analysedRangeEnd(), ranges[1].end); + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); @@ -3909,11 +3932,13 @@ private slots: QVERIFY(!before.pitch.empty()); // Stop splices the recording in and starts the analysis of it - // there and then + // there and then. Held, so that it is still running when Undo is + // pressed, and the redo's run when its range is read m_window->seekTo(sv::sv_frame_t(2.0 * rate)); startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(700); + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(m_window->analysingRange(), @@ -3936,8 +3961,10 @@ private slots: // Redo: the range is analysed again, and this time the result // reaches the take's pitch track QCOMPARE(redoOnce(), QString("Record Singing")); + QVERIFY(m_window->analysingRange()); QCOMPARE(m_window->analysedRangeStart(), analysedStart); QCOMPARE(m_window->analysedRangeEnd(), analysedEnd); + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); auto ranges = m_window->takes()->getCoverage().getRanges(); @@ -5270,12 +5297,18 @@ private slots: QTest::qWait(700); // Stop splices the recording in and starts the analysis of it there - // and then, so it is running when the session is saved + // and then, so it is running when the session is saved. Held, and + // released from the first turn of the event loop, which is the + // save's own wait: the merge can only come while the save waits + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(m_window->analysingRange(), "the test shows nothing: no analysis was running when the " "session was saved"); + QTimer::singleShot(0, m_window, [this]() { + m_window->releaseTakeAnalysis(); + }); QString session = m_dir.filePath("mid-analysis.ton"); QVERIFY(m_window->saveSessionFile(session)); @@ -5526,7 +5559,10 @@ private slots: // thread ("Timers cannot be stopped from another thread") and the // process dies with an access violation soon after. A regression // shows up as a crash of the whole test program, and not reliably: - // run it under load to check. + // run it under load to check. The analysis is held, so that its run + // is there, unmerged, at the close however quick the machine; whether + // its thread is still going then as well depends on the machine and + // its load. void close_session_during_analysis() { FakeAudioIO::Config config; config.input = tone(highHz, 3.0); @@ -5541,13 +5577,16 @@ private slots: // Stop splices the recording into the take and starts the analysis // of the result there and then, so it is running when the session // is closed. Otherwise this test shows nothing + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY2(sv::ModelTransformerFactory::getInstance() ->haveRunningTransformers(), "the race was not set up: no analysis was running when " "the session was about to be closed"); + QVERIFY(m_window->analysingRange()); m_window->doCloseSession(); + m_window->releaseTakeAnalysis(); QVERIFY(!m_window->analyser2()); QCOMPARE(m_window->paneStack()->getPaneCount(), 0); diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index b6968bb3..0b9f78b5 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -1156,14 +1156,18 @@ private slots: QTest::qWait(600); // Stop, and the moment after it: the recorded range is being - // analysed, and erasing would throw that away + // analysed, and erasing would throw that away. Held, so that the + // moment is there however quick the machine + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(m_window->analysingRange()); QVERIFY2(!erase->isEnabled(), "Erase can be used while the take is being analysed"); - // ... and everything back, by itself + // ... and everything back, by itself: the release does nothing + // but let the merge happen + m_window->releaseTakeAnalysis(); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); QTRY_VERIFY2(erase->isEnabled(), "Erase did not come back after the analysis"); @@ -1417,7 +1421,10 @@ private slots: } // Checklist: stop a take and close the window at once: no crash. - // Closed and deleted while the take is still being analysed + // Closed and deleted while the take is still being analysed. The + // analysis is held, so that its run is there, unmerged, when the + // window goes, however quick the machine; whether pYIN's thread is + // still going then as well depends on the machine and its load void stop_then_close_the_window_at_once() { FakeAudioIO::Config config; config.input = tone(highHz, 4.0); @@ -1429,6 +1436,7 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(1000); + m_window->holdTakeAnalysis(); m_window->doRecord(); QVERIFY(m_window->analysingRange()); From e5f3354795f0fe21ed4a7fff419ad63de255a9d9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:40:05 +0000 Subject: [PATCH 178/275] test: svcore with its reader test reading nothing before a decode starts svcore's AudioFileReaderTest compared the first reference frames of a non-gapless read with memory before the decoded samples when the offset it found was negative, as it is for the 32000 hz mp3 read at 48000 on macos, which failed there with one bad sample at frame 0. the fork's f9f3330 skips those frames and names the offset in its errors. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- repoint-lock.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoint-lock.json b/repoint-lock.json index 5869abea..22930b2e 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -4,7 +4,7 @@ "pin": "d7ceb7d1d490674c93d334e5378108c4328e9e05" }, "svcore": { - "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" + "pin": "f9f333084b608a2c5b31ea7522399be10f959ba2" }, "svgui": { "pin": "a34646ac8888690724d890063789b123f7921885" From 845134469b00cb6b84aa79c5d295741c33b22c90 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:45:23 +0000 Subject: [PATCH 179/275] fix: picking files on android: parentheses, drive, downloads, menus qt's content file engine re-encodes '(' and ')' in a picked uri and then has no grant for it, so a session named "(vocals) ..." was reported missing; tony now opens picked files through its own picker and a descriptor for the uri exactly as android wrote it. the picker offers every file, so drive's are no longer disabled by a type filter, and the extension is checked after. downloads and the picker's recent and audio views are found through mediastore with all files access. every refusal names the file's provider, the path looked for and why; open recent says a moved file is gone. menus keep clear of the screen's edges and system bars. tony keeps its log in a file too, which help > save log writes out for a user without adb. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/AndroidFiles.cpp | 201 +++++++++++++++++++-- main/AndroidFiles.h | 88 +++++++++- main/AndroidStorage.cpp | 308 ++++++++++++++++++++++++++++++++- main/AndroidStorage.h | 30 +++- main/LogFile.cpp | 78 +++++++++ main/LogFile.h | 61 +++++++ main/MainWindow.cpp | 292 ++++++++++++++++++++++++++----- main/MainWindow.h | 19 +- main/PopupArea.cpp | 79 +++++++++ main/PopupArea.h | 79 +++++++++ main/TouchMenuStyle.cpp | 101 ++++++++++- main/TouchMenuStyle.h | 23 +++ main/main.cpp | 70 +++++++- main/test/TestAndroidFiles.h | 258 ++++++++++++++++++++++++++- main/test/TestLogFile.h | 121 +++++++++++++ main/test/TestPopupArea.h | 147 ++++++++++++++++ main/test/TestTouchMenuStyle.h | 124 ++++++++++++- main/test/tony-core-test.cpp | 14 ++ meson.build | 4 + 19 files changed, 2011 insertions(+), 86 deletions(-) create mode 100644 main/LogFile.cpp create mode 100644 main/LogFile.h create mode 100644 main/PopupArea.cpp create mode 100644 main/PopupArea.h create mode 100644 main/test/TestLogFile.h create mode 100644 main/test/TestPopupArea.h diff --git a/main/AndroidFiles.cpp b/main/AndroidFiles.cpp index c4acec08..d4c5cdad 100644 --- a/main/AndroidFiles.cpp +++ b/main/AndroidFiles.cpp @@ -115,6 +115,18 @@ AndroidFiles::checkVampPlugin(QString path) QString AndroidFiles::copyIn(QString source, QString name, QString dir, QString &error) +{ + QFile in(source); + if (!in.open(QIODevice::ReadOnly)) { + error = QString("cannot read it: %1").arg(in.errorString()); + return ""; + } + return copyIn(in, source, name, dir, error); +} + +QString +AndroidFiles::copyIn(QIODevice &in, QString sourceName, QString name, + QString dir, QString &error) { if (!QDir().mkpath(dir)) { error = QString("cannot create the folder %1").arg(dir); @@ -123,12 +135,6 @@ AndroidFiles::copyIn(QString source, QString name, QString dir, QString target = QDir(dir).filePath(safeFileName(name)); - QFile in(source); - if (!in.open(QIODevice::ReadOnly)) { - error = QString("cannot read it: %1").arg(in.errorString()); - return ""; - } - // QSaveFile writes a file of its own and renames it over the target // at the end, so a copy that fails half-way leaves the earlier one QSaveFile out(target); @@ -165,8 +171,8 @@ AndroidFiles::copyIn(QString source, QString name, QString dir, return ""; } - SVCERR << "AndroidFiles: copied " << total << " bytes from " << source - << " to " << target << endl; + SVCERR << "AndroidFiles: copied " << total << " bytes from " + << sourceName << " to " << target << endl; return target; } @@ -203,11 +209,20 @@ pathUnder(QString root, QString relative) return QDir::cleanPath(root + "/" + relative); } -QString -AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) +static const QString externalStorageProvider +("com.android.externalstorage.documents"); +static const QString downloadsProvider +("com.android.providers.downloads.documents"); +static const QString mediaProvider +("com.android.providers.media.documents"); + +// The authority of a content:// URI and the parts of its path, decoded; +// false if it is not content:// +static bool +splitContentUri(QString uri, QString &authority, QStringList &parts) { const QString scheme("content://"); - if (!uri.startsWith(scheme, Qt::CaseInsensitive)) return ""; + if (!uri.startsWith(scheme, Qt::CaseInsensitive)) return false; // Nothing the picker gives has a query or a fragment QString rest = uri.mid(scheme.size()); @@ -218,23 +233,67 @@ AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) // Split before decoding: a '/' inside the id is %2F in every form of // the URI, Android's and QUrl's, while spaces and letters such as 'ä' // may come either way - QStringList parts = rest.split('/'); - QString authority = parts.takeFirst(); + parts = rest.split('/'); + authority = parts.takeFirst(); for (QString &part : parts) { part = QUrl::fromPercentEncoding(part.toUtf8()); } + return true; +} + +// The authority and the decoded document id of a content:// URI that +// names a document, alone or under a folder grant; false for any other +static bool +splitDocumentUri(QString uri, QString &authority, QString &id) +{ + QStringList parts; + if (!splitContentUri(uri, authority, parts)) return false; - QString id; if (parts.size() == 2 && parts[0] == "document") { id = parts[1]; } else if (parts.size() == 4 && parts[0] == "tree" && parts[2] == "document") { id = parts[3]; } else { - return ""; + return false; } + return true; +} + +// Whether id is "" +static bool +isNumberedId(QString id, QString prefix) +{ + static const QRegularExpression number("^[0-9]+$"); + return id.startsWith(prefix) && + number.match(id.mid(prefix.size())).hasMatch(); +} + +QString +AndroidFiles::grantedUri(const QUrl &picked) +{ + // QUrl keeps the encoding it was given of everything but the + // characters no one encodes (letters, digits, "-._~"), which Android + // does not encode either: so this is the string Android wrote + return picked.toString(QUrl::FullyEncoded); +} + +QString +AndroidFiles::providerOf(QString uri) +{ + QString authority; + QStringList parts; + if (!splitContentUri(uri, authority, parts)) return ""; + return authority; +} + +QString +AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) +{ + QString authority, id; + if (!splitDocumentUri(uri, authority, id)) return ""; - if (authority == "com.android.externalstorage.documents") { + if (authority == externalStorageProvider) { int colon = id.indexOf(':'); if (colon <= 0) return ""; @@ -255,7 +314,7 @@ AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) return ""; } - if (authority == "com.android.providers.downloads.documents") { + if (authority == downloadsProvider) { // Only these carry a path; the rest are numbers in a database const QString raw("raw:/"); if (!id.startsWith(raw)) return ""; @@ -265,6 +324,114 @@ AndroidFiles::pathFromContentUri(QString uri, QString primaryRoot) return ""; } +AndroidFiles::PathLookup +AndroidFiles::pathLookupFor(QString uri) +{ + QString authority, id; + if (!splitDocumentUri(uri, authority, id)) return PathLookup::None; + + if (authority == externalStorageProvider) { + // A volume and a path: pathFromContentUri() says which it takes + return (id.indexOf(':') > 0 ? PathLookup::InUri : PathLookup::None); + } + + if (authority == downloadsProvider) { + if (id.startsWith("raw:/")) return PathLookup::InUri; + if (isNumberedId(id, "msf:")) return PathLookup::MediaStore; + if (isNumberedId(id, "")) return PathLookup::ByNameAndSize; + return PathLookup::None; // "msd:", a folder, and the rest + } + + if (authority == mediaProvider && mediaStoreUriFor(uri) != "") { + return PathLookup::MediaStore; + } + + return PathLookup::None; +} + +QString +AndroidFiles::mediaStoreUriFor(QString uri) +{ + QString authority, id; + if (!splitDocumentUri(uri, authority, id)) return ""; + + // MediaStore..getContentUri("external", n): "external" is + // every volume. The providers' own ids, from DownloadStorageProvider + // (MediaStoreDownloadsHelper) and MediaDocumentsProvider + // (getUriForDocumentId()) + const QString media("content://media/external/"); + + if (authority == downloadsProvider) { + if (isNumberedId(id, "msf:")) { + return media + "downloads/" + id.mid(4); + } + return ""; + } + + if (authority == mediaProvider) { + const QStringList types { "audio", "image", "video", "document" }; + const QStringList collections { + "audio/media", "images/media", "video/media", "file" + }; + for (int i = 0; i < types.size(); ++i) { + QString prefix = types[i] + ":"; + if (isNumberedId(id, prefix)) { + return media + collections[i] + "/" + id.mid(prefix.size()); + } + } + } + + return ""; +} + +QString +AndroidFiles::chooseDownload(QStringList candidates, QString primaryRoot) +{ + candidates.removeAll(QString()); + candidates.removeDuplicates(); + if (candidates.size() == 1) return candidates[0]; + + // The download manager saves into the Download folder: the one of + // several that lies there directly, if only one does. (Android's + // paths, as strings: QFileInfo would give them a drive on Windows) + if (primaryRoot == "") return ""; + QString downloads = QDir::cleanPath(primaryRoot + "/Download"); + QString chosen; + for (QString path : candidates) { + QString clean = QDir::cleanPath(path); + if (clean.left(clean.lastIndexOf('/')) == downloads) { + if (chosen != "") return ""; + chosen = path; + } + } + return chosen; +} + +bool +AndroidFiles::hasExtensionIn(QString name, QString patterns) +{ + QString suffix = QFileInfo(name).suffix().toLower(); + if (suffix == "") return false; + for (QString pattern : patterns.split(' ', Qt::SkipEmptyParts)) { + if (pattern.toLower() == "*." + suffix) return true; + } + return false; +} + +QStringList +AndroidFiles::usableRecentFiles(QStringList identifiers) +{ + // Two letters at least: "C:" is a Windows drive + static const QRegularExpression scheme("^[a-zA-Z][a-zA-Z0-9+.-]+:"); + QStringList usable; + for (QString identifier : identifiers) { + if (scheme.match(identifier).hasMatch()) continue; + if (!QFileInfo(identifier).isFile()) continue; + usable << identifier; + } + return usable; +} + QString AndroidFiles::suggestedSessionName(QString sessionPath, QString audioPath) { diff --git a/main/AndroidFiles.h b/main/AndroidFiles.h index ec7c2326..9e5884e8 100644 --- a/main/AndroidFiles.h +++ b/main/AndroidFiles.h @@ -18,6 +18,9 @@ #include #include +class QIODevice; +class QUrl; + /** * File work that only the Android build calls: Android hands Tony * neither its Vamp plugins under their own names nor the files the user @@ -66,6 +69,14 @@ class AndroidFiles static QString copyIn(QString source, QString name, QString dir, QString &error); + /** + * The same, from in, open for reading: on Android a file descriptor + * the file's provider gave (AndroidStorage::openDocument()), which + * may be a pipe. sourceName says where it came from, for the log. + */ + static QString copyIn(QIODevice &in, QString sourceName, QString name, + QString dir, QString &error); + /** * name as a file name that can be used in any directory: path * separators and the characters Windows refuses become '_', and a @@ -73,11 +84,31 @@ class AndroidFiles */ static QString safeFileName(QString name); + /** + * The URI of a file picked in Android's picker, as the string Android + * wrote: Android lets Tony read that document through that exact + * string only (its grants are matched by string), and Qt's + * QFileDialog::selectedFiles() gives it partly decoded (spaces and + * letters such as 'ä' unencoded, "%3A" and "%2F" kept). Qt's own + * content file engine turns that back into a URI with '(' and ')' + * (and "!'*") encoded, which Android leaves as they are, so for a + * name holding any of those it finds no grant and reports the file + * missing. picked is the QUrl the picker gave (selectedUrls()). + */ + static QString grantedUri(const QUrl &picked); + + /** + * The authority of a content:// URI: which app's provider the file + * comes from, such as com.google.android.apps.docs.storage (Google + * Drive); "" if uri is not content://. + */ + static QString providerOf(QString uri); + /** * The real path of the file a content:// URI from Android's file - * picker names, when it lies in the phone's own storage; "" for any - * other URI (a cloud provider's, the media provider's, a download - * known only by number), which has no path Tony could use. + * picker names, when the URI itself says it; "" for any other URI (a + * cloud provider's, and the documents whose path is looked up in + * MediaStore: see pathLookupFor()). * * The external storage provider's documents, alone or under a folder * grant (.../document/ and .../tree//document/), have ids @@ -90,6 +121,57 @@ class AndroidFiles */ static QString pathFromContentUri(QString uri, QString primaryRoot); + /** + * How the real path of a picked document can be found, if it is in + * the phone's own storage. Only InUri needs nothing more than the + * URI; the rest need All files access, without which MediaStore + * shows Tony none of the files other apps put there. + */ + enum class PathLookup { + None, // A cloud provider's, or not a file: no path + InUri, // pathFromContentUri() + MediaStore, // The _data column of mediaStoreUriFor()'s row + ByNameAndSize // A numbered download: see chooseDownload() + }; + static PathLookup pathLookupFor(QString uri); + + /** + * The MediaStore row (content://media/external/...) whose _data + * column holds the path of the file behind uri, for the downloads + * provider's "msf:" (a download MediaStore knows) and the media + * provider's "audio:", "image:", "video:" and + * "document:" (the picker's Audio, Images, Videos and Documents, + * and much of its Recent); "" for any other uri. + */ + static QString mediaStoreUriFor(QString uri); + + /** + * The downloads provider names a file the download manager fetched + * by its number there (""), and the download manager shows other + * apps none of its records. Such a file is found in MediaStore, where + * the download manager puts every download, by the name and size the + * provider gives: candidates are the paths of the files of that name + * and size. Returns the only one, or else the only one in the + * Download folder of primaryRoot; "" if that does not settle it. + */ + static QString chooseDownload(QStringList candidates, QString primaryRoot); + + /** + * Whether name has one of the extensions in patterns, a list such as + * svcore's getKnownExtensions() give ("*.wav *.mp3"), in any case. + * The picker offers every file, and Android's file types cannot say + * which are Tony's (see MainWindow::getOpenFileName()). + */ + static bool hasExtensionIn(QString name, QString patterns); + + /** + * The entries of the recent files list that can be opened: paths of + * files that are there. A file moved or deleted since, and anything + * that is not a path (a content:// URI, which the picker's grant no + * longer covers), is left out. + */ + static QStringList usableRecentFiles(QStringList identifiers); + /** * The name Save Session As suggests on Android, where the system's * picker suggests none of its own: the session's name if it has a diff --git a/main/AndroidStorage.cpp b/main/AndroidStorage.cpp index 3893f1c4..7be6c6a3 100644 --- a/main/AndroidStorage.cpp +++ b/main/AndroidStorage.cpp @@ -22,6 +22,7 @@ #include #include #include +#include #include #include @@ -56,10 +57,313 @@ AndroidStorage::primaryRoot() ("getAbsolutePath", "()Ljava/lang/String;").toString(); } +// ContentResolver calls are made through JNI directly rather than through +// QJniObject, which clears an exception a call throws (a +// SecurityException for a URI without a grant, say) and tells only the +// system log: here it is caught and said, for the user's message and +// Tony's own log +namespace { + +// The exception the last call left, as Java names it, cleared; "" if +// there is none +QString +takeException(QJniEnvironment &env) +{ + if (!env->ExceptionCheck()) return ""; + jthrowable thrown = env->ExceptionOccurred(); + env->ExceptionClear(); + QString text("an unknown exception"); + if (thrown) { + jclass thrownClass = env->GetObjectClass(thrown); + jmethodID toString = env->GetMethodID + (thrownClass, "toString", "()Ljava/lang/String;"); + jobject description = + (toString ? env->CallObjectMethod(thrown, toString) : nullptr); + if (env->ExceptionCheck()) env->ExceptionClear(); + if (description) { + text = QJniObject::fromLocalRef(description).toString(); + } + env->DeleteLocalRef(thrownClass); + env->DeleteLocalRef(thrown); + } + return text; +} + +QJniObject +contentResolver() +{ + QJniObject context = QNativeInterface::QAndroidApplication::context(); + if (!context.isValid()) return QJniObject(); + return context.callObjectMethod + ("getContentResolver", "()Landroid/content/ContentResolver;"); +} + +// Uri.parse(): the same string, to the character, so that a grant for it +// is found +QJniObject +parseUri(QString uri) +{ + return QJniObject::callStaticObjectMethod + ("android/net/Uri", "parse", "(Ljava/lang/String;)Landroid/net/Uri;", + QJniObject::fromString(uri).object()); +} + +jobjectArray +stringArray(QJniEnvironment &env, QStringList strings) +{ + jclass stringClass = env->FindClass("java/lang/String"); + jobjectArray array = env->NewObjectArray(jsize(strings.size()), + stringClass, nullptr); + for (int i = 0; i < strings.size(); ++i) { + QJniObject s = QJniObject::fromString(strings[i]); + env->SetObjectArrayElement(array, jsize(i), s.object()); + } + env->DeleteLocalRef(stringClass); + return array; +} + +// ContentResolver.query(): each row the values of columns in order, as +// strings ("" for none). Empty, with error saying why, if it fails +QList +query(QString uri, QStringList columns, QString selection, + QStringList arguments, QString &error) +{ + QList rows; + + QJniObject resolver = contentResolver(); + QJniObject parsed = parseUri(uri); + if (!resolver.isValid() || !parsed.isValid()) { + error = "no content resolver"; + return rows; + } + + QJniEnvironment env; + jclass resolverClass = env->GetObjectClass(resolver.object()); + jmethodID queryMethod = env->GetMethodID + (resolverClass, "query", + "(Landroid/net/Uri;[Ljava/lang/String;Ljava/lang/String;" + "[Ljava/lang/String;Ljava/lang/String;)Landroid/database/Cursor;"); + env->DeleteLocalRef(resolverClass); + if (!queryMethod) { + error = takeException(env); + return rows; + } + + jobjectArray projection = stringArray(env, columns); + jobjectArray selectionArgs = + (arguments.empty() ? nullptr : stringArray(env, arguments)); + QJniObject selectionString; + if (selection != "") selectionString = QJniObject::fromString(selection); + + jobject cursor = env->CallObjectMethod + (resolver.object(), queryMethod, parsed.object(), projection, + selectionString.isValid() ? selectionString.object() : nullptr, + selectionArgs, nullptr); + QString thrown = takeException(env); + + env->DeleteLocalRef(projection); + if (selectionArgs) env->DeleteLocalRef(selectionArgs); + + if (thrown != "") { + error = thrown; + return rows; + } + if (!cursor) { + error = "the provider answered nothing"; + return rows; + } + + QJniObject c = QJniObject::fromLocalRef(cursor); + while (c.callMethod("moveToNext", "()Z")) { + QStringList row; + for (int i = 0; i < columns.size(); ++i) { + row << c.callObjectMethod("getString", "(I)Ljava/lang/String;", + jint(i)).toString(); + } + rows << row; + } + c.callMethod("close", "()V"); + return rows; +} + +} + +QString +AndroidStorage::pathFor(QString uri, QString &why) +{ + QString app = QApplication::applicationName(); + QString error; + + switch (AndroidFiles::pathLookupFor(uri)) { + + case AndroidFiles::PathLookup::None: + why = tr("its provider gives no path for it"); + return ""; + + case AndroidFiles::PathLookup::InUri: { + QString path = AndroidFiles::pathFromContentUri(uri, primaryRoot()); + if (path == "") why = tr("its path could not be read from its URI"); + return path; + } + + case AndroidFiles::PathLookup::MediaStore: { + if (!hasAllFilesAccess()) { + why = tr("%1 has no All files access, without which it cannot " + "look up where the file is").arg(app); + return ""; + } + QString row = AndroidFiles::mediaStoreUriFor(uri); + QList rows = query(row, { "_data" }, "", {}, error); + if (rows.size() == 1 && rows[0].size() == 1 && rows[0][0] != "") { + return rows[0][0]; + } + why = (error != "" ? + tr("MediaStore could not be asked for %1: %2").arg(row, error) : + tr("MediaStore has no path for %1").arg(row)); + return ""; + } + + case AndroidFiles::PathLookup::ByNameAndSize: { + if (!hasAllFilesAccess()) { + why = tr("%1 has no All files access, without which it cannot " + "look up where the file is").arg(app); + return ""; + } + // The name and size the downloads provider gives, through the + // picker's grant; then the files of that name and size + QList document = + query(uri, { "_display_name", "_size" }, "", {}, error); + if (document.size() != 1 || document[0].size() != 2 || + document[0][0] == "") { + why = (error != "" ? + tr("its provider could not be asked for its name: %1") + .arg(error) : + tr("its provider gives no name for it")); + return ""; + } + QString name = document[0][0]; + QString size = document[0][1]; + QList files = + query("content://media/external/file", { "_data" }, + "_display_name = ? AND _size = ?", { name, size }, error); + QStringList candidates; + for (const QStringList &file : files) { + if (!file.empty()) candidates << file[0]; + } + QString path = AndroidFiles::chooseDownload(candidates, primaryRoot()); + if (path == "") { + why = (error != "" ? + tr("MediaStore could not be asked for it: %1").arg(error) : + tr("MediaStore has %1 files named \"%2\" of %3 bytes, " + "not one").arg(QString::number(candidates.size()), + name, size)); + } + return path; + } + } + + return ""; +} + +QString +AndroidStorage::displayName(QString uri) +{ + QString error; + QList rows = query(uri, { "_display_name" }, "", {}, error); + if (rows.size() == 1 && rows[0].size() == 1) return rows[0][0]; + if (error != "") { + cerr << "AndroidStorage: no name for " << uri.toStdString() << ": " + << error.toStdString() << endl; + } + return ""; +} + +int +AndroidStorage::openDocument(QString uri, QString mode, QString &error) +{ + QJniObject resolver = contentResolver(); + QJniObject parsed = parseUri(uri); + if (!resolver.isValid() || !parsed.isValid()) { + error = "no content resolver"; + return -1; + } + + QJniEnvironment env; + jclass resolverClass = env->GetObjectClass(resolver.object()); + jmethodID open = env->GetMethodID + (resolverClass, "openFileDescriptor", + "(Landroid/net/Uri;Ljava/lang/String;)" + "Landroid/os/ParcelFileDescriptor;"); + env->DeleteLocalRef(resolverClass); + if (!open) { + error = takeException(env); + return -1; + } + + QJniObject modeString = QJniObject::fromString(mode); + jobject descriptor = env->CallObjectMethod + (resolver.object(), open, parsed.object(), modeString.object()); + QString thrown = takeException(env); + if (thrown != "") { + error = thrown; + return -1; + } + if (!descriptor) { + error = "the provider gave nothing to read"; + return -1; + } + + // Ours to close from here + QJniObject pfd = QJniObject::fromLocalRef(descriptor); + int fd = pfd.callMethod("detachFd", "()I"); + if (fd < 0) error = "the provider gave no file descriptor"; + return fd; +} + +bool +AndroidStorage::removeIfEmpty(QString uri) +{ + QString error; + QList rows = query(uri, { "_size" }, "", {}, error); + if (rows.size() != 1 || rows[0].size() != 1 || rows[0][0] != "0") { + return false; + } + + QJniObject resolver = contentResolver(); + QJniObject parsed = parseUri(uri); + if (!resolver.isValid() || !parsed.isValid()) return false; + + QJniEnvironment env; + jclass contract = env->FindClass("android/provider/DocumentsContract"); + jmethodID remove = (contract ? env->GetStaticMethodID + (contract, "deleteDocument", + "(Landroid/content/ContentResolver;" + "Landroid/net/Uri;)Z") : nullptr); + bool removed = false; + if (remove) { + removed = env->CallStaticBooleanMethod + (contract, remove, resolver.object(), parsed.object()); + } + QString thrown = takeException(env); + if (contract) env->DeleteLocalRef(contract); + + if (thrown != "") { + cerr << "AndroidStorage: could not remove the empty " + << uri.toStdString() << ": " << thrown.toStdString() << endl; + return false; + } + if (removed) { + cerr << "AndroidStorage: removed the empty " << uri.toStdString() + << endl; + } + return removed; +} + QString -AndroidStorage::pathFor(QString uri) const +AndroidStorage::logPath() { - return AndroidFiles::pathFromContentUri(uri, primaryRoot()); + return QStandardPaths::writableLocation(QStandardPaths::AppDataLocation) + + "/log/tony.log"; } bool diff --git a/main/AndroidStorage.h b/main/AndroidStorage.h index 633ce855..dfca1f79 100644 --- a/main/AndroidStorage.h +++ b/main/AndroidStorage.h @@ -44,9 +44,33 @@ class AndroidStorage // /storage/emulated/0; "" if Android does not say static QString primaryRoot(); - // The real path of a content:// URI the file picker gave, if it names - // a file in the phone's own storage; else "" - QString pathFor(QString uri) const; + // The real path of the file a content:// URI the file picker gave + // names, if it is in the phone's own storage: from the URI, or from + // MediaStore, which shows Tony other apps' files only with All files + // access (AndroidFiles::pathLookupFor()). Else "", with why saying + // why not. Whether the file is there is not checked. The URI as + // Android wrote it (AndroidFiles::grantedUri()): the picker's grant + // is for that string only + static QString pathFor(QString uri, QString &why); + + // The name the file's provider gives the document at uri; "" if it + // gives none + static QString displayName(QString uri); + + // A file descriptor for the document at uri, open to read ("r") or + // write ("w"), which the caller closes (QFile's AutoCloseHandle); -1 + // on failure, with error saying why. Called here rather than through + // QFile, whose content file engine rebuilds the URI, differently for + // names with parentheses, and then has no grant for it + static int openDocument(QString uri, QString mode, QString &error); + + // Removes the document at uri if it is empty: the one the picker + // makes for a save, when the save is not made there. True if it did + static bool removeIfEmpty(QString uri); + + // The file Tony's output is kept in as well as the system log + // (main.cpp), which Help > Save Log... saves a copy of (LogFile) + static QString logPath(); // Asks for All files access: why, in a box, and then the system's // settings page for it; back from there, checks again. True if Tony diff --git a/main/LogFile.cpp b/main/LogFile.cpp new file mode 100644 index 00000000..dd599f6c --- /dev/null +++ b/main/LogFile.cpp @@ -0,0 +1,78 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LogFile.h" + +#include +#include + +LogFile::LogFile(QString path, qint64 limit) : + m_path(path), + m_limit(limit) +{ +} + +QString +LogFile::olderPath(QString path) +{ + return path + ".1"; +} + +bool +LogFile::begin() +{ + // What is there now becomes the older part, the older part goes + if (QFileInfo::exists(m_path)) { + QFile::remove(olderPath(m_path)); + QFile::rename(m_path, olderPath(m_path)); + } + + m_file.setFileName(m_path); + return m_file.open(QIODevice::WriteOnly | QIODevice::Truncate); +} + +bool +LogFile::open() +{ + QDir().mkpath(QFileInfo(m_path).absolutePath()); + return begin(); +} + +void +LogFile::write(const QByteArray &line) +{ + if (!m_file.isOpen()) return; + + m_file.write(line); + m_file.write("\n", 1); + m_file.flush(); + + if (m_file.size() >= m_limit) { + m_file.close(); + begin(); + } +} + +QByteArray +LogFile::contents(QString path) +{ + QByteArray all; + for (QString part : { olderPath(path), path }) { + QFile file(part); + if (file.open(QIODevice::ReadOnly)) { + all += file.readAll(); + } + } + return all; +} diff --git a/main/LogFile.h b/main/LogFile.h new file mode 100644 index 00000000..216bf5d5 --- /dev/null +++ b/main/LogFile.h @@ -0,0 +1,61 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LOG_FILE_H +#define TONY_LOG_FILE_H + +#include +#include +#include + +/** + * A log file that stays small. On Android Tony's output goes to the + * system log, which no one reads without a computer and adb; a copy + * kept here can be saved from Tony (Help > Save Log...) and sent. + * + * When opened, the file an earlier run left becomes .1, in place + * of an older one, and whenever the file grows past limit bytes it goes + * the same way and a new one is begun: the two hold the last limit to + * twice limit bytes, the end of the run before included when this one's + * is short. One thread writes; it is not locked. + */ +class LogFile +{ +public: + LogFile(QString path, qint64 limit); + + // Moves an earlier run's file aside and begins a new one; false if + // it cannot be written + bool open(); + + bool isOpen() const { return m_file.isOpen(); } + + // Appends line and a newline, at once, in case Tony ends soon after + void write(const QByteArray &line); + + // .1, the older part + static QString olderPath(QString path); + + // The older part and the newer, in that order: the log to be sent + static QByteArray contents(QString path); + +private: + QString m_path; + qint64 m_limit; + QFile m_file; + + bool begin(); +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 1b18a580..7a072ca4 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -29,12 +29,16 @@ #ifdef Q_OS_ANDROID #include "AndroidFiles.h" #include "AndroidStorage.h" +#include "LogFile.h" #include "OboeAudioIO.h" +#include "data/fileio/AudioFileReaderFactory.h" +#include #include #include #include #include #include +#include #endif #include "framework/Document.h" @@ -1324,10 +1328,20 @@ MainWindow::setupHelpMenu() connect(action, SIGNAL(triggered()), this, SLOT(whatsNew())); menu->addAction(action); - action = new QAction(tr("&About %1").arg(name), this); - action->setStatusTip(tr("Show information about %1").arg(name)); + action = new QAction(tr("&About %1").arg(name), this); + action->setStatusTip(tr("Show information about %1").arg(name)); connect(action, SIGNAL(triggered()), this, SLOT(about())); menu->addAction(action); + +#ifdef Q_OS_ANDROID + // The phone's system log needs a computer to read; this copy of what + // Tony wrote to it can be sent instead + menu->addSeparator(); + action = new QAction(tr("Save &Log..."), this); + action->setStatusTip(tr("Save a copy of %1's log, to send when something went wrong").arg(name)); + connect(action, &QAction::triggered, this, [this]() { saveLog(); }); + menu->addAction(action); +#endif } void @@ -1335,6 +1349,13 @@ MainWindow::setupRecentFilesMenu() { m_recentFilesMenu->clear(); vector files = m_recentFiles.getRecent(); +#ifdef Q_OS_ANDROID + // Only the files that are still where they were: nothing else could + // be opened from here, a content:// URI whose grant has gone included + QStringList usable = AndroidFiles::usableRecentFiles + (QStringList(files.begin(), files.end())); + files = vector(usable.begin(), usable.end()); +#endif for (size_t i = 0; i < files.size(); ++i) { QString path = files[i]; QAction *action = m_recentFilesMenu->addAction(path); @@ -2755,80 +2776,243 @@ MainWindow::closeSession() QString MainWindow::getOpenFileName(FileFinder::FileType type) { - QString path = MainWindowBase::getOpenFileName(type); - if (!path.startsWith("content:")) return path; + // Layers and other data go through svgui's dialog as before + if (type != FileFinder::AudioFile && + type != FileFinder::SessionOrAudioFile && + type != FileFinder::SessionFile) { + return MainWindowBase::getOpenFileName(type); + } QString app = QApplication::applicationName(); - // The name the user knows the file by, which Qt asks the file's - // provider for: the URI need not contain it - QString name = QFileInfo(path).fileName(); + // Every file is offered: a provider disables the files whose type is + // not among those asked for, and Android knows no type for .ton, nor + // Qt one Drive gives every audio file. What was picked is checked + // below instead + QFileDialog dialog(this, (type == FileFinder::AudioFile ? + tr("Select an audio file") : + tr("Select a session or audio file"))); + dialog.setAcceptMode(QFileDialog::AcceptOpen); + dialog.setFileMode(QFileDialog::ExistingFile); + + if (!dialog.exec()) return ""; + QList urls = dialog.selectedUrls(); + if (urls.empty() || urls[0].isEmpty()) return ""; + if (urls[0].isLocalFile()) return urls[0].toLocalFile(); + + QString uri = AndroidFiles::grantedUri(urls[0]); + + // The name the user knows the file by, which the file's provider + // gives: the URI need not contain it + QString name = AndroidStorage::displayName(uri); + if (name == "") name = urls[0].fileName(); + cerr << "MainWindow::getOpenFileName: picked " << uri << ", \"" + << name << "\"" << endl; + bool session = (QFileInfo(name).suffix().toLower() == "ton"); + bool audio = AndroidFiles::hasExtensionIn + (name, AudioFileReaderFactory::getKnownExtensions()); + if (!audio && (!session || type == FileFinder::AudioFile)) { + QMessageBox::warning + (this, tr("Cannot open the file"), + (type == FileFinder::AudioFile ? + tr("\"%1\" is not an audio file %2 can open

%2 opens audio files such as WAV and MP3.

") : + tr("\"%1\" is not a file %2 can open

%2 opens sessions (.ton) and audio files such as WAV and MP3.

")) + .arg(name.toHtmlEscaped(), app) + pickDetails(uri, "", "")); + return ""; + } // A file in the phone's own storage has a path, and with All files // access that is what is opened, as on the desktop: a session finds // its audio and takes folder beside it, and a session saved beside // audio opened so finds the audio. Asked for with a session, which // cannot do without it, and the first time with audio, which can - QString local = m_storage->pathFor(path); - if (local != "") { - if (!AndroidStorage::hasAllFilesAccess() && - (session || !m_storage->hasAsked())) { - m_storage->ask - (session ? - tr("A session keeps its audio and its takes folder beside " - "it, and %1 opens and saves it there, where it is.") - .arg(app) : - tr("Audio opened where it is can have its session saved " - "beside it. Otherwise %1 copies the audio into its own " - "storage.").arg(app)); - } - if (AndroidStorage::hasAllFilesAccess()) { - if (QFileInfo(local).isFile()) { - cerr << "MainWindow::getOpenFileName: opening " << path - << " where it is, " << local << endl; - return local; - } - cerr << "MainWindow::getOpenFileName: " << path << " should be " - << local << ", which is not there" << endl; + bool hasPath = + (AndroidFiles::pathLookupFor(uri) != AndroidFiles::PathLookup::None); + if (hasPath && !AndroidStorage::hasAllFilesAccess() && + (session || !m_storage->hasAsked())) { + m_storage->ask + (session ? + tr("A session keeps its audio and its takes folder beside " + "it, and %1 opens and saves it there, where it is.") + .arg(app) : + tr("Audio opened where it is can have its session saved " + "beside it. Otherwise %1 copies the audio into its own " + "storage.").arg(app)); + } + + QString local, why; + if (!hasPath) { + why = tr("its provider gives no path for it"); + } else if (!AndroidStorage::hasAllFilesAccess()) { + why = tr("%1 has no All files access").arg(app); + } else { + local = AndroidStorage::pathFor(uri, why); + if (local != "" && QFileInfo(local).isFile()) { + cerr << "MainWindow::getOpenFileName: opening " << uri + << " where it is, " << local << endl; + return local; } + if (local != "") why = tr("there is no file at that path"); } + cerr << "MainWindow::getOpenFileName: no path for " << uri << ": " + << why << endl; // A session read through the picker comes without the audio and // takes beside it: the picker lets Tony read the one file only if (session) { - if (local == "") { + if (!hasPath) { QMessageBox::warning (this, tr("Cannot open the session"), - tr("The session cannot be opened from here

A session needs its audio and its takes folder beside it, and from here (Downloads, or a cloud app such as Drive) %1 is given the one file only.

Keep sessions in a folder of the phone's own storage, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer, and open them by browsing to that folder in the picker.

") - .arg(app)); + tr("The session cannot be opened from here

A session needs its audio and its takes folder beside it, and from here (a cloud app such as Drive) %1 is given the one file only.

Keep sessions in a folder of the phone's own storage, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer, and open them by browsing to that folder in the picker.

") + .arg(app) + pickDetails(uri, local, why)); } else if (!AndroidStorage::hasAllFilesAccess()) { QMessageBox::warning (this, tr("Cannot open the session"), tr("%1 may not open the session where it is

A session needs its audio and its takes folder beside it, and %1 can read those only with All files access. Allow it when %1 asks, or in the phone's Settings, Apps, %1.

") - .arg(app)); + .arg(app) + pickDetails(uri, local, why)); } else { QMessageBox::warning (this, tr("Cannot open the session"), - tr("The session was not found where it should be

%1 looked for it at \"%2\".

") - .arg(app, local.toHtmlEscaped())); + tr("The session was not found where it should be") + + pickDetails(uri, local, why)); } return ""; } + // Audio without a path is read through the picker's grant, from a + // file descriptor the provider opens: Qt's own content file engine + // rebuilds the URI, differently for a name with parentheses, and is + // then refused QString dir = QStandardPaths::writableLocation (QStandardPaths::AppDataLocation) + "/imported"; QString error; - QString copy = AndroidFiles::copyIn(path, name, dir, error); + QString copy; + int fd = AndroidStorage::openDocument(uri, "r", error); + if (fd >= 0) { + QFile in; + if (in.open(fd, QIODevice::ReadOnly, QFileDevice::AutoCloseHandle)) { + copy = AndroidFiles::copyIn(in, uri, name, dir, error); + } else { + ::close(fd); + error = in.errorString(); + } + } if (copy == "") { QMessageBox::critical (this, tr("Failed to open file"), - tr("File open failed

\"%1\" could not be copied into %2's own storage: %3") - .arg(name.toHtmlEscaped(), app, error.toHtmlEscaped())); + tr("File open failed

\"%1\" could not be copied into %2's own storage: %3

") + .arg(name.toHtmlEscaped(), app, error.toHtmlEscaped()) + + pickDetails(uri, local, why)); } return copy; } +QString +MainWindow::pickDetails(QString uri, QString path, QString why) const +{ + // Small print for the user to pass on: which app's provider the file + // came from, where Tony looked for it, and why that did not do + QString provider = AndroidFiles::providerOf(uri); + QString details = "

"; + details += tr("From: %1").arg((provider != "" ? provider : uri) + .toHtmlEscaped()); + if (path != "") { + details += "
" + tr("Looked for at: %1").arg(path.toHtmlEscaped()); + } + if (why != "") { + details += "
" + tr("Because: %1").arg(why.toHtmlEscaped()); + } + details += "

"; + return details; +} + +void +MainWindow::saveLog() +{ + QString app = QApplication::applicationName(); + + QByteArray log = LogFile::contents(AndroidStorage::logPath()); + if (log.isEmpty()) { + QMessageBox::information + (this, tr("No log"), + tr("There is no log to save

%1 could not keep one on this phone.

").arg(app)); + return; + } + + QFileDialog dialog(this, tr("Save the log")); + dialog.setAcceptMode(QFileDialog::AcceptSave); + dialog.setFileMode(QFileDialog::AnyFile); + dialog.setMimeTypeFilters({ "text/plain" }); + dialog.selectFile(QString("tony-log-%1.txt") + .arg(QDateTime::currentDateTime() + .toString("yyyyMMdd-HHmmss"))); + if (!dialog.exec()) return; + QList urls = dialog.selectedUrls(); + if (urls.empty() || urls[0].isEmpty()) return; + + // A new document the picker has made, through its grant: one file, + // which any provider can take, a cloud app's included + QString target = (urls[0].isLocalFile() ? urls[0].toLocalFile() : + AndroidFiles::grantedUri(urls[0])); + QString error; + QFile out; + bool opened = false; + if (urls[0].isLocalFile()) { + out.setFileName(target); + opened = out.open(QIODevice::WriteOnly | QIODevice::Truncate); + if (!opened) error = out.errorString(); + } else { + int fd = AndroidStorage::openDocument(target, "w", error); + if (fd >= 0) { + opened = out.open(fd, QIODevice::WriteOnly, + QFileDevice::AutoCloseHandle); + if (!opened) { + ::close(fd); + error = out.errorString(); + } + } + } + + bool written = opened && out.write(log) == log.size(); + if (opened && !written) error = out.errorString(); + if (opened) out.close(); + + if (!written) { + cerr << "MainWindow::saveLog: could not write " << target << ": " + << error << endl; + QMessageBox::critical + (this, tr("Failed to save the log"), + tr("The log was not saved

%1

") + .arg(error.toHtmlEscaped()) + pickDetails(target, "", "")); + return; + } + cerr << "MainWindow::saveLog: saved " << log.size() << " bytes to " + << target << endl; +} + +bool +MainWindow::recentFileIsThere(QString path) +{ + if (!AndroidFiles::usableRecentFiles({ path }).empty()) return true; + + QString app = QApplication::applicationName(); + QString own = QStandardPaths::writableLocation + (QStandardPaths::AppDataLocation); + if (!path.startsWith(own) && !AndroidStorage::hasAllFilesAccess()) { + QMessageBox::warning + (this, tr("Cannot open the file"), + tr("%1 may not open \"%2\" where it is

That needs All files access, which %1 no longer has. Allow it in the phone's Settings, Apps, %1, or open the file with File, Open.

") + .arg(app, path.toHtmlEscaped())); + } else { + QMessageBox::warning + (this, tr("File not found"), + tr("\"%1\" is no longer there

It has been moved or deleted since it was last opened. Open it with File, Open from where it is now.

") + .arg(path.toHtmlEscaped())); + } + return false; +} + QString MainWindow::getSaveFileName(FileFinder::FileType type) { @@ -2865,19 +3049,28 @@ MainWindow::getSaveFileName(FileFinder::FileType type) if (suggested != "") dialog.selectFile(suggested); if (!dialog.exec()) return ""; - QStringList selected = dialog.selectedFiles(); - if (selected.empty() || selected[0] == "") return ""; - QString picked = selected[0]; + QList urls = dialog.selectedUrls(); + if (urls.empty() || urls[0].isEmpty()) return ""; + QString picked = (urls[0].isLocalFile() ? urls[0].toLocalFile() : + AndroidFiles::grantedUri(urls[0])); // The picker has made an empty document of that name by now - QString local = (picked.startsWith("content:") ? - m_storage->pathFor(picked) : picked); + QString local = picked, why; + if (!urls[0].isLocalFile()) { + local = AndroidStorage::pathFor(picked, why); + if (local != "" && !QFileInfo(local).exists()) { + why = tr("there is no file at that path"); + local = ""; + } + } if (local == "") { - AndroidFiles::removeIfEmpty(picked); + cerr << "MainWindow::getSaveFileName: no path for " << picked + << ": " << why << endl; + AndroidStorage::removeIfEmpty(picked); QMessageBox::warning (this, tr("Cannot save the session there"), - tr("The session cannot be saved there

A session keeps its audio and its takes folder beside it, which %1 can write only in a folder of the phone's own storage, not in Downloads or through a cloud app.

Browse to a folder of the phone's own storage in the picker, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer.

") - .arg(app)); + tr("The session cannot be saved there

A session keeps its audio and its takes folder beside it, which %1 can write only in a folder of the phone's own storage, not through a cloud app.

Browse to a folder of the phone's own storage in the picker, such as one a sync app (Syncthing, FolderSync) keeps in step with your computer.

") + .arg(app) + pickDetails(picked, "", why)); return ""; } @@ -6431,6 +6624,15 @@ MainWindow::openRecentFile() QString path = action->objectName(); if (path == "") return; +#ifdef Q_OS_ANDROID + // A file moved or deleted since, said so, rather than "could not be + // opened" + if (!recentFileIsThere(path)) { + setupRecentFilesMenu(); + return; + } +#endif + FileOpenStatus status = openPath(path, ReplaceSession); if (status == FileOpenFailed) { diff --git a/main/MainWindow.h b/main/MainWindow.h index 9ffa0293..c7ac547f 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -914,9 +914,21 @@ protected slots: // is, by its path, once Tony has All files access (AndroidStorage), // so that a session finds its audio and takes beside it. Other audio // is copied into the app's own storage and the copy's path returned; - // other sessions are refused + // other sessions are refused. Tony's own picker, not svgui's dialog: + // that asks Qt whether the URI's file exists, and Qt's content file + // engine says no for a name with parentheses (AndroidFiles:: + // grantedUri()); and it offers only the types Qt names for its + // filters, which are not always Android's QString getOpenFileName(sv::FileFinder::FileType type) override; + // What to tell the user of a picked file that could not be opened or + // saved, to pass on: its provider, the path looked for, and why + QString pickDetails(QString uri, QString path, QString why) const; + + // Help > Save Log...: Tony's log (main.cpp) through the save picker, + // for a user who cannot read the system log to send + void saveLog(); + // Save Session As: Tony's own picker, which suggests a name (svgui's // suggests none), and then the path of the file picked in the phone's // own storage, or nothing. The picker has made an empty document by @@ -925,6 +937,11 @@ protected slots: QString getSaveFileName(sv::FileFinder::FileType type) override; AndroidStorage *m_storage; + // A recent file that has been moved or deleted is said to be so, and + // is no longer offered (RecentFiles keeps it, having no way to drop + // one) + bool recentFileIsThere(QString path); + // Android sends Tony to the background: playback stops, a take being // recorded is finished as Stop finishes it, and the session is saved // as Save saves it, if it has a file of its own. Android holds Tony's diff --git a/main/PopupArea.cpp b/main/PopupArea.cpp new file mode 100644 index 00000000..64e322bc --- /dev/null +++ b/main/PopupArea.cpp @@ -0,0 +1,79 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "PopupArea.h" + +#include +#include + +QRect +PopupArea::usable(QRect available, QRect window, QMargins safeArea, + int edgeMargin) +{ + if (!available.isValid()) return available; + + QRect area = available; + if (!window.isNull()) { + area &= window.marginsRemoved(safeArea); + } + + edgeMargin = std::max(0, edgeMargin); + area.setTop(std::max(area.top(), available.top() + edgeMargin)); + area.setBottom(std::min(area.bottom(), available.bottom() - edgeMargin)); + + if (!area.isValid()) return available; + return area; +} + +int +PopupArea::menuFrame(QRect available, QRect usable, int base) +{ + int frame = std::max(0, base); + if (!available.isValid() || !usable.isValid()) return frame; + + frame = std::max(frame, usable.left() - available.left()); + frame = std::max(frame, usable.top() - available.top()); + frame = std::max(frame, available.right() - usable.right()); + frame = std::max(frame, available.bottom() - usable.bottom()); + return frame; +} + +QRect +PopupArea::fit(QRect rect, QRect usable) +{ + if (!usable.isValid()) return rect; + + rect.setWidth(std::min(rect.width(), usable.width())); + rect.setHeight(std::min(rect.height(), usable.height())); + + if (rect.right() > usable.right()) rect.moveRight(usable.right()); + if (rect.left() < usable.left()) rect.moveLeft(usable.left()); + if (rect.bottom() > usable.bottom()) rect.moveBottom(usable.bottom()); + if (rect.top() < usable.top()) rect.moveTop(usable.top()); + return rect; +} + +int +PopupArea::pixels(double mm, double dotsPerInch) +{ + if (mm <= 0 || dotsPerInch <= 0) return 0; + return int(std::lround(mm * dotsPerInch / 25.4)); +} + +int +PopupArea::fingerWidth(double dotsPerInch) +{ + if (dotsPerInch < 100 || dotsPerInch > 300) dotsPerInch = 160; + return pixels(8, dotsPerInch); +} diff --git a/main/PopupArea.h b/main/PopupArea.h new file mode 100644 index 00000000..a94317cc --- /dev/null +++ b/main/PopupArea.h @@ -0,0 +1,79 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_POPUP_AREA_H +#define TONY_POPUP_AREA_H + +#include +#include + +/** + * Where on a phone's screen menus and other popups may go. + * + * Qt places a popup inside the screen's available geometry, and on + * Android that is the whole of the activity, system bars included: from + * Android 15 on an app draws edge to edge, under the status bar, the + * navigation bar and the camera's cutout, and is told of them only as + * its windows' safe area margins (QWindow::safeAreaMargins()), which + * the main window's layout keeps clear of and a popup's placement does + * not. A tap on a menu item under the status bar goes to the status + * bar. Plain arithmetic, for TouchMenuStyle, which applies it. + */ +class PopupArea +{ +public: + /** + * The part of available (the screen's available geometry) a popup + * may cover: inside window (the application's main window, in the + * same coordinates) less safeArea, its safe area margins, and at + * least edgeMargin from the top and bottom of available, where a + * finger does not reach well. A null window stands for all of + * available. Never empty: if the margins leave nothing, available. + */ + static QRect usable(QRect available, QRect window, QMargins safeArea, + int edgeMargin); + + /** + * The margin QMenu keeps between a menu and every side of the screen + * (the style's PM_MenuDesktopFrameWidth, one number for all four + * sides) for its menus to lie inside usable: the widest of the four + * gaps between available and usable, and at least base, the style's + * own. QMenu places, sizes and scrolls a menu by that margin alone, + * so a menu is kept inside by it rather than moved after it is + * placed, which would leave QMenu's scrolling out of step. + */ + static int menuFrame(QRect available, QRect usable, int base); + + /** + * rect moved, and shortened or narrowed where it has to be, to lie + * inside usable: for a combo box's list, which scrolls whatever its + * height. + */ + static QRect fit(QRect rect, QRect usable); + + /** + * A length in pixels: mm millimetres at dotsPerInch. + */ + static int pixels(double mm, double dotsPerInch); + + /** + * About a finger's width, 8 mm, in Qt's pixels on a phone whose + * screen has dotsPerInch of them to the inch. That is near 160 + * (Android's dp, which Qt's pixels are there); a figure far from it, + * as some phones report, is taken to be 160. + */ + static int fingerWidth(double dotsPerInch); +}; + +#endif diff --git a/main/TouchMenuStyle.cpp b/main/TouchMenuStyle.cpp index 07137b76..67640902 100644 --- a/main/TouchMenuStyle.cpp +++ b/main/TouchMenuStyle.cpp @@ -13,23 +13,89 @@ */ #include "TouchMenuStyle.h" +#include "PopupArea.h" #include #include +#include #include #include +#include #include +#include #include +#include TouchMenuStyle::TouchMenuStyle(QStyle *base) : QProxyStyle(base), m_pressY(0), m_scrolledToY(0), - m_dragging(false) + m_dragging(false), + m_edgeMargin(0) { } +void +TouchMenuStyle::setSafeAreaWindow(QWidget *window) +{ + m_safeAreaWindow = window; +} + +void +TouchMenuStyle::setEdgeMargin(int pixels) +{ + m_edgeMargin = pixels; +} + +QRect +TouchMenuStyle::usableArea(const QWidget *popup) const +{ + QScreen *screen = (popup ? popup->screen() : nullptr); + if (!screen) screen = QGuiApplication::primaryScreen(); + if (!screen) return QRect(); + + // Android's available geometry is the whole of the activity, bars + // and all (qandroidplatformscreen.cpp, handleLayoutSizeChanged() in + // androidjnimain.cpp); the bars are the window's safe area margins + QRect window; + QMargins safeArea; + if (QWidget *w = m_safeAreaWindow.data()) { + if (w->isVisible()) { + window = QRect(w->mapToGlobal(QPoint(0, 0)), w->size()); +#if QT_VERSION >= QT_VERSION_CHECK(6, 9, 0) + if (QWindow *handle = w->windowHandle()) { + safeArea = handle->safeAreaMargins(); + } +#endif + } + } + + QRect available = screen->availableGeometry(); + QRect usable = PopupArea::usable(available, window, safeArea, + m_edgeMargin); + + // Said when it changes, for a report from a phone + if (usable != m_lastUsable) { + m_lastUsable = usable; + std::cerr << "TouchMenuStyle: popups within " << usable.x() << "," + << usable.y() << " " << usable.width() << "x" + << usable.height() << " of " << available.width() << "x" + << available.height() << ", safe area margins " + << safeArea.left() << "," << safeArea.top() << "," + << safeArea.right() << "," << safeArea.bottom() + << std::endl; + } + return usable; +} + +bool +TouchMenuStyle::isComboList(const QWidget *widget) +{ + return widget && widget->windowType() == Qt::Popup && + qobject_cast(widget->parentWidget()); +} + int TouchMenuStyle::styleHint(StyleHint hint, const QStyleOption *option, @@ -52,6 +118,20 @@ TouchMenuStyle::pixelMetric(PixelMetric metric, // beside it if (metric == PM_MenuScrollerHeight) return 3 * value; + // QMenu keeps this far from every side of the screen's available + // geometry, when it places a menu, sizes one too tall to fit and + // scrolls it (QMenuPrivate::popup(), scrollMenu() in qmenu.cpp): so + // the whole of what it does stays inside the usable area + if (metric == PM_MenuDesktopFrameWidth && + qobject_cast(widget)) { + QScreen *screen = widget->screen(); + if (!screen) screen = QGuiApplication::primaryScreen(); + if (screen) { + return PopupArea::menuFrame(screen->availableGeometry(), + usableArea(widget), value); + } + } + return value; } @@ -59,16 +139,16 @@ void TouchMenuStyle::polish(QWidget *widget) { QProxyStyle::polish(widget); - if (QMenu *menu = qobject_cast(widget)) { - menu->installEventFilter(this); + if (qobject_cast(widget) || isComboList(widget)) { + widget->installEventFilter(this); } } void TouchMenuStyle::unpolish(QWidget *widget) { - if (QMenu *menu = qobject_cast(widget)) { - menu->removeEventFilter(this); + if (qobject_cast(widget) || isComboList(widget)) { + widget->removeEventFilter(this); } QProxyStyle::unpolish(widget); } @@ -88,6 +168,17 @@ TouchMenuStyle::rowHeight(QMenu *menu) bool TouchMenuStyle::eventFilter(QObject *object, QEvent *event) { + // A combo box's list is placed by QComboBox::showPopup() inside the + // available geometry and then shown: moved into the usable area + // before it is, as its list scrolls whatever its height + if (event->type() == QEvent::Show && object->isWidgetType() && + isComboList(static_cast(object))) { + QWidget *list = static_cast(object); + QRect fitted = PopupArea::fit(list->geometry(), usableArea(list)); + if (fitted != list->geometry()) list->setGeometry(fitted); + return false; + } + QMenu *menu = qobject_cast(object); if (!menu) return false; diff --git a/main/TouchMenuStyle.h b/main/TouchMenuStyle.h index 6c776278..4e144ba3 100644 --- a/main/TouchMenuStyle.h +++ b/main/TouchMenuStyle.h @@ -33,6 +33,11 @@ class QMenu; * QMenu scrolls by; the lift that ends a drag chooses nothing. A tap * chooses an item as before. * + * Menus and combo box lists stay inside the part of the screen a popup + * may use (PopupArea): clear of the system bars an app drawn edge to + * edge lies under, which the main window's safe area margins give, and + * of the top and bottom edges of the screen by edgeMargin. + * * Built everywhere and tested on the desktop, whose style is left as it * is: only main() on Android installs it. */ @@ -44,6 +49,17 @@ class TouchMenuStyle : public QProxyStyle // The application's style, or base if given, with menus for touch TouchMenuStyle(QStyle *base = nullptr); + // The window whose safe area popups keep inside: the main window, + // which covers the screen on a phone. None by default + void setSafeAreaWindow(QWidget *window); + + // How far popups keep from the top and bottom of the screen, in + // pixels: 0 by default + void setEdgeMargin(int pixels); + + // Where popup may be, in global coordinates + QRect usableArea(const QWidget *popup) const; + int styleHint(StyleHint hint, const QStyleOption *option = nullptr, const QWidget *widget = nullptr, @@ -66,12 +82,19 @@ class TouchMenuStyle : public QProxyStyle // The height a finger moves to scroll the menu by one row static int rowHeight(QMenu *menu); + // Whether widget is a combo box's list, a popup window of its own + static bool isComboList(const QWidget *widget); + // The menu pressed on, where, how far the scrolling has followed the // finger, and whether the press has become a drag QPointer m_menu; double m_pressY; double m_scrolledToY; bool m_dragging; + + QPointer m_safeAreaWindow; + int m_edgeMargin; + mutable QRect m_lastUsable; }; #endif diff --git a/main/main.cpp b/main/main.cpp index cbeef1ee..7b320cf4 100644 --- a/main/main.cpp +++ b/main/main.cpp @@ -45,13 +45,19 @@ #ifdef Q_OS_ANDROID #include "AndroidFiles.h" +#include "AndroidStorage.h" +#include "LogFile.h" +#include "PopupArea.h" #include "TouchMenuStyle.h" #include +#include #include #include +#include #include #include #include +#include #endif #include "../version.h" @@ -144,10 +150,50 @@ class TonyApplication : public QApplication }; #ifdef Q_OS_ANDROID +// Each line that goes to the system log goes to a file in the app's own +// storage as well, from when keepSystemLogInFile() is called (it needs +// the application): the user, who has no adb to read the system log +// with, can save a copy of it with Help > Save Log... and send it. The +// lines before that are held until then +static std::mutex logFileMutex; +static LogFile *logFile = nullptr; +static std::vector linesBeforeLogFile; + +static void +writeToLogFile(const std::string &line) +{ + std::lock_guard lock(logFileMutex); + if (logFile) { + logFile->write(QByteArray::fromStdString(line)); + } else if (linesBeforeLogFile.size() < 1000) { + linesBeforeLogFile.push_back(line); + } +} + +static void +keepSystemLogInFile(QString path) +{ + std::lock_guard lock(logFileMutex); + LogFile *file = new LogFile(path, 512 * 1024); + if (!file->open()) { + delete file; + linesBeforeLogFile.clear(); + __android_log_write(ANDROID_LOG_WARN, "Tony", + "Cannot write the log file"); + return; + } + for (const std::string &line : linesBeforeLogFile) { + file->write(QByteArray::fromStdString(line)); + } + linesBeforeLogFile.clear(); + logFile = file; +} + // An Android app's stdout and stderr lead nowhere, and Tony and svcore // report through cerr. Send both into a pipe, and have a thread pass what // comes out of it on to the system log (logcat), a line at a time, under -// the tag "Tony". Qt's own messages go to the system log already. +// the tag "Tony", and to the log file. Qt's own messages go to the system +// log only. static void sendOutputToSystemLog() { @@ -172,6 +218,7 @@ sendOutputToSystemLog() // The system log cuts longer lines short if (buffer[i] == '\n' || line.size() >= 1000) { __android_log_write(ANDROID_LOG_INFO, "Tony", line.c_str()); + writeToLogFile(line); line.clear(); } if (buffer[i] != '\n') line += buffer[i]; @@ -310,6 +357,13 @@ main(int argc, char **argv) QApplication::setOrganizationDomain("sonicvisualiser.org"); QApplication::setApplicationName("Tony"); +#ifdef Q_OS_ANDROID + keepSystemLogInFile(AndroidStorage::logPath()); + cerr << "Tony " << TONY_VERSION << " on Android API " + << QNativeInterface::QAndroidApplication::sdkVersion() + << ", Qt " << qVersion() << endl; +#endif + QStringList pluginProblems = setupTonyVampPath(); QStringList args = application.arguments(); @@ -399,6 +453,20 @@ main(int argc, char **argv) QScreen *screen = QApplication::primaryScreen(); QRect available = screen->availableGeometry(); +#ifdef Q_OS_ANDROID + // Menus clear of the system bars, which the main window's safe area + // margins give, and a finger's width from the screen's top and bottom + if (TouchMenuStyle *style = + qobject_cast(QApplication::style())) { + int margin = PopupArea::fingerWidth(screen->physicalDotsPerInchY()); + style->setSafeAreaWindow(gui); + style->setEdgeMargin(margin); + cerr << "Menus keep " << margin << " px from the screen's top and " + << "bottom (" << screen->physicalDotsPerInchY() << " px per " + << "inch)" << endl; + } +#endif + int width = (available.width() * 2) / 3; int height = available.height() / 2; if (height < 450) height = (available.height() * 2) / 3; diff --git a/main/test/TestAndroidFiles.h b/main/test/TestAndroidFiles.h index 54e82b7c..c52a7a17 100644 --- a/main/test/TestAndroidFiles.h +++ b/main/test/TestAndroidFiles.h @@ -16,11 +16,12 @@ // Tier 1: the file work the Android build does at start (the Vamp plugin // links) and when a file is picked (the copy into app storage, the path -// of a picked content:// URI, the names Save Session As suggests and -// accepts), done here on plain files in a temporary directory and on -// URIs written as Android writes them. What only a phone has -- a -// provider behind the URI, the installed library directory -- is not -// here. +// of a picked content:// URI or where to look it up, the URI string +// Android granted, the files Tony opens, the names Save Session As +// suggests and accepts, the recent files that are still there), done +// here on plain files in a temporary directory and on URIs written as +// Android writes them. What only a phone has -- a provider behind the +// URI, MediaStore, the installed library directory -- is not here. #include "../AndroidFiles.h" @@ -28,6 +29,7 @@ #include #include +#include #include #include #include @@ -143,6 +145,26 @@ private slots: QCOMPARE(QDir(into).entryList(QDir::Files), QStringList({ "take.wav" })); } + void a_file_is_copied_from_what_its_provider_hands_over() { + // On Android: a file descriptor from the provider, through the URI + // as Android wrote it, which may be a pipe with no size + QByteArray content("ID3 and the rest of an MP3"); + QBuffer pipe(&content); + QVERIFY(pipe.open(QIODevice::ReadOnly)); + + QString into = newPath("imported"); + QString error; + QString copy = AndroidFiles::copyIn + (pipe, "content://com.google.android.apps.docs.storage/document/" + "acc%3D1%3Bdoc%3Dencoded%3Dabc", + "(vocals) Avi Kaplan - Peace Somehow.mp3", into, error); + + QCOMPARE(error, QString()); + QCOMPARE(copy, QDir(into).filePath + ("(vocals) Avi Kaplan - Peace Somehow.mp3")); + QCOMPARE(readFile(copy), content); + } + void a_name_cannot_put_the_copy_elsewhere() { QString source = newDir("provider") + "/document"; QVERIFY(writeFile(source, "x")); @@ -250,7 +272,8 @@ private slots: "acc%3D1%3Bdoc%3Dencoded%3DabcDEF", "content://com.dropbox.android.document/document/" "%2FSongs%2Ftest.ton", - // The media provider and a download, by number + // The media provider and a download, by number: looked up in + // MediaStore instead (pathLookupFor()) "content://com.android.providers.media.documents/document/audio%3A42", "content://com.android.providers.downloads.documents/document/msf%3A1234", "content://com.android.providers.downloads.documents/document/1234", @@ -295,6 +318,229 @@ private slots: QString()); } + // The user's session, whose name has spaces and parentheses, in each + // form its URI can come in: as Android writes it; as + // QFileDialog::selectedFiles() gives it (what the "File does not + // exist" box showed); and as Qt's content file engine rebuilds it, + // with the parentheses encoded as well + void the_users_session_has_its_path_in_every_form_of_its_uri() { + QString root = "/storage/emulated/0"; + QString provider = + "content://com.android.externalstorage.documents/document/"; + QString name = "(vocals) Avi Kaplan - Peace Somehow.ton"; + QString inMusic = "/storage/emulated/0/Music/" + name; + QString inDownload = "/storage/emulated/0/Download/" + name; + + QString android = provider + "primary%3AMusic%2F" + "(vocals)%20Avi%20Kaplan%20-%20Peace%20Somehow.ton"; + QString shown = provider + "primary%3AMusic%2F" + "(vocals) Avi Kaplan - Peace Somehow.ton"; + QString rebuilt = provider + "primary%3AMusic%2F" + "%28vocals%29%20Avi%20Kaplan%20-%20Peace%20Somehow.ton"; + + QCOMPARE(pickedAsQtGivesIt(android), shown); + QCOMPARE(AndroidFiles::pathFromContentUri(android, root), inMusic); + QCOMPARE(AndroidFiles::pathFromContentUri(shown, root), inMusic); + QCOMPARE(AndroidFiles::pathFromContentUri(rebuilt, root), inMusic); + QCOMPARE(AndroidFiles::pathLookupFor(shown), + AndroidFiles::PathLookup::InUri); + + // In Download, by the phone's storage or by the Downloads root + QString download = provider + "primary%3ADownload%2F" + "(vocals)%20Avi%20Kaplan%20-%20Peace%20Somehow.ton"; + QCOMPARE(AndroidFiles::pathFromContentUri(download, root), inDownload); + QCOMPARE(AndroidFiles::pathFromContentUri + (pickedAsQtGivesIt(download), root), inDownload); + QString raw = "content://com.android.providers.downloads.documents/" + "document/raw%3A%2Fstorage%2Femulated%2F0%2FDownload%2F" + "(vocals)%20Avi%20Kaplan%20-%20Peace%20Somehow.ton"; + QCOMPARE(AndroidFiles::pathFromContentUri(raw, root), inDownload); + QCOMPARE(AndroidFiles::pathFromContentUri + (pickedAsQtGivesIt(raw), root), inDownload); + + // Under a grant of the folder + QString tree = "content://com.android.externalstorage.documents/" + "tree/primary%3AMusic/document/primary%3AMusic%2F" + "(vocals)%20Avi%20Kaplan%20-%20Peace%20Somehow.ton"; + QCOMPARE(AndroidFiles::pathFromContentUri(tree, root), inMusic); + QCOMPARE(AndroidFiles::pathFromContentUri + (pickedAsQtGivesIt(tree), root), inMusic); + } + + // What Tony opens a picked document by, when it has no path: the URI + // exactly as Android wrote it, which is what Android granted + void the_uri_is_opened_as_android_wrote_it() { + QStringList uris = { + "content://com.android.externalstorage.documents/document/" + "primary%3AMusic%2F(vocals)%20Avi%20Kaplan%20-%20Peace%20" + "Somehow.ton", + // Uri.encode() leaves "_-!.~'()*" as they are, and encodes + // the rest as UTF-8, in capitals + "content://com.android.externalstorage.documents/document/" + "primary%3ADownload%2FIt's%20a%20*star*!%20%C3%84%C3%A4ni" + "%20%2B%26%3D%3B%2C%24%23%3F%25%5B%5D.wav", + "content://com.android.externalstorage.documents/document/" + "primary%3ADownload%2F%7B%7D%7C%5C%5E%60%22%3C%3E%40.wav", + "content://com.android.externalstorage.documents/tree/" + "primary%3AMusic/document/primary%3AMusic%2F(a)%20b.ton", + "content://com.google.android.apps.docs.storage/document/" + "acc%3D1%3Bdoc%3Dencoded%3DAbC-_x%2Fy%2Bz%3D%3D", + "content://com.android.providers.downloads.documents/document/" + "msf%3A1000001234", + "content://com.android.providers.downloads.documents/document/" + "raw%3A%2Fstorage%2Femulated%2F0%2FDownload%2Fx%20(1).ton", + "content://com.android.providers.downloads.documents/document/1234", + "content://com.android.providers.media.documents/document/" + "audio%3A42", + }; + for (QString uri : uris) { + QCOMPARE(AndroidFiles::grantedUri(QUrl(uri)), uri); + } + + // Not the string the picker's selectedFiles() gives: that has the + // spaces decoded, and is another URI to Android + QVERIFY(pickedAsQtGivesIt(uris[0]) != uris[0]); + } + + void the_provider_is_named() { + QCOMPARE(AndroidFiles::providerOf + ("content://com.google.android.apps.docs.storage/document/" + "acc%3D1%3Bdoc%3Dencoded%3Dabc"), + QString("com.google.android.apps.docs.storage")); + QCOMPARE(AndroidFiles::providerOf + ("content://com.android.externalstorage.documents/root/primary"), + QString("com.android.externalstorage.documents")); + QCOMPARE(AndroidFiles::providerOf("/storage/emulated/0/x.ton"), + QString()); + QCOMPARE(AndroidFiles::providerOf(""), QString()); + } + + void downloads_and_media_are_looked_up_in_mediastore() { + QString downloads = + "content://com.android.providers.downloads.documents/document/"; + QString media = + "content://com.android.providers.media.documents/document/"; + using L = AndroidFiles::PathLookup; + + QCOMPARE(AndroidFiles::pathLookupFor(downloads + "msf%3A1000001234"), + L::MediaStore); + QCOMPARE(AndroidFiles::mediaStoreUriFor(downloads + "msf%3A1000001234"), + QString("content://media/external/downloads/1000001234")); + // As the picker's selectedFiles() gives it + QCOMPARE(AndroidFiles::mediaStoreUriFor(downloads + "msf:77"), + QString("content://media/external/downloads/77")); + + QCOMPARE(AndroidFiles::pathLookupFor(media + "audio%3A42"), + L::MediaStore); + QCOMPARE(AndroidFiles::mediaStoreUriFor(media + "audio%3A42"), + QString("content://media/external/audio/media/42")); + QCOMPARE(AndroidFiles::mediaStoreUriFor(media + "image%3A7"), + QString("content://media/external/images/media/7")); + QCOMPARE(AndroidFiles::mediaStoreUriFor(media + "video%3A8"), + QString("content://media/external/video/media/8")); + QCOMPARE(AndroidFiles::mediaStoreUriFor(media + "document%3A9"), + QString("content://media/external/file/9")); + + // A download known by its number in the download manager + QCOMPARE(AndroidFiles::pathLookupFor(downloads + "1234"), + L::ByNameAndSize); + QCOMPARE(AndroidFiles::mediaStoreUriFor(downloads + "1234"), QString()); + + // The path is in the URI + QCOMPARE(AndroidFiles::pathLookupFor + (downloads + "raw%3A%2Fstorage%2Femulated%2F0%2FDownload%2Fa.ton"), + L::InUri); + QCOMPARE(AndroidFiles::pathLookupFor + ("content://com.android.externalstorage.documents/document/" + "primary%3AMusic%2Fa.ton"), + L::InUri); + + // Nothing to look up: folders, the providers' roots and groups, a + // cloud provider's documents, ids that are not numbers + QStringList none = { + downloads + "msd%3A55", + downloads + "downloads", + downloads + "msf%3A12a", + downloads + "raw%3ASong.mp3", + media + "audio_root", + media + "album%3A3", + media + "artist%3A4", + media + "images_bucket%3A5", + media + "audio%3A", + "content://com.android.providers.media.documents/tree/audio_root", + "content://com.google.android.apps.docs.storage/document/" + "acc%3D1%3Bdoc%3Dencoded%3Dabc", + "content://com.android.externalstorage.documents/document/Music", + "content://media/external/audio/media/42", + "/storage/emulated/0/Download/a.ton", + "", + }; + for (QString uri : none) { + QCOMPARE(AndroidFiles::pathLookupFor(uri), L::None); + QCOMPARE(AndroidFiles::mediaStoreUriFor(uri), QString()); + } + } + + void a_numbered_download_is_found_by_name_and_size() { + QString root = "/storage/emulated/0"; + QString download = "/storage/emulated/0/Download/Song (1).mp3"; + QString copy = "/storage/emulated/0/Music/Song (1).mp3"; + QString deeper = "/storage/emulated/0/Download/Old/Song (1).mp3"; + + QCOMPARE(AndroidFiles::chooseDownload({ download }, root), download); + QCOMPARE(AndroidFiles::chooseDownload({ copy }, root), copy); + + // Several of that name and size: the one in Download + QCOMPARE(AndroidFiles::chooseDownload({ copy, download, deeper }, root), + download); + QCOMPARE(AndroidFiles::chooseDownload({ download, copy }, root + "/"), + download); + QCOMPARE(AndroidFiles::chooseDownload({ download, download }, root), + download); + + // Not settled + QCOMPARE(AndroidFiles::chooseDownload({ copy, deeper }, root), QString()); + QCOMPARE(AndroidFiles::chooseDownload({ download, copy }, ""), QString()); + QCOMPARE(AndroidFiles::chooseDownload({}, root), QString()); + QCOMPARE(AndroidFiles::chooseDownload({ "" }, root), QString()); + } + + void only_tonys_own_files_are_opened() { + QString session = "*.ton"; + QString audio = "*.aiff *.flac *.mp3 *.ogg *.opus *.wav"; + + QVERIFY(AndroidFiles::hasExtensionIn + ("(vocals) Avi Kaplan - Peace Somehow.ton", session)); + QVERIFY(AndroidFiles::hasExtensionIn("Song.MP3", audio)); + QVERIFY(AndroidFiles::hasExtensionIn("Song v1.2.wav", audio)); + QVERIFY(AndroidFiles::hasExtensionIn("a.WaV", "*.WAV")); + + QVERIFY(!AndroidFiles::hasExtensionIn("Song.ton", audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("Notes.pdf", session + " " + audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("Song", audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("Song.mp3.part", audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("mp3", audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("", audio)); + QVERIFY(!AndroidFiles::hasExtensionIn("Song.mp3", "")); + } + + void recent_files_that_are_gone_are_left_out() { + QString dir = newDir("recent"); + QString here = dir + "/Here.ton"; + QString moved = dir + "/Moved.ton"; + QVERIFY(writeFile(here, "BZh91AY&SY")); + + QStringList recent = { + moved, + here, + "content://com.android.externalstorage.documents/document/" + "primary%3AMusic%2FSong.mp3", + newDir("a-folder"), + "", + }; + QCOMPARE(AndroidFiles::usableRecentFiles(recent), QStringList({ here })); + } + // --- The name Save Session As suggests and accepts --- void the_suggested_name_is_the_sessions_or_the_references() { diff --git a/main/test/TestLogFile.h b/main/test/TestLogFile.h new file mode 100644 index 00000000..09ed91e9 --- /dev/null +++ b/main/test/TestLogFile.h @@ -0,0 +1,121 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LOG_FILE_H +#define TEST_LOG_FILE_H + +// Tier 1: the copy of Tony's output kept on Android for Help > Save +// Log... (LogFile): an earlier run's kept once, the whole never much +// more than twice its limit, and read back oldest first. + +#include "../LogFile.h" + +#include +#include +#include +#include +#include +#include + +class TestLogFile : public QObject +{ + Q_OBJECT + + QTemporaryDir m_dir; + int m_counter = 0; + + QString newPath() { + return m_dir.filePath(QString("log-%1/tony.log").arg(++m_counter)); + } + + static bool writeFile(QString path, QByteArray content) { + QDir().mkpath(QFileInfo(path).absolutePath()); + QFile file(path); + if (!file.open(QIODevice::WriteOnly)) return false; + bool ok = (file.write(content) == content.size()); + file.close(); + return ok; + } + + static qint64 size(QString path) { + return QFileInfo(path).size(); + } + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void a_log_is_begun_in_a_folder_not_there_yet() { + QString path = newPath(); + LogFile log(path, 1000); + QVERIFY(!log.isOpen()); + QVERIFY(log.open()); + QVERIFY(log.isOpen()); + + log.write("Setting VAMP_PATH to /data/user/0/io.github.jhhr.tony/files/vamp"); + QCOMPARE(LogFile::contents(path), + QByteArray("Setting VAMP_PATH to /data/user/0/" + "io.github.jhhr.tony/files/vamp\n")); + } + + void the_run_before_is_kept_and_the_one_before_that_goes() { + QString path = newPath(); + QVERIFY(writeFile(LogFile::olderPath(path), "two runs ago\n")); + QVERIFY(writeFile(path, "the last run\n")); + + LogFile log(path, 1000); + QVERIFY(log.open()); + log.write("this run"); + + QCOMPARE(LogFile::contents(path), + QByteArray("the last run\nthis run\n")); + } + + void a_log_past_its_limit_begins_again() { + QString path = newPath(); + const qint64 limit = 100; + LogFile log(path, limit); + QVERIFY(log.open()); + + // Each line 10 bytes with its newline + for (int i = 0; i < 50; ++i) { + log.write(QString("line %1...").arg(i, 2, 10, QChar('0')).toUtf8() + .left(9)); + } + + QVERIFY(size(path) < limit); + QVERIFY(size(LogFile::olderPath(path)) <= limit); + + QByteArray all = LogFile::contents(path); + QVERIFY(all.size() <= 2 * limit); + QVERIFY(all.size() >= limit); + QVERIFY(all.endsWith("line 49..\n")); + QVERIFY(!all.contains("line 00")); + + // In order, oldest first, no line cut + QList lines = all.split('\n'); + lines.removeLast(); + int first = lines.first().mid(5, 2).toInt(); + for (int i = 0; i < lines.size(); ++i) { + QCOMPARE(lines[i], QString("line %1..").arg(first + i, 2, 10, + QChar('0')).toUtf8()); + } + } + + void there_is_nothing_to_read_before_a_log_is_begun() { + QCOMPARE(LogFile::contents(newPath()), QByteArray()); + } +}; + +#endif diff --git a/main/test/TestPopupArea.h b/main/test/TestPopupArea.h new file mode 100644 index 00000000..c94a9439 --- /dev/null +++ b/main/test/TestPopupArea.h @@ -0,0 +1,147 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_POPUP_AREA_H +#define TEST_POPUP_AREA_H + +// Tier 1: where menus may go on a phone drawn edge to edge (PopupArea): +// inside the main window's safe area, clear of the top and bottom of +// the screen, and the one margin QMenu is given for that. The figures +// are a phone's in landscape, in device-independent pixels: the screen +// 915 by 412, a camera cutout at the left, the status bar at the top, +// gesture navigation at the bottom. + +#include "../PopupArea.h" + +#include +#include + +class TestPopupArea : public QObject +{ + Q_OBJECT + + const QRect screen = QRect(0, 0, 915, 412); + const QMargins bars = QMargins(35, 24, 0, 16); + +private slots: + void a_phone_in_landscape_keeps_popups_off_its_bars_and_edges() { + QRect usable = PopupArea::usable(screen, screen, bars, 50); + + // Clear of the cutout at the side, and of the top and bottom + // edges by the margin, which is wider than either bar + QCOMPARE(usable, QRect(QPoint(35, 50), QPoint(914, 361))); + } + + void a_bar_wider_than_the_margin_is_kept_clear_of() { + QRect usable = PopupArea::usable(screen, screen, + QMargins(0, 60, 48, 70), 50); + QCOMPARE(usable, QRect(QPoint(0, 60), QPoint(914 - 48, 411 - 70))); + } + + void a_window_short_of_the_screen_is_kept_inside() { + // In split screen, Tony has part of it + QRect window(0, 0, 915, 200); + QRect usable = PopupArea::usable(screen, window, QMargins(0, 24, 0, 0), + 10); + QCOMPARE(usable, QRect(QPoint(0, 24), QPoint(914, 199))); + } + + void without_a_window_the_screen_less_its_edges() { + QCOMPARE(PopupArea::usable(screen, QRect(), bars, 50), + QRect(QPoint(0, 50), QPoint(914, 361))); + QCOMPARE(PopupArea::usable(screen, QRect(), QMargins(), 0), screen); + + // As on the desktop, where the screen does not start at 0 + QRect desktop(0, 30, 1920, 1050); + QCOMPARE(PopupArea::usable(desktop, QRect(), QMargins(), 0), desktop); + QCOMPARE(PopupArea::usable(desktop, desktop, QMargins(), 20), + QRect(QPoint(0, 50), QPoint(1919, 1059))); + } + + void margins_that_leave_nothing_leave_the_screen() { + QCOMPARE(PopupArea::usable(screen, screen, bars, 300), screen); + QCOMPARE(PopupArea::usable(screen, QRect(2000, 0, 10, 10), bars, 0), + screen); + } + + void qmenu_is_given_the_widest_gap_as_its_margin() { + QRect usable = PopupArea::usable(screen, screen, bars, 50); + QCOMPARE(PopupArea::menuFrame(screen, usable, 0), 50); + + // The style's own, if wider + QCOMPARE(PopupArea::menuFrame(screen, usable, 60), 60); + + // A bar at the side wider than the top and bottom margins + usable = PopupArea::usable(screen, screen, QMargins(0, 24, 80, 0), 50); + QCOMPARE(PopupArea::menuFrame(screen, usable, 0), 80); + + // Nothing to keep clear of + QCOMPARE(PopupArea::menuFrame(screen, screen, 0), 0); + QCOMPARE(PopupArea::menuFrame(screen, QRect(), 3), 3); + } + + // Where QMenu puts a menu with that margin: inside the usable area + // (its placement in qmenu.cpp, QMenuPrivate::popup(), for a menu + // taller than the screen popped up at its top, which scrolls) + void a_menu_placed_by_that_margin_is_inside() { + QRect usable = PopupArea::usable(screen, screen, bars, 50); + int frame = PopupArea::menuFrame(screen, usable, 0); + + int top = screen.top() + frame; + int height = screen.bottom() - frame * 2 - top; + QRect menu(screen.left() + frame, top, 300, height); + QVERIFY(usable.contains(menu)); + } + + void a_combo_boxs_list_is_moved_and_shortened_to_fit() { + QRect usable(QPoint(35, 50), QPoint(914, 361)); + + // Up from the box near the top, under the status bar + QCOMPARE(PopupArea::fit(QRect(100, 10, 200, 150), usable), + QRect(100, 50, 200, 150)); + // Down past the bottom + QCOMPARE(PopupArea::fit(QRect(100, 300, 200, 150), usable), + QRect(100, 212, 200, 150)); + // Taller than the whole area + QCOMPARE(PopupArea::fit(QRect(100, 0, 200, 500), usable), + QRect(100, 50, 200, 312)); + // Off the side, and wider than the area + QCOMPARE(PopupArea::fit(QRect(0, 100, 200, 100), usable), + QRect(35, 100, 200, 100)); + QCOMPARE(PopupArea::fit(QRect(0, 100, 1000, 100), usable), + QRect(35, 100, 880, 100)); + // Inside already + QCOMPARE(PopupArea::fit(QRect(100, 100, 200, 100), usable), + QRect(100, 100, 200, 100)); + // Nothing to fit into + QCOMPARE(PopupArea::fit(QRect(1, 2, 3, 4), QRect()), QRect(1, 2, 3, 4)); + } + + void millimetres_become_pixels() { + QCOMPARE(PopupArea::pixels(8, 160), 50); + QCOMPARE(PopupArea::pixels(25.4, 96), 96); + QCOMPARE(PopupArea::pixels(0, 160), 0); + QCOMPARE(PopupArea::pixels(8, 0), 0); + QCOMPARE(PopupArea::pixels(-1, 160), 0); + } + + void a_fingers_width_is_eight_millimetres() { + QCOMPARE(PopupArea::fingerWidth(160), 50); + QCOMPARE(PopupArea::fingerWidth(140), 44); + // Figures no phone's screen has, in Qt's pixels + QCOMPARE(PopupArea::fingerWidth(0), 50); + QCOMPARE(PopupArea::fingerWidth(480), 50); + } +}; + +#endif diff --git a/main/test/TestTouchMenuStyle.h b/main/test/TestTouchMenuStyle.h index 8b638139..2e24ce43 100644 --- a/main/test/TestTouchMenuStyle.h +++ b/main/test/TestTouchMenuStyle.h @@ -16,15 +16,19 @@ // Tier 5: menus taller than the screen, as a phone has them // (TouchMenuStyle): on the screen in one column, scrolled by a finger -// dragged over them, every item reachable, and a tap still a choice. -// The menus are real popups on the offscreen platform's screen; the -// mouse events are what Qt makes of a finger on Android. +// dragged over them, every item reachable, and a tap still a choice; +// menus and combo box lists clear of the system bars and of the top and +// bottom of the screen. The menus are real popups on the offscreen +// platform's screen; the mouse events are what Qt makes of a finger on +// Android. #include "../TouchMenuStyle.h" #include #include +#include #include +#include #include #include #include @@ -202,6 +206,120 @@ private slots: QCOMPARE(triggered.count(), 1); QCOMPARE(triggered.at(0).at(0).value(), shown); } + + // --- Clear of the system bars and the screen's edges --- + + static QRect onScreen(QWidget *w) { + return QRect(w->mapToGlobal(QPoint(0, 0)), w->size()); + } + + // Inside area, give or take the two pixels the offscreen platform + // moves a popup by + static bool within(QRect r, QRect area) { + return area.adjusted(-2, -2, 2, 2).contains(r); + } + + static QString describe(QRect r, QRect area) { + return QString("%1,%2 %3x%4 in %5,%6 %7x%8") + .arg(r.x()).arg(r.y()).arg(r.width()).arg(r.height()) + .arg(area.x()).arg(area.y()).arg(area.width()).arg(area.height()); + } + + void a_long_menu_keeps_clear_of_the_top_and_bottom_and_still_scrolls() { + m_style->setEdgeMargin(60); + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + + QRect usable = m_style->usableArea(menu); + QCOMPARE(usable, screen().adjusted(0, 60, 0, -60)); + QVERIFY2(within(onScreen(menu), usable), + qPrintable(describe(onScreen(menu), usable))); + + QPoint low(menu->width() / 2, menu->height() * 3 / 4); + QPoint high(menu->width() / 2, menu->height() / 4); + for (int i = 0; i < 10 && !shows(itemCount - 1); ++i) { + drag(low, high); + } + QVERIFY(shows(itemCount - 1)); + QVERIFY2(within(onScreen(menu), usable), + qPrintable(describe(onScreen(menu), usable))); + } + + void a_short_menu_at_the_very_top_is_moved_down() { + m_style->setEdgeMargin(60); + m_menu.reset(new QMenu); + m_menu->setStyle(m_style.get()); + m_menu->addAction("Open..."); + m_menu->addAction("Open Recent"); + m_menu->addAction("Save Session"); + m_menu->popup(screen().topLeft()); + QVERIFY(QTest::qWaitForWindowExposed(m_menu.get())); + + QRect usable = m_style->usableArea(m_menu.get()); + QVERIFY2(within(onScreen(m_menu.get()), usable), + qPrintable(describe(onScreen(m_menu.get()), usable))); + QVERIFY(onScreen(m_menu.get()).top() >= screen().top() + 58); + } + + void menus_keep_inside_the_main_windows_safe_area() { + // The offscreen platform has no system bars and no safe area + // margins: a main window short of the screen on every side + // stands for the part the bars leave + QWidget window; + window.setGeometry(screen().adjusted(40, 80, -40, -100)); + window.show(); + QVERIFY(QTest::qWaitForWindowExposed(&window)); + m_style->setSafeAreaWindow(&window); + + QMenu *menu = longMenu(); + QVERIFY(QTest::qWaitForWindowExposed(menu)); + + QRect usable = m_style->usableArea(menu); + QCOMPARE(usable, onScreen(&window) & screen()); + QVERIFY2(within(onScreen(menu), usable), + qPrintable(describe(onScreen(menu), usable))); + + m_menu.reset(); + } + + void a_combo_boxs_list_keeps_clear_too() { + m_style->setEdgeMargin(60); + + // At the top of the screen, as the take box is in the compact + // layout's toolbar, its list longer than the screen is high + QComboBox combo; + combo.setStyle(m_style.get()); + for (int i = 0; i < itemCount; ++i) { + combo.addItem(QString("Take %1").arg(i)); + } + combo.setCurrentIndex(itemCount / 2); + QWidget *list = combo.view()->parentWidget(); + list->setStyle(m_style.get()); + combo.move(screen().topLeft()); + combo.show(); + QVERIFY(QTest::qWaitForWindowExposed(&combo)); + + combo.showPopup(); + QVERIFY(QTest::qWaitForWindowExposed(list)); + + QRect usable = m_style->usableArea(list); + QVERIFY2(within(onScreen(list), usable), + qPrintable(describe(onScreen(list), usable))); + + combo.hidePopup(); + } + + void without_margins_menus_are_placed_as_before() { + // As the desktop has it: the style's own margin, and the screen + TouchMenuStyle plain; + m_menu.reset(new QMenu); + m_menu->addAction("Item"); + QCOMPARE(plain.usableArea(m_menu.get()), screen()); + QCOMPARE(plain.pixelMetric(QStyle::PM_MenuDesktopFrameWidth, nullptr, + m_menu.get()), + plain.baseStyle()->pixelMetric + (QStyle::PM_MenuDesktopFrameWidth, nullptr, m_menu.get())); + } }; #endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index ae9dab5a..52e33e1b 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -16,7 +16,9 @@ #include "TestRealtimePitchTracker.h" #include "TestLatencyShift.h" #include "TestCoverage.h" +#include "TestLogFile.h" #include "TestPinchZoom.h" +#include "TestPopupArea.h" #include "TestStreamLatency.h" #include "TestTakeAudio.h" #include "TestTakeEvents.h" @@ -77,12 +79,24 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestLogFile t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestPinchZoom t; if (runSuite(&t, argc, argv)) ++good; else ++bad; } + { + TestPopupArea t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestStreamLatency t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 9d3bed38..a7c5b31d 100644 --- a/meson.build +++ b/meson.build @@ -1170,7 +1170,9 @@ tony_entry_files = [ tony_core_files = [ 'main/AndroidFiles.cpp', 'main/Coverage.cpp', + 'main/LogFile.cpp', 'main/PinchZoom.cpp', + 'main/PopupArea.cpp', 'main/RealtimePitchTracker.cpp', 'main/SingingTakes.cpp', 'main/StreamLatency.cpp', @@ -1483,7 +1485,9 @@ if system != 'android' 'main/test/TestRealtimePitchTracker.h', 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', + 'main/test/TestLogFile.h', 'main/test/TestPinchZoom.h', + 'main/test/TestPopupArea.h', 'main/test/TestStreamLatency.h', 'main/test/TestTakeAudio.h', 'main/test/TestTakeEvents.h', From 967fb8b2ee7405f935b7b65a961d2874c39efaf1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:45:23 +0000 Subject: [PATCH 180/275] docs: phase a7b done, the evidence behind it, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 34f36661..77604fb5 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -160,7 +160,7 @@ builds happen in the container.) - A5 — Compact touch mode. Done. - A6 — Oboe audio backend. Done. - A7 — Sessions in place on the phone, and fixes from the first phone test. Done. -- A7b — Fixes from the second phone test: menus, the picker, Downloads. +- A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. - A4b — Vertical zoom and scroll by touch. - A8 — Documentation pass. @@ -367,6 +367,12 @@ granted; playback stopping in the background. Did not: `ContentResolver`); map those too. The refusal message names the provider, so that a case still unmapped can be reported; and log the URI. +Later evidence (the user): "File or URL ... could not be opened" came from Open Recent +after the file had been moved; "File does not exist ... content://...primary%3AMusic%2F(vocals) +Avi Kaplan - Peace Somehow.ton" came from svgui's `InteractiveFileFinder`, whose +`QFileInfo::exists()` on the URI Qt's content file engine answers wrongly for a name with +parentheses. + ### A4b — Vertical zoom and scroll by touch The same phone test: the pitch track is tiny, because the pane's frequency range is far @@ -623,3 +629,23 @@ hide() does not end exec()): DontUseNativeDialog. `activeModalWidget()` misses n Left open: not run on a phone. Below Android 11 no in-place sessions. Downloads (`msf:` ids) refused for sessions. Save to Audio Path with copied audio saves into app storage. For A8: port-android.md "Files, storage..." (bundle superseded) and "Permissions and lifecycle". + +### Phase A7b — 2026-09-26 (the agent stopped twice; the lead finished MainWindow.cpp) +Built: `PopupArea` (menus kept inside the safe area, a finger's width from the screen's top +and bottom; TouchMenuStyle applies it), `LogFile` (the logcat lines kept in +`/log/tony.log`, 512 KB and one older part; Help > Save Log... writes a copy through +the save picker), `AndroidFiles::grantedUri()`, `providerOf()`, `pathLookupFor()`, +`mediaStoreUriFor()`, `chooseDownload()`, `hasExtensionIn()`, `usableRecentFiles()`; +`AndroidStorage::pathFor(uri, why)` (static; MediaStore `_data` for `msf:`/`audio:` ids, +name and size for numbered downloads), `displayName()`, `openDocument()` (a descriptor +from the ContentResolver), `removeIfEmpty(uri)`. MainWindow: Tony's own Open picker for +sessions and audio (no type filter; the extension checked after), mapped path used only if +it is a file, otherwise audio copied in through `openDocument()`; every refusal carries +`pickDetails()` (provider, path looked for, why); Open Recent says a moved file is gone, +and the menu lists only files that are there. +Cause found: Qt's content file engine re-encodes '(' and ')' and then has no grant for the +URI, so `QFileInfo::exists()` and `QFile` fail for such names; hence descriptors from the +ContentResolver with the URI exactly as Android wrote it. +Tests seen failing: `the_uri_is_opened_as_android_wrote_it` with `PrettyDecoded` (lead). +Left open: none of it run on a phone. Layer import still goes through svgui's dialog. + From 5f7b2b83e7c0c75c1de7d19e5950f2a8df8f01ad Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:57:35 +0000 Subject: [PATCH 181/275] fix: a note held across the start of a punch-in stays one note Two punch-ins meeting inside a held note left no note after the join: the first's note stopped where its recording did, and the second punch-in's run found the note going on from before the merge window, so the merge by onset added nothing. An old note sounding at the window's start that also ends inside the window now takes the end of the run's note sounding there, cut back as before at any onset the run found inside it. The audio at the window's start has not changed, so the two are one note. It keeps its onset, value and label, and the change is recorded for undo. The dev check for joins (item 10) found it and now passes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 24 ++++++++ docs/calibrate-audio.md | 4 +- docs/takes.md | 27 +++++++-- main/Analyser.cpp | 80 ++++++++++++++++++--------- main/test/TestDevChecks.h | 28 ++++------ main/test/TestRecordWorkflow.h | 86 +++++++++++++++++++++++++++++ main/test/TestSingingAnalysis.h | 79 ++++++++++++++++++++++++++ 7 files changed, 277 insertions(+), 51 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index ec2e9003..f42cf144 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -526,3 +526,27 @@ The next phase must know: the margin is the most frames received across one look an OS stall between those reads (not the event loop) widens it for the whole take and can leave no look in any gap: now "not judged", not a Fail. Left open: `test-tony-dev` 13 tests, about 228 s (two runs of about 22 s added). + +### Phase C2c — 2026-09-26 +Built: `Analyser::mergeRangedAnalysis()`: an old note sounding at W's start and ending inside +W takes the end of the run's note sounding there (onset before W), that end found as for an +added note (`newNoteEnd()`: `endBeyondRun` if the run's end cut it off, cut back at the next +old onset after W), then cut back to the first added onset as before. Recorded in +`m_rangedNotesChange` like the old cut, so undo restores the old note. Tests: +`TestRecordWorkflow::join_inside_a_held_note_keeps_one_note` (Record into Selection +[0.5, 1.5] then [1.5, 2.5] s on one held tone: one note; undo gives the first's note back +exactly, redo the one note); `TestSingingAnalysis::ranged_join_inside_a_note`, rows "the same +note going on" (one note) and "a new note from the join" (two, the first ending at J). +`TestDevChecks`: item 10's `QEXPECT_FAIL` gone; `dev_checks_fail_with_the_round_trip_off` +now requires item 10 to pass (it allowed either). +Choices: "the same note" = both sounding at W's start, where the audio has not changed. No +pitch tolerance (the values are medians over different stretches; a pitch the user corrected +must not stop the note). Only an old note ending inside W: one running past W keeps its end +(`ranged_keeps_a_note_across_the_window_edge` compares it exactly). A run's note beginning +before W with no old note sounding there is still not added: before W the models' notes stand. +Seen failing: the same-note cases of both new tests before the fix (the first's note alone, +0.517-1.509 s); the "new note" row with a naive join (an old note carried over a new note +beginning within 4 hops of its end, and the cut at the first added onset off). +Left open (`docs/takes.md`, known limits): a note running on past W keeps its old end where +the new audio stopped it inside W; a note whose onset in the run and in the models lie a hop +or two either side of W's start is lost (pre-existing, found by reading). diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index de5c2cb2..6034382d 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -115,7 +115,7 @@ The numbers are those of `docs/manual-checklist.md` before that merge. | 7 | Record from a position; overwrite question | **Automated** (placement) | Placement as in 1. Outside the new range, the take's audio is bit-identical and pitch and notes are unchanged beyond ±0.25 s. The question, No, and "Don't ask again" stay with the app suite (`record_over_existing_question`) and a glance. | | 8 | Cursor from P, pane follows, all in one place | **Measured** | *Automated:* the cursor is at S when the take starts, and inside the visible range throughout. *Reported:* the dot-to-cursor offset. *Eyes:* the rest. | | 9 | Stop on a 4-minute song | **Automated** | A generated 4-minute reference, punch-in near the end. Time from Stop to new pitch, against a threshold. Pitch outside the range unchanged, which proves the ranged path ran. | -| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. *Found in C2:* the note fails on today's code. The second punch-in's analysis starts 0.5 s before the join, inside the tone, so its note begins before the merge window and is dropped; the first's note ends at the join. A whole-take analysis gives one note. | +| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. *Found in C2:* the note failed: the second punch-in's analysis starts 0.5 s before the join, inside the tone, so its note began before the merge window and was dropped, and the first's note ended at the join. A whole-take analysis gives one note. *Fixed in C2c:* an old note sounding at the merge window's start and ending inside it takes the end of the run's note sounding there (`docs/takes.md`). | | 11 | Is 3 s right, is the countdown readable | **Manual** | A judgement. | | 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. *Found in C1c:* on a noiseless loopback the earlier take is silent wherever the reference is, so a take played out shows in no gap; the app test gives the fake a −60 dBFS noise floor, as a room gives a real mic. | | 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown counts only the 1 s there is (*found in C1c:* plus the round trip, by design, so it starts at 2; for the instant before the round trip is known it shows 1), placement is right. | @@ -333,7 +333,7 @@ marked "Done" when it is committed. 5. **C2** Long song and joins: items 9 and 10. Done. - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). Done. - **C2c** The notes merge keeps one note across a join: the defect item 10 found (the - user chose to fix it on this branch). + user chose to fix it on this branch). Done. 6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. 7. **Release build** by the lead (§8, "Release builds must stay clean"). 8. **D** Docs, from the code and the phase log: diff --git a/docs/takes.md b/docs/takes.md index 7a0db5d0..bfb8fd5a 100644 --- a/docs/takes.md +++ b/docs/takes.md @@ -119,10 +119,21 @@ as a transform's output never was) and `rangedAnalysisMerged()` then what the grid alignment is for. - **Pitch**: old events in W go, new events in W are added. - **Notes, by onset**: old notes with onset in W go; new notes with onset in W are added. - An old note from before W is cut back only if a new note overlaps it. A new note that - would overlap an old note starting at or after the end of W is cut back to that onset. A - new note cut off by the *end of the run* (ends within four hops of it) takes the end of - the old note that ran past, if there is one. + A new note that would overlap an old note starting at or after the end of W is cut back + to that onset. A new note cut off by the *end of the run* (ends within four hops of it) + takes the end of the old note that ran past, if there is one. +- **A note running into W from before it** keeps its onset. If it also *ends inside W*, it + takes the end of the run's note that is sounding at W's start (that end found as for a + new note, above). The audio at W's start has not changed, so two notes sounding there + are one note; this is what keeps one note through a join inside a held note, where the + old note stops where the old recording did and the run's note began before W. No pitch + tolerance: the two values are medians over different stretches, and a pitch the user + corrected by hand must not stop the note. One that runs on past W keeps its end, in + unchanged audio. Either way it is cut back to the onset of a new note that starts inside + it, so it never runs over a note the run found in W. +- A run's note that begins before W is otherwise **not added**, even where no old note + sounds at W's start: before W the notes the models hold stand (the old analysis had more + context, or the user deleted that note). - **pYIN stamps one frame of every run twice** in fixed-lag mode: the last frame of `process()` comes again first from `getRemainingFeatures()`, 100 hops before the end. Whole-file tracks have it too, harmlessly. In a ranged run it falls inside W, so the @@ -256,8 +267,12 @@ Things to know, none of which stops the feature being used. See also next ordinary save brings them back. - About 11 ms of pitch at the very start of a coverage range cannot be produced (pYIN's first two hops). -- One sung note can still become two when a note runs into W from before it *and* its - audio changed. Deliberate trade for not splitting notes in unchanged audio. +- One sung note becomes two only where the run finds an onset inside it (a re-attack, + say). A note that runs on past W keeps its old end even where the new audio stopped it + inside W, up to the next onset the run found. +- A note whose old onset lies just inside W and whose onset in the run lies just before + it (a hop or two either side of W's start) is lost: the old one goes with W, and the + run's is not added. Found by reading the merge, not seen. - A splice or erase that succeeded on disk but could not be shown is not rolled back; the user gets a dialog naming the file. - Playing a wave model with a **positive** start frame plays up to a block early and diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 15faf8d1..c0c0aabd 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -1337,44 +1337,74 @@ Analyser::mergeRangedAnalysis() } } - for (const Event &e : oldNotes) { - sv_frame_t f = e.getFrame(); - if (f >= wFrom && f < noteTo) { - notes->remove(e); - m_rangedNotesChange.removed.push_back(e); - } else if (f < wFrom && firstAdded >= 0 && - f + e.getDuration() > firstAdded) { - // A note that runs into the window from before it is left as - // it is unless one of the new notes starts inside it, when it - // is cut back to that onset. Where the audio has not - // changed the run finds that note going on from before the - // window, has nothing to add inside it, and the one note - // stays one note - notes->remove(e); - notes->add(e.withDuration(firstAdded - f)); - m_rangedNotesChange.removed.push_back(e); - m_rangedNotesChange.added.push_back(e.withDuration(firstAdded - f)); - } - } - for (Event e : adding) { + // Where a new note of the run ends in the models + auto newNoteEnd = [&](const Event &e) { + sv_frame_t end = e.getFrame() + e.getDuration(); // A note that the end of the run cut off goes on to where the // old note it belongs to ended. A few hops of slack: the run's // last note ends within a block or so of where it stopped if (endBeyondRun > 0) { - sv_frame_t end = e.getFrame() + e.getDuration(); sv_frame_t slack = 4 * analysisStepSize; if (end > m_rangedEnd - slack && end < m_rangedEnd + slack && endBeyondRun > end) { - e = e.withDuration(endBeyondRun - e.getFrame()); + end = endBeyondRun; } } // The far edge the same way round: an old note that begins at or // after the window keeps its onset, and a new note that would run // over it is cut back - if (nextOldOnset > e.getFrame() && - e.getFrame() + e.getDuration() > nextOldOnset) { - e = e.withDuration(nextOldOnset - e.getFrame()); + if (nextOldOnset > e.getFrame() && end > nextOldOnset) { + end = nextOldOnset; + } + return end; + }; + + // The new note sounding where the window begins. It began before + // the window, so it is not added, but the audio there has not + // changed: an old note sounding at the same frame is the same note, + // and the run knows better where it ends + sv_frame_t carriedEnd = -1; + for (const Event &e : newNoteEvents) { + if (e.getFrame() < wFrom && e.getFrame() + e.getDuration() > wFrom) { + carriedEnd = newNoteEnd(e); + break; } + } + + for (const Event &e : oldNotes) { + sv_frame_t f = e.getFrame(); + if (f >= wFrom && f < noteTo) { + notes->remove(e); + m_rangedNotesChange.removed.push_back(e); + continue; + } + if (f >= wFrom) continue; + + // A note that runs into the window from before it keeps its + // onset. If it also ends inside the window, where it ends is the + // run's to say: it takes the end of the new note sounding at the + // start of the window, so that a note held across the start of a + // punch-in is one note, not the old one stopping where the old + // recording did. One that runs on past the window keeps its end, + // in audio that has not changed. Either way it is cut back to the + // onset of a new note that starts inside it + sv_frame_t end = f + e.getDuration(); + sv_frame_t to = end; + if (carriedEnd > 0 && end > wFrom && end < noteTo) { + to = carriedEnd; + } + if (firstAdded >= 0 && to > firstAdded) { + to = firstAdded; + } + if (to != end) { + notes->remove(e); + notes->add(e.withDuration(to - f)); + m_rangedNotesChange.removed.push_back(e); + m_rangedNotesChange.added.push_back(e.withDuration(to - f)); + } + } + for (Event e : adding) { + e = e.withDuration(newNoteEnd(e) - e.getFrame()); notes->add(e); m_rangedNotesChange.added.push_back(e); } diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index 6b69923d..42944043 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -732,24 +732,16 @@ private slots: QVERIFY2(!joins->message.contains(part), describe()); } - // And one note through the join, which the take does not have: - // the second punch-in's analysis starts 0.5 s before the join, - // inside the tone, so its note begins before the merge window and - // is not merged in, while the first's note, which ends at the - // join, stays. The tone after the join has no note - const bool joinsPass = joins->verdict == CheckResult::Verdict::Pass; - QEXPECT_FAIL("", "one note should run through a join inside a held " - "tone, but the notes merge by onset keeps the first " - "punch-in's note, ending at the join, and drops the " - "second's, which begins before the merge window " - "(docs/takes.md, \"Notes, by onset\")", Continue); - QVERIFY2(joinsPass, describe()); + // And one note through the join. The second punch-in's analysis + // starts 0.5 s before the join, inside the tone, so its note begins + // before the merge window: the first's note, which ended at the + // join, is that note going on and takes its end + QVERIFY2(joins->verdict == CheckResult::Verdict::Pass, describe()); QVERIFY(QFileInfo(m_report.reportPath).fileName() == "DevChecks.txt"); QVERIFY(TakesFile::isInFolder(reportDirectory(), m_report.reportPath)); QCOMPARE(lastReportLine(), - QString("Totals: %1 passed, %2 failed, 1 measured, 0 skipped") - .arg(joinsPass ? 10 : 9).arg(joinsPass ? 0 : 1)); + QString("Totals: 10 passed, 0 failed, 1 measured, 0 skipped")); QVERIFY(m_report.sessionPath != ""); QCOMPARE(m_window->sessionFile(), m_report.sessionPath); @@ -841,12 +833,12 @@ private slots: check(9)->verdict == CheckResult::Verdict::Skipped && check(9)->message == "The long song was left out of this " "run.", describe()); - const bool joinsPass = - check(10) && check(10)->verdict == CheckResult::Verdict::Pass; + QVERIFY2(check(10) && + check(10)->verdict == CheckResult::Verdict::Pass, + describe()); QCOMPARE(lastReportLine(), QString("Totals: %1 passed, %2 failed, 1 measured, 1 skipped") - .arg((dotsPass ? 4 : 3) + (joinsPass ? 1 : 0)) - .arg((dotsPass ? 4 : 5) + (joinsPass ? 0 : 1))); + .arg(dotsPass ? 5 : 4).arg(dotsPass ? 4 : 5)); } // The loopback heard a second time, 50 ms later at half the level, as diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 741d465d..d991f8af 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1708,6 +1708,92 @@ private slots: verifyPlaySourceClean(); } + // Two punch-ins that meet at J inside a note held through both. The + // first's note ends at J, where its recording stopped; the analysis + // of the second starts half a second before J, inside the note, so + // the note it finds begins before the window the merge replaces. It + // is the same note going on, and the take has one note through J. + // Undoing the second punch-in gives the first's note back exactly + void join_inside_a_held_note_keeps_one_note() { + FakeAudioIO::Config config; + config.input = tone(lowHz, 3.0); + makeWindow(config); + m_window->setPlayReferenceWhileRecording(false); + m_window->setRecordIntoSelection(true); + openReference(writeWav(tone(highHz, 3.5))); + if (QTest::currentTestFailed()) return; + + const sv::sv_frame_t P = sv::sv_frame_t(0.5 * rate); + const sv::sv_frame_t J = sv::sv_frame_t(1.5 * rate); + const sv::sv_frame_t E = sv::sv_frame_t(2.5 * rate); + + // Record into Selection, so that each punch-in stops exactly at + // the end of its selection and the two meet at J + auto punchIn = [this](sv::sv_frame_t from, sv::sv_frame_t to) { + m_window->clearSelections(); + m_window->selectRange(from, to); + m_window->seekTo(from); + startTake(); + if (QTest::currentTestFailed()) return; + QTRY_VERIFY_WITH_TIMEOUT + (!m_window->recordTarget()->isRecording(), 5000); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + }; + auto describe = [](const sv::EventVector ¬es) { + QStringList out; + for (const auto &e : notes) { + out << QString("%1 to %2 s at %3 Hz") + .arg(double(e.getFrame()) / rate, 0, 'f', 3) + .arg(double(e.getFrame() + e.getDuration()) / rate, + 0, 'f', 3) + .arg(double(e.getValue()), 0, 'f', 1); + } + return "[" + out.join(", ") + "]"; + }; + + punchIn(P, J); + if (QTest::currentTestFailed()) return; + TakeSnapshot first = snapshotTake(); + QCOMPARE(int(first.coverage.size()), 1); + QCOMPARE(first.coverage[0], Coverage::Range(P, J)); + + // The first punch-in's note runs to the join + QVERIFY2(first.notes.size() == 1, qPrintable(describe(first.notes))); + const sv::Event held = first.notes[0]; + QVERIFY2(std::llabs(held.getFrame() - P) <= 4 * hop && + std::llabs(held.getFrame() + held.getDuration() - J) + <= 4 * hop, qPrintable(describe(first.notes))); + + punchIn(J, E); + if (QTest::currentTestFailed()) return; + TakeSnapshot second = snapshotTake(); + QCOMPARE(int(second.coverage.size()), 1); + QCOMPARE(second.coverage[0], Coverage::Range(P, E)); + + // One note, the first's carried on through J to the end of the + // second punch-in: its onset, pitch and label as they were + QVERIFY2(second.notes.size() == 1 && + second.notes[0].getFrame() == held.getFrame() && + second.notes[0].getValue() == held.getValue() && + second.notes[0].getLabel() == held.getLabel() && + std::llabs(second.notes[0].getFrame() + + second.notes[0].getDuration() - E) <= 4 * hop, + qPrintable(QString("the notes after the second punch-in " + "are %1; the first's was %2, the join " + "is at %3 s") + .arg(describe(second.notes)) + .arg(describe(first.notes)) + .arg(double(J) / rate, 0, 'f', 3))); + + // Undo gives the first punch-in's note back, exactly as it was, + // and redo the one note again + QCOMPARE(undoOnce(), QString("Record Singing")); + verifyTakeMatches(first); + if (QTest::currentTestFailed()) return; + QCOMPARE(redoOnce(), QString("Record Singing")); + verifyTakeMatches(second); + } + // Recording again while the analysis of the range just recorded is // still running. That analysis is lost -- the swap releases the models // it was to be merged into -- so the analysis that follows has to diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 02129077..a41f9d23 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -1030,6 +1030,85 @@ private slots: .arg(describeNotes(now)).arg(to).arg(wasEnd))); } + void ranged_join_inside_a_note_data() { + QTest::addColumn("sameNote"); + QTest::newRow("the same note going on") << true; + QTest::newRow("a new note from the join") << false; + } + + void ranged_join_inside_a_note() { + // Two punch-ins meeting at J, as MainWindow analyses them: the + // first's range alone, its coverage ending at J, then the second's + // with the coverage grown to take it in. The second's run starts + // half a second before J, inside the first's note, so the note it + // finds there begins before the merge window. When the singer + // holds the note through J, that is the first's note going on, and + // the two are one. When a different note begins at J, they are two: + // the first's note must not be carried on over the new one + QFETCH(bool, sameNote); + + std::vector data; + appendSilence(data, 0.2); + auto held = tone(singingHz, 1.3); // 0.2 to 1.5 s + data.insert(data.end(), held.begin(), held.end()); + auto then = tone(sameNote ? singingHz : referenceHz, 1.3); + data.insert(data.end(), then.begin(), then.end()); // to 2.8 s + appendSilence(data, 0.3); + + const sv::sv_frame_t P = frameAt(0.5), J = frameAt(1.5), + E = frameAt(2.5); + sv::sv_frame_t wFrom, wTo; + mergeWindow(J, E, P, E, wFrom, wTo); + QVERIFY(wFrom > P && wFrom < J); + + Analyser analyser(Analyser::SecondaryColors); + setUpEmpty(analyser, addSingingModel(data)); + if (QTest::currentTestFailed()) return; + + QSignalSpy done(&analyser, SIGNAL(initialAnalysisCompleted())); + QCOMPARE(analyser.analyseRange(P, J, P, J), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + // The first punch-in's note, running from near P to J + sv::EventVector first = noteEvents(analyser); + QVERIFY2(first.size() == 1 && + std::abs(first[0].getFrame() - P) <= 4 * hop && + std::abs(first[0].getFrame() + first[0].getDuration() - J) + <= 4 * hop, qPrintable("first: " + describeNotes(first))); + + done.clear(); + QCOMPARE(analyser.analyseRange(J, E, P, E), QString()); + waitForRange(analyser, done); + if (QTest::currentTestFailed()) return; + + sv::EventVector now = noteEvents(analyser); + QString what = QString("the first punch-in's note %1 and the second's " + "(merge window %2 to %3) became %4") + .arg(describeNotes(first)).arg(wFrom).arg(wTo) + .arg(describeNotes(now)); + + // The first's note keeps its onset and pitch either way + QVERIFY2(!now.empty() && now[0].getFrame() == first[0].getFrame() && + now[0].getValue() == first[0].getValue(), qPrintable(what)); + sv::sv_frame_t end = now[0].getFrame() + now[0].getDuration(); + + if (sameNote) { + // and runs on through J to the end of the coverage + QVERIFY2(now.size() == 1 && std::abs(end - E) <= 4 * hop, + qPrintable(what)); + } else { + // and still ends at J, where the new note begins, sung at its + // own pitch + QVERIFY2(now.size() == 2 && std::abs(end - J) <= 4 * hop && + now[1].getFrame() >= end && + std::abs(now[1].getFrame() - J) <= 4 * hop && + std::abs(TestSignals::centsBetween + (now[1].getValue(), referenceHz)) < 50.0, + qPrintable(what)); + } + } + void ranged_at_the_edge_of_coverage() { // Where the caller's coverage limit clips an edge of the run there // is nothing beyond it but silence and no context worth keeping, From 84f85a18bfff9d4a54d890ef7f5a3b954791e359 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 09:58:11 +0000 Subject: [PATCH 182/275] docs: calibrate audio work orders, C3 refined Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 45 +++++++++++++++++++++++++---- 1 file changed, 39 insertions(+), 6 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index f42cf144..c6d98362 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,7 +194,7 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`), C2c (`5f7b2b8`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). @@ -388,11 +388,44 @@ and `TestSingingAnalysis.h` for "note"); C2's log entry. ### C3 — Retire `test-tony-device` -To be refined by the lead after C2. Outline: remove `TestRealDevice.h`, -`tony-device-check.cpp` and its target once everything it checks is in the dev run (its -"no input does no harm" case included, as an app test if not already covered); update the -docs that name it in the same commit (`AGENTS.md`, `building.md`, `testing.md`, and -`manual-checklist.md` section 1, which becomes "Calibrate Audio with the dev checks"). +Read also: `main/test/TestRealDevice.h` whole (607 lines: what is being retired); +`docs/manual-checklist.md` whole; `docs/testing.md`'s table and the passages naming +`test-tony-device`; `main/dev/DevChecks.h` (the items and the report); the runner's +`Recording` step in `AudioCheckRunner.cpp`. + +- **Check coverage first**, test by test of `TestRealDevice.h`, and write the mapping into + your log entry: `device()` → the report header; `takes_line_up_with_the_reference` → + items 1 and 2; `nothing_of_the_take_comes_back_out` → item 4; + `stop_is_quicker_than_a_whole_song` → item 9; `live_dots_were_drawn` → items 3 and 5; + `no_input_does_no_harm` → see below. Anything it checks that the dev run does not: + report it, and port it if small. +- **A device that opens but delivers nothing** (its `no_input_does_no_harm` and the + QFAIL "the device opened, but … delivered no input at all"): the runner today waits + out its `Recording` step and ends with "A take did not stop at the end of its range", + which misleads. Make the runner end a take that has received no frames at all by the + time the take should be over, with a message that says the device delivered no input, + through the Stop path, leaving no take and no harm; and an app test in + `TestAudioCheck` with a fake device that opens and never calls back (see how + `FakeAudioIO` and `TestMainWindow` make one; add a field if needed). A device that + cannot be opened at all is covered already (`check_fails_without_a_device`). +- **Remove** `main/test/TestRealDevice.h`, `main/test/tony-device-check.cpp` and the + `test-tony-device` target in `meson.build`, and `TestMainWindow`'s + `setUseRealDevice()` and what serves only it, if nothing else uses them. +- **Docs, in the same change** (AGENTS.md's "Keeping the docs true"): + - `AGENTS.md` and `docs/building.md`: the build command without `test-tony-device.exe`; + - `docs/testing.md`: its table row and paragraph; + - `docs/manual-checklist.md` section 1 becomes "The device check: Calibrate Audio with + the dev checks": in a development build, Playback ▸ Calibrate Audio… with the + checkbox on; the earcup against the microphone; the report file `DevChecks.txt` in + Tony's application data folder; what each item's numbers mean in a line each; what + fails on MME today (items 1, 2 and 10's offsets, spec §8) and why. Keep the list of + what it covers and the dated notes that still hold; drop what was only about the + executable (`TONY_DEVICE_CHECK_FAKE`: `TestDevChecks` checks the check now). + - `main/dev/DevChecks.h`'s comments that name `test-tony-device`. + - Leave `docs/calibrate-audio*.md` beyond your log entry and spec §7's "Done": phase D + brings them up to date. +- **Tests:** the new `TestAudioCheck` test, seen failing before the runner change. The + three suites green; `test-tony-device` no longer builds or exists. ### Lead — release build From b899fe809bfd3e89c91e745402cad0900fbc3bb7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 10:21:00 +0000 Subject: [PATCH 183/275] docs: the work order for phase a7c, android's decoders for m4a the user's desktop session with an m4a reference opened on the phone but could not read the audio: the android build has no aac decoder. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 77604fb5..4063f2b4 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -162,6 +162,7 @@ builds happen in the container.) - A7 — Sessions in place on the phone, and fixes from the first phone test. Done. - A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. - A4b — Vertical zoom and scroll by touch. +- A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -386,6 +387,31 @@ wider than the singing, and pinch zooms only the time axis. (`MainWindow::editDisplayExtents()`) uses. Check the scale (log for pitch), the limits, and what the other layers in the pane do when it changes. +### A7c — M4A/AAC through Android's decoders; no autosave of an incomplete session + +The user's third finding (2026-09-26): a desktop session whose reference is +"(vocals) Avi Kaplan - Peace Somehow.m4a" opens on the phone (the reference is found by name +next to the .ton, the stored path being `c:/Users/.../OneDrive/Singing/...`), but reports +"Incomplete session loaded": the Android build has no AAC decoder. On Windows, M4A is read +by bqaudiostream's `MediaFoundationReadStream`; on Android only WAV (sndfile), MP3 (mad) and +Opus (opusfile) are read. + +- An `AudioReadStream` over the NDK's `AMediaExtractor` and `AMediaCodec` (libmediandk, + API 21+), in `main/`, compiled for Android only, registered with bqaudiostream's factory + the way its own readers are (`static AudioReadStreamBuilder<...>` with a URI and the + extensions; see `bqaudiostream/src/MediaFoundationReadStream.cpp` ~84): m4a, aac, mp4, + and whatever else the phone's decoders take that Tony cannot read already (flac, ogg, + opus are candidates; say which, and what wins when two readers claim an extension). + svcore's `BQAFileReader` asks that factory, so no fork changes. The registration must + survive linking (`link_whole` into the application library keeps it; check). +- Decoded to float at the file's own rate and channels, as the other readers give; Tony + resamples on load (A1's notes). Seeking is not needed if the other readers do not seek. +- Pure parts (format bookkeeping, sample conversion) testable on the desktop; the decoder + itself only on the phone. +- A session that loaded incomplete (audio it names could not be read) is never saved + without the user asking: not by the save on suspend (A7), and Save warns first. Find + where svapp reports "Incomplete session loaded" and how Tony can know it happened. + ### A8 — Documentation pass - Bring the docs pages up to date from the code and the log: building.md (the container From e03f9d2a710e84874960b4369ad644242865da11 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 10:34:21 +0000 Subject: [PATCH 184/275] test: retire test-tony-device, whose checks the dev run now makes Everything the real-device executable measured is in Calibrate Audio's dev run: latency and several recordings in one take, live dots, nothing back out, the mic's input, Stop on a long song, and the device's reported figures. Its one other case, a device that opens but delivers nothing, now ends the audio check with a message that says so, through the Stop path and leaving no take, instead of waiting out the step and blaming the take for not stopping. The manual checklist's device check is now Calibrate Audio with the dev checks: how to run it, what each item's numbers mean, and what fails on MME today and why. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- AGENTS.md | 2 +- docs/README.md | 2 +- docs/building.md | 2 +- docs/calibrate-audio-work-orders.md | 28 +- docs/calibrate-audio.md | 2 +- docs/manual-checklist.md | 118 ++++-- docs/open-points.md | 5 +- docs/testing.md | 15 +- main/AudioCheckRunner.cpp | 25 +- main/AudioCheckRunner.h | 16 + main/dev/DevChecks.h | 6 +- main/test/FakeAudioIO.h | 7 +- main/test/TestAudioCheck.h | 56 +++ main/test/TestMainWindow.h | 18 +- main/test/TestRealDevice.h | 607 ---------------------------- main/test/tony-device-check.cpp | 59 --- meson.build | 31 -- 17 files changed, 228 insertions(+), 771 deletions(-) delete mode 100644 main/test/TestRealDevice.h delete mode 100644 main/test/tony-device-check.cpp diff --git a/AGENTS.md b/AGENTS.md index 365d0102..6417f46d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,7 +27,7 @@ From **Git Bash** (the usual agent shell). `build.bat` does not run from sh. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe test-tony-device.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log ``` diff --git a/docs/README.md b/docs/README.md index 2ced9f0c..84455fb4 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,5 +18,5 @@ methods, and they do not tell the story of fixed bugs. | [takes.md](takes.md) | Takes: decisions, the audio swap, ranged analysis and merge, undo, the coverage strip, files and sessions, limitations | | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | -| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | +| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — little of it tried yet | | [calibrate-audio.md](calibrate-audio.md) | Plan, not built: a Calibrate Audio button that measures the round trip through a speaker-to-mic loopback, and dev checks that automate most of the manual checklist | diff --git a/docs/building.md b/docs/building.md index 15d082e8..97f9f79d 100644 --- a/docs/building.md +++ b/docs/building.md @@ -24,7 +24,7 @@ From PowerShell its output is safe to capture: `.\build.bat *> tmp\build.log`. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe test-tony-device.exe > tmp/build.log 2>&1 +ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log tail -20 tmp/build.log ``` diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index c6d98362..68a17083 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -54,7 +54,7 @@ MinGW setup. - Build: cd /home/user/tony - ninja -j 4 -C build_linux tony test-tony-core test-tony-app test-tony-dev test-tony-device pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log + ninja -j 4 -C build_linux tony test-tony-core test-tony-app test-tony-dev pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log Search the log for `error:`; never read it whole. meson reconfigures by itself after `meson.build` changes. Targets have no `.exe`. @@ -583,3 +583,29 @@ beginning within 4 hops of its end, and the cut at the first added onset off). Left open (`docs/takes.md`, known limits): a note running on past W keeps its old end where the new audio stopped it inside W; a note whose onset in the run and in the models lie a hop or two either side of W's start is lost (pre-existing, found by reading). + +### Phase C3 — 2026-09-26 +Built: `test-tony-device` gone (`TestRealDevice.h`, `tony-device-check.cpp`, its target; +`TestMainWindow`'s `setUseRealDevice()`, `haveAudioDevice()`, `haveRecordingDevice()`, used by +it alone). `AudioCheckRunner::deliveredNothing()`: not one frame received (the count restarts +with each take) `kNoInputTimeoutMs` (2 s) past the take's lead-in and range: the take is +stopped through the Stop path (`finishSingingTake()` finds nothing to use, so no take) and the +run ends "The audio device delivered no input ...". `FakeAudioIO::Config::neverCallsBack`. +`TestAudioCheck::check_ends_when_the_device_delivers_nothing` (not before length + 2 s − one +poll; no take; the next file analysed). Docs: AGENTS.md, building.md, testing.md, checklist +§1 rewritten, `DevChecks.h`; one line each in open-points.md and docs/README.md. +Coverage: `device()` → report header, its "can record" → the runner's "The take did not +start"; `record_the_reference_through_the_air` → stage 1 (near 61 and 151 s), channel levels +→ item 5, a range per recording → "A take was not added"; `takes_line_up_with_the_reference` +→ items 1, 2 (±2 ms against its ±10 ms and 5 ms; "match < 0.1" → NoSignal); +`nothing_of_the_take_comes_back_out` → 4; `stop_is_quicker_than_a_whole_song` → 9 (same 0.5); +`live_dots_were_drawn` → 3, 5 (same > 10); `no_input_does_no_harm` → the new ending and test, +and `check_fails_without_a_device`. Not in the dev run (not ported): a take with no lead-in and +not into a selection, stopped by hand; "no dialog" over every take (item 14: stages 3, 4 only); +with no device, the "Couldn't open audio device" warning per file opened (a dated note now). +Choices: not a fixed time after `record()`: a device slow to start (a Bluetooth headset +switching to its mic) is never taken for a dead one, and a working take has stopped itself by +then. One frame leaves the old limit in charge. No `TakeLatency` pushed (no take). +Seen failing: the new test before the change ("did not stop", 14 s); with the rule cut to 2 s +after `record()`, its timing (ended 2104 ms into a 3000 ms take). +Found: items 7 and 13 judge placement as 1 and 2 do (`offsetsOf()`): on MME they fail too. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 6034382d..a4fb84b1 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -334,7 +334,7 @@ marked "Done" when it is committed. - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). Done. - **C2c** The notes merge keeps one note across a join: the defect item 10 found (the user chose to fix it on this branch). Done. -6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. +6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. Done. 7. **Release build** by the lead (§8, "Release builds must stay clean"). 8. **D** Docs, from the code and the phase log: - `manual-checklist.md`: an automated item keeps its text and gets "*automated: diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 385338d9..39dca241 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -8,52 +8,98 @@ be on this page is now checked automatically: screen. Each of its tests begins with a comment `// Checklist:` quoting the item it replaced; `grep -n "Checklist:" main/test/*.h` lists them. With `TONY_TEST_SHOT_DIR` set it saves what it looked at as PNG files. -- **`test-tony-device`** checks the real device: section 1. +- **Calibrate Audio**, carrying on into the development checks, tries the real device in + the app itself: section 1. What is left needs a real device, real ears, or a decision. When an item has been checked, note the date and the result next to it; when a change touches an area, the items of that area are what to ask the user to try. Launch with `.\build.bat run`. -## 1. The device check +## 1. The device check: Calibrate Audio with the dev checks Once per machine, and again for each output device sung with (Bluetooth headphones have a latency of their own). It covers: latency on this machine; several recordings in one take, each in time; nothing of the take coming back out of the speakers; how long Stop takes on a four-minute song; live dots from whichever input the microphone is on; and a device that -records nothing. - -It plays a four-minute reference of a tone and clicks, records it through the air at two -places into one take, and measures where the clicks landed in the take. - -1. In Tony, choose the devices under **Playback > Audio Output Device** and **Audio Input - Device** (or leave the system default). The check reads that choice, and nothing else, - from Tony's settings. -2. Speakers at a moderate volume and the microphone where it hears them; with headphones, - hold an ear cup against the microphone. A quiet room. It takes about a minute. -3. From Git Bash: - - ```sh - export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" - ninja -j 3 -C build_mingw test-tony-device.exe > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log - cd build_mingw && mkdir -p ../tmp/tl - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-device.exe > ../tmp/test.log 2>&1; echo "exit:$?" - grep -a "^FAIL\|^ Loc\|^QINFO\|^Totals" ../tmp/tl/TestRealDevice.txt - ``` - -4. Read the `QINFO` lines, one per recording: `clicks +x ms from the reference` (within - ±10 ms passes; the sign says late or early), `match` (below 0.1 the microphone did not - hear the speakers), `stop took`, `live dots`, and `channels (dB)`, which shows which - input the microphone is on. With a stereo interface, run it with the microphone on - input 2: the dots must still be there. - -`TONY_DEVICE_CHECK_FAKE=n` runs it with no hardware, on the fake device with its output fed -back into its input `n` frames later than it reports: for checking the check. - -Cloud run, 2026-09-25 (Linux, no sound card): with no device at all Record does no harm and -the next file is analysed (the "Couldn't open audio device" warning comes back once per file -opened); with a device that opens but delivers nothing the take is dropped quietly, no -harm. On the fake: +0.0 ms at both places with `n = 0`, and +50.0 ms, failing, with -`n = 2205`. **Not yet run on real hardware.** +records nothing. Besides those: recording over part of a take, the lead-in, the pre-roll +near the start, two takes meeting inside a note, and Record into Selection stopping by +itself. + +It calibrates, then records test references of sweeps and tones through the air, placing +the takes with the round trip it has just measured, for the run only: a four-minute song +with two takes far apart, then a shorter reference with punch-ins, a re-recording over one +of them, a take near the start and two takes that meet. It finds each sweep in the take's +file, and compares the take before and after each punch-in. + +1. A development build: any build type but `release` (`build.bat` builds + `debugoptimized`). +2. Choose the devices under **Playback > Audio Output Device** and **Audio Input Device** + (or leave the system default): the run records with them, as any take does. +3. With wired headphones, hold one earcup against the microphone, off your ears; with + speakers, a moderate volume and the microphone where it hears them. A quiet room. +4. **Playback > Calibrate Audio...**, with **Run the dev checks after calibrating** on (it + is by default), then **Start**. A few minutes; leave the window alone meanwhile (Cancel + stops the run). The dev checks run only after a calibration that can be used (verdict + Ok or Unsteady). No signal, or a fading one, means the microphone did not hear the + sweeps, or Windows' audio enhancements took them out. The stored latency changes only + through **Use this latency**. +5. The report is on the dialog's result page and in `DevChecks.txt` in Tony's application + data folder (`%APPDATA%\sonic-visualiser\Tony` on Windows). The test session is saved + beside it in a `dev-checks-` folder and left open, to be looked at and played. + +The report's header names the devices, the audio drivers built in, the playback and record +latencies the device reports, and the round trip used. Then, item by item: + +- **1** `latency_on_this_machine`: where each sweep of each punch-in landed against the + reference, in ms (+ is late), within ±2 ms; the same after saving and reopening. +- **2** `several_phrases_in_one_take`: each punch-in's median offset and measured start + gap; every punch-in of one take within ±2 ms. +- **3** `live_dots`: the dots drawn in each punch-in (more than 10, lying on the reference's + tones), and how far they trail the cursor, median and spread. +- **4** `nothing_of_the_take_in_the_speakers`: a second arrival of the sweeps (the input + played back out and heard again), and the loudest output while the reference is silent; + it passes with neither. +- **5** `mic_on_input_2`: each input's peak, and which one carries the microphone. Judged + only when that is input 2, where the dots must still be drawn; measured otherwise. With a + stereo interface, run it once with the microphone on input 2. +- **7** `record_from_a_position`: a re-recording's placement; the take's audio outside its + range the same bit for bit, its pitch and notes beyond ±0.25 s unchanged. +- **9** `stop_on_a_long_song`: the whole song's analysis time, and each punch-in's time + from Stop to its pitch merged, which must be under half of it. +- **10** `the_joins`: where two punch-ins meet inside a held tone, the step in the samples + (dB), the largest gap in the pitch, and one note across the join. +- **12** `nothing_heard_or_changed_in_the_lead_in`: the output in the silent gaps of a + re-recording's lead-in, and the take before the punch-in unchanged; "not judged" when the + window stalled over every gap. +- **13** `pre_roll_near_the_start`: a punch-in at 1 s asking for a 3 s pre-roll: playback + from the song's start and never before it, a countdown no longer than the lead-in there + is, and placement. +- **14** `record_into_selection_stops_by_itself`: how far past its selection each take + recorded before it stopped itself, the coverage it added, and no dialog during it. + +**What fails on MME today, and why.** Every take restarts the audio stream, and on MME the +offset between input and output moves by about 13 ms from one restart to the next, while +the sweeps within one take agree to 0.3 ms; the start gap does not see it. No one round +trip then places every take within ±2 ms: expect items 1 and 2 to fail, items 7 and 13 +whenever their punch-in lands more than 2 ms off, and item 10 at the join (the step and the +pitch) when its two punch-ins land apart. That is the true reading, not a fault of the +check: the remedy is a lower-latency driver, the next project +([calibrate-audio.md](calibrate-audio.md), §8). + +A device that opens but delivers nothing ends the run with "The audio device delivered no +input" once the take's lead-in and range and 2 s more have gone by without one frame, and +leaves no take; a device that cannot be opened ends it with "The take did not start". +`TestAudioCheck` checks both on the fake device. + +User's PC, 2026-09-26 (MME, wired microphone and headphones, one earcup to the microphone): +Calibrate Audio measured 301 and 295 ms, verdict Unsteady; with the microphone between both +cups, Scattered. A dev run with items 1 and 2 only: round trip 303.5 ms, the two punch-ins +at +0.6 and -12.9 ms, the sweeps of each within 0.3 ms. **The whole dev run not yet.** + +Cloud run, 2026-09-25 (Linux, no sound card, with the test executable this replaced): with +no device at all Record does no harm and the next file is analysed (the "Couldn't open +audio device" warning comes back once per file opened); with a device that opens but +delivers nothing the take is dropped quietly, no harm. ## 2. Still by hand diff --git a/docs/open-points.md b/docs/open-points.md index 4c6141fa..84a68bbb 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -13,8 +13,9 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. - **Take operations clear the undo history with no prompt** (all but Rename). - **The alternate pitch track at ±3 octaves** of a 220 Hz reference (28 Hz, 1.8 kHz) is outside the range the pane shows, and nothing scrolls to it; ±2 is in view. -- Of the [manual checklist](manual-checklist.md), the device check has been run only in - the cloud (no sound card, and the fake device); nothing yet on real hardware. +- Of the [manual checklist](manual-checklist.md), the device check (Calibrate Audio with + the dev checks) has been run on real hardware only in part: Calibrate Audio, and a dev + run with items 1 and 2 only (the user's PC, MME, 2026-09-26). The whole dev run not yet. ## Not built diff --git a/docs/testing.md b/docs/testing.md index 21fb150b..ba8f015b 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,20 +1,21 @@ # Testing QtTest suites in `main/test/`, in two executables that mirror the two libraries -(see [architecture.md](architecture.md)), plus one for the development checks and one for -the real device. The commands are in [AGENTS.md](../AGENTS.md). +(see [architecture.md](architecture.md)), plus one for the development checks. The +commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | | `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | | `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about a minute and growing: each test records several takes in real time | -| `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | -`meson test` / `build.bat test` runs the first three plus four svcore suites. `test-tony-device` -is built with them and never run by `meson test`: it needs a microphone that hears the -speakers. `test-tony-dev` is apart from `test-tony-app` so that the everyday runs stay -shorter: run it when a change touches what the development checks drive (see +`meson test` / `build.bat test` runs these three (`test-tony-dev` where it is built) plus +four svcore suites. No suite uses the +real audio device: that is checked in the app itself, by Calibrate Audio with the +development checks (section 1 of the [manual checklist](manual-checklist.md)). +`test-tony-dev` is apart from `test-tony-app` so that the everyday runs stay shorter: run +it when a change touches what the development checks drive (see [AGENTS.md](../AGENTS.md)). - The `tony-app` meson test has `timeout: 900`; the suite took about 277 s unloaded when diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index 49a984e1..f87d3355 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -69,7 +69,8 @@ AudioCheckRunner::AudioCheckRunner(MainWindow *window) : m_punchIn(0), m_openingReference(false), m_inPoll(false), - m_stepLimitMs(0) + m_stepLimitMs(0), + m_takeMs(0) { m_timer->setInterval(kPollMs); connect(m_timer, &QTimer::timeout, this, &AudioCheckRunner::poll); @@ -295,6 +296,14 @@ AudioCheckRunner::poll() if (!m_window->m_recordTarget || !m_window->m_recordTarget->isRecording()) { takeStopped(); + } else if (deliveredNothing()) { + // Such a take never stops itself: it stops when what it has + // received reaches the end of its range + stopTake(); + end(tr("The audio device delivered no input: it opened, but in " + "%1 s of recording not one frame came from it. Is the " + "microphone connected, and allowed to be used?") + .arg(double(m_stepClock.elapsed()) / 1000.0, 0, 'f', 1)); } else if (stepTimedOut()) { stopTake(); end(tr("A take did not stop at the end of its range.")); @@ -483,7 +492,8 @@ AudioCheckRunner::startPunchIn() // take to see that it has reached the end const double seconds = double(m_window->m_takePreRoll + (to - from)) / m_result.referenceRate; - setStep(Step::Recording, qint64(seconds * 1000.0) + kTakeStopTimeoutMs); + m_takeMs = qint64(seconds * 1000.0); + setStep(Step::Recording, m_takeMs + kTakeStopTimeoutMs); } void @@ -653,6 +663,17 @@ AudioCheckRunner::stepTimedOut() const return m_stepLimitMs > 0 && m_stepClock.elapsed() > m_stepLimitMs; } +bool +AudioCheckRunner::deliveredNothing() const +{ + // The count starts again at 0 with every take (startRecording()). + // One frame is enough to wait for the take as usual: a device that + // delivers late or too little is what stepTimedOut() is for + return m_step == Step::Recording && m_window->m_recordTarget && + m_window->m_recordTarget->getFramesReceived() == 0 && + m_stepClock.elapsed() > m_takeMs + kNoInputTimeoutMs; +} + void AudioCheckRunner::stopTake() { diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index c3aadb09..1f5df604 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -141,6 +141,15 @@ class AudioCheckRunner : public QObject static constexpr int kTakeAnalysisTimeoutMs = 30000; static constexpr int kTakeStopTimeoutMs = 10000; + /// How long past its own length (lead-in and range) a take may go + /// without a single frame from the device before the run says the + /// device delivered no input. Not sooner: a stream that is only slow + /// to start (a Bluetooth headset switching to its microphone, say) + /// delivers its first block within a second or two, and a device + /// that has sent nothing for the whole length of the take plus this + /// has recorded none of it anyway + static constexpr int kNoInputTimeoutMs = 2000; + /// What a run records struct Plan { LatencyCheck::Layout layout; @@ -322,6 +331,13 @@ class AudioCheckRunner : public QObject QElapsedTimer m_stepClock; qint64 m_stepLimitMs; + /// The length of the take being recorded, lead-in and range, in ms + qint64 m_takeMs; + + /// The take has gone kNoInputTimeoutMs past its length with not one + /// frame from the device + bool deliveredNothing() const; + void poll(); void openReference(); diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index 9da67a7f..67adf70b 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -151,8 +151,7 @@ class DevChecks : public QObject /// may land, either way static constexpr double kPlacementSeconds = 0.002; - /// Item 3: more live dots than this in every punch-in, as - /// test-tony-device asked + /// Item 3: more live dots than this in every punch-in static constexpr int kMinDots = 10; /// Item 3: a dot is on one of the reference's sounds when it lies @@ -184,8 +183,7 @@ class DevChecks : public QObject static constexpr double kStopMarginSeconds = 0.25; /// Item 9: each punch-in into the long song has its pitch merged - /// within this share of the time the whole song's analysis took, as - /// test-tony-device asked + /// within this share of the time the whole song's analysis took static constexpr double kStopShare = 0.5; /// How often a stage is looked at diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h index 988b5004..b1c8a107 100644 --- a/main/test/FakeAudioIO.h +++ b/main/test/FakeAudioIO.h @@ -84,6 +84,11 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO // left and right, as PortAudioIO does for its level meters bool reportLevels = false; + // The device opens, and suspends and resumes as asked, but never + // calls back: no input comes in and no output is asked for, as + // with a driver whose stream starts and then delivers nothing + bool neverCallsBack = false; + // Whether the application keeps the input it is given just // now. It discards input until its recording file is open, // which is some time after it resumes the device. If unset, @@ -208,7 +213,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO next += period; std::this_thread::sleep_until(next); std::lock_guard guard(m_mutex); - if (m_suspended) { + if (m_suspended || m_config.neverCallsBack) { // don't try to catch up on the time spent suspended next = steady_clock::now(); continue; diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 4ddecc3f..8a0236a1 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -46,6 +46,7 @@ #include #include #include +#include #include #include #include @@ -717,6 +718,61 @@ private slots: QVERIFY(!m_window->audioCheckTakes()); } + // A device that opens but never calls back: not one frame comes in. + // Once the take should have been over, and not before, so that a + // device slow to start is not taken for one that delivers nothing, + // the run stops the take through the Stop path and ends saying that + // the device delivered no input, not that the take did not stop. No + // take is left, and no harm: the next file opened is analysed + void check_ends_when_the_device_delivers_nothing() { + FakeAudioIO::Config config = loopback(); + config.neverCallsBack = true; + makeWindow(config); + + // How long after the take began to record the run ended; the + // connections go with the guard when the test returns + QObject guard; + QElapsedTimer recording; + qint64 endedAfterMs = -1; + connect(m_window->audioCheck(), &AudioCheckRunner::progress, &guard, + [&recording](const AudioCheckRunner::Progress &p) { + if (p.step == AudioCheckRunner::Step::Recording && + !recording.isValid()) { + recording.start(); + } + }); + connect(m_window->audioCheck(), &AudioCheckRunner::finished, &guard, + [&](const AudioCheckResult &) { + if (recording.isValid()) { + endedAfterMs = recording.elapsed(); + } + }); + + runCheck(onePunchIn()); + if (QTest::currentTestFailed()) return; + + QVERIFY2(m_result.failure.contains("delivered no input"), + describe(m_result).constData()); + const qint64 takeMs = + qint64((AudioCheckRunner::kPreRollSeconds + 2.0) * 1000.0); + QVERIFY2(endedAfterMs >= takeMs + AudioCheckRunner::kNoInputTimeoutMs - + AudioCheckRunner::kPollMs, + qPrintable(QString("ended %1 ms into a take of %2 ms") + .arg(endedAfterMs).arg(takeMs))); + + QVERIFY(!m_result.calibrationUsable()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(!m_window->recordingInProgress()); + QVERIFY(!m_window->recordingAsSingingTrack()); + QVERIFY(!m_window->audioCheckTakes()); + QVERIFY2(!m_window->takes()->haveTake(), + "a recording of nothing was kept as a take"); + + openSong(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_finished, 1); + } + // With automatic analysis switched off the reference is never // analysed, and the check does not wait for it: it needs none void check_runs_without_automatic_analysis() { diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index b64a54aa..0e4df6a6 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -15,7 +15,7 @@ #define TEST_MAIN_WINDOW_H // The real MainWindow for the suites that drive it: TestRecordWorkflow, -// TestUiChecks, TestAudioCheck, TestDevChecks and the real-device check +// TestUiChecks, TestAudioCheck and TestDevChecks #include "FakeAudioIO.h" @@ -41,7 +41,6 @@ /** * MainWindow with the fake device in place of a real one, and the * protected state of the singing workflow opened up for inspection. - * The real device can be asked for instead (setUseRealDevice()). */ class TestMainWindow : public MainWindow { @@ -53,16 +52,6 @@ class TestMainWindow : public MainWindow FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } - // The audio device Tony itself would open, from the settings, in - // place of the fake: for the checks that need real hardware. Set - // before the first file is opened, which is when the device is made - void setUseRealDevice(bool on) { m_useRealDevice = on; } - bool haveAudioDevice() { return m_audioIO || m_playTarget; } - - // A device that records as well: without one there is only a play - // target (MainWindowBase::createAudioIO()) - bool haveRecordingDevice() { return m_audioIO != nullptr; } - void doRecord() { record(); } void doPlay() { play(); } // and again to stop void doAnalyseNow() { analyseNow(); } @@ -239,10 +228,6 @@ class TestMainWindow : public MainWindow protected: void createAudioIO() override { if (m_audioIO || m_playTarget) return; - if (m_useRealDevice) { - MainWindow::createAudioIO(); - return; - } if (!m_installDevice) return; m_fakeConfig.inputIsKept = [this]() { return m_recordTarget->isRecording(); @@ -276,7 +261,6 @@ class TestMainWindow : public MainWindow private: FakeAudioIO::Config m_fakeConfig; bool m_installDevice; - bool m_useRealDevice = false; bool m_recordOverAnswer = true; bool m_recordOverInDialog = false; int m_recordOverQuestions = 0; diff --git a/main/test/TestRealDevice.h b/main/test/TestRealDevice.h deleted file mode 100644 index cd8cdaec..00000000 --- a/main/test/TestRealDevice.h +++ /dev/null @@ -1,607 +0,0 @@ -/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ -/* - Tony - An intonation analysis and annotation tool - Centre for Digital Music, Queen Mary, University of London. - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. See the file - COPYING included with this distribution for more information. -*/ - -#ifndef TEST_REAL_DEVICE_H -#define TEST_REAL_DEVICE_H - -// The checks that need the real audio device: the real MainWindow, -// opening the device the user chose in Tony, recording what its -// microphone hears of its own speakers. -// -// The reference is four minutes of a repeating two-second pattern: a -// second of a tone, for the live dots and pYIN, then a second holding -// three clicks at uneven spacing. Two recordings are made into one take -// with Play Reference While Recording on, and each recorded range of -// the take is lined up against the reference by its clicks. A take -// whose latency was compensated right has its clicks exactly where the -// reference has them. -// -// Not part of any automated run: see docs/manual-checklist.md for how -// to set it up. With no device to record from, or one that delivers -// nothing, the one check that runs is that Record does no harm. - -#include "TestSignals.h" -#include "TestMainWindow.h" - -#include "version.h" - -#include "layer/Layer.h" -#include "data/model/SparseTimeValueModel.h" -#include "data/fileio/WavFileReader.h" -#include "data/fileio/WavFileWriter.h" -#include "data/fileio/FileSource.h" -#include "base/RecordDirectory.h" -#include "transform/ModelTransformerFactory.h" -#include "widgets/InteractiveFileFinder.h" - -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include - -class TestRealDevice : public QObject -{ - Q_OBJECT - - static constexpr double rate = 44100.0; - static constexpr double songSeconds = 240.0; - static constexpr double patternSeconds = 2.0; - - // Where in each pattern the clicks are, in seconds: uneven, so that - // only the right lag lines all three up - static std::vector clickOffsets() { return { 1.10, 1.37, 1.71 }; } - - // How far a recorded range may sit from the reference and still be - // called in time: a few milliseconds of sound travelling from the - // speaker to the microphone, and the rest for the device - static constexpr double toleranceMs = 10.0; - - // The recordings: where, and for how long - static std::vector takePositions() { return { 60.0, 150.0 }; } - static constexpr int takeMs = 5000; - - struct Recorded { - Coverage::Range range; - double lagMs = 0.0; // later than the reference if positive - double peak = 0.0; // normalised correlation at that lag - double echoMs = 0.0; // strongest later peak, after the first - double echo = 0.0; // ... and its height - double stopSeconds = 0.0; // from Stop to the analysis merged - int dots = 0; // live dots drawn during it - std::vector channelLevels; // dB, of the raw recording - }; - - QTemporaryDir m_dir; - TestMainWindow *m_window = nullptr; - QTimer m_watchdog; - QStringList m_dialogs; - std::vector m_reference; - QString m_referencePath; - double m_wholeSongSeconds = 0.0; - std::vector m_recorded; - bool m_deliveredNothing = false; - - static sv::sv_frame_t frames(double seconds) { - return sv::sv_frame_t(std::lround(seconds * rate)); - } - - // The tone half of each pattern, and the clicks: short bursts of - // noise, which line up by cross-correlation with no ambiguity of a - // period - std::vector makeReference(bool clicksOnly) { - std::vector data(size_t(frames(songSeconds)), 0.f); - const int clickFrames = int(0.003 * rate); - std::mt19937 random(42); - std::uniform_real_distribution noise(-1.f, 1.f); - std::vector click(static_cast(clickFrames), 0.f); - for (int i = 0; i < clickFrames; ++i) { - double window = 0.5 - 0.5 * std::cos(2.0 * TestSignals::kPi * i / - (clickFrames - 1)); - click[size_t(i)] = float(0.8 * window) * noise(random); - } - auto saw = TestSignals::sawtooth(220.5, rate, int(frames(1.0)), 0.25f); - - for (double t = 0.0; t + patternSeconds <= songSeconds; - t += patternSeconds) { - size_t at = size_t(frames(t)); - if (!clicksOnly) { - std::copy(saw.begin(), saw.end(), data.begin() + long(at)); - } - for (double offset : clickOffsets()) { - size_t c = size_t(frames(t + offset)); - std::copy(click.begin(), click.end(), data.begin() + long(c)); - } - } - return data; - } - - QString writeWav(const std::vector &data, QString name) { - QString path = m_dir.filePath(name); - sv::WavFileWriter writer(path, rate, 1, - sv::WavFileWriter::WriteToTarget); - const float *ptr = data.data(); - if (!writer.isOK() || - !writer.writeSamples(&ptr, sv::sv_frame_t(data.size())) || - !writer.close()) { - return {}; - } - return path; - } - - static std::vector readMono(QString path, sv::sv_frame_t from, - sv::sv_frame_t count) { - sv::WavFileReader reader { sv::FileSource(path) }; - if (!reader.isOK()) return {}; - int ch = reader.getChannelCount(); - auto data = reader.getInterleavedFrames(from, count); - std::vector mono(data.size() / size_t(ch), 0.f); - for (size_t i = 0; i < mono.size(); ++i) { - for (int c = 0; c < ch; ++c) mono[i] += data[i * size_t(ch) + size_t(c)]; - } - return mono; - } - - // The lag, in frames within +-maxLag, at which the recorded audio - // best matches the clicks of the reference, and how well: the peak of - // the normalised cross-correlation. The tone half of the reference - // is left out, so that only the clicks decide it - static std::pair bestLag(const std::vector &clicks, - const std::vector &recorded, - long maxLag, - std::vector *curve) { - // clicks runs from maxLag before the recorded range to maxLag - // after it - double er = 0.0; - for (float v : recorded) er += double(v) * v; - long best = 0; - double bestValue = -1.0; - for (long lag = -maxLag; lag <= maxLag; ++lag) { - double sum = 0.0, ec = 0.0; - for (size_t i = 0; i < recorded.size(); ++i) { - double c = clicks[size_t(long(i) + maxLag - lag)]; - sum += c * recorded[i]; - ec += c * c; - } - double value = (ec > 0.0 && er > 0.0) ? sum / std::sqrt(ec * er) - : 0.0; - if (curve) curve->push_back(value); - if (value > bestValue) { - bestValue = value; - best = lag; - } - } - return { best, bestValue }; - } - - static double levelDb(const std::vector &data) { - double sum = 0.0; - for (float v : data) sum += double(v) * v; - double rms = data.empty() ? 0.0 : std::sqrt(sum / double(data.size())); - return rms > 0.0 ? 20.0 * std::log10(rms) : -200.0; - } - - static bool analysed(Analyser *a) { - return a && a->getLayer(Analyser::PitchTrack) && - a->getLayer(Analyser::Notes) && - a->getInitialAnalysisCompletion() >= 100 && - !a->isAnalysingRange() && - !sv::ModelTransformerFactory::getInstance() - ->haveRunningTransformers(); - } - - // The newest raw recording: the file the device was recorded into, - // with every channel it had - QString newestRecording() { - QString newest; - QDateTime newestTime; - QDirIterator it(sv::RecordDirectory::getRecordContainerDirectory(), - { "recorded-*.wav" }, QDir::Files, - QDirIterator::Subdirectories); - while (it.hasNext()) { - QString path = it.next(); - QDateTime t = QFileInfo(path).lastModified(); - if (newest == "" || t > newestTime) { - newest = path; - newestTime = t; - } - } - return newest; - } - - // Not a slot: QtTest would run it as a test - void dismissDialog() { - QWidget *modal = QApplication::activeModalWidget(); - if (!modal) return; - QString description = modal->windowTitle(); - if (auto box = qobject_cast(modal)) { - description += ": " + box->text(); - QList buttons = box->buttons(); - m_dialogs.push_back(description); - if (!buttons.isEmpty()) { - buttons.last()->click(); - return; - } - } else { - m_dialogs.push_back(description); - } - if (auto dialog = qobject_cast(modal)) dialog->reject(); - else modal->close(); - } - - bool haveDevice() { return m_window && m_window->haveAudioDevice(); } - bool canRecord() { return m_window && m_window->haveRecordingDevice(); } - -private slots: - void initTestCase() { - QVERIFY(m_dir.isValid()); - QSettings().clear(); - - // The devices Tony uses, from Tony's own settings: whatever was - // chosen in its Playback menu is what is checked here. The rest of - // Tony's settings are not read, and nothing is written to them - { - QSettings tony("sonic-visualiser", "Tony"); - tony.beginGroup("Preferences"); - QSettings mine; - mine.beginGroup("Preferences"); - for (QString key : tony.childKeys()) { - if (key.startsWith("audio-")) { - mine.setValue(key, tony.value(key)); - qInfo("Tony's setting %s = \"%s\"", qPrintable(key), - qPrintable(tony.value(key).toString())); - } - } - mine.setValue(QString("network-permission-%1").arg(TONY_VERSION), - false); - mine.endGroup(); - } - - QSettings settings; - settings.beginGroup("MainWindow"); - settings.setValue("playrefwhilerecording", true); - settings.setValue("preroll", false); - settings.setValue("recordintoselection", false); - settings.endGroup(); - SingingTakes::setOverwriteConfirmationWanted(true); - - sv::InteractiveFileFinder::getInstance() - ->setApplicationSessionExtension("ton"); - sv::RecordDirectory::setRecordContainerDirectory - (m_dir.filePath("recorded")); - - connect(&m_watchdog, &QTimer::timeout, - this, [this]() { dismissDialog(); }); - m_watchdog.start(50); - - m_reference = makeReference(false); - m_referencePath = writeWav(m_reference, "reference.wav"); - QVERIFY(m_referencePath != ""); - - // TONY_DEVICE_CHECK_FAKE=n checks the check itself, with no - // hardware: the fake device, its output fed back into its input as - // speakers into a microphone, reporting a round trip of 2048 - // frames and taking 2048 + n. With n = 0 everything passes; the - // recordings come out n frames late otherwise - FakeAudioIO::Config fake; - bool useFake = qEnvironmentVariableIsSet("TONY_DEVICE_CHECK_FAKE"); - if (useFake) { - fake.playbackLatency = 1024; - fake.recordLatency = 1024; - fake.inputDelay = 2048 + - qEnvironmentVariableIntValue("TONY_DEVICE_CHECK_FAKE"); - fake.loopback = true; - qInfo("the fake device, %d frames from output to input", - fake.inputDelay); - } - m_window = new TestMainWindow(fake); - m_window->setUseRealDevice(!useFake); - m_window->setPlayReferenceWhileRecording(true); - m_window->resize(1200, 800); - m_window->show(); - - // The whole song's analysis, timed: what Stop must take much less - // time than - QElapsedTimer timer; - timer.start(); - QCOMPARE(m_window->openPath(m_referencePath, - MainWindow::ReplaceSession), - MainWindow::FileOpenSucceeded); - QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 300000); - m_wholeSongSeconds = double(timer.elapsed()) / 1000.0; - qInfo("analysis of the whole %.0f s reference took %.1f s", - songSeconds, m_wholeSongSeconds); - } - - void cleanupTestCase() { - m_watchdog.stop(); - if (m_window) { - if (m_window->recordTarget()->isRecording()) m_window->doRecord(); - QTRY_VERIFY_WITH_TIMEOUT - (!sv::ModelTransformerFactory::getInstance() - ->haveRunningTransformers(), 30000); - m_window->doCloseSession(); - delete m_window; - m_window = nullptr; - } - sv::RecordDirectory::setRecordContainerDirectory(""); - } - - // Which device, and what it says of itself - void device() { - std::vector names = - breakfastquay::AudioFactory::getImplementationNames(); - QStringList implementations; - for (auto n : names) implementations << QString::fromStdString(n); - qInfo("audio drivers built in: %s", qPrintable(implementations.join(", "))); - - if (!haveDevice()) { - qInfo("no audio device could be opened: %s", - qPrintable(m_dialogs.join(" | "))); - m_dialogs.clear(); - QSKIP("no audio device: only no_input_does_no_harm applies"); - } - qInfo("playback latency reported: %lld frames (%.1f ms)", - (long long)m_window->playSource()->getTargetPlayLatency(), - double(m_window->playSource()->getTargetPlayLatency()) * 1000.0 / rate); - qInfo("record latency reported: %d frames (%.1f ms)", - m_window->recordTarget()->getSystemRecordLatency(), - m_window->recordTarget()->getSystemRecordLatency() * 1000.0 / rate); - QVERIFY2(canRecord(), "the device can play but not record: is a " - "microphone connected and allowed?"); - } - - // Two recordings into one take, at two places in the song, with the - // reference playing: the microphone hears it and the take records it - void record_the_reference_through_the_air() { - if (!canRecord()) QSKIP("no device to record from"); - - for (double at : takePositions()) { - Recorded r; - m_window->seekTo(frames(at)); - m_window->doRecord(); - QVERIFY2(m_window->recordTarget()->isRecording(), - qPrintable("Record did not start: " + m_dialogs.join(" | "))); - QTest::qWait(takeMs); - if (auto dots = sv::ModelById::getAs - (m_window->realtimeModelId())) { - r.dots = dots->getEventCount(); - } - sv::sv_frame_t received = - m_window->recordTarget()->getFramesReceived(); - QElapsedTimer timer; - timer.start(); - m_window->doRecord(); - QVERIFY(!m_window->recordTarget()->isRecording()); - if (received == 0) { - m_deliveredNothing = true; - QTRY_VERIFY(!m_window->recordingInProgress()); - QFAIL(qPrintable(QString("the device opened, but in %1 s it " - "delivered no input at all") - .arg(takeMs / 1000))); - } - QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 60000); - r.stopSeconds = double(timer.elapsed()) / 1000.0; - - // Each channel of what the device delivered, to see which - // input the microphone is on - sv::WavFileReader reader { sv::FileSource(newestRecording()) }; - QVERIFY(reader.isOK()); - int channels = reader.getChannelCount(); - auto data = reader.getInterleavedFrames(0, reader.getFrameCount()); - for (int c = 0; c < channels; ++c) { - std::vector one; - for (size_t i = size_t(c); i < data.size(); i += size_t(channels)) { - one.push_back(data[i]); - } - r.channelLevels.push_back(levelDb(one)); - } - m_recorded.push_back(r); - } - - auto ranges = m_window->takes()->getCoverage().getRanges(); - QCOMPARE(ranges.size(), m_recorded.size()); - for (size_t i = 0; i < ranges.size(); ++i) { - m_recorded[i].range = ranges[i]; - } - QVERIFY(m_dialogs.isEmpty()); - } - - // Checklist: no input device, or one that records nothing: Record - // does nothing harmful, and the next file opened is analysed as usual - void no_input_does_no_harm() { - if (canRecord() && !m_deliveredNothing) { - QSKIP("the device records: this is for one that does not"); - } - m_dialogs.clear(); - m_window->seekTo(frames(30.0)); - m_window->doRecord(); - QTest::qWait(1000); - if (m_window->recordTarget()->isRecording()) m_window->doRecord(); - QTest::qWait(500); - qInfo("Record with no input: %s", - m_dialogs.isEmpty() ? "no dialog" - : qPrintable("dialog " + m_dialogs.join(" | "))); - m_dialogs.clear(); - QVERIFY(!m_window->recordTarget()->isRecording()); - QVERIFY(!m_window->recordingInProgress()); - QVERIFY(!m_window->recordingAsSingingTrack()); - QVERIFY2(!m_window->takes()->haveTake(), - "a recording of nothing was kept as a take"); - - auto other = TestSignals::sawtooth(294.0, rate, int(frames(3.0)), 0.5f); - m_window->discardModifications(); - QCOMPARE(m_window->openPath(writeWav(other, "next.wav"), - MainWindow::ReplaceSession), - MainWindow::FileOpenSucceeded); - QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser()), 30000); - auto pitch = sv::ModelById::getAs - (m_window->analyser()->getLayer(Analyser::PitchTrack)->getModel()); - QVERIFY(pitch && pitch->getEventCount() > 10); - - // With no device at all, MainWindowBase tries again to open one - // for every file opened, and says so each time when it cannot. - // That warning is expected; anything else is not - QStringList others; - for (QString d : m_dialogs) { - if (!d.startsWith("Couldn't open audio device")) others << d; - } - qInfo("%d warnings about the device while opening the next file", - int(m_dialogs.size() - others.size())); - m_dialogs.clear(); - QVERIFY2(others.isEmpty(), qPrintable(others.join(" | "))); - } - - // Checklist: the take lines up with the reference (latency on this - // machine), and every recording in one take does (several phrases) - void takes_line_up_with_the_reference() { - if (m_recorded.empty()) QSKIP("nothing was recorded"); - const std::vector clicks = makeReference(true); - const long maxLag = long(frames(0.3)); - QString audio = m_window->takes()->getAudioPath(); - - for (Recorded &r : m_recorded) { - // Leave out the edges of the range: the start is where the - // reference only began to play - sv::sv_frame_t from = r.range.start + frames(0.5); - sv::sv_frame_t to = r.range.end - frames(0.2); - QVERIFY(to - from > frames(2.0)); - auto recorded = readMono(audio, from, to - from); - QCOMPARE(sv::sv_frame_t(recorded.size()), to - from); - - // Only the half of each pattern that holds the clicks: the - // tone of the other half is nothing to line up by, and would - // count against how well the clicks match - for (size_t i = 0; i < recorded.size(); ++i) { - double t = std::fmod(double(from + sv::sv_frame_t(i)) / rate, - patternSeconds); - if (t < 1.0) recorded[i] = 0.f; - } - std::vector ref(clicks.begin() + long(from) - maxLag, - clicks.begin() + long(to) + maxLag); - std::vector curve; - auto best = bestLag(ref, recorded, maxLag, &curve); - r.lagMs = double(best.first) * 1000.0 / rate; - r.peak = best.second; - - // The strongest match later than the first, beyond the length - // of a click and its ringing: an echo of the take played back - // out of the speakers would put the clicks there a second time - for (long lag = best.first + long(frames(0.015)); lag <= maxLag; - ++lag) { - double v = curve[size_t(lag + maxLag)]; - if (v > r.echo) { - r.echo = v; - r.echoMs = double(lag - best.first) * 1000.0 / rate; - } - } - - qInfo("recording at %.1f s: clicks %+.1f ms from the reference " - "(match %.2f); stop took %.1f s; %d live dots; channels " - "(dB) %s", - double(r.range.start) / rate, r.lagMs, r.peak, - r.stopSeconds, r.dots, - qPrintable([&]() { - QStringList l; - for (double d : r.channelLevels) { - l << QString::number(d, 'f', 1); - } - return l.join(" "); - }())); - } - - for (const Recorded &r : m_recorded) { - QVERIFY2(r.peak > 0.1, - qPrintable(QString("the clicks of the reference are " - "hardly in the recording at %1 s " - "(match %2): can the microphone hear " - "the speakers?") - .arg(double(r.range.start) / rate) - .arg(r.peak, 0, 'f', 2))); - QVERIFY2(std::fabs(r.lagMs) <= toleranceMs, - qPrintable(QString("the recording at %1 s is %2 ms %3 " - "the reference") - .arg(double(r.range.start) / rate) - .arg(std::fabs(r.lagMs), 0, 'f', 1) - .arg(r.lagMs > 0 ? "later than" : "earlier " - "than"))); - } - if (m_recorded.size() > 1) { - double spread = std::fabs(m_recorded[0].lagMs - - m_recorded[1].lagMs); - QVERIFY2(spread <= 5.0, - qPrintable(QString("the two recordings of one take are " - "%1 ms apart in their timing") - .arg(spread, 0, 'f', 1))); - } - } - - // Checklist: nothing of the take in the speakers while recording. - // If the input were played back out, the microphone would hear every - // click a second time, a round trip later - void nothing_of_the_take_comes_back_out() { - if (m_recorded.empty()) QSKIP("nothing was recorded"); - for (const Recorded &r : m_recorded) { - qInfo("recording at %.1f s: strongest later match %.2f at " - "+%.1f ms, against %.2f for the clicks themselves", - double(r.range.start) / rate, r.echo, r.echoMs, r.peak); - QVERIFY2(r.echo < 0.5 * r.peak, - qPrintable(QString("the clicks come again %1 ms later " - "at %2 of their strength: is the " - "input being played back out, by " - "Tony or by the system (\"Listen to " - "this device\")?") - .arg(r.echoMs, 0, 'f', 1) - .arg(r.echo / r.peak, 0, 'f', 2))); - } - } - - // Checklist: how long Stop takes on a four-minute song: new pitch only - // where the singing was, not a whole-song analysis - void stop_is_quicker_than_a_whole_song() { - if (m_recorded.empty()) QSKIP("nothing was recorded"); - for (const Recorded &r : m_recorded) { - QVERIFY2(r.stopSeconds < 0.5 * m_wholeSongSeconds, - qPrintable(QString("Stop took %1 s, the whole song's " - "analysis %2 s") - .arg(r.stopSeconds, 0, 'f', 1) - .arg(m_wholeSongSeconds, 0, 'f', 1))); - } - } - - // Checklist: live dots appear, whichever input the microphone is on - void live_dots_were_drawn() { - if (m_recorded.empty()) QSKIP("nothing was recorded"); - for (const Recorded &r : m_recorded) { - QVERIFY2(r.dots > 10, - qPrintable(QString("only %1 live dots during the " - "recording at %2 s") - .arg(r.dots) - .arg(double(r.range.start) / rate))); - } - } -}; - -#endif diff --git a/main/test/tony-device-check.cpp b/main/test/tony-device-check.cpp deleted file mode 100644 index 54028756..00000000 --- a/main/test/tony-device-check.cpp +++ /dev/null @@ -1,59 +0,0 @@ -/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ -/* - Tony - An intonation analysis and annotation tool - Centre for Digital Music, Queen Mary, University of London. - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. See the file - COPYING included with this distribution for more information. -*/ - -// The checks that need a real audio device and a microphone that can -// hear the speakers. Run by hand, never by meson test: see -// docs/manual-checklist.md - -#include "TestRealDevice.h" - -#include "RunSuite.h" - -#include "system/Init.h" - -#include -#include -#include - -#include - -using namespace std; -using namespace sv; - -int main(int argc, char *argv[]) -{ - svSystemSpecificInitialisation(); - - // Not offscreen by default, unlike the other suites: the window is - // there to be watched while it records - - // Names distinct from the application's: Tony's own settings are only - // read, for the choice of device - QApplication app(argc, argv); - app.setOrganizationName("tony-tests"); - app.setApplicationName("test-tony-device"); - - qputenv("VAMP_PATH", - QDir::toNativeSeparators(app.applicationDirPath()).toLocal8Bit()); - - TestRealDevice t; - bool ok = runSuite(&t, argc, argv); - - if (!ok) { - SVCERR << "\n********* the device check failed\n" << endl; - return 1; - } else { - SVCERR << "The device check passed" << endl; - return 0; - } -} diff --git a/meson.build b/meson.build index c98d491a..4d4992d0 100644 --- a/meson.build +++ b/meson.build @@ -1463,37 +1463,6 @@ tony_app_test_exe = executable( win_subsystem: 'console' ) -# The checks that need a real audio device, with its microphone hearing -# its speakers. Built with the rest, run only by hand: not a meson test -tony_device_test_moc_files = qt.preprocess( - moc_headers: [ - 'main/test/TestRealDevice.h', -]) - -tony_device_test_exe = executable( - 'test-tony-device', - tony_device_test_moc_files, - 'main/test/tony-device-check.cpp', - dependencies: [ - tony_app_dep, - tony_core_dep, - svcore_dep, - qt_dep, - feature_dependencies, - os_dep, - dl_dep, - ], - cpp_args: [ - feature_defines, - general_defines, - ], - link_args: [ - feature_additional_libs, - general_link_args, - ], - win_subsystem: 'console' -) - test('tony-core', tony_core_test_exe) test('tony-app', tony_app_test_exe, depends: pyin_plugin, From 57696f213e90cc0c676036089b375e6da56abaf1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 10:35:11 +0000 Subject: [PATCH 185/275] docs: calibrate audio work orders, D refined Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio-work-orders.md | 39 ++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md index 68a17083..fcb80bc9 100644 --- a/docs/calibrate-audio-work-orders.md +++ b/docs/calibrate-audio-work-orders.md @@ -194,7 +194,7 @@ coloured fringes on the scale's labels read as live dots in `TestUiChecks`. ## 4. Phases -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`), C2c (`5f7b2b8`). +Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`), C2c (`5f7b2b8`), C3 (`e03f9d2`). Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). @@ -448,9 +448,40 @@ A build directory of type `release`: it must compile and link with no `main/dev/ the Calibrate Audio "Not built" item and add what is left open, the svapp `aboutToBeDeleted()` warning included. - Make `docs/calibrate-audio.md` describe what was built, with a "Known limitations and - open points" section. -- Delete this work-orders file and the log file, and remove the spec's links to them. -- No code. Suspected bugs go in the report. + open points" section. It is a plan today, with "Found in …" and "Since …" notes + layered on (§2's note that 4 × 3 does not fit is out of date; §4's table uses the + checklist's old numbering; §5's `waitUntil()` and modal dialog are gone). Rewrite it as + the design as built: the button, calibration, the dev run's stages and items, the + architecture, the tests, and the decisions table; keep the facts of §11 that still + hold, and the user's runs as dated notes. Say plainly what each verdict and each dev + check means, and which numbers the user should send back. +- **`AGENTS.md`**: a row in its "Read … before …" table for `docs/calibrate-audio.md` + (touching the audio check, the dev checks, or latency calibration), if you judge it + earns one; nothing else there unless it is false. +- **Open points to carry** into `open-points.md` (and the spec's limitations), each + checked against the code first: + - the next project, a lower-latency driver (spec §7): the rate fix first, the + `jhhr/bqaudioio` fork (created, attached, not yet pinned), a driver type in Tony, MME + the default until a run shows better; + - MME's restart jitter and what it fails today (spec §8, `manual-checklist.md` §1); + - items 4 and 12 read Pass when their gap part was "not judged" (C2b): the Totals line + then overstates; the user has not decided whether it should count otherwise; + - `AudioCheckRunner::kReferenceTimeoutMs` (60 s) against the long song's analysis + (10.4 s here): a PC six times slower ends the dev run at its first stage; + - the countdown reads 1, 2, 1 at P = 1 s with a 3 s pre-roll (C1c); + - live dots trail the cursor by about the round trip (C1b), spec §8 "Cursor versus + dots"; + - items 3 and 5 judge the fresh punch-ins only; item 14 cannot see an overwrite + question (asked inside `record()`, before the observer starts); + - not covered by the dev run (C3's list); + - the svapp `aboutToBeDeleted()` warning (merge log). +- **`forks.md`**: whether `jhhr/bqaudioio` belongs in its table yet (it is not used by + the build) or only in open points; say which you chose. +- Do not rewrite what `default` wrote in the shared docs beyond what this work made false; + C3 already rewrote `manual-checklist.md` §1: review it, keep it. +- Delete this work-orders file and the log file, and remove every link to them. +- No code, no builds, no test runs (the lead is building a release configuration at the + same time). Suspected bugs go in the report. ## 5. Log (newest last; 25 lines at most per entry) From bf39289a6fbcfa3fc4ef418da3e1820e0f972e13 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 10:49:34 +0000 Subject: [PATCH 186/275] feat: zoom and scroll the frequency range by touch a pinch zooms each axis by the fingers' spread along it: across, the time axis as before; up the pane, the analyser's frequency range, on the pane's log scale, about the frequency between the fingers. two fingers dragged up or down scroll the range. an axis counts only past a dead zone and when the movement along it is at least half that across it, so a pinch along one axis leaves the other alone. the range stays within a0 to c8 and no narrower than a major third. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/MainWindow.cpp | 33 ++- main/PinchZoom.cpp | 27 +++ main/PinchZoom.h | 34 +++- main/TouchGestures.cpp | 104 ++++++++-- main/TouchGestures.h | 54 ++++- main/VerticalZoom.cpp | 147 ++++++++++++++ main/VerticalZoom.h | 88 ++++++++ main/test/TestTouchGestures.h | 369 +++++++++++++++++++++++++++++++++- main/test/TestVerticalZoom.h | 244 ++++++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 2 + 11 files changed, 1088 insertions(+), 21 deletions(-) create mode 100644 main/VerticalZoom.cpp create mode 100644 main/VerticalZoom.h create mode 100644 main/test/TestVerticalZoom.h diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 7a072ca4..0a562258 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -6648,7 +6648,38 @@ void MainWindow::paneAdded(Pane *pane) { pane->setPlaybackFollow(PlaybackScrollPage); - new TouchGestures(pane); // owned by the pane + + // Two fingers zoom and scroll the frequency range as well, in the + // pane the reference's analyser shows it in: every pitch and note + // layer there is drawn on it, aligned to its spectrogram's scale + TouchGestures::VerticalRange range; + range.get = [this, pane](VerticalZoom::Range &shown) { + double min, max; + if (!m_analyser || m_analyser->getPane() != pane || + !m_analyser->getDisplayFrequencyExtents(min, max)) { + return false; + } + // What the pane draws values in Hz on, which is that range + // unless something else has come to set the scale + CoordinateScale scale = pane->getEffectiveVerticalExtents("Hz"); + if (scale.getDisplayMinimum() != min || + scale.getDisplayMaximum() != max || + !(scale.isLogarithmic() || scale.isLinear())) { + return false; + } + shown.min = min; + shown.max = max; + shown.log = scale.isLogarithmic(); + return true; + }; + range.set = [this](const VerticalZoom::Range &wanted) { + m_analyser->setDisplayFrequencyExtents(wanted.min, wanted.max); + }; + range.limits = VerticalZoom::pitchLimits(); + + TouchGestures *gestures = new TouchGestures(pane); // owned by the pane + gestures->setVerticalRange(range); + m_paneStack->sizePanesEqually(); if (m_overview) m_overview->registerView(pane); } diff --git a/main/PinchZoom.cpp b/main/PinchZoom.cpp index 0844ab01..de84046c 100644 --- a/main/PinchZoom.cpp +++ b/main/PinchZoom.cpp @@ -101,4 +101,31 @@ centreFor(double frame, ZoomLevel z, int width, double x) return sv_frame_t(std::llround(frame - dx * framesPerPixel(z))); } +AxisMovement::AxisMovement(double deadZone) : + m_deadZone(deadZone), + m_counting(false), + m_offset(0.0) +{ +} + +double +AxisMovement::update(double along, double across) +{ + if (!m_counting && + std::fabs(along) > m_deadZone && + 2.0 * std::fabs(along) >= std::fabs(across)) { + m_counting = true; + m_offset = (along > 0.0 ? m_deadZone : -m_deadZone); + } + return m_counting ? along - m_offset : 0.0; +} + +void +AxisMovement::start(double along) +{ + if (m_counting) return; + m_counting = true; + m_offset = along; +} + } diff --git a/main/PinchZoom.h b/main/PinchZoom.h index 9a2297a7..571ecd53 100644 --- a/main/PinchZoom.h +++ b/main/PinchZoom.h @@ -21,8 +21,9 @@ /** * The arithmetic of a two-finger gesture on a pane's time axis: which * zoom level a pinch asks for, and where the pane must be centred for - * a frame to stay under the fingers. TouchGestures does as the - * answers say. + * a frame to stay under the fingers; and, for either axis, when the + * fingers' movement along it counts. TouchGestures does as the + * answers say (VerticalZoom has the vertical axis). * * The x mapping is svgui's View::getFrameForX() made continuous: the * same centre, rounded the same way, so that a frame placed at x by @@ -63,6 +64,35 @@ namespace PinchZoom */ sv::sv_frame_t centreFor(double frame, sv::ZoomLevel z, int width, double x); + + /** + * How much of the fingers' movement along one axis counts: the + * change in their spread, or the travel of the point between them, + * since they came down. None until the movement is plainly meant: + * more than the dead zone, and at least half as much as the same + * movement across the axis, so that a pinch or a drag along one + * axis leaves the other alone. From then on all of it, less what + * was taken for the dead zone, so that nothing jumps when it + * starts to count. + */ + class AxisMovement + { + public: + explicit AxisMovement(double deadZone = 0.0); + + /// What counts of the movement along, with across the other's + double update(double along, double across); + + /// Count from now on, from along as it is now + void start(double along); + + bool isCounting() const { return m_counting; } + + private: + double m_deadZone; + bool m_counting; + double m_offset; + }; } #endif diff --git a/main/TouchGestures.cpp b/main/TouchGestures.cpp index b22930c1..e28cfc80 100644 --- a/main/TouchGestures.cpp +++ b/main/TouchGestures.cpp @@ -21,13 +21,13 @@ #include #include #include -#include #include #include #include #include #include +#include #include using namespace sv; @@ -109,6 +109,12 @@ TouchGestures::~TouchGestures() stopWatchingMenu(); } +void +TouchGestures::setVerticalRange(const VerticalRange &range) +{ + m_verticalRange = range; +} + bool TouchGestures::eventFilter(QObject *watched, QEvent *event) { @@ -396,14 +402,40 @@ TouchGestures::beginPinch() QPointF a = m_points.value(m_pinchA); QPointF b = m_points.value(m_pinchB); - m_startSpan = QLineF(a, b).length(); + m_startCentre = (a + b) / 2.0; + m_startSpreadX = spread(a.x() - b.x()); + m_startSpreadY = spread(a.y() - b.y()); + + // Two fingers dragged together change their spread a little: that + // is not yet a pinch. Twice the slop, as Android's own detector has. + // Up the pane the same goes for their travel: the range stays put + // while the fingers scroll or zoom in time + double deadZone = 2 * slop(); + m_spreadX = PinchZoom::AxisMovement(deadZone); + m_spreadY = PinchZoom::AxisMovement(deadZone); + m_travelY = PinchZoom::AxisMovement(deadZone); + m_startFramesPerPixel = PinchZoom::framesPerPixel(m_pane->getZoomLevel()); - m_zooming = false; m_anchorFrame = PinchZoom::frameAtX (m_pane->getCentreFrame(), m_pane->getZoomLevel(), - m_pane->width(), (a.x() + b.x()) / 2.0); + m_pane->width(), m_startCentre.x()); + + // The range the fingers start from, within its limits (one typed + // in may not be), and the value under the point between them + m_haveRange = false; + double height = m_pane->height(); + VerticalZoom::Range shown; + if (height > 0 && m_verticalRange.get && m_verticalRange.set && + m_verticalRange.get(shown)) { + m_haveRange = true; + m_shownRange = shown; + m_startRange = VerticalZoom::limited + (shown, m_verticalRange.limits, height, m_startCentre.y()); + m_anchorValue = VerticalZoom::valueAtY + (m_startRange, height, m_startCentre.y()); + } } void @@ -418,19 +450,25 @@ TouchGestures::updatePinch() QPointF a = m_points.value(m_pinchA); QPointF b = m_points.value(m_pinchB); - double span = QLineF(a, b).length(); + QPointF travel = (a + b) / 2.0 - m_startCentre; + double spreadX = spread(a.x() - b.x()); + double spreadY = spread(a.y() - b.y()); double x = (a.x() + b.x()) / 2.0; - // Two fingers dragged together change their distance a little: that - // is not yet a pinch. Twice the slop, as Android's own detector has - if (!m_zooming && std::fabs(span - m_startSpan) > 2 * slop()) { - m_zooming = true; - } + // Each axis zooms by the spread along it, once that counts + m_spreadX.update(spreadX - m_startSpreadX, spreadY - m_startSpreadY); + double pinchY = m_spreadY.update(spreadY - m_startSpreadY, + spreadX - m_startSpreadX); - if (m_zooming && m_startSpan > 0.0 && span > 0.0) { + // A range zoomed about the fingers goes with them up and down, as the + // time axis goes with them across + if (m_spreadY.isCounting()) m_travelY.start(travel.y()); + double travelY = m_travelY.update(travel.y(), travel.x()); + + if (m_spreadX.isCounting()) { ZoomLevel current = m_pane->getZoomLevel(); ZoomLevel level = PinchZoom::pinchedLevel - (current, m_startFramesPerPixel * m_startSpan / span); + (current, m_startFramesPerPixel * m_startSpreadX / spreadX); if (!(level == current)) { m_pane->setZoomLevel(level); } @@ -455,6 +493,39 @@ TouchGestures::updatePinch() if (centre != m_pane->getCentreFrame()) { m_pane->setCentreFrame(centre); } + + updateVerticalRange((m_startSpreadY + pinchY) / m_startSpreadY, + travelY); +} + +void +TouchGestures::updateVerticalRange(double factor, double travel) +{ + // Left as it is until the fingers plainly move up or down + if (!m_haveRange) return; + if (!m_spreadY.isCounting() && !m_travelY.isCounting()) return; + + double height = m_pane->height(); + if (!(height > 0)) return; + + // The value that was between the fingers goes where they are now, + // with the range narrowed by as much as they have spread + double y = m_startCentre.y() + travel; + VerticalZoom::Range wanted = VerticalZoom::zoomedAbout + (m_startRange, factor, m_anchorValue, height, y); + VerticalZoom::Range range = VerticalZoom::limited + (wanted, m_verticalRange.limits, height, y); + + // Held at a limit, the range stays while the fingers go on: what is + // under them then is what they hold, as at the ends of the audio + if (range != wanted) { + m_anchorValue = VerticalZoom::valueAtY(range, height, y); + } + + if (range != m_shownRange) { + m_shownRange = range; + m_verticalRange.set(range); + } } void @@ -535,3 +606,12 @@ TouchGestures::slop() const // phone are the platform's density-independent ones return QGuiApplication::styleHints()->startDragDistance(); } + +double +TouchGestures::spread(double distance) const +{ + // The fingers' spread along one axis, but never less than four + // times the slop: two fingers side by side are hardly apart up the + // pane, and a ratio of two such spreads could be anything + return std::max(std::fabs(distance), 4.0 * std::max(slop(), 1)); +} diff --git a/main/TouchGestures.h b/main/TouchGestures.h index 9e07f595..987e849f 100644 --- a/main/TouchGestures.h +++ b/main/TouchGestures.h @@ -15,6 +15,9 @@ #ifndef TONY_TOUCH_GESTURES_H #define TONY_TOUCH_GESTURES_H +#include "PinchZoom.h" +#include "VerticalZoom.h" + #include "base/BaseTypes.h" #include @@ -25,6 +28,7 @@ #include #include +#include #include #include @@ -36,9 +40,16 @@ class Pane; } /** - * Touch on one pane: pinch to zoom the time axis about the fingers, - * two fingers dragged to scroll it, and a long press for the pane's - * right-button menu. MainWindow gives every pane one; the pane owns it. + * Touch on one pane: pinch to zoom about the fingers, two fingers + * dragged to scroll, and a long press for the pane's right-button + * menu. MainWindow gives every pane one; the pane owns it. + * + * The time axis zooms by the fingers' spread across the pane and + * follows them across it. Given a vertical range (setVerticalRange()), + * the range zooms by their spread up the pane and follows them up and + * down, each axis only once the fingers plainly move along it + * (PinchZoom::AxisMovement): a pinch across the pane leaves the range + * alone, and a pinch up the pane the zoom level. * * One finger is left to Qt, which makes mouse events of a touch that * nothing accepts: tapping, dragging in Navigate mode and selecting in @@ -73,6 +84,21 @@ class TouchGestures : public QObject explicit TouchGestures(sv::Pane *pane); virtual ~TouchGestures(); + /** + * The range of values the pane's layers are drawn over, bottom to + * top, for two fingers to zoom and scroll. Changing it makes no + * undo step. A pane without one moves in time only. + */ + struct VerticalRange { + /// The range shown now, on the pane's scale; false if none + std::function get; + /// Show another + std::function set; + VerticalZoom::Limits limits; + }; + + void setVerticalRange(const VerticalRange &range); + /// How long one finger must rest for a long press, in ms static const int longPressMs = 500; @@ -99,6 +125,7 @@ class TouchGestures : public QObject void startTwoFingers(); void beginPinch(); void updatePinch(); + void updateVerticalRange(double factor, double travel); void hold(QMouseEvent *); void replayHeld(); @@ -110,6 +137,7 @@ class TouchGestures : public QObject void removePoint(int id); QPointF sourcePosition() const; int slop() const; + double spread(double distance) const; sv::Pane *m_pane; State m_state = State::Idle; @@ -134,10 +162,26 @@ class TouchGestures : public QObject // The two points of the pinch, and the view when they came down int m_pinchA = -1; int m_pinchB = -1; - double m_startSpan = 0.0; + QPointF m_startCentre; + double m_startSpreadX = 0.0; + double m_startSpreadY = 0.0; double m_startFramesPerPixel = 1.0; double m_anchorFrame = 0.0; - bool m_zooming = false; + + // How much of the fingers' movement since then counts: the change + // in their spread across the pane and up it, and the travel of + // the point between them up it + PinchZoom::AxisMovement m_spreadX; + PinchZoom::AxisMovement m_spreadY; + PinchZoom::AxisMovement m_travelY; + + // The vertical range, as it was when they came down and as last + // shown, and the value that was under the point between them + VerticalRange m_verticalRange; + bool m_haveRange = false; + VerticalZoom::Range m_startRange; + VerticalZoom::Range m_shownRange; + double m_anchorValue = 0.0; }; #endif diff --git a/main/VerticalZoom.cpp b/main/VerticalZoom.cpp new file mode 100644 index 00000000..7f1783cb --- /dev/null +++ b/main/VerticalZoom.cpp @@ -0,0 +1,147 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "VerticalZoom.h" + +#include +#include + +namespace VerticalZoom +{ + +namespace { + +// Onto the axis the pane spaces evenly: log10, as svgui's LogRange +double map(bool log, double value) +{ + return log ? std::log10(value) : value; +} + +double unmap(bool log, double point) +{ + return log ? std::pow(10.0, point) : point; +} + +// False for NaN as well +bool shows(const Range &range) +{ + return range.max > range.min && (!range.log || range.min > 0.0); +} + +} + +Limits +pitchLimits() +{ + Limits limits; + limits.lowest = 27.5; // A0 + limits.highest = 4186.009; // C8 + limits.narrowest = std::pow(2.0, 4.0 / 12.0); // a major third + return limits; +} + +double +valueAtY(const Range &range, double height, double y) +{ + double bottom = map(range.log, range.min); + double top = map(range.log, range.max); + return unmap(range.log, bottom + (top - bottom) * (height - y) / height); +} + +double +yForValue(const Range &range, double height, double value) +{ + double bottom = map(range.log, range.min); + double top = map(range.log, range.max); + return height - height * (map(range.log, value) - bottom) / (top - bottom); +} + +Range +zoomedAbout(const Range &range, double factor, double value, + double height, double y) +{ + double span = (map(range.log, range.max) - map(range.log, range.min)) + / factor; + double bottom = map(range.log, value) - span * (height - y) / height; + + Range zoomed; + zoomed.min = unmap(range.log, bottom); + zoomed.max = unmap(range.log, bottom + span); + zoomed.log = range.log; + return zoomed; +} + +Range +limited(const Range &range, const Limits &limits, double height, double y) +{ + bool log = range.log; + + Range all; + all.min = limits.lowest; + all.max = limits.highest; + all.log = log; + + if (!shows(range)) return all; + + Range result = range; + + if (result.max < result.min * limits.narrowest) { + + // Widened about what is at y, or at the edge the fingers are + // beyond, as far as the limits have it + double at = std::min(std::max(y, 0.0), height); + double value = std::min(std::max(valueAtY(result, height, at), + limits.lowest), + limits.highest); + + if (log) { + double span = std::log10(result.max / result.min); + result = zoomedAbout(result, + span / std::log10(limits.narrowest), + value, height, at); + } else { + // min + (max - min) * above = value, with max = narrowest * min + double above = (height - at) / height; + result.min = value / (1.0 + above * (limits.narrowest - 1.0)); + result.max = result.min * limits.narrowest; + } + } + + double span = map(log, result.max) - map(log, result.min); + double lowest = map(log, limits.lowest); + double highest = map(log, limits.highest); + + if (span >= highest - lowest) return all; + + // Moved along the mapped axis, so that it keeps its size on screen + double shift = 0.0; + if (map(log, result.min) < lowest) { + shift = lowest - map(log, result.min); + } else if (map(log, result.max) > highest) { + shift = highest - map(log, result.max); + } + + if (shift != 0.0) { + double bottom = map(log, result.min) + shift; + result.min = unmap(log, bottom); + result.max = unmap(log, bottom + span); + // Exactly on the limit, not a rounding error either side of it + if (shift > 0.0) result.min = limits.lowest; + else result.max = limits.highest; + } + + return result; +} + +} diff --git a/main/VerticalZoom.h b/main/VerticalZoom.h new file mode 100644 index 00000000..e6556441 --- /dev/null +++ b/main/VerticalZoom.h @@ -0,0 +1,88 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_VERTICAL_ZOOM_H +#define TONY_VERTICAL_ZOOM_H + +/** + * The arithmetic of two fingers on a pane's vertical axis: the value + * at a height in the pane, the range that zooms about a value held at + * a height, and the limits a range is kept within. TouchGestures does + * as the answers say. + * + * The mapping is svgui's CoordinateScale's for a vertical scale: y + * from the top of the pane, the range's minimum at the bottom edge (y + * is the height) and its maximum at the top (y is 0), the values + * between spaced evenly on a linear scale and by ratio on a + * logarithmic one. Zooming and scrolling happen on that mapped axis, + * so that on a log scale an octave keeps its height wherever it is. + * + * Nothing here touches a view, so all of it is tested without a + * window (TestVerticalZoom). + */ +namespace VerticalZoom +{ + /// The values at the bottom and the top of a pane, and the scale + struct Range { + double min = 0.0; + double max = 0.0; + bool log = false; + }; + + inline bool operator==(const Range &a, const Range &b) { + return a.min == b.min && a.max == b.max && a.log == b.log; + } + inline bool operator!=(const Range &a, const Range &b) { + return !(a == b); + } + + /// How far a range may go + struct Limits { + double lowest = 0.0; // its min no lower than this + double highest = 0.0; // its max no higher + double narrowest = 1.0; // its max at least this many times its min + }; + + /** + * For a pitch track: from A0 to C8, the piano's range, which holds + * any note that is sung and the default range with room to spare; + * and four semitones at the narrowest. + */ + Limits pitchLimits(); + + /// The value at y in a pane of the given height showing range + double valueAtY(const Range &range, double height, double y); + + /// Where value is in a pane of the given height showing range + double yForValue(const Range &range, double height, double value); + + /** + * The range factor times narrower than range, on its own scale, + * that shows value at y: above 1 zooms in, below 1 out. With a + * factor of 1 it is range scrolled, whatever it had at y. + */ + Range zoomedAbout(const Range &range, double factor, double value, + double height, double y); + + /** + * range within limits: widened about y, if narrower than they + * allow; then moved inside them, or made all of them if wider. + * A range within the limits comes back as it is. One that shows + * nothing (no width, or a log scale from zero) becomes all of them. + */ + Range limited(const Range &range, const Limits &limits, + double height, double y); +} + +#endif diff --git a/main/test/TestTouchGestures.h b/main/test/TestTouchGestures.h index 29684916..b7e55d94 100644 --- a/main/test/TestTouchGestures.h +++ b/main/test/TestTouchGestures.h @@ -16,7 +16,9 @@ // Tier 5: touch on the panes of the real MainWindow (TouchGestures). // Pinch, two fingers dragged and a long press; one finger and the -// mouse as they were. +// mouse as they were. Up and down, two fingers zoom and scroll the +// analyser's frequency range, which is checked through svgui's own +// mapping of the pane. // // The touch goes in where a platform's does (QTest::touchEvent goes // through QWindowSystemInterface), so Qt makes mouse events of it as it @@ -28,14 +30,21 @@ #include "../MainWindow.h" #include "../Analyser.h" +#include "../AlternatePitchTrack.h" #include "../PinchZoom.h" #include "../TouchGestures.h" +#include "../VerticalZoom.h" #include "version.h" #include "view/Pane.h" #include "view/PaneStack.h" #include "view/ViewManager.h" +#include "layer/CoordinateScale.h" +#include "layer/Layer.h" +#include "layer/TimeValueLayer.h" +#include "widgets/CommandHistory.h" +#include "widgets/InteractiveFileFinder.h" #include "data/fileio/WavFileWriter.h" #include "data/model/RelativelyFineZoomConstraint.h" #include "transform/ModelTransformerFactory.h" @@ -49,10 +58,12 @@ #include #include #include +#include #include #include #include +#include #include #include @@ -69,6 +80,8 @@ class TouchTestWindow : public MainWindow sv::PaneStack *paneStack() { return m_paneStack; } sv::ViewManager *viewManager() { return m_viewManager; } Analyser *analyser() { return m_analyser; } + AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } + void toggleAlternatePitch() { alternatePitchToggled(); } void discardModifications() { m_documentModified = false; } void doCloseSession() { discardModifications(); closeSession(); } @@ -120,6 +133,51 @@ class TestTouchGestures : public QObject return m_window->viewManager()->getSelections(); } + // The analyser's frequency range, which the pane draws pitch on + VerticalZoom::Range frequencyRange() { + VerticalZoom::Range r; + r.log = true; + m_window->analyser()->getDisplayFrequencyExtents(r.min, r.max); + return r; + } + + static double octaves(const VerticalZoom::Range &r) { + return std::log2(r.max / r.min); + } + + static QString text(const VerticalZoom::Range &r) { + return QString("%1 to %2 Hz").arg(r.min).arg(r.max); + } + + // Where the pane draws a frequency, and what it draws at y: svgui's + // mapping, not the one under test + double yForFrequency(double f) { + return pane()->getEffectiveVerticalExtents("Hz") + .getCoordForValue(pane(), f); + } + + double frequencyAtY(double y) { + return pane()->getEffectiveVerticalExtents("Hz") + .getValueForCoord(pane(), y); + } + + // How far the fingers go up or down before that counts + static int deadZone() { + return 2 * QGuiApplication::styleHints()->startDragDistance(); + } + + // Two fingers down together, moved in ten even steps, and lifted + // together + void twoFingers(sv::Pane *p, QPoint a0, QPoint b0, QPoint a1, QPoint b1) { + Touch t = touch(); + t.press(0, a0, p).press(1, b0, p).commit(); + for (int i = 1; i <= 10; ++i) { + t.move(0, a0 + (a1 - a0) * i / 10, p) + .move(1, b0 + (b1 - b0) * i / 10, p).commit(); + } + t.release(0, a1, p).release(1, b1, p).commit(); + } + bool analysed() { Analyser *a = m_window->analyser(); return a && a->getLayer(Analyser::PitchTrack) && @@ -196,6 +254,10 @@ private slots: false); settings.endGroup(); + // As main() does; without it a .ton file is not a session + sv::InteractiveFileFinder::getInstance() + ->setApplicationSessionExtension("ton"); + connect(&m_watchdog, &QTimer::timeout, this, [this]() { dismissDialog(); }); m_watchdog.start(50); @@ -557,6 +619,311 @@ private slots: QVERIFY(p->getZoomLevel() == expected); QCOMPARE(p->getCentreFrame(), centre); } + + // Fingers spread up the pane: the frequency range narrows by as + // much as they spread, less what the dead zone took, about the + // frequency between them, which stays there. The time axis stays as + // it was, the pitch layers in the pane are drawn on the new range, + // and nothing goes into the undo history + void vertical_pinch_narrows_the_frequency_range() { + openWindow(); + if (QTest::currentTestFailed()) return; + + m_window->toggleAlternatePitch(); + QVERIFY(m_window->alternatePitch()->isShown()); + + sv::Pane *p = pane(); + QVERIFY2(p->height() >= 300, + qPrintable(QString("the pane is %1 high").arg(p->height()))); + int x = p->width() / 2 + 100; + int y = p->height() / 2 - 60; + sv::sv_frame_t centre = p->getCentreFrame(); + VerticalZoom::Range before = frequencyRange(); + QCOMPARE(before.min, 40.0); + QCOMPARE(before.max, 1500.0); + double held = frequencyAtY(y); + + int commands = 0; + auto counted = connect(sv::CommandHistory::getInstance(), + qOverload<>(&sv::CommandHistory::commandExecuted), + this, [&commands]() { ++commands; }); + twoFingers(p, QPoint(x, y - 50), QPoint(x, y + 50), + QPoint(x, y - 100), QPoint(x, y + 100)); + disconnect(counted); + QCOMPARE(commands, 0); + + VerticalZoom::Range after = frequencyRange(); + double factor = (200.0 - deadZone()) / 100.0; + double narrowed = octaves(before) / octaves(after); + QVERIFY2(std::fabs(narrowed - factor) < 0.01 * factor, + qPrintable(QString("%1, then %2: %3 times narrower, not %4") + .arg(text(before)).arg(text(after)) + .arg(narrowed).arg(factor))); + QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, + qPrintable(QString("%1 Hz was at %2, then at %3") + .arg(held).arg(y).arg(yForFrequency(held)))); + + // As far as the centre goes, to the whole pixel the fingers hold + QCOMPARE(framesPerPixel(), double(startLevel)); + QVERIFY(std::abs(p->getCentreFrame() - centre) < startLevel); + + // The reference's pitch and notes, and the alternate pitch track + // that follows them + Analyser *a = m_window->analyser(); + std::vector layers { + a->getLayer(Analyser::PitchTrack), + a->getLayer(Analyser::Notes), + m_window->alternatePitch()->getLayer() + }; + for (sv::Layer *layer : layers) { + QVERIFY(layer); + sv::CoordinateScale scale = + p->getEffectiveVerticalExtentsForLayer(layer); + QVERIFY2(scale.isLogarithmic() && + scale.getDisplayMinimum() == after.min && + scale.getDisplayMaximum() == after.max, + qPrintable(QString("%1 is drawn from %2 to %3") + .arg(layer->objectName()) + .arg(scale.getDisplayMinimum()) + .arg(scale.getDisplayMaximum()))); + } + + QCOMPARE(m_window->menuRequests, 0); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // Fingers brought together up the pane: the range widens, about + // the frequency between them + void vertical_pinch_widens_the_frequency_range() { + openWindow(); + if (QTest::currentTestFailed()) return; + + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(100, 800)); + + sv::Pane *p = pane(); + int x = p->width() / 2 - 100; + int y = p->height() / 2 - 40; + VerticalZoom::Range before = frequencyRange(); + QCOMPARE(before.min, 100.0); + double held = frequencyAtY(y); + + twoFingers(p, QPoint(x, y - 100), QPoint(x, y + 100), + QPoint(x, y - 50), QPoint(x, y + 50)); + + VerticalZoom::Range after = frequencyRange(); + double factor = (100.0 + deadZone()) / 200.0; + double narrowed = octaves(before) / octaves(after); + QVERIFY2(std::fabs(narrowed - factor) < 0.01 * factor, + qPrintable(QString("%1, then %2: %3 times narrower, not %4") + .arg(text(before)).arg(text(after)) + .arg(narrowed).arg(factor))); + QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, + qPrintable(QString("%1 Hz was at %2, then at %3") + .arg(held).arg(y).arg(yForFrequency(held)))); + QCOMPARE(framesPerPixel(), double(startLevel)); + } + + // Fingers spread across the pane, not quite level, and drifting down + // a little: the time axis zooms, and the frequency range is left + // exactly as it was. Their spread up the pane grows by more than the + // dead zone, but by less than half as much as across + void horizontal_pinch_leaves_the_frequency_range() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 2; + int drift = deadZone() / 2; + VerticalZoom::Range before = frequencyRange(); + + twoFingers(p, QPoint(x - 50, y - 15), QPoint(x + 50, y + 15), + QPoint(x - 150, y - 35 + drift), + QPoint(x + 150, y + 35 + drift)); + + QVERIFY(framesPerPixel() < startLevel / 2.0); + VerticalZoom::Range after = frequencyRange(); + QCOMPARE(after.min, before.min); + QCOMPARE(after.max, before.max); + } + + // Spread along both: both zoom, each about the fingers + void diagonal_pinch_zooms_time_and_frequency() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2 - 100; + int y = p->height() / 2 - 30; + sv::sv_frame_t frame = p->getFrameForX(x); + VerticalZoom::Range before = frequencyRange(); + double held = frequencyAtY(y); + + twoFingers(p, QPoint(x - 50, y - 50), QPoint(x + 50, y + 50), + QPoint(x - 100, y - 100), QPoint(x + 100, y + 100)); + + QCOMPARE(framesPerPixel(), startLevel / 2.0); + QVERIFY2(std::abs(p->getFrameForX(x) - frame) <= startLevel / 2, + qPrintable(QString("frame %1 under the fingers, then %2") + .arg(frame).arg(p->getFrameForX(x)))); + + VerticalZoom::Range after = frequencyRange(); + double factor = (200.0 - deadZone()) / 100.0; + double narrowed = octaves(before) / octaves(after); + QVERIFY2(std::fabs(narrowed - factor) < 0.01 * factor, + qPrintable(QString("%1, then %2: %3 times narrower, not %4") + .arg(text(before)).arg(text(after)) + .arg(narrowed).arg(factor))); + QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, + qPrintable(QString("%1 Hz was at %2, then at %3") + .arg(held).arg(y).arg(yForFrequency(held)))); + } + + // Two fingers side by side dragged down the pane: what was between + // them goes down with them, as far as they went beyond the dead + // zone, and the range keeps its height. Time stays where it was + void vertical_two_finger_drag_scrolls_the_frequency_range() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 2 - 80; + sv::sv_frame_t centre = p->getCentreFrame(); + VerticalZoom::Range before = frequencyRange(); + double held = frequencyAtY(y); + + twoFingers(p, QPoint(x - 50, y), QPoint(x + 50, y), + QPoint(x - 50, y + 120), QPoint(x + 50, y + 120)); + + VerticalZoom::Range after = frequencyRange(); + double expected = y + 120 - deadZone(); + QVERIFY2(std::fabs(yForFrequency(held) - expected) <= 1.0, + qPrintable(QString("%1 Hz was at %2, then at %3, not %4") + .arg(held).arg(y).arg(yForFrequency(held)) + .arg(expected))); + QVERIFY2(after.max > before.max && + std::fabs(octaves(after) - octaves(before)) < 0.01, + qPrintable(QString("%1, then %2") + .arg(text(before)).arg(text(after)))); + + // As far as the centre goes, to the whole pixel the fingers hold + QCOMPARE(framesPerPixel(), double(startLevel)); + QVERIFY(std::abs(p->getCentreFrame() - centre) < startLevel); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // No narrower than a major third, no wider than the piano, and no + // further up or down: held there, the range stays while the fingers + // go on, and goes back with them as soon as they turn. The range is + // whole Hz (the spectrogram's), hence the half hertz allowed + void frequency_range_stays_within_its_limits() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 2; + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(200, 260)); + twoFingers(p, QPoint(x, y - 50), QPoint(x, y + 50), + QPoint(x, y - 120), QPoint(x, y + 120)); + VerticalZoom::Range r = frequencyRange(); + double semitones = 12.0 * octaves(r); + QVERIFY2(semitones > 3.9 && semitones < 4.1, + qPrintable(text(r) + QString(": %1 semitones") + .arg(semitones))); + + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(40, 1500)); + twoFingers(p, QPoint(x, y - 120), QPoint(x, y + 120), + QPoint(x, y - 20), QPoint(x, y + 20)); + r = frequencyRange(); + QVERIFY2(std::fabs(r.min - limits.lowest) <= 0.5 && + std::fabs(r.max - limits.highest) <= 0.5, + qPrintable(text(r))); + + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(40, 1500)); + VerticalZoom::Range start = frequencyRange(); + int top = 20; + int bottom = p->height() - 40; + Touch t = touch(); + t.press(0, QPoint(x - 50, top), p).press(1, QPoint(x + 50, top), p) + .commit(); + for (int i = 1; i <= 10; ++i) { + int at = top + (bottom - top) * i / 10; + t.move(0, QPoint(x - 50, at), p).move(1, QPoint(x + 50, at), p) + .commit(); + } + r = frequencyRange(); + QVERIFY2(std::fabs(r.max - limits.highest) <= 0.5 && + std::fabs(octaves(r) - octaves(start)) < 0.01, + qPrintable(text(start) + ", then " + text(r))); + + double atTurn = frequencyAtY(bottom); + for (int i = 1; i <= 5; ++i) { + int at = bottom - 10 * i; + t.move(0, QPoint(x - 50, at), p).move(1, QPoint(x + 50, at), p) + .commit(); + } + t.release(0, QPoint(x - 50, bottom - 50), p) + .release(1, QPoint(x + 50, bottom - 50), p).commit(); + QVERIFY2(std::fabs(yForFrequency(atTurn) - (bottom - 50)) <= 1.0, + qPrintable(QString("%1 Hz was at %2, then at %3") + .arg(atTurn).arg(bottom) + .arg(yForFrequency(atTurn)))); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // The range the fingers leave is the session's: saved with it, and + // there again when it is opened + void frequency_range_is_saved_with_the_session() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 2; + twoFingers(p, QPoint(x, y - 50), QPoint(x, y + 50), + QPoint(x, y - 100), QPoint(x, y + 120)); + VerticalZoom::Range zoomed = frequencyRange(); + QVERIFY(zoomed.min > 40.0 && zoomed.max < 1500.0); + + QString session = m_dir.filePath("zoomed.ton"); + QVERIFY(m_window->saveSessionFile(session)); + m_window->doCloseSession(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QTRY_VERIFY_WITH_TIMEOUT(analysed(), 30000); + + VerticalZoom::Range restored = frequencyRange(); + QCOMPARE(restored.min, zoomed.min); + QCOMPARE(restored.max, zoomed.max); + sv::CoordinateScale scale = pane()->getEffectiveVerticalExtentsForLayer + (m_window->analyser()->getLayer(Analyser::PitchTrack)); + QCOMPARE(scale.getDisplayMinimum(), zoomed.min); + QCOMPARE(scale.getDisplayMaximum(), zoomed.max); + } + + // The selection strip shows no frequencies: two fingers dragged up + // it leave the range alone + void frequency_range_is_the_pitch_pane_s_only() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *s = strip(); + int x = s->width() / 2; + int y = s->height() - 5; + VerticalZoom::Range before = frequencyRange(); + + twoFingers(s, QPoint(x - 50, y), QPoint(x + 50, y), + QPoint(x - 50, y - 60), QPoint(x + 50, y - 60)); + + VerticalZoom::Range after = frequencyRange(); + QCOMPARE(after.min, before.min); + QCOMPARE(after.max, before.max); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } }; #endif diff --git a/main/test/TestVerticalZoom.h b/main/test/TestVerticalZoom.h new file mode 100644 index 00000000..f7bc8985 --- /dev/null +++ b/main/test/TestVerticalZoom.h @@ -0,0 +1,244 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_VERTICAL_ZOOM_H +#define TEST_VERTICAL_ZOOM_H + +// Tier 2: the arithmetic of two fingers on a pane's vertical axis, and +// when a movement of the fingers along an axis counts. Ranges and +// pixels in, ranges and pixels out; no view. That the mapping is the +// pane's own is checked against svgui in TestTouchGestures. + +#include "../PinchZoom.h" +#include "../VerticalZoom.h" + +#include +#include + +#include +#include + +class TestVerticalZoom : public QObject +{ + Q_OBJECT + + typedef VerticalZoom::Range Range; + + static Range range(double min, double max, bool log) { + Range r; + r.min = min; + r.max = max; + r.log = log; + return r; + } + + static QString text(const Range &r) { + return QString("%1 to %2 (%3)").arg(r.min).arg(r.max) + .arg(r.log ? "log" : "linear"); + } + + // The height of the range on its own scale: octaves on a log one + static double span(const Range &r) { + return r.log ? std::log2(r.max / r.min) : r.max - r.min; + } + + static bool near(double a, double b, double tolerance) { + return std::fabs(a - b) <= tolerance; + } + +private slots: + // Bottom edge the minimum, top edge the maximum; half way up a log + // scale the geometric mean, a linear one the arithmetic + void value_at_y_on_both_scales() { + Range log = range(40.0, 1500.0, true); + QCOMPARE(VerticalZoom::valueAtY(log, 400.0, 400.0), 40.0); + QCOMPARE(VerticalZoom::valueAtY(log, 400.0, 0.0), 1500.0); + QVERIFY(near(VerticalZoom::valueAtY(log, 400.0, 200.0), + std::sqrt(40.0 * 1500.0), 1.0e-9)); + + Range linear = range(100.0, 300.0, false); + QCOMPARE(VerticalZoom::valueAtY(linear, 200.0, 50.0), 250.0); + QCOMPARE(VerticalZoom::valueAtY(linear, 200.0, 200.0), 100.0); + } + + void y_for_value_is_the_inverse() { + for (bool log : { true, false }) { + Range r = range(55.0, 880.0, log); + for (double y : { 0.0, 13.5, 200.0, 377.0, 400.0 }) { + double value = VerticalZoom::valueAtY(r, 400.0, y); + double back = VerticalZoom::yForValue(r, 400.0, value); + QVERIFY2(near(back, y, 1.0e-9), + qPrintable(QString("%1: y %2 came back as %3") + .arg(text(r)).arg(y).arg(back))); + } + } + } + + // Twice as far in shows half as many octaves, with the value asked + // for where it was asked for + void zoom_keeps_the_value_at_y() { + for (bool log : { true, false }) { + Range start = range(40.0, 1500.0, log); + for (double factor : { 0.5, 1.0, 1.7, 2.0, 8.0 }) { + for (double y : { 0.0, 120.0, 250.0, 400.0 }) { + double value = 220.0; + Range r = VerticalZoom::zoomedAbout + (start, factor, value, 400.0, y); + QString what = QString("%1 by %2 about %3 at %4: %5") + .arg(text(start)).arg(factor).arg(value).arg(y) + .arg(text(r)); + QVERIFY2(r.log == log, qPrintable(what)); + QVERIFY2(near(VerticalZoom::yForValue(r, 400.0, value), + y, 1.0e-6), qPrintable(what)); + QVERIFY2(near(span(r), span(start) / factor, + 1.0e-9 * span(start)), qPrintable(what)); + } + } + } + } + + // A factor of one moves the range: what was at 300 is at 200, and + // the range is as tall as it was. A quarter of the pane is half an + // octave, which the range goes down by + void factor_one_scrolls() { + Range start = range(100.0, 400.0, true); + double value = VerticalZoom::valueAtY(start, 400.0, 300.0); + Range r = VerticalZoom::zoomedAbout(start, 1.0, value, 400.0, 200.0); + QVERIFY(near(VerticalZoom::yForValue(r, 400.0, value), 200.0, 1.0e-6)); + QVERIFY(near(span(r), 2.0, 1.0e-9)); + QVERIFY(near(r.min, 100.0 / std::sqrt(2.0), 1.0e-9)); + } + + void pitch_limits_are_the_piano_and_a_major_third() { + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + QCOMPARE(limits.lowest, 27.5); + QVERIFY(near(limits.highest, 4186.009, 0.001)); + QVERIFY(near(12.0 * std::log2(limits.narrowest), 4.0, 1.0e-9)); + } + + void a_range_within_the_limits_is_left_as_it_is() { + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + for (bool log : { true, false }) { + for (Range r : { range(40.0, 1500.0, log), + range(27.5, 4186.009, log), + range(200.0, 252.0, log) }) { + r.log = log; + Range l = VerticalZoom::limited(r, limits, 400.0, 100.0); + QVERIFY2(l == r, qPrintable(text(r) + " became " + text(l))); + } + } + } + + // Too narrow: widened to a major third about the value at y, which + // stays there + void a_narrow_range_is_widened_about_y() { + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + for (bool log : { true, false }) { + for (double y : { 0.0, 100.0, 400.0 }) { + Range r = range(200.0, 210.0, log); + double value = VerticalZoom::valueAtY(r, 400.0, y); + Range l = VerticalZoom::limited(r, limits, 400.0, y); + QString what = text(r) + " became " + text(l); + QVERIFY2(near(l.max / l.min, limits.narrowest, 1.0e-9), + qPrintable(what)); + QVERIFY2(near(VerticalZoom::valueAtY(l, 400.0, y), value, + 1.0e-9), qPrintable(what)); + } + } + } + + // Past an end: moved back inside, as tall as it was + void a_range_past_an_end_is_moved_inside() { + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + + Range low = VerticalZoom::limited(range(20.0, 100.0, true), + limits, 400.0, 200.0); + QCOMPARE(low.min, 27.5); + QVERIFY(near(low.max, 137.5, 1.0e-6)); + + Range high = VerticalZoom::limited(range(3000.0, 6000.0, true), + limits, 400.0, 200.0); + QCOMPARE(high.max, limits.highest); + QVERIFY(near(high.min, limits.highest / 2.0, 1.0e-6)); + + Range linear = VerticalZoom::limited(range(10.0, 110.0, false), + limits, 400.0, 200.0); + QCOMPARE(linear.min, 27.5); + QVERIFY(near(linear.max, 127.5, 1.0e-6)); + } + + // Wider than the limits, or showing nothing: all of them + void a_wide_or_empty_range_is_all_of_the_limits() { + VerticalZoom::Limits limits = VerticalZoom::pitchLimits(); + double nan = std::numeric_limits::quiet_NaN(); + for (Range r : { range(10.0, 20000.0, true), + range(0.0, 1500.0, true), + range(-5.0, 1500.0, true), + range(500.0, 500.0, true), + range(600.0, 500.0, true), + range(nan, 500.0, true), + range(10.0, 20000.0, false), + range(40.0, nan, false) }) { + Range l = VerticalZoom::limited(r, limits, 400.0, 200.0); + QVERIFY2(l.min == limits.lowest && l.max == limits.highest && + l.log == r.log, + qPrintable(text(r) + " became " + text(l))); + } + } + + // Nothing within the dead zone; beyond it, all but the dead zone, + // and then all the way back through it + void movement_counts_beyond_the_dead_zone() { + PinchZoom::AxisMovement m(20.0); + QCOMPARE(m.update(10.0, 0.0), 0.0); + QCOMPARE(m.update(-20.0, 0.0), 0.0); + QVERIFY(!m.isCounting()); + QCOMPARE(m.update(25.0, 0.0), 5.0); + QVERIFY(m.isCounting()); + QCOMPARE(m.update(40.0, 0.0), 20.0); + QCOMPARE(m.update(0.0, 0.0), -20.0); + QCOMPARE(m.update(-30.0, 500.0), -50.0); + + PinchZoom::AxisMovement down(20.0); + QCOMPARE(down.update(-50.0, 0.0), -30.0); + } + + // Less than half as much as across: not meant + void movement_across_the_axis_does_not_count() { + PinchZoom::AxisMovement m(20.0); + QCOMPARE(m.update(30.0, 100.0), 0.0); + QCOMPARE(m.update(49.0, -100.0), 0.0); + QVERIFY(!m.isCounting()); + QCOMPARE(m.update(50.0, 100.0), 30.0); + QVERIFY(m.isCounting()); + } + + // Started from outside: it counts from where it is, however far + // that is, so that nothing jumps + void movement_started_counts_from_there() { + PinchZoom::AxisMovement m(20.0); + m.start(8.0); + QVERIFY(m.isCounting()); + QCOMPARE(m.update(8.0, 0.0), 0.0); + QCOMPARE(m.update(18.0, 0.0), 10.0); + m.start(100.0); // already counting: no change + QCOMPARE(m.update(18.0, 0.0), 10.0); + + PinchZoom::AxisMovement far(20.0); + far.start(-50.0); + QCOMPARE(far.update(-50.0, 0.0), 0.0); + QCOMPARE(far.update(-60.0, 500.0), -10.0); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 52e33e1b..ceee6d6c 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -25,6 +25,7 @@ #include "TestSingingTakes.h" #include "TestTakesFile.h" #include "TestTakeTiming.h" +#include "TestVerticalZoom.h" #include "RunSuite.h" @@ -133,6 +134,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestVerticalZoom t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + (void)good; if (bad > 0) { diff --git a/meson.build b/meson.build index a7c5b31d..d0a2f2c5 100644 --- a/meson.build +++ b/meson.build @@ -1180,6 +1180,7 @@ tony_core_files = [ 'main/TakeEvents.cpp', 'main/TakesFile.cpp', 'main/TakeTiming.cpp', + 'main/VerticalZoom.cpp', ] tony_app_files = [ @@ -1494,6 +1495,7 @@ if system != 'android' 'main/test/TestSingingTakes.h', 'main/test/TestTakesFile.h', 'main/test/TestTakeTiming.h', + 'main/test/TestVerticalZoom.h', ]) tony_core_test_exe = executable( From b2cfa87cb2cf6229688293e0d77bbdf82243e773 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 10:49:34 +0000 Subject: [PATCH 187/275] docs: phase a4b done, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 4063f2b4..af2aa85b 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -161,7 +161,7 @@ builds happen in the container.) - A6 — Oboe audio backend. Done. - A7 — Sessions in place on the phone, and fixes from the first phone test. Done. - A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. -- A4b — Vertical zoom and scroll by touch. +- A4b — Vertical zoom and scroll by touch. Done. - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. - A8 — Documentation pass. @@ -675,3 +675,28 @@ ContentResolver with the URI exactly as Android wrote it. Tests seen failing: `the_uri_is_opened_as_android_wrote_it` with `PrettyDecoded` (lead). Left open: none of it run on a phone. Layer import still goes through svgui's dialog. +### Phase A4b — 2026-09-26 +Built: `main/VerticalZoom` (tony_core; `TestVerticalZoom`): value at y and back on a log or +linear scale (svgui's CoordinateScale mapping), zoom about a value held at y, `limited()` +(`pitchLimits()`: A0 to C8, a major third at the narrowest). `PinchZoom::AxisMovement`: a +movement along an axis counts past a dead zone (2x slop) if at least half that across, then +less the dead zone. `TouchGestures`: each axis zooms by the fingers' spread along it (never +under 4x slop); the time axis as A4 had it, with the spread across in place of the distance; +the range about the value between the fingers, following them up and down past the dead zone; +held at a limit, re-anchored. `TouchGestures::VerticalRange` (get/set/limits); `paneAdded()` +gives it the reference analyser's `get/setDisplayFrequencyExtents()`, in its pane only, and +only while the pane draws Hz on that range. 8 app tests in `TestTouchGestures`. +Found: the range is the primary analyser's `SpectrogramLayer` (MelodicRange: log, 40-1500 Hz, +dormant). Pitch, notes, the take's layers, live dots, the alternate track are AutoAlignScale: +they defer to the topmost "Hz" layer with a scale of its own, dormant or not (View:: +getEffectiveVerticalExtents), which is it. Not undoable, no modified flag; saved in the session +(the spectrogram's minFrequency/maxFrequency) and restored with it; a new reference resets it. +Choices / deviations: +- The spectrogram keeps whole Hz (int, lrint): steps of up to ~12 px at a major third about + 110 Hz on a 300 px pane. svgui fork change to fix: `m_minFrequency`/`m_maxFrequency` double, + no lrint in `setDisplayExtents()`, `toDouble()` in `setProperties()`. Not made. +- Time scroll keeps A4's (no dead zone); the range is sticky: a drag within ~27 degrees of + across never moves it. +Tests seen failing: across rule removed; zoom about the middle; no limits; spectrogram unsaved. +Left open: not on a phone. A drag up may also scroll the pane stack, if it can scroll. + From 449c7e4a3168bfd40d595c84dea584067c272f32 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:04:14 +0000 Subject: [PATCH 188/275] docs: Calibrate Audio and the dev checks as built calibrate-audio.md describes what was built instead of the plan: the button, the verdicts, the dev run's stages and checks, what to send back, the architecture, tests, decisions, open points and the user's runs. recording.md gets the round trip's two sources and the audio check's takes; testing.md the dev checks' suite and the fake device's new fields; architecture.md the new classes and who owns them; building.md that Ubuntu's Qt 6.4 now works; open-points.md the lower-latency driver project, MME's restart jitter and the weak spots the checks left; takes.md two limits found on the way. The phase work orders and their log are removed with the work they steered. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- AGENTS.md | 8 +- docs/README.md | 6 +- docs/architecture.md | 26 +- docs/building.md | 40 +- docs/calibrate-audio-log.md | 572 -------------- docs/calibrate-audio-work-orders.md | 642 ---------------- docs/calibrate-audio.md | 1097 +++++++++++++++------------ docs/forks.md | 10 +- docs/manual-checklist.md | 16 +- docs/open-points.md | 56 +- docs/recording.md | 52 +- docs/takes.md | 10 +- docs/testing.md | 121 ++- 13 files changed, 898 insertions(+), 1758 deletions(-) delete mode 100644 docs/calibrate-audio-log.md delete mode 100644 docs/calibrate-audio-work-orders.md diff --git a/AGENTS.md b/AGENTS.md index 6417f46d..01d228a7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,6 +18,7 @@ code. | [docs/architecture.md](docs/architecture.md) | touching layers, models, the document, commands, playback or the session file | | [docs/recording.md](docs/recording.md) | touching `record()`, the Stop path, latency, pre-roll, the live tracker | | [docs/takes.md](docs/takes.md) | touching takes, the audio swap, ranged analysis, undo, the coverage strip, save/restore | +| [docs/calibrate-audio.md](docs/calibrate-audio.md) | touching Calibrate Audio (`AudioCheckRunner`, `CalibrateAudioDialog`), the dev checks (`main/dev/`) or the measured latency (`LatencyCheck`, `LatencyCalibration`) | | [docs/forks.md](docs/forks.md) | needing a change in `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` | | [docs/open-points.md](docs/open-points.md), [docs/manual-checklist.md](docs/manual-checklist.md) | choosing what to do next, or saying what the user should try by hand | @@ -42,7 +43,7 @@ Run tests from `build_mingw/` with the same environment: ```sh mkdir -p ../tmp/tl TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core.exe > ../tmp/test.log 2>&1; echo "exit:$?" -TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe > ../tmp/test.log 2>&1; echo "exit:$?" # ~5 min, real time +TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe > ../tmp/test.log 2>&1; echo "exit:$?" # ~9 min, real time TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe undo_two_takes_in_order > ../tmp/test.log 2>&1 grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt ``` @@ -52,11 +53,12 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt that lack it fail, so the exit status is only meaningful for a run with no names. - Run named tests while working; run **both whole suites** before calling anything done. - `test-tony-dev.exe` (development builds only) holds the development checks' suite, - about a minute and more of real-time takes. Run it as well, whole, when a change touches + about four minutes of real-time takes. Run it as well, whole, when a change touches the take path (`record()`, Stop, latency, pre-roll), `AudioCheckRunner`, `CalibrateAudioDialog` or `main/dev/`; "both whole suites" then means all three. - From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. -- Give the app suite a tool timeout of 10 minutes. +- Give the app suite a tool timeout of 10 minutes, or run it in the background: it comes + close to that. ## Rules for working here diff --git a/docs/README.md b/docs/README.md index 84455fb4..cf264464 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,11 +12,11 @@ methods, and they do not tell the story of fixed bugs. | Page | What is in it | | --- | --- | | [building.md](building.md) | The MinGW build, and every way the environment has gone wrong | -| [testing.md](testing.md) | The two test executables, the fakes and helpers, how tests turned out to be worthless, races | +| [testing.md](testing.md) | The test executables (core, app, and the dev checks'), the fakes and helpers, how tests turned out to be worthless, races | | [architecture.md](architecture.md) | What the fork adds, `tony_core` / `tony_app`, who owns what, and the rules for layers, models, commands, playback and session files | | [recording.md](recording.md) | Record to Stop, step by step and why in that order; latency; pre-roll; record into selection; the live tracker | | [takes.md](takes.md) | Takes: decisions, the audio swap, ranged analysis and merge, undo, the coverage strip, files and sessions, limitations | | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | -| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — little of it tried yet | -| [calibrate-audio.md](calibrate-audio.md) | Plan, not built: a Calibrate Audio button that measures the round trip through a speaker-to-mic loopback, and dev checks that automate most of the manual checklist | +| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes, starting with the device check (Calibrate Audio with the dev checks). Tried so far: Calibrate Audio and part of a dev run on the user's PC, and the looks from cloud screenshots; the rest not yet | +| [calibrate-audio.md](calibrate-audio.md) | Calibrate Audio, which measures the round trip through an earcup held to the mic, and the dev checks that settle the checklist's device items: what they do, what each verdict and check means, which numbers to send back, design, tests, decisions, open points, the user's runs | diff --git a/docs/architecture.md b/docs/architecture.md index 090310a3..49f5a647 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -13,7 +13,8 @@ Upstream Tony analyses the pitch of one recording. This fork makes it a singing 3. **pYIN analysis of what was just recorded** replaces the dots when the take stops. 4. **Partial recordings and takes**: record from the playhead into part of the song, keep the rest, erase, undo, and keep several takes. See [takes.md](takes.md). -5. Around that: play the reference while recording, latency compensation, pre-roll, +5. Around that: play the reference while recording, latency compensation (with a round + trip Calibrate Audio can measure), pre-roll, record into selection, an octave-shifted "alternate" pitch track to follow, and a background music track that is played but never analysed. @@ -30,8 +31,13 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `ModelChangeThrottle`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h` | -| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `TakeCommands`, `TakeLayers`, `PaneUtils` | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `ModelChangeThrottle`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `LatencyUtils.h`, `LatencyCheck`, `LatencyCalibration`, `TakeDiff` | +| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `TakeCommands`, `TakeLayers`, `PaneUtils`, `AudioCheckRunner`, `CalibrateAudioDialog`; in development builds only, `main/dev/` (`DevChecks`, `TakeObserver`) | + +Development builds are every build type but `release`: they define `TONY_DEV_CHECKS` and +compile `main/dev/`, and everything that uses it elsewhere is inside `#ifdef +TONY_DEV_CHECKS`, so that a release build compiles with no `main/dev/` file +([calibrate-audio.md](calibrate-audio.md), §6). When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` list **only if** it has `Q_OBJECT`. Logic that can be written as pure functions or a plain @@ -59,6 +65,15 @@ follow. `MainWindow` then only fills the struct in and puts the answer on screen `WritableWaveFileModel` and emits `pitchDetected(frame, hz)`. It never touches the pitch model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued connection). Stop the tracker **before** releasing the model it reads. +- **Calibrate Audio** ([calibrate-audio.md](calibrate-audio.md), §9): `MainWindow` owns + the `AudioCheckRunner` and, in development builds, the `DevChecks` (both made with the + window), and the `CalibrateAudioDialog` (made the first time it is asked for). + `DevChecks` drives the runner and owns a `TakeObserver`. The runner, the dev checks and + the observer are `friend`s of `MainWindow`, as they drive the take path and read the + take's state; the development ones only under `#ifdef TONY_DEV_CHECKS`. `~MainWindow` + deletes the dialog, then the dev checks, then the runner, before anything they read; + `closeSession()` tells the runner, then the dev checks. They are driven by timers and + signals, never a nested event loop: the window can be closed during a run. The reference is the pane's **work model** (`Pane::setWorkModel()`, svgui fork, set in `analyseNewMainModel()`). Without that the pane greys itself out from the end of the @@ -129,6 +144,11 @@ leaves whoever keeps a pointer to it holding a layer that the redo stack owns an and `Analyser::setVisible()` **write QSettings keys that both analysers share**; use them only for the user's own toggles, never for temporary states such as "during a take". For temporary hiding use `showLayer(pane, false)`. +- The toolbar's level controls (`LevelPanToolButton`) answer a gain between their notches + by moving to the nearest notch and emitting it, and the window then sets that gain + through `Analyser::setGain()` and `setAudible()`, writing the shared settings. A play gain + set directly on the reference (as the audio check's is) needs the control moved first, + under a `QSignalBlocker`, before `updateLayerStatuses()` shows it. ### Selection and tools diff --git a/docs/building.md b/docs/building.md index 97f9f79d..f989c982 100644 --- a/docs/building.md +++ b/docs/building.md @@ -68,26 +68,35 @@ echo "exit:$?" >> tmp/build.log ## On Linux (a cloud session) -Not how the project is developed, but it builds and both suites run; this is how it was done -on 2026-09-25 (Ubuntu 24.04, no sound card): +Not how the project is developed, but it builds and the suites run; this is how it was done +on 2026-09-25 and 2026-09-26 (Ubuntu 24.04, no sound card): - Packages: the `apt-get install` list of `.github/workflows/linux.yml` (`smlnj` and `mercurial` are not needed, and `libboost-dev` does for `libboost-all-dev`), plus `librubberband-dev`, `libjack-jackd2-dev`, `libasound2-dev`, `libopusenc-dev`, `meson`. -- **Qt 6.11 from conda-forge, not Ubuntu's 6.4.** Under 6.4 the string-based connects of - `Analyser` with `sv::` types do not resolve ("No such slot - Analyser::layerCompletionChanged(ModelId)"), so pYIN's completion never arrives and every - analysing test times out. download.qt.io's mirrors are blocked by the session's proxy; - conda-forge is not: +- **Qt: Ubuntu's 6.4 does** (the `qt6-*` packages of that list), now that no string-based + connect names `ModelId` or `sv_frame_t`. Qt 6.4 does not match such a string with + moc's `sv::` names, and the connection fails at run time with "No such slot" (for + `Analyser::layerCompletionChanged(ModelId)` pYIN's completion never arrived and every + analysing test timed out); the user's Qt 6.11 matches it, so a string connect can pass + on Windows and fail here. Use member-pointer connects, and grep a test log for "No such + slot". Code built here can use no Qt API newer than 6.4. The app and dev test mains draw + text without sub-pixel anti-aliasing, which Ubuntu's fontconfig asks Qt 6.4 for + ([testing.md](testing.md)). +- To build against the user's Qt version instead: download.qt.io's mirrors are blocked by + the session's proxy; conda-forge is not: `micromamba create -p /opt/qt611 -c conda-forge qt6-main=6.11.1`, then a directory with links to only its `Qt6*.pc` files, so that nothing else of conda's is picked up: `PKG_CONFIG_PATH= meson setup build_qt611`, and - `LD_LIBRARY_PATH=/opt/qt611/lib` to run. + `LD_LIBRARY_PATH=/opt/qt611/lib` to run (2026-09-25). +- Build the targets without `.exe`, and `pyin.so` with them: `ninja -C build_linux tony + test-tony-core test-tony-app test-tony-dev pyin.so`. Without the plugin every app test + that waits for an analysis hangs ([testing.md](testing.md)). - The libraries by `git clone` at the pins of `repoint-lock.json`. sourcehut (the `hg` ones) was unreachable; their GitHub mirrors (`github.com/breakfastquay/...`) are at the same tips. -- `-j 4` on four cores; the whole build takes about 20 minutes. Run the app suite with - nothing else building: it records in real time. +- `-j 4` on four cores; the whole build takes about 20 minutes. Run the app and dev suites + one at a time with nothing else building: they record in real time. - Four tests of `TestTakesFile` fail on Linux and nowhere else: they are about Windows paths (backslashes, drive letters, case). @@ -99,8 +108,15 @@ on 2026-09-25 (Ubuntu 24.04, no sound card): include directories for `opus`, `sord-0`, `serd-0`. - `-DHAVE_MEDIAFOUNDATION` with `-lmfplat -lmfreadwrite -lmfuuid -lpropsys`; needs the `bqaudiostream` fork. -- `tony_core` / `tony_app` static libraries and the two test executables; see +- `tony_core` / `tony_app` static libraries and the test executables; see [architecture.md](architecture.md) for what goes where. A new source file goes into `tony_core_files` or `tony_app_files`, and its header into the matching `*_moc_files` only if it declares `Q_OBJECT`. -- Windows headers define `near` and `far` as macros. Do not use them as identifiers. +- Any build type but `release` (`build.bat`'s is `debugoptimized`) is a development build: + `-DTONY_DEV_CHECKS` for the compiler and for moc, `main/dev/` compiled into `tony_app`, + and `test-tony-dev` built. `meson.build`'s default and the CI workflows use `release`, + which has none of it. After a change to how the dev checks are wired in, set up a + `release` build directory and build it: it must compile with no `main/dev/` file + ([calibrate-audio.md](calibrate-audio.md), §6). +- Windows headers define macros named `near` and `far` (empty), `min`, `max`, `ERROR`, `IN` + and `OUT`. Do not use them as identifiers: a build on Linux does not catch it. diff --git a/docs/calibrate-audio-log.md b/docs/calibrate-audio-log.md deleted file mode 100644 index cb70be20..00000000 --- a/docs/calibrate-audio-log.md +++ /dev/null @@ -1,572 +0,0 @@ -# Calibrate Audio: work orders and log of the finished phases - -The work orders of the phases already built, and their hand-over log, moved out of -[calibrate-audio-work-orders.md](calibrate-audio-work-orders.md) once they were done. -**Phase agents do not read this file**; the work orders' section 3 says what they need of -it. The documentation phase (D) reads the log, and deletes this file with the work orders. - -## Work orders of the finished phases - -### A1 — Test reference and sweep finder (spec §5 "tony_core", §6 core suite) - -- New `main/LatencyCheck.{h,cpp}` in `tony_core`, pure. -- **Generator.** - - A layout gives the length and the events. Make three: *calibration* (about 26 s), - *dev* (adds 3 s held tones) and *long* (4 minutes). - - Each event: a sweep of 1 → 8 kHz, 200 ms, 10 ms raised-cosine edges, −12 dBFS - peak; then a tone at a pitch with a whole number of samples per period at 44.1 kHz - (196, 220.5, 245, 294 Hz; see `docs/testing.md` on pYIN subharmonics). - - Gaps between events are irregular, 1.6–2.6 s, all different by at least 0.1 s. - - Deterministic. It returns the samples and each sweep's exact frame. - - Exponential or linear sweep: choose one and say why. The spec leans neither way. -- **Finder.** Given take audio (samples and rate) and an expected event time, search - ±0.8 s: - - FFT matched filter against the sweep (bqfft; see how `RealtimePitchTracker` uses - it), weighted to the sweep's band; - - the envelope of the result; - - the **earliest** peak within 6 dB of the largest; - - two confidences: the peak over the window's median in dB, and the peak over the - second-best peak outside ±10 ms in dB; - - the error in frames and in ms. - - The thresholds are named constants (15 dB and 6 dB to start). -- **Not in this phase:** aggregation over events and punch-ins, verdicts, second arrivals. -- **Tests,** in a new core test class `TestLatencyCheck`: - - generator: deterministic, events where it says, gaps all different, peak level; - - finder on synthetic takes, made by shifting the reference: - - shifts across ±0.75 s, plus one fractional shift (resample or interpolate); - - white noise at 0 and −10 dB SNR; - - high-pass at 1 kHz and low-pass at 4 kHz; - - polarity inverted; - - a reflection 6 dB **stronger** 7 ms after the direct sound: the direct sound - must be found; - - silence: no confident peak. - - **Show that the reflection test fails** when the finder takes the largest peak. - -### A2 — Verdicts and calibration arithmetic (spec §5 "tony_core") - -- In `LatencyCheck`: given the events inside a take's coverage ranges (one range per - punch-in), and the finder's results, compute: - - median and spread per punch-in and across punch-ins; - - the slope of offset over position; - - the input peak, for clipping; - - whether confidences fall steadily over time, for fading; - - a **second arrival**: a second confident peak at a consistent extra delay across - events, which means monitoring echo. -- `Verdict`: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent, with - the thresholds of spec §5 as named constants. -- The calibration arithmetic as a pure function: new round trip = used round trip + - median offset, in seconds. A take that lands late was spliced from too early a frame. -- **Refined by the lead after A1:** - - **Entry point for B1.** One function takes a layout, the take's samples and rate, - and the punch-ins. Each punch-in is its timeline range in seconds, in the order - recorded. The function returns a summary with the verdict, all flags that applied, - per-punch-in figures, and per-event results. - - **Which events are judged.** An event is *judged* in a punch-in when its sweep and - the finder's window sit inside the range with a margin, a named constant. The splice - cuts content at the range ends, and a sweep cut in half is not a failure of the - path. Count judged and found separately; NoSignal is about found out of judged. - - **Second arrivals.** `findSweep()` computes the second peak but does not return - its position; add it, and its level against the chosen one, to `Arrival`. Monitoring - echo is a second arrival at a consistent extra delay (a few ms of spread) across - most found events, not more than some named dB below the direct sound. - - **Verdict order.** When several apply, pick one and document it, with all flags kept. - Suggested: NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok. - - **PositionDependent against Scattered.** Fit offset over punch-in position. - PositionDependent is a slope above 0.5 % whose fit leaves little residual; large - residuals are Scattered. -- **Tests:** - - one event missing, the rest found; - - fading; - - clipped; - - misplacement that grows with position (the 48000/44100 case) → PositionDependent; - - two punch-ins 20 ms apart → Scattered or Unsteady by threshold; - - monitoring echo detected; - - an event cut by a range end is not judged; - - the arithmetic with both signs. **Show that the sign test fails** when flipped. - - A1 found that a take of 48 kHz frames read as 44.1 kHz finds nothing, because the - sweeps are stretched. Build the rate case as punch-ins displaced by - P·(1 − 44100/48000), not as a stretch. - -### B1 — The alignment check runner (spec §2, §5 "App, every build", §6 app suite) - -Read also: `docs/recording.md` whole (154 lines), `docs/architecture.md` sections on -layers, models and commands (search the headings), `docs/testing.md` "What there is to -reuse". In `MainWindow.cpp`, read by range: `record()`, the deferred lambda in -`recordingStarted()`, `pollTakeProgress()`, `finishSingingTake()` and -`wantedPreRollFrames()`. - -- **Pure helper first**, in `LatencyCheck`, with a core test: - - `punchInsFor(layout, count, eventsEach)` returns `count` consecutive punch-in ranges - in seconds. Each holds `eventsEach` events that `judgeTake()` will judge, using its - margins. - - The calibration uses 4 × 3 on the calibration layout. Tests use a short layout of - 2 × 2 (≈ 8 s) to keep real time down. -- **New `main/AudioCheckRunner.{h,cpp}`** in `tony_app_files`. A QObject owned and wired - by `MainWindow`. It is driven by signals and a polling timer, like - `pollTakeProgress()`, **not by nested event loops**: it runs in every build, and the - user can close the window at any moment. -- **Steps:** - 1. Write the reference WAV to the app data directory, overwriting the old one. Mono, - 44.1 kHz. - 2. `checkSaveModified()`, then `openPath(path, ReplaceSession)`. Wait for the - reference's analysis, as `openReference()` in the tests does. - 3. For each punch-in: select its range and call `record()`; the take stops itself. - Wait for the take's analysis before the next punch-in. It is not strictly needed, - but it keeps pYIN's CPU load out of the next take's timing. - 4. Read the take's audio: the model `analyser2()->getMainModelId()`, mixed to mono, - at its own rate. Call `judgeTake()`. -- **Record into Selection, Play Reference While Recording and a 1 s pre-roll** apply to - the check's own takes through an override in `MainWindow`. `record()`, - `recordingStarted()` and `wantedPreRollFrames()` consult it. **Never through - `setChecked()`**: those actions write QSettings. -- **What each take used.** Keep, per take, the round trip it used: today - `computeRecordingLatency(out, in)`, which B2 will change. Also keep the reported - output and input latency, and the **recording's sample rate**. -- **The result** carries: - - the `TakeSummary`; - - the round trip used, and the reported pair; - - the recording's rate and the reference's; - - a **rate-mismatch flag**, set when the two rates differ, whatever the sweeps say - (A2's finding: a 48 kHz take runs off the finder's reach); - - the calibrated round trip from `calibratedRoundTrip()`, meaningful only on Ok or - Unsteady. - - Emitted as a signal when done. Failures (no device, recording refused, the session - closed mid-run) end the run with a reason. -- **Cancel** stops a take in progress through the normal Stop path and clears the - override. `closeSession()` and `~MainWindow()` cancel a running check. -- **Not in this phase:** storing or using the result (B2), any dialog or menu entry (B3). -- **App tests.** Choose between a new class and adding to `TestRecordWorkflow`; say - which. `TestMainWindow` and the fixtures live in `TestRecordWorkflow.h`. Use - `FakeAudioIO` `loopback = true` and the short layout. - - Wrong reported latencies (e.g. 2·4096 and 4096), `inputDelay` = 3·4096 + 123. The - median offset equals the difference, in seconds at 44.1 kHz, to within a few - frames. The verdict is Ok. The calibrated round trip equals `inputDelay`. - - Fake at 48 kHz: the rate mismatch is flagged and both rates are given. Assert only - what the runner reports, and that nothing crashes. What Tony does with 48 kHz takes - is a separate, known bug. - - After a check, the three toggles and their QSettings values are as before. - - Cancel during a take leaves no recording in progress and the override cleared. - - **Show failure** for the first test with the override's Play Reference half - removed: nothing is heard, so NoSignal. - -### B2 — Measured round trip in use (spec §5 `LatencyCalibration`, "MainWindow") - -Read also: `docs/recording.md` "Latency"; `main/LatencyUtils.h` whole; in -`MainWindow.cpp`, the deferred lambda in `recordingStarted()` (search -`m_takeLatency.roundTrip`) and `refineRecordingLatency()`; `AudioCheckRunner.h` -(`AudioCheckResult`, `calibrationUsable()`). - -- **New `main/LatencyCalibration.{h,cpp}`** in `tony_core`: - - **Key.** The three Preferences values `createAudioIO()` reads (`audio-target`, - then `audio-playback-device` and `audio-record-device`, each suffixed with the - implementation when one is pinned; see `audioDeviceSettingKey()` in - `MainWindow.cpp`) and the recording's rate. Device names can hold `/` and non-ASCII - characters: encode them, so that QSettings does not make subgroups. - - **Stored:** round trip and spread in seconds, the date, and the reported output - and input latency, each **in seconds**. The reported pair is the staleness - fingerprint: a stored figure is stale when either differs by more than a named - tolerance. - - `store`, `load`, `forget`, and a staleness test. QSettings group - `LatencyCalibration`. -- **Use in `recordingStarted()`.** The round trip is the stored one when there is one - for this key and it is not stale; otherwise the reported sum. Either way it is - **converted to frames at the recording's rate**, from seconds. - - Fix the reported sum's units while you are there. `getTargetPlayLatency()` counts - frames at the session's rate (`ResamplerWrapper` converts it), and - `getSystemRecordLatency()` at the device's. Today the two are added as they come; - B1 measured 242 ms instead of 256 at 48 kHz. - - The recording's model (`m_currentRecordingModelId`) exists by the time the lambda - runs, and gives the rate. - - Keep in `m_takeLatency` which source was used (reported or measured), and log it. - - The start gap and everything downstream stay as they are. -- **MainWindow API for B4.** Store a check's result, forget the stored figure, and - describe the figure in use (source, milliseconds, date). -- **Not in this phase:** any dialog, menu entry or playback change. -- **Tests:** - - **Core.** Store, load and forget round trip, with a device name holding `/` and - `ä`; the staleness tolerance on both sides; different keys stay apart. Use a - QSettings scope the tests own and clear. - - **App.** - - `latency_end_to_end`'s recipe with wrong reported latencies and a stored figure - equal to `inputDelay`: the sung step lands on the reference's. **The same test - without the stored figure must fail**; show it. - - A stale stored figure (fingerprint differs): the reported sum is used. - - 48 kHz fake: the reported sum, in recording frames, equals - `playbackLatency + recordLatency`, both device frames, to within a frame or two. - - Check, store, check again (short plan): the second check's median offset is - within a few frames of 0. This is the strongest test in the feature, and it - takes about 25 s. - -### B3 — The check's playback, and progress (spec §2, §8 "Loudness") - -Read also: `docs/architecture.md` on play parameters and on `Analyser::setAudible()` -(search "audible"); in `Analyser.cpp`, where the reference's pan and the -sonification's audibility are set (search `setPlayPan`, `PlayParameters`). - -- For the check's session only, the reference plays **centred** and at **−12 dBFS - peak** after normalisation: the model is normalised to full scale when it is read, so - the gain has to come off at playback. The pitch-track sonification is **silent**. - - Do it through the play parameters of the check session's models and layers, never - `Analyser::setAudible()`, which writes the shared settings (see `AGENTS.md`). - - Make sure nothing puts them back later in the run, for example the analysis - finishing, or the next take. - - Nothing of the user's own sessions changes. -- **A progress signal** on the runner: step, punch-in *k* of *n*, and the seconds left - where known, for B4's dialog. -- **Two runner fixes from B2's report:** - - **Reference file name.** The reference WAV gets a new name each run (a counter or a - timestamp), and old ones are removed when no session holds them. **Check Again** - otherwise rewrites the file that the check session it replaces still has open; on - Windows that write can fail. Linux cannot show it. - - **Reported output latency.** `AudioCheckRunner` divides the reported output latency - by the reference's rate. Have `TakeLatency` carry both reported figures **in - seconds**, as `MainWindow::roundTripAt()` works them out, and have the result use - those. Then the take path and the check can never disagree. -- **Tests:** - - During a check the reference's play parameters are centred at the planned gain, - and the sonification is not audible. - - Afterwards, a newly opened ordinary file plays as before: reference pan and - sonification as the settings say. - - The sweeps reach `FakeAudioIO`'s captured output at about −12 dBFS on the mixed - channel. - - Progress reports each punch-in in order. - -### B4 — Calibrate Audio dialog and menu (spec §2) - -Read also: how an existing Tony dialog is built and tested (search -`confirmRecordingOverTake` and `askForTakeName` in `MainWindow.cpp` and -`TestRecordWorkflow.h`). - -- **`main/CalibrateAudioDialog.{h,cpp}`**, non-modal and thin. Its pages: - 1. **Instructions:** the output and input device names, the latency in use with its - source, "hold one earcup against the mic, off your ears", moderate volume. - 2. **Progress,** from B3's signal, with Cancel. - 3. **Result:** - - the verdict in plain words, with the fix for each failure (spec §2 and §8); - - the measured round trip against the driver's figure; - - the spread; - - both rates, with a plain sentence when they differ; - - the input peak; - - the echo, if one was heard. - - **Use this latency** (only when `calibrationUsable()`), **Check Again** and - **Close**. -- **Menu, Playback:** - - **Calibrate Audio…**, disabled while recording; - - a disabled line saying the latency in use ("Latency: measured 187 ms, 25 Sep" or - "Latency: driver's figure, 400 ms"); - - **Forget Measured Latency**. -- The calibration plan is 4 punch-ins × 3 events on the calibration layout; it fits - since the lead's spacing change. -- **The device key is taken when the check starts.** Carry it in `AudioCheckResult`, - and have `storeMeasuredLatency()` use it, not the Preferences at the moment the - button is pressed. The dialog is non-modal, so the user could change device in - between (B2's report). Also disable both Audio Device menus while a check runs. -- Anything that asks the user goes through a virtual seam, as - `confirmRecordingOverTake()` does. The app tests drive the dialog's slots directly; - the dialog watchdog fails a test on any unexpected modal dialog. -- **Tests:** - - the menu starts a check; - - Use this latency stores (B2's API), and the menu line changes; - - Forget clears it; - - the dialog's result words for NoSignal and for a rate mismatch; - - Calibrate Audio is disabled during an ordinary take. -- **After B4 the user runs it on Windows** (the checkpoint in spec §7). - -### C0 — TakeDiff (spec §5 "tony_core") - -Read also: `docs/takes.md` on the splice's edge fades and on ranged analysis's merge -window (search "fade", "±", "W"); `main/TakeAudio.h` and `main/TakeEvents.h` for the -existing vocabulary. `TakeEvents` may already hold part of what is needed: reuse it and -do not duplicate it. - -- **New `main/TakeDiff.{h,cpp}`** in `tony_core`, pure. The comparisons the dev checks - (C1–C4) use on a real take. Each returns a plain result: pass or fail, and the numbers - behind it, so that a check can report them. The inputs are sample buffers, event - vectors and frame ranges; no models. -- **Audio unchanged outside a range.** Two sample buffers (before and after a - punch-in, as read from the take's file) are **bit-identical** outside - `[start, end)`, allowing for where the splice's fades fall. Find that in - `TakeAudio.cpp`; do not guess. Report the first differing frame. -- **Events unchanged outside a range ± margin.** Two event vectors (pitch, and notes - with durations) are identical outside `[start − margin, end + margin]`. A note that - crosses the boundary counts as inside. Report what was added, removed and changed. -- **Pitch continuous across a join.** Given pitch events and a join frame: - - no gap longer than N hops within ±W of the join; - - no two events at the same frame; - - frames strictly increasing. -- **One note across a join.** Exactly one note spans the join frame, and no note - begins or ends within ±X of it. X is a named constant. -- **No step at a join.** The largest first difference of the samples within ±2 ms of - the join, against the typical first difference over the 50 ms around it, in dB. It - should stay near 0 dB when the splice is clean, and a hard cut shows as a large - excess. The threshold is a named constant; justify it. -- **Tests,** in a new core class `TestTakeDiff`, on synthetic data: - - each comparison passing and failing on purpose: one sample changed just outside - the range; one pitch event dropped at the join; a doubled frame; a note split in - two at the join; a hard cut in a sine. - - **Show failure** for two of them by breaking the code. - -### C1a — Dev-check framework, items 1 and 2 (spec §2 point 5, §3, §4, §5 "development builds only") - -Read also: `main/AudioCheckRunner.{h,cpp}` whole (about 1000 lines together; you extend -it), `main/CalibrateAudioDialog.h`, `main/test/TestAudioCheck.h` for the fixture -(`loopback()`, `shortPlan()`, `makeWindow()`), `docs/takes.md` on where take files go -before and after a save, and spec §4's rows for items 1 and 2. - -- **Build flag** (spec §3). In `meson.build`, a build type not starting with `release` - adds `-DTONY_DEV_CHECKS` to `general_defines`, and only then are `main/dev/*.cpp` - compiled into `tony_app` and their headers moc'd. `build_linux` is `debugoptimized`, so - it has them. Everything under `main/dev/`, and every use of it elsewhere (a `friend` - line, a member, a dialog widget, the test class's registration), is inside - `#ifdef TONY_DEV_CHECKS`. A release build must compile with no `main/dev/` file; the - lead builds one later, so keep the `#ifdef`s tidy. -- **Runner extensions** (every build; small; each tested in `TestAudioCheck`): - - **`Plan` ranges.** Explicit punch-ins in seconds; when given, they replace - `punchInsFor()`. `start()` refuses ranges that overlap, are out of order or lie - outside the layout. - - **`Plan` keeps the session.** Record into the session open now, which the caller - says is a check reference of the plan's layout: no reference written or opened, no - save question; straight on to the reference's analysis wait and the punch-ins. The - take keeps what earlier runs recorded; only this plan's punch-ins are judged. - - **`Plan` round trip, for the run only** (spec §10 "a dev run uses the new figure - for itself only"). Seconds; unset means the window's own. The window uses it for the - check's takes only, where `recordingStarted()` takes `roundTripAt()` now. It never - touches the stored figure or the Playback menu's line. Log it as the check's own. - - **`TakeLatency` gets the start gap** each take used, and whether it was measured or - only estimated (item 2 reports it per punch-in). - - **No save question for the check's own session.** When the session open has never - been saved and its main file is in `referenceDirectory()`, replacing it asks - nothing: it is the check before. This also ends B4's Check Again prompt. - - **Record during a check** (from B4): greyed while a check runs, and `record()` - ignores the user's press then. Pressing it today ends the check's take early. -- **`main/dev/DevChecks.{h,cpp}`**, a `QObject`. - - **No nested event loop.** Spec §5 said `waitUntil()` with a `QEventLoop`; the lead - changed it: a list of stages driven by the runner's `finished()` and a polling timer, - as the runner is driven, for the runner's reason (the window can be closed at any - moment). Each stage starts something and says when it is done; a stage that times - out fails the run. - - `start(Options)`, `cancel()`, `isRunning()`, `sessionClosing()` (as the runner's: - ends the run unless the run itself is replacing the session). Signals `progress` - (stage name, n of m) and `finished(DevReport)`, once however the run ends. - - `Options`: the round trip for the run (seconds), the report directory ("" for - `TONY_TEST_LOG_DIR` if set, else `AppDataLocation`), the scratch directory ("" for - `AppDataLocation`). - - `CheckResult { item, name, verdict (Pass, Fail, Measured, Skipped), numbers (label and - value pairs, as text), message }`; `DevReport { checks, failure, reportPath, - sessionPath }`. A run that ends early marks the checks it did not reach Skipped, - with the reason. - - Owned by `MainWindow` in dev builds, like the runner; a `friend` of it under the - `#ifdef`. `~MainWindow` deletes it after the dialog and before the runner; - `closeSession()` calls its `sessionClosing()`. -- **The stages of C1a** (C1b inserts more before the last): - 1. **Fresh punch-ins.** A runner run on `devLayout()`, opening a new reference (no - save question: see above), with two explicit punch-ins in separate regions of the - calibration part, each holding two sweeps, and the run's round trip. Choose ranges - that leave the held tones (after 25 s) and the start (before 3 s) free for later - stages, and say which you chose. - 2. **Save and reopen.** Save the session into a scratch folder (below) with - `MainWindow`'s own save path, no dialog; reopen it; wait for the analyses; read the - take's file again. -- **Items:** - - **1** Pass when every judged sweep of every punch-in lands within ±2 ms (a named - constant), and after the reopen the take's audio judged again gives the same offsets - and its pitch and notes are the same events as before the save. Numbers: offsets - per punch-in, largest offset, the round trip used. - - **2** Pass when every punch-in was added to the take, each placed within ±2 ms, each - with a measured start gap of its own. Numbers: per punch-in its median offset and - start gap. -- **Scratch folders.** Not deleted at the end: the session open afterwards lives in it - (spec §2: the test session stays open). As `nextReferencePath()` does for references: - numbered folders, and at the start of a run every one that the open session does not - use is removed. The report names the folder. -- **Report.** - - Text file `DevChecks.txt` in the report directory: the run's date, devices and round - trip, then one block per check grouped by checklist item (verdict, message, - numbers), ending `Totals: N passed, N failed, N measured, N skipped`. - - Tests pass a report directory of their own: a failing run's report must not land - among the suites' own files, where the lead greps for `^FAIL`. -- **Dialog** (dev builds only). The instructions page gets a checkbox, "Run the dev - checks after calibrating", on by default and not remembered. With it on and the - calibration usable, the progress page goes straight on into the dev checks with - `calibratedRoundTrip`, Cancel cancels whichever is running, and the result page - shows the calibration as today plus the dev report: one line per check, then the - report file's path. With the calibration unusable, the dev checks do not run and the - page says so. -- **Tests,** new class `TestDevChecks` in the app suite, compiled and registered only - in dev builds; the fixture copied from `TestAudioCheck`, not shared by editing it: - - passing on a loopback fake with its true round trip: items 1 and 2 pass, the report - file ends with `Totals:`, and the session open afterwards is the one in the scratch - folder; - - failing with the round trip 20 ms off: items 1 and 2 fail, and the report shows the - offsets; - - cancelled mid-run, and the session closed mid-run: `finished` once, no take left - recording, the user's three toggles and stored latency untouched; - - the dialog with the checkbox on runs the dev checks after the calibration (the short - plan for the calibration). - - **Show failure** for item 1's tolerance and for the run round trip being ignored. - - Report the real time `TestDevChecks` adds. Spec §6 says more than about a minute - over all the dev phases moves them to a third executable, which the lead will ask - the user about; do not create one. - -## Log - -### Phase A1 — 2026-09-25 -Built: `main/LatencyCheck.{h,cpp}` in `tony_core`: `calibrationLayout()`, `devLayout()`, `longLayout()` (rate defaults to 44100), `sweep(rate)`, `generate(layout)`, `findSweep(samples, count, rate, expectedSeconds)` → `Arrival {found, errorFrames, errorSeconds, peakOverMedianDb, peakOverSecondDb, levelDb, inputPeak}`. Thresholds are `k…` constants in the header. `main/test/TestLatencyCheck.h`: 16 tests, 1.4 s. -Choices / deviations: -- Linear sweep: flat spectrum, narrowest peak; an exponential sweep's harmonics match it 67/106 ms *early*, where the earliest-peak rule looks. -- Envelope = magnitude of the analytic signal (a second inverse FFT gives the Hilbert part). -- Earlier peak counts if it is a local maximum, within 6 dB, and ≥ 1 ms before the largest. No dip rule: one arrival with a hole in its band beats, with deep dips, so only time tells arrivals apart (`finder_takes_one_arrival_as_one`). -- No band mask beyond the matched filter itself: a 0 dBFS 100 Hz hum already comes through 100 dB down; a mask changed nothing measurable. -- Confidences are the chosen (earliest) peak's; "second" is the envelope's maximum more than 10 ms from it. -- A gap is sweep to sweep. Calibration sweeps at 1.0, 3.1, 4.7, 7.2, 9.1, 11.4, 13.1, 15.7, 17.7, 19.5, 21.9, 24.1 s; each tone starts 0.3 s after its sweep, 0.8 s long. Dev adds 3 s held tones after sweeps at 26.9, 30.9, 35.2 s (40 s). Long: 113 events, 240 s. -- Reflection test at 5.5 dB (direct found) and 7 dB (reflection taken): exactly 6 dB passes here only by rounding (6.1 dB flips). -The next phase must know: -- Pass the take's samples at their own rate, and the expected time in seconds (layout frame / layout rate). -- A take of 48 kHz samples read as 44.1 kHz (sweeps stretched 8.8%) finds nothing: level −22 dB, 0.1 dB over the second. `findSweep` at 48000 finds them. A2's "resampled by 48000/44100" case must be built as misplaced frames, or try both rates. -- Measured: noise alone 6–12 dB over the median (threshold 15); SNR 0 / −10 dB: 38 / 28 dB over the median, 26 / 16 dB over the second. In digital silence the median is ~0, so over-the-median reads up to the 200 dB clamp. -- About 20 ms per call. The second peak's position is computed but not returned; second arrivals (A2) need it. -Left open: every threshold untuned; nothing reads `inputPeak` yet. - -### Phase A2 — 2026-09-25 -Built: in `main/LatencyCheck.{h,cpp}`: `Arrival::secondDelaySeconds`, `secondLevelDb`; `judgeTake(layout, take, count, rate, punchIns)` → `TakeSummary {verdict, flags, punchIns[], events[], judged, found, medianOffset, spread, slope, slopeResidual, inputPeak, fadingDb, echo}`; `PunchIn {start, end}` in timeline seconds; `Verdict`, declared in precedence order (NoSignal, Clipped, Fading, PositionDependent, Scattered, Unsteady, Ok); `verdictName()`; `calibratedRoundTrip(used, offset)` = used + offset. 9 tests in `TestLatencyCheck` itself (reusing its helpers); the class now takes 2.9 s. -Choices / deviations: -- Judged: the finder's window, plus a sweep's length past it, inside the range with 50 ms to spare (the splice crossfades 5 ms). An event under a later, overlapping punch-in is judged in that one only. Ranges stop at the take's end. -- Across = median of the punch-ins' medians (each stream start counts once); spread = their max − min. Unsteady/Scattered take the larger of that and any spread within one punch-in. PositionDependent: 3 punch-ins at least (a line through two always fits), |slope| > 0.5 %, and what the least-squares line leaves ≤ 5 ms. NoSignal also when nothing is found, or nothing judged. -- Fading reads `levelDb`, not a confidence: over the median of near silence a confidence runs up to the 200 dB clamp. Median of the first half of the judged events (as recorded) minus that of the second ≥ 10 dB; needs 6 events. -- Echo is judged over events *heard* (≥ 15 dB over the median), not found: an echo within 6 dB leaves nothing found. Added `kEchoMinDelaySeconds` = 20 ms: a reflection 9 ms after the direct sound and 5.5 dB stronger leaves the tail of its peak at 10.1 ms, 9–23 dB down, after every sweep, and was reported as an echo. Also ≤ 30 dB down, within 3 ms of the median delay, in more than half of the heard events and 3 at least. -- Clipped: the largest sample inside the ranges ≥ −0.2 dBFS. -The next phase must know: -- **At 48 kHz, §2's punch-ins at 14 and 20 s land 1.14 and 1.63 s early, beyond the finder's 0.8 s reach.** Measured: the finder takes the neighbouring sweeps, fully confident (+662, +575 ms), and the verdict is Scattered. PositionDependent needs punch-ins that start before about 9.8 s, so §6's "a 48 kHz fake reports PositionDependent" fails with §2's punch-ins. The rates themselves can name the rate. -- §2's punch-ins as [2,7], [8,13], [14,19], [20,25] s judge 7 events (2, 2, 2, 1). A punch-in shorter than 1.95 s judges none. -- An echo under 20 ms (an interface's direct monitor) is not seen. -Left open: every threshold untuned. The verdict thresholds came from the lead's brief; spec §5 has none of them. - -### Phase B1 — 2026-09-26 -Built: `LatencyCheck::punchInsFor()` (+ `kPunchInSlackSeconds`, 10 ms) and core test `punch_ins_hold_the_events_asked_for`. `TakeLatency` in `LatencyUtils.h`. `main/AudioCheckRunner.{h,cpp}` (`tony_app`): `Plan`, `start()`, `cancel()`, `sessionClosing()`, `finished(AudioCheckResult)`. `MainWindow`: `friend class AudioCheckRunner`, the override `m_audioCheckTakes` (read by `record()`, the `recordingStarted()` lambda, `wantedPreRollFrames()`), `m_takeLatency`, the runner made in the constructor, deleted first in `~MainWindow`, told by `closeSession()`. New app class `main/test/TestAudioCheck.h`: 6 tests, 32 s. -Choices / deviations: -- **4 × 3 does not fit the calibration layout** (spec §2 now says why). `punchInsFor()` returns nothing then; 4 × 2 and 3 × 3 fit. B3 needs the lead's choice. -- The take is read from its **file**, not its model: the model is peak-normalised as read (measured: every take Clipped) and resampled to 44.1 kHz. -- `friend` over accessors: the runner needs about nine internals. A new test class, not `TestRecordWorkflow` (5475 lines); its watchdog and init/cleanup are copied. -- Selection: `clearSelections()` + `addSelectionQuietly()`, so the reference is not re-analysed during a take; each punch-in adds one or two "Select" undo steps, as a user's selection does. -- Waits: poll every 50 ms for "nothing being analysed", not "analysed": with auto-analysis off the reference never gets layers (test `check_runs_without_automatic_analysis`). Limits 60 s reference, 30 s a take's analysis, take length + 10 s to stop (then the Stop path); each ends the run with a reason. -- `~MainWindow` deletes the runner: the run ends silently and a take in progress is left to the destructor (the Stop path would splice and start pYIN mid-teardown). `closeSession()` → Stop path + `finished()`. -- Reference: `AppDataLocation/calibrate-audio-reference.wav` unless the plan names a path (tests: their temp dir). Save question first, then write, then open. -- `calibrationUsable()`: Ok or Unsteady, and no rate mismatch. -- No loopback gain was needed (see below). -The next phase must know: -- `Analyser` pans the reference hard left, sonification hard right: only the **left earcup** carries sweeps. Normalised, the reference plays at 0 dBFS, not −12. The fake averages channels: sweeps loop back at half level (peak 0.905 with the synth). -- `getTargetPlayLatency()` counts session frames, `getSystemRecordLatency()` device frames; the result converts each at its own rate. At 48 kHz the round trip used was 242 ms, not 256. -- 48 kHz fake: Scattered, 3 of 3 found, offsets −200 ms median, as A2 foresaw. -- No progress signal yet; B3's dialog may want one (`m_punchIn`). -Left open: no test deletes the window mid-check. Seen while proving the session-close hook: closing a session during an **ordinary** take, then pressing Stop, hangs (pre-existing). - -### Lead — 2026-09-26, after B1 -- Reordered the calibration spacings to `{21,16,25,19,17,23,26,20,24,18,22}` (B1's suggestion) so that 4 × 3 punch-ins fit; `punch_ins_hold_the_events_asked_for` now asks for 4 × 3 and failed on the old order. `judge_only_events_inside_a_punch_in` names its events from the layout instead of 9.1 and 11.4 s. -- Split B3 into B3 (the check's playback and progress) and B4 (dialog and menu), after B1 needed 370k tokens. -- Calibration sweeps now at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, 17.7, 20.1, 21.9, 24.1 s. - -### Phase B2 — 2026-09-26 -Built: `main/LatencyCalibration.{h,cpp}` (`tony_core`, namespace): `Key`, `currentKey(settings, rate)`, `Figure`, `store`/`load`/`forget` (all take a `QSettings &`), `isStale`, `kStaleToleranceSeconds` = 1 ms, `Source`, `InUse {source, roundTrip, date, stale}`, `roundTripInUse()`, `reportedSeconds()`, `toFrames()`. `TakeLatency::measured`. `MainWindow`: `roundTripAt(rate)`, used by the `recordingStarted()` lambda; B4's API `storeMeasuredLatency(result)`, `forgetMeasuredLatency()`, `latencyInUse()`. Core class `TestLatencyCalibration` (6 tests); app tests `latency_measured_round_trip_used`, `latency_stale_round_trip_ignored`, `latency_reported_at_the_device_rate`, `latency_reported_with_device_opened_first` (TestRecordWorkflow), `check_stored_round_trip_is_used` (TestAudioCheck, 25 s). -Choices / deviations: -- Settings: `LatencyCalibration/||//`. In names only `%`, `/`, `\` and `|` are percent-encoded. Registry key names stop at 255 characters, and full encoding of long non-ASCII names could pass that. Values are stored as text (`'g'`, 17 digits) and the date as ISO UTC. -- The key follows `createAudioIO()`, not `audioDeviceSettingKey()`: they differ for `audio-target` = "auto", which Tony never writes. -- **The output latency's unit** (spec §11, corrected): frames at `m_playSource->getDeviceSampleRate()`, or at the recording's rate when that is 0. This is not always the session's rate. If a device is chosen before any file is opened (or a standalone take is the first action), `ResamplerWrapper` has no source rate yet. It passes the device's figure through unconverted and tells the play source 0. Using the session's rate there gave 13012 frames instead of 12288 at 48 kHz (`latency_reported_with_device_opened_first`). -- `storeMeasuredLatency()` stores only when `calibrationUsable()`. The key uses the current Preferences, the rate is the result's, and the fingerprint is the result's reported pair. -- `latencyInUse()`/`forget` use the rate of the last take placed with a round trip. Before any take they use the session's rate, the only rate at which a usable check stores. That rate is reset when a device is chosen from the menu. The device's rate cannot be known before a take: `AudioCallbackRecordTarget` has no getter for it. -- A stale figure is not deleted; it becomes valid again if the driver goes back to reporting the old pair. -The next phase must know: -- With no figure stored at 44.1 kHz, the round trip is exactly the old sum; this is tested in core across a grid of values. At 48 kHz the check now uses 256 ms, not 242. -- The runner's `reportedOutputLatency` still divides by the reference's rate. It matches the take path at 44.1 kHz, the only rate that is stored. -- `storeMeasuredLatency()` reads the device from the Preferences when "Use this latency" is pressed. If B4's non-modal dialog lets the device change in between, the figure is stored under the new device. -Left open: `computeRecordingLatency()` is unused outside `TestLatencyShift`. The svapp fork could add `AudioCallbackRecordTarget::getRecordSampleRate()` so that `latencyInUse()` knows the rate before the first take. - -### Phase B3 — 2026-09-26 -Built: `AudioCheckRunner::setPlayback()`: on the play parameters of the check session's own models, the reference audible, centred, gain 10^(kPeakDbfs/20) (1 if the normalise preference is off); its pitch and notes muted. Applied once the reference is open and again before every punch-in. Public `Step`, `Progress {step, punchIn, punchIns, secondsLeft}`, signal `progress()`. `referenceDirectory()`, `nextReferencePath(dir, inUse)`. `LatencyCalibration::InUse::reportedOutput/Input` (seconds, whichever source won); `TakeLatency::reportedOutput/Input` are now those seconds, and `end()` copies them. App tests `check_plays_the_reference_centred_and_quiet` (13 s), `check_leaves_the_next_session_alone` (4 s), `check_reference_gets_a_file_of_its_own`; core `round_trip_in_use` extended. Four existing tests now compare the reported pair in seconds. -Choices / deviations: -- The toolbar's reference level control (`m_audioLPW`) answers a gain between its notches (−12 dB lies between −11.25 and −20) by emitting the nearest; `audioGainChanged()` then sets it through `Analyser::setGain()`/`setAudible()`, writing `Analyser/audible-0`. `setPlayback()` moves the control first under a `QSignalBlocker`. Without it both new session tests fail on the settings. -- Reference files `calibrate-audio-reference-N.wav`: every such file in the directory but the open session's main model file is removed, then the lowest free N is taken, so names alternate 1, 2. A timestamp per run would add a dead Recent Files entry per run (`RecentFiles` has no remove, and keeps 20). -- The check session keeps its playback after the run. The reference is made audible even where the user's settings mute it. -- `progress()` comes from `poll()` only, never from inside `start()`/`cancel()`: on each step change, and each whole second less while recording. `secondsLeft` counts recording to come (min(1 s, start) + range per take), not analyses. No metatype: direct connections only. -- "Afterwards" is tested after a cancelled run, against the same file opened before the check: the settings alone do not say how a session plays (below). -The next phase must know: -- Pre-existing, not fixed: (a) in the first file of a window the same feedback moves the pitch and notes gain from 0.5 to 0.562 and forces both audible, writing the settings; (b) `audible-0` is overridden at load by `audible-3`: the spectrogram is a layer on the reference's model, so it plays whenever `audible-3` is true. -- B4: add a few seconds per analysis to `secondsLeft` for a rough total. -Left open: the default reference path is exercised only through `nextReferencePath()`; the tests name their own file. - -### Lead — 2026-09-26, after B3 -- De-raced `TestSingingAnalysis::waitForRange()` (`2a306fb`): `initialAnalysisCompleted()` also fires from `layerCompletionChanged()` before a ranged merge; it failed once in a full run. -- For phase D's `open-points.md`, older bugs B3 found: (a) in a window's first file the toolbar level control's notches move the pitch/notes gain 0.5 → 0.562, force both audible and write that to the settings; (b) `audible-0` (Play Audio) is overridden on load by `audible-3` (the spectrogram layer on the same model, loaded last). Also B1's: closing a session during an ordinary take, then Stop, hangs. - -### Phase B4 — 2026-09-26 -Built: `main/CalibrateAudioDialog.{h,cpp}` (`tony_app`), a `QDialog`, not modal, with three pages (instructions, progress, result): `present()`, `startCheck()` (Start and Check Again), `cancelCheck()`, `useLatency()`, `showResult()`, `reject()`; for tests `page()`, `pageText()`, `canUseLatency()`, `setPlan()`; static `describeLatency(InUse)`. `AudioCheckRunner::calibrationPlan()` (4 × 3). `AudioCheckResult::key`: the Preferences' devices taken in `start()`, the rate set in `end()`; `storeMeasuredLatency()` stores under it. `MainWindow`: Playback ▸ Calibrate Audio..., a disabled line "Latency: ...", Forget Measured Latency, after the device submenus; `calibrateAudio()`, `updateLatencyMenuLine()`. `updateMenuStates()` shuts Calibrate Audio during any take or check, and both device menus during a check; the runner's `progress` and `finished` call it. `TestAudioCheck`: 5 tests `calibrate_audio_*`, one full run (13 s); `TestMainWindow` accessors. -Choices / deviations: -- The window owns the dialog, makes it on first use, and deletes it in `~MainWindow` before the runner. The dialog calls only `latencyInUse()` and `storeMeasuredLatency()`, and shows only runs it started itself. -- Closing it (title bar, Esc, Close) while its check runs cancels the check. Opened again, it shows the instructions. -- Menu line: "Latency: measured 281 ms, 26 Sep" (the year only when not this one), "Latency: driver's figure, 279 ms", plus " (the measured one is out of date)" when stale, or "not known yet" while the device reports 0. Forget is enabled while a figure is kept, stale or not. The line is refreshed on the menu's `aboutToShow`, in `updateMenuStates()`, and by store and forget. -- Result page: one sentence for the verdict, then its fix. A rate mismatch replaces the verdict's words, and an echo adds a paragraph. Then a table: the round trip measured (not for NoSignal or a rate mismatch) against the driver's (out + in), what the takes were placed with, each punch-in's offset, the spread, found of judged, both rates, the input peak, the echo, and the devices. The text is selectable, to copy. -- Progress: the runner's seconds left plus `kSecondsPerAnalysis` = 3 s for each analysis to come. The bar never goes back. -- Nothing new asks the user, so there is no new seam: Forget asks nothing. -The next phase must know: -- A run replacing a check session asks "Session modified: save?" (its takes mark it modified). Check Again always meets it; answer No. The runner could skip the question for its own reference's session. -- Not on the result page: §2's mic channel and noise floor. The runner measures neither. -Left open: Record stays enabled during a check. Pressing it there goes through the Stop path and ends the check's take early; what the run then makes of it was not tried. - -### Lead — 2026-09-26, after B4 -- The button is complete; the user's Windows run is the checkpoint (spec §7). C0 onwards goes on meanwhile. -- Left for C1 (small, in passing): Record stays enabled during a check, and pressing it ends the check's take early; grey it while a check runs. -- Left for D: spec §2 promises the mic channel and noise floor on the result page, which nothing measures yet (C2's item 5 measures the channel); §8 says "modal progress dialog", true only of the dev run; Check Again always asks to save the check's own session (answer No), a possible later nicety. - -### Phase C0 — 2026-09-26 -Built: `main/TakeDiff.{h,cpp}` (`tony_core`, namespace). Five comparisons, each returning a struct with `pass` and its numbers: `audioOutside()` → `AudioDiff {firstDifference, differences, largestDifference}`; `eventsOutside()` → `EventDiff {added, removed, changed (before, after), firstDifference, window}`; `pitchAcross()` → `PitchJoin {events, largestGap, largestGapFrom, doubled, firstDoubled, outOfOrder, firstOutOfOrder, window}`; `notesAcross()` → `NoteJoin {spanning, edgesNear, nearestEdge}`; `stepAt()` → `SampleStep {stepDb, largest, typical, largestAt, channel}`. Samples are interleaved (`const float *`, frames, channels); times are named constants in seconds, with the rate passed. Core class `TestTakeDiff`: 19 tests, 10 ms. -Choices / deviations: -- Fades: `weightAt()` in `TakeAudio.cpp` mixes only frames of [start, end), both edge frames included, and copies every other frame; take files are float WAV. So `audioOutside()` excuses nothing outside the range: pass the placed range (the coverage added). A test runs the real `splice()` and `erase()` through files: identical outside, and the range less one frame at either end differs exactly at that frame. Frames past a buffer's end are silence; bits are compared, not values. -- Events: "inside" is what `TakeEvents::eraseNotes()` of the window would touch (a crossing note is inside; a pitch event goes by its frame). Window [start − 0.25 s, end + 0.25 s), half-open. Multiset difference; what is left at one frame is "changed". A note that grew from outside into the window reads as removed. -- Pitch window ±0.5 s (`kPitchWindowSeconds`), not ±0.25: the second run's merge seam is 0.25 s before the join, and would sit on the edge of a ±0.25 s window. Gap limit 1 hop (`kMaxGapHops`). Gaps run to the neighbours beyond the window, or to its edge if there are none, so a hole reaching in, a track stopping inside, or an empty window all fail. -- Notes: X = 0.5 s (`kNoteClearanceSeconds`), closed. A note is [f, f + d), so one ending at the join does not hold it. -- Step: the largest |x[i] − x[i−1]| within ±2 ms, against the 95th percentile over the 50 ms around, in the worst channel. Measured: steady tone 0.03 dB, noise 2.7 dB (6.7 at most in 200 simulated trials), hard cut 36 dB, and the real splice 0.03 dB with its fade, 36 dB without. `kMaxStepDb` = 10 dB; about one random cut in ten reads under it (the two sides nearly meeting). -The next phase must know: -- **Two punch-ins that meet at J leave a 10 ms dip, not a crossfade.** Each fades against what the file held there, which is silence: [J − 5 ms, J) fades out and [J, J + 5 ms) fades in (read from `weightAt()`, not measured). `stepAt()` reads it as no step. -- The notes merge adds a new note only if its onset is in W (`Analyser.cpp`, "Notes go by their onset"). A second run's note that begins before J − 0.25 s is not added, and the dip may split the note at J. Either way, item 10's "one note across the join" may fail on today's code. C3 should measure it, not assume it. -Left open: every threshold untuned; nothing calls `TakeDiff` yet. - -### Lead — 2026-09-26, after C0 -- C1 is split into C1a (framework, items 1 and 2) and C1b (observer, items 7, 12, 13, 14); spec §7 says so. -- DevChecks runs as stages driven by the runner and a timer, not a nested event loop; scratch folders stay with the open session and the next run removes the old ones. Spec §5 and §8 changed to match. -- C0's warning about item 10 (a 10 ms dip at two punch-ins' join; notes merged by onset) stands for C3. - -### Phase C1a — 2026-09-26 -Built: `meson.build`: `dev_checks` = build type not starting with `release`; then `-DTONY_DEV_CHECKS` in `general_defines` and moc, `main/dev/DevChecks.cpp`, `TestDevChecks.h`. Runner: `Plan::ranges` (`punchInsOf()`), `keepSession`, `roundTrip` (window's `m_audioCheckRoundTrip`, read only in the `recordingStarted()` lambda), `abandon()`, public `analysing()` and `readTakeFile()`, no save question for a check's own session. `TakeLatency::startGap/startGapMeasured`. `MainWindow`: Record's action calls `recordPressed()`, which ignores presses while `audioCheckRunning()`; Record greyed then; `m_devChecks` (friend). `main/dev/DevChecks.{h,cpp}`: stages, `CheckResult`, `DevReport`, items 1 `latency_on_this_machine` and 2 `several_phrases_in_one_take`, `DevChecks.txt`, `nextScratchFolder()`. Dialog: checkbox, dev run, `setDevChecks()`, `setDevOptions()`. Tests: 6 in `TestAudioCheck` (~27 s), `TestDevChecks` 7 (~66 s). -Choices / deviations: -- Stage 1 ranges [6.3, 10.2] and [16.8, 21.2] s (sweeps 7.2/9.1, 17.7/20.1; 50 ms spare). Free: before 4.3 s (P = 1 s judging 3.1 s), 10.2–16.8 and 21.2–25 s, the held tones. A re-record starting at 17.9–19.2 s has its lead-in over B's first sweep and still judges its second. -- A stage that fails or times out ends the run: `failure` names it ("Stage 2 of 2, "Save and reopen", did not finish within 60 s."), and every check whose data is missing is Skipped with that text. Checks are worked out at the end: item 2 needs stage 1 only, item 1 both. -- Record goes through `recordPressed()`, not a guard in `record()`: the runner and `pollTakeProgress()` call `record()` during the check's take too, and it cannot tell them from a press. -- After the reopen pitch and notes are restored, not analysed: compared by value with values rounded as `Event::toXml()` writes them; offsets to the frame. Save: `saveSessionToPath()` (a dialog only on failure). -- A run's own round trip counts as `TakeLatency::measured`. The dev reference goes to `referenceDirectory()`, so a cancelled dev run leaves a session the next check replaces without asking. -- `TestAudioCheck`'s fixture deletes each window's `DevChecks`: in a dev build its B4 dialog tests would carry on into them (the checkbox is on) and write `DevChecks.txt` into the log directory. -The next phase must know: -- **B3 bug fixed:** `getLocalFilename()` is the decoded cache copy (spec §11), so `nextReferencePath()` never kept the open reference: always `-1`, written over while open. `mainModelFile()` now. -- On the fake with its true round trip every sweep lands at 0 frames; the fake's reported pair is 2.8 ms short, so a run ignoring its round trip fails item 1. -- `latencyInUse()`'s reported pair differs before any file is open: store test fingerprints after opening one. -- The watchdog answers "Session modified" with No; `QStandardPaths::setTestModeEnabled()` keeps references out of the test app's data directory. -Left open: the app suite now takes about 7m40s; the dev checks add about 66 s, over spec §6's minute. - -### Merge of default — 2026-09-26 -Merged `origin/default` at `92b8b5f` (15 commits); svgui at its new pin `a34646a`. -Conflicts: -- `TestRecordWorkflow.h`: `TestMainWindow` lives in default's `TestMainWindow.h` now. This branch's 17 accessors were merged into it three-way against the base's class (no overlap with default's `setUseRealDevice()` and the rest), and `dev/DevChecks.h` went with them, under `#ifdef TONY_DEV_CHECKS`. -- `TestSingingAnalysis.h`: both `waitForRange()` fixes are the same code; default's comment kept. -- `MainWindow.h` (includes), `meson.build`, `tony-core-test.cpp`, `tony-app-test.cpp`: both sides kept. `TestUiChecks` runs right after `TestRecordWorkflow`, as on default, then `TestAudioCheck` and `TestDevChecks`. -Beyond the conflicts: -- `TestAudioCheck.h` and `TestDevChecks.h` include `TestMainWindow.h` and what they use, not `TestRecordWorkflow.h`. -- `tony-app-test.cpp` draws text without sub-pixel anti-aliasing. Ubuntu's fontconfig asks for it (`10-sub-pixel-rgb.conf`) and Qt 6.4 follows it: the orange fringes of the waveform scale's labels at the pane's left edge were taken for the first live dot, and `TestUiChecks::live_dots_under_the_cursor` failed. Default ran on conda-forge's Qt 6.11, whose text was grey. -- `test-tony-device`'s moc needs no `dev_moc_args`: `TestMainWindow` has no `Q_OBJECT`, and `TestRealDevice.h` no `#ifdef`. -Checked: no string connect naming `ModelId` or `sv_frame_t` is left, and all of `e2cf7c0` survived. `liftPlaySelectionForTake()` (default) is called in the `recordingStarted()` lambda after the round trip, so a check's takes lift the constraint too. `closeSession()` tells the runner and the dev checks first, then restores the constraint. -For D: -- `testing.md` still calls `TestMainWindow` shared by three suites, and lists neither `TestAudioCheck` nor `TestDevChecks`. -- `building.md`'s "Qt 6.11, not Ubuntu's 6.4" is no longer needed for the connects. -- `README.md` says the manual checklist is "none of it tried yet"; `open-points.md` says the device check has been run in the cloud. -Seen, not fixed: every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()" from `svapp/audio/AudioCallbackRecordTarget.cpp:291`. The signal does not exist, so the warning is old and appears on any Qt. -App suite: 549 s, of which `TestUiChecks` takes 82 s. diff --git a/docs/calibrate-audio-work-orders.md b/docs/calibrate-audio-work-orders.md deleted file mode 100644 index fcb80bc9..00000000 --- a/docs/calibrate-audio-work-orders.md +++ /dev/null @@ -1,642 +0,0 @@ -# Calibrate Audio: work orders for phase agents - -You are one of a line of agents, each building **one small phase** of the Calibrate -Audio work. A lead reviews your work when you report back. You have no memory of earlier -phases; what you need is here. This file is working memory for the feature branch, and -the documentation phase removes it. - -**Your context is the budget.** Aim to finish well under 200k tokens. The rules below -say how. They are about not reading huge files whole and not maintaining big documents; -they are **not** a licence to skip what you need to understand. Careful, correct work -comes first. - -## 1. What to read, and what not to - -1. This file, all of it. Not `docs/calibrate-audio-log.md` (the finished phases), unless - you are phase D. -2. `AGENTS.md` at the repository root. Its rules apply, except that the **build and test - commands in section 2 below replace its Windows ones**. -3. `docs/calibrate-audio.md` (the spec, about 400 lines): always §1, §5, §10 and §11, - plus the sections your work order names. Search for the heading and read that range. -4. `docs/testing.md`, "What there is to reuse" and "Design principles", if you write app - tests. `docs/recording.md`, "Latency", if you touch the take path. -5. Code: - - `main/MainWindow.cpp` is over 6000 lines and `main/test/TestRecordWorkflow.h` over - 2000. **Never read them whole**: search for the function, then read that range. - - Before writing a test, read an existing test next to where yours will go and copy - its shape. - -## 2. Rules - -**Scope** - -- Build your phase only. If something from a later phase is needed, build the smallest - part of it and say so. -- The spec is agreed with the user. Where it is silent, choose the simpler option and say - so. Where it is **wrong or impossible**, do not improvise another design: finish what - can be finished, leave the tree building and green, and report. -- **Do not edit** `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` (forks) or any other - top-level library directory (`bqaudioio/`, `pyin/`, `vamp-plugin-sdk/`, …). These are - separate repositories, gitignored here. If one needs a change, report exactly which - change; the lead makes it. -- Match the surrounding code: naming, comment density, idiom. Comments say why, in plain - words. Every new source file starts with the project's GPL header (copy it from - `main/TakeTiming.h`). -- Put pure logic in `tony_core` (listed in `meson.build` as `tony_core_files`), as plain - structs and functions like `TakeTiming`. State gets a class and files of its own; - `MainWindow` only wires it. The layer, model and command rules in `AGENTS.md` apply. -- **Qt here is 6.4.2; the user builds with Qt 6.11 on Windows.** Use no Qt API newer than - 6.4, and nothing specific to Linux. - -**Build and test.** This session builds on Linux in `build_linux/`, not the user's -MinGW setup. - -- Build: - - cd /home/user/tony - ninja -j 4 -C build_linux tony test-tony-core test-tony-app test-tony-dev pyin.so > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log - - Search the log for `error:`; never read it whole. meson reconfigures by itself after - `meson.build` changes. Targets have no `.exe`. -- **`pyin.so` must be built too.** The app suite sets `VAMP_PATH` to its own directory. - Without the plugin, every test that waits for pitch analysis hangs until QtTest's - 5-minute watchdog aborts the run. -- Test, from `build_linux/`: - - mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core > ../tmp/test.log 2>&1; echo "exit:$?" - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app > ../tmp/test-app.log 2>&1; echo "exit:$?" - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-dev > ../tmp/test-dev.log 2>&1; echo "exit:$?" - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-dev some_test_name > ../tmp/test-dev.log 2>&1 - grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - - - Read results from the per-suite files, not stdout. - - A test name on the command line goes to every suite in the executable, so the exit - status of a run with names is meaningless. - - New test classes are registered in `main/test/tony-core-test.cpp`, - `tony-app-test.cpp` or `tony-dev-test.cpp` and added to `meson.build`. - `TestDevChecks` is the only suite of `test-tony-dev`. -- While working, run **only your tests**. Run all three whole suites **once**, at the - end, and again only if something failed. `test-tony-app` takes about 8 minutes: run it - with the Bash tool's `run_in_background: true` and wait for the completion notice, as - a foreground call can hit the 10-minute tool limit. Never run two suites at once, nor a - suite while building: the app tests record in real time. -- App tests run in real time against `FakeAudioIO`: keep them short (seconds, not tens - of seconds). -- Every behaviour gets a test that can fail. Show it for the two or three that matter - most by breaking the code for a moment. Undo the break **by hand**: never - `git checkout` or `git restore` a file to revert an experiment. -- Do not weaken or delete an existing test to get green. If one is wrong because the - behaviour was meant to change, change it and say so. -- **Linux traps:** - - Section 3 gives the suite results to expect. - - Qt 6.4 does not match a `SIGNAL()`/`SLOT()` string naming `ModelId` or `sv_frame_t` - with moc's `sv::` names: the connection fails at run time with "No such slot". The - user's Qt happens to match them, so a string connect can pass there and fail here. - Use member-pointer `connect` only, and grep test output for "No such slot". -- **Windows traps** (the user builds with MinGW; Linux will not catch these): - - Windows headers define `near` and `far` as empty macros, so a variable named `near` - compiles here and breaks the user's build. Avoid those two names, and other - Windows macro names such as `min`, `max`, `ERROR`, `IN`, `OUT`. - -**Docs: almost none.** The documentation phase (D) brings `docs/` up to date. You -write only: - -- one entry in the log (section 5), **25 lines at most**, appended at the end; -- "Done" on your phase's line in spec §7, and a correction of any spec statement your - work proved wrong. - -Edit documents with the Edit/Write tools only. A shell heredoc or one-liner containing -backticks, `$` or non-ASCII text gets mangled, and has corrupted documents before. - -**Git.** The lead commits. Leave your work uncommitted, building and green. Never commit, -push, amend, stash, or `git add -A`. - -**Report** (your final message, and all the lead sees; under 60 lines) - -- What was built, by file, briefly. -- The `Totals` lines of the final full runs of all three suites, copied, not paraphrased. -- Which tests you saw fail without the change. -- Choices made, deviations, anything fragile or unfinished. Say it plainly: a problem - reported is cheap, one found later is not. - -## 3. State of the code - -As of 2026-09-26, after C1a and the merge of `default`. The work orders and log of the -finished phases are in `docs/calibrate-audio-log.md`: **do not read it**; what you need of -it is here. - -**What exists** - -- `tony_core`: `LatencyCheck` (the three layouts, the sweep finder, `judgeTake()` with its - verdicts and second-arrival `echo`, `punchInsFor()`), `LatencyCalibration` (the stored - round trip per device key and rate), `TakeDiff` (audio bit-identical outside a range, - events unchanged outside a range ± 0.25 s, pitch and notes across a join, sample step at - a join). Core tests: `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`. -- `AudioCheckRunner` (every build): a `Plan` (layout; `punchIns` × `eventsEach` or explicit - `ranges`; `keepSession`; a `roundTrip` for the run only), steps driven by a 50 ms timer, - never a nested event loop. Its takes go through `MainWindow::record()` with the - override `m_audioCheckTakes` (Record into Selection, Play Reference While Recording, - a 1 s pre-roll, whatever the toolbar says) and are judged from the take's own file. - `progress()` reports the step (`Recording`, `AnalysingTake`, …) and punch-in. - App tests: `TestAudioCheck` in `test-tony-app` (23 tests, about 2 minutes). -- `CalibrateAudioDialog` (every build): instructions, progress, result; in dev builds a - checkbox that carries on into the dev checks with the calibrated round trip. -- `MainWindow`: `m_takeLatency` (`TakeLatency`: round trip, reported pair, rate, start gap - and whether it was measured) for the last take; `roundTripAt()`; - `m_audioCheckRoundTrip`, used only in `recordingStarted()`'s deferred lambda; Record's - action goes to `recordPressed()`, which ignores presses while `audioCheckRunning()`. -- `main/dev/DevChecks` (dev builds, `TONY_DEV_CHECKS`): stages driven by a timer and the - runner's `finished()`. Stage 1 "Fresh punch-ins": the dev reference (`devLayout()`) opened - as a session of its own, punch-ins [6.3, 10.2] and [16.8, 21.2] s. Stage 2 "Save and - reopen" into a numbered scratch folder. Items 1 (`latency_on_this_machine`) and 2 - (`several_phrases_in_one_take`), ±2 ms per sweep. `CheckResult`, `DevReport`, the report - `DevChecks.txt`. Free for new punch-ins: before 4.3 s, 10.2–16.8 s, 21.2–25 s, and the - held tones from 26.9 s (C2's joins). - Tests: `TestDevChecks`, in its own executable `test-tony-dev` (9 tests, about 66 s). -- **From `default`:** `TestUiChecks` (in `test-tony-app`) automates the checklist's screen, - keyboard and dialog items on the fake device. `test-tony-device` (`TestRealDevice.h`, - run by hand) records a 4-minute click track through the air and checks latency, two - recordings in one take, echo, Stop time, input channel levels and live dots. **The user - decided that the dev run takes these over, and `test-tony-device` is retired (C3).** - `TestMainWindow` lives in `main/test/TestMainWindow.h`, with this work's accessors - (`audioCheck()`, `devChecks()`, `takeLatency()`, the menus and actions). - -**Facts found along the way** - -- A model's `getLocalFilename()` is the decoded copy in the temporary directory ("normalise - audio" is on); the file opened is `AudioCheckRunner::mainModelFile()`. -- On the loopback fake with its true round trip every sweep lands at 0 frames; its - reported pair is 2.8 ms short of the true round trip. -- After a reopen, pitch and notes are restored from the session, not analysed. -- `getOutputLevels()` / `getInputLevels()` give the peak since the previous call, and reset - it: one reader only, or each takes the other's peaks. -- `TestAudioCheck`'s fixture deletes each window's `DevChecks`, so that its dialog tests - do not carry on into them. Test references go to Qt's test location - (`QStandardPaths::setTestModeEnabled`); reports and scratch folders to the test's own - directory. -- Every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()" (from - svapp, old, harmless here); ignore it. - -**The user's first run on Windows** (MME, wired mic and headphones, one earcup to the mic): -round trip 301 and 295 ms, verdict Unsteady both times (the spread across or within -punch-ins between 5 and 15 ms); with the mic between both cups, Scattered (over 15 ms). -The detailed figures are awaited. So the ±2 ms of items 1 and 2 will fail on the user's -machine as things stand: thresholds get tuned from real report files, not now. Whether to -keep the stream running between takes (an svapp change) waits on those figures. - -**Linux baseline** (Qt 6.4.2): core all green but 4 `TestTakesFile` tests -(`takes_folder`, `relative_audio_path`, `resolve_audio_path`, `in_folder`: Windows paths); -your final runs must show exactly these 4. `test-tony-app` green in about 8 minutes -(`TestRecordWorkflow` 98, `TestUiChecks` 19, `TestAudioCheck` 23, …); `test-tony-dev` -green. `tony-app-test.cpp` draws text without sub-pixel anti-aliasing, or Qt 6.4's -coloured fringes on the scale's labels read as live dots in `TestUiChecks`. - -## 4. Phases - -Done: A1 (`944df7c`), A2 (`a03b7ec`), B1 (`58de074`), B2 (`47944f2`), B3 (`8524d5f`), B4 (`9b1fb6c`), C0 (`1ef2494`), C1b (`276036e`), C1c (`b1b8f08`), C2 (`fbdce6c`), C2b (`4fb73ac`), C2c (`5f7b2b8`), C3 (`e03f9d2`). - -Also done: C1a (`4370131`), the merge of `default` (`c8b9585`), `test-tony-dev` (lead). - -**Order from here:** C1b, C1c, C2, C2b, C2c, C3, then the lead's release build, then D. The spec's -old C2 (observer group) and C4 (smoke group) are gone: `TestUiChecks` covers the smoke -items and the screen, and what the dev run measures on the device is now in C1b–C2. - -**Budget.** C1a took over 500k tokens against a target of 200k. Read what you need, but -build only your phase, keep tests few and meaningful, and stop to report rather than -redesign. - -### C1b — Take observer; items 3, 4, 5 (spec §4 rows 3, 4, 5, 8) - -Read also: `main/dev/DevChecks.{h,cpp}` whole; `main/test/TestDevChecks.h` for the -fixture; in `main/test/TestRealDevice.h` the tests `record_the_reference_through_the_air`, -`nothing_of_the_take_comes_back_out` and `live_dots_were_drawn` (search `Checklist:`), -which these checks take over; `docs/recording.md` on the live tracker and the cursor -during a take; `main/test/FakeAudioIO.h`. - -- **`main/dev/TakeObserver.{h,cpp}`** (dev builds). DevChecks starts one when the - runner's `progress()` reports `Recording` for a punch-in, and stops it when that - punch-in's analysis is done. Every 20 ms it records, with the time: the playback frame - (the `ViewManager`'s), the output levels, the input levels, the frames received, the - status text, whether a modal widget is up; and each live dot as it first appears (the - realtime model, `m_realtimePitchModelId`, looked up each poll), with its frame, its - value and the playback frame at that moment; and when the take stopped recording. -- **Levels, a checked fact.** `getOutputLevels()` / `getInputLevels()` return the peak - since the previous call and reset it. `ViewManager::checkPlayStatus()` (svgui) already - reads them every 20 ms: **while recording it reads the input levels only**, and emits - `monitoringLevelsChanged(left, right)` when they change; while only playing, the - output levels. So during a take the observer takes the input levels from that signal, - and is the only reader of the output levels. Find out whether the lead-in counts as - recording there, and say. -- **Items**, from the punch-ins of stage 1 (they then hold for every later punch-in too): - - **3 Live dots** (and row 8's number). Pass when each punch-in drew more than 10 dots - (as `test-tony-device`), and the dots lie on the reference's tones: frames within the - tone's span ± 1 hop, value within 50 cents of the tone. Numbers: dots per punch-in, - and the dot-to-cursor offset (dot frame against the playback frame when it - appeared), median and spread in ms: the "Cursor versus dots" risk of spec §8. - - **4 Nothing of the take in the speakers.** Fail on a second arrival - (`TakeSummary::echo`, as the calibration already detects it), or on any output level - above zero in a poll that lies wholly in one of the reference's silent gaps, with a - margin you work out from the output latency and the poll interval; also Play Singing - Audio's state the same before and after the take. Numbers: echo delay and level, - the largest output level in the gaps. - - **5 Mic channel.** Per-channel level of each punch-in's raw recording (the newest - `recorded-*.wav`, as `newestRecording()` finds it in `TestRealDevice.h`, or better the - path the window itself knows, if it does). Measured: which input carries the mic, and - its level. When it is input 2, item 3's dots are the check (Pass/Fail); otherwise - "not applicable here". -- **The report header** gains what `test-tony-device`'s `device()` test logs: the audio - drivers built in, the reported playback and record latencies. -- **`FakeAudioIO`**: a second loopback tap (delay and gain), new fields that change no - existing meaning, for item 4's failing test. The input on one channel exists already - (`inputChannel`). -- **Tests** (`TestDevChecks`): passing on the loopback fake; item 4 failing with the echo - tap; item 5 with the input on channel 2 (dots still drawn, "input 2"). Show failure for - the dots-on-tones check and the silent-gap check by breaking them. - -### C1c — Re-record and pre-roll stages; items 7, 12, 13, 14 (spec §4 rows 7, 12, 13, 14) - -Read also: `main/dev/DevChecks.{h,cpp}` and `main/dev/TakeObserver.h`; `main/TakeDiff.h`; -`main/TakeTiming.h` (`shouldStopAt()`, `countdownText`); in `main/MainWindow.cpp` -`wantedPreRollFrames()`, `pollTakeProgress()` and the start of `record()`; -`docs/recording.md` on pre-roll and Record into Selection; spec §4 rows 7, 12, 13, 14. - -- **Snapshots.** Before and after each punch-in of the new stages: the take's samples from - its file (`AudioCheckRunner::readTakeFile()`), its pitch and notes (by value, looked up - afresh), and its coverage. Compared with `TakeDiff`. -- **Stage "Re-record"**, after "Fresh punch-ins": one runner run keeping the session, one - range starting inside stage 1's second punch-in ([16.8, 21.2] s) between 17.9 and 19.2 s - and ending by 21.2 s. Its 1 s lead-in then plays over the earlier punch-in's recorded - sweep at 17.7 s, and it still judges the sweep at 20.1 s (C1a's note). Items: - - **7** Placement as in items 1 and 2; outside the placed range the take's audio is - bit-identical (`audioOutside()`), and its pitch and notes are unchanged beyond - ± 0.25 s (`eventsOutside()`). - - **12** The lead-in: nothing before P changed (the same comparisons, their part before - P, reported apart), and C1b's output-in-the-gaps check over the lead-in's looks, which - now lie over take audio. **This is the case C1b could not show failing**: show it here. - - **14** The take stopped by itself: the frames recorded past what `shouldStopAt()` - needed, in seconds, within 0.25 s plus one poll of the take timer (100 ms); the - coverage added is exactly the selection; no modal widget seen during the take. The - check's takes always record into a selection, and `record()` asks the overwrite - question only when not (`end < 0`), so "no question" is watched, not arranged. -- **Stage "Pre-roll near the start"**: one range from P = 1 s judging the sweep at 3.1 s, - with a pre-roll of 3 s for this plan only: a new `Plan` field, read where - `wantedPreRollFrames()` reads `kPreRollSeconds` today. Item **13**: playback ran from - frame 0 and never before it (the observer's cursor), the countdown began at 1 and not - 3 (the observer's status text), placement right. -- Items 1 and 2 then cover every punch-in of the run; say how their numbers read now. -- **Tests** (`TestDevChecks`): the passing run gains the two stages (keep the whole run - under about 25 s on the fake). Failing cases, sharing runs where they can: - - the take audible during the re-record's lead-in (set its play parameters audible when - the runner reports `Recording` for that punch-in, from the test): items 4 and 12 fail; - - show failure by breaking the code for item 7 (e.g. the splice's fade beyond its range) - and item 13 (e.g. the pre-roll not clamped at 0), and undo by hand. - -### C2 — Long song and joins; items 9, 10 (spec §4 rows 9, 10) - -Read also: `main/dev/DevChecks.{h,cpp}` (its stages, `runs()`, `Snapshot`); `main/TakeDiff.h` -(`stepAt()`, `pitchAcross()`, `notesAcross()`, `eventsOutside()`); `main/LatencyCheck.{h,cpp}` -for `longLayout()` and the dev layout's held tones; `docs/takes.md` on ranged analysis and -the merge window W; in `main/test/TestRealDevice.h`, `stop_is_quicker_than_a_whole_song` -and how it times the whole-song analysis; spec §4 rows 9 and 10. - -- **Stage "Long song", the first of the run**, before "Fresh punch-ins": the long - reference as a session of its own (a new reference, not `keepSession`), timed from the - session opening to the reference analysed; then two punch-ins far apart in one runner - run (as `test-tony-device`, near 60 s and 150 s), each timed from its Stop to its pitch - merged (the runner's `AnalysingTake` step). First, so that the dev reference then - replaces it as a check's own unsaved session, and the run still ends on the saved dev - session; no saved session is ever replaced. - - **9** Pass when each punch-in's Stop-to-merged time is under half the whole-song - analysis time, and the take's pitch outside the range ± 0.25 s is unchanged - (`eventsOutside()`), which shows the ranged path ran. Numbers: both times. - - These punch-ins join `runs()` for items 1, 2 and 4: placement far into a song is where - a rate mismatch shows. - - **Test time**: a 4-minute reference in every passing test is too slow. Give - `longLayout()` a length parameter (default 240 s, core test for it) and `Options` the - long reference's length; tests use about 60 s with punch-ins to match. Report what - that makes the whole-song and ranged times on the fake. -- **Stage "Joins"**, after "Pre-roll near the start", keeping the session: two punch-ins - in one runner run that meet at J in the middle of one of the dev layout's held tones - (3 s tones after the sweeps at 26.9, 30.9 and 35.2 s). The first holds that tone's - sweep; the second runs on past the next event's sweep so that both are judged, and its - lead-in plays over the first's recording. Snapshots before and after. - - **10** At J: `stepAt()` on the take's file, `pitchAcross()` and `notesAcross()` on the - take's pitch and notes; and `eventsOutside()` over the two ranges together. Numbers: - the step in dB, the largest pitch gap, the notes at J and the nearest note edge. - - C0 found (from the code) that the join is a 10 ms dip, and that the notes merge by - onset may drop or split the note. **Measure and report the result as it is.** If it - fails on today's code, do not fix `Analyser` or the splice: the app test expects that - failure with `QEXPECT_FAIL` and the reason, as `default` does for its defects, and the - report says so. - - On the user's MME, two punch-ins carry different restart offsets (spec §8), so the - join may fail there for that reason too: the message should say which part failed. -- **Tests**: the passing run gains both stages; keep `test-tony-dev` under about 3 minutes - in all (135 s now). Show failure for item 9 (e.g. Stop analysing the whole song) and one - part of item 10 by breaking the code, and undo by hand. - -### C2b — Item 12 robust against a stalled event loop (a de-race, its own commit) - -Read also: `main/dev/DevChecks.cpp` (`gapLooks()`, the "Re-record" stage, item 12) and -`main/dev/TakeObserver.{h,cpp}`; C1b's and C1c's log entries. - -- **The flake (found in C2):** item 12 judges the output levels in the silent gaps of the - re-record's lead-in. Its only gap there, 18.8–19.2 s, holds 15–16 looks; a stall of the - GUI thread of about 340 ms (seen on this VM, nothing in the log) left 9, once 0, and - item 12 failed in a clean run. A real device can stall too, and MME's larger blocks - widen the margin and shrink the gap further. -- **The fix, in the design, not the tolerance:** - - Give the re-record stage's plan a longer pre-roll (`Plan::preRoll`), so that its - lead-in also spans the 0.9 s gap at 16.8–17.7 s inside the earlier punch-in - [16.8, 21.2]: a pre-roll of 2.4 s starts it at 16.8 s. Check that items 7, 12 and 14 - still read as before, and say what the lead-in now covers. - - Look at what a stall does to a look (the frames received jump; the look's window then - spans a sweep and is left out). Where a gap check ends with no look in any gap, the - part is "not judged" with the reason, not a Fail: it has not seen the take played - out. Item 4 the same. -- **Tests:** a test that stalls the GUI thread for about 0.4 s during the re-record's - lead-in (a single-shot timer that busy-waits, started when the runner reports that - punch-in recording): item 12 still judges and passes; and the take-heard fault still - fails with such a stall. Show the stall test failing on the code before the fix. - -### C2c — The notes merge keeps one note across a join (`Analyser`, its own commit) - -Read also: `docs/takes.md` "Ranged analysis and merge" and its known limits (search "by -onset", "deliberate trade"); `Analyser::analyseRange()`'s merge in `main/Analyser.cpp` -(search "Notes go by their onset"); the existing merge tests (search `TestRecordWorkflow.h` -and `TestSingingAnalysis.h` for "note"); C2's log entry. - -- **The defect (found by item 10):** two punch-ins meeting at J inside a held tone. The - first's note ends at J (the audio after J was silence when it was analysed). The second - punch-in's run starts 0.5 s before J, inside the tone, so its note begins before W (the - range ± 0.25 s): the merge adds only notes with their onset in W, and the tone after J - has no note. Analysing the whole take gives one note, 27.21–30.20 s. -- **Wanted:** one note across the join. When an old note runs into W from before it and - a new note that begins before W overlaps it there, the old note keeps its onset (the - audio before W has not changed) and takes the new note's end, cut back as today at an - old onset after W. Decide, and justify, what happens when a new note begins before W - with no old note there. Keep everything else the merge promises: notes in unchanged - audio not split, the change record (`m_rangedNotesChange`) that undo reverses, and the - "deliberate trade" of `docs/takes.md` unless the fix removes it (then say so). -- **Tests:** - - an app test on the fake: two punch-ins meeting inside a held tone leave one note - across J; seen failing before the fix; - - undo of the second punch-in gives back the first's note exactly; - - every existing merge test green and unchanged; - - `TestDevChecks`: item 10's `QEXPECT_FAIL` removed; the check passes. -- **Docs:** `docs/takes.md`'s "Notes, by onset" and its known limit, in the same commit. - -### C3 — Retire `test-tony-device` - -Read also: `main/test/TestRealDevice.h` whole (607 lines: what is being retired); -`docs/manual-checklist.md` whole; `docs/testing.md`'s table and the passages naming -`test-tony-device`; `main/dev/DevChecks.h` (the items and the report); the runner's -`Recording` step in `AudioCheckRunner.cpp`. - -- **Check coverage first**, test by test of `TestRealDevice.h`, and write the mapping into - your log entry: `device()` → the report header; `takes_line_up_with_the_reference` → - items 1 and 2; `nothing_of_the_take_comes_back_out` → item 4; - `stop_is_quicker_than_a_whole_song` → item 9; `live_dots_were_drawn` → items 3 and 5; - `no_input_does_no_harm` → see below. Anything it checks that the dev run does not: - report it, and port it if small. -- **A device that opens but delivers nothing** (its `no_input_does_no_harm` and the - QFAIL "the device opened, but … delivered no input at all"): the runner today waits - out its `Recording` step and ends with "A take did not stop at the end of its range", - which misleads. Make the runner end a take that has received no frames at all by the - time the take should be over, with a message that says the device delivered no input, - through the Stop path, leaving no take and no harm; and an app test in - `TestAudioCheck` with a fake device that opens and never calls back (see how - `FakeAudioIO` and `TestMainWindow` make one; add a field if needed). A device that - cannot be opened at all is covered already (`check_fails_without_a_device`). -- **Remove** `main/test/TestRealDevice.h`, `main/test/tony-device-check.cpp` and the - `test-tony-device` target in `meson.build`, and `TestMainWindow`'s - `setUseRealDevice()` and what serves only it, if nothing else uses them. -- **Docs, in the same change** (AGENTS.md's "Keeping the docs true"): - - `AGENTS.md` and `docs/building.md`: the build command without `test-tony-device.exe`; - - `docs/testing.md`: its table row and paragraph; - - `docs/manual-checklist.md` section 1 becomes "The device check: Calibrate Audio with - the dev checks": in a development build, Playback ▸ Calibrate Audio… with the - checkbox on; the earcup against the microphone; the report file `DevChecks.txt` in - Tony's application data folder; what each item's numbers mean in a line each; what - fails on MME today (items 1, 2 and 10's offsets, spec §8) and why. Keep the list of - what it covers and the dated notes that still hold; drop what was only about the - executable (`TONY_DEVICE_CHECK_FAKE`: `TestDevChecks` checks the check now). - - `main/dev/DevChecks.h`'s comments that name `test-tony-device`. - - Leave `docs/calibrate-audio*.md` beyond your log entry and spec §7's "Done": phase D - brings them up to date. -- **Tests:** the new `TestAudioCheck` test, seen failing before the runner change. The - three suites green; `test-tony-device` no longer builds or exists. - -### Lead — release build - -A build directory of type `release`: it must compile and link with no `main/dev/` file -(spec §8). The lead does this, not an agent. - -### D — Documentation pass - -- Read the log in `docs/calibrate-audio-log.md` as well as this one: you are the one - phase that does. -- Bring `docs/` up to date from the code, the logs and the spec: - - `recording.md`: the latency section; - - `testing.md`: a Dev checks section, `TestAudioCheck`, the loopback fake, and who uses - `TestMainWindow`; - - `manual-checklist.md`: what the dev run settles, and what is left by hand; - - `architecture.md`, if new classes change who owns what; - - `building.md`: whether "Qt 6.11, not Ubuntu's 6.4" still holds now that the connects - are member-pointer ones; - - `README.md` and `open-points.md`: agree on what of the checklist has been tried; remove - the Calibrate Audio "Not built" item and add what is left open, the svapp - `aboutToBeDeleted()` warning included. -- Make `docs/calibrate-audio.md` describe what was built, with a "Known limitations and - open points" section. It is a plan today, with "Found in …" and "Since …" notes - layered on (§2's note that 4 × 3 does not fit is out of date; §4's table uses the - checklist's old numbering; §5's `waitUntil()` and modal dialog are gone). Rewrite it as - the design as built: the button, calibration, the dev run's stages and items, the - architecture, the tests, and the decisions table; keep the facts of §11 that still - hold, and the user's runs as dated notes. Say plainly what each verdict and each dev - check means, and which numbers the user should send back. -- **`AGENTS.md`**: a row in its "Read … before …" table for `docs/calibrate-audio.md` - (touching the audio check, the dev checks, or latency calibration), if you judge it - earns one; nothing else there unless it is false. -- **Open points to carry** into `open-points.md` (and the spec's limitations), each - checked against the code first: - - the next project, a lower-latency driver (spec §7): the rate fix first, the - `jhhr/bqaudioio` fork (created, attached, not yet pinned), a driver type in Tony, MME - the default until a run shows better; - - MME's restart jitter and what it fails today (spec §8, `manual-checklist.md` §1); - - items 4 and 12 read Pass when their gap part was "not judged" (C2b): the Totals line - then overstates; the user has not decided whether it should count otherwise; - - `AudioCheckRunner::kReferenceTimeoutMs` (60 s) against the long song's analysis - (10.4 s here): a PC six times slower ends the dev run at its first stage; - - the countdown reads 1, 2, 1 at P = 1 s with a 3 s pre-roll (C1c); - - live dots trail the cursor by about the round trip (C1b), spec §8 "Cursor versus - dots"; - - items 3 and 5 judge the fresh punch-ins only; item 14 cannot see an overwrite - question (asked inside `record()`, before the observer starts); - - not covered by the dev run (C3's list); - - the svapp `aboutToBeDeleted()` warning (merge log). -- **`forks.md`**: whether `jhhr/bqaudioio` belongs in its table yet (it is not used by - the build) or only in open points; say which you chose. -- Do not rewrite what `default` wrote in the shared docs beyond what this work made false; - C3 already rewrote `manual-checklist.md` §1: review it, keep it. -- Delete this work-orders file and the log file, and remove every link to them. -- No code, no builds, no test runs (the lead is building a release configuration at the - same time). Suspected bugs go in the report. - -## 5. Log (newest last; 25 lines at most per entry) - -Template: - - ### Phase — - Built: ... - Choices / deviations: ... - The next phase must know: ... - Left open: ... - -### Phase C1b — 2026-09-26 -Built: `main/dev/TakeObserver` (a sample every 20 ms: cursor frame, frames received just -before and after the output-level read, output levels while recording only, input levels -from `monitoringLevelsChanged`, status text, modal; each dot on first sight with the cursor -then; the raw recording's path; when it stopped; Play Singing Audio before and after). -`DevChecks` starts one on the runner's `Recording` progress, keeps it (`Watched`) when the -next punch-in records or the runner finishes; items 3 `live_dots`, 4 -`nothing_of_the_take_in_the_speakers`, 5 `mic_on_input_2` from stage 1; report header -with drivers and reported latencies. `FakeAudioIO`: `echoDelay`/`echoGain`, opt-in -`reportLevels`. `TestDevChecks`: 5 checks; `dev_checks_echo_and_the_mic_on_input_2`. -Choices / deviations: output is placed from frames received and the measured start gap -(input, then output, per callback), not from time or the output latency, which the levels -precede. Margin one block either side; the block bounded by the most frames received -between two looks not held up (35 ms on the fake). Item 3: dots from a sound's start -−1 hop to its end + half the tracker window +1 hop (±1 hop failed the clean fake: dots run -440 frames past a tone); sweep ends make 2–3 dots at 860–980 Hz, counted apart, pitch not -judged. Cursor = the ViewManager's frame (S + recorded), read only while recording: dots -trail it by the round trip + about 40 ms (+322 ms at 281 ms on the fake). Item 5 "not -applicable" is Measured; on input 2 alone it passes on more than 10 dots per punch-in. -Echo and input 2 share one run. -The next phase must know: an observer runs for every punch-in of a runner run DevChecks -starts (`m_watched`, cleared per stage); stage 1's kept in `m_freshWatched`. -Left open: at −20 ms item 3 passes or fails with the hop phase (the test allows both). -The gap check was seen failing on a non-silent reference, not a take played back out: -stage 1 has no take audio where it plays (C1c's re-record has). Dot spread 40–230 ms. - -### Phase C1c — 2026-09-26 -Built: `AudioCheckRunner::Plan::preRoll` (default `kPreRollSeconds`, negative refused), -handed to `MainWindow::m_audioCheckPreRoll`, which `wantedPreRollFrames()` reads. -`DevChecks` stages 2 "Re-record" [19.2, 21.2] and 3 "Pre-roll near the start" [1.0, 4.2] -with 3 s (both `keepSession`), a `Snapshot` (take file, pitch, notes, coverage) before and -after each; items 7 `record_from_a_position`, 12 `nothing_heard_or_changed_in_the_lead_in`, -13 `pre_roll_near_the_start`, 14 `record_into_selection_stops_by_itself`. Items 1, 2, 4 now -cover every punch-in (`runs()`, numbered 1–4 along the run); item 1's reopen compares with -the file judged just before the save over all ranges. C1b's gap logic is `gapLooks(until)`. -Choices: P = 19.2 s: shortest range judging 20.1 s, a gap (18.8–19.2) in the lead-in -(16 looks), a whole note before P − 0.25. Item 7 excuses only the selection; item 12 is the -same before P, plus the looks that end by P. Item 14: raw recording frames minus (round -trip + start gap + R + E − P), within [0, 0.25 s + take timer interval + one block], not via -`TakeTiming`, whose margin it checks. Item 13: highest countdown shown ≤ ceil(min(3 s, P) -+ round trip + start gap + 50 ms), ending at 1. The fault test cancels at stage 3. -Found: on a noiseless loopback the take equals the reference, so one played out shows in -no gap: `loopbackInARoom()` adds −60 dBFS noise (pass and fault runs). The countdown reads -1, 2, 1: `record()` shows it before the round trip is known (harmless, reported). -The next phase must know: a new recording stage joins `runs()` for items 1, 2 and 4; a -cancelled run still works out the checks of the stages it finished; the runner repeats -`progress(Recording)` as the seconds left tick down. -Left open: an overwrite question would come inside `record()`, before the observer starts, -so item 14 cannot see it. Items 3 and 5 judge stage 1 only. `test-tony-dev` about 135 s. - -### Phase C2 — 2026-09-26 -Built: `LatencyCheck::longLayout(rate, seconds)`, default `kLongSeconds` (240), core test -`long_layout_of_another_length`. `DevChecks` stage 1 "Long song" (`Options::longSeconds`, -0 leaves it out and item 9 Skipped; `longPunchIns()`: the shortest range judging the first -sweep from 1/4 and from 5/8 of the song, 61.04–62.96 and 150.84–152.76 s at 240 s) and -stage 5 "Joins" (`joinPunchIns()`: [26.0, 28.7] and [28.7, 32.0], J = 28.7 s, the middle of -the tone 27.2–30.2 s; they judge 26.9 and 30.9 s). Items 9 `stop_on_a_long_song`, 10 -`the_joins`. `Run` carries its layout (`gapLooks()` takes it); items 1, 2, 4 count the long -song's punch-ins first (1–8 now); the reopen's `punchInsSoFar()` keeps to the dev take. -Choices: timed by `longSongStep()` from the runner's reports: whole song = its -AnalysingReference step (writing and opening before it, 0.14 s at 60 s, not counted); a -punch-in = its AnalysingTake step, ending at the next Recording report (after `record()`) -or at `finished()` (after the judging). Item 9's pitch before a punch-in is read at its first -Recording report. Item 10's messages name the part: "step:", "pitch:", "note:", "outside:". -On the fake: 60 s song 2.9–3.0 s, punch-ins 0.59–0.70 s (20–24 %); 240 s: 10.4 s, 0.63 and -0.75 s (6–7 %). At J the step reads −8.3 dB (the two 5 ms fades, a dip), pitch gap 1 hop. -Found: item 10 fails, the note only: the first punch-in's note ends 5.7 ms past J, the tone -after J has none (the ranged run starts 0.5 s before J, so its note begins before W and is -dropped). Analysing the whole take gives one note, 27.21–30.20 s. `QEXPECT_FAIL` in the test. -Seen failing: item 9 (Stop analysing the whole take: 40 % and 72 %, pitch changed), item -10's step (no fade-in: 29.4 dB). Tests: the passing run and the cancel, close and dialog tests -have a 60 s long song; the fault runs and the deletion test leave it out. -Left open: item 12 (C1c) once found 0 looks in its 0.4 s lead-in gap (15–16 usual, 9 once): -a silent GUI stall of 340 ms or more there, likely this VM's disk; a real run could show it. -The runner's 60 s limit on the reference's analysis is 6× 240 s's 10 s. `test-tony-dev` 180 s. - -### Phase C2b — 2026-09-26 -Built: `DevChecks::kReRecordPreRollSeconds` (2.4 s) for stage 3: its lead-in runs from 16.8 s, -where stage 2's second punch-in begins, over two silent gaps (16.8–17.7 and 18.8–19.2 s), the -sweep at 17.7 s and the tone from 18 s; item 12's looks there went from 15–16 to 57. -`GapLooks::longestWait` (the longest wait between two looks begun before `until`), a number -of item 12. Items 4 and 12: no look in any gap makes that part "not judged", with the reason -(longest wait, margin) in the message, not a Fail. `TestDevChecks`: -`dev_checks_lead_in_through_a_stall`, two rows: 0.45 s over the gap before P (judged, Pass) -and over the whole lead-in (not judged, Pass); `dev_checks_take_heard_during_the_lead_in` -gains the 0.45 s stall and still fails items 4 and 12 on the take heard; `describe(item)`. -Choices: the stall is a busy-wait in a 5 ms timer's slot, due by frames received since the -record start (the record duration is counted on the GUI thread and stands still in a stall). -A part not judged leaves the verdict to the other parts: Pass if they pass (Measured and -Skipped mean other things), the message saying what was not judged. -Items 7 and 14 read as before (14: 0.299 s past, earlier runs 0.31–0.35 s); item 13 unchanged. -Seen failing: before the fix, both tests (0 looks, item 12 Fail: the flake); the whole-lead-in -row with "not judged" put back among the problems. -The next phase must know: the margin is the most frames received across one look's two reads; -an OS stall between those reads (not the event loop) widens it for the whole take and can -leave no look in any gap: now "not judged", not a Fail. -Left open: `test-tony-dev` 13 tests, about 228 s (two runs of about 22 s added). - -### Phase C2c — 2026-09-26 -Built: `Analyser::mergeRangedAnalysis()`: an old note sounding at W's start and ending inside -W takes the end of the run's note sounding there (onset before W), that end found as for an -added note (`newNoteEnd()`: `endBeyondRun` if the run's end cut it off, cut back at the next -old onset after W), then cut back to the first added onset as before. Recorded in -`m_rangedNotesChange` like the old cut, so undo restores the old note. Tests: -`TestRecordWorkflow::join_inside_a_held_note_keeps_one_note` (Record into Selection -[0.5, 1.5] then [1.5, 2.5] s on one held tone: one note; undo gives the first's note back -exactly, redo the one note); `TestSingingAnalysis::ranged_join_inside_a_note`, rows "the same -note going on" (one note) and "a new note from the join" (two, the first ending at J). -`TestDevChecks`: item 10's `QEXPECT_FAIL` gone; `dev_checks_fail_with_the_round_trip_off` -now requires item 10 to pass (it allowed either). -Choices: "the same note" = both sounding at W's start, where the audio has not changed. No -pitch tolerance (the values are medians over different stretches; a pitch the user corrected -must not stop the note). Only an old note ending inside W: one running past W keeps its end -(`ranged_keeps_a_note_across_the_window_edge` compares it exactly). A run's note beginning -before W with no old note sounding there is still not added: before W the models' notes stand. -Seen failing: the same-note cases of both new tests before the fix (the first's note alone, -0.517-1.509 s); the "new note" row with a naive join (an old note carried over a new note -beginning within 4 hops of its end, and the cut at the first added onset off). -Left open (`docs/takes.md`, known limits): a note running on past W keeps its old end where -the new audio stopped it inside W; a note whose onset in the run and in the models lie a hop -or two either side of W's start is lost (pre-existing, found by reading). - -### Phase C3 — 2026-09-26 -Built: `test-tony-device` gone (`TestRealDevice.h`, `tony-device-check.cpp`, its target; -`TestMainWindow`'s `setUseRealDevice()`, `haveAudioDevice()`, `haveRecordingDevice()`, used by -it alone). `AudioCheckRunner::deliveredNothing()`: not one frame received (the count restarts -with each take) `kNoInputTimeoutMs` (2 s) past the take's lead-in and range: the take is -stopped through the Stop path (`finishSingingTake()` finds nothing to use, so no take) and the -run ends "The audio device delivered no input ...". `FakeAudioIO::Config::neverCallsBack`. -`TestAudioCheck::check_ends_when_the_device_delivers_nothing` (not before length + 2 s − one -poll; no take; the next file analysed). Docs: AGENTS.md, building.md, testing.md, checklist -§1 rewritten, `DevChecks.h`; one line each in open-points.md and docs/README.md. -Coverage: `device()` → report header, its "can record" → the runner's "The take did not -start"; `record_the_reference_through_the_air` → stage 1 (near 61 and 151 s), channel levels -→ item 5, a range per recording → "A take was not added"; `takes_line_up_with_the_reference` -→ items 1, 2 (±2 ms against its ±10 ms and 5 ms; "match < 0.1" → NoSignal); -`nothing_of_the_take_comes_back_out` → 4; `stop_is_quicker_than_a_whole_song` → 9 (same 0.5); -`live_dots_were_drawn` → 3, 5 (same > 10); `no_input_does_no_harm` → the new ending and test, -and `check_fails_without_a_device`. Not in the dev run (not ported): a take with no lead-in and -not into a selection, stopped by hand; "no dialog" over every take (item 14: stages 3, 4 only); -with no device, the "Couldn't open audio device" warning per file opened (a dated note now). -Choices: not a fixed time after `record()`: a device slow to start (a Bluetooth headset -switching to its mic) is never taken for a dead one, and a working take has stopped itself by -then. One frame leaves the old limit in charge. No `TakeLatency` pushed (no take). -Seen failing: the new test before the change ("did not stop", 14 s); with the rule cut to 2 s -after `record()`, its timing (ended 2104 ms into a 3000 ms take). -Found: items 7 and 13 judge placement as 1 and 2 do (`offsetsOf()`): on MME they fail too. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index a4fb84b1..ffb11dfc 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -1,505 +1,646 @@ -# Calibrate Audio: plan - -Version 2. One button that measures the audio path. In development builds it also runs -every manual-checklist item that a speaker-to-mic loopback can settle. Nothing is built -yet. The research behind the figures quoted here (OboeTester, Bucket Brigade, Ardour's -MTDM, PortAudio's latency reporting, Audacity's measurements) was a separate report, -not kept in the repository. - -Setup assumed: wired headphones and a wired mic. For the run, hold one earcup against the -mic, off your ears. No cable is needed. A 3.5 mm loop cable would give the same result -with less noise. +# Calibrate Audio and the development checks + +**Playback ▸ Calibrate Audio…**, in every build, measures the round trip of the audio path +(from Tony handing a sound to the output device to the microphone's recording of it coming +back) through the ordinary take path, with one earcup of wired headphones held against the +microphone. **Use this latency** then places every take on those devices with the measured +figure instead of the one the driver reports. In development builds the same dialog can +carry on into the **dev checks**: a scripted run that records test references through the +air and settles, with a verdict and numbers, the device items of the manual checklist. + +This page is the design as built: why, what the button and the dev run do, what each +verdict and check means, which numbers to send back, the architecture, the tests, the +decisions and what is still open. How to run it on a machine, and what fails on MME today, +is in [manual-checklist.md](manual-checklist.md), section 1. The research behind the +approach (OboeTester, Bucket Brigade, Ardour's MTDM, PortAudio's latency reporting, +Audacity's measurements) was a separate report, not kept in the repository. ## 1. Why -- **Every take is shifted by the wrong amount.** Tony shifts it by *reported output - latency + reported input latency + start gap* (`MainWindow::recordingStarted()`, - `LatencyUtils.h`). The start gap is measured and right. The reported pair comes from +- **The driver's latency is wrong.** A take is placed by taking the round trip and the + start gap off the front of its recording ([recording.md](recording.md#latency)). Without + a measured figure the round trip is the sum of the output and input latency the device + reports. The start gap is measured and right; the reported pair comes from `Pa_GetStreamInfo()`, which on MME, DirectSound and WASAPI is buffer sizes only. Audacity measured it off by −5 to +155 ms. bqaudioio opens the stream with `suggestedLatency = 0.2` on both sides. - **The device's sample rate is not checked.** The device opens at PortAudio's default - rate. For "(System Default)" through MME that is most likely 44.1 kHz. For a device - chosen from the menu whose name exists only under WASAPI or WDM-KS, it is often 48 kHz. - `TakeAudio::splice()` writes the first recording of a take at that rate and places it - with frames of the 44.1 kHz reference. -- **Most of the manual checklist has never been run** (`docs/manual-checklist.md`, - 36 items). Many items ask whether logic that the app suite proves on `FakeAudioIO` - also holds on a real device. A loopback run in the real app can answer that without a - person listening. + rate: for "(System Default)" through MME most likely 44.1 kHz, for a device whose name + exists only under WASAPI or WDM-KS often 48 kHz. A take's first recording is written at + that rate and placed frame for frame on the 44.1 kHz reference, so at 48 kHz it lands + early by 8 % of its position. The check names the mismatch; it does not fix it (§10). +- **The manual checklist's device items had never been run.** Many ask whether logic the + app suite proves on `FakeAudioIO` holds on a real device. A loopback run in the real app + answers that without a person listening. ## 2. The button -**Playback ▸ Calibrate Audio…** is in every build. - -1. **Instructions:** - - an earcup against the mic, off your ears; - - moderate volume, quiet room; - - names the output and input device and the latency now in use (reported or - measured). -2. **Test session.** Tony writes a generated test reference and opens it the way - File ▸ Open does. You are asked to save your work first. The run never touches your - song's takes or undo history. - *Since C1a:* not when the session open is a check's own (never saved, its reference - in the check's directory): that is replaced without asking. -3. **Calibration, about 30 s.** Four punch-ins, each spanning three of the reference's - sweeps with the room the finder needs around them. The ranges come from the layout, - not from fixed times: a 5 s punch-in judges only one or two sweeps. They use the - ordinary take path with Play Reference While Recording on. Every punch-in restarts the - stream, as a real take does. - *Found in B1:* the calibration layout cannot hold four times three. Where one - punch-in ends and the next begins, two sweeps must be 1.9 s apart (what the finder - reads around each), and the layout's boundaries after its 3rd, 6th and 9th sweeps - are 2.5, 1.7 and 1.8 s. `punchInsFor()` returns nothing for 4 × 3; 4 × 2 and 3 × 3 - fit. Swapping two pairs of spacings (`{21,16,25,19,17,23,26,20,24,18,22}`) would - make 4 × 3 fit, ending at 25.2 s. -4. **Result page:** - - **Round trip:** measured, next to the driver's figure. - - **Spread between punch-ins:** how much the driver's timing moves from one stream - start to the next. - - **Sample rates:** the recording's rate against the reference's 44100 Hz. - - **Mic channel:** which input channel carries the mic. - - **Levels:** noise floor, and the input peak (clipping). - - **Verdict:** in plain words, with the fix for each failure. - - **Use this latency** stores the figure. -5. **Development builds only:** the dialog carries on into the **dev checks**, about - 4 minutes (section 5), unless a checkbox on the instructions page (on by default) - is cleared. It ends with a report page and a report file. - -The test session stays open afterwards, so the reference's and the takes' pitch tracks -can be looked at. You go back to your song through Recent Files. - -## 3. Dev mode - -`meson.build` already switches on the build type (`WANT_TIMING` versus `NO_TIMING`). Add: - -- `-DTONY_DEV_CHECKS` when the build type does not start with `release`; -- the dev-check source files, compiled only then. - -`build.bat` sets up `build_mingw` as `debugoptimized`, so your builds have the checks. -`meson.build`'s default and the deploy scripts use `release`, so packages have none of -the code. - -A runtime flag, so that a release build on someone else's PC could run the checks, is -possible later. It would ship the check code in every package, so it is not part of this -plan. - -## 4. What the loopback run settles, item by item - -*Since the merge of `default` (2026-09-26):* `default` automated much of the checklist on -its own. `TestUiChecks` covers the screen, keyboard and dialog items on the fake device, -the smoke items below among them. `test-tony-device`, run by hand, covers the device items -(1, 2, 4, 5, 6, 9). The user decided that the dev run takes the device items over and -`test-tony-device` is retired; the smoke group and items 15 and 16 are dropped from the dev -run. The table keeps the old numbering, which `default` has since rewritten; phase D -brings the two together. - -The numbers are those of `docs/manual-checklist.md` before that merge. - -- **Automated:** the dev checks pass or fail it. -- **Measured:** the dev checks report numbers; a person still judges how it looks or - feels. -- **Smoke:** the logic is already covered by the app suite; the dev checks repeat it on - the real device and real timing. Optional, last. -- **Manual:** stays on the checklist. - -| # | Item | Verdict | How | +**The Playback menu**, after the two audio device submenus: + +- **Calibrate Audio…**, disabled during any take and while a check runs; +- a line that cannot be chosen, the latency takes are placed with now: "Latency: measured + 281 ms, 26 Sep" (the year only when it is not this one), or "Latency: driver's figure, + 279 ms", with "(the measured one is out of date)" when a stored figure is stale, or "not + known yet" while the device reports nothing. Brought up to date whenever the menu opens; +- **Forget Measured Latency**, enabled while a figure is kept for these devices, stale or + not. + +While a check runs, Record and both device submenus are disabled as well. + +**The dialog** is not modal: the check's session is in the window and can be looked at +meanwhile. Closing the dialog while its check runs cancels the check, since nothing else +would show how the run ended. Three pages: + +1. **Instructions:** one earcup against the microphone, off your ears; a moderate volume + and a quiet room; the output and input devices and the latency in use; how long it + takes. In development builds the checkbox **Run the dev checks after calibrating**, on + every time the page is shown and not remembered. **Start**. +2. **Progress:** the step and the punch-in, a bar and the time left (the recording to come, + plus a guess of 3 s for each analysis), **Cancel**. During the dev checks, their stage. +3. **Result** (§4): **Use this latency** (only when the calibration is usable), **Check + Again**, **Close**. The text can be selected and copied. + +**What a check does:** + +1. **A session of its own.** Tony writes the calibration reference (§3), a WAV of sweeps + and tones, into the application data directory and opens it as File ▸ Open does, + replacing the session. It asks to save the session open first, unless that is a + check's own: never saved, and playing a reference from that directory. The reference is + `calibrate-audio-reference-N.wav` with the lowest N free, after every other such file + in the directory is removed: a check never writes over the file the session open now + plays (Windows will not let it), and Recent Files, which lists every file opened and has + no remove, gets two of these names at most. +2. **Four punch-ins of three sweeps each** into the reference's take, about 6 s each. + Each is an ordinary take: its range is selected, `record()` is called with Record into + Selection, Play Reference While Recording and a 1 s pre-roll whatever the toolbar says, + and the take stops itself at the end of the selection through the Stop path, is spliced + and analysed. So every punch-in restarts the audio stream, as a real take does, and is + placed with the round trip every take is placed with, which is what is being measured. + The next punch-in waits for the take's analysis, which keeps pYIN's load out of its + timing. +3. **The check's playback.** Tony normalises every audio file to full scale as it reads it + and pans the reference hard left and its pitch and notes sonification hard right. The + check's session plays the reference centred, at a gain that brings it back to the + −12 dBFS it was made at, and the sonification silent. It stays so after the run; a + session opened afterwards plays as before. +4. **Judged from the take's file** (`AudioCheckRunner::readTakeFile()`), mixed to one + channel at the rate it was recorded, never from the take's model: the model is + normalised to full scale as it is read, so every take would read as clipped, and + resampled to the session's rate. + +Under a minute in all. Every step has a limit and ends the run with a reason: 60 s for the +reference's analysis, 30 s for a take's, a take's lead-in and range plus 10 s to stop +itself. A device that opens but has delivered not one frame 2 s after the take's lead-in +and range should have gone by ends the run with "The audio device delivered no input", +stopping the take through the Stop path, which then finds nothing to splice. Timed from +the take's own length, not from `record()`: a device slow to start (a Bluetooth headset +switching to its microphone) is never taken for a dead one. A device that cannot be opened +ends it with "The take did not start". Cancel, and a session closed during a run, stop a +take in progress through the Stop path. The window's destructor abandons a run without the +Stop path, which would splice and start pYIN in the middle of the teardown. + +The test session stays open afterwards, so that its pitch tracks can be looked at. The +song comes back through Recent Files. + +## 3. Finding the sweeps, and the verdicts + +**The reference** (`LatencyCheck`). Each event is a linear sweep from 1 to 8 kHz, 200 ms, +with 10 ms raised-cosine edges and a −12 dBFS peak; 0.1 s of silence; a tone of 0.8 s at +196, 220.5, 245 or 294 Hz in turn (a whole number of samples per period at 44.1 kHz, or +pYIN reports a subharmonic: [testing.md](testing.md)); silence to the next event. Linear +rather than exponential: its spectrum is flat, so its matched filter gives the narrowest +peak, and the harmonics a small speaker adds to an exponential sweep match the sweep itself +shifted 67 and 106 ms earlier, where the earliest-peak rule looks. The spacings from sweep to +sweep are irregular, 1.6 to 2.6 s and all different by 0.1 s at least, so that a take +misplaced by whole events cannot look right. + +- *Calibration*, 26 s: twelve events, sweeps at 1.0, 3.1, 4.7, 7.2, 9.1, 10.8, 13.1, 15.7, + 17.7, 20.1, 21.9 and 24.1 s. The order of the spacings is what lets four punch-ins of + three events fit: where one punch-in ends and the next begins, 1.9 s of spacing is lost. +- *Dev*, 40 s: the calibration's events, then three with 3 s held tones, after sweeps at + 26.9, 30.9 and 35.2 s. +- *Long*, 240 s or any length (the start of the 240 s one): the calibration's spacings over + and over. No more than eleven spacings can differ by 0.1 s within 1.6 to 2.6 s. + +**The finder** looks within ±0.8 s of where a sweep should be: an FFT matched filter, the +envelope of its output, and the **earliest** peak within 6 dB of the largest (at least 1 ms +before it), so that a reflection stronger than the direct sound is not taken for it. A sweep +is *found* when its peak stands 15 dB over the window's median and 6 dB over the highest +peak more than 10 ms from it. Its **offset** is where it was found minus where the +reference has it: positive is late. + +**Judging a take.** An event is judged in a punch-in only when all the finder reads for it +(its window, and a sweep's length past it) lies inside the punch-in's range with 50 ms to +spare: the splice cuts and crossfades at the range's ends, and a sweep cut in half says +nothing about the audio path. Where a later punch-in overlaps an earlier one, the event is +judged in the later only. Each punch-in has the median and spread of its offsets; across +punch-ins, the offset is the median of their medians (each stream start counts once), with +their spread and a line fitted over position. + +**Verdicts**, in order of precedence: the first that applies is the verdict, and all are +kept. + +| Verdict | When | What it means, and the fix the result page gives | Usable | +| --- | --- | --- | --- | +| NoSignal | fewer than 2/3 of the judged sweeps found, or none | the microphone did not hear the sweeps: volume, a muted or wrong input, Windows' audio enhancements, a Bluetooth headset's Hands-Free device | no | +| Clipped | the input reached −0.2 dBFS in the punch-ins | too loud, which can move where sweeps are found: turn down, or hold the earcup a little away | no | +| Fading | the sweeps' level fell by 10 dB from the first half of the judged events to the second (six at least) | something filters the microphone: echo cancellation, noise suppression, audio enhancements | no | +| PositionDependent | over three punch-ins at least, the offsets lie on a line steeper than 0.5 % that leaves less than 5 ms | recording and playback run at different speeds | no | +| Scattered | the offsets disagree by more than 15 ms, across punch-ins or within one | the driver's timing varies too much for one latency: close other programs that use sound | no | +| Unsteady | they disagree by 5 to 15 ms | small enough: the measured round trip is the middle of it, and a take may land up to half the spread off | yes | +| Ok | none of these | the sweeps came back steadily | yes | + +Two findings stand beside the verdict: + +- **A rate mismatch**, the recording's rate not the reference's. It replaces the verdict's + words, and the calibration is not usable whatever the verdict says. It comes from the + two rates, not from the sweeps: at 48 kHz a punch-in from about 10 s into the reference + lands further off than the finder searches, and the sweeps then read as Scattered. +- **An echo**: a second peak at the same delay, within 3 ms, after more than half of the + sweeps heard and three at least, 20 ms late or more and no more than 30 dB down: the + input is played back out somewhere (Windows' "Listen to this device", an interface's + monitor) and heard again. A paragraph on the result page; not a verdict. An echo under + 20 ms is not seen: that is where the tail of a close reflection lies. + +**The calibrated round trip** is the one the takes were placed with plus the median offset. +A take that landed late was spliced from too early a frame of its recording, so the round +trip grows. + +Every threshold is a starting value. The finder's were checked on synthetic takes (noise +alone reads 6 to 12 dB over the median, against 15; noise at 0 and −10 dB SNR still reads 38 +and 28 dB over it); none has been tuned on a real device yet. + +## 4. The result page + +The verdict in one sentence and its fix; the echo, if one was heard; then a table: the +round trip measured (not for NoSignal or a rate mismatch) against the driver's, output plus +input; what the takes were placed with (measured before, or the driver's figure); where each +punch-in landed (+ is late); the spread; sweeps found of those judged; both rates; the input +peak in dBFS; the echo; the devices. A failed run shows why it ended instead. + +**Use this latency** keeps the calibrated round trip for the devices the check started on, +not for those the Preferences name when it is pressed (the result stays on show for as long +as the user likes, and the device submenus open again when the run ends), and for the rate +the takes were recorded at. + +## 5. The measured round trip in use + +`LatencyCalibration` keeps one figure per **key**: the audio driver and the playback and +record devices as the Preferences name them for `MainWindowBase::createAudioIO()` +(`audio-target`, then `audio-playback-device` and `audio-record-device`, each suffixed with +the driver when one is named), and the rate the device records at. In QSettings: +`LatencyCalibration/||//`. Device names can hold `/` and +non-ASCII characters; only `%`, `/`, `\` and `|` are percent-encoded, so that no subgroups +are made and a long non-ASCII name stays within the 255 characters of a Windows registry +key. The key follows `createAudioIO()`, not `audioDeviceSettingKey()`: they differ only for +an `audio-target` of "auto", which Tony never writes. + +Stored: the round trip and its spread in seconds, the date, and the output and input +latency the device reported then, in seconds. That pair is the **staleness fingerprint**: +when either latency the device reports now differs by more than 1 ms, its buffers have +changed and the round trip with them, and takes go back to the reported pair until the +check is run again. A stale figure is not deleted: it applies again if the driver goes back +to its old buffers. + +At every take, `MainWindow::roundTripAt()` gives the stored figure for the devices and the +recording's rate if there is one and it is not stale, else the reported sum, each reported +latency converted to seconds at the rate it is counted in. Then it is turned into frames of +the recording ([recording.md](recording.md#latency)). With no figure stored, at 44.1 kHz, +the round trip is exactly the old sum; a core test checks it over a grid of values. + +The menu line, Forget Measured Latency and the dialog's instructions use the rate of the +last take placed with a round trip, or before any take the session's: the device's rate is +not known before a take (`AudioCallbackRecordTarget` has no getter for it), and the session's +is the only rate a usable check stores at. Choosing a device from the menu resets it. + +A dev run places its takes with the round trip the calibration before it measured, for the +run only: nothing is stored, the menu line goes on describing the window's own figure, and +the log calls it the audio check's own. + +## 6. Development builds + +`meson.build`: a build type that does not start with `release` (debug, debugoptimized, +plain, minsize, custom) adds `-DTONY_DEV_CHECKS` to the compiler's and moc's defines, +compiles `main/dev/`, and builds `test-tony-dev`. `build.bat` sets `build_mingw` up as +`debugoptimized`, so the development machine's builds have the checks; `meson.build`'s +default and the CI workflows use `release`, so packages have none of the code. Every use of +`main/dev/` elsewhere (a `friend` line, a member, the dialog's checkbox, the test class) is +inside `#ifdef TONY_DEV_CHECKS`: a `release` build must compile with no `main/dev/` file, +and building one is how to check it. + +Verified by the lead on 2026-09-26, on Linux: `meson setup build_release +--buildtype=release`, then `tony`, `test-tony-core` and `test-tony-app` built. No compile +command carried `-DTONY_DEV_CHECKS`, no `main/dev/` file was compiled, there was no +`test-tony-dev`, and `nm -C tony` found no `DevChecks` or `TakeObserver` symbol. Both +release suites were green: the core suite but for the four `TestTakesFile` tests that fail +on Linux only ([building.md](building.md)), the app suite whole (`TestAudioCheck` 25 passed, +`TestRecordWorkflow` 99, `TestUiChecks` 19, among others). `TestAudioCheck` runs the same in +both kinds of build: its fixture deletes the dev checks of a development build's window. + +Not done: a runtime flag, so that a release build on someone else's PC could run the +checks. It would ship the check code in every package. + +## 7. The dev run + +It starts once the calibration is done, when the checkbox was on and the calibration is +usable (otherwise the result page says why it did not run), and ends on the result page: +the calibration as above, a line for each check, and the report file's path. Cancel ends it. +A few minutes in all. + +### Stages + +| # | Stage | What it records | Items | | --- | --- | --- | --- | -| 1 | Latency on this machine, also after save and reopen | **Automated** | Every sweep of every punch-in lands within ±2 ms (§6). Save the test session to a temp `.ton`, reopen it, analyse again: unchanged. | -| 2 | Several phrases in one take | **Automated** | Punch-ins at four positions of one take, each placed right. Each measures its own start gap. | -| 3 | Live dots | **Measured** | *Automated:* dots sit on the reference's tones on the timeline (±1 hop; *found in C1b:* a dot trails the sound it heard by up to half the tracker's window, so from a tone's start to half a window past its end, ±1 hop); they stay after Stop until the pitch track arrives, then go; the status bar stops changing; the take's own pitch and notes are hidden during the take and back after. *Reported:* how far behind the cursor a dot appears, in ms. *Eyes:* does it look right. | -| 4 | Nothing of the take in the speakers | **Automated** | Re-record over earlier material. Tony's output peak (`getOutputLevels()`) is exactly zero in the reference's silent gaps, so no take audio and no synth. The mic hears no second arrival of each sweep; one would mean the input is monitored somewhere (Windows "Listen to this device", or an interface's direct monitor). Play Singing Audio keeps its state. | -| 5 | Mic on input 2 of a stereo interface | **Measured** | Per-channel input peaks show which input the mic is on. If it is input 2, dots appearing is the check; otherwise "not applicable here". | -| 6 | No input device / device in use | **Manual** | Needs the device gone or busy. | -| 7 | Record from a position; overwrite question | **Automated** (placement) | Placement as in 1. Outside the new range, the take's audio is bit-identical and pitch and notes are unchanged beyond ±0.25 s. The question, No, and "Don't ask again" stay with the app suite (`record_over_existing_question`) and a glance. | -| 8 | Cursor from P, pane follows, all in one place | **Measured** | *Automated:* the cursor is at S when the take starts, and inside the visible range throughout. *Reported:* the dot-to-cursor offset. *Eyes:* the rest. | -| 9 | Stop on a 4-minute song | **Automated** | A generated 4-minute reference, punch-in near the end. Time from Stop to new pitch, against a threshold. Pitch outside the range unchanged, which proves the ranged path ran. | -| 10 | The joins | **Automated** | Two punch-ins that meet in the middle of a held tone: no step in the samples at the join; pitch continuous (no gap, no doubled frame); **one** note across the join; nothing moves outside ±0.25 s. *Found in C2:* the note failed: the second punch-in's analysis starts 0.5 s before the join, inside the tone, so its note began before the merge window and was dropped, and the first's note ended at the join. A whole-take analysis gives one note. *Fixed in C2c:* an old note sounding at the merge window's start and ending inside it takes the end of the run's note sounding there (`docs/takes.md`). | -| 11 | Is 3 s right, is the countdown readable | **Manual** | A judgement. | -| 12 | Lead-in: nothing heard back, nothing before P changed | **Automated** | Output peaks during the lead-in are the reference's only. Audio and events before P are unchanged. *Found in C1c:* on a noiseless loopback the earlier take is silent wherever the reference is, so a take played out shows in no gap; the app test gives the fake a −60 dBFS noise floor, as a room gives a real mic. | -| 13 | Pre-roll near the start | **Automated** | Punch-in at P = 1 s: playback runs from 0, the countdown counts only the 1 s there is (*found in C1c:* plus the round trip, by design, so it starts at 2; for the instant before the round trip is known it shows 1), placement is right. | -| 14 | Record into Selection stops by itself | **Automated** | Stops within 0.25 s plus one poll of the end. The coverage added is exactly the selection. No dialog. | -| 15 | The practice loop | **Measured** | *Automated:* it stops by itself, the playhead is back at P, and Play plays the take. *Yours:* "is anything else needed?". | -| 16 | Constrain Playback to Selection with a pre-roll | **Measured** | Reports whether the lead-in was played in full. The decision is yours. | -| 17 | Coverage band readable | **Manual** | Eyes. | -| 18 | Band cannot be touched | **Manual** | Could become an app-suite test with synthetic mouse events; it needs no device. | -| 19 | Select Recording at Playhead, then Erase | Smoke | Audio zero in the range, band and events gone, no analysis. | -| 20 | Erase/Select greyed out when they should be | Smoke | Action states sampled through a real take and its real analysis time. | -| 21 | Ctrl+Z three times | Smoke | On real takes, with the menu texts. | -| 22 | Ctrl+Z straight after Stop | Smoke | With real analysis timing. | -| 23 | Undo of the very first recording | Smoke | | -| 24 | New Empty Take, switching | Smoke | Switch time measured. The inactive take is silent: output peaks over its region while the active take is empty there. | -| 25 | Duplicate, record into the copy | Smoke | The original's file is bit-identical afterwards. | -| 26 | Delete asks first; undo clearing acceptable? | **Manual** | Dialogs and a judgement. | -| 27 | Combo and Takes menu greyed during a take | Smoke | Sampled during a real take. | -| 28 | Analyse Now on a take | Smoke | | -| 29–35 | Sessions and files | **Manual** | File system and OS. Mostly covered by the app suite already; item 35 needs a log-out. | -| 36 | Looks | **Manual** | Eyes. | - -**Tally:** - -| Category | Items | Count | -| --- | --- | --- | -| Automated | 1, 2, 4, 7, 9, 10, 12, 13, 14 | 9 | -| Measured, judgement left | 3, 5, 8, 15, 16 | 5 | -| Smoke | 19–25, 27, 28 | 9 | -| Manual | 6, 11, 17, 18, 26, 29–36 | 13 | - -Also newly checked, though not a checklist item yet: the device's sample rate. - -## 5. Architecture - -Following `AGENTS.md`: state gets its own class, `MainWindow` only wires, and pure logic -goes in `tony_core`. - -### `tony_core` (pure, core suite) - -- **`LatencyCheck`** - - **Generator.** Fixed layouts: - - *calibration*: 26 s; - - *dev*: adds held tones of 3 s for the join check; - - *long*: 4 minutes. - - Each event is a sweep of 1 → 8 kHz, 200 ms, with 10 ms edges and a −12 dBFS peak, - followed by a tone at a pitch pYIN tracks (196, 220.5, 245 or 294 Hz; see - `docs/testing.md`). Gaps between events are irregular (1.6–2.6 s, all different). - Only eleven gaps can differ by 0.1 s in that range, so the *long* layout repeats - the calibration's eleven, and the gaps around *dev*'s held tones are longer. - The generator returns every sweep's exact frame. - - **Analysis:** - - FFT matched filter in the sweep band (bqfft), then its envelope; - - the **earliest** peak within 6 dB of the largest; - - confidence: ≥ 15 dB over the window's median, ≥ 6 dB over the second peak outside - ±10 ms (starting values, to tune); - - error per sweep; median and spread per punch-in and across punch-ins; slope over - position; - - **second-arrival detection**, for monitoring echo; - - verdicts: Ok, NoSignal, Fading, Clipped, Scattered, Unsteady, PositionDependent. - - **Calibration arithmetic:** `newRoundTrip = usedRoundTrip + median offset`. A take - that lands late was spliced from too early a frame. -- **`TakeDiff`** - - audio bit-identity outside a range; - - pitch and note events unchanged outside a range ± margin; - - pitch continuity across a join (gaps, doubled frames); - - notes spanning a frame; - - sample step at a join. -- **`LatencyCalibration`** - - **Key:** the Preferences values `createAudioIO()` reads, plus the recording rate. - - **Value:** round trip in seconds, spread, date, and the reported pair at the time, - which works as a staleness fingerprint. - -### App, every build - -- **`AudioCheckRunner`.** Primitives on the live window: - - generate and open a test reference; - - `punchIn(P, E)`, recorded with Record into Selection, Play Reference While Recording - on and a 1 s pre-roll, **without writing the user's settings** (the three actions - write QSettings when toggled, so this needs an override inside `record()`, not - `setChecked()`); - - wait for the splice and for the analysis; - - read the take's audio, coverage and events. - - A **`TakeObserver`** polls every 20 ms during a punch-in and records: - - output and input peaks per channel; - - each live dot, with the cursor frame when it was added; - - playback frame, pane centre, status text, action states; - - frames received, and when the take stopped. -- **`CalibrateAudioDialog`.** Instructions, progress, the result page, Use this latency. -- **`MainWindow`.** - - The menu entry. - - In `recordingStarted()`: "stored round trip if valid for this key, else the reported - sum", logged either way; converted at the recording's rate. - - A record, kept at take start, of the round trip each take used. - - "Forget Measured Latency". - -### App, development builds only (`main/dev/`) - -- **`DevChecks`.** A list of checks. Each returns a plain - `CheckResult { item, name, verdict (Pass/Fail/Measured/Skipped), numbers, message }`. - The run is a list of stages, each starting something and saying when it is done, - driven by the runner's `finished()` and a polling timer as the runner itself is, and - never by a nested event loop: the window can be closed at any moment. Cancel stops - the take and leaves the test session open. - - They are **not** QtTest functions. A QVERIFY failure cannot be asserted from inside - another QtTest, and the app suite has to prove each check can fail. -- **Access.** `DevChecks` is a `friend` of `MainWindow` under `#ifdef TONY_DEV_CHECKS`, - so there is no public surface in release builds. -- **Report.** A page in the dialog, grouped by checklist item, with the measured numbers. - A text file goes to `TONY_TEST_LOG_DIR` if that is set, else to the app data directory, - and ends with a `Totals:` line like the suites. -- **Scratch files.** The run saves its test session into a numbered scratch folder, so - every take file lands in `.takes/`, and the report names it. The folder stays - after the run, since the session open then lives in it; the next run removes every - one the open session does not use. - -### The dev run, one scripted sequence (~4 min) - -Calibration first. The dev checks then use the new figure for the run only; your stored -setting changes only through Use this latency. - -| Step | What it does | Items | Phase | +| 1 | Long song | a 240 s long reference as a session of its own; two punch-ins of about 2 s, the shortest ranges that judge the first sweep from a quarter and from five eighths of the way in (61.0–63.0 and 150.8–152.8 s) | 9; and 1, 2, 4 | +| 2 | Fresh punch-ins | the dev reference as a session of its own; [6.3, 10.2] and [16.8, 21.2] s, judging two sweeps each | 1, 2, 3, 4, 5 | +| 3 | Re-record | into the same take, [19.2, 21.2] s over the end of stage 2's second punch-in, with a 2.4 s pre-roll | 7, 12, 14; and 1, 2, 4 | +| 4 | Pre-roll near the start | [1.0, 4.2] s, judging the sweep at 3.1 s, asking for a 3 s pre-roll | 13, 14; and 1, 2, 4 | +| 5 | Joins | [26.0, 28.7] then [28.7, 32.0] s, meeting at J = 28.7 s, the middle of the held tone from 27.2 to 30.2 s | 10; and 1, 2, 4 | +| 6 | Save and reopen | the session saved into a scratch folder the way Save As saves once it has a name, opened again, and its take's file judged again | 1 | + +Why so: + +- **The long song first**, so that the dev reference then replaces its session as a + check's own, unsaved, without asking, and the run still ends on the saved session: no + saved session is ever replaced. Far into a song is also where a rate mismatch shows. +- **Stage 2's ranges** leave room for the rest: the start (before 4.3 s) for stage 4; the + held tones for stage 5; and each range holds two sweeps, so that stage 3 can start inside + the second one past its first sweep and still judge the other. +- **Stage 3** starts at 19.2 s: the shortest range that judges the sweep at 20.1 s, with a + whole note of the take (18 to 18.8 s) before it that the ranged analysis must leave + alone. Its lead-in plays from 16.8 s over what stage 2 recorded: two gaps where the + reference is silent (16.8–17.7 and 18.8–19.2 s), a sweep and a tone. Two gaps, so that a + window held up over one still has the other for item 12 to judge. +- **Stage 5's** second punch-in's 1 s lead-in plays over what the first recorded; J lies + 1.5 s from either end of the tone, further than item 10 looks around it. + +A stage has 4 minutes (the reopen 1 minute), a backstop behind the runner's own limits, +which end a run first and give their reason. A stage that fails ends the run; the checks are +then worked out from the stages it got through, and the rest are Skipped with the reason. + +A **`TakeObserver`** watches every punch-in from its Recording step until its analysis is +done. Every 20 ms it keeps the cursor (the `ViewManager`'s playback frame), the frames +received just before and just after reading the output levels, the output levels, the input +levels (from the level meter's signal), the status bar's text and whether a modal dialog is +up; each live dot as it first appears, with the cursor at that moment; the raw recording's +path, when the take stopped, and Play Singing Audio before and after. + +### The checks + +The items are numbered as the manual checklist was when they were planned, before `default` +rewrote it; the report and [manual-checklist.md](manual-checklist.md) §1 use these numbers. +"Placed within ±2 ms" means every judged sweep found, with its offset within ±2 ms. + +| Item | Check | Passes when | Numbers | | --- | --- | --- | --- | -| 1 | Long reference, two punch-ins far apart | 9 (and 1, 2, 4) | C2 | -| 2 | Two punch-ins into fresh regions, observer on | 1, 2, 3, 4, 5, 8 | C1a, C1b | -| 3 | Re-record over an earlier punch-in, through its lead-in | 7, 12, 14 | C1c | -| 4 | Punch-in at P = 1 s with a 3 s pre-roll | 13 | C1c | -| 5 | Two adjacent punch-ins meeting inside a held tone | 10 | C2 | -| 6 | Save to the scratch `.ton` and reopen | 1 | C1a | - -*Since C2* the long reference comes first: the dev reference then replaces its session -as a check's own, unsaved, without asking, and the run still ends on the saved session. - -Items 15 and 16 and the smoke group are no longer in the dev run (§4). - -## 6. Tests - -- **Core suite, `TestLatencyCheck` and `TestTakeDiff`.** Synthetic takes: - - every shift from −0.75 to +0.75 s, including a fractional one; - - noise at 0 and −10 dB SNR; - - small-speaker band limiting; - - polarity inverted; - - a stronger reflection 7 ms after the direct sound; - - an event missing, silence, fading events, clipping; - - resampled by 48000/44100; - - two punch-ins 20 ms apart; - - a monitoring echo; - - the join cases. -- **App suite.** `TestMainWindow` with `FakeAudioIO` `loopback = true`: - - **Calibration.** Wrong reported latencies are measured right. With the stored - figure, `latency_end_to_end`'s recipe lines up; without it, the same test fails. A - stale key falls back to the reported sum. The user's three toggles and their - settings are untouched. A 48 kHz fake is reported as a rate mismatch, naming - both rates. This comes from comparing the recording's rate with the reference's, - not from the sweeps: from about 10 s into the reference, a 48 kHz take lands further - off than the finder searches, and the sweeps then read as Scattered (found in A2). - - **Every dev check at least twice.** Once passing on a calibrated fake, and once - **failing** under an injected fault: - - an uncalibrated offset, for items 1, 2, 7 and 13; - - a monitoring echo, from a second loopback tap in the fake, for item 4; - - a splice offset broken on purpose, for item 10; - - a 48 kHz device. - - That is the "can fail" proof for each one. - - **`FakeAudioIO` additions**, all new fields that change no existing meaning: - loopback gain, a second echo tap and noise. - - **Where they run.** If the dev-check tests add more than about a minute of real time - to the app suite, they move to a third executable, `test-tony-dev`. That would - change the "run both whole suites" rule in `AGENTS.md`, so ask first. - -## 7. Order of work - -Every step ends with both whole suites green. The work is split into phases for a line -of agents; their work orders are in -[calibrate-audio-work-orders.md](calibrate-audio-work-orders.md). Each phase's line is -marked "Done" when it is committed. - -1. **Core.** - - **A1** Test reference and sweep finder: `LatencyCheck` generator and per-event - analysis. Done. - - **A2** Verdicts and calibration arithmetic: aggregation over events and punch-ins. - Done. -2. **Runner, dialog and calibration page** (every build), with app tests. - - **B1** The alignment check runner and its app tests. Done. - - **B2** Storing the measured round trip and using it in takes (`LatencyCalibration`, - `recordingStarted()`, staleness). This was step 3 below; it moved up because the - dialog needs it. Done. - - **B3** The check's playback: the reference centred at −12 dBFS, the sonification - silent, and a progress signal. Done. - - **B4** The Calibrate Audio dialog and menu entry. Done. - - **Then you run it on your PC.** Its numbers settle three things: how wrong the - driver's figure is, whether the offset holds across stream restarts on MME, and - whether your device's rate hits the takes. Work goes on meanwhile: only the - thresholds and the restart-jitter remedy wait on those numbers. - *First run, 2026-09-26* (MME, wired mic and headphones): round trip 301 and 295 ms - with one earcup to the mic, verdict Unsteady both times (5–15 ms); Scattered with the - mic between both cups. *A dev run (C1a's):* round trip 303.5 ms; the two sweeps of - each punch-in agree to 0.2–0.3 ms, but punch-in 1 landed at +0.6 ms and punch-in 2 at - −12.9 ms; the measured start gap was 0 frames both times. So the finder is precise, and - what moves is the stream's input-to-output offset at each restart, which the start gap - does not see. The decision is in §8, "Restart jitter". -3. **Calibration in use:** built in B2 (Use this latency and Forget in B4). -4. **Dev-check framework:** - - **C0** `TakeDiff`, pure. Done. - - **C1a** Build flag, `DevChecks`, report, friend access, the dialog's dev run. - Items 1 and 2. Done. - - *After the merge of `default`:* `TestDevChecks` moved into an executable of its - own, `test-tony-dev`, run when a change touches what the checks drive (the user's - decision). The phases below were cut again (§4). - - **C1b** `TakeObserver`. Items 3, 4, 5 (and 8's number). Done. - - **C1c** Re-record and pre-roll stages. Items 7, 12, 13, 14. Done. -5. **C2** Long song and joins: items 9 and 10. Done. - - **C2b** Item 12 robust against a stalled event loop (a flake C2 found). Done. - - **C2c** The notes merge keeps one note across a join: the defect item 10 found (the - user chose to fix it on this branch). Done. -6. **C3** Retire `test-tony-device`, once all it checks is in the dev run. Done. -7. **Release build** by the lead (§8, "Release builds must stay clean"). -8. **D** Docs, from the code and the phase log: - - `manual-checklist.md`: an automated item keeps its text and gets "*automated: - dev check ``*"; a measured item keeps only the question for a person. The - list becomes what a person must do after a dev run. - - `testing.md`: a Dev checks section. - - `recording.md`: the latency section. - - `open-points.md`. - -**Next project, after this branch: a lower-latency driver** (the user's decision, -2026-09-26, from the restart jitter in §8). In order: - -1. **The device-rate mismatch.** A take recorded at 48 kHz is placed frame for frame - into a 44.1 kHz session. The robust fix converts when the take is spliced, whatever - the device's rate; the other way, the record target asking for the session's rate - (`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork), fails - where the device only runs at its mixer's rate. It comes first, because WASAPI opens - at the Windows mixer's rate, usually 48 kHz. The button then shows the fix working. -2. **A `bqaudioio` fork**, `jhhr/bqaudioio` (created 2026-09-26, the remote `jhhr` in - `bqaudioio/`; `repoint-project.json` still takes sourcehut until this phase): - choosing the host API, WASAPI's automatic rate conversion, and the 0.2 s - `suggestedLatency` made settable. -3. **Driver type in Tony:** MME, DirectSound or WASAPI; the device menus list that - type's devices only (today every host API's are listed, and `getDeviceIndex()` takes - the first name that matches, which is MME's); the stored round trip kept per type. +| 1 | `latency_on_this_machine` | every punch-in of every stage placed within ±2 ms; after the reopen, the take's file judged again over the dev take's punch-ins gives the same offsets to the frame, and the pitch and notes the session restored are the same events (values as the file rounds them) | offsets of each punch-in, the largest, the round trip used, the same after reopening | +| 2 | `several_phrases_in_one_take` | at least two punch-ins; each wholly in the take's coverage, its median offset within ±2 ms, its own start gap measured | each punch-in's median offset and start gap | +| 3 | `live_dots` | stage 2: more than 10 dots in each punch-in, each on one of the reference's sounds, and those on tones within 50 cents of the tone. A sound's dots lie from its start, less a hop, to half the tracker's window and a hop past its end: a dot is drawn at the middle of its window, but YIN hears mostly the first half. Dots on the sweeps (a subharmonic of their top) are counted apart | dots per punch-in, on tones, on sweeps, elsewhere; how far behind the cursor they appeared, median and spread | +| 4 | `nothing_of_the_take_in_the_speakers` | no echo in any stage; an output level of exactly 0 at every look that lies wholly in one of the reference's silent gaps; Play Singing Audio the same after each take as before, and the take heard or not as it says | the second arrival; the largest output level in the gaps, and over how many looks; the largest output level; the margin of a look | +| 5 | `mic_on_input_2` | stage 2: judged only when the microphone is on input 2 alone (an input within 20 dB of the loudest carries it), and then passes when every punch-in drew more than 10 dots; otherwise Measured, "not applicable here"; a Fail when no input recorded anything | each input's peak in each punch-in; which inputs carry the microphone | +| 7 | `record_from_a_position` | stage 3: placed within ±2 ms; outside the selection the take's audio the same bit for bit, and its pitch and notes beyond ±0.25 s unchanged | the range, offsets, audio, pitch and notes outside | +| 9 | `stop_on_a_long_song` | stage 1: each punch-in's time from Stop to its pitch merged under half the whole song's analysis time, and the take's pitch beyond ±0.25 s of each range unchanged, which shows only the range was analysed | the whole song's analysis, and each punch-in's time and share of it | +| 10 | `the_joins` | stage 5, at J: no step in the samples (the largest first difference within ±2 ms of J at most 10 dB over the 95th percentile of the 50 ms around); the pitch track running through (no gap over one hop within ±0.5 s, no frame twice, in order); exactly one note holding J and no note beginning or ending within 0.5 s; outside the two ranges ± 0.25 s, pitch and notes unchanged. A failure names its part: "step:", "pitch:", "note:", "outside:" | offsets, the second punch-in against the first, the step in dB, the pitch across J, the notes at J and within 1 s, the nearest note edge | +| 12 | `nothing_heard_or_changed_in_the_lead_in` | stage 3: before P the take's audio the same bit for bit and its pitch and notes beyond 0.25 s unchanged; and item 4's looks that end by P, now over the take's own audio, read 0 in the silent gaps | audio, pitch and notes before P; the output in the lead-in's gaps, and the looks; the longest wait between two looks | +| 13 | `pre_roll_near_the_start` | stage 4: a lead-in no longer than the 1 s of song before P; playback from frame 0, and the cursor never before it; a countdown shown, ending at 1, and starting no higher than the lead-in, round trip and start gap (and 50 ms) rounded up; placed within ±2 ms | the lead-in, where playback started, the cursor's lowest, the countdown, offsets | +| 14 | `record_into_selection_stops_by_itself` | stages 3 and 4: what was recorded past the selection's end, from the raw recording, between 0 and 0.25 s plus one look of the take timer (100 ms) and a block; the coverage after is the coverage before plus the selection, exactly; no modal dialog from the take's start to its analysis done | seconds past the end, and allowed; the coverage after; dialogs | + +Item 10 does not judge placement: items 1 and 2 do. Item 14 works out what the take needed +from the round trip, start gap, lead-in and range itself, not with `TakeTiming`, whose +margin is part of what it checks. + +**How items 4 and 12 read the output.** A look's output level is the loudest sample handed +to the device since the look before. It is placed on the reference's timeline from the +frames received, since each callback takes in a block of input and then hands out a block +of output, and from the measured start gap; not from the clock or the output latency, since +the levels are of what was handed to the device, before its latency. Either side of a look +lies a margin of one block: the most frames received between two looks that were not held +up. A look counts only when it lies wholly in a gap where the reference is silent. + +- A take played back out shows in a gap only if the take holds something there. A real + microphone records the room; a noiseless loopback would record the reference alone, and + the tests give the fake a noise floor for that reason. +- A stall of the GUI thread (a disk, the system) makes the frames received jump, and a look + across it spans a sound and is left out. When no look at all lay in a gap, that part is + **not judged**: the message says so, with the longest wait and the margin, and the + verdict is left to the check's other parts, so it can read Pass (§10). + +**Verdicts:** Pass; Fail; Measured, numbers only (item 5 when the microphone is not on +input 2 alone); Skipped, when the run ended before the stage the check needs, or the long +song was left out (`Options::longSeconds` of 0, for tests only). + +### The report + +`DevChecks.txt` in `TONY_TEST_LOG_DIR` if that is set, else in the application data +directory (`%APPDATA%\sonic-visualiser\Tony` on Windows), written over by each run. Its +header: the date, the output and input devices, the audio drivers built in, the playback and +record latencies the device reports (frames and ms), the round trip for the run, the +scratch folder, the session saved, and why the run ended early if it did. Then each check +under "Item N", with its verdict, message and numbers, and last `Totals: N passed, N failed, +N measured, N skipped`, as the suites end. + +The run saves its session as `dev-checks.ton` in a scratch folder `dev-checks-N` in the +application data directory, so that its take files land in a folder of their own, and +leaves it open to be looked at and played. The next run removes every such folder the +session open then does not use. + +## 8. What to send back + +After a run on a new machine or device: the result page's text (select it all and copy) +and the whole `DevChecks.txt`. What the numbers decide: + +- **The round trip measured against the driver's, and where each punch-in landed:** how + wrong the driver is, and how far the offset moves from one stream start to the next (the + Unsteady and Scattered thresholds, and the restart jitter of §10). +- **Sweeps found, the input peak, the echo:** the finder's thresholds, NoSignal and Clipped. +- **The recording's rate:** whether the device runs at 44.1 kHz, which the rate fix of §10 + is for. +- **The report's header:** the drivers built in and what the device reports. +- **Items 1 and 2**, each sweep's offset and each start gap: the ±2 ms. **Item 3**, how far + the dots trail the cursor. **Items 4 and 12**, the margin, the looks in the gaps and the + longest wait: how the gap check fares with a real device's blocks. **Item 5**, each + input's peak. **Item 9**, both times. **Item 10**, the step in dB and the notes at J. + **Item 13**, the countdown. **Item 14**, the seconds past the selection's end. + +## 9. Architecture + +Following `AGENTS.md`: pure logic in `tony_core`, state in classes of its own, `MainWindow` +only wires. + +- **`tony_core`** (the core suite): `LatencyCheck` (the layouts and the generator, the + finder, `judgeTake()` and its verdicts, `punchInsFor()`, the calibration arithmetic), + `LatencyCalibration` (§5), and `TakeDiff`, the comparisons the dev checks make on a take: + audio bit-identical outside a range, events unchanged outside a range ± 0.25 s, pitch and + notes across a join, a step in the samples at a join. `TakeDiff` excuses nothing outside + the range it is given: the splice's fades lie inside the range (`weightAt()` in + `TakeAudio.cpp`). +- **`tony_app`, every build:** `AudioCheckRunner` (a run of the check: its plan, its steps, + its result), `CalibrateAudioDialog` (a view over the runner: it starts and cancels the + runs it started, shows their results and hands one to `storeMeasuredLatency()`; it + touches no model, layer or take), and in `MainWindow` the menu, `roundTripAt()`, the + record of what the last take was placed with (`m_takeLatency`) and the override for the + check's takes. +- **`main/dev/`, development builds only:** `DevChecks` (the stages, the checks, the + report), which drives the runner, and `TakeObserver`, which it owns and starts for each + punch-in. + +What it rests on: + +- **No nested event loop.** The runner and the dev checks are driven by polling timers and + the runner's `finished()`: they run in the live window, which can be closed, or its + session replaced, at any moment. Each says `finished()` once, however the run ends: + never from inside `start()` (whatever goes wrong is found at the first poll), but at once + from inside `cancel()`, so that the caller knows the run is over when it returns. + `progress()` never comes from inside either. A step that shows a dialog runs an event + loop of its own in which the timer goes on firing, so `poll()` does not re-enter. +- **Ownership.** `MainWindow` owns the runner (made with the window), the dialog (made the + first time it is asked for) and, in development builds, the dev checks (made with the + window). `~MainWindow` deletes the dialog, then the dev checks, then the runner, before + anything they read; the dev checks and the runner then end a run without a word and + without the Stop path, leaving a take in progress to the window. `closeSession()` tells + the runner first, then the dev checks: a run that is itself replacing the session (the + runner opening a reference, the dev checks' reopen) carries on. +- **Friends.** The runner, the dev checks and the observer are friends of `MainWindow`: + they drive the take path and read the take's state. The development ones' `friend` + lines are inside the `#ifdef`, so a release build has no such surface. +- **The check's takes never go through the toolbar.** Record into Selection, Play Reference + While Recording and Pre-roll write QSettings when toggled, so the runner sets an override + (`m_audioCheckTakes`, with the plan's pre-roll and round trip) that `record()`, + `recordingStarted()` and `wantedPreRollFrames()` read, and clears it when the take stops + or the run ends. +- **Record during a check.** The Record action goes to `recordPressed()`, which ignores a + press while a check runs: `record()` itself cannot tell a press from the runner's calls or + `pollTakeProgress()`'s, and a press would stop the check's take or record one of the + user's into the check's session. +- **The check's playback** is set on the play parameters of the session's own models, + never through `Analyser::setAudible()` and the like, which write settings every session + reads ([architecture.md](architecture.md)). The toolbar's reference level control + answers a gain between its notches by moving to the nearest and saying so, and the window + then sets that gain through `Analyser::setGain()` and `setAudible()`: the runner moves the + control first, under a `QSignalBlocker`. +- **The levels have one reader each.** `getOutputLevels()` and `getInputLevels()` give the + peak since the previous call and reset it. While recording, lead-in included, + `ViewManager::checkPlayStatus()` reads the input levels only, for the meter's signal; the + observer then reads the output levels, and only then, and takes the input levels from + that signal. +- **A take's audio is read from its file**, never from its model (§2). + +## 10. Known limitations and open points + +The open points are also in [open-points.md](open-points.md), briefly; this section has +the reasons. + +**Next: a lower-latency driver** (the user's decision, 2026-09-26, from the restart jitter +below). In order: + +1. **The device-rate mismatch.** A take recorded at 48 kHz is placed frame for frame into + a 44.1 kHz session. The robust fix converts when the take is spliced, whatever the + device's rate; the other way, the record target asking for the session's rate + (`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork), fails where + the device runs only at its mixer's rate. It comes first because WASAPI opens at the + Windows mixer's rate, usually 48 kHz. The button then shows the fix working. +2. **A `bqaudioio` fork**, `jhhr/bqaudioio` (created 2026-09-26; the remote `jhhr` in + `bqaudioio/`). Nothing uses it yet: `repoint-project.json` still takes bqaudioio from + sourcehut, and nothing is pinned to the fork. For choosing the host API, WASAPI's + automatic rate conversion, and a `suggestedLatency` that can be set. +3. **A driver type in Tony**: MME, DirectSound or WASAPI; the device menus list that type's + devices only (today every host API's are listed, and `getDeviceIndex()` takes the first + name that matches, which is MME's); the stored round trip kept per type. 4. **Measure** with Calibrate Audio and a dev run on each type, on the user's PC. -**MME stays the default** until such a run shows WASAPI (or another type) better. - -## 8. Risks - -- **Restart jitter.** If the spread across punch-ins is ≥ 10 ms, no stored figure fits - every take. *Measured on the user's PC (MME):* about 13 ms between two takes, 5–20 ms - over three calibrations, 0.3 ms within a take (§7). Considered: tuning items 1 and 2 - to ±15 ms, and keeping the stream running between takes (an svapp change, which would - make one session's takes agree with each other but not with the reference). Chosen: a - lower-latency driver, the next project (§7). Until then items 1 and 2 keep ±2 ms and - fail on MME, which is the true reading. -- **Windows enhancements or echo cancellation** can remove the sweeps. Tony cannot ask for - raw capture: bqaudioio is upstream and MME has no raw mode. The NoSignal and Fading - verdicts point to *Sound settings ▸ device ▸ Audio enhancements: Off*. -- **Thresholds are guesses** until real runs exist. Every check reports its numbers as - well as pass or fail, and the report file is what tunes them. -- **The live app during a run.** No nested event loops: the runner and the dev checks - are driven by timers, and the dialog is not modal, so the window can be used, and - closed, while they run. Cancel, and a session closed mid-run, must always leave a - clean state. `closeSession()` already stops take polling. -- **Loudness.** The sweeps are −12 dBFS with earcups off the ears; the dialog says so - before starting. *Found in B1:* not as played. Tony normalises every audio file to - full scale as it reads it (`Preferences::setNormaliseAudio(true)`), so the reference - plays at 0 dBFS, in the left channel only (§11). *Since B3* the check's session - plays it centred, with a play gain that brings it back to −12 dBFS, and its - sonification silent. Other sessions play as before. -- **Cursor versus dots.** The cursor subtracts the *reported* output latency. With a - measured round trip the dots move to the right place and may sit off the cursor. Item - 8's number will show how much. Fixing it needs the round trip split between output - and input, which is not in this plan. -- **Release builds must stay clean.** A `release`-type build must compile with no - `main/dev/` file, so build one before calling step 4 done. - -## 9. Later candidates for the button - -- **A noise gate for live dots.** `RealtimePitchTracker` has no level gate (YIN is - scale-free), so room noise can make dots. The measured noise floor could set one. -- **A quick re-measure** after a Bluetooth reconnect, without a test session. -- **Items 18 and 6**, as app-suite tests: synthetic mouse events, and a fake device that - fails to open. - -## 10. Decisions +**MME stays the default** until such a run shows WASAPI, or another type, better. + +**Restart jitter on MME.** Every take restarts the stream. On the user's PC the offset +between input and output moved by about 13 ms between two takes and by 5 to 20 ms over three +calibrations, while the sweeps within one take agreed to 0.3 ms; the start gap, measured at +0 frames both times, does not see it. No one stored figure then places every take. +Considered: widening items 1 and 2 to ±15 ms; keeping the stream running between takes (an +svapp change, which would make one session's takes agree with each other but not with the +reference). Chosen: the driver project. Until then items 1 and 2 keep ±2 ms and fail on MME, +the true reading, and so do items 7 and 13 whenever their punch-in lands more than 2 ms off. +Item 10, by reading the code, does not fail for it (the join is a dip, below). + +**For the user to decide:** + +- **"Not judged" reads Pass.** Items 4 and 12 pass when their gap part could not be judged + (no look lay in a gap), with a message that says so; the Totals line then overstates. + Whether that should count otherwise (Measured, say) is not decided. +- **The thresholds**, all starting values, from the report files of real runs (§8). + +**Weak spots:** + +- **Release builds must stay clean**, with no `main/dev/` code: checked 2026-09-26 (§6). + Build a `release` directory again after a change to how the dev checks are wired in. +- The runner allows the reference's analysis 60 s (`kReferenceTimeoutMs`), and the long + song's took 10.4 s on the cloud machine: a PC six times slower ends the dev run at its + first stage, "The test reference was not analysed in time". +- **The countdown** at P = 1 s with a 3 s pre-roll reads 1, 2, 1: `record()` shows it + before the round trip is known, and from the deferred start on it counts the lead-in and + the round trip still to come, the time until what is sung is kept. Harmless; item 13 + allows it. +- **Live dots trail the cursor by about the round trip.** During a take the cursor is where + playback started plus what has been recorded (the svgui fork's `ViewManager`), which runs + ahead of what is heard, while each dot is drawn where its sound belongs on the reference. + On the fake they trail it by the round trip and about 40 ms more. Item 3 reports it; the + manual checklist asks how it looks. Not changed. +- **A join is a 10 ms dip.** Two punch-ins that meet each fade over 5 ms against the + silence the file held there, not into each other ([takes.md](takes.md#known-limitations)). + Item 10's step reads the dip as no step, and so, by reading the code, cannot see a jump + in phase either: two punch-ins that a restart placed differently show only in its number + "second punch-in against the first", not in its verdict. Not seen on a device. (A comment + in `DevChecks::joinsCheck()` still expects them to show in the step and the pitch.) +- Item 3 is no check of placement: dots 20 ms early (the round trip 20 ms off) pass or fail + with where the tracker's hops fall, and the test allows either. Items 1 and 2 judge + placement. +- Items 3 and 5 judge stage 2's punch-ins only. Item 14 cannot see an overwrite question: + `record()` would ask it before the observer starts. The check's takes record into a + selection, where none is asked, so "no question" is watched for, not arranged. +- The calibration's result page does not give the microphone's channel or the noise floor: + nothing in the calibration measures them (item 5 of the dev run finds the channel). +- Windows' audio enhancements, echo cancellation or noise suppression can take the sweeps + out, and Tony cannot ask for raw capture (bqaudioio is upstream, and MME has no raw + mode): NoSignal and Fading tell the user to turn them off. +- Not in the dev run, of what the retired `test-tony-device` did: a take with no lead-in + and not into a selection, stopped by hand; "no dialog" over every take (item 14 watches + stages 3 and 4); with no device at all, the "Couldn't open audio device" warning once for + every file opened (a dated note in the manual checklist). +- Every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()", from + svapp's `AudioCallbackRecordTarget` ([forks.md](forks.md), known defects). Harmless here. +- A dev run's session refers to its reference in the application data directory, which the + next check removes unless that session is open, and the next dev run removes its scratch + folder: only the latest run can be looked at. + +**Later candidates:** + +- A noise gate for live dots: `RealtimePitchTracker` has none (YIN is scale-free), so room + noise can make dots. A measured noise floor could set one. +- A quick re-measure after a Bluetooth reconnect, without a test session. +- The microphone's channel and the noise floor on the calibration's result page. +- A getter for the rate the record target records at (svapp fork), so that the menu line + and Forget know the device's rate before the first take (§5). + +## 11. Tests + +- **Core** (`test-tony-core`): `TestLatencyCheck`: the generator; the finder on takes made + by shifting the reference (every shift across ±0.75 s and a fractional one), with noise at + 0 and −10 dB SNR, band limits, the polarity inverted, a reflection 7 ms late and + stronger than the direct sound (5.5 dB: the direct sound is taken; 7 dB: the reflection), + and silence; the verdicts, with the 48 kHz case built as punch-ins displaced by + P·(1 − 44100/48000) (a take stretched by 8.8 % finds nothing); the arithmetic's sign; + `punchInsFor()`; long layouts of other lengths. `TestLatencyCalibration`: a figure + stored and read back under a device name holding `/` and `ä`, keys kept apart, the key + as the Preferences give it, staleness either side of the tolerance, the round trip in use + and its frames at the recording's rate. `TestTakeDiff`: each comparison passing and + failing on purpose, and the real `splice()` and `erase()` through files, whose fades lie + inside the range. +- **The round trip in the take path** (`TestRecordWorkflow`, `latency_*`): a stored figure + lines a take up where the reported pair does not, a stale one is ignored, and the + reported pair is converted at the device's rate, also when the device was opened before + any file. +- **`TestAudioCheck`** (`test-tony-app`): the check on the loopback fake, whose device + reports 2 × 4096 frames out and 4096 in while the true round trip is 123 frames longer. + The check measures the true one, and once it is stored a second check finds its takes + in place; a 48 kHz fake is a rate mismatch; cancel, a closed session, no device, a device + that never calls back; the check's playback, and a session opened after it playing as + before; the user's toggles and their settings untouched; plans refused; the plan's round + trip and pre-roll; keeping the session; replacing a check's own session without asking, + and asking before the user's; Record ignored; the menu and the dialog. Its runs are two + punch-ins of two sweeps on the first 11 s of the calibration reference, about 13 s each. +- **`TestDevChecks`** (`test-tony-dev`, development builds only): whole dev runs on the + loopback fake. Passing, with the fake's true round trip and a long song of 60 s (240 s + would add most of a minute to every passing run). Failing: the round trip 20 ms off (items + 1, 2, 7 and 13; item 10 still passes, both punch-ins moved alike); an echo tap, with the + microphone on input 2 (item 4 fails, item 5 judged on input 2); the take made audible + during the re-record's lead-in (items 4 and 12, also through a stall); a stall of the GUI + thread in the lead-in (item 12 still judged, or not judged, never failed). Also cancel, a + closed session, the dev checks deleted during a run, the scratch folders, and the dialog + carrying on into them. Parts that no fault run makes fail (among them item 3's dots on + the tones, item 9, and item 10's step) were seen failing with the code broken for a + moment. About 4 minutes. + +How the tests are built, and what to watch for: [testing.md](testing.md), "The audio check +and the dev checks". + +**What a passing run reads on the fake** (Linux, 2026-09-26), to set a real report against: + +- every sweep at 0 frames, since the fake's delay is exactly its round trip; its reported + pair is 2.8 ms short, so a run that ignored the calibrated figure would fail item 1; +- the dots trail the cursor by the round trip and about 40 ms more (322 ms at a 281 ms round + trip), their spread 40 to 230 ms; +- item 9: a 60 s song analysed in 2.9 to 3.0 s, its punch-ins merged in 0.59 to 0.70 s (20 + to 24 %); a 240 s song in 10.4 s, its punch-ins in 0.63 and 0.75 s (6 to 7 %); +- item 10: the step reads −8.3 dB, the dip; the largest pitch gap is one hop; +- item 12: 57 looks in the lead-in's gaps; item 14: 0.30 to 0.35 s past the selection's end. + +## 12. Decisions | Question | Decision | | --- | --- | -| Where the check runs | In a session of its own, opened from a generated WAV; the user is asked to save first | -| How the round trip is measured | Through ordinary takes (§2), not a separate audio IO | -| When a measured figure is used | After **Use this latency**; a dev run uses the new figure for itself only | -| Dev mode | Any build type that does not start with `release` (`TONY_DEV_CHECKS`) | -| Form of a dev check | A function returning a plain `CheckResult`, not a QtTest function | -| Checkpoint after B3 | The user runs it on Windows when they can; C0 onwards does not wait | -| Commits | The lead commits each phase after review and pushes `feat/calibrateaudiotests` | -| `test-tony-device` (from `default`) | Its checks move into the dev run, and it is retired (C3) | -| Restart jitter on MME (about 13 ms) | Not tuned away: a lower-latency driver project after this branch; MME the default until a run shows another type better | -| Where the dev checks' tests run | `test-tony-dev`, a third executable in dev builds, run when a change touches the take path, the audio check or the dev checks | - -## 11. Facts checked in the code - -Checked on 2026-09-25, so that phases do not re-derive them. - -- **The latency today.** In `MainWindow::recordingStarted()`'s deferred lambda: L = - `computeRecordingLatency(getTargetPlayLatency(), getSystemRecordLatency())` + start - gap. It applies only with Play Reference While Recording on; otherwise L = 0. The - start gap comes from the play-start callback set in `MainWindow`'s constructor: - `getFramesReceived() − blockFrames` on the first output block with audio. - `refineRecordingLatency()` and `currentRecordingLatency()` swap the estimate for the - measurement. *Found in B1:* `getTargetPlayLatency()` counts frames of the session's - rate (bqaudioio's `ResamplerWrapper` converts it), `getSystemRecordLatency()` the - device's, and L is taken off the recording, in the device's. They differ only when - the device is not at 44.1 kHz. *Found in B2:* the wrapper converts only if the - session had a rate when the device was opened. A device chosen before any file is - opened gets its figure passed on in its own frames, and the play source is told a - device rate of 0. So the output latency counts frames at the play source's - `getDeviceSampleRate()`, or at the device's rate when that is 0. - *Since B2* the round trip is the stored figure (`LatencyCalibration`) when one is - valid, otherwise the reported pair, each converted to seconds at its own rate. It - is then turned into recording frames. `computeRecordingLatency()` is no longer - called. -- **Where the reference is heard** (found in B1). `Analyser` pans the reference hard - left and its pitch and notes sonification hard right, so only the left earcup - carries the sweeps; the right one carries the synth. *Since B3* the check's own - session sets its models' play parameters instead: the reference centred at - −12 dBFS, the sonification muted. -- **bqaudioio `PortAudioIO`** (upstream, not a fork): +| Where the check runs | In a session of its own, opened from a generated WAV; the user is asked to save first, unless the session open is a check's own | +| How the round trip is measured | Through ordinary takes, not a separate audio IO | +| When a measured figure is used | After **Use this latency**, for the devices and rate it was measured on; a dev run uses the new figure for itself only | +| Dev mode | Any build type that does not start with `release` (`TONY_DEV_CHECKS`); no runtime flag | +| Form of a dev check | A function returning a plain `CheckResult`, not a QtTest function: the suite has to show that each check can fail, which a QVERIFY inside another test cannot | +| How runs are driven | Polling timers and signals, never a nested event loop | +| `test-tony-device` (from `default`) | Its checks moved into the dev run, and it is retired | +| Where the dev checks' tests run | `test-tony-dev`, a third executable in development builds, run when a change touches what the checks drive (`AGENTS.md`) | +| Restart jitter on MME (about 13 ms) | Not tuned away: a lower-latency driver is the next project; MME stays the default until a run shows another type better | +| The notes merge at a join inside a held note | Fixed on this branch: one note across the join ([takes.md](takes.md)) | + +## 13. Facts checked in the code + +So that later work does not derive them again. + +- **bqaudioio's `PortAudioIO`** (upstream, not a fork): - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; - - input goes to the record target **before** output is asked for, in the same - callback; + - input goes to the record target **before** output is asked for, in the same callback; - `suspend()`/`resume()` are `Pa_StopStream`/`Pa_StartStream`. `MainWindowBase::stop()` suspends and `record()` resumes, so **every take restarts the stream**; - it exposes no device names and ignores PortAudio's callback time info. - **Device rate.** - `AudioCallbackPlaySource::getApplicationSampleRate()` and `AudioCallbackRecordTarget::getApplicationSampleRate()` both return 0, so the device - opens at PortAudio's default rate. - - `ResamplerWrapper` resamples the play source to it. The record target records at it. - - `MainWindow` sets `Preferences::setFixedSampleRate(44100)`. - - `TakeAudio::splice()` writes a take's first recording at the recording's rate - without converting positions. Later recordings are refused only when their rate - differs from the take file's. - - PortAudio's MME default rate is the first of {44100, 48000, …} that the device - accepts. + opens at the output device's default rate. PortAudio's MME default is the first of + 44100, 48000, … that the device accepts. + - `ResamplerWrapper` resamples the play source to it; the record target records at it. + `MainWindow` sets `Preferences::setFixedSampleRate(44100)`. + - `TakeAudio::splice()` writes a take's first recording at the recording's rate without + converting positions, and refuses a later one whose rate differs from the take file's. +- **The reported latencies** count frames at two rates: `getTargetPlayLatency()` at the + play source's `getDeviceSampleRate()`, which is the session's when bqaudioio's + `ResamplerWrapper` converted it, but the device's own when a device was opened before + any file (the wrapper then passes the figure through and tells the play source 0); + `getSystemRecordLatency()` at the device's. They differ only when the device is not at + 44.1 kHz. - **Device choice.** `getDeviceIndex()` takes the first PortAudio device with the given name, across host APIs. MME names are cut to 31 characters. -- **Settings the check must not write.** The toggles `m_recordIntoSelection` +- **Settings a check must not write:** the toggles `m_recordIntoSelection` (`MainWindow/recordintoselection`), `m_playRefWhileRecording` and `m_preRoll` write - QSettings when toggled. `wantedPreRollFrames()` reads `MainWindow/prerollseconds`. -- **Opening a reference.** - - The tests use `openPath(path, MainWindow::ReplaceSession)` after - `discardModifications()`. - - `checkSaveModified()` is what asks the user to save. - - The reference is analysed when `Analyser::getInitialAnalysisCompletion() >= 100` and - the layers exist. See `analysed()` in `TestRecordWorkflow.h`. - - *Found in C1a:* with "normalise audio" on, a model's `getLocalFilename()` is the - reader's decoded copy in the temporary directory, not the file opened. The file - opened is `getLocation()` resolved through `FileSource` - (`AudioCheckRunner::mainModelFile()`). -- **The take after Stop.** - - Its audio is the model `analyser2()->getMainModelId()`, and its file is - `m_takes->getAudioPath()`. *Found in B1:* the model is normalised to full scale - and resampled to 44.1 kHz as it is read, so every take read from it is Clipped; - the check reads the file. - - Coverage is `m_takes->getCoverage().getRanges()`. - - The take is analysed when `analysed(analyser2())` holds. -- **Levels.** `getOutputLevels()` and `getInputLevels()`, on the play source and record - target, return per-channel peaks since the last call. *Found in C1b:* while recording, - lead-in included, `ViewManager::checkPlayStatus()` reads the input levels only, so the - observer is then the output levels' one reader. They are of what is handed to the - device, before its output latency. `FakeAudioIO` reports levels only with - `reportLevels`. The play source's `getTargetBlockSize()` is always its default 1024: - bqaudioio's `ResamplerWrapper` does not pass the device's block on. -- **Fake device.** `FakeAudioIO::Config::loopback` adds the output to the input - `inputDelay` frames late; `TestAudioCheck` uses it. The reported latencies are - independent of the real delay. `TestMainWindow::createAudioIO()` installs the fake. - Its output is the mean of the channels, so a hard-left reference loops back at - half level; the check's, centred since B3, at the level it is played at. -- **Menus.** The Playback menu is built in `MainWindow::setupToolbars()` - (`m_playbackMenu`). The audio device submenus are there too. -- **Build types.** `build.bat` uses `debugoptimized`; `meson.build` defaults to - `release`, which the deploy scripts use. `meson.build` already switches on - `buildtype.startswith('release')` for `WANT_TIMING` / `NO_TIMING`. -- **Qt Test** is already in `qt_dep`'s modules, so every target links it. + QSettings when toggled; `wantedPreRollFrames()` reads `MainWindow/prerollseconds`. +- **Opening a file.** With "normalise audio" on, a model's `getLocalFilename()` is the + reader's decoded copy in the temporary directory, not the file opened; the file opened is + `getLocation()` resolved through `FileSource` (`AudioCheckRunner::mainModelFile()`). +- **When analysis is done** (`AudioCheckRunner::analysing()`): no transform running, no + ranged run, and the first analysis complete if there is a pitch layer. With automatic + analysis off the reference never gets one, and the check needs none. +- **After a reopen** a take's pitch and notes are restored from the session, not analysed. +- **The take after Stop:** its model is `analyser2()->getMainModelId()`, normalised and + resampled as it is read; its file is `m_takes->getAudioPath()`; its coverage + `m_takes->getCoverage()`. +- **Levels** are of what is handed to the device, before its output latency. The play + source's `getTargetBlockSize()` is always its default, 1024: `ResamplerWrapper` does not + pass the device's block on, so the observer bounds a block by the frames received. +- **The fake device.** `FakeAudioIO::Config::loopback` adds the output, the mean of its + channels, to the input `inputDelay` frames late; the latencies it reports are independent + of that delay. It reports levels only with `reportLevels`. + +## 14. The user's runs + +- **2026-09-26**, Windows, MME, wired microphone and headphones, one earcup to the + microphone: Calibrate Audio measured 301 and 295 ms, verdict Unsteady both times (5 to + 15 ms); with the microphone between both cups, Scattered. +- **2026-09-26**, a dev run of an early build, items 1 and 2 only: round trip 303.5 ms; the + two sweeps of each punch-in agree to 0.2–0.3 ms, but punch-in 1 landed at +0.6 ms and + punch-in 2 at −12.9 ms; the measured start gap was 0 frames both times. So the finder is + precise, and what moves is the stream's offset between input and output at each restart. +- The whole dev run: not yet. diff --git a/docs/forks.md b/docs/forks.md index 0f60d1c8..3b6f155c 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -11,7 +11,11 @@ forks under `github.com/jhhr` that exist only for this Tony fork: | `svapp/` | `jhhr/svapp` `tony-customizations` | See below. | | `bqaudiostream/` | `jhhr/bqaudiostream` `master` | `` instead of `` under MinGW, needed for `-DHAVE_MEDIAFOUNDATION`. | -`pyin/` and the rest are upstream and must stay untouched. +`pyin/` and the rest are upstream and must stay untouched. `bqaudioio/` too, for now: a +fork of it, `jhhr/bqaudioio`, was created on 2026-09-26 for the lower-latency driver work +([open-points.md](open-points.md)), and the checkout has it as the remote `jhhr`, but +`repoint-project.json` still takes bqaudioio from sourcehut and nothing is pinned to the +fork. It joins the table when that work first pins it. ## Changing a fork @@ -103,7 +107,9 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` ## Known defects in the forks, not fixed - `svapp/audio/AudioCallbackRecordTarget.cpp` connects to `SIGNAL(aboutToBeDeleted())`, - which the model class does not have, so its `modelAboutToBeDeleted()` never runs. The + which the model class does not have, so its `modelAboutToBeDeleted()` never runs, and + every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()", on any + Qt: expected in test logs, not a new fault. The signal to use would be `Document::modelAboutToBeReleased(ModelId)`. Tony avoids the consequence by stopping the recording and releasing the model in a fixed order (see [recording.md](recording.md)); whether `m_model` can dangle otherwise was not looked into. diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 39dca241..a2580788 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -40,9 +40,9 @@ file, and compares the take before and after each punch-in. 4. **Playback > Calibrate Audio...**, with **Run the dev checks after calibrating** on (it is by default), then **Start**. A few minutes; leave the window alone meanwhile (Cancel stops the run). The dev checks run only after a calibration that can be used (verdict - Ok or Unsteady). No signal, or a fading one, means the microphone did not hear the - sweeps, or Windows' audio enhancements took them out. The stored latency changes only - through **Use this latency**. + Ok or Unsteady, recorded at the reference's rate, 44.1 kHz). No signal, or a fading one, + means the microphone did not hear the sweeps, or Windows' audio enhancements took them + out. The stored latency changes only through **Use this latency**. 5. The report is on the dialog's result page and in `DevChecks.txt` in Tony's application data folder (`%APPDATA%\sonic-visualiser\Tony` on Windows). The test session is saved beside it in a `dev-checks-` folder and left open, to be looked at and played. @@ -80,11 +80,11 @@ latencies the device reports, and the round trip used. Then, item by item: **What fails on MME today, and why.** Every take restarts the audio stream, and on MME the offset between input and output moves by about 13 ms from one restart to the next, while the sweeps within one take agree to 0.3 ms; the start gap does not see it. No one round -trip then places every take within ±2 ms: expect items 1 and 2 to fail, items 7 and 13 -whenever their punch-in lands more than 2 ms off, and item 10 at the join (the step and the -pitch) when its two punch-ins land apart. That is the true reading, not a fault of the -check: the remedy is a lower-latency driver, the next project -([calibrate-audio.md](calibrate-audio.md), §8). +trip then places every take within ±2 ms: expect items 1 and 2 to fail, and items 7 and 13 +whenever their punch-in lands more than 2 ms off. Item 10 shows two punch-ins that landed +apart only in its number "second punch-in against the first": the join is a 10 ms dip, which +hides a jump. That is the true reading, not a fault of the check: the remedy is a +lower-latency driver, the next project ([calibrate-audio.md](calibrate-audio.md), §10). A device that opens but delivers nothing ends the run with "The audio device delivered no input" once the take's lead-in and range and 2 s more have gone by without one frame, and diff --git a/docs/open-points.md b/docs/open-points.md index 84a68bbb..20886908 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -15,12 +15,26 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. outside the range the pane shows, and nothing scrolls to it; ±2 is in view. - Of the [manual checklist](manual-checklist.md), the device check (Calibrate Audio with the dev checks) has been run on real hardware only in part: Calibrate Audio, and a dev - run with items 1 and 2 only (the user's PC, MME, 2026-09-26). The whole dev run not yet. + run of an early build with items 1 and 2 only (the user's PC, MME, 2026-09-26). The whole + dev run not yet. Of section 2, only the looks, from cloud screenshots (2026-09-25). +- **The dev checks' "not judged" reads Pass.** Items 4 and 12 pass when no look at the + output lay in a silent gap, with a message that says so, and the report's Totals then + overstate. Should it count otherwise, as Measured say? + ([calibrate-audio.md](calibrate-audio.md), §10.) +- **The thresholds** of the sweep finder, the verdicts and the dev checks are starting + values, to be tuned from the report files of real runs (what to send back: + [calibrate-audio.md](calibrate-audio.md), §8). ## Not built -- **Calibrate Audio**: a measured round trip in place of PortAudio's reported latency, - and dev checks for the manual checklist. Planned in [calibrate-audio.md](calibrate-audio.md). +- **A lower-latency driver**, the next project (the user's decision, 2026-09-26): first + the device-rate fix (a take recorded at 48 kHz is placed frame for frame into the + 44.1 kHz session; convert when it is spliced), then a `bqaudioio` fork for choosing the + host API, WASAPI's rate conversion and a settable `suggestedLatency` (`jhhr/bqaudioio` + exists, the remote `jhhr` in `bqaudioio/`, not pinned yet: [forks.md](forks.md)), then a + driver type in Tony with the stored round trip per type, then Calibrate Audio and a dev + run on each type. MME stays the default until a run shows another better + ([calibrate-audio.md](calibrate-audio.md), §10). - Showing two takes at once, or any comparison of takes other than switching. - Singing track gain and pan are not saved in the session. - Background music is not saved in the session; it is reloaded by hand. @@ -57,3 +71,39 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. fork; the 30 s give-up of `waitForRangedAnalysis()`; `commitData()` relocating takes on Windows (the test runs elsewhere only); the two other ways `MainWindowBase::record()` can fail. +- **Closing the session during an ordinary take, then pressing Stop, hung** (seen once, + 2026-09-26, while testing the audio check; not looked into). `closeSession()` stops a + check's take through the Stop path, but not the user's own. +- **The toolbar's level controls write the settings on their own.** In a window's first + file, `updateLayerStatuses()` shows the pitch and notes gain of 0.5, which lies between + two notches; the control moves to the nearest, 0.562, emits it, and the window makes + both audible and writes that to the shared settings. +- **Play Audio's setting is overridden on load by the spectrogram's**: the spectrogram + layer is on the reference's model and shares its play parameters, and `Analyser` loads + `audible-3` after `audible-0`. + +### Calibrate Audio and the dev checks + +The reasons are in [calibrate-audio.md](calibrate-audio.md), §10. + +- **Restart jitter on MME.** Every take restarts the stream, and on the user's PC the + offset between input and output moved by about 13 ms from one start to the next. No one + round trip then places every take: the dev checks' items 1 and 2, and 7 and 13 whenever + their punch-in lands more than 2 ms off, fail on MME today. The remedy is the driver + project above. +- The runner allows the reference's analysis 60 s (`kReferenceTimeoutMs`); the 4-minute + song's took 10.4 s on the cloud machine, so a PC six times slower ends the dev run at its + first stage. +- The countdown at a punch-in 1 s into the song with a 3 s pre-roll reads 1, 2, 1: it is + shown before the round trip is known. +- Live dots trail the cursor by about the round trip (and 40 ms more on the fake): the + cursor runs with what has been recorded. +- Items 3 and 5 judge the fresh punch-ins only. Item 14 cannot see an overwrite question, + which `record()` would ask before the observer starts. +- Not in the dev run, of what the retired `test-tony-device` did: a take with no lead-in + and not into a selection, stopped by hand; "no dialog" over every take (item 14 watches + two stages). +- Every take logs "No such signal sv::WritableWaveFileModel::aboutToBeDeleted()", from the + svapp fork ([forks.md](forks.md), known defects). +- The calibration's result page does not give the microphone's channel or the noise + floor, which nothing in it measures. diff --git a/docs/recording.md b/docs/recording.md index 3af9cf0f..64b80dec 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -15,7 +15,7 @@ Symbols used throughout, all in frames on the reference's timeline | **E** | where it stops itself (`m_takeEnd`), -1 when the user presses Stop | | **R** | pre-roll lead-in actually available before P (`m_takePreRoll`), 0 without one | | **S** | where playback starts: P − R | -| **L** | recording latency: output + input latency + start gap (`m_recordingLatencyFrames`) | +| **L** | recording latency: round trip (measured, or output + input latency as reported) + start gap (`m_recordingLatencyFrames`) | The recording's own file always begins at the press of Record. What was sung in answer to the reference at P is therefore at file frame **L + R**, and the splice reads from there. @@ -53,8 +53,8 @@ the reference at P is therefore at file frame **L + R**, and the splice reads fr makes the cursor run with the reference instead of crawling from frame 0. 7. `recordStatusChanged(true)` → `recordingStarted()` fires *inside* the base call, before the model is in the document. It defers with `QTimer::singleShot(0)`: - `setupRealtimePitchLayer()`, and, if Play Reference While Recording is on, the latency - estimate and `m_playSource->play(S)`. + `setupRealtimePitchLayer()`, and, if Play Reference While Recording is on (or the take + is the audio check's, below), the latency estimate and `m_playSource->play(S)`. 8. `modelAdded()` sees `m_recordingAsSingingTrack`, stores `m_currentRecordingModelId` and returns. The recording is raw material, not the singing track; no analyser is made. 9. After the base call: if `isRecording()` is false (no device, device busy) the take @@ -100,10 +100,25 @@ dot model outlives the take. ## Latency -- **Estimate** (GUI thread, in the deferred lambda): - `computeRecordingLatency(getTargetPlayLatency(), getSystemRecordLatency())` plus - `m_recordTarget->getFramesReceived()` just before `play()` — the *start gap*, the part - of the recording made before the reference began to play. +L is the **round trip** plus the **start gap**, both in frames of the recording. + +- **The round trip** (GUI thread, in the deferred lambda): `roundTripAt()` gives the figure + Calibrate Audio measured and the user kept for these devices and the recording's rate, + unless it is stale, and otherwise the sum of the output and input latency the device + reports ([calibrate-audio.md](calibrate-audio.md), §5). It is worked out in seconds and + then turned into frames of the recording, whose rate its model gives, because the two + reported latencies count frames at different rates: `getTargetPlayLatency()` at the play + source's `getDeviceSampleRate()` (the session's, when bqaudioio's `ResamplerWrapper` + converted it; the device's own when the device was opened before any file, when the + wrapper passes the figure through and tells the play source 0), and + `getSystemRecordLatency()` at the device's. They differ only when the device is not at + 44.1 kHz. `computeRecordingLatency()` is no longer used here; the tests keep it as the + reported sum to compare with. +- A run of the audio check may bring a round trip of its own for its takes (the dev checks, + with the one the calibration before them measured). Nothing is stored, and the Playback + menu goes on describing the window's own figure. +- **Start gap estimate** (same place): `m_recordTarget->getFramesReceived()` just before + `play()`, the part of the recording made before the reference began to play. - **Measurement** (audio callback): the lambda given to `m_playSource->setPlayStartCallback()` runs with the first block after `play()` and stores `getFramesReceived() − blockFrames` in an atomic. Drivers deliver a block's input @@ -120,7 +135,13 @@ dot model outlives the take. `setStartFrame(-L)` on the model is the old route; `TestLatencyShift` and `shift_aligns_onset` still cover it, and the svapp fork still restores the `start` attribute so that older `.ton` files open right. -- L is per take, not per device: each recording measures its start gap afresh. +- L is per take, not per device: each recording measures its start gap afresh. The round + trip is per device and rate, but every take restarts the stream, and on MME the offset + between input and output moves by about 13 ms from one start to the next, which the start + gap does not see ([calibrate-audio.md](calibrate-audio.md), §10). +- `m_takeLatency` keeps what the last take was placed with: the round trip, whether it was + measured, the reported pair in seconds, the recording's rate, and the start gap and + whether it was measured. The audio check reads it for each of its takes. ## Pre-roll and Record into Selection @@ -146,6 +167,21 @@ dot model outlives the take. written to the status bar from a timer of your own will be overwritten before it can be read. +## The audio check's takes + +Calibrate Audio and the dev checks ([calibrate-audio.md](calibrate-audio.md)) record +through this same path, so that what they measure is what a take does. Their takes record +into the selection the runner makes, play the reference and have a lead-in of the run's +own, whatever the toolbar says: the three toggles write QSettings when they are toggled, so +the runner never touches them. It sets an override instead (`m_audioCheckTakes`, with the +run's pre-roll and round trip), which `record()`, the deferred lambda and +`wantedPreRollFrames()` read, and clears it when the take stops. + +Record's action goes to `recordPressed()`, which ignores a press while a check runs (the +button is greyed then as well). The guard is not in `record()`: the runner and +`pollTakeProgress()` start and stop the check's takes through `record()`, and it cannot tell +their calls from a press. + ## The live tracker `RealtimePitchTracker::run()`: read a 2048-frame window of the **mixdown** diff --git a/docs/takes.md b/docs/takes.md index bfb8fd5a..eeea278b 100644 --- a/docs/takes.md +++ b/docs/takes.md @@ -277,5 +277,11 @@ Things to know, none of which stops the feature being used. See also user gets a dialog naming the file. - Playing a wave model with a **positive** start frame plays up to a block early and without an edge fade. This design avoids it: a take's file always starts at frame 0. -- A recording device whose sample rate differs from the reference's is unexercised; - `TakeAudio` refuses the splice. +- A recording device whose sample rate differs from the reference's: a take's first + recording is written at the device's rate and placed frame for frame, so at 48 kHz it + lands early by 8 % of its position; a later recording at another rate than the take + file's is refused. Calibrate Audio names the mismatch; the fix is planned + ([calibrate-audio.md](calibrate-audio.md), §10). +- Two recordings that meet at a frame J each fade over 5 ms against what the take held + there, not into each other: where that was silence, the join is a 10 ms dip. The dev + checks' join check reads it as no step, and a pitch gap of about one hop. diff --git a/docs/testing.md b/docs/testing.md index ba8f015b..b67c55e1 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -6,9 +6,9 @@ commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestModelChangeThrottle` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks` | about 5 minutes (measured 2026-09-25 on Linux), nearly all of it `TestRecordWorkflow` and `TestUiChecks`: takes are recorded in real time | -| `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about a minute and growing: each test records several takes in real time | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`, `TestModelChangeThrottle` | seconds | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestRecordWorkflow`, `TestUiChecks`, `TestAudioCheck` | 8 to 9 minutes (measured 2026-09-26 on Linux), nearly all of it `TestRecordWorkflow`, `TestUiChecks` and `TestAudioCheck` (about 2 minutes): takes are recorded in real time | +| `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about 4 minutes (2026-09-26, Linux): each test records a dev run's takes, or part of them, in real time | `meson test` / `build.bat test` runs these three (`test-tony-dev` where it is built) plus four svcore suites. No suite uses the @@ -18,15 +18,17 @@ development checks (section 1 of the [manual checklist](manual-checklist.md)). it when a change touches what the development checks drive (see [AGENTS.md](../AGENTS.md)). -- The `tony-app` meson test has `timeout: 900`; the suite took about 277 s unloaded when - that was set. Every workflow test adds real time, so if the suite comes near it, raise it - in `meson.build`: `meson test` reports a timeout even when every test passes. Running the - executable by hand has no timeout. -- `main()` of the app suite replaces `VAMP_PATH` with the executable's directory, so an - installed pYIN is never the one tested; the meson test `depends:` on `pyin_plugin` - because nothing else builds `pyin.dll`. Build `pyin.dll` too when running by hand after - a clean. -- Both mains set the organisation/application names to `tony-tests` / `test-tony-*` and +- The `tony-app` and `tony-dev` meson tests have `timeout: 900`; the app suite took about + 277 s unloaded when that was set, and about 550 s on Linux on 2026-09-26. Every workflow + test adds real time, so if a suite comes near it, raise it in `meson.build`: `meson test` + reports a timeout even when every test passes. Running the executable by hand has no + timeout. +- `main()` of the app and dev suites replaces `VAMP_PATH` with the executable's directory, + so an installed pYIN is never the one tested; their meson tests `depends:` on + `pyin_plugin` because nothing else builds `pyin.dll`. Build `pyin.dll` (`pyin.so` on + Linux) too when running by hand after a clean: without it every test that waits for an + analysis hangs until QtTest's five-minute watchdog aborts the run. +- The mains set the organisation/application names to `tony-tests` / `test-tony-*` and every suite works in a `QTemporaryDir`, so the user's QSettings and record directory are never touched. - `Tony.exe` links both libraries with `link_whole:`. A new source file that is in neither @@ -36,7 +38,9 @@ it when a change touches what the development checks drive (see Suites are header-only classes (`TestX.h`). A new suite needs: the header, an `#include` and a `runSuite()` block in `tony-core-test.cpp` or `tony-app-test.cpp`, and the header in -the matching `*_test_moc_files` list in `meson.build`. A new test function in an existing +the matching `*_test_moc_files` list in `meson.build`. `tony-dev-test.cpp` runs +`TestDevChecks` alone, and its moc list is inside meson's `if dev_checks` with the +`TONY_DEV_CHECKS` define for moc. A new test function in an existing suite needs nothing but itself (a private slot). **Every private slot runs as a test**, so helpers must not be slots; connect to lambdas instead. For access to private statics use `friend class TestX;`, as `RealtimePitchTracker.h` does. @@ -49,7 +53,8 @@ helpers must not be slots; connect to lambdas instead. For access to private sta - Test function names on the command line are passed to **every** suite in the executable. The ones that do not have the function report it as unknown and fail, so the exit status of a run with names is always 1. Only a run with no names has a - meaningful exit status. + meaningful exit status; `test-tony-dev` has one suite, so there a run with names has one + too. - `QT_QPA_PLATFORM=offscreen` is set by `main()` when not given. ## Design principles @@ -58,7 +63,8 @@ helpers must not be slots; connect to lambdas instead. For access to private sta (`LatencyUtils.h`, `TakeTiming` — core suite) and, separately, as "the application applies the number it was given" (app suite, with `FakeAudioIO` reporting latencies chosen by the test and delaying its input by exactly that much). The real figure of a real device is - for the [manual checklist](manual-checklist.md). + measured in the app, by Calibrate Audio ([manual checklist](manual-checklist.md), + section 1). - **Pure logic goes in `tony_core`** so that it can have many cheap tests. The app suite is for order-of-events and ownership: what is in the document, the pane, the play source and the undo history after a workflow. @@ -72,13 +78,20 @@ helpers must not be slots; connect to lambdas instead. For access to private sta callback in real time, input first and then output, as PortAudio and JACK do. `Config` sets rate, block size, reported latencies, a programmed mono input, its delay, and whether the input clock starts at the first audible output sample ("a singer exactly on - time"), `loopback` (the output fed back into the input, as speakers into a microphone), - and `inputChannel` (the input on one channel only, as a microphone on input 2). It - captures the output, so tests can assert what reached the speakers. -- `TestMainWindow` (`TestMainWindow.h`, shared by the three suites that drive a window): - subclass of `MainWindow` that exposes protected operations as `doRecord()`, - `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, installs the fake device - through `createAudioIO()` (or the real one, `setUseRealDevice()`), and **answers dialogs + time"), `loopback` (the output, the mean of its channels, fed back into the input + `inputDelay` frames late, as speakers into a microphone), `echoDelay` / `echoGain` (a + second arrival of the loopback, as an input played back out and heard again), + `inputChannel` (the input on one channel only, as a microphone on input 2), + `reportLevels` (the peaks of each block, as `PortAudioIO` reports them for the meters) + and `neverCallsBack` (a device that opens and then delivers nothing). It captures the + output, so tests can assert what reached the speakers. +- `TestMainWindow` (`TestMainWindow.h`, shared by the four suites that drive a window: + `TestRecordWorkflow`, `TestUiChecks` and `TestAudioCheck` in `test-tony-app`, + `TestDevChecks` in `test-tony-dev`): subclass of `MainWindow` that exposes protected + operations as `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, + and the audio check's parts (`audioCheck()`, `devChecks()`, `takeLatency()`, the + Playback menu's actions). It installs the fake device through `createAudioIO()`, or no + device at all when made with `installDevice` false, and **answers dialogs through virtual seams**: `confirmRecordingOverTake()`, `confirmDeleteTake()`, `askForTakeName()`, each with a `set...Answer()` and a counter of questions asked. Anything new that asks the user needs such a virtual. `setRecordOverAskedInDialog()` @@ -116,6 +129,9 @@ point is that something survived: the same layer and model objects before and af on the test thread. - In `TestMainWindow` override only `createAudioIO()`: `~MainWindowBase` calls `deleteAudioIO()` non-virtually. +- The app and dev mains draw text without sub-pixel anti-aliasing. Ubuntu's fontconfig asks + for it and Qt 6.4 follows it: the scale's labels then have orange fringes, which + `TestUiChecks` takes for live dots. A new main that shows a window needs the same. - With `FakeAudioIO`'s `inputFollowsPlayback`, `inputDelay` must be at least one block. The device delivers about two blocks before the application sets its recording flag; `inputIsKept` counts only input that was kept, which exact start-gap tests need. @@ -215,6 +231,67 @@ it draws is judged by pixels: With `TONY_TEST_SHOT_DIR` set, the suite saves the images it judged, and some of the whole window, as `-.png`, for the [manual checklist](manual-checklist.md)'s look. +## The audio check and the dev checks (`TestAudioCheck`, `TestDevChecks`) + +What they cover is in [calibrate-audio.md](calibrate-audio.md), section 11. Both fixtures +follow `TestRecordWorkflow`'s (a `TestMainWindow`, the dialog watchdog, the user's toggles +reset in `init()`), and `TestDevChecks`' is a copy of `TestAudioCheck`'s, not shared: each +class keeps its own. `cleanup()` also removes any round trip a test stored, which would +place the next test's takes. How they are built, and what to keep in mind when adding to +them: + +- **The loopback fake.** Both record through `FakeAudioIO` with `loopback` on (their + `loopback()`). The device reports 2 × 4096 frames out and 4096 in, and the true round + trip (`inputDelay`) is 123 frames longer: a check that works measures the true one, and + with it every sweep lands at 0 frames, while a run placed with the reported pair lands + 2.8 ms off. `TestDevChecks`' loopback also has `reportLevels` on, for the observer's + output levels. +- **Keep them short.** Every run records in real time. + - `TestAudioCheck`'s `shortPlan()` is two punch-ins of two sweeps on the calibration + reference cut short after its fifth event (10.8 s), about 13 s a run. + - A dev run takes `DevChecks::Options`, which `options()` fills in: the round trip for + the run, the report and scratch directories of the test's own, and the long song's + length (`longSeconds`). The passing run's long song is 60 s, not 240: long enough that + its whole analysis takes well over twice a punch-in's (item 9), and about 50 s for the + whole run. Runs that look at other things leave the long song out (0), and a fault run + may cancel the run once the stage it needs is done (at "Pre-roll near the start", for + the re-record's faults): the checks of the stages it got through are worked out all the + same. `runDevChecks()` waits up to 120 s. +- **A dev check is not a QtTest function.** Each returns a `CheckResult`; a test runs a + dev run, or part of one, and asserts on its report: `check(item)` gives a check, + `number(check, label)` one of its numbers by its label, `describe()` every check's + verdict, message and numbers for a failure message (QtTest cuts a long one: pass the + item). Each check has a run where it passes and one where it fails, from a fault given to + the fake or the window, or was seen failing with the code broken for a moment. The + faults: the round trip given 20 ms off; `echoDelay` / `echoGain` with the input on + channel 2 (`inputChannel`); the take made audible as the re-record's punch-in starts + recording (a lambda on the runner's `progress()` that sets its play parameters); a stall. +- **A noise floor for the output checks.** On a noiseless loopback the take holds the + reference and nothing else, silent wherever the reference is, so a take played back out + shows in no silent gap, and items 4 and 12 cannot fail. `loopbackInARoom()` adds a + programmed input of white noise at −60 dBFS, as a room gives a microphone: too quiet for + the sweep finder, and no pitch for the live tracker or pYIN. The passing run and the + lead-in's fault and stall runs use it. +- **Holding the GUI thread up on purpose** (`stallTheReRecording()`, + `dev_checks_lead_in_through_a_stall`): a busy-wait in the slot of a 5 ms timer, due by + where the reference is being handed out (playback start plus the frames received), not by + the take's recorded duration, which is counted on the GUI thread and stands still while + it is held up. +- **Where files go.** `TestAudioCheck`'s plans name a reference file in the test's own + directory; its tests of the check's own directory, and all of `TestDevChecks`, write + references where the check writes them, the application data directory, with + `QStandardPaths::setTestModeEnabled()` on so that it is Qt's test location. Reports and + scratch folders go to the test's own directory, never `TONY_TEST_LOG_DIR`: a failing + run's `DevChecks.txt` would land among the suites' result files, which are grepped for + `^FAIL` and `Totals`. The passing test prints the report line by line after `report:`, so + that its Totals never begins a line of the suite's log. +- **`TestAudioCheck`'s windows have no dev checks.** Its fixture deletes each window's + `DevChecks` (`doDeleteDevChecks()`): in a development build its dialog tests would + otherwise carry on into them, the checkbox being on. So it runs the same in a `release` + build, where it is the only test of the button. +- **No device, and a dead one.** `makeWindow(config, false)` makes a window with no audio + device at all; `neverCallsBack` a device that opens and delivers nothing. + ## What stays manual Anything about a real device, real timing by ear, or how something looks: From cf8f47f686a27f48bc7f486abd1ea80025ece562 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:28:45 +0000 Subject: [PATCH 189/275] feat: m4a, flac and ogg through android's decoders a bqaudiostream reader over the ndk's amediaextractor and amediacodec lets the android build open audio its libraries cannot read, such as a desktop session's m4a reference. decodedpcm, the conversion and buffering, is tested on the desktop. encoder delay and padding are kept, as windows' media foundation keeps them, so that a desktop session's reference lines up; each file's decoder, and any failure, goes to the log. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- deploy/android/build-tony.sh | 2 +- main/AndroidMediaReadStream.cpp | 537 ++++++++++++++++++++++++++++++++ main/AndroidMediaReadStream.h | 74 +++++ main/DecodedPcm.cpp | 186 +++++++++++ main/DecodedPcm.h | 112 +++++++ main/test/TestDecodedPcm.h | 300 ++++++++++++++++++ main/test/tony-core-test.cpp | 7 + meson.build | 17 +- 8 files changed, 1232 insertions(+), 3 deletions(-) create mode 100644 main/AndroidMediaReadStream.cpp create mode 100644 main/AndroidMediaReadStream.h create mode 100644 main/DecodedPcm.cpp create mode 100644 main/DecodedPcm.h create mode 100644 main/test/TestDecodedPcm.h diff --git a/deploy/android/build-tony.sh b/deploy/android/build-tony.sh index d152d2a9..030b28da 100755 --- a/deploy/android/build-tony.sh +++ b/deploy/android/build-tony.sh @@ -150,7 +150,7 @@ check() { echo " it exports $symbol" } -system_libraries='lib(c|m|dl|log|z|android|c\+\+_shared)\.so' +system_libraries='lib(c|m|dl|log|z|android|mediandk|c\+\+_shared)\.so' check libTony_arm64-v8a.so main "$system_libraries|libQt6[A-Za-z]+_arm64-v8a\.so" check pyin.so vampGetPluginDescriptor "$system_libraries" diff --git a/main/AndroidMediaReadStream.cpp b/main/AndroidMediaReadStream.cpp new file mode 100644 index 00000000..eb2918c7 --- /dev/null +++ b/main/AndroidMediaReadStream.cpp @@ -0,0 +1,537 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AndroidMediaReadStream.h" + +#include "DecodedPcm.h" + +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +using namespace breakfastquay; + +namespace { + +// MediaFormat's KEY_ENCODER_DELAY and KEY_ENCODER_PADDING: their NDK +// names are of API level 29, and Tony's is 28 +const char *const keyEncoderDelay = "encoder-delay"; +const char *const keyEncoderPadding = "encoder-padding"; + +// How long one wait for decoded output lasts, and how long the decoder +// may take nothing and give nothing before it is taken to have stopped +const int64_t waitMicroseconds = 5000; +const auto stalledAfter = std::chrono::seconds(10); + +struct FormatDeleter { + void operator()(AMediaFormat *f) const { AMediaFormat_delete(f); } +}; +typedef std::unique_ptr Format; + +// The format a MIME type names, for the log: "AAC (audio/mp4a-latm)" +std::string +describeMime(const std::string &mime) +{ + static const char *const names[][2] = { + { "audio/mp4a-latm", "AAC" }, + { "audio/flac", "FLAC" }, + { "audio/vorbis", "Vorbis" }, + { "audio/opus", "Opus" }, + { "audio/mpeg", "MP3" }, + { "audio/3gpp", "AMR-NB" }, + { "audio/amr-wb", "AMR-WB" }, + { "audio/alac", "Apple Lossless" }, + { "audio/ac3", "AC-3" }, + { "audio/eac3", "E-AC-3" }, + { "audio/raw", "PCM" }, + }; + for (const auto &n : names) { + if (mime == n[0]) return std::string(n[1]) + " (" + mime + ")"; + } + return mime; +} + +std::string +seconds(double s) +{ + std::ostringstream os; + os << std::fixed << std::setprecision(2) << s << " s"; + return os.str(); +} + +// One line to the log at once, so that another thread's cannot break it +void +logLine(const std::string &line) +{ + std::cerr << ("AndroidMediaReadStream: " + line + "\n") << std::flush; +} + +} + +// Tags: the extensions of files Android's extractors read (Android's +// "Supported media formats") and the Android build cannot read +// otherwise. Not wav, mp3 or opus: libsndfile, libmad and opusfile read +// those, and where two readers claim one extension the first registered +// wins, in an order that depends on the linker +static AudioReadStreamBuilder +androidMediaBuilder("https://github.com/jhhr/tony/AndroidMediaReadStream", + AndroidMediaReadStream::getExtensions()); + +std::vector +AndroidMediaReadStream::getExtensions() +{ + return { + "m4a", "mp4", "aac", "3gp", // AAC, and AMR in 3GPP + "amr", + "flac", + "ogg", "oga", // Vorbis or Opus + "webm", "mka", "mkv" // Vorbis or Opus, and others + }; +} + +class AndroidMediaReadStream::D +{ +public: + D(std::string p) : path(p) { } + ~D(); + + std::string path; + std::string error; + + int fd = -1; + AMediaExtractor *extractor = nullptr; + AMediaCodec *codec = nullptr; + bool started = false; + + std::string mime; + std::string codecName; + int64_t durationUs = 0; + int32_t delay = 0; + int32_t padding = 0; + int trackRate = 0; + int trackChannels = 0; + + // The decoder's output format as it stands + int rate = 0; + int channels = 0; + int encoding = DecodedPcm::Int16; + + // Made at the first decoded audio, whose channels and rate are the + // stream's from then on + std::unique_ptr pcm; + int streamRate = 0; + bool rateChangeLogged = false; + + bool inputEnded = false; + bool outputEnded = false; + int64_t framesQueued = 0; + + void open(); + void decodeUntil(bool (D::*done)() const); + bool hasDecoded() const { return pcm != nullptr || outputEnded; } + bool hasAvailable() const { + return (pcm && pcm->getAvailable() > 0) || outputEnded; + } + +private: + bool step(); + void takeOutputFormat(); + void takeOutput(ssize_t index, const AMediaCodecBufferInfo &info); + [[noreturn]] void fail(std::string why); +}; + +AndroidMediaReadStream::D::~D() +{ + if (codec) { + if (started) AMediaCodec_stop(codec); + AMediaCodec_delete(codec); + } + if (extractor) AMediaExtractor_delete(extractor); + // After the extractor, which may still be reading it + if (fd >= 0) ::close(fd); +} + +void +AndroidMediaReadStream::D::fail(std::string why) +{ + error = why; + logLine("\"" + path + "\": " + why); + throw InvalidFileFormat(path, why); +} + +void +AndroidMediaReadStream::D::open() +{ + fd = ::open(path.c_str(), O_RDONLY | O_CLOEXEC); + if (fd < 0) { + int e = errno; + error = std::string("cannot open the file: ") + std::strerror(e); + logLine("\"" + path + "\": " + error); + if (e == ENOENT) throw FileNotFound(path); + throw FileReadFailed(path); + } + + struct stat st; + if (fstat(fd, &st) != 0) { + fail(std::string("cannot read the file: ") + std::strerror(errno)); + } + + extractor = AMediaExtractor_new(); + if (!extractor) fail("Android gave no media extractor"); + + media_status_t status = + AMediaExtractor_setDataSourceFd(extractor, fd, 0, st.st_size); + if (status != AMEDIA_OK) { + fail("Android does not know the file's format (media error " + + std::to_string(int(status)) + ")"); + } + + // The first audio track + size_t tracks = AMediaExtractor_getTrackCount(extractor); + Format format; + size_t track = 0; + std::string others; + for (size_t i = 0; i < tracks; ++i) { + Format f(AMediaExtractor_getTrackFormat(extractor, i)); + const char *m = nullptr; + if (!f || !AMediaFormat_getString(f.get(), AMEDIAFORMAT_KEY_MIME, &m) || + !m) { + continue; + } + std::string trackMime(m); + if (!format && trackMime.compare(0, 6, "audio/") == 0) { + format = std::move(f); + track = i; + mime = trackMime; + } else { + others += (others == "" ? "" : ", ") + trackMime; + } + } + if (!format) { + fail("the file has no audio track" + + (others == "" ? std::string() : " (it holds " + others + ")")); + } + + status = AMediaExtractor_selectTrack(extractor, track); + if (status != AMEDIA_OK) { + fail("Android cannot read the audio track (media error " + + std::to_string(int(status)) + ")"); + } + + int32_t v = 0; + if (AMediaFormat_getInt32(format.get(), AMEDIAFORMAT_KEY_SAMPLE_RATE, &v)) { + trackRate = rate = v; + } + if (AMediaFormat_getInt32(format.get(), AMEDIAFORMAT_KEY_CHANNEL_COUNT, &v)) { + trackChannels = channels = v; + } + int64_t d = 0; + if (AMediaFormat_getInt64(format.get(), AMEDIAFORMAT_KEY_DURATION, &d)) { + durationUs = d; + } + + // The decoder would trim the delay and padding its format declares, + // which Media Foundation seems not to (see the header): so it is told + // of none + if (AMediaFormat_getInt32(format.get(), keyEncoderDelay, &delay) && + delay != 0) { + AMediaFormat_setInt32(format.get(), keyEncoderDelay, 0); + } + if (AMediaFormat_getInt32(format.get(), keyEncoderPadding, &padding) && + padding != 0) { + AMediaFormat_setInt32(format.get(), keyEncoderPadding, 0); + } + + codec = AMediaCodec_createDecoderByType(mime.c_str()); + if (!codec) { + fail("this phone has no decoder for " + describeMime(mime) + + ", the format of the file's audio"); + } + + char *name = nullptr; + if (AMediaCodec_getName(codec, &name) == AMEDIA_OK && name) { + codecName = name; + AMediaCodec_releaseName(codec, name); + } + + status = AMediaCodec_configure(codec, format.get(), nullptr, nullptr, 0); + if (status != AMEDIA_OK) { + fail("the decoder " + codecName + " does not take this " + + describeMime(mime) + " track (media error " + + std::to_string(int(status)) + ")"); + } + + status = AMediaCodec_start(codec); + if (status != AMEDIA_OK) { + fail("the decoder " + codecName + " did not start (media error " + + std::to_string(int(status)) + ")"); + } + started = true; + + // Until the first audio, whose format is the stream's + decodeUntil(&D::hasDecoded); + if (!pcm) { + fail("the decoder " + codecName + " gave no audio from this " + + describeMime(mime) + " track"); + } +} + +void +AndroidMediaReadStream::D::decodeUntil(bool (D::*done)() const) +{ + auto lastProgress = std::chrono::steady_clock::now(); + while (!(this->*done)()) { + if (step()) { + lastProgress = std::chrono::steady_clock::now(); + } else if (std::chrono::steady_clock::now() - lastProgress > + stalledAfter) { + fail("the decoder " + codecName + " stopped giving audio after " + + std::to_string(pcm ? pcm->getFramesAdded() : 0) + " frames"); + } + } +} + +// The next compressed frame to the decoder, if it has room for one, and +// what it has decoded from the decoder, if it has anything. False if +// neither happened, having waited a little for output +bool +AndroidMediaReadStream::D::step() +{ + bool progress = false; + + if (!inputEnded) { + ssize_t index = AMediaCodec_dequeueInputBuffer(codec, 0); + if (index >= 0) { + size_t capacity = 0; + uint8_t *buffer = AMediaCodec_getInputBuffer + (codec, size_t(index), &capacity); + if (!buffer) fail("the decoder gave no input buffer"); + + ssize_t size = -1; + if (AMediaExtractor_getSampleTrackIndex(extractor) >= 0) { + size = AMediaExtractor_readSampleData + (extractor, buffer, capacity); + if (size < 0) { + fail("Android could not read compressed frame " + + std::to_string(framesQueued + 1) + ", of " + + std::to_string(AMediaExtractor_getSampleSize(extractor)) + + " bytes, into the decoder's buffer of " + + std::to_string(capacity)); + } + } + + media_status_t status; + if (size < 0) { + // No frames left: the decoder is told, and gives what it + // still holds, the last with the end-of-stream flag + status = AMediaCodec_queueInputBuffer + (codec, size_t(index), 0, 0, 0, + AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM); + inputEnded = true; + } else { + int64_t time = AMediaExtractor_getSampleTime(extractor); + status = AMediaCodec_queueInputBuffer + (codec, size_t(index), 0, size_t(size), + uint64_t(time < 0 ? 0 : time), 0); + ++framesQueued; + AMediaExtractor_advance(extractor); + } + if (status != AMEDIA_OK) { + fail("the decoder did not take a compressed frame (media " + "error " + std::to_string(int(status)) + ")"); + } + progress = true; + } else if (index != AMEDIACODEC_INFO_TRY_AGAIN_LATER) { + fail("the decoder gave no input buffer (media error " + + std::to_string(index) + ")"); + } + } + + // Wait for output only if there was no input to give: then there is + // nothing to do but wait + AMediaCodecBufferInfo info; + ssize_t index = AMediaCodec_dequeueOutputBuffer + (codec, &info, progress ? 0 : waitMicroseconds); + + if (index >= 0) { + takeOutput(index, info); + progress = true; + } else if (index == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED) { + takeOutputFormat(); + progress = true; + } else if (index == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED) { + // Nothing to do: buffers are asked for by index each time + progress = true; + } else if (index != AMEDIACODEC_INFO_TRY_AGAIN_LATER) { + fail("the decoder gave no output (media error " + + std::to_string(index) + ")"); + } + + return progress; +} + +void +AndroidMediaReadStream::D::takeOutputFormat() +{ + Format format(AMediaCodec_getOutputFormat(codec)); + if (!format) return; + + int32_t v = 0; + if (AMediaFormat_getInt32(format.get(), AMEDIAFORMAT_KEY_SAMPLE_RATE, &v) && + v > 0) { + rate = v; + } + if (AMediaFormat_getInt32(format.get(), AMEDIAFORMAT_KEY_CHANNEL_COUNT, &v) && + v > 0) { + channels = v; + } + + // Without it the samples are 16-bit integers (MediaFormat's + // KEY_PCM_ENCODING) + int32_t e = DecodedPcm::Int16; + AMediaFormat_getInt32(format.get(), AMEDIAFORMAT_KEY_PCM_ENCODING, &e); + encoding = e; + + if (!DecodedPcm::isKnownEncoding(encoding)) { + fail("the decoder " + codecName + " gives samples in " + + DecodedPcm::describe(encoding) + ", which Tony cannot read"); + } +} + +void +AndroidMediaReadStream::D::takeOutput(ssize_t index, + const AMediaCodecBufferInfo &info) +{ + size_t size = 0; + uint8_t *buffer = AMediaCodec_getOutputBuffer(codec, size_t(index), &size); + + bool usable = buffer && info.size > 0 && info.offset >= 0 && + size_t(info.offset) + size_t(info.size) <= size && + !(info.flags & AMEDIACODEC_BUFFER_FLAG_CODEC_CONFIG); + + if (usable) { + if (!pcm) { + if (channels < 1 || rate < 1) { + AMediaCodec_releaseOutputBuffer(codec, size_t(index), false); + fail("the decoder " + codecName + " gives audio of " + + std::to_string(channels) + " channels at " + + std::to_string(rate) + " Hz"); + } + pcm.reset(new DecodedPcm(channels)); + streamRate = rate; + } else if (rate != streamRate && !rateChangeLogged) { + // It cannot be resampled here: the rest plays at the wrong + // speed. Not known to happen + logLine("\"" + path + "\": the decoder's rate changed from " + + std::to_string(streamRate) + " to " + std::to_string(rate) + + " Hz part way through; the rest is taken as " + + std::to_string(streamRate) + " Hz"); + rateChangeLogged = true; + } + pcm->add(buffer + info.offset, size_t(info.size), encoding, channels); + } + + AMediaCodec_releaseOutputBuffer(codec, size_t(index), false); + + if ((info.flags & AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM) && !outputEnded) { + outputEnded = true; + int64_t frames = pcm ? pcm->getFramesAdded() : 0; + std::ostringstream os; + os << "\"" << path << "\": decoded " << frames << " frames"; + if (streamRate > 0) { + os << " (" << seconds(double(frames) / streamRate) << ")"; + } + os << " from " << framesQueued << " compressed frames"; + if (pcm && pcm->getBytesDropped() > 0) { + os << "; " << pcm->getBytesDropped() + << " bytes that were not whole frames left out"; + } + logLine(os.str()); + } +} + +AndroidMediaReadStream::AndroidMediaReadStream(std::string path) : + m_d(new D(path)) +{ + m_channelCount = 0; + m_sampleRate = 0; + m_seekable = false; + + m_d->open(); + + m_channelCount = size_t(m_d->pcm->getChannelCount()); + m_sampleRate = size_t(m_d->streamRate); + if (m_d->durationUs > 0) { + m_estimatedFrameCount = size_t(std::llround + (double(m_d->durationUs) * double(m_sampleRate) / 1.0e6)); + } + + // Which decoder, and what it gives: the first thing to look for in a + // log when a file sounds or lines up wrong + std::ostringstream os; + os << "\"" << path << "\": " << describeMime(m_d->mime) + << " by " << (m_d->codecName == "" ? "a decoder" : m_d->codecName) + << ": " << m_sampleRate << " Hz, " << m_channelCount << " channel(s), " + << DecodedPcm::describe(m_d->encoding); + if (m_d->trackRate != int(m_sampleRate) || + m_d->trackChannels != int(m_channelCount)) { + os << " (the file says " << m_d->trackRate << " Hz, " + << m_d->trackChannels << " channel(s))"; + } + if (m_d->durationUs > 0) { + os << "; " << seconds(double(m_d->durationUs) / 1.0e6) << ", about " + << m_estimatedFrameCount << " frames"; + } + if (m_d->delay != 0 || m_d->padding != 0) { + os << "; encoder delay " << m_d->delay << " and padding " + << m_d->padding << " frames kept, not trimmed"; + } + logLine(os.str()); +} + +AndroidMediaReadStream::~AndroidMediaReadStream() +{ +} + +std::string +AndroidMediaReadStream::getError() const +{ + return m_d ? m_d->error : std::string(); +} + +size_t +AndroidMediaReadStream::getFrames(size_t count, float *frames) +{ + size_t got = 0; + while (got < count) { + got += m_d->pcm->read(frames + got * m_channelCount, count - got); + if (got == count || m_d->outputEnded) break; + m_d->decodeUntil(&D::hasAvailable); + } + return got; +} diff --git a/main/AndroidMediaReadStream.h b/main/AndroidMediaReadStream.h new file mode 100644 index 00000000..d5fd5636 --- /dev/null +++ b/main/AndroidMediaReadStream.h @@ -0,0 +1,74 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_ANDROID_MEDIA_READ_STREAM_H +#define TONY_ANDROID_MEDIA_READ_STREAM_H + +#include + +#include +#include +#include + +/** + * Audio files read through Android's own extractors and decoders (the + * NDK's AMediaExtractor and AMediaCodec, in libmediandk): what the + * Android build cannot read otherwise. Its libsndfile has neither FLAC + * nor Ogg, and there is no Media Foundation, which reads M4A and AAC on + * Windows. Built for Android only. + * + * A bqaudiostream reader, registered with bqaudiostream's factory for + * its extensions (getExtensions()) as bqaudiostream's own readers are, + * so that svcore's BQAFileReader uses it. WAV, MP3 and Opus are not + * among them: libsndfile, libmad and opusfile read those as before. + * The registration is a static object in this file that nothing else + * refers to, kept in the application because it links tony_core whole. + * + * The first audio track is decoded from start to end: there is no + * seeking, as in bqaudiostream's Opus and Media Foundation readers. + * The channel count and rate are those of the decoder's output, which + * may differ from the container's (HE-AAC doubles the rate), and the + * samples in whatever PCM encoding the decoder gives (DecodedPcm). + * + * The delay and padding an encoder left, which an M4A declares (its + * iTunSMPB or edit list), stay in: the decoder is told of none, so that + * it trims nothing. Media Foundation seems to keep them on Windows (a + * reference it read for a user's session is a whole number of AAC + * frames long), and a session's reference has to be the same audio on + * both, or the pitch the session holds is out of line with it. + * + * Each file's format, decoder and length go to the log (stderr, which + * Help > Save Log... keeps), and so does why a file cannot be read. + */ +class AndroidMediaReadStream : public breakfastquay::AudioReadStream +{ +public: + AndroidMediaReadStream(std::string path); + ~AndroidMediaReadStream() override; + + std::string getTrackName() const override { return ""; } + std::string getArtistName() const override { return ""; } + std::string getError() const override; + + /// The file extensions it is registered for, in lower case + static std::vector getExtensions(); + +protected: + size_t getFrames(size_t count, float *frames) override; + +private: + class D; + std::unique_ptr m_d; +}; + +#endif diff --git a/main/DecodedPcm.cpp b/main/DecodedPcm.cpp new file mode 100644 index 00000000..35f125f0 --- /dev/null +++ b/main/DecodedPcm.cpp @@ -0,0 +1,186 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "DecodedPcm.h" + +#include +#include + +bool +DecodedPcm::isKnownEncoding(int encoding) +{ + return bytesPerSample(encoding) > 0; +} + +int +DecodedPcm::bytesPerSample(int encoding) +{ + switch (encoding) { + case Int16: return 2; + case Int8: return 1; + case Float: return 4; + case Int24Packed: return 3; + case Int32: return 4; + default: return 0; + } +} + +std::string +DecodedPcm::describe(int encoding) +{ + switch (encoding) { + case Int16: return "16-bit integer"; + case Int8: return "8-bit integer"; + case Float: return "float"; + case Int24Packed: return "24-bit integer"; + case Int32: return "32-bit integer"; + default: return "unknown encoding " + std::to_string(encoding); + } +} + +// One sample, from the device's byte order, which is little-endian on +// every Android ABI; memcpy, as a buffer need not be aligned for it +static float +sampleAt(const uint8_t *p, int encoding) +{ + switch (encoding) { + case DecodedPcm::Int16: { + int16_t v; + std::memcpy(&v, p, sizeof(v)); + return float(v) / 32768.f; + } + case DecodedPcm::Int8: + return (float(p[0]) - 128.f) / 128.f; + case DecodedPcm::Float: { + float v; + std::memcpy(&v, p, sizeof(v)); + return v; + } + case DecodedPcm::Int24Packed: { + // Into the top three bytes of an int32, which carries the sign + uint32_t u = (uint32_t(p[0]) << 8) | (uint32_t(p[1]) << 16) | + (uint32_t(p[2]) << 24); + int32_t v; + std::memcpy(&v, &u, sizeof(v)); + return float(double(v) / 2147483648.0); + } + case DecodedPcm::Int32: { + int32_t v; + std::memcpy(&v, p, sizeof(v)); + return float(double(v) / 2147483648.0); + } + default: + return 0.f; + } +} + +void +DecodedPcm::convert(const uint8_t *data, int encoding, + int sourceChannels, size_t frames, + float *out, int targetChannels) +{ + const int bytes = bytesPerSample(encoding); + if (bytes == 0 || sourceChannels < 1 || targetChannels < 1) return; + + const size_t frameBytes = size_t(bytes) * size_t(sourceChannels); + + for (size_t f = 0; f < frames; ++f) { + + const uint8_t *frame = data + f * frameBytes; + float *target = out + f * size_t(targetChannels); + + if (sourceChannels <= targetChannels) { + for (int c = 0; c < targetChannels; ++c) { + int s = c % sourceChannels; + target[c] = sampleAt(frame + s * bytes, encoding); + } + continue; + } + + for (int c = 0; c < targetChannels; ++c) { + float sum = 0.f; + int n = 0; + for (int s = c; s < sourceChannels; s += targetChannels) { + sum += sampleAt(frame + s * bytes, encoding); + ++n; + } + target[c] = sum / float(n); + } + } +} + +DecodedPcm::DecodedPcm(int channels) : + m_channels(std::max(channels, 1)), + m_readPosition(0), + m_added(0), + m_read(0), + m_dropped(0) +{ +} + +size_t +DecodedPcm::add(const void *data, size_t bytes, int encoding, int channels) +{ + const int sampleBytes = bytesPerSample(encoding); + if (sampleBytes == 0 || channels < 1 || !data) { + m_dropped += int64_t(bytes); + return 0; + } + + const size_t frameBytes = size_t(sampleBytes) * size_t(channels); + const size_t frames = bytes / frameBytes; + m_dropped += int64_t(bytes - frames * frameBytes); + if (frames == 0) return 0; + + // What has been read goes before more is added, once it is at least + // half of what is held, so that the buffer does not grow for ever + if (m_readPosition > 0 && m_readPosition * 2 >= m_samples.size()) { + m_samples.erase(m_samples.begin(), + m_samples.begin() + ptrdiff_t(m_readPosition)); + m_readPosition = 0; + } + + size_t start = m_samples.size(); + m_samples.resize(start + frames * size_t(m_channels)); + convert(static_cast(data), encoding, channels, frames, + m_samples.data() + start, m_channels); + + m_added += int64_t(frames); + return frames; +} + +size_t +DecodedPcm::getAvailable() const +{ + return (m_samples.size() - m_readPosition) / size_t(m_channels); +} + +size_t +DecodedPcm::read(float *out, size_t frames) +{ + size_t n = std::min(frames, getAvailable()); + if (n == 0) return 0; + + size_t samples = n * size_t(m_channels); + std::memcpy(out, m_samples.data() + m_readPosition, + samples * sizeof(float)); + m_readPosition += samples; + + if (m_readPosition == m_samples.size()) { + m_samples.clear(); + m_readPosition = 0; + } + + m_read += int64_t(n); + return n; +} diff --git a/main/DecodedPcm.h b/main/DecodedPcm.h new file mode 100644 index 00000000..4005bf05 --- /dev/null +++ b/main/DecodedPcm.h @@ -0,0 +1,112 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_DECODED_PCM_H +#define TONY_DECODED_PCM_H + +#include +#include +#include +#include + +/** + * What a platform's audio decoder hands out, made into what svcore's + * readers give (bqaudiostream's AudioReadStream::getInterleavedFrames()): + * float frames, interleaved, at one channel count, read in pieces of + * any size. + * + * For AndroidMediaReadStream, whose decoder (Android's MediaCodec) + * gives buffers of PCM in whatever encoding its output format names, + * 16-bit integers when it names none, and may change that format part + * way through: the channel count the stream reports is fixed by its + * first buffer, and a buffer with another count is folded into it. + * + * Nothing here touches a decoder, so all of it is tested on the desktop + * (TestDecodedPcm). + */ +class DecodedPcm +{ +public: + /** + * Android's PCM encodings, the values of AudioFormat.ENCODING_PCM_* + * that a MediaFormat's "pcm-encoding" holds. Each sample is in the + * device's own byte order (little-endian), 24-bit ones as three + * bytes; 8-bit ones are unsigned about 128. + */ + enum Encoding { + Int16 = 2, + Int8 = 3, + Float = 4, + Int24Packed = 21, + Int32 = 22 + }; + + /// Whether the value is one of the encodings above + static bool isKnownEncoding(int encoding); + + /// Bytes per sample of an encoding, 0 for one not known + static int bytesPerSample(int encoding); + + /// The encoding in words, for the log: "16-bit integer" and so on + static std::string describe(int encoding); + + /** + * Convert frames of interleaved samples in an encoding, and in + * sourceChannels channels, into float frames in targetChannels. + * Fewer channels than wanted: channel c is source channel c modulo + * their number (mono goes to every channel). More: target channel c + * is the mean of the source channels whose index is c modulo the + * target's number (all of them for mono), so that no channel, and + * no voice in a centre channel, is lost. + */ + static void convert(const uint8_t *data, int encoding, + int sourceChannels, size_t frames, + float *out, int targetChannels); + + explicit DecodedPcm(int channels); + + int getChannelCount() const { return m_channels; } + + /** + * Take a buffer of bytes of samples in the encoding, interleaved in + * the given number of channels. Returns the whole frames taken; a + * part of a frame at the end, which a decoder should never give, is + * left out and counted (getBytesDropped()). Nothing is taken in an + * encoding that is not known, or with no channels. + */ + size_t add(const void *data, size_t bytes, int encoding, int channels); + + /// Frames taken and not yet read + size_t getAvailable() const; + + /** + * Copy up to the given number of frames into out, which has room + * for that many at getChannelCount(), and return how many there + * were. + */ + size_t read(float *out, size_t frames); + + int64_t getFramesAdded() const { return m_added; } + int64_t getFramesRead() const { return m_read; } + int64_t getBytesDropped() const { return m_dropped; } + +private: + int m_channels; + std::vector m_samples; + size_t m_readPosition; // in samples, into m_samples + int64_t m_added; + int64_t m_read; + int64_t m_dropped; +}; + +#endif diff --git a/main/test/TestDecodedPcm.h b/main/test/TestDecodedPcm.h new file mode 100644 index 00000000..ace10b2c --- /dev/null +++ b/main/test/TestDecodedPcm.h @@ -0,0 +1,300 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_DECODED_PCM_H +#define TEST_DECODED_PCM_H + +// Tier 2: the output of Android's decoders as svcore's readers give +// audio. Bytes in the encodings Android names, float frames out; the +// decoder itself can only be tried on a phone. + +#include "../DecodedPcm.h" + +#include +#include + +#include +#include +#include +#include + +class TestDecodedPcm : public QObject +{ + Q_OBJECT + + // The samples as bytes in an encoding, little-endian as on every + // Android ABI: the inverse of the conversion, for the tests that go + // through every encoding. The byte-level tests below do not use it, + // so that a mistake made the same way both ways is still caught + static std::vector encode(const std::vector &samples, + int encoding) { + std::vector bytes; + for (float v : samples) { + switch (encoding) { + case DecodedPcm::Int16: { + long i = std::lrint(double(v) * 32768.0); + i = std::min(32767L, std::max(-32768L, i)); + bytes.push_back(uint8_t(i & 0xff)); + bytes.push_back(uint8_t((i >> 8) & 0xff)); + break; + } + case DecodedPcm::Int8: { + long i = std::lrint(double(v) * 128.0) + 128; + i = std::min(255L, std::max(0L, i)); + bytes.push_back(uint8_t(i)); + break; + } + case DecodedPcm::Float: { + uint8_t b[4]; + std::memcpy(b, &v, 4); + bytes.insert(bytes.end(), b, b + 4); + break; + } + case DecodedPcm::Int24Packed: { + long i = std::lrint(double(v) * 8388608.0); + i = std::min(8388607L, std::max(-8388608L, i)); + bytes.push_back(uint8_t(i & 0xff)); + bytes.push_back(uint8_t((i >> 8) & 0xff)); + bytes.push_back(uint8_t((i >> 16) & 0xff)); + break; + } + case DecodedPcm::Int32: { + long long i = std::llrint(double(v) * 2147483648.0); + i = std::min(2147483647LL, std::max(-2147483648LL, i)); + for (int k = 0; k < 4; ++k) { + bytes.push_back(uint8_t((i >> (8 * k)) & 0xff)); + } + break; + } + } + } + return bytes; + } + + // The smallest step of an encoding, as a float + static double step(int encoding) { + switch (encoding) { + case DecodedPcm::Int16: return 1.0 / 32768.0; + case DecodedPcm::Int8: return 1.0 / 128.0; + case DecodedPcm::Int24Packed: return 1.0 / 8388608.0; + case DecodedPcm::Int32: return 1.0 / 2147483648.0; + default: return 0.0; + } + } + + // Frame f, channel c, of a signal whose every sample says where it + // came from, within [-1, 1) + static float ramp(size_t f, int c) { + return float((double(f % 200) - 100.0) / 128.0 + 0.001 * c); + } + + static std::vector ramps(size_t frames, int channels, size_t from = 0) { + std::vector v; + for (size_t f = from; f < from + frames; ++f) { + for (int c = 0; c < channels; ++c) v.push_back(ramp(f, c)); + } + return v; + } + + static std::vector convertBytes(std::vector bytes, + int encoding, int channels, + int targetChannels) { + size_t frames = bytes.size() / + size_t(DecodedPcm::bytesPerSample(encoding) * channels); + std::vector out(frames * size_t(targetChannels), -99.f); + DecodedPcm::convert(bytes.data(), encoding, channels, frames, + out.data(), targetChannels); + return out; + } + +private slots: + + // Android's own byte layouts (AudioFormat): signed and little-endian, + // 8-bit unsigned about 128, 24-bit as three bytes + void each_encoding_from_its_bytes() { + QCOMPARE(convertBytes({ 0x00, 0x40, 0x00, 0xc0, 0xff, 0x7f, 0x00, 0x80 }, + DecodedPcm::Int16, 1, 1), + (std::vector { 0.5f, -0.5f, 32767.f / 32768.f, -1.f })); + QCOMPARE(convertBytes({ 0xc0, 0x40, 0x80, 0x00 }, + DecodedPcm::Int8, 1, 1), + (std::vector { 0.5f, -0.5f, 0.f, -1.f })); + QCOMPARE(convertBytes({ 0x00, 0x00, 0x40, 0x00, 0x00, 0xc0, + 0xff, 0xff, 0xff, 0x00, 0x00, 0x80, + 0x01, 0x00, 0x00 }, + DecodedPcm::Int24Packed, 1, 1), + (std::vector { 0.5f, -0.5f, -1.f / 8388608.f, -1.f, + 1.f / 8388608.f })); + QCOMPARE(convertBytes({ 0x00, 0x00, 0x00, 0x40, 0x00, 0x00, 0x00, 0xc0, + 0x00, 0x00, 0x00, 0x80 }, + DecodedPcm::Int32, 1, 1), + (std::vector { 0.5f, -0.5f, -1.f })); + float values[] = { 0.25f, -0.75f }; + std::vector floats(sizeof(values)); + std::memcpy(floats.data(), values, sizeof(values)); + QCOMPARE(convertBytes(floats, DecodedPcm::Float, 1, 1), + (std::vector { 0.25f, -0.75f })); + } + + // The same stereo signal in every encoding comes out the same, to + // within the encoding's step, and in its frames and channels + void every_encoding_gives_the_signal() { + const int encodings[] = { + DecodedPcm::Int16, DecodedPcm::Int8, DecodedPcm::Float, + DecodedPcm::Int24Packed, DecodedPcm::Int32 + }; + std::vector signal = ramps(300, 2); + for (int encoding : encodings) { + std::vector out = + convertBytes(encode(signal, encoding), encoding, 2, 2); + QCOMPARE(out.size(), signal.size()); + for (size_t i = 0; i < signal.size(); ++i) { + QVERIFY2(std::fabs(out[i] - signal[i]) <= step(encoding) * 0.51 + 1e-9, + qPrintable(QString("%1: sample %2 is %3, not %4") + .arg(QString::fromStdString + (DecodedPcm::describe(encoding))) + .arg(i).arg(out[i]).arg(signal[i]))); + } + } + } + + void encodings_known_and_described() { + QVERIFY(DecodedPcm::isKnownEncoding(2)); + QVERIFY(DecodedPcm::isKnownEncoding(4)); + QVERIFY(!DecodedPcm::isKnownEncoding(0)); + QVERIFY(!DecodedPcm::isKnownEncoding(1)); // ENCODING_DEFAULT + QVERIFY(!DecodedPcm::isKnownEncoding(5)); // AC3: not PCM + QCOMPARE(DecodedPcm::bytesPerSample(21), 3); + QCOMPARE(QString::fromStdString(DecodedPcm::describe(4)), + QString("float")); + QVERIFY(QString::fromStdString(DecodedPcm::describe(7)) + .contains("7")); + } + + // Fewer channels than the stream's: each is used in turn, a mono + // buffer in both channels of a stereo stream + void fewer_channels_are_repeated() { + std::vector mono = { 0.5f, -0.25f }; + std::vector out = convertBytes + (encode(mono, DecodedPcm::Float), DecodedPcm::Float, 1, 2); + QCOMPARE(out, (std::vector { 0.5f, 0.5f, -0.25f, -0.25f })); + + std::vector stereo = { 0.5f, -0.25f }; + out = convertBytes(encode(stereo, DecodedPcm::Float), + DecodedPcm::Float, 2, 3); + QCOMPARE(out, (std::vector { 0.5f, -0.25f, 0.5f })); + } + + // More channels than the stream's: folded in, none dropped; 5.1 to + // stereo keeps the centre channel, where a voice usually is + void more_channels_are_folded_in() { + std::vector stereo = { 0.5f, -0.25f }; + std::vector out = convertBytes + (encode(stereo, DecodedPcm::Float), DecodedPcm::Float, 2, 1); + QCOMPARE(out, (std::vector { 0.125f })); + + // FL FR FC LFE BL BR + std::vector six = { 0.1f, 0.2f, 0.4f, 0.8f, 0.3f, 0.6f }; + out = convertBytes(encode(six, DecodedPcm::Float), + DecodedPcm::Float, 6, 2); + QCOMPARE(out.size(), size_t(2)); + QVERIFY(std::fabs(out[0] - (0.1f + 0.4f + 0.3f) / 3.f) < 1e-6); + QVERIFY(std::fabs(out[1] - (0.2f + 0.8f + 0.6f) / 3.f) < 1e-6); + } + + // Buffers of any size in, reads of any size out: every frame once, + // in order, and the counts agree + void read_in_pieces_of_any_size() { + DecodedPcm pcm(2); + QCOMPARE(pcm.getChannelCount(), 2); + QCOMPARE(pcm.getAvailable(), size_t(0)); + + // Reads that leave a little behind, so that what has been read is + // cleared away before more is added, and reads of more than is + // there + const size_t adds[] = { 1024, 1, 333, 2048, 7, 1024, 500 }; + const size_t reads[] = { 1000, 20, 1, 2000, 300, 37, 5000 }; + + size_t added = 0, readFrames = 0; + size_t r = 0; + std::vector got; + + for (size_t a : adds) { + std::vector bytes = + encode(ramps(a, 2, added), DecodedPcm::Int16); + QCOMPARE(pcm.add(bytes.data(), bytes.size(), + DecodedPcm::Int16, 2), a); + added += a; + QCOMPARE(pcm.getAvailable(), added - readFrames); + + std::vector out(reads[r] * 2, -99.f); + size_t n = pcm.read(out.data(), reads[r]); + QCOMPARE(n, std::min(reads[r], added - readFrames)); + ++r; + readFrames += n; + got.insert(got.end(), out.begin(), out.begin() + ptrdiff_t(n * 2)); + } + + // The rest, and then nothing + std::vector out(added * 2); + size_t n = pcm.read(out.data(), added); + QCOMPARE(n, added - readFrames); + readFrames += n; + got.insert(got.end(), out.begin(), out.begin() + ptrdiff_t(n * 2)); + QCOMPARE(pcm.read(out.data(), 10), size_t(0)); + + QCOMPARE(pcm.getFramesAdded(), int64_t(added)); + QCOMPARE(pcm.getFramesRead(), int64_t(added)); + QCOMPARE(pcm.getAvailable(), size_t(0)); + + std::vector expected = ramps(added, 2); + QCOMPARE(got.size(), expected.size()); + for (size_t i = 0; i < got.size(); ++i) { + QVERIFY2(std::fabs(got[i] - expected[i]) <= 1.0 / 65536.0, + qPrintable(QString("sample %1 is %2, not %3") + .arg(i).arg(got[i]).arg(expected[i]))); + } + } + + // A buffer in another channel count is taken at the stream's count: + // the format changed part way through + void a_change_of_channels_keeps_the_stream_s_count() { + DecodedPcm pcm(2); + std::vector stereo = encode({ 0.5f, -0.5f }, DecodedPcm::Int16); + std::vector mono = encode({ 0.25f, 0.75f }, DecodedPcm::Float); + QCOMPARE(pcm.add(stereo.data(), stereo.size(), DecodedPcm::Int16, 2), + size_t(1)); + QCOMPARE(pcm.add(mono.data(), mono.size(), DecodedPcm::Float, 1), + size_t(2)); + std::vector out(6); + QCOMPARE(pcm.read(out.data(), 3), size_t(3)); + QCOMPARE(out, (std::vector { 0.5f, -0.5f, 0.25f, 0.25f, + 0.75f, 0.75f })); + } + + // What is not whole frames, or not PCM Tony knows, is not taken + void part_frames_and_unknown_encodings_are_dropped() { + DecodedPcm pcm(2); + std::vector bytes = encode({ 0.5f, -0.5f, 0.25f }, DecodedPcm::Int16); + QCOMPARE(pcm.add(bytes.data(), bytes.size(), DecodedPcm::Int16, 2), + size_t(1)); + QCOMPARE(pcm.getBytesDropped(), int64_t(2)); + QCOMPARE(pcm.add(bytes.data(), bytes.size(), 5, 2), size_t(0)); + QCOMPARE(pcm.add(bytes.data(), bytes.size(), DecodedPcm::Int16, 0), + size_t(0)); + QCOMPARE(pcm.getBytesDropped(), int64_t(2 + 6 + 6)); + QCOMPARE(pcm.getFramesAdded(), int64_t(1)); + QCOMPARE(pcm.getAvailable(), size_t(1)); + } +}; + +#endif diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index ceee6d6c..cf56c64a 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -16,6 +16,7 @@ #include "TestRealtimePitchTracker.h" #include "TestLatencyShift.h" #include "TestCoverage.h" +#include "TestDecodedPcm.h" #include "TestLogFile.h" #include "TestPinchZoom.h" #include "TestPopupArea.h" @@ -80,6 +81,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestDecodedPcm t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestLogFile t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index d0a2f2c5..d62d035e 100644 --- a/meson.build +++ b/meson.build @@ -539,9 +539,11 @@ elif system == 'android' '-Wl,--version-script=' + meson.current_source_dir() / 'vamp-plugin-sdk/vamp-plugin.map' ] - # Android's system log, which main.cpp passes stdout and stderr on to + # Android's system log, which main.cpp passes stdout and stderr on to; + # and its media extractors and decoders (main/AndroidMediaReadStream) feature_additional_libs = [ - '-llog' + '-llog', + '-lmediandk', ] else @@ -1170,6 +1172,7 @@ tony_entry_files = [ tony_core_files = [ 'main/AndroidFiles.cpp', 'main/Coverage.cpp', + 'main/DecodedPcm.cpp', 'main/LogFile.cpp', 'main/PinchZoom.cpp', 'main/PopupArea.cpp', @@ -1183,6 +1186,15 @@ tony_core_files = [ 'main/VerticalZoom.cpp', ] +# Android's decoders for the audio its libraries cannot read (M4A, FLAC, +# Ogg): a bqaudiostream reader that registers itself, which only link_whole +# keeps in the application, nothing referring to it +if system == 'android' + tony_core_files += [ + 'main/AndroidMediaReadStream.cpp', + ] +endif + tony_app_files = [ 'main/AlternatePitchTrack.cpp', 'main/CompactLayout.cpp', @@ -1486,6 +1498,7 @@ if system != 'android' 'main/test/TestRealtimePitchTracker.h', 'main/test/TestLatencyShift.h', 'main/test/TestCoverage.h', + 'main/test/TestDecodedPcm.h', 'main/test/TestLogFile.h', 'main/test/TestPinchZoom.h', 'main/test/TestPopupArea.h', From 306ebfa448720dd66d875f9174957ad8f099e53d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:28:46 +0000 Subject: [PATCH 190/275] fix: an incomplete session is saved only when the user says so a session whose audio could not be loaded would be saved without any reference to it. save, save as and save in audio path now ask first, and the save on suspend on android passes such a session by, so that a sync app cannot carry the damage back to the desktop. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/MainWindow.cpp | 62 ++++++++++- main/MainWindow.h | 16 +++ main/test/TestRecordWorkflow.h | 194 +++++++++++++++++++++++++++++++++ 3 files changed, 269 insertions(+), 3 deletions(-) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 0a562258..004f5f33 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3122,8 +3122,15 @@ MainWindow::applicationStateChanged(Qt::ApplicationState state) } // Only a session that has a file of its own: one never saved stays as - // it is, as there is no one to ask where it should go - if (m_sessionFile == "" || !m_documentModified) return; + // it is, as there is no one to ask where it should go. Nor one that + // loaded without some of its audio, which its file would lose + if (!maySaveUnasked()) { + if (m_documentModified && sessionIsIncomplete()) { + cerr << "MainWindow::applicationStateChanged: the session " + << "loaded incomplete; it is not saved" << endl; + } + return; + } m_suspendSavePath = m_sessionFile; saveWhenSuspended(); @@ -3136,7 +3143,7 @@ MainWindow::saveWhenSuspended() // Another session since, or saved since if (m_suspendSavePath == "" || m_suspendSavePath != m_sessionFile || - !m_documentModified) { + !maySaveUnasked()) { m_suspendSavePath = ""; return; } @@ -6839,6 +6846,42 @@ MainWindow::checkSaveModified() return false; } +bool +MainWindow::sessionIsIncomplete() const +{ + // Set by svapp's session reader for audio it could not read, and kept + // by the document until it is replaced (or saved, below) + return m_document && m_document->isIncomplete(); +} + +bool +MainWindow::confirmSaveOfIncompleteSession() +{ + if (!sessionIsIncomplete()) return true; + return askToSaveIncompleteSession(); +} + +bool +MainWindow::askToSaveIncompleteSession() +{ + // svapp's words at the load, asked again when it comes to it + return QMessageBox::warning + (this, tr("Save incomplete session?"), + tr("Save this session without all of its audio?

Some of " + "the audio content referred to by the original session file " + "could not be loaded. If you save this session, it will be " + "saved without any reference to that audio, and information " + "may be lost.

"), + QMessageBox::Save | QMessageBox::Cancel, + QMessageBox::Cancel) == QMessageBox::Save; +} + +bool +MainWindow::maySaveUnasked() const +{ + return m_sessionFile != "" && m_documentModified && !sessionIsIncomplete(); +} + bool MainWindow::waitForInitialAnalysis() { @@ -6921,6 +6964,9 @@ MainWindow::waitForRangedAnalysis() void MainWindow::saveSession() { + // (Save As asks for itself) + if (m_sessionFile != "" && !confirmSaveOfIncompleteSession()) return; + // We do not want to save mid-analysis regions -- that would cause // confusion on reloading m_analyser->clearReAnalysis(); @@ -6934,6 +6980,7 @@ MainWindow::saveSession() } else { CommandHistory::getInstance()->documentSaved(); documentRestored(); + if (m_document) m_document->setIncomplete(false); } } else { saveSessionAs(); @@ -6945,6 +6992,8 @@ MainWindow::saveSessionInAudioPath() { if (m_audioFile == "") return; + if (!confirmSaveOfIncompleteSession()) return; + if (!waitForInitialAnalysis()) return; // We do not want to save mid-analysis regions -- that would cause @@ -7004,6 +7053,9 @@ MainWindow::saveSessionToPath(QString path) // from now on is written into its folder (spec 6.4) m_sessionFile = path; + // and the file is what the session is: nothing it names is missing + if (m_document) m_document->setIncomplete(false); + CommandHistory::getInstance()->documentSaved(); documentRestored(); m_recentFiles.addFile(path); @@ -7013,6 +7065,10 @@ MainWindow::saveSessionToPath(QString path) void MainWindow::saveSessionAs() { + // Before the picker, which on Android has made the file by the time + // it returns + if (!confirmSaveOfIncompleteSession()) return; + // We do not want to save mid-analysis regions -- that would cause // confusion on reloading m_analyser->clearReAnalysis(); diff --git a/main/MainWindow.h b/main/MainWindow.h index c7ac547f..79d4e365 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -908,6 +908,22 @@ protected slots: bool checkSaveModified(); bool waitForInitialAnalysis(); + // A session that loaded without some of the audio it names (svapp's + // "Incomplete session loaded") is saved without any mention of that + // audio, so the file it came from, saved over, would lose it. It is + // saved only when the user asks and then says yes: Save, Save As and + // Save Session in Audio Path ask first, and a save no one asked for + // (Android's on suspend) passes it by. Once saved it is what its new + // file says, and is not asked about again + bool sessionIsIncomplete() const; + bool confirmSaveOfIncompleteSession(); + // The question, which the tests answer: they cannot answer a dialog + virtual bool askToSaveIncompleteSession(); + + // Whether the session may be saved with no one asked: it has a file + // of its own, has been changed, and is not incomplete + bool maySaveUnasked() const; + #ifdef Q_OS_ANDROID // Android's file picker gives content:// URIs, which svcore's readers // cannot open. A file in the phone's own storage is opened where it diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 06da55b7..e8d17790 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -144,6 +144,25 @@ class TestMainWindow : public MainWindow bool doSaveSessionAs(QString path) { return saveSessionToPath(path); } QString sessionFile() { return m_sessionFile; } + // Save and Save As, as the File menu does them. Save As is given the + // file here, not by a dialog, if the test has named one + void doSaveSession() { saveSession(); } + void doSaveSessionAsAsked() { saveSessionAs(); } + void setSaveFileNameAnswer(QString path) { m_saveFileNameAnswer = path; } + int saveFileNameQuestions() const { return m_saveFileNameQuestions; } + + // A session that loaded without some of its audio, and the question + // asked before it is saved, answered from here + bool isSessionIncomplete() const { return sessionIsIncomplete(); } + bool doMaySaveUnasked() const { return maySaveUnasked(); } + void setSaveIncompleteAnswer(bool yes) { m_saveIncompleteAnswer = yes; } + int saveIncompleteQuestions() const { return m_saveIncompleteQuestions; } + + // As a change to the session does, and the session's own file as + // svapp's session reader would set it + void markModified() { documentModified(); } + void setSessionFile(QString path) { m_sessionFile = path; } + // As answering "No" to "do you want to save?" void discardModifications() { m_documentModified = false; } bool isDocumentModified() { return m_documentModified; } @@ -249,6 +268,17 @@ class TestMainWindow : public MainWindow return m_takeNameAnswer == "" ? current : m_takeNameAnswer; } + bool askToSaveIncompleteSession() override { + ++m_saveIncompleteQuestions; + return m_saveIncompleteAnswer; + } + + QString getSaveFileName(sv::FileFinder::FileType type) override { + if (m_saveFileNameAnswer == "") return MainWindow::getSaveFileName(type); + ++m_saveFileNameQuestions; + return m_saveFileNameAnswer; + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -260,6 +290,10 @@ class TestMainWindow : public MainWindow bool m_deleteTakeAnswer = true; int m_deleteTakeQuestions = 0; QString m_takeNameAnswer; + bool m_saveIncompleteAnswer = false; + int m_saveIncompleteQuestions = 0; + QString m_saveFileNameAnswer; + int m_saveFileNameQuestions = 0; }; class TestRecordWorkflow : public QObject @@ -566,6 +600,43 @@ class TestRecordWorkflow : public QObject return ""; } + // A file's bytes, as they are on disk + static QByteArray fileContents(QString path) { + QFile f(path); + return f.open(QIODevice::ReadOnly) ? f.readAll() : QByteArray(); + } + + // Put text into a saved session file before the first occurrence of + // another, as removeTakesElement() takes some out. "" on success, + // else what went wrong + QString insertIntoSession(QString path, QByteArray before, + QByteArray text) { + QByteArray document; + { + sv::BZipFileDevice file(path); + if (!file.open(QIODevice::ReadOnly)) { + return "could not read " + path + ": " + file.errorString(); + } + document = file.readAll(); + file.close(); + } + int at = document.indexOf(before); + if (at < 0) return "no " + QString(before) + " in " + path; + document.insert(at, text); + + sv::BZipFileDevice out(path); + if (!out.open(QIODevice::WriteOnly)) { + return "could not write " + path + ": " + out.errorString(); + } + qint64 written = out.write(document); + out.close(); + if (written != document.size()) { + return QString("wrote %1 of %2 bytes to ").arg(written) + .arg(document.size()) + path; + } + return ""; + } + // The dialogs the watchdog has dismissed so far, taken off the list so // that cleanup() does not fail the test with them: for a test that // expects one @@ -5259,6 +5330,129 @@ private slots: QCOMPARE(stripEvents(), before.strip); } + // The reference of a session cannot be read when it is opened (on the + // phone: a format it has no decoder for), and svapp says the session + // loaded incomplete. Saved over, its file would lose the reference: it + // is saved only when the user asks and then says yes, and a save no + // one asked for (Android's on suspend) passes it by + void incomplete_session_not_saved_unasked() { + FakeAudioIO::Config config; + makeWindow(config); + QString reference = writeWav(tone(lowHz, 1.0)); + openReference(reference); + if (QTest::currentTestFailed()) return; + + QString session = m_dir.filePath("no-reference.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + QVERIFY(!m_window->isSessionIncomplete()); + m_window->markModified(); + QVERIFY2(m_window->doMaySaveUnasked(), + "a whole session, changed, with a file of its own, may not " + "be saved unasked"); + m_window->doCloseSession(); + QByteArray saved = fileContents(session); + QVERIFY(!saved.isEmpty()); + + QVERIFY(QFile::remove(reference)); + m_window->discardModifications(); + QCOMPARE(m_window->openPath(session, MainWindow::ReplaceSession), + MainWindow::FileOpenSucceeded); + QCOMPARE(dialogsMatching("Incomplete session loaded").size(), 1); + QVERIFY2(m_window->isSessionIncomplete(), + "the session is not known to have loaded incomplete"); + + // svapp gives such a session no file of its own; nor is it saved + // unasked if it has one + QCOMPARE(m_window->sessionFile(), QString()); + m_window->markModified(); + QVERIFY(!m_window->doMaySaveUnasked()); + m_window->setSessionFile(session); + QVERIFY2(!m_window->doMaySaveUnasked(), + "an incomplete session may be saved over its file unasked"); + + // Save, over that file: asked, and "no" saves nothing + m_window->setSaveIncompleteAnswer(false); + m_window->doSaveSession(); + QCOMPARE(m_window->saveIncompleteQuestions(), 1); + QCOMPARE(fileContents(session), saved); + QVERIFY(m_window->isDocumentModified()); + + // Save with no file of its own is Save As, which asks before the + // file is picked (on Android the picker makes the file) + m_window->setSessionFile(""); + QString other = m_dir.filePath("no-reference-saved.ton"); + m_window->setSaveFileNameAnswer(other); + m_window->doSaveSession(); + QCOMPARE(m_window->saveIncompleteQuestions(), 2); + m_window->doSaveSessionAsAsked(); + QCOMPARE(m_window->saveIncompleteQuestions(), 3); + QCOMPARE(m_window->saveFileNameQuestions(), 0); + QVERIFY(!QFileInfo::exists(other)); + QVERIFY(m_window->isSessionIncomplete()); + QVERIFY(m_window->isDocumentModified()); + QVERIFY(takeDialogs().isEmpty()); + } + + // The same, with the reference there and other audio the session names + // missing, so that it can be saved: once the user says yes, the session + // is what its new file says, and is neither asked about nor passed by + // again + void incomplete_session_saved_when_the_user_says_so() { + FakeAudioIO::Config config; + makeWindow(config); + openReference(writeWav(tone(lowHz, 1.0))); + if (QTest::currentTestFailed()) return; + + QString session = m_dir.filePath("missing-audio.ton"); + QVERIFY(m_window->doSaveSessionAs(session)); + m_window->doCloseSession(); + QString missing = m_dir.filePath("missing-audio.wav"); + QString error = insertIntoSession + (session, "", + QString("\n").arg(missing) + .toUtf8()); + QVERIFY2(error == "", qPrintable(error)); + QByteArray saved = fileContents(session); + + reopenSession(session); + if (QTest::currentTestFailed()) return; + QCOMPARE(dialogsMatching("Incomplete session loaded").size(), 1); + QVERIFY(m_window->isSessionIncomplete()); + QCOMPARE(m_window->sessionFile(), QString()); + m_window->markModified(); + QVERIFY(!m_window->doMaySaveUnasked()); + + // Saved over the file it came from, as the user may choose: which + // then no longer names the audio that was missing + m_window->setSaveIncompleteAnswer(true); + m_window->setSaveFileNameAnswer(session); + m_window->doSaveSession(); + QCOMPARE(m_window->saveIncompleteQuestions(), 1); + QCOMPARE(m_window->saveFileNameQuestions(), 1); + QVERIFY2(fileContents(session) != saved, "the session was not saved"); + QCOMPARE(m_window->sessionFile(), session); + QVERIFY(!m_window->isDocumentModified()); + QVERIFY(!m_window->isSessionIncomplete()); + { + sv::BZipFileDevice file(session); + QVERIFY(file.open(QIODevice::ReadOnly)); + QByteArray document = file.readAll(); + file.close(); + QVERIFY(!document.contains("missing-audio.wav")); + QCOMPARE(int(document.count("type=\"wavefile\"")), 1); + } + + m_window->markModified(); + QVERIFY(m_window->doMaySaveUnasked()); + m_window->doSaveSession(); + QCOMPARE(m_window->saveIncompleteQuestions(), 1); + QCOMPARE(m_window->saveFileNameQuestions(), 1); + QVERIFY(!m_window->isDocumentModified()); + QVERIFY(takeDialogs().isEmpty()); + } + // --- The takes folder of a session (spec 6.4) --- // // A take's combined audio belongs to the song: it lives in From a1cc110e1e31b02cfb5ea841d16a65cfd5489f97 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:28:46 +0000 Subject: [PATCH 191/275] docs: phase a7c done, and its log entry Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index af2aa85b..1a347613 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -162,7 +162,7 @@ builds happen in the container.) - A7 — Sessions in place on the phone, and fixes from the first phone test. Done. - A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. - A4b — Vertical zoom and scroll by touch. Done. -- A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. +- A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -700,3 +700,28 @@ Choices / deviations: Tests seen failing: across rule removed; zoom about the middle; no limits; spectrogram unsaved. Left open: not on a phone. A drag up may also scroll the pane stack, if it can scroll. +### Phase A7c — 2026-09-26 +Built: `main/AndroidMediaReadStream` (tony_core, Android only, `-lmediandk`): a bqaudiostream reader +over AMediaExtractor/AMediaCodec for m4a mp4 aac 3gp amr flac ogg oga webm mka mkv. Nothing else +claims those on Android (libsndfile.a has no FLAC or OGG format; oggz/fishsound are out); wav, mp3 +and opus stay with sndfile, mad, opusfile (two readers of one tag: the first registered wins, in +link order). First audio track, in order, no seek; channels and rate of the decoder's output at +its first audio; `pcm-encoding` (16-bit if absent). Logged: per file MIME, codec, rate, channels, +encoding, duration, delay/padding; at the end frames decoded and compressed frames; each refusal +("this phone has no decoder for AAC (audio/mp4a-latm)"). `main/DecodedPcm` (core, +`TestDecodedPcm`): five PCM encodings to float, channels folded, reads of any size. `MainWindow`: +`sessionIsIncomplete()` (the document's flag, from SVFileReader), `maySaveUnasked()` (the suspend +save's test), `askToSaveIncompleteSession()` before Save with a file, Save As (before the picker) +and Save in Audio Path; a save clears the flag. build-tony.sh allows libmediandk.so. +Choices / deviations: +- Encoder delay/padding kept: the decoder gets 0 for both (AOSP trims them otherwise, from + memory; its sources were unreachable). The user's .ton has Media Foundation's decode as 9352192 + = 9133 x 1024 frames, whole AAC frames: trimming would move the reference 2112 frames (48 ms) + against the saved pitch. The phone's log gives its count to compare. 16-bit output, as MF's. +- A file that fails is tried twice (by extension, then by every reader): logged twice. +Found: svapp gives an incomplete session no file (`m_sessionFile` ""), so A7's suspend save +already passed it by. With the reference missing, Save As waits for an analysis that never comes +("Waiting for analysis" until Cancel): not fixed; `waitForInitialAnalysis()` could pass then. +Tests seen failing: DecodedPcm compaction; `maySaveUnasked()`'s guard; Save As's question. +Left open: the decoder not run on a phone. SDK sources 36 installed in /opt/android/sdk. + From cad69933cae57d5899be01dde10c35b3ccb34fcc Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:54:31 +0000 Subject: [PATCH 192/275] docs: the joins check's comment says what a misplaced punch-in shows Two punch-ins placed differently do not show in the step at the join: each splice fades against silence, and the dip hides a jump of phase. Only the "second punch-in against the first" number shows it. The item numbers are the checklist's from before default rewrote it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- main/dev/DevChecks.cpp | 8 +++++--- main/dev/DevChecks.h | 4 +++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index 67b38f9b..eacb52e7 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -1931,9 +1931,11 @@ DevChecks::joinsCheck(QString reason) const // without a step, the pitch track without a hole or a frame twice, // one note runs through it, and outside the two ranges the take's // pitch and notes are as they were. Each part is named when it - // fails: two punch-ins placed differently, as on a device whose - // offset moves when its stream restarts, show in the step and the - // pitch, and the note merge may fail on its own + // fails. Two punch-ins placed differently, as on a device whose + // offset moves when its stream restarts, do not show in the step: + // each splice fades against the silence the file held there, so the + // join is a dip of a few ms that hides a jump of phase. Hence the + // number "second punch-in against the first", which does const sv_samplerate_t rate = stage.result.referenceRate; const LatencyCheck::PunchIn &first = s.punchIns[0].range; const LatencyCheck::PunchIn &second = s.punchIns[1].range; diff --git a/main/dev/DevChecks.h b/main/dev/DevChecks.h index 67adf70b..923ed1bb 100644 --- a/main/dev/DevChecks.h +++ b/main/dev/DevChecks.h @@ -48,7 +48,9 @@ struct CheckResult { enum class Verdict { Pass, Fail, Measured, Skipped }; - /// The item of docs/manual-checklist.md it settles, and its name + /// The item it settles, in the manual checklist's numbering from + /// before default's rewrite of it (docs/calibrate-audio.md explains), + /// and its name int item; QString name; From 0d0cdf1749ab0bda39fd832d56e23ed20e7b73ee Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 11:54:31 +0000 Subject: [PATCH 193/275] test: races with a ranged analysis are set up by holding its merge Five tests need the analysis of a just-stopped take still running when they act. On a faster machine pYIN finished a one-second range before analyseRange() returned, the merge was over inside Stop, and each test reported that its race was not set up. The tests' window now holds a finished ranged analysis's merge until the test lets it go (Analyser::setRangedMergeHeld(), never set in the application), so each race is set up whatever the machine's speed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/testing.md | 4 ++++ main/Analyser.cpp | 17 ++++++++++++++++- main/Analyser.h | 14 ++++++++++++++ main/test/TestMainWindow.h | 20 ++++++++++++++++++++ main/test/TestRecordWorkflow.h | 29 +++++++++++++++++++++++------ main/test/TestUiChecks.h | 9 +++++++++ 6 files changed, 86 insertions(+), 7 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index b67c55e1..8d3af75c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -178,6 +178,10 @@ it; the marker goes in the commit that fixes it. There are none at present. - A take's analysis lands in two steps, `rangedAnalysisMerged()` then `initialAnalysisCompleted()`. Read results after the merge (`analysingRange()` false), not after some other signal that happens to come at about the same time. +- pYIN may analyse a short recorded range before Stop returns, so nothing is being analysed + after it. A test that acts during that analysis calls `holdRangedMerges(true)` on its + `TestMainWindow` before Stop (`Analyser::setRangedMergeHeld()`), and `false` before it + waits for `analysed()`, or from a timer where a save's own wait has to let the merge go. - The status bar is written by three base-class timers; a test that reads it must go through what `showTakeCountdown()` controls. - Deleting a derived layer does not stop its transform; only diff --git a/main/Analyser.cpp b/main/Analyser.cpp index c0c0aabd..2eff40d2 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -67,7 +67,8 @@ Analyser::Analyser(ColorScheme colorScheme) : m_rangedEnd(0), m_rangedMergeStart(0), m_rangedMergeEnd(0), - m_rangedClippedEnd(false) + m_rangedClippedEnd(false), + m_rangedMergeHeld(false) { QSettings settings; settings.beginGroup("LayerDefaults"); @@ -1204,9 +1205,23 @@ Analyser::rangedAnalysisCompletionChanged(ModelId) // at 100 in both means a result if (!newPitch->isReady() || !newNotes->isReady()) return; + // Finished, but a test is holding it (setRangedMergeHeld()); never + // so in the application + if (m_rangedMergeHeld) return; + mergeRangedAnalysis(); } +void +Analyser::setRangedMergeHeld(bool held) +{ + m_rangedMergeHeld = held; + + // A run that finished while held gets no second completion signal: + // look now, as that signal would have + if (!held) rangedAnalysisCompletionChanged({}); +} + void Analyser::mergeRangedAnalysis() { diff --git a/main/Analyser.h b/main/Analyser.h index 2f90469c..ebe4c332 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -236,6 +236,17 @@ class Analyser : public QObject, */ void cancelRangedAnalysis() { discardRangedAnalysis(); } + /** + * For the tests only: while held, a ranged analysis that has + * finished is not merged and counts as running. A test that has to + * act during the analysis of a short range cannot count on it + * otherwise: pYIN may finish before analyseRange() returns, and the + * merge is then over with. Letting go merges a finished result at + * once, as its completion would have; one still running is merged + * when it finishes. Never held in the application. + */ + void setRangedMergeHeld(bool held); + /** * What the last ranged merge took out of and put into the pitch * track and the notes. Reversing these two changes undoes the @@ -400,6 +411,9 @@ protected slots: // cannot stamp anything before its own first two hops anyway bool m_rangedClippedEnd; + // Set only by a test (setRangedMergeHeld()) + bool m_rangedMergeHeld; + // What the last merge did, for the undo command of the recording // that asked for the analysis (see getRangedPitchChange()) TakeEvents::Change m_rangedPitchChange; diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index 0e4df6a6..51fe3f4d 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -98,6 +98,15 @@ class TestMainWindow : public MainWindow sv::sv_frame_t analysedRangeStart() { return m_takeAnalysisRange.start; } sv::sv_frame_t analysedRangeEnd() { return m_takeAnalysisRange.end; } + // Hold the merge of the analysis of each recorded range until let go, + // in the take's analyser and in every one made after it, so that a + // test can act while a range is being analysed: pYIN may analyse a + // short one before Stop returns (Analyser::setRangedMergeHeld()) + void holdRangedMerges(bool hold) { + m_holdRangedMerges = hold; + if (m_analyser2) m_analyser2->setRangedMergeHeld(hold); + } + // Save As, with the file name given here instead of by a dialog: the // session's own file is set, so that what is recorded next goes into // its takes folder @@ -255,6 +264,16 @@ class TestMainWindow : public MainWindow return m_takeNameAnswer == "" ? current : m_takeNameAnswer; } + // Every take analyser is made here, for a take's first recording and + // for each swap of its audio, before its range is analysed + void setupSingingTrackAnalyser(sv::ModelId singingModelId, + bool deferAnalysis = false) override { + MainWindow::setupSingingTrackAnalyser(singingModelId, deferAnalysis); + if (m_analyser2 && m_holdRangedMerges) { + m_analyser2->setRangedMergeHeld(true); + } + } + // The base class deleteAudioIO() deletes m_audioIO, which is right // for the fake as well @@ -267,6 +286,7 @@ class TestMainWindow : public MainWindow bool m_deleteTakeAnswer = true; int m_deleteTakeQuestions = 0; QString m_takeNameAnswer; + bool m_holdRangedMerges = false; }; #endif diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index d991f8af..86d228f0 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1806,6 +1806,11 @@ private slots: openReference(writeWav(tone(lowHz, 5.0))); if (QTest::currentTestFailed()) return; + // The merges are held until both takes have stopped: pYIN may + // analyse the first range before its Stop returns, and then there + // is nothing left to lose + m_window->holdRangedMerges(true); + startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(900); @@ -1822,11 +1827,9 @@ private slots: QVERIFY(firstEnd > sv::sv_frame_t(0.7 * rate)); // A second take in a gap, recorded without letting the event - // loop run: the result of a ranged analysis is merged from a - // queued call, so the first one cannot have finished by the time - // this one stops, however quick the machine is. (The device - // records from a thread of its own, and the record target's ring - // buffer holds ten seconds.) + // loop run, as it was before the merges could be held. (The + // device records from a thread of its own, and the record + // target's ring buffer holds ten seconds.) const sv::sv_frame_t P = sv::sv_frame_t(3.0 * rate); m_window->seekTo(P); startTake(); @@ -1834,7 +1837,7 @@ private slots: QThread::msleep(250); QVERIFY2(m_window->analysingRange(), "the first range's analysis finished before the second take " - "stopped: something ran the event loop"); + "stopped"); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); @@ -1843,6 +1846,7 @@ private slots: QCOMPARE(m_window->analysedRangeStart(), sv::sv_frame_t(0)); QVERIFY(m_window->analysedRangeEnd() > P); + m_window->holdRangedMerges(false); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); auto events = pitchEvents(m_window->analyser2()); QVERIFY2(!eventsBetween(events, 0, firstEnd).empty(), @@ -1864,6 +1868,11 @@ private slots: openReference(writeWav(tone(lowHz, 3.0))); if (QTest::currentTestFailed()) return; + // Held, so that each merge is still to come when its models go, + // however quickly pYIN analyses the range. Never let go: it is + // torn down each time + m_window->holdRangedMerges(true); + startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(900); @@ -5351,6 +5360,10 @@ private slots: openReference(writeWav(tone(lowHz, 2.0))); if (QTest::currentTestFailed()) return; + // Held, however quickly pYIN analyses the range, until the save + // runs the event loop + m_window->holdRangedMerges(true); + startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(700); @@ -5363,6 +5376,10 @@ private slots: "the test shows nothing: no analysis was running when the " "session was saved"); + // Let go from the event loop, which only a save that waits runs + QTimer::singleShot(0, m_window, [this]() { + m_window->holdRangedMerges(false); + }); QString session = m_dir.filePath("mid-analysis.ton"); QVERIFY(m_window->saveSessionFile(session)); diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 64d6b925..edffd983 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -1141,6 +1141,10 @@ private slots: QVERIFY(!select->isEnabled()); m_window->clearSelections(); + // The merge of the take's analysis is held until the menus have + // been looked at: pYIN may analyse the range before Stop returns + m_window->holdRangedMerges(true); + startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(200); @@ -1162,6 +1166,7 @@ private slots: "Erase can be used while the take is being analysed"); // ... and everything back, by itself + m_window->holdRangedMerges(false); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); QTRY_VERIFY2(erase->isEnabled(), "Erase did not come back after the analysis"); @@ -1422,6 +1427,10 @@ private slots: openReference(writeWav(tone(lowHz, 3.0))); if (QTest::currentTestFailed()) return; + // Held, so that the merge is still to come when the window goes, + // however quickly pYIN analyses the range + m_window->holdRangedMerges(true); + startTake(); if (QTest::currentTestFailed()) return; QTest::qWait(1000); From 49f2271c21e4334306bbd6cb028d5a8672d05abf Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:03:57 +0000 Subject: [PATCH 194/275] docs: work orders for phases a9 and a10 From the fourth phone test: the live dots lag seconds behind while recording, and the plot elements are a third of their desktop size. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 89 +++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 1a347613..f757b58a 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -163,6 +163,8 @@ builds happen in the container.) - A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. - A4b — Vertical zoom and scroll by touch. Done. - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. +- A9 — Live dots in real time on the phone. +- A10 — Plot elements sized for the screen. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -412,6 +414,93 @@ Opus (opusfile) are read. without the user asking: not by the save on suspend (A7), and Save warns first. Find where svapp reports "Incomplete session loaded" and how Tony can know it happened. +### A9 — Live dots in real time on the phone + +The fourth phone test (2026-09-26): the Avi Kaplan session and a direct `.m4a` open, but +while recording the orange live dots lag **several seconds** behind the singing on the +phone. On the desktop they keep up. Nothing else about recording was reported yet. + +What is known from the code (not measured): + +- `RealtimePitchTracker` (its own thread) emits `pitchDetected()` **once per hop**, 256 + frames, about 172 signals a second. Each is a queued call into + `MainWindow::onRealtimePitchDetected()`, which adds one point to the live model + (`SparseTimeValueModel::add`, a model change the pane repaints for) and sets the status + bar text. A GUI thread that needs more than about 5.8 ms per estimate falls behind for + good, and the lag grows for as long as the take lasts. +- The recorded audio reaches the model on the GUI thread too: svapp's + `AudioCallbackRecordTarget::updateModel()` every 10 ms (the fork's timeout; upstream about + 200 ms) writes to the file and calls `WritableWaveFileModel::updateModel()`, which closes + and reopens the file (`WavFileReader::updateFrameCount()`). The tracker can only see + what that has written. +- svgui's `View` paints layers into an image at `ceil(devicePixelRatio)`: 3 on the phone + (2.75), nine times the pixels of the desktop. The pane follows the playback cursor (20 ms + timer, svgui fork) while recording. +- `main.cpp` copies every `cerr` line to logcat and to the log file. + +Wanted: + +- **Measure first, in the container**: the desktop build, at `QT_SCALE_FACTOR=3` with a + window of a phone's logical size (about 400 x 850), a reference of a few minutes with its + pitch track and notes, recording through `FakeAudioIO`. Find what the GUI thread spends + per estimate, per record update, and per paint of the pane, and whether the tracker's + thread itself keeps up. A phone core is several times slower than this container's: + say which costs would scale to a lag. +- **Fix so the lag is bounded by design**, whatever the phone's speed: the dots come to the + GUI thread in batches (everything the tracker found since the last one) at a paced rate, + go into the model together, and the status bar is set once per batch. If the GUI thread + is slow, the dots arrive later in bigger batches, but never a growing queue behind. Fix + any other per-estimate or per-update cost the measurement shows, the simpler way. +- **A log line once a second while recording**, so the phone's log (Help > Save Log...) + says whether it holds: seconds recorded, seconds the tracker has reached, seconds of + dots drawn, and the GUI-side costs measured above (e.g. slot time, paint time of the + pane, largest and average). Keep it to one line a second. +- A test that fails with the per-estimate design: e.g. a GUI thread made slow on purpose + in the test, and the newest dot must stay within a bound of the recording. +- If the measurement points into svgui (the pane repaints its whole image for each point, + say), you may change the fork `svgui/` for it: on its branch `tony-customizations`, + committed there with its own style (`view: what`), not pushed; the lead pushes it and + pins it. `svcore/` and `svapp/`: report the change, do not make it. + +### A10 — Plot elements sized for the screen + +The same phone test: the pitch tracks are thin lines and the notes thin bars, hard to make +out, while the rest of the GUI is sized for the phone. A4b's vertical zoom spreads the +pitch but does not thicken what is drawn. The user: "the GUI elements have been adequately +resized to fit the higher DPI resolution of the mobile screen but the graph plot elements +have not." + +What is known from the code: svgui's `View` renders the layers into an image at +`ceil(devicePixelRatio)` (3 on the phone) through a `ViewProxy` whose coordinates are in +those physical pixels. Sizes that the layers give in pixels are then physical pixels: +`TimeValueLayer`'s `PlotPoints` (Tony's pitch tracks and the live dots) draws each point as +`drawRect(x, y - 1, w, 2)`; `FlexiNoteLayer` draws notes `NOTE_HEIGHT` (16) high; and +`ViewProxy::scalePenWidth()` scales pens by only the square root of the ratio. So on the +phone a pitch point is about a third, and a note a third, of its desktop height in logical +pixels. + +Wanted: + +- What Tony draws in its panes (the reference and singing pitch tracks, the live dots, the + notes, the coverage strip, anything else of Tony's that looks too thin) keeps its size + in **logical pixels** at any pixel ratio: at ratio 3 it is three times as many physical + pixels as at ratio 1. Note editing (hit areas that use `NOTE_HEIGHT`) must match what is + drawn. +- A **plot size** setting for making them bigger still: View menu, a few steps (say 100%, + 150%, 200%), remembered in `QSettings`, applied at once. Default 100% on the desktop, + 150% on Android. The same on both platforms otherwise. +- The desktop at ratio 1 and 100% draws **exactly as before**: prove it with a test that + renders the layers into images before/after, or equivalent. +- The change is in the fork `svgui/`, which you may edit for this phase: on its branch + `tony-customizations`, as small and general as it can be (e.g. a plot scale in + `ViewManager` that `View`/`ViewProxy` apply, and the layers using `scalePixelSize()` for + their hard-coded sizes), committed there with its own style (`layer: what`, + `view: what`), not pushed; the lead pushes it and pins it. Tony's side (the setting, the + menu) in `main/`. Say which svgui layers change on a hi-DPI desktop (Windows at 150% or + 200%), since they do too. +- Tests: the sizes at ratio 1 and 3 and the setting's steps, in images rendered offscreen + (`QT_SCALE_FACTOR` or a `QImage` with a device pixel ratio). + ### A8 — Documentation pass - Bring the docs pages up to date from the code and the log: building.md (the container From b7293835d094899564aff506e385b754a5e42c10 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:18:26 +0000 Subject: [PATCH 195/275] docs: plan for sharded test runs on windows The test runner shards the suites over processes on Linux only, where each process gets a HOME of its own. On Windows settings, data and temp directories are keyed by the application name, so the plan gives each shard an application name of its own, on both platforms. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0137vMTCch66TVceRM316MAF --- docs/README.md | 1 + docs/windows-shards.md | 134 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 135 insertions(+) create mode 100644 docs/windows-shards.md diff --git a/docs/README.md b/docs/README.md index 905cce37..a394ae9b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,3 +20,4 @@ methods, and they do not tell the story of fixed bugs. | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | | [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | | [calibrate-audio.md](calibrate-audio.md) | Plan, not built: a Calibrate Audio button that measures the round trip through a speaker-to-mic loopback, and dev checks that automate most of the manual checklist | +| [windows-shards.md](windows-shards.md) | Plan, not built: sharded test runs from Git Bash on Windows, each shard kept apart by an application name of its own | diff --git a/docs/windows-shards.md b/docs/windows-shards.md new file mode 100644 index 00000000..b71b7037 --- /dev/null +++ b/docs/windows-shards.md @@ -0,0 +1,134 @@ +# Sharded test runs on Windows: plan + +Plan, not built. For a Linux cloud session to carry out; the last section is for the +Windows machine afterwards. When all of it is done, what lasts goes into +[testing.md](testing.md) and [AGENTS.md](../AGENTS.md), and this file and its row in +[README.md](README.md) are deleted. + +## Goal + +On Linux the test executables already run as shards in parallel processes +(`TONY_TEST_SHARD`, `main/test/RunSuite.h`, `deploy/linux/run-tests.sh`): the app suite +takes a minute and a half instead of eight. On the Windows machine the same runs are still +one process each, about 13.5 minutes for core, app and dev together. Make the runner work +from Git Bash on Windows, with one way of keeping shards apart that serves both platforms. + +## Measured on Windows (2026-09-26, one process) + +| Suite | Tests | Wall | CPU | +| --- | --- | --- | --- | +| `TestRecordWorkflow` | 165 | 430 s | 164 s | +| `TestAudioCheck` | 23 | 185 s | 69 s | +| `TestUiChecks` | 19 | 116 s | 56 s | +| the other four app suites | 60 | 11 s | 6 s | +| whole `test-tony-app` | | 743 s | 295 s | +| `test-tony-dev` | 9 | 67 s | | +| `test-tony-core` | | 17 s | | + +The app suite uses 0.4 of one core on average (Linux: nearer 0.25); the machine has four +cores and no hyper-threading. So there is room for several processes at once, fewer than on +Linux. + +## Why the Linux runner does not work on Windows + +Processes sharing a settings store clear each other's settings: `initTestCase()` of +`TestRecordWorkflow` calls `QSettings().clear()`, and the shards of `test-tony-dev` failed +on Linux in exactly this way until each got a `HOME` of its own. `run-tests.sh` gives every +process its own `HOME` and XDG directories. + +On Windows neither moves anything. QSettings in its native format is the registry key +`HKCU\Software\tony-tests\`, and `QStandardPaths`, which svcore's +`ResourceFinder` uses, asks Windows for its known folders instead of reading environment +variables. These are also shared by processes of one application name: + +- `AppDataLocation`, where `AudioCheckRunner` and `DevChecks` keep files. +- svcore's `TempDirectory`, under the user resource prefix. At start-up it deletes every + `sv_*` directory with no `.pid` file in it, which includes a directory another process + has created a moment ago and not yet written its `.pid` into. +- svcore's debug log, under the same prefix. + +All of them are keyed by the application name. So a shard runs under an application name +of its own, on both platforms. + +## Design (decided) + +1. **`RunSuite.h`**: a pure function that gives the application name for a shard, taking + the base name and the value of `TONY_TEST_SHARD`. No value: the base name, unchanged. A + valid `i/n`: `-shardof`. An invalid value: the base name (`runSuite()` + already refuses to run with it). One function parses `i/n` for both this and + `runSuite()`, so that the two cannot disagree. +2. **The mains** that run suites, `tony-core-test.cpp`, `tony-app-test.cpp` and + `tony-dev-test.cpp`, set the application name through that function instead of the + literal, right after constructing the application and before anything reads settings. + `tony-device-check.cpp` runs by hand against a real device and is never sharded: leave + it. +3. **`deploy/linux/run-tests.sh`**: drop the per-process `HOME` and XDG directories, so + that Linux runs rely on the same thing Windows will, and keep proving it. Keep the + per-process log directories. The script stays where it is. Its header comment says it + runs from Git Bash on Windows too, with the environment AGENTS.md gives and executable + names with `.exe`. Use nothing Git for Windows' bash lacks: `nproc`, `seq`, `awk`, + `grep` and `xargs` are there. Leave the default `-j` (twice the cores) alone; the + Windows step sets its own. + +Not part of this: sharded `meson test` definitions (`meson test` and `build.bat test` stay +the one-process run that [testing.md](testing.md) asks for after changes to lifetimes, +threads or teardown), a PowerShell runner, any change in the library forks. If the work +seems to need a fork change (for instance to `TempDirectory`), do not make it: report it. + +## Steps for the cloud session + +Read AGENTS.md, then [testing.md](testing.md) (Running, Shards, the Linux failures, Timing +and races), `run-tests.sh`, `RunSuite.h` and `TestRunSuite.h`. Build as +[building.md](building.md#building-on-linux) says. + +1. **Baseline.** Run `run-tests.sh` for `test-tony-core`, `test-tony-app` and + `test-tony-dev` as it is, and record what fails. Only tests from testing.md's list of + Linux failures should. +2. **The failure this prevents, before the change.** Copy the script to `tmp/` (not + committed) and make every process share one `HOME` and one set of XDG directories. + That is the Windows situation on Linux: nothing but the application name can then keep + shards apart. Run it for `test-tony-dev` and `test-tony-app` and record what fails. If + nothing does, run it three times. If still nothing fails, say so in the report: step 4 + then proves less. +3. **Build design 1 and 2**, with tests in `TestRunSuite`: + - no shard gives exactly the base name; + - every `i` of one `n` gives a different name; + - the same `i` with a different `n` gives a different name; + - an invalid value gives the base name. + + Break the function for a moment and see a test fail (AGENTS.md), then put it back. +4. **Proof.** The shared-`HOME` script from step 2 must now pass three runs in a row for + `test-tony-dev` and `test-tony-app`, apart from the known Linux failures. If it does + not, stop there and report what failed: Windows would fail the same way, and a + per-process `HOME` cannot help it there. +5. **Design 3**, then run core, app and dev through the real script three times each. +6. **One-process runs** of all three executables, as AGENTS.md gives them, from `build/`: + the path without `TONY_TEST_SHARD` is the one `meson test` and named tests use. +7. **Docs.** [testing.md](testing.md)'s Shards paragraph: shards are kept apart by their + application name (settings, data directory, temp directories), on Linux and Windows; + remove "so the script is for Linux". Fix anything else the change makes false + ([building.md](building.md#building-on-linux) mentions the script). Leave AGENTS.md's + Windows commands and times alone: they need measuring on Windows. +8. **Commits and PR.** One commit per step as AGENTS.md says. For example: the shard's + application name with its tests (`test:`), the script (`test:`), the docs (`docs:`). + Push the session's branch and open a PR against `default` with `gh`. Report as + AGENTS.md asks: the Totals of the final runs, the failures seen in steps 1 and 2, and + anything that did not go as planned. + +## On the Windows machine, after the merge + +From Git Bash with AGENTS.md's environment, after a build: + +1. `deploy/linux/run-tests.sh -j N build_mingw test-tony-app.exe` for N = 4, 6 and 8, + watching CPU in Task Manager. Take the largest N that passes three runs in a row with + the CPU clearly below full; tests that race the analysis are the ones to watch. +2. The same for `test-tony-dev.exe` and `test-tony-core.exe`. +3. Check that each shard left its own key under `HKCU\Software\tony-tests` and its own + folders under `%APPDATA%` and `%LOCALAPPDATA%` (`qttest` included, for the two suites + that turn on `QStandardPaths`' test mode). There should be one per `i` and `n`, so their + number stays bounded. +4. AGENTS.md: the full Windows run goes through `run-tests.sh` with that N and the + measured times, and gets a tool timeout to match. The one-process command stays, for + named tests and for changes to lifetimes, threads or teardown. Update the times in + [testing.md](testing.md) too. +5. Delete this file and its row in [README.md](README.md). From ba2f6323174081d0937daaec8daf762e47bdd5c9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:29:59 +0000 Subject: [PATCH 196/275] docs: a9 re-scoped after the merge of default Default's throttle and cache exclusion are in; A9 measures what is left at the phone's pixel ratio and batches the dots. Fork work goes on the fork's feat/tonyandroid branch; the sharded runner is described. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 46 +++++++++++++++++++++++++++++-------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index f757b58a..90c1b3f8 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -69,6 +69,13 @@ first. grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt ``` + For the whole suites, `default`'s sharded runner is quicker (the app suite in about a + minute and a half) and prints a per-suite summary with every failure; from the repo + root: `deploy/linux/run-tests.sh test-tony-core > tmp/core-run.log 2>&1` and the same + for `test-tony-app` and `test-tony-dev` (the dev checks' suite, since the merge of + `default`; build it with the others). Under its load `stale_pitch_event_ignored` failed + once at `QVERIFY(model)` and passed alone and on the next run. + No `.exe` on Linux; the plugin target is `pyin.so`. Tony needs Qt 6.5 or later at run time (string connects with `sv::` types, see the A0 log entry). - Send build output to a log file with the exit status written into it; look at the tail @@ -127,6 +134,13 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: ## 4. State of the code (kept by the lead; as of 2026-09-25, after phase A1) +- 2026-09-26, after A7c: `default` merged in (`fb5fa6f`): lyrics, Calibrate Audio (a + measured round trip, `roundTripAt()`, which converts through seconds per rate), the dev + checks (`test-tony-dev`; compiled into non-release builds, the Android one included), the + sharded test runner, the live dot throttle and cache exclusion. svgui is pinned at + `c685b97`, checked out as its branch `feat/tonyandroid`. The test window class is in + `main/test/TestMainWindow.h`. All three suites green; the Android build links. + - The branch holds `default`, the research docs, the container setup script `deploy/linux/container-setup.sh`, and A1's fix. - Both suites pass on Linux (Qt 6.11.2 from conda-forge). `TestTakesFile`'s Windows path @@ -420,14 +434,22 @@ The fourth phone test (2026-09-26): the Avi Kaplan session and a direct `.m4a` o while recording the orange live dots lag **several seconds** behind the singing on the phone. On the desktop they keep up. Nothing else about recording was reported yet. +That APK was built before `default` was merged in (2026-09-26, `fb5fa6f`). `default` had +fixed the same symptom on the desktop in two steps: `ModelChangeThrottle` tells the pane of +new dots at most every 40 ms instead of once per dot, and the dots layer is kept out of the +pane's cache (`Layer::setCachedInView`, svgui fork), so a notice no longer has the pane draw +the reference's waveform, pitch track and notes again (see `docs/recording.md` and the +messages of `2a20de5` and `7fb4174`: GUI thread from ~80 % to ~22 % of a core in a +1920 px window on this container). Nobody has measured that on a phone. So: + What is known from the code (not measured): -- `RealtimePitchTracker` (its own thread) emits `pitchDetected()` **once per hop**, 256 - frames, about 172 signals a second. Each is a queued call into - `MainWindow::onRealtimePitchDetected()`, which adds one point to the live model - (`SparseTimeValueModel::add`, a model change the pane repaints for) and sets the status - bar text. A GUI thread that needs more than about 5.8 ms per estimate falls behind for - good, and the lag grows for as long as the take lasts. +- `RealtimePitchTracker` (its own thread) still emits `pitchDetected()` **once per hop**, + 256 frames, about 172 signals a second. Each is a queued call into + `MainWindow::onRealtimePitchDetected()`, which adds one point to the live model, tells + the throttle, and sets the status bar text. A GUI thread that needs more than about + 5.8 ms per estimate falls behind for good, and the lag grows for as long as the take + lasts. - The recorded audio reaches the model on the GUI thread too: svapp's `AudioCallbackRecordTarget::updateModel()` every 10 ms (the fork's timeout; upstream about 200 ms) writes to the file and calls `WritableWaveFileModel::updateModel()`, which closes @@ -449,8 +471,10 @@ Wanted: - **Fix so the lag is bounded by design**, whatever the phone's speed: the dots come to the GUI thread in batches (everything the tracker found since the last one) at a paced rate, go into the model together, and the status bar is set once per batch. If the GUI thread - is slow, the dots arrive later in bigger batches, but never a growing queue behind. Fix - any other per-estimate or per-update cost the measurement shows, the simpler way. + is slow, the dots arrive later in bigger batches, but never a growing queue behind. Keep + `default`'s throttle and cache exclusion; fold the throttle into the batching if that is + simpler, saying so. Fix any other per-estimate or per-update cost the measurement shows, + the simpler way. - **A log line once a second while recording**, so the phone's log (Help > Save Log...) says whether it holds: seconds recorded, seconds the tracker has reached, seconds of dots drawn, and the GUI-side costs measured above (e.g. slot time, paint time of the @@ -458,7 +482,8 @@ Wanted: - A test that fails with the per-estimate design: e.g. a GUI thread made slow on purpose in the test, and the newest dot must stay within a bound of the recording. - If the measurement points into svgui (the pane repaints its whole image for each point, - say), you may change the fork `svgui/` for it: on its branch `tony-customizations`, + say), you may change the fork `svgui/` for it: on its branch `feat/tonyandroid` + (checked out; fork work for a Tony branch goes on a fork branch of the same name), committed there with its own style (`view: what`), not pushed; the lead pushes it and pins it. `svcore/` and `svapp/`: report the change, do not make it. @@ -492,7 +517,8 @@ Wanted: - The desktop at ratio 1 and 100% draws **exactly as before**: prove it with a test that renders the layers into images before/after, or equivalent. - The change is in the fork `svgui/`, which you may edit for this phase: on its branch - `tony-customizations`, as small and general as it can be (e.g. a plot scale in + `feat/tonyandroid` (checked out; fork work for a Tony branch goes on a fork branch of + the same name), as small and general as it can be (e.g. a plot scale in `ViewManager` that `View`/`ViewProxy` apply, and the layers using `scalePixelSize()` for their hard-coded sizes), committed there with its own style (`layer: what`, `view: what`), not pushed; the lead pushes it and pins it. Tony's side (the setting, the From 40ce2f5e35b72ef8ac7cadbb4dab4ec3e574030c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:54:09 +0000 Subject: [PATCH 197/275] test: the app and dev suites' qtest timeout is meson's own qt 6.4's qtest watchdog can miss the end of a quick test function and then times out whichever one is running five minutes after, so since the merge of the lyrics, whose tests take testrecordworkflow past five minutes, a random test there failed with "test function timed out" at about 300 s of the suite and the process hung until meson killed it: in two of three local runs with ubuntu's qt, which linux ci uses. with a two-minute timeout it struck reload_singing_track at 120.6 s. meson.build now sets QTEST_FUNCTION_TIMEOUT to meson's 900 s for both suites, and testing.md says so. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- docs/testing.md | 6 ++++++ meson.build | 8 ++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/testing.md b/docs/testing.md index 929dde9c..c2faedcc 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -72,6 +72,12 @@ Windows path would start an escape in the C string. paths (`C:\...`, case-insensitive) on Windows only, and the tests that race the analysis of a take hold it ("Timing and races"): on a fast machine they used to fail with "the race was not set up". +- **Qt 6.4's QTest watchdog can time out a test that did not take long.** It can miss the + end of a quick test function and then times out whichever one is running five minutes + later: a suite that runs longer than that (`TestRecordWorkflow` does) fails at random + with "Test function timed out", its totals at about 300 s, and the process then hangs. + `meson.build` sets `QTEST_FUNCTION_TIMEOUT` to meson's own timeout for the app and dev + suites; set it by hand for a one-process run of those built against that Qt. - CI runs every suite on Linux (Ubuntu 24.04, Qt 6.4), macOS and Windows (MSYS2), one suite at a time. When a run fails, its `test-failures` step lists each failed test with the lines QTest indents under it, from meson's full log. diff --git a/meson.build b/meson.build index bc387a8d..63cdd7e7 100644 --- a/meson.build +++ b/meson.build @@ -1511,10 +1511,14 @@ tony_device_test_exe = executable( ) test('tony-core', tony_core_test_exe) +# QTEST_FUNCTION_TIMEOUT as long as meson's own timeout: Qt 6.4's watchdog +# can miss the end of a quick test function and time out whichever one +# is running five minutes later, so a suite longer than that fails at +# random, and the process then hangs until meson kills it test('tony-app', tony_app_test_exe, depends: pyin_plugin, timeout: 900, - env: [ 'QT_QPA_PLATFORM=offscreen' ]) + env: [ 'QT_QPA_PLATFORM=offscreen', 'QTEST_FUNCTION_TIMEOUT=900000' ]) # The development checks' suite, only where the checks are built. Its # own executable: every test records takes in real time, and it is run @@ -1554,7 +1558,7 @@ if dev_checks test('tony-dev', tony_dev_test_exe, depends: pyin_plugin, timeout: 900, - env: [ 'QT_QPA_PLATFORM=offscreen' ]) + env: [ 'QT_QPA_PLATFORM=offscreen', 'QTEST_FUNCTION_TIMEOUT=900000' ]) endif test('svcore-base', svcore_base_test_exe) From c459b8a37ba97e90c325138f23015439958f9179 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:54:09 +0000 Subject: [PATCH 198/275] test: a singing analysis test waits for the transform to go analyse() returned at the first initialAnalysisCompleted(), which comes with the first of the two outputs' notices at 100. the other's can still be queued behind it, and ranged_cancelled counted it as a late merge after its cancel: twice on linux ci. analyse() now also waits for the factory to let the transform go, which it does only after its notices have been delivered. not seen failing here, loaded or not. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- main/test/TestSingingAnalysis.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/main/test/TestSingingAnalysis.h b/main/test/TestSingingAnalysis.h index 02129077..c58d2507 100644 --- a/main/test/TestSingingAnalysis.h +++ b/main/test/TestSingingAnalysis.h @@ -37,6 +37,7 @@ #include "data/model/WritableWaveFileModel.h" #include "data/model/SparseTimeValueModel.h" #include "data/model/NoteModel.h" +#include "transform/ModelTransformerFactory.h" #include "base/PlayParameters.h" #include @@ -174,6 +175,14 @@ class TestSingingAnalysis : public QObject QVERIFY(analyser.getLayer(Analyser::Notes)); QVERIFY2(done.count() > 0 || done.wait(30000), "pYIN did not complete within 30 seconds"); + + // and for the transform to be gone. The first notice at 100 comes + // from one of the two outputs, and the other's can still be + // queued behind it: a test would count that as a second + // initialAnalysisCompleted(), long after pYIN finished. The + // factory lets a transform go only after its notices are in + QTRY_VERIFY_WITH_TIMEOUT(!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); } sv::ModelId addSingingModel(const std::vector &data) { From 263dcffd275f255495efd1e975abb94f2592cec4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:54:09 +0000 Subject: [PATCH 199/275] test: the lyrics failure dialogs are matched by their text as well lyrics_import_failure and lyrics_export_failure found their dialog by its title, which macos neither shows nor keeps on a message box, and failed there. messagesMatching() matches by the text, and by the title too where there is one. seen failing on linux with the import error's title taken away. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- main/test/TestRecordWorkflow.h | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 2bdcd86f..62b530df 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -428,6 +428,22 @@ class TestRecordWorkflow : public QObject return matching; } + // As dialogsMatching(), for message boxes with this title as well. + // Not on macOS, which shows no title on a message box, and Qt keeps + // none there: the text alone must tell the box apart + QStringList messagesMatching(QString title, QString text) { + QStringList matching; + for (const QString &dialog : dialogsMatching(text)) { +#ifdef Q_OS_MACOS + Q_UNUSED(title); + matching.push_back(dialog); +#else + if (dialog.startsWith(title + ": ")) matching.push_back(dialog); +#endif + } + return matching; + } + // Audio in the session besides the reference: one model per take that // is on show, and nothing left over from a session load int audioModelsBesidesReference() { @@ -6278,7 +6294,8 @@ private slots: for (const auto &f : failures) { QVERIFY2(!m_window->doImportLyricsFrom(f.first), qPrintable(f.first)); - QStringList dialogs = dialogsMatching("Could not import lyrics"); + QStringList dialogs = + messagesMatching("Could not import lyrics", f.second); QCOMPARE(dialogs.size(), 1); QVERIFY2(dialogs[0].contains(f.second), qPrintable(dialogs[0])); QVERIFY(!lyrics->isShown()); @@ -6293,7 +6310,8 @@ private slots: m_window->discardModifications(); QVERIFY(!m_window->doImportLyricsFrom(untimed)); - QCOMPARE(dialogsMatching("Could not import lyrics").size(), 1); + QCOMPARE(messagesMatching("Could not import lyrics", + "No timed lyrics were found").size(), 1); QVERIFY(lyrics->getModelId() == model); QCOMPARE(lyricsEvents(), events); QCOMPARE(lyricsLayersInDocument(), 1); @@ -6565,7 +6583,8 @@ private slots: for (QString path : { noFolder, folder }) { m_window->setLyricsExportAnswer(path); m_window->exportLyricsAction()->trigger(); - QStringList dialogs = dialogsMatching("Could not export lyrics"); + QStringList dialogs = + messagesMatching("Could not export lyrics", path); QCOMPARE(dialogs.size(), 1); QVERIFY2(dialogs[0].contains(path), qPrintable(dialogs[0])); QCOMPARE(m_window->statusText(), status); From 991095069a78fccfd7cddbb2458355d9ac0c72f0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:54:09 +0000 Subject: [PATCH 200/275] test: windows ci lists failed tests from the suites' result files on windows qtest's output to stdout did not reach meson's log: tony-app said "1 test suite(s) failed!" and the list of failures was empty. the test step sets TONY_TEST_LOG_DIR, so the suites write their results to files as well, and the failure step reads those after meson's log. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/windows.yml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/workflows/windows.yml b/.github/workflows/windows.yml index f38c0640..6eb6171f 100644 --- a/.github/workflows/windows.yml +++ b/.github/workflows/windows.yml @@ -67,11 +67,15 @@ jobs: export MINGW_PREFIX=$(cygpath -m /mingw64) ninja -C build + # Tony's suites also write their results to files: here QTest's + # output to stdout does not reach meson's log - name: test id: test shell: msys2 {0} run: | export MINGW_PREFIX=$(cygpath -m /mingw64) + mkdir -p build/test-results + export TONY_TEST_LOG_DIR=$(cygpath -m "$PWD/build/test-results") meson test -C build --print-errorlogs --num-processes 1 # meson shows at most the last 100 lines of a failing log, and one # test executable runs several QTest suites: name every failed test, @@ -86,3 +90,12 @@ jobs: /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } /^[A-Z!]+ *: / { fail = 0 } fail && /^ / { print }' build/meson-logs/testlog.txt + awk 'function show(line) { + if (!shown) { print "suite: " FILENAME; shown = 1 } + print line + } + FNR == 1 { shown = 0; fail = 0 } + /^Totals:/ { if ($0 !~ /, 0 failed/) show($0); next } + /^(FAIL!|XPASS|QFATAL)/ { show($0); fail = 1; next } + /^[A-Z!]+ *: / { fail = 0 } + fail && /^ / { show($0) }' build/test-results/*.txt From dbce0cb5528050e10495caa11581d2b5c5549542 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:56:40 +0000 Subject: [PATCH 201/275] test: each shard runs under an application name of its own Settings, the data location and svcore's temp directory and log are all keyed by the application name. On Windows they are the registry and known folders, which no environment variable moves, so a HOME per process cannot keep shards apart there: the name can, on both platforms. One parse of TONY_TEST_SHARD serves the name and runSuite(), and now refuses parts that are not whole numbers, which used to run as shard 0. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0137vMTCch66TVceRM316MAF --- main/test/RunSuite.h | 52 ++++++++++++++++++++++-- main/test/TestRunSuite.h | 79 +++++++++++++++++++++++++++++++++++- main/test/tony-app-test.cpp | 7 +++- main/test/tony-core-test.cpp | 7 +++- main/test/tony-dev-test.cpp | 7 +++- 5 files changed, 141 insertions(+), 11 deletions(-) diff --git a/main/test/RunSuite.h b/main/test/RunSuite.h index ad2e7b4e..c8bcb191 100644 --- a/main/test/RunSuite.h +++ b/main/test/RunSuite.h @@ -50,6 +50,50 @@ shardFunctions(const QObject *suite, int shard, int count) return names; } +/** + * Read a value of TONY_TEST_SHARD, "i/n". True only for two whole + * numbers with 0 <= i < n, which are then set in \a shard and \a count. + * runSuite() and shardApplicationName() both read the value through + * this, so that they cannot disagree on what is a shard. + */ +inline bool +parseShard(const QString &value, int &shard, int &count) +{ + QStringList parts = value.split('/'); + if (parts.size() != 2) { + return false; + } + bool iok = false, nok = false; + int i = parts[0].toInt(&iok); + int n = parts[1].toInt(&nok); + if (!iok || !nok || n < 1 || i < 0 || i >= n) { + return false; + } + shard = i; + count = n; + return true; +} + +/** + * The application name for a process of the test executable \a base + * that runs the shard \a shard (the value of TONY_TEST_SHARD): + * "-shardof", or \a base itself for no value or one that + * is not a shard. Settings, QStandardPaths' data location and svcore's + * temp directory and log are all keyed by the application name, and on + * Windows, where they are the registry and known folders, no + * environment variable moves them: so the name is all that keeps shards + * running at once from clearing each other's settings and files. + */ +inline QString +shardApplicationName(const QString &base, const QString &shard) +{ + int i = 0, n = 0; + if (!parseShard(shard, i, n)) { + return base; + } + return QString("%1-shard%2of%3").arg(base).arg(i).arg(n); +} + /** * Run one suite with the command-line arguments given. If the * environment variable TONY_TEST_LOG_DIR is set, the suite's results @@ -61,6 +105,8 @@ shardFunctions(const QObject *suite, int shard, int count) * whole suite between them in about 1/n of the time: most tests wait * on a fake device playing in real time. A suite with nothing in the * shard is not run. Not for use with test names on the command line. + * The mains give each shard an application name of its own: + * shardApplicationName(). */ inline bool runSuite(QObject *suite, int argc, char *argv[]) @@ -71,10 +117,8 @@ runSuite(QObject *suite, int argc, char *argv[]) } QString shard = qEnvironmentVariable("TONY_TEST_SHARD"); if (shard != "") { - QStringList parts = shard.split('/'); - int n = (parts.size() == 2 ? parts[1].toInt() : 0); - int i = (parts.size() == 2 ? parts[0].toInt() : -1); - if (n < 1 || i < 0 || i >= n) { + int i = 0, n = 0; + if (!parseShard(shard, i, n)) { qWarning("TONY_TEST_SHARD must be i/n with 0 <= i < n, not \"%s\"", qPrintable(shard)); return false; diff --git a/main/test/TestRunSuite.h b/main/test/TestRunSuite.h index 5d2a5c65..c60cc926 100644 --- a/main/test/TestRunSuite.h +++ b/main/test/TestRunSuite.h @@ -16,7 +16,8 @@ // Which test functions a shard of a suite runs (TONY_TEST_SHARD): each // exactly once over the shards, and nothing QTest would not run as a -// test. +// test. And the application name a shard's process runs under: one of +// its own, which is what keeps shards running at once apart. #include "RunSuite.h" @@ -93,6 +94,82 @@ private slots: QCOMPARE(shardFunctions(&fixture, 4, 6), QStringList({ "fifth" })); QVERIFY(shardFunctions(&fixture, 5, 6).isEmpty()); } + + // A run that is not sharded keeps the executable's own name, the one + // meson test and named tests have always used + void no_shard_is_the_base_name() { + QCOMPARE(shardApplicationName("test-tony-app", QString()), + QString("test-tony-app")); + QCOMPARE(shardApplicationName("test-tony-app", ""), + QString("test-tony-app")); + } + + // The name says which shard of how many, after the base name + void a_shard_name_says_which_shard() { + QCOMPARE(shardApplicationName("test-tony-app", "3/8"), + QString("test-tony-app-shard3of8")); + } + + // The shards of one run are processes at once: none may share a name + void every_shard_of_a_count_has_its_own_name() { + for (int count = 1; count <= 8; ++count) { + QStringList names; + for (int shard = 0; shard < count; ++shard) { + names << shardName(shard, count); + } + QVERIFY2(QSet(names.begin(), names.end()).size() == count, + qPrintable(QString("%1 shards are named %2") + .arg(count).arg(names.join(", ")))); + } + } + + // Shard i of another count runs other tests, so it is another name + void the_same_shard_of_another_count_has_another_name() { + for (int count = 1; count <= 8; ++count) { + for (int other = count + 1; other <= 8; ++other) { + for (int shard = 0; shard < count; ++shard) { + QString name = shardName(shard, count); + QVERIFY2(shardName(shard, other) != name, + qPrintable(QString("%1/%2 and %1/%3 are both %4") + .arg(shard).arg(count).arg(other) + .arg(name))); + } + } + } + } + + // runSuite() refuses to run with a value that is not a shard, and + // the name is then the base name, as for no value + void an_invalid_shard_is_the_base_name() { + for (QString value : { "x", "2/2", "-1/2", "0/0", "1/2/3", "/" }) { + QString name = shardApplicationName("test-tony-app", value); + QVERIFY2(name == "test-tony-app", + qPrintable(QString("\"%1\" gives %2") + .arg(value).arg(name))); + } + } + + // Both parts must be whole numbers. Otherwise a slip such as "1x/2" + // would be read as shard 0, and run it a second time under its name + void a_shard_is_two_whole_numbers() { + int shard = -1, count = -1; + QVERIFY(parseShard("3/8", shard, count)); + QCOMPARE(shard, 3); + QCOMPARE(count, 8); + for (QString value : { "a/2", "/2", "1x/2", "1/x", "1/2x", "1.0/2" }) { + QVERIFY2(!parseShard(value, shard, count), + qPrintable(QString("\"%1\" taken as %2/%3") + .arg(value).arg(shard).arg(count))); + QCOMPARE(shardApplicationName("test-tony-app", value), + QString("test-tony-app")); + } + } + +private: + static QString shardName(int shard, int count) { + return shardApplicationName + ("test-tony-app", QString("%1/%2").arg(shard).arg(count)); + } }; #endif diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index 0de3c289..02905e3c 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -46,10 +46,13 @@ int main(int argc, char *argv[]) } // Names distinct from the application's, so that nothing here reads - // or writes the user's real Tony settings + // or writes the user's real Tony settings, and a shard's distinct + // from the other shards', so that shards running at once keep apart QApplication app(argc, argv); app.setOrganizationName("tony-tests"); - app.setApplicationName("test-tony-app"); + app.setApplicationName(shardApplicationName + ("test-tony-app", + qEnvironmentVariable("TONY_TEST_SHARD"))); // Text in shades of grey, whatever the machine's fontconfig asks for. // Ubuntu's asks for sub-pixel anti-aliasing, and Qt 6.4 follows it: diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index bab0dd72..cf15ebd7 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -47,10 +47,13 @@ int main(int argc, char *argv[]) svSystemSpecificInitialisation(); // Names distinct from the application's, so that nothing here reads - // or writes the user's real Tony settings + // or writes the user's real Tony settings, and a shard's distinct + // from the other shards', so that shards running at once keep apart QCoreApplication app(argc, argv); app.setOrganizationName("tony-tests"); - app.setApplicationName("test-tony-core"); + app.setApplicationName(shardApplicationName + ("test-tony-core", + qEnvironmentVariable("TONY_TEST_SHARD"))); { TestRealtimeYin t; diff --git a/main/test/tony-dev-test.cpp b/main/test/tony-dev-test.cpp index 033f81b3..582a7858 100644 --- a/main/test/tony-dev-test.cpp +++ b/main/test/tony-dev-test.cpp @@ -42,10 +42,13 @@ int main(int argc, char *argv[]) } // Names distinct from the application's, so that nothing here reads - // or writes the user's real Tony settings + // or writes the user's real Tony settings, and a shard's distinct + // from the other shards', so that shards running at once keep apart QApplication app(argc, argv); app.setOrganizationName("tony-tests"); - app.setApplicationName("test-tony-dev"); + app.setApplicationName(shardApplicationName + ("test-tony-dev", + qEnvironmentVariable("TONY_TEST_SHARD"))); // Text in shades of grey, as in test-tony-app QFont font = QApplication::font(); From 53a835f40f71032e622e0110d2fa068b4d424cf0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:37:58 +0000 Subject: [PATCH 202/275] test: wait until stop would keep a take before stopping it Three tests stopped a take straight after looking at it. When Stop came before the fake device's first block, finishSingingTake() rightly dropped the empty take, and lyrics_import_disabled_while_recording then waited 30 s for an analysis that never started: it failed in about half of the app runs. The tests now wait until the take holds more than the latency and the lead-in, as finishSingingTake() counts them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0137vMTCch66TVceRM316MAF --- main/test/TestMainWindow.h | 21 +++++++++++++++++++++ main/test/TestRecordWorkflow.h | 13 +++++++++++++ 2 files changed, 34 insertions(+) diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index a1be511a..70539d56 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -31,6 +31,7 @@ #include "view/ViewManager.h" #include "audio/AudioCallbackPlaySource.h" #include "audio/AudioCallbackRecordTarget.h" +#include "data/model/WritableWaveFileModel.h" #include #include @@ -190,6 +191,26 @@ class TestMainWindow : public MainWindow sv::sv_frame_t takeEnd() { return m_takeEnd; } bool takeTimerRunning() { return m_takeTimer && m_takeTimer->isActive(); } + // Whether Stop would keep the take being recorded. finishSingingTake() + // drops one no longer than the latency and the lead-in, as a take + // stopped straight after Record is when the device has not delivered + // a block yet. Counted as it counts them, and only once nothing can + // move the latency any more: the take's deferred start has run (it + // sets up the live dots, and then the latency), and the start of the + // reference, if that plays, has been measured + bool stopWouldKeepTake() { + if (!m_recordingInProgress || !m_realtimePitchLayer) return false; + if (m_awaitingReferenceStart) return false; + if (m_playSource && m_playSource->isPlaying() && + m_recordingStartGapMeasured < 0) { + return false; + } + auto recording = sv::ModelById::getAs + (m_currentRecordingModelId); + return recording && + recording->getFrameCount() > currentTakeTiming().spliceOffset(); + } + void seekTo(sv::sv_frame_t frame) { m_viewManager->setPlaybackFrame(frame); } diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 8653f5d3..d9599aba 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -170,6 +170,13 @@ class TestRecordWorkflow : public QObject QVERIFY(m_window->recordTarget()->isRecording()); } + // For a test that stops a take as soon as it has looked at it: until + // Stop would keep the take. One stopped before the device's first + // block has nothing in it and is dropped, with no analysis to wait for + void waitUntilStopKeepsTake() { + QTRY_VERIFY_WITH_TIMEOUT(m_window->stopWouldKeepTake(), 2000); + } + // Stop, then wait for pYIN on the take void stopTake() { QVERIFY(m_window->recordTarget()->isRecording()); @@ -4982,6 +4989,8 @@ private slots: m_window->doNewEmptyTake(); QCOMPARE(m_window->takes()->getTakeCount(), 1); + waitUntilStopKeepsTake(); + if (QTest::currentTestFailed()) return; stopTake(); if (QTest::currentTestFailed()) return; m_window->doUpdateMenuStates(); @@ -5899,6 +5908,8 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QVERIFY(!reference->isLayerDormant(pane)); + waitUntilStopKeepsTake(); + if (QTest::currentTestFailed()) return; stopTake(); } @@ -6331,6 +6342,8 @@ private slots: QVERIFY(!m_window->doImportLyricsFrom(path)); QVERIFY(!m_window->lyrics()->isShown()); + waitUntilStopKeepsTake(); + if (QTest::currentTestFailed()) return; stopTake(); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(import->isEnabled(), 2000); From 43733277d1839de934d0daedc1c5d1980d22ce9d Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:56:44 +0000 Subject: [PATCH 203/275] test: shards share HOME, kept apart by their application names run-tests.sh gave every process a HOME and XDG directories of its own, which move nothing on Windows. The shards' application names now keep their settings, data and temp directories apart on both platforms, so Linux runs rely on what Windows will and keep proving it. The header says the script runs from Git Bash on Windows too. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0137vMTCch66TVceRM316MAF --- deploy/linux/run-tests.sh | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/deploy/linux/run-tests.sh b/deploy/linux/run-tests.sh index 30b03f4d..e5aca887 100755 --- a/deploy/linux/run-tests.sh +++ b/deploy/linux/run-tests.sh @@ -17,17 +17,19 @@ # container's 4 cores, 8 processes run it in a minute and a half instead # of eight, and the load stays under 2. # -# Each process has a log directory, a HOME and XDG directories of its own. -# The suites keep QSettings per user, and processes sharing the file -# would read and clear each other's settings; a suite that turns on -# QStandardPaths' test mode keeps them in $HOME/.qttest, which the XDG -# variables do not move. On Windows QSettings is the registry, which -# neither moves: this is for Linux. +# Each process has a log directory of its own, and runs under an +# application name of its own (shardApplicationName(), +# main/test/RunSuite.h). The name keeps its settings, data location and +# svcore temp directory apart from the other processes', on Linux and on +# Windows alike, so they all share the user's HOME. The script runs from +# Git Bash on Windows too, with the environment AGENTS.md gives and +# executable names with .exe. # # Usage, from anywhere: # deploy/linux/run-tests.sh [-j N] [BUILD_DIR] EXECUTABLE # # deploy/linux/run-tests.sh test-tony-app +# deploy/linux/run-tests.sh -j 6 build_mingw test-tony-app.exe # # N defaults to twice the number of cores, BUILD_DIR to build. The # results are in tmp/tl/EXECUTABLE/SHARD/SUITE.txt; the summary gives @@ -71,12 +73,10 @@ mkdir -p "$out" start=$SECONDS for i in $(seq 0 $((jobs - 1))); do dir=$out/$i - mkdir -p "$dir/home" "$dir/xdg/config" "$dir/xdg/data" "$dir/xdg/cache" + mkdir -p "$dir" ( cd "$build" && - TONY_TEST_SHARD=$i/$jobs TONY_TEST_LOG_DIR=$dir HOME=$dir/home \ - XDG_CONFIG_HOME=$dir/xdg/config XDG_DATA_HOME=$dir/xdg/data \ - XDG_CACHE_HOME=$dir/xdg/cache \ + TONY_TEST_SHARD=$i/$jobs TONY_TEST_LOG_DIR=$dir \ "./$exe" > "$dir/stdout.log" 2>&1 echo $? > "$dir/exit" ) & From 4502fdc8d98d707093ea9ee03e7b851fe8208bd1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:59:34 +0000 Subject: [PATCH 204/275] fix: a ranged analysis is merged from the event loop, however soon it ends analyseRange() merged a run that pYIN had already finished by the time createDerivedLayers() returned, within the call, so Stop found nothing running after a short take. The tests of what happens during a take's analysis then lost their race now and then under the sharded runner (one in 7 of 10 runs once the live dots cost less). The check now goes through the event loop: the caller always finds the range being analysed and waits for its merge as it does after a longer take. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- main/Analyser.cpp | 11 +++++++++-- main/MainWindow.cpp | 5 +++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 4726c889..54d8c252 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -1206,8 +1206,15 @@ Analyser::analyseRange(sv_frame_t start, sv_frame_t end, // createDerivedLayers() returns only once the transform has set both // outputs' completion to 0, so no signal can have been missed above. - // A very short range could have finished by now all the same - rangedAnalysisCompletionChanged({}); + // A very short range could have finished by now all the same. It is + // looked at from the event loop, not here: then the caller always + // finds the range being analysed when this returns, whether pYIN took + // a second or was done within the call, and waits for the merge the + // same way. (It was done within the call often enough, on a quiet + // machine, to make tests of what happens during an analysis fail.) + QMetaObject::invokeMethod(this, [this]() { + rangedAnalysisCompletionChanged({}); + }, Qt::QueuedConnection); return ""; } diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index abd3a14e..93b04299 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -6201,8 +6201,9 @@ MainWindow::startTakeAnalysis(sv_frame_t start, sv_frame_t end) return false; } - // A range short enough to have been analysed and merged before the - // call returned leaves nothing to wait for + // The analysis is merged from the event loop, however soon it is + // done (Analyser::analyseRange()); this is for an analyser that + // could not start one after all if (!m_analyser2->isAnalysingRange()) return false; m_takeAnalysisRange = Coverage::Range(start, end); From 0df08a542a670d2a078dbced64626790700504af Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:59:42 +0000 Subject: [PATCH 205/275] fix: live dots reach the GUI thread in batches and redraw only their strip The tracker made one queued call per hop, about 170 a second, so a GUI thread slower than a hop would fall behind for good. LiveDotsFeed takes everything found once every 40 ms: the dots are added together, the status bar is set once, and the pane is drawn again only over the new dots rather than whole (GUI thread 30 % to 19 % of a core at pixel ratio 3 in a phone-sized window). It replaces ModelChangeThrottle, and logs once a second how far the recording, the tracker and the dots have got and what they cost, so a phone's log says whether they keep up. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 28 ++- docs/architecture.md | 9 +- docs/recording.md | 23 ++- docs/testing.md | 2 +- main/LiveDotsFeed.cpp | 184 ++++++++++++++++++ main/LiveDotsFeed.h | 123 ++++++++++++ main/MainWindow.cpp | 131 +++++++++---- main/MainWindow.h | 19 +- main/ModelChangeThrottle.cpp | 86 --------- main/ModelChangeThrottle.h | 65 ------- main/PaneUtils.cpp | 13 ++ main/PaneUtils.h | 12 ++ main/RealtimePitchTracker.cpp | 16 +- main/RealtimePitchTracker.h | 41 ++-- main/test/TestLiveDotsFeed.h | 277 +++++++++++++++++++++++++++ main/test/TestMainWindow.h | 21 +- main/test/TestModelChangeThrottle.h | 173 ----------------- main/test/TestRealtimePitchTracker.h | 28 ++- main/test/TestRecordWorkflow.h | 60 ++++++ main/test/TestUiChecks.h | 56 ++++++ main/test/tony-core-test.cpp | 4 +- meson.build | 4 +- 22 files changed, 964 insertions(+), 411 deletions(-) create mode 100644 main/LiveDotsFeed.cpp create mode 100644 main/LiveDotsFeed.h delete mode 100644 main/ModelChangeThrottle.cpp delete mode 100644 main/ModelChangeThrottle.h create mode 100644 main/test/TestLiveDotsFeed.h delete mode 100644 main/test/TestModelChangeThrottle.h diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 90c1b3f8..9053643c 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -177,7 +177,7 @@ builds happen in the container.) - A7b — Fixes from the second phone test: menus, the picker, Downloads. Done. - A4b — Vertical zoom and scroll by touch. Done. - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. -- A9 — Live dots in real time on the phone. +- A9 — Live dots in real time on the phone. Done. - A10 — Plot elements sized for the screen. - A8 — Documentation pass. @@ -840,3 +840,29 @@ already passed it by. With the reference missing, Save As waits for an analysis Tests seen failing: DecodedPcm compaction; `maySaveUnasked()`'s guard; Save As's question. Left open: the decoder not run on a phone. SDK sources 36 installed in /opt/android/sdk. +### Phase A9 — 2026-09-26 +Measured (container; `QT_SCALE_FACTOR=3`, 400x850 window, compact layout, 3-minute reference +with pitch and notes, 4.4 s page, take with the reference playing): GUI thread 29-30% of a core. +Per estimate 16 us (0.3%), record update 61 us per 10 ms (0.6%), tracker thread 1.3%: all keep +up. Pane paints ~26%: each of the dots' 25 notices a second drew the whole pane (7 ms in a take; +3.2 ms idle, 4-4.9 at ratio 2.75, the smooth downscale); the pointer's ~50 strips/s 2.7-3 ms each. +Built: `RealtimePitchTracker` keeps its estimates (`takeEstimates()`, `getFramesAnalysed()`), no +signal per hop. `LiveDotsFeed` (core, `TestLiveDotsFeed`): a 40 ms timer on the GUI thread hands +all found since the last look to `onRealtimePitchDetected(estimates)`: dots added together, status +bar set once, the pane drawn again only over the batch's frames (`PaneUtils::updateViewFrames()`). +It times batches and the pane's paints (an event filter that delivers them itself) and logs once +a second ("MainWindow: live dots: 12.03 s recorded, tracker at 12.01 s, dots to 12.01 s; 25 +batches of 6.8 dots, ..."). `ModelChangeThrottle` is gone (the interval is the throttle). After: +18-20% of a core. Found: the phone's APK predates the merge: dots cached, told nothing, drawn at +page turns only (~3.5 s on a 4.4 s page), which is "several seconds behind". +On a core 4-5x slower the pointer's strip paints would dominate (50-75% of a core; they coalesce: +fewer frames, no growing lag): `TimeValueLayer::paint` costs ~10 us a dot (`getModelsEndFrame()`, +two `QFontMetrics`, a pen, a brush per point), paints 100 physical px past the area; svgui's +pointer strip is 65 px. svgui candidates, not made. +Tests seen failing: `live_dots_keep_up_with_a_slow_gui` on HEAD (1.9 s behind at 4.3 s); +`live_dots_draw_only_where_they_are` with a whole-pane update; the feed per estimate. +Flaky: tests needing the ranged analysis running just after Stop lose when pYIN ends inside +`createDerivedLayers()` (~16 ms) and merges at once (seen in a log): one in 7 of 10 runner runs +here, 0 of 3 at HEAD, which also lost one in a parallel repeat. Lead: `analyseRange()` now looks +at a finished run from the event loop, never within the call; 5 of 5 runner runs green after. +The lead also brought the docs naming `ModelChangeThrottle` up to date. Not on a phone. diff --git a/docs/architecture.md b/docs/architecture.md index 1ee78b23..2322013c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -32,7 +32,7 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `ModelChangeThrottle`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LyricsTtml`, `LyricsEdit`, `LatencyUtils.h` | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `LiveDotsFeed`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LyricsTtml`, `LyricsEdit`, `LatencyUtils.h` | | `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `LyricsEditor`, `TakeCommands`, `TakeLayers`, `PaneUtils` | When adding a file: put it in the right `*_files` list, and in the matching `*_moc_files` @@ -60,9 +60,10 @@ follow. `MainWindow` then only fills the struct in and puts the answer on screen (as must `m_analyser2`). `LyricsEditor` owns no layer: it finds the lyrics through `LyricsTrack` at every event, and is deleted before it. - `RealtimePitchTracker` is a `QThread` that only **reads** the recording's - `WritableWaveFileModel` and emits `pitchDetected(frame, hz)`. It never touches the pitch - model; `MainWindow::onRealtimePitchDetected()` writes it on the GUI thread (queued - connection). Stop the tracker **before** releasing the model it reads. + `WritableWaveFileModel` and keeps its estimates for `takeEstimates()`. It never touches + the pitch model; `LiveDotsFeed` takes the estimates on the GUI thread every 40 ms and + `MainWindow::onRealtimePitchDetected()` writes them there. Stop the feed, then the + tracker, **before** releasing the model it reads (`stopRealtimePitchTracker()`). The reference is the pane's **work model** (`Pane::setWorkModel()`, svgui fork, set in `analyseNewMainModel()`). Without that the pane greys itself out from the end of the diff --git a/docs/recording.md b/docs/recording.md index 3af9cf0f..20e0d278 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -149,17 +149,22 @@ dot model outlives the take. ## The live tracker `RealtimePitchTracker::run()`: read a 2048-frame window of the **mixdown** -(`getData(-1, ...)`, so a mic on input 2 works), YIN, emit, advance 256; sleep 5 ms when -there is not a full window yet. `kHopSize` is also the resolution of the dot model, whose +(`getData(-1, ...)`, so a mic on input 2 works), YIN, keep the estimate, advance 256; +sleep 5 ms when there is not a full window yet. `kHopSize` is also the resolution of the dot model, whose unit must be `"Hz"` for the layer to align to the pane's log-frequency scale. -**Telling the pane of the dots.** Every notice of a change to the dot model is a redraw of -the pane, and a notice for each dot would be about 170 a second. So the dot model is made -with `notifyOnAdd` false, and then it tells nobody of a dot at all: the dots were drawn -only when one widened the model's pitch range, and stalled within half a second on a -steady note. `onRealtimePitchDetected()` hands each dot's frames to -`m_realtimeDotsNotifier` (`ModelChangeThrottle`, `tony_core`), which tells the pane at -once and then at most every 40 ms. +**Getting the dots to the pane.** The tracker finds about 170 estimates a second. Handed +to the GUI thread one at a time (a queued call each), a GUI thread that needs longer for one +than the tracker takes to find the next (5.8 ms) falls behind for good, and the dots trail +the singing more and more for as long as the take lasts; a phone is that slow. So the +tracker keeps its estimates, and `m_liveDotsFeed` (`LiveDotsFeed`, `tony_core`) takes all +of them every 40 ms and hands them to `onRealtimePitchDetected()` as one batch: a slow GUI +thread gets bigger batches, never a queue. The dot model is made with `notifyOnAdd` false, +so it tells nobody of a dot (a notice has the pane draw all of itself; with no notice at +all the dots were drawn only when one widened the model's pitch range); instead each batch +has the pane draw only the strip where its dots go (`updateViewFrames()`, `PaneUtils`). +Once a second the feed's costs go to the log ("live dots: ... s recorded, tracker at ... +s, dots to ... s; ..."), which is how a phone's log says whether the dots keep up. **The dots are kept out of the pane's cache** (`Layer::setCachedInView(false)`, svgui fork). Told of a change to the model of a layer in its cache, a pane draws every layer in diff --git a/docs/testing.md b/docs/testing.md index b3350df8..526f5bc9 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -6,7 +6,7 @@ the real device. The commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`, `TestModelChangeThrottle`, `TestRunSuite` | seconds | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`, `TestLiveDotsFeed`, `TestRunSuite` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow`, `TestUiChecks`, `TestAudioCheck` | about 9 minutes in one process, a minute and a half in eight (measured 2026-09-26 on Linux), nearly all of it `TestRecordWorkflow`, `TestAudioCheck` and `TestUiChecks`: takes are recorded in real time | | `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about a minute and growing: each test records several takes in real time | | `test-tony-device` | as `test-tony-app`, but with the **real** audio device | `TestRealDevice` | about a minute; run by hand only, see the [manual checklist](manual-checklist.md) | diff --git a/main/LiveDotsFeed.cpp b/main/LiveDotsFeed.cpp new file mode 100644 index 00000000..b6ea67c5 --- /dev/null +++ b/main/LiveDotsFeed.cpp @@ -0,0 +1,184 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LiveDotsFeed.h" + +#include + +#include + +#if defined(Q_OS_LINUX) || defined(Q_OS_MACOS) +#include +#endif + +using namespace sv; + +// CPU time this thread has used, in ms; negative where there is no way +// of asking +static double +threadTimeMs() +{ +#if defined(Q_OS_LINUX) || defined(Q_OS_MACOS) + timespec ts; + if (clock_gettime(CLOCK_THREAD_CPUTIME_ID, &ts) == 0) { + return double(ts.tv_sec) * 1000.0 + double(ts.tv_nsec) / 1.0e6; + } +#endif + return -1.0; +} + +static double +elapsedMs(const QElapsedTimer &timer) +{ + return double(timer.nsecsElapsed()) / 1.0e6; +} + +void +LiveDotsFeed::Times::add(double ms) +{ + ++count; + totalMs += ms; + maxMs = std::max(maxMs, ms); +} + +LiveDotsFeed::LiveDotsFeed(int intervalMs) : + m_running(false), + m_threadTimeAtReport(-1.0) +{ + m_timer.setInterval(intervalMs); + connect(&m_timer, &QTimer::timeout, this, [this]() { tick(); }); +} + +LiveDotsFeed::~LiveDotsFeed() +{ + stop(); +} + +void +LiveDotsFeed::start(Source source, Handler handler) +{ + stop(); + m_source = source; + m_handler = handler; + m_running = true; + m_report.newestFrame = -1; + startPeriod(); + m_sinceTick.invalidate(); + m_timer.start(); +} + +void +LiveDotsFeed::stop() +{ + m_timer.stop(); + m_running = false; + m_source = {}; + m_handler = {}; + if (m_painted) m_painted->removeEventFilter(this); + m_painted = nullptr; +} + +void +LiveDotsFeed::timePaintsOf(QObject *widget) +{ + if (m_painted) m_painted->removeEventFilter(this); + m_painted = widget; + if (m_painted) m_painted->installEventFilter(this); +} + +void +LiveDotsFeed::startPeriod() +{ + sv_frame_t newest = m_report.newestFrame; + m_report = Report(); + m_report.newestFrame = newest; + m_sinceReport.start(); + m_threadTimeAtReport = threadTimeMs(); +} + +void +LiveDotsFeed::tick() +{ + if (!m_running) return; + + // How long since the last look, which is the interval when the GUI + // thread has the time, and longer when it has not + if (m_sinceTick.isValid()) { + m_report.intervals.add(elapsedMs(m_sinceTick)); + } + m_sinceTick.start(); + + Estimates batch = m_source(); + if (!batch.empty()) { + QElapsedTimer timer; + timer.start(); + m_handler(batch); + m_report.batches.add(elapsedMs(timer)); + m_report.estimates += int(batch.size()); + m_report.newestFrame = batch.back().frame; + // (the handler may have stopped the feed) + if (!m_running) return; + } + + if (m_reporter && m_sinceReport.elapsed() >= 1000) { + m_report.seconds = elapsedMs(m_sinceReport) / 1000.0; + double threadTime = threadTimeMs(); + if (threadTime >= 0.0 && m_threadTimeAtReport >= 0.0 && + m_report.seconds > 0.0) { + m_report.guiThreadShare = (threadTime - m_threadTimeAtReport) / + (m_report.seconds * 1000.0); + } + m_reporter(m_report); + startPeriod(); + } +} + +bool +LiveDotsFeed::eventFilter(QObject *object, QEvent *event) +{ + if (object != m_painted || event->type() != QEvent::Paint) { + return false; + } + + // Delivered here, because nothing comes after a paint event to say + // when it is over + QElapsedTimer timer; + timer.start(); + object->event(event); + m_report.paints.add(elapsedMs(timer)); + return true; +} + +QString +LiveDotsFeed::describe(const Report &r) +{ + QString text = QString("%1 batches of %2 dots, %3 ms each (at most %4), " + "every %5 ms (at most %6); pane painted %7 times, " + "%8 ms each (at most %9)") + .arg(r.batches.count) + .arg(r.batches.count > 0 ? double(r.estimates) / r.batches.count : 0.0, + 0, 'f', 1) + .arg(r.batches.averageMs(), 0, 'f', 2) + .arg(r.batches.maxMs, 0, 'f', 1) + .arg(r.intervals.averageMs(), 0, 'f', 0) + .arg(r.intervals.maxMs, 0, 'f', 0) + .arg(r.paints.count) + .arg(r.paints.averageMs(), 0, 'f', 2) + .arg(r.paints.maxMs, 0, 'f', 1); + if (r.guiThreadShare >= 0.0) { + text += QString("; GUI thread %1% of a core") + .arg(r.guiThreadShare * 100.0, 0, 'f', 0); + } + return text; +} diff --git a/main/LiveDotsFeed.h b/main/LiveDotsFeed.h new file mode 100644 index 00000000..5617dab3 --- /dev/null +++ b/main/LiveDotsFeed.h @@ -0,0 +1,123 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LIVE_DOTS_FEED_H +#define TONY_LIVE_DOTS_FEED_H + +#include "RealtimePitchTracker.h" + +#include +#include +#include +#include +#include + +#include + +/** + * Hands the live tracker's pitch estimates to the GUI thread in + * batches: everything found since the last batch, once an interval. + * + * The tracker finds about 170 estimates a second. Handed over one at a + * time, each is a call queued on the GUI thread, and a GUI thread that + * needs longer for one than the tracker takes to find the next (5.8 ms) + * falls behind for good: the dots trail the singing further and further + * for as long as the take lasts. A phone's GUI thread is several times + * slower than a desktop's. Taken in batches, a slower thread takes + * bigger batches, later, and nothing queues up. The interval is also + * how often the pane is told of new dots, each time a redraw of it. + * + * Once a second it reports what the batches, and the paints of the + * pane the dots are in, have cost the GUI thread: for the log, since + * whether the dots keep up on a phone cannot be seen from the desktop. + * + * Lives on the GUI thread, as the handler does. + */ +class LiveDotsFeed : public QObject +{ +public: + typedef RealtimePitchTracker::Estimates Estimates; + typedef std::function Source; + typedef std::function Handler; + + explicit LiveDotsFeed(int intervalMs); + ~LiveDotsFeed(); + + /// Take what the source has found once an interval, and give it to + /// the handler, if there is anything + void start(Source source, Handler handler); + + /// No more batches or reports, and no more paints timed. What the + /// source still holds is left with it + void stop(); + + bool isRunning() const { return m_running; } + + /// Time the paint events of this widget (the pane the dots are in) + /// for the reports, until stop(). The feed delivers them to the + /// widget itself, to see when they are over: event filters the + /// widget had before do not see its paint events meanwhile + void timePaintsOf(QObject *widget); + + struct Times { + int count = 0; + double totalMs = 0.0; + double maxMs = 0.0; + void add(double ms); + double averageMs() const { return count > 0 ? totalMs / count : 0.0; } + }; + + struct Report { + double seconds = 0.0; ///< since the last report + int estimates = 0; ///< handed over since then + /// Centre frame of the newest estimate handed over (not only + /// since the last report), or -1 + sv::sv_frame_t newestFrame = -1; + Times batches; ///< the handler's time, each batch + Times intervals; ///< between one look at the source and the next + Times paints; ///< the widget's paint events + /// Of one core, over the period; negative where unknown + double guiThreadShare = -1.0; + }; + + /// Called with what happened since the last report, once a second + /// while running + void setReporter(std::function reporter) { + m_reporter = reporter; + } + + /// The costs of a report, for a log line + static QString describe(const Report &report); + +protected: + bool eventFilter(QObject *object, QEvent *event) override; + +private: + void tick(); + void startPeriod(); + + QTimer m_timer; + bool m_running; + Source m_source; + Handler m_handler; + std::function m_reporter; + QPointer m_painted; + + Report m_report; + QElapsedTimer m_sinceTick; + QElapsedTimer m_sinceReport; + double m_threadTimeAtReport; +}; + +#endif diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 93b04299..428fe3ab 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -161,7 +161,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_analyser2(nullptr), m_realtimePitchTracker(nullptr), m_realtimePitchLayer(nullptr), - m_realtimeDotsNotifier(40), + m_liveDotsFeed(40), m_overview(0), m_compactLayout(nullptr), m_playAction(nullptr), @@ -593,6 +593,7 @@ MainWindow::~MainWindow() // Clean up secondary state that may not have been torn down if the // window was closed without going through closeSession() (e.g. on // application exit via the window close button). + m_liveDotsFeed.stop(); if (m_realtimePitchTracker) { m_realtimePitchTracker->stop(); delete m_realtimePitchTracker; @@ -4891,9 +4892,10 @@ MainWindow::setupRealtimePitchLayer() // the pane's log-frequency coordinate system (same as the pYIN pitch track). // // notifyOnAdd false: a notice for each of the ~170 dots a second - // would be a redraw for each. The model then tells nobody of a dot, - // though, so m_realtimeDotsNotifier tells the pane of what was added, - // 25 times a second + // would be a redraw for each, and a notice has the pane draw all of + // itself. The model then tells nobody of a dot, though, so + // onRealtimePitchDetected() has the pane draw the part of it where + // each batch of dots goes, 25 times a second auto pitchModel = std::make_shared (sr, RealtimePitchTracker::kHopSize, false); pitchModel->setObjectName(tr("Realtime Pitch (Live)")); @@ -4920,13 +4922,13 @@ MainWindow::setupRealtimePitchLayer() // Associate our pre-filled SparseTimeValueModel with the layer. // The model was already registered via addNonDerivedModel above. m_document->setModel(m_realtimePitchLayer, m_realtimePitchModelId); - m_realtimeDotsNotifier.setModel(m_realtimePitchModelId); m_realtimePitchLayer->setVerticalScale(TimeValueLayer::AutoAlignScale); m_realtimePitchLayer->setPlotStyle(TimeValueLayer::PlotPoints); // Out of the pane's cache: told of a change to the model of a layer // in it, the pane draws every layer in it again -- the reference's - // pitch track, notes and waveform, 25 times a second + // pitch track, notes and waveform, 25 times a second. Out of it, a + // part of the pane can be drawn with the dots new there m_realtimePitchLayer->setCachedInView(false); // Singing/recording track uses the "Orange" colour so it is visually @@ -4937,15 +4939,26 @@ MainWindow::setupRealtimePitchLayer() m_document->attachLayerToView(pane, m_realtimePitchLayer); // Create and start the pitch tracker. Its thread reads new frames - // from audioSourceId (the WritableWaveFileModel) and emits - // pitchDetected(); onRealtimePitchDetected() writes the estimates into - // m_realtimePitchModelId on this thread. + // from audioSourceId (the WritableWaveFileModel) and keeps what it + // finds; m_liveDotsFeed takes it from there to + // onRealtimePitchDetected() in batches, which writes the estimates + // into m_realtimePitchModelId on this thread. The feed is stopped + // before the tracker goes (stopRealtimePitchTracker()) m_realtimePitchTracker = new RealtimePitchTracker( audioSourceId, this); - connect(m_realtimePitchTracker, &RealtimePitchTracker::pitchDetected, - this, &MainWindow::onRealtimePitchDetected); m_realtimePitchTracker->start(); + RealtimePitchTracker *tracker = m_realtimePitchTracker; + m_liveDotsFeed.setReporter([this](const LiveDotsFeed::Report &report) { + logLiveDots(report); + }); + m_liveDotsFeed.start + ([tracker]() { return tracker->takeEstimates(); }, + [this](const RealtimePitchTracker::Estimates &estimates) { + onRealtimePitchDetected(estimates); + }); + m_liveDotsFeed.timePaintsOf(pane); + cerr << "setupRealtimePitchLayer: realtime pitch tracking started " << "(audio source model " << audioSourceId << ", sr=" << sr << ")" << endl; } @@ -4953,6 +4966,8 @@ MainWindow::setupRealtimePitchLayer() void MainWindow::stopRealtimePitchTracker() { + // First: the feed takes from the tracker + m_liveDotsFeed.stop(); if (m_realtimePitchTracker) { m_realtimePitchTracker->stop(); delete m_realtimePitchTracker; @@ -4964,7 +4979,6 @@ void MainWindow::teardownRealtimePitchLayer() { stopRealtimePitchTracker(); - m_realtimeDotsNotifier.setModel({}); if (m_realtimeLayerTeardownConnection) { disconnect(m_realtimeLayerTeardownConnection); @@ -5785,16 +5799,19 @@ MainWindow::wantedPreRollFrames() const } void -MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) +MainWindow::onRealtimePitchDetected +(const RealtimePitchTracker::Estimates &estimates) { - // Called on the GUI thread via Qt::QueuedConnection (RealtimePitchTracker - // emits from its background thread). Write the point into the model here - // so all model mutations stay on the GUI thread. + // Everything the tracker has found since the last batch, handed over + // by m_liveDotsFeed on the GUI thread, so that all model mutations + // stay on this thread. One estimate at a time, as they were found, + // a GUI thread slower than the tracker would fall further behind it + // for as long as the take lasted. // - // Events still queued when the take ended arrive here as well. The - // dots may still be on show then, waiting for pYIN, but the take is - // over: leave them, and the status bar, alone. - if (!m_recordingInProgress) return; + // The feed stops with the tracker when the take ends. A batch that + // came even so would find the dots perhaps still on show, waiting for + // pYIN, but the take over: leave them, and the status bar, alone. + if (!m_recordingInProgress || estimates.empty()) return; // Draw the dot where the finished pitch track will put this sound: the // take is spliced into the singing track from m_takePosition on, with @@ -5809,21 +5826,41 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) for (const Event &e : m->getAllEvents()) m->remove(e); } - // The frame is the recording's, at the device's rate, and the answer - // the reference's. A negative answer is sound sung during the lead-in - // of a pre-roll, or before the reference started at all: no dot for it - sv_frame_t intoTake = currentTakeTiming().liveFrameIntoTake(frame); - if (intoTake < 0) return; - sv_frame_t dotFrame = m_takePosition + intoTake; + // The frames are the recording's, at the device's rate, and the + // answers the reference's. A negative answer is sound sung during the + // lead-in of a pre-roll, or before the reference started at all: no + // dot for it + TakeTiming timing = currentTakeTiming(); + sv_frame_t from = 0, to = 0; + const RealtimePitchTracker::Estimate *newest = nullptr; + for (const auto &estimate : estimates) { + sv_frame_t intoTake = timing.liveFrameIntoTake(estimate.frame); + if (intoTake < 0) continue; + sv_frame_t dotFrame = m_takePosition + intoTake; + if (m) m->add(Event(dotFrame, float(estimate.hz), tr(""))); + sv_frame_t dotEnd = dotFrame + RealtimePitchTracker::kHopSize; + if (!newest) { + from = dotFrame; + to = dotEnd; + } else { + from = std::min(from, dotFrame); + to = std::max(to, dotEnd); + } + newest = &estimate; + } + if (!newest) return; - if (m) { - m->add(Event(dotFrame, float(hz), tr(""))); - m_realtimeDotsNotifier.changed - (dotFrame, dotFrame + RealtimePitchTracker::kHopSize); + // The pane draws again only where the new dots are. The model's own + // notice would have it draw all of itself, which at a phone's pixel + // ratio is several times the cost, 25 times a second + if (m && m_paneStack && m_paneStack->getPaneCount() > 0) { + updateViewFrames(m_paneStack->getPane(0), from, to); } - // Convert Hz to MIDI note number and cents deviation. - // MIDI note 69 = A4 = 440 Hz. + // The status bar says what is being sung just now: the newest + // estimate of the batch. Convert Hz to MIDI note number and cents + // deviation. MIDI note 69 = A4 = 440 Hz. + double hz = newest->hz; double midiNote = 12.0 * std::log2(hz / 440.0) + 69.0; int nearestNote = int(std::round(midiNote)); int cents = int(std::round((midiNote - nearestNote) * 100.0)); @@ -5853,6 +5890,36 @@ MainWindow::onRealtimePitchDetected(sv::sv_frame_t frame, double hz) .arg(centsStr)); } +void +MainWindow::logLiveDots(const LiveDotsFeed::Report &report) +{ + // Once a second during a take, so that the log of a phone says + // whether the dots keep up with the singing there: how much has been + // recorded, how far the tracker has analysed it, and how far the dots + // handed to the pane have got, all in seconds of the recording; then + // what the dots cost the GUI thread + sv_samplerate_t rate = 0; + if (auto recording = ModelById::getAs + (m_currentRecordingModelId)) { + rate = recording->getSampleRate(); + } + if (rate <= 0 || !m_recordTarget || !m_realtimePitchTracker) return; + + auto seconds = [rate](sv_frame_t frame) { + return QString::number(double(frame) / rate, 'f', 2); + }; + // A dot is at the middle of the window it was found in: the end of + // that window is the recording it has caught up with + sv_frame_t dotsTo = report.newestFrame < 0 ? 0 : + report.newestFrame + RealtimePitchTracker::kWindowSize / 2; + + cerr << "MainWindow: live dots: " + << seconds(m_recordTarget->getFramesReceived()) << " s recorded, " + << "tracker at " << seconds(m_realtimePitchTracker->getFramesAnalysed()) + << " s, dots to " << seconds(dotsTo) << " s; " + << LiveDotsFeed::describe(report) << endl; +} + void MainWindow::recordingFinishedFull(Analyser *analysing) { diff --git a/main/MainWindow.h b/main/MainWindow.h index 37abddbc..70944d28 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -28,7 +28,7 @@ #include "TakeTiming.h" #include "LatencyUtils.h" #include "LatencyCalibration.h" -#include "ModelChangeThrottle.h" +#include "LiveDotsFeed.h" #include #include @@ -343,7 +343,6 @@ protected slots: // --- Real-time pitch tracking during microphone recording --- virtual void recordingStarted(); - virtual void onRealtimePitchDetected(sv::sv_frame_t frame, double hz); virtual void recordingFinishedFull(Analyser *analysing = nullptr); virtual void finishSingingTake(); @@ -377,9 +376,9 @@ protected slots: // Model backing the realtime layer (owned by the document). sv::ModelId m_realtimePitchModelId; - // Tells the pane of the dots added to that model, which tells nobody - // itself (see setupRealtimePitchLayer()) - ModelChangeThrottle m_realtimeDotsNotifier; + // Brings the tracker's estimates to onRealtimePitchDetected() in + // batches, and reports once a second what they cost + LiveDotsFeed m_liveDotsFeed; sv::Overview *m_overview; @@ -864,6 +863,16 @@ protected slots: virtual void teardownRealtimePitchLayer(); virtual void stopRealtimePitchTracker(); + // A batch of the live tracker's estimates, everything it has found + // since the last: dots for them, the pane told of them, the status + // bar set from the newest. From m_liveDotsFeed + virtual void onRealtimePitchDetected + (const RealtimePitchTracker::Estimates &estimates); + + // The once-a-second log line of a take: how far the recording, the + // tracker and the dots have got, and what the dots cost + void logLiveDots(const LiveDotsFeed::Report &report); + // The raw recording of a take needs a layer of its own to hold it in // the document: the singing analyser is busy with the take's audio, // which stays on show while the recording is made. The layer is diff --git a/main/ModelChangeThrottle.cpp b/main/ModelChangeThrottle.cpp deleted file mode 100644 index 6b362cfa..00000000 --- a/main/ModelChangeThrottle.cpp +++ /dev/null @@ -1,86 +0,0 @@ -/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ - -/* - Tony - An intonation analysis and annotation tool - Centre for Digital Music, Queen Mary, University of London. - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. See the file - COPYING included with this distribution for more information. -*/ - -#include "ModelChangeThrottle.h" - -#include - -using namespace sv; - -ModelChangeThrottle::ModelChangeThrottle(int intervalMs) : - m_pending(false), - m_from(0), - m_to(0) -{ - m_timer.setInterval(intervalMs); - QObject::connect(&m_timer, &QTimer::timeout, - &m_timer, [this]() { intervalUp(); }); -} - -ModelChangeThrottle::~ModelChangeThrottle() -{ -} - -void -ModelChangeThrottle::setModel(ModelId model) -{ - m_timer.stop(); - m_pending = false; - m_model = model; -} - -void -ModelChangeThrottle::changed(sv_frame_t from, sv_frame_t to) -{ - if (m_model.isNone()) return; - - if (m_pending) { - m_from = std::min(m_from, from); - m_to = std::max(m_to, to); - } else { - m_from = from; - m_to = to; - m_pending = true; - } - - // Quiet until now: tell at once, and hold back what comes next for an - // interval - if (!m_timer.isActive()) { - tell(); - m_timer.start(); - } -} - -void -ModelChangeThrottle::intervalUp() -{ - // Nothing came in the interval: quiet again - if (!m_pending) { - m_timer.stop(); - return; - } - tell(); -} - -void -ModelChangeThrottle::tell() -{ - m_pending = false; - auto model = ModelById::get(m_model); - if (!model) return; - - // The model's own notice, as it would have sent it itself for each - // change: a view redraws only what it is told of - emit model->modelChangedWithin(m_model, m_from, m_to); -} diff --git a/main/ModelChangeThrottle.h b/main/ModelChangeThrottle.h deleted file mode 100644 index 3c6dbc9d..00000000 --- a/main/ModelChangeThrottle.h +++ /dev/null @@ -1,65 +0,0 @@ -/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ - -/* - Tony - An intonation analysis and annotation tool - Centre for Digital Music, Queen Mary, University of London. - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. See the file - COPYING included with this distribution for more information. -*/ - -#ifndef TONY_MODEL_CHANGE_THROTTLE_H -#define TONY_MODEL_CHANGE_THROTTLE_H - -#include "base/BaseTypes.h" -#include "data/model/Model.h" - -#include - -/** - * Tells the views of a model what has changed in it, at most once an - * interval: for a model written to faster than it is worth drawing, and - * made to hold back its own change notices (notifyOnAdd false), such as - * the live pitch dots of a take. - * - * A pane draws itself again for every notice it is told, and the dots - * come about 170 a second. Told nothing, a pane never draws the dots - * at all. - * - * The first change after a quiet interval is told at once; changes - * that follow within the interval are told together at its end. Lives - * on the GUI thread, as the model's writer and its views do. - */ -class ModelChangeThrottle -{ -public: - explicit ModelChangeThrottle(int intervalMs); - ~ModelChangeThrottle(); - - ModelChangeThrottle(const ModelChangeThrottle &) = delete; - ModelChangeThrottle &operator=(const ModelChangeThrottle &) = delete; - - /// The model whose views are told. A model of none (the default) - /// stops, and a change not yet told is forgotten - void setModel(sv::ModelId model); - sv::ModelId getModel() const { return m_model; } - - /// Frames [from, to) of the model have changed - void changed(sv::sv_frame_t from, sv::sv_frame_t to); - -private: - void intervalUp(); - void tell(); - - sv::ModelId m_model; - QTimer m_timer; - bool m_pending; - sv::sv_frame_t m_from; - sv::sv_frame_t m_to; -}; - -#endif diff --git a/main/PaneUtils.cpp b/main/PaneUtils.cpp index 209b48e8..0c57b5ad 100644 --- a/main/PaneUtils.cpp +++ b/main/PaneUtils.cpp @@ -90,3 +90,16 @@ pruneExtraPane(Document *document, PaneStack *paneStack, Pane *extra, if (overview) overview->unregisterView(extra); if (paneStack) paneStack->deletePane(extra); } + +void +updateViewFrames(View *view, sv_frame_t from, sv_frame_t to) +{ + if (!view || to <= from) return; + + // A pixel either side: a layer draws at the view's x for a frame, + // scaled to the pixel ratio, and at least one physical pixel wide + int x0 = view->getXForFrame(from) - 1; + int x1 = view->getXForFrame(to) + 1; + if (x1 < 0 || x0 >= view->width()) return; + view->update(x0, 0, x1 - x0 + 1, view->height()); +} diff --git a/main/PaneUtils.h b/main/PaneUtils.h index 4a9058aa..f134a6c7 100644 --- a/main/PaneUtils.h +++ b/main/PaneUtils.h @@ -22,6 +22,7 @@ class Document; class PaneStack; class Pane; class Overview; +class View; } /** @@ -45,4 +46,15 @@ void pruneExtraPane(sv::Document *document, sv::ModelId ownedModelId, sv::Overview *overview = nullptr); +/** + * Have the view draw again only the part of itself over frames [from, + * to), for a layer that draws there alone what changed there, and + * that is out of the view's cache, such as the live dots. A model's + * own change notice has the view draw all of itself: at a pixel ratio + * of 3 that is nine times the pixels, of every layer, the cached ones + * copied from the cache. + */ +void updateViewFrames(sv::View *view, + sv::sv_frame_t from, sv::sv_frame_t to); + #endif diff --git a/main/RealtimePitchTracker.cpp b/main/RealtimePitchTracker.cpp index 271d6369..5de6024e 100644 --- a/main/RealtimePitchTracker.cpp +++ b/main/RealtimePitchTracker.cpp @@ -35,7 +35,8 @@ RealtimePitchTracker::RealtimePitchTracker(ModelId audioSourceId, m_audioSourceId(audioSourceId), m_minFreq(60.0), m_maxFreq(1000.0), - m_threshold(0.15) + m_threshold(0.15), + m_framesAnalysed(0) { } @@ -57,6 +58,15 @@ RealtimePitchTracker::stop() wait(); } +RealtimePitchTracker::Estimates +RealtimePitchTracker::takeEstimates() +{ + Estimates taken; + std::lock_guard guard(m_estimatesMutex); + taken.swap(m_estimates); + return taken; +} + void RealtimePitchTracker::run() { @@ -110,11 +120,13 @@ RealtimePitchTracker::run() double hz = sr / lagSamples; if (hz >= m_minFreq && hz <= m_maxFreq) { sv_frame_t centreFrame = nextFrameToProcess + kWindowSize / 2; - emit pitchDetected(centreFrame, hz); + std::lock_guard guard(m_estimatesMutex); + m_estimates.push_back({ centreFrame, hz }); processedAny = true; } } + m_framesAnalysed = nextFrameToProcess + kWindowSize; nextFrameToProcess += kHopSize; } diff --git a/main/RealtimePitchTracker.h b/main/RealtimePitchTracker.h index 9b9cbbe1..f3f4eb62 100644 --- a/main/RealtimePitchTracker.h +++ b/main/RealtimePitchTracker.h @@ -17,6 +17,8 @@ #include +#include +#include #include #include "base/BaseTypes.h" @@ -35,16 +37,17 @@ class FFT; * continuously polls a WritableWaveFileModel for new audio samples, * estimating pitch in real time using FFT-accelerated YIN. * - * pitch estimates are reported via pitchDetected() signals; the - * connection to the GUI thread is automatically a QueuedConnection so - * the slot (which writes to the model and updates the status bar) runs - * safely on the GUI thread without blocking audio or rendering. + * The estimates are kept until the GUI thread takes them, all at once + * (takeEstimates()): there is one for each hop, about 170 a second, and + * a signal for each would queue a call on the GUI thread that a slow + * GUI thread falls behind with for good (LiveDotsFeed). * * Usage: * 1. Create a RealtimePitchTracker with the ModelId of the * WritableWaveFileModel being recorded into. * 2. Call start() — the background thread starts immediately. - * 3. Call stop() when recording ends — blocks until the thread exits. + * 3. Take what it has found with takeEstimates(), from any thread. + * 4. Call stop() when recording ends — blocks until the thread exits. */ class RealtimePitchTracker : public QThread { @@ -91,16 +94,24 @@ class RealtimePitchTracker : public QThread void setThreshold(double t) { m_threshold = t; } double getThreshold() const { return m_threshold; } -signals: + /** A voiced pitch estimate. */ + struct Estimate { + sv::sv_frame_t frame; ///< centre frame of the analysis window + double hz; ///< always > 0 + }; + typedef std::vector Estimates; + /** - * Emitted from the background thread each time a new voiced pitch - * estimate is available. Via Qt::AutoConnection this arrives in the - * GUI thread's event loop (QueuedConnection cross-thread). - * - * @param frame Centre frame of the analysis window. - * @param hz Pitch in Hz (always > 0 when emitted). + * The estimates found since the last call, oldest first; they are + * not kept any longer. Any thread. */ - void pitchDetected(sv::sv_frame_t frame, double hz); + Estimates takeEstimates(); + + /** + * The frame of the recording the tracker has analysed up to: the + * end of its latest window, voiced or not. Any thread. + */ + sv::sv_frame_t getFramesAnalysed() const { return m_framesAnalysed; } protected: /** The background polling loop — do not call directly. */ @@ -115,6 +126,10 @@ class RealtimePitchTracker : public QThread double m_maxFreq; double m_threshold; + std::mutex m_estimatesMutex; + Estimates m_estimates; + std::atomic m_framesAnalysed; + // --- YIN helpers (all called only from run()) --- static void yinDifferenceFFT(const std::vector &buf, diff --git a/main/test/TestLiveDotsFeed.h b/main/test/TestLiveDotsFeed.h new file mode 100644 index 00000000..5882496a --- /dev/null +++ b/main/test/TestLiveDotsFeed.h @@ -0,0 +1,277 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_LIVE_DOTS_FEED_H +#define TEST_LIVE_DOTS_FEED_H + +// Tier 2: the live tracker's estimates brought to the GUI thread in +// batches. That a take's dots keep up with a slow GUI thread is +// TestRecordWorkflow's business (live_dots_keep_up_with_a_slow_gui). + +#include "../LiveDotsFeed.h" + +#include +#include +#include +#include + +#include +#include +#include +#include + +class TestLiveDotsFeed : public QObject +{ + Q_OBJECT + + typedef RealtimePitchTracker::Estimate Estimate; + typedef RealtimePitchTracker::Estimates Estimates; + + static constexpr int kInterval = 20; + + // What the tracker would hold: found on another thread, taken whole + struct Found { + std::mutex mutex; + Estimates estimates; + sv::sv_frame_t next = 1024; + + void add(int n) { + std::lock_guard guard(mutex); + for (int i = 0; i < n; ++i) { + estimates.push_back({ next, 220.0 }); + next += 256; + } + } + Estimates take() { + Estimates taken; + std::lock_guard guard(mutex); + taken.swap(estimates); + return taken; + } + int waiting() { + std::lock_guard guard(mutex); + return int(estimates.size()); + } + }; + + static void busy(int ms) { + QElapsedTimer timer; + timer.start(); + while (timer.elapsed() < ms) { } + } + + // A widget, as far as its paint events go: they take a while + class Painted : public QObject + { + public: + int paints = 0; + bool event(QEvent *e) override { + if (e->type() == QEvent::Paint) { + ++paints; + busy(5); + return true; + } + return QObject::event(e); + } + }; + +private slots: + void hands_over_everything_at_once() { + Found found; + std::vector batches; + LiveDotsFeed feed(kInterval); + found.add(100); + feed.start([&]() { return found.take(); }, + [&](const Estimates &e) { batches.push_back(e); }); + QTest::qWait(kInterval * 5); + feed.stop(); + + QCOMPARE(int(batches.size()), 1); + QCOMPARE(int(batches[0].size()), 100); + for (int i = 1; i < 100; ++i) { + QVERIFY(batches[0][i].frame > batches[0][i-1].frame); + } + QCOMPARE(found.waiting(), 0); + } + + void nothing_found_nothing_handed_over() { + int looked = 0, handed = 0; + LiveDotsFeed feed(kInterval); + feed.start([&]() { ++looked; return Estimates(); }, + [&](const Estimates &) { ++handed; }); + QTest::qWait(kInterval * 10); + feed.stop(); + QVERIFY(looked >= 3); + QCOMPARE(handed, 0); + } + + // The point of it all: estimates come in faster than a slow GUI + // thread could take them one by one, and it keeps up all the same, + // with bigger batches, one a look + void a_slow_gui_takes_bigger_batches() { + Found found; + std::atomic produced(0); + std::atomic producing(true); + std::thread producer([&]() { + // one a hop, as the tracker finds them: 5.8 ms + while (producing) { + found.add(1); + ++produced; + std::this_thread::sleep_for(std::chrono::microseconds(5800)); + } + }); + + int handedOver = 0, calls = 0, worstBehind = 0; + LiveDotsFeed feed(kInterval); + QElapsedTimer timer; + timer.start(); + feed.start([&]() { return found.take(); }, + [&](const Estimates &e) { + ++calls; + handedOver += int(e.size()); + // far more than an estimate's worth of time + busy(25); + worstBehind = std::max + (worstBehind, produced - handedOver); + }); + QTest::qWait(1500); + producing = false; + producer.join(); + QTest::qWait(kInterval * 5); + feed.stop(); + qint64 elapsed = timer.elapsed(); + + // Everything arrived, and never more than a look and a slow + // batch's worth of them waited: 45 ms is 8 estimates; handed over + // one at a time, 25 ms each, some 1100 would be waiting by now + QCOMPARE(handedOver, int(produced)); + QCOMPARE(found.waiting(), 0); + QVERIFY2(worstBehind <= 40, + qPrintable(QString("%1 estimates were waiting at worst") + .arg(worstBehind))); + QVERIFY2(calls <= elapsed / kInterval + 2, + qPrintable(QString("%1 batches in %2 ms") + .arg(calls).arg(elapsed))); + QVERIFY2(double(handedOver) / calls >= 4.0, + qPrintable(QString("%1 estimates in %2 batches") + .arg(handedOver).arg(calls))); + } + + void stop_leaves_the_rest_with_the_source() { + Found found; + int handed = 0; + LiveDotsFeed feed(kInterval); + feed.start([&]() { return found.take(); }, + [&](const Estimates &) { ++handed; }); + QVERIFY(feed.isRunning()); + feed.stop(); + QVERIFY(!feed.isRunning()); + found.add(10); + QTest::qWait(kInterval * 5); + QCOMPARE(handed, 0); + QCOMPARE(found.waiting(), 10); + } + + void reports_once_a_second() { + Found found; + std::vector reports; + int handedOver = 0; + sv::sv_frame_t newest = -1; + LiveDotsFeed feed(kInterval); + feed.setReporter([&](const LiveDotsFeed::Report &r) { + reports.push_back(r); + }); + feed.start([&]() { return found.take(); }, + [&](const Estimates &e) { + handedOver += int(e.size()); + newest = e.back().frame; + }); + QElapsedTimer timer; + timer.start(); + while (timer.elapsed() < 2300) { + found.add(3); + QTest::qWait(30); + } + feed.stop(); + + QCOMPARE(int(reports.size()), 2); + int reported = 0; + for (const auto &r : reports) { + QVERIFY2(r.seconds >= 0.95 && r.seconds < 1.5, + qPrintable(QString("a report of %1 s").arg(r.seconds))); + QVERIFY(r.batches.count > 10); + QVERIFY(r.intervals.count > 10); + QVERIFY2(r.intervals.averageMs() >= kInterval * 0.8 && + r.intervals.averageMs() < kInterval * 3, + qPrintable(QString("looked every %1 ms") + .arg(r.intervals.averageMs()))); + QCOMPARE(r.paints.count, 0); + reported += r.estimates; + } + QVERIFY(reported > 0 && reported <= handedOver); + QVERIFY(reports.back().newestFrame > 0); + QVERIFY(reports.back().newestFrame <= newest); +#if defined(Q_OS_LINUX) || defined(Q_OS_MACOS) + QVERIFY(reports.back().guiThreadShare >= 0.0); +#endif + } + + void times_the_paints_of_a_widget() { + Found found; + Painted painted; + std::vector reports; + LiveDotsFeed feed(kInterval); + feed.setReporter([&](const LiveDotsFeed::Report &r) { + reports.push_back(r); + }); + feed.start([&]() { return found.take(); }, + [&](const Estimates &) { }); + feed.timePaintsOf(&painted); + for (int i = 0; i < 3; ++i) { + QEvent paint(QEvent::Paint); + QCoreApplication::sendEvent(&painted, &paint); + } + // and they still reach the widget + QCOMPARE(painted.paints, 3); + QTRY_VERIFY_WITH_TIMEOUT(!reports.empty(), 3000); + QCOMPARE(reports[0].paints.count, 3); + QVERIFY(reports[0].paints.averageMs() >= 4.5); + QVERIFY(reports[0].paints.maxMs >= 4.5); + + // Not timed after the take, and not held up either + feed.stop(); + QEvent paint(QEvent::Paint); + QCoreApplication::sendEvent(&painted, &paint); + QCOMPARE(painted.paints, 4); + } + + void describes_a_report() { + LiveDotsFeed::Report r; + r.seconds = 1.0; + r.estimates = 175; + for (int i = 0; i < 25; ++i) r.batches.add(i == 3 ? 1.5 : 0.25); + for (int i = 0; i < 25; ++i) r.intervals.add(i == 7 ? 95.0 : 40.0); + for (int i = 0; i < 49; ++i) r.paints.add(i == 0 ? 18.0 : 3.0); + r.guiThreadShare = 0.284; + QString text = LiveDotsFeed::describe(r); + QVERIFY2(text.startsWith("25 batches of 7.0 dots, 0.30 ms each " + "(at most 1.5), every 42 ms (at most 95); " + "pane painted 49 times, 3.31 ms each " + "(at most 18.0); GUI thread 28% of a core"), + qPrintable(text)); + r.guiThreadShare = -1.0; + QVERIFY(!LiveDotsFeed::describe(r).contains("GUI thread")); + } +}; + +#endif diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index e36b303a..dc2eabf3 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -34,6 +34,7 @@ #include #include +#include #include #include #include @@ -293,13 +294,30 @@ class TestMainWindow : public MainWindow int lyricsShiftQuestions() const { return m_shiftQuestions; } void whileAskingLyricsShift(std::function f) { m_whileAskingShift = f; } + // An estimate of the live tracker's, handed to the window as the + // live dots feed does, in a batch of its own void doRealtimePitchDetected(sv::sv_frame_t frame, double hz) { - onRealtimePitchDetected(frame, hz); + onRealtimePitchDetected({ { frame, hz } }); } + + // A GUI thread that takes this long, on top of the real work, each + // time the live dots are handed to it: a phone, several times slower + // than the machine the tests run on + void setLiveDotsDelay(int ms) { m_liveDotsDelayMs = ms; } QString statusText() { return getStatusLabel()->text(); } void setStatusText(QString text) { getStatusLabel()->setText(text); } protected: + void onRealtimePitchDetected + (const RealtimePitchTracker::Estimates &estimates) override { + if (m_liveDotsDelayMs > 0) { + QElapsedTimer timer; + timer.start(); + while (timer.elapsed() < m_liveDotsDelayMs) { } + } + MainWindow::onRealtimePitchDetected(estimates); + } + void createAudioIO() override { if (m_audioIO || m_playTarget) return; if (m_useRealDevice) { @@ -392,6 +410,7 @@ class TestMainWindow : public MainWindow FakeAudioIO::Config m_fakeConfig; bool m_installDevice; bool m_useRealDevice = false; + int m_liveDotsDelayMs = 0; bool m_recordOverAnswer = true; bool m_recordOverInDialog = false; int m_recordOverQuestions = 0; diff --git a/main/test/TestModelChangeThrottle.h b/main/test/TestModelChangeThrottle.h deleted file mode 100644 index 5a51a2d2..00000000 --- a/main/test/TestModelChangeThrottle.h +++ /dev/null @@ -1,173 +0,0 @@ -/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ -/* - Tony - An intonation analysis and annotation tool - Centre for Digital Music, Queen Mary, University of London. - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of the - License, or (at your option) any later version. See the file - COPYING included with this distribution for more information. -*/ - -#ifndef TEST_MODEL_CHANGE_THROTTLE_H -#define TEST_MODEL_CHANGE_THROTTLE_H - -// Tier 2: telling a model's views of its changes, at most once an -// interval. What a pane does when told is TestUiChecks' business -// (live_dots_under_the_cursor). - -#include "../ModelChangeThrottle.h" - -#include "data/model/SparseTimeValueModel.h" - -#include -#include -#include - -#include -#include -#include - -class TestModelChangeThrottle : public QObject -{ - Q_OBJECT - - typedef sv::sv_frame_t frame_t; - - static constexpr int kInterval = 50; - - // A model like the live dots': changes held back by the model itself - std::shared_ptr m_model; - sv::ModelId m_id; - std::vector> m_told; - -private slots: - void init() { - m_model = std::make_shared(44100, 256, false); - m_id = sv::ModelById::add(m_model); - m_told.clear(); - connect(m_model.get(), &sv::Model::modelChangedWithin, - this, [this](sv::ModelId, frame_t from, frame_t to) { - m_told.push_back({ from, to }); - }); - } - - void cleanup() { - m_model.reset(); - sv::ModelById::release(m_id); - } - - // Why there is a throttle at all: a dot added to such a model tells - // nobody, so nothing would ever draw it - void the_model_tells_nothing_itself() { - m_model->add(sv::Event(1000, 220.f, "")); - m_model->add(sv::Event(1256, 221.f, "")); - QCoreApplication::processEvents(); - QCOMPARE(int(m_told.size()), 0); - } - - void first_change_is_told_at_once() { - ModelChangeThrottle throttle(kInterval); - throttle.setModel(m_id); - throttle.changed(1000, 1256); - QCOMPARE(int(m_told.size()), 1); - QCOMPARE(m_told[0], std::make_pair(frame_t(1000), frame_t(1256))); - } - - // ... and what follows within the interval is told together, once, - // when it is up - void changes_within_an_interval_are_told_together() { - ModelChangeThrottle throttle(kInterval); - throttle.setModel(m_id); - throttle.changed(1000, 1256); - throttle.changed(2000, 2256); - throttle.changed(1500, 1756); - QCOMPARE(int(m_told.size()), 1); - - QTRY_COMPARE_WITH_TIMEOUT(int(m_told.size()), 2, kInterval * 10); - QCOMPARE(m_told[1], std::make_pair(frame_t(1500), frame_t(2256))); - - // Nothing more to tell - QTest::qWait(kInterval * 3); - QCOMPARE(int(m_told.size()), 2); - } - - // A steady stream is told once an interval, however many changes - void a_stream_is_told_once_an_interval() { - ModelChangeThrottle throttle(kInterval); - throttle.setModel(m_id); - QElapsedTimer timer; - timer.start(); - int changes = 0; - while (timer.elapsed() < kInterval * 10) { - throttle.changed(changes * 256, changes * 256 + 256); - ++changes; - // A change every 2 ms by the clock, the throttle's timer - // running meanwhile. Not qWait(2): on Windows a wait that - // short lasts a timer tick of about 15 ms - QElapsedTimer step; - step.start(); - while (step.nsecsElapsed() < 2000000) { - QCoreApplication::processEvents(); - } - } - QVERIFY(changes > 100); - QVERIFY2(m_told.size() >= 5 && m_told.size() <= 13, - qPrintable(QString("%1 changes over ten intervals were told " - "%2 times") - .arg(changes).arg(m_told.size()))); - // Between them the notices cover every change - QTest::qWait(kInterval * 2); - QCOMPARE(m_told.front().first, frame_t(0)); - QCOMPARE(m_told.back().second, frame_t(changes * 256)); - for (size_t i = 1; i < m_told.size(); ++i) { - QVERIFY(m_told[i].first <= m_told[i-1].second); - } - } - - // After a quiet interval the next change is told at once again - void quiet_then_told_at_once() { - ModelChangeThrottle throttle(kInterval); - throttle.setModel(m_id); - throttle.changed(1000, 1256); - QTest::qWait(kInterval * 3); - QCOMPARE(int(m_told.size()), 1); - throttle.changed(5000, 5256); - QCOMPARE(int(m_told.size()), 2); - } - - // A change not yet told goes with the model, and with no model - // nothing is told at all - void no_model_tells_nothing() { - ModelChangeThrottle throttle(kInterval); - throttle.changed(1000, 1256); - QCOMPARE(int(m_told.size()), 0); - - throttle.setModel(m_id); - throttle.changed(1000, 1256); - throttle.changed(2000, 2256); - QCOMPARE(int(m_told.size()), 1); - throttle.setModel({}); - QTest::qWait(kInterval * 3); - QCOMPARE(int(m_told.size()), 1); - } - - // The model released under it: nothing to tell, and no harm - void model_gone() { - ModelChangeThrottle throttle(kInterval); - sv::ModelId id = m_id; - throttle.setModel(id); - throttle.changed(1000, 1256); - throttle.changed(2000, 2256); - m_model.reset(); - sv::ModelById::release(id); - m_id = {}; - QTest::qWait(kInterval * 3); - throttle.changed(3000, 3256); - QCOMPARE(int(m_told.size()), 1); - } -}; - -#endif diff --git a/main/test/TestRealtimePitchTracker.h b/main/test/TestRealtimePitchTracker.h index 3fc9ebba..40f6dca7 100644 --- a/main/test/TestRealtimePitchTracker.h +++ b/main/test/TestRealtimePitchTracker.h @@ -31,13 +31,11 @@ #include #include -// Collects pitchDetected() on the test (GUI) thread through a queued -// connection, which is how MainWindow receives it. QSignalSpy would -// connect directly and be written to from the tracker thread. -class PitchCollector : public QObject +// Takes the tracker's estimates on the test (GUI) thread, which is how +// MainWindow gets them (LiveDotsFeed): whatever has been found since the +// last look, whenever the count is asked for +class PitchCollector { - Q_OBJECT - public: struct Event { sv::sv_frame_t frame; @@ -46,17 +44,17 @@ class PitchCollector : public QObject std::vector events; - PitchCollector(RealtimePitchTracker *tracker) { - connect(tracker, &RealtimePitchTracker::pitchDetected, - this, &PitchCollector::pitchDetected); - } - - int count() const { return int(events.size()); } + PitchCollector(RealtimePitchTracker *tracker) : m_tracker(tracker) { } -public slots: - void pitchDetected(sv::sv_frame_t frame, double hz) { - events.push_back({ frame, hz }); + int count() { + for (const auto &e : m_tracker->takeEstimates()) { + events.push_back({ e.frame, e.hz }); + } + return int(events.size()); } + +private: + RealtimePitchTracker *m_tracker; }; class TestRealtimePitchTracker : public QObject diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 6d243078..54403375 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1786,6 +1786,66 @@ private slots: QCOMPARE(m_window->statusText(), QString("after the analysis")); } + // A phone's GUI thread is several times slower than this machine's. + // Made slower than the tracker finds pitch (an estimate a hop, 5.8 + // ms), it must still keep the dots up with the singing: given the + // dots one at a time it would fall further behind for as long as the + // take lasted + void live_dots_keep_up_with_a_slow_gui() { + FakeAudioIO::Config config; + config.input = tone(highHz, 5.0); + makeWindow(config); + openReference(writeWav(tone(lowHz, 5.0))); + if (QTest::currentTestFailed()) return; + + m_window->setLiveDotsDelay(10); + startTake(); + if (QTest::currentTestFailed()) return; + + // With no reference playing there is no latency, so a dot is at + // the frame of the recording its window was centred on. The most + // the newest may trail what the device has recorded: half a + // window, a record update, the tracker's poll, the wait for the + // next batch and the slow GUI thread, with room to spare. Per + // estimate, this GUI thread is 0.4 s behind after a second, 1.2 s + // after three + const sv::sv_frame_t bound = sv::sv_frame_t(0.4 * rate); + QElapsedTimer timer; + timer.start(); + sv::sv_frame_t worst = 0; + int looked = 0; + QString detail; + while (timer.elapsed() < 3000) { + QTest::qWait(100); + // (the tracker starts an event-loop turn after the take) + if (timer.elapsed() < 500) continue; + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + auto events = model->getAllEvents(); + if (events.empty()) continue; + ++looked; + sv::sv_frame_t recorded = + m_window->recordTarget()->getFramesReceived(); + sv::sv_frame_t behind = recorded - events.back().getFrame(); + if (behind > worst) { + worst = behind; + detail = QString("%1 ms into the take the newest dot is at " + "%2 s, %3 ms behind the %4 s recorded") + .arg(timer.elapsed()) + .arg(double(events.back().getFrame()) / rate, 0, 'f', 2) + .arg(1000.0 * double(behind) / rate, 0, 'f', 0) + .arg(double(recorded) / rate, 0, 'f', 2); + } + } + m_window->setLiveDotsDelay(0); + qInfo("%s", qPrintable(detail)); + QVERIFY2(worst <= bound, qPrintable(detail)); + QVERIFY2(looked >= 10, "hardly any live dots"); + + stopTake(); + } + // Review finding 8: during the take, the dots sit where the pitch // track will. Same device and singer as latency_end_to_end, but // looked at before Stop diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 64d6b925..d84a37a2 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -61,6 +61,7 @@ #include #include #include +#include #include #include #include @@ -532,6 +533,61 @@ private slots: sv::RecordDirectory::setRecordContainerDirectory(""); } + // The pane draws again only where new dots go, not all of itself for + // each batch of them: at a phone's pixel ratio a whole pane costs + // several times as much, 25 times a second + void live_dots_draw_only_where_they_are() { + FakeAudioIO::Config config; + config.input = tone(highHz, 4.0); + makeWindow(config); + if (QTest::currentTestFailed()) return; + openReference(writeWav(tone(lowHz, 12.0))); + if (QTest::currentTestFailed()) return; + + // A take that stays on its page, and away from the start of the + // song, where the recording's own frames would be + const sv::sv_frame_t P = frames(6.0); + showSeconds(5.0, 11.0); + m_window->seekTo(P); + startTake(); + if (QTest::currentTestFailed()) return; + QTest::qWait(500); + + // Seen before the paint event reaches anything else + struct PaintWatch : public QObject { + int whole = 0, parts = 0; + bool eventFilter(QObject *object, QEvent *e) override { + if (e->type() == QEvent::Paint) { + auto *widget = static_cast(object); + QRect r = static_cast(e)->rect(); + if (r.width() >= widget->width() - 2) ++whole; + else ++parts; + } + return false; + } + } watch; + sv::Pane *pane = pane0(); + sv::sv_frame_t pageStart = pane->getStartFrame(); + auto model = sv::ModelById::getAs + (m_window->realtimeModelId()); + QVERIFY(model); + int dotsBefore = model->getEventCount(); + pane->installEventFilter(&watch); + QTest::qWait(1500); + pane->removeEventFilter(&watch); + + QCOMPARE(pane->getStartFrame(), pageStart); + QVERIFY2(model->getEventCount() > dotsBefore + 100, + "hardly any dots came"); + QVERIFY2(watch.parts > 10, "the pane was hardly drawn at all"); + // (a dot that widens the model's pitch range has it drawn whole) + QVERIFY2(watch.whole <= 5, + qPrintable(QString("the pane was drawn whole %1 times in " + "1.5 s, and in part %2 times") + .arg(watch.whole).arg(watch.parts))); + stopTake(); + } + // Checklist: live dots appear under the playback cursor, not behind // it; during a take at P > 0 the cursor starts at P, the pane follows // it, and cursor and dots are in the same place. Singing exactly in diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index b38553d3..834ffaab 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -33,7 +33,7 @@ #include "TestLatencyCheck.h" #include "TestLatencyCalibration.h" #include "TestTakeDiff.h" -#include "TestModelChangeThrottle.h" +#include "TestLiveDotsFeed.h" #include "TestRunSuite.h" #include "RunSuite.h" @@ -192,7 +192,7 @@ int main(int argc, char *argv[]) } { - TestModelChangeThrottle t; + TestLiveDotsFeed t; if (runSuite(&t, argc, argv)) ++good; else ++bad; } diff --git a/meson.build b/meson.build index 6fd5bfd0..8b89eb2c 100644 --- a/meson.build +++ b/meson.build @@ -1191,10 +1191,10 @@ tony_core_files = [ 'main/LogFile.cpp', 'main/LatencyCalibration.cpp', 'main/LatencyCheck.cpp', + 'main/LiveDotsFeed.cpp', 'main/Lyrics.cpp', 'main/LyricsEdit.cpp', 'main/LyricsTtml.cpp', - 'main/ModelChangeThrottle.cpp', 'main/PinchZoom.cpp', 'main/PopupArea.cpp', 'main/RealtimePitchTracker.cpp', @@ -1558,7 +1558,7 @@ if system != 'android' 'main/test/TestLatencyCheck.h', 'main/test/TestLatencyCalibration.h', 'main/test/TestTakeDiff.h', - 'main/test/TestModelChangeThrottle.h', + 'main/test/TestLiveDotsFeed.h', 'main/test/TestRunSuite.h', ]) From 2e318930d7bd7f18d8735d86d76de4c87f05f976 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:03:57 +0000 Subject: [PATCH 206/275] docs: shards are kept apart by their application names testing.md says how shards keep apart on Linux and Windows alike and names the wait before stopping a short take. windows-shards.md says what was built and proven on Linux and keeps what is left for the Windows machine. forks.md lists the unlocked play-parameter map that crashed the fill thread once in a run whose shards shared their settings. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0137vMTCch66TVceRM316MAF --- docs/README.md | 2 +- docs/forks.md | 5 +++ docs/testing.md | 28 +++++++++---- docs/windows-shards.md | 90 ++++++++++++++++++++---------------------- 4 files changed, 68 insertions(+), 57 deletions(-) diff --git a/docs/README.md b/docs/README.md index a394ae9b..a7a11f85 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,4 +20,4 @@ methods, and they do not tell the story of fixed bugs. | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | | [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes — none of it tried yet | | [calibrate-audio.md](calibrate-audio.md) | Plan, not built: a Calibrate Audio button that measures the round trip through a speaker-to-mic loopback, and dev checks that automate most of the manual checklist | -| [windows-shards.md](windows-shards.md) | Plan, not built: sharded test runs from Git Bash on Windows, each shard kept apart by an application name of its own | +| [windows-shards.md](windows-shards.md) | Plan, built and proven on Linux, left to try on Windows: sharded test runs from Git Bash on Windows, each shard kept apart by an application name of its own | diff --git a/docs/forks.md b/docs/forks.md index d214a5c0..b5e2d6fc 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -167,6 +167,11 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` [recording.md](recording.md)); whether `m_model` can dangle otherwise was not looked into. - The play-start callback is passed the frames actually got, not the requested block size. - `View::removeLayer()` does not disconnect `layerMeasurementRectsChanged`. +- `svcore/base/PlayParameterRepository.cpp` keeps its play parameters in a `std::map` with + no lock, which the audio fill thread reads (`AudioGenerator::mixModel()` through + `getPlayParameters()`) while the GUI thread adds and removes playables. Seen once as a + crash of `test-tony-app` in the fill thread, in a sharded run whose processes shared + their settings; not seen otherwise. ## Changes that would tidy Tony up but were not made diff --git a/docs/testing.md b/docs/testing.md index b3350df8..7b002aaa 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -25,9 +25,9 @@ shorter: run it when a change touches what the development checks drive (see installed pYIN is never the one tested; the meson test `depends:` on `pyin_plugin` because nothing else builds `pyin.dll`. Build `pyin.dll` too when running by hand after a clean. -- Both mains set the organisation/application names to `tony-tests` / `test-tony-*` and - every suite works in a `QTemporaryDir`, so the user's QSettings and record directory - are never touched. +- The mains set the organisation/application names to `tony-tests` / `test-tony-*` (a + shard's name with a suffix of its own, see "Running") and every suite works in a + `QTemporaryDir`, so the user's QSettings and record directory are never touched. - `Tony.exe` links both libraries with `link_whole:`. A new source file that is in neither `tony_core_files` nor `tony_app_files` is invisible to the tests. - `build_mingw/meson-logs/testlog.txt` contains a dump of the whole inherited environment. @@ -59,11 +59,18 @@ Windows path would start an escape in the C string. not run. The app suite nearly only waits on `FakeAudioIO`'s real-time clock, so n processes at once take about 1/n of the time: on four cores the load stayed under 2 with eight, and reached 3.5 with twelve. `deploy/linux/run-tests.sh` starts them and adds up - their results. Each process needs a `HOME` and XDG directories of its own: the suites' - QSettings are per user, and processes sharing them clear each other's settings. - `TestDevChecks` turns on `QStandardPaths`' test mode, which keeps them in `~/.qttest` - whatever the XDG variables say. On Windows QSettings is the registry, so the script is - for Linux. Do not combine shards with test names on the command line. + their results. Processes running at once must not share settings: the suites clear and + rewrite them, and would do it under each other. So each shard runs under an application + name of its own, `-shardof` (`RunSuite.h`), and its settings, data location, + svcore temp directory and log are all keyed by that name, in `QStandardPaths`' test mode + too. That holds on Linux and Windows alike (on Windows they are the registry and known + folders, which no environment variable moves), so every process shares the user's + `HOME`; each shard leaves a settings file and a data folder of its own, one per `i` and + `n`. The script is written to run from Git Bash on Windows too, with the environment + AGENTS.md gives and executable names with `.exe`. It has not run there yet, and how many + processes suit that machine is not measured + ([windows-shards.md](windows-shards.md#on-the-windows-machine-after-the-merge)). Do not + combine shards with test names on the command line. - A sharded run is a whole run of the suites, but the tests that share a process are other ones. After a change to object lifetimes, threads or teardown (see "Timing and races"), run the one-process run as well. @@ -136,6 +143,11 @@ Windows path would start an escape in the C string. dialogs a test does expect. - `analysed()` waits for analysis completion, no running transformers **and** no ranged run. `snapshotTake()`, `verifyStripMatchesTake()`, `takeLayers()`. +- `waitUntilStopKeepsTake()` waits until Stop would keep the take being recorded + (`stopWouldKeepTake()` in `TestMainWindow`: more recorded than `finishSingingTake()` + discards, once the latency is final). A test that stops a take straight after looking at it + calls it first: Stop may otherwise come before the fake device's first block, and the + take is dropped with nothing to analyse. - `TestSignals.h`: sine, sawtooth, seeded noise, comparison in cents. `TestSingingAnalysis.h` works the expected ranges out for itself (`widenRange()`, `mergeWindow()`, `comparePitchAcrossFile()`, `verifyNothingLeftOver()`). diff --git a/docs/windows-shards.md b/docs/windows-shards.md index b71b7037..da76e2ed 100644 --- a/docs/windows-shards.md +++ b/docs/windows-shards.md @@ -1,9 +1,8 @@ # Sharded test runs on Windows: plan -Plan, not built. For a Linux cloud session to carry out; the last section is for the -Windows machine afterwards. When all of it is done, what lasts goes into -[testing.md](testing.md) and [AGENTS.md](../AGENTS.md), and this file and its row in -[README.md](README.md) are deleted. +Built and proven on Linux; the last section is left for the Windows machine. When that is +done, what lasts goes into [testing.md](testing.md) and [AGENTS.md](../AGENTS.md), and +this file and its row in [README.md](README.md) are deleted. ## Goal @@ -29,11 +28,11 @@ The app suite uses 0.4 of one core on average (Linux: nearer 0.25); the machine cores and no hyper-threading. So there is room for several processes at once, fewer than on Linux. -## Why the Linux runner does not work on Windows +## Why the Linux runner did not work on Windows Processes sharing a settings store clear each other's settings: `initTestCase()` of `TestRecordWorkflow` calls `QSettings().clear()`, and the shards of `test-tony-dev` failed -on Linux in exactly this way until each got a `HOME` of its own. `run-tests.sh` gives every +on Linux in exactly this way until each got a `HOME` of its own. `run-tests.sh` gave every process its own `HOME` and XDG directories. On Windows neither moves anything. QSettings in its native format is the registry key @@ -50,7 +49,7 @@ variables. These are also shared by processes of one application name: All of them are keyed by the application name. So a shard runs under an application name of its own, on both platforms. -## Design (decided) +## Design (built) 1. **`RunSuite.h`**: a pure function that gives the application name for a shard, taking the base name and the value of `TONY_TEST_SHARD`. No value: the base name, unchanged. A @@ -70,50 +69,40 @@ of its own, on both platforms. `grep` and `xargs` are there. Leave the default `-j` (twice the cores) alone; the Windows step sets its own. +All three are built as decided, but for one thing: the parse of `i/n` is stricter than +`runSuite()`'s was. A part that is not a whole number (`a/2`, `1x/2`, `1.0/2`) used to +run as shard 0; it is now invalid, so `runSuite()` refuses it and the name stays the base +name. + Not part of this: sharded `meson test` definitions (`meson test` and `build.bat test` stay the one-process run that [testing.md](testing.md) asks for after changes to lifetimes, threads or teardown), a PowerShell runner, any change in the library forks. If the work seems to need a fork change (for instance to `TempDirectory`), do not make it: report it. -## Steps for the cloud session - -Read AGENTS.md, then [testing.md](testing.md) (Running, Shards, the Linux failures, Timing -and races), `run-tests.sh`, `RunSuite.h` and `TestRunSuite.h`. Build as -[building.md](building.md#building-on-linux) says. - -1. **Baseline.** Run `run-tests.sh` for `test-tony-core`, `test-tony-app` and - `test-tony-dev` as it is, and record what fails. Only tests from testing.md's list of - Linux failures should. -2. **The failure this prevents, before the change.** Copy the script to `tmp/` (not - committed) and make every process share one `HOME` and one set of XDG directories. - That is the Windows situation on Linux: nothing but the application name can then keep - shards apart. Run it for `test-tony-dev` and `test-tony-app` and record what fails. If - nothing does, run it three times. If still nothing fails, say so in the report: step 4 - then proves less. -3. **Build design 1 and 2**, with tests in `TestRunSuite`: - - no shard gives exactly the base name; - - every `i` of one `n` gives a different name; - - the same `i` with a different `n` gives a different name; - - an invalid value gives the base name. - - Break the function for a moment and see a test fail (AGENTS.md), then put it back. -4. **Proof.** The shared-`HOME` script from step 2 must now pass three runs in a row for - `test-tony-dev` and `test-tony-app`, apart from the known Linux failures. If it does - not, stop there and report what failed: Windows would fail the same way, and a - per-process `HOME` cannot help it there. -5. **Design 3**, then run core, app and dev through the real script three times each. -6. **One-process runs** of all three executables, as AGENTS.md gives them, from `build/`: - the path without `TONY_TEST_SHARD` is the one `meson test` and named tests use. -7. **Docs.** [testing.md](testing.md)'s Shards paragraph: shards are kept apart by their - application name (settings, data directory, temp directories), on Linux and Windows; - remove "so the script is for Linux". Fix anything else the change makes false - ([building.md](building.md#building-on-linux) mentions the script). Leave AGENTS.md's - Windows commands and times alone: they need measuring on Windows. -8. **Commits and PR.** One commit per step as AGENTS.md says. For example: the shard's - application name with its tests (`test:`), the script (`test:`), the docs (`docs:`). - Push the session's branch and open a PR against `default` with `gh`. Report as - AGENTS.md asks: the Totals of the final runs, the failures seen in steps 1 and 2, and - anything that did not go as planned. +## Proven on Linux + +A copy of the old script in which every process shared one `HOME` and one set of XDG +directories stood in for Windows: with one `HOME`, as on Windows, nothing but the +application name can keep shards apart. + +- **Before the change**, both suites failed. The suites clear the settings in + `initTestCase()` and write their own in `init()` and `cleanup()`, and with one settings + file the processes did it under each other: a setting such as the pre-roll read back as + another process had left it, and the network-permission setting, once cleared, brought + its dialog up. `test-tony-dev` failed two tests; `test-tony-app` had nine failures in + one run and twelve in another, nearly all of them that way. +- **After it**, three runs of `test-tony-dev` with the shared `HOME` were clean, and so + were those of `test-tony-app`, but for a race in one test (a take stopped before the + fake device's first block), which failed with a `HOME` per process as well and is since + fixed. Now that the script itself shares `HOME`, three runs each of core, app and dev + through it, and one-process runs of all three, failed only the Linux failures + [testing.md](testing.md#running) lists. +- **What each shard left**: one settings file under `~/.config/tony-tests/` and one data + folder, holding its log and any temp directory of svcore's, under + `~/.local/share/tony-tests/`, both named for the shard; the two suites that turn on + `QStandardPaths`' test mode put theirs under `~/.qttest/`, named the same way. One set + per `i` and `n`, so their number stays bounded. A one-process run adds only the base + name's. ## On the Windows machine, after the merge @@ -121,7 +110,11 @@ From Git Bash with AGENTS.md's environment, after a build: 1. `deploy/linux/run-tests.sh -j N build_mingw test-tony-app.exe` for N = 4, 6 and 8, watching CPU in Task Manager. Take the largest N that passes three runs in a row with - the CPU clearly below full; tests that race the analysis are the ones to watch. + the CPU clearly below full; tests that race the analysis are the ones to watch. On the + first run, check that each shard's results land in `tmp/tl/test-tony-app.exe//`: + the script passes `TONY_TEST_LOG_DIR` as a `/c/...` path and relies on Git Bash + converting it for a native program. If they are not there, the summary has no suite + counts. 2. The same for `test-tony-dev.exe` and `test-tony-core.exe`. 3. Check that each shard left its own key under `HKCU\Software\tony-tests` and its own folders under `%APPDATA%` and `%LOCALAPPDATA%` (`qttest` included, for the two suites @@ -131,4 +124,5 @@ From Git Bash with AGENTS.md's environment, after a build: measured times, and gets a tool timeout to match. The one-process command stays, for named tests and for changes to lifetimes, threads or teardown. Update the times in [testing.md](testing.md) too. -5. Delete this file and its row in [README.md](README.md). +5. Delete this file, its row in [README.md](README.md) and the link to it in + [testing.md](testing.md#running). From 1979cccc0d766b287da582a248ba5ee28b9f4b7a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:30:31 +0000 Subject: [PATCH 207/275] feat: pitch and notes sized for the screen, and a plot size setting On a phone at pixel ratio 3 the pitch tracks, live dots and notes were drawn at a third of their size while the rest of the window was scaled. The svgui fork now sizes them in logical pixels at any ratio, pens included (pins svgui 8286def on its feat/tonyandroid branch), and View > Plot Size draws them at 100, 150 or 200 %, remembered; 150 % by default on Android. The desktop at 100 % draws as before, pixel for pixel. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 ++- docs/forks.md | 11 ++ main/MainWindow.cpp | 7 +- main/MainWindow.h | 5 + main/PlotSize.cpp | 97 +++++++++ main/PlotSize.h | 70 +++++++ main/test/TestCompactLayout.h | 42 ++++ main/test/TestPlotSize.h | 361 ++++++++++++++++++++++++++++++++++ main/test/TestViewCache.h | 9 + main/test/tony-app-test.cpp | 7 + meson.build | 3 + repoint-lock.json | 2 +- 12 files changed, 638 insertions(+), 3 deletions(-) create mode 100644 main/PlotSize.cpp create mode 100644 main/PlotSize.h create mode 100644 main/test/TestPlotSize.h diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 9053643c..b2fcd7e1 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -178,7 +178,7 @@ builds happen in the container.) - A4b — Vertical zoom and scroll by touch. Done. - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. - A9 — Live dots in real time on the phone. Done. -- A10 — Plot elements sized for the screen. +- A10 — Plot elements sized for the screen. Done. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -866,3 +866,28 @@ Flaky: tests needing the ranged analysis running just after Stop lose when pYIN here, 0 of 3 at HEAD, which also lost one in a parallel repeat. Lead: `analyseRange()` now looks at a finished run from the event loop, never within the call; 5 of 5 runner runs green after. The lead also brought the docs naming `ModelChangeThrottle` up to date. Not on a phone. + +### Phase A10 — 2026-09-26 +Built: svgui `a082647` (view): `ViewManager::setPlotScale()` and `plotScaleChanged()` (views drop +their cache and repaint); `LayerGeometryProvider::scalePlotSize()` (logical px x ratio x plot +scale, no font factor: the identity at 1 and 1) and `scalePlotPixelSize()`; +`ViewProxy::scalePenWidth()` by ratio x plot scale, not sqrt(ratio). `8286def` (layer): +TimeValueLayer's point height and least width, FlexiNoteLayer's note height, outline and hit +area (`getRelativeMousePosition()`, `getFeatureDescription()`) through it. Neither pushed nor +pinned. `main/PlotSize` (app): View > Plot Size 100/150/200 %, `MainWindow/plotsize`, default 150 +on Android and 100 elsewhere; nothing stored until a step is chosen, anything else stored is the +default. Tests: `TestPlotSize`, one each in `TestViewCache` and `TestCompactLayout`. +Measured: a whole pane of Tony's at ratio 1 (waveform, pitch, notes) was byte-identical before +and after (a one-off shot, since removed). Ratio 3, 400x850 compact: pitch layer 6.2 ms a paint +before and after at 100 %, 6.7-7.5 ms at 150/200 %; notes 0.05 ms; whole pane 17-19 ms either +way. At ratio 1, 150/200 % doubles the pitch layer (2.5 to 6 ms): a pen wider than a pixel leaves +Qt's fast path, which at ratio 3 the old 1.7 px pen had left already. +Hi-DPI desktop (ratio 2 at Windows 150/200 %) changes too: TimeValueLayer (points 2 logical px +high, not 1; pens 2 px, not 1.4), FlexiNoteLayer (notes 16 logical px, not 8, now as high as +their hit area), the pens of RegionLayer's bar styles and of SliceLayer/SpectrumLayer. Not the +coverage strip or lyrics (already `scalePixelSize()`), waveform, spectrogram or time ruler. +Left: the ruler's ticks and the waveform's lines stay 1 physical px (looked fine at ratio 3); the +strip and lyrics do not follow the plot size; forks.md not updated (A8). Not on a phone. +Tests seen failing: ViewProxy without the ratio (the old sizes at 3): sizes, hit area; plot +scale x1.1 at 100 %: desktop_draws_as_before; no connection: the cache test; PlotSize not +applying: its step test and the menu test. diff --git a/docs/forks.md b/docs/forks.md index d214a5c0..3f8bc112 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -157,6 +157,17 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` - `View::paintEvent()` on a cache hit no longer has the cached layers draw into its buffer, where the cache then covered them. Upstream has done that since 2018, so the cache saved nothing and every paint, down to the play pointer's few pixels, drew every layer. +- Plot elements keep their size in logical pixels (branch `feat/tonyandroid`, for the + Android port). `View` draws its layers at the whole pixel ratio (3 on a phone at 2.75), + but `TimeValueLayer`'s points (2 px high) and `FlexiNoteLayer`'s notes (`NOTE_HEIGHT`) + were sized in those physical pixels and pens scaled by only the square root of the ratio, + so on a phone pitch and notes were a third of their size. Now + `LayerGeometryProvider::scalePlotSize()` (logical px x ratio x plot scale, no font factor: + unchanged at ratio 1 and scale 1) sizes them, the notes' hit areas use it too, and + `ViewProxy::scalePenWidth()` scales by the whole ratio and the plot scale. + `ViewManager::setPlotScale()` / `plotScaleChanged()` is Tony's View > Plot Size; each + view drops its cache on a change. On a hi-DPI desktop (ratio 2) this doubles points and + notes, and thickens the pens of every layer drawn through a `ViewProxy`. ## Known defects in the forks, not fixed diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 428fe3ab..4fe9c8af 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -19,6 +19,7 @@ #include "NetworkPermissionTester.h" #include "Analyser.h" #include "CompactLayout.h" +#include "PlotSize.h" #include "AudioCheckRunner.h" #include "CalibrateAudioDialog.h" #include "LatencyUtils.h" @@ -164,6 +165,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_liveDotsFeed(40), m_overview(0), m_compactLayout(nullptr), + m_plotSize(nullptr), m_playAction(nullptr), m_recordAction(nullptr), m_zoomInAction(nullptr), @@ -512,8 +514,9 @@ MainWindow::MainWindow(AudioMode audioMode, m_takeTimer->setInterval(100); connect(m_takeTimer, SIGNAL(timeout()), this, SLOT(pollTakeProgress())); - // Before the menus: its switch is in the View menu + // Before the menus: their switches are in the View menu m_compactLayout = new CompactLayout(this); + m_plotSize = new PlotSize(m_viewManager, this); setupMenus(); setupToolbars(); @@ -1159,6 +1162,8 @@ MainWindow::setupViewMenu() menu->addSeparator(); menu->addAction(m_compactLayout->getAction()); + QMenu *plotSizeMenu = menu->addMenu(tr("Plot &Size")); + plotSizeMenu->addActions(m_plotSize->getActions()); // Enabled and checked in updateLayerStatuses(). Not "Show &Lyrics": // Peek Left has the L m_showLyrics = new QAction(tr("Show L&yrics"), this); diff --git a/main/MainWindow.h b/main/MainWindow.h index 70944d28..a406d02d 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -46,6 +46,7 @@ class QComboBox; class QActionGroup; class QToolBar; class CompactLayout; +class PlotSize; class AudioCheckRunner; struct AudioCheckResult; @@ -387,6 +388,10 @@ protected slots: // the parts (setupCompactLayout()): the actions below, which are made // with the menus and toolbars, and others that have members already CompactLayout *m_compactLayout; + + // View > Plot Size: how large the panes draw pitch and notes + PlotSize *m_plotSize; + QAction *m_playAction; QAction *m_recordAction; QAction *m_zoomInAction; diff --git a/main/PlotSize.cpp b/main/PlotSize.cpp new file mode 100644 index 00000000..d207b08e --- /dev/null +++ b/main/PlotSize.cpp @@ -0,0 +1,97 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "PlotSize.h" + +#include "view/ViewManager.h" + +#include +#include +#include + +static const char *settingsGroup = "MainWindow"; +static const char *settingsKey = "plotsize"; + +PlotSize::PlotSize(sv::ViewManager *viewManager, QObject *parent) : + QObject(parent), + m_viewManager(viewManager), + m_group(new QActionGroup(this)), + m_percent(0) +{ + m_group->setExclusive(true); + + for (int percent : getSteps()) { + QAction *action = new QAction(tr("%1%").arg(percent), m_group); + action->setCheckable(true); + action->setData(percent); + action->setStatusTip + (tr("Draw the pitch tracks and the notes at %1% of their " + "normal size").arg(percent)); + connect(action, &QAction::triggered, + this, [this, percent]() { setPercent(percent); }); + } + + QSettings settings; + settings.beginGroup(settingsGroup); + int percent = settings.value(settingsKey, getDefaultPercent()).toInt(); + settings.endGroup(); + + if (!getSteps().contains(percent)) { + percent = getDefaultPercent(); + } + apply(percent); +} + +QList +PlotSize::getActions() const +{ + return m_group->actions(); +} + +int +PlotSize::getDefaultPercent() +{ +#ifdef Q_OS_ANDROID + return 150; +#else + return 100; +#endif +} + +void +PlotSize::setPercent(int percent) +{ + if (!getSteps().contains(percent)) return; + + apply(percent); + + QSettings settings; + settings.beginGroup(settingsGroup); + settings.setValue(settingsKey, percent); + settings.endGroup(); +} + +void +PlotSize::apply(int percent) +{ + m_percent = percent; + + for (QAction *action : m_group->actions()) { + if (action->data().toInt() == percent) action->setChecked(true); + } + + if (m_viewManager) { + m_viewManager->setPlotScale(percent / 100.0); + } +} diff --git a/main/PlotSize.h b/main/PlotSize.h new file mode 100644 index 00000000..763dc93f --- /dev/null +++ b/main/PlotSize.h @@ -0,0 +1,70 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_PLOT_SIZE_H +#define TONY_PLOT_SIZE_H + +#include +#include + +class QAction; +class QActionGroup; + +namespace sv { +class ViewManager; +} + +/** + * How large the panes draw the pitch tracks, the live dots and the + * notes, as View > Plot Size sets it: 100%, 150% or 200% of their + * normal size in logical pixels. It is the view manager's plot scale + * (svgui fork), which the panes apply together with the screen's pixel + * ratio, so the steps look alike on a phone and on a desktop. Chosen, + * a step takes effect at once and is remembered in the settings. + * + * The default is 150% on Android, where a thin line on a small screen + * held at arm's length is hard to follow, and 100% elsewhere, where the + * panes then draw exactly as they always have. + */ +class PlotSize : public QObject +{ + Q_OBJECT + +public: + /// Reads the setting and applies it to the view manager + PlotSize(sv::ViewManager *viewManager, QObject *parent); + + /// The checkable steps, one of them checked, for a submenu + QList getActions() const; + + int getPercent() const { return m_percent; } + + static QList getSteps() { return { 100, 150, 200 }; } + + /// 150 on Android, 100 elsewhere + static int getDefaultPercent(); + +public slots: + /// Applies and remembers a step; anything else is ignored + void setPercent(int percent); + +private: + void apply(int percent); + + sv::ViewManager *m_viewManager; + QActionGroup *m_group; + int m_percent; +}; + +#endif diff --git a/main/test/TestCompactLayout.h b/main/test/TestCompactLayout.h index 0d6c45ca..242e2ac4 100644 --- a/main/test/TestCompactLayout.h +++ b/main/test/TestCompactLayout.h @@ -302,6 +302,7 @@ private slots: delete m_window; m_window = nullptr; } + QSettings().remove("MainWindow/plotsize"); QVERIFY2(m_dialogs.isEmpty(), qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); } @@ -683,6 +684,47 @@ private slots: } QVERIFY(m_window->takeBox()->isVisible()); } + + // View > Plot Size (PlotSize): the steps with the default checked, a + // step applied to the panes at once and taken up at the next start. + // On a phone it is in the menu button's popup with the rest of the + // View menu + void plot_size_in_the_view_menu() { + QSettings().remove("MainWindow/plotsize"); + openWindow(); + if (QTest::currentTestFailed()) return; + + QMenu *plotSize = nullptr; + for (QAction *menu: m_window->menuBar()->actions()) { + if (!menu->menu() || !menu->menu()->actions() + .contains(m_window->compact()->getAction())) continue; + for (QAction *action: menu->menu()->actions()) { + if (action->menu() && action->text() == "Plot &Size") { + plotSize = action->menu(); + } + } + } + QVERIFY(plotSize); + + QStringList texts, checked; + for (QAction *action: plotSize->actions()) { + texts << action->text(); + if (action->isChecked()) checked << action->text(); + } + QCOMPARE(texts, QStringList({ "100%", "150%", "200%" })); + QCOMPARE(checked, QStringList({ "100%" })); + QCOMPARE(m_window->viewManager()->getPlotScale(), 1.0); + + plotSize->actions().at(2)->trigger(); + QCOMPARE(m_window->viewManager()->getPlotScale(), 2.0); + + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + openWindow(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->viewManager()->getPlotScale(), 2.0); + } }; #endif diff --git a/main/test/TestPlotSize.h b/main/test/TestPlotSize.h new file mode 100644 index 00000000..bc2d9f5b --- /dev/null +++ b/main/test/TestPlotSize.h @@ -0,0 +1,361 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_PLOT_SIZE_H +#define TEST_PLOT_SIZE_H + +// Tier 3: how large the panes draw the pitch tracks and the notes (the +// svgui fork's plot scale, which a pane applies with its pixel ratio), +// and View > Plot Size (PlotSize). The layers are painted into images +// as a pane paints them into its buffer, through a ViewProxy whose +// pixels are the buffer's, at ratio 1 and at a phone's ratio 3. No +// MainWindow; the menu in the window is TestCompactLayout's. + +#include "../PlotSize.h" + +#include "view/Pane.h" +#include "view/ViewManager.h" +#include "view/ViewProxy.h" +#include "layer/TimeValueLayer.h" +#include "layer/FlexiNoteLayer.h" +#include "layer/ColourDatabase.h" +#include "layer/CoordinateScale.h" +#include "data/model/SparseTimeValueModel.h" +#include "data/model/NoteModel.h" + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +class TestPlotSize : public QObject +{ + Q_OBJECT + + static constexpr double kRate = 44100.0; + static constexpr int kHop = 256; + static constexpr int kWidth = 400; + static constexpr int kHeight = 200; + + sv::ViewManager *m_viewManager = nullptr; + sv::Pane *m_pane = nullptr; + sv::TimeValueLayer *m_pitch = nullptr; + sv::FlexiNoteLayer *m_notes = nullptr; + sv::ModelId m_pitchModel; + sv::ModelId m_noteModel; + + // Columns of the pane, in logical pixels: one through the first + // note's pitch, one through the middle of that note, well away from + // its ends + static constexpr int kPitchX = 50; + static constexpr int kNoteX = 100; + + static sv::sv_frame_t frames(double seconds) { + return sv::sv_frame_t(seconds * kRate); + } + + static QColor colourOf(sv::SingleColourLayer *layer) { + return sv::ColourDatabase::getInstance()->getColour + (layer->getBaseColour()); + } + + // The layer painted into a white image as the pane paints it into + // its buffer at this pixel ratio + QImage render(sv::Layer *layer, int ratio) { + QImage image(kWidth * ratio, kHeight * ratio, + QImage::Format_ARGB32_Premultiplied); + image.fill(Qt::white); + QPainter painter(&image); + sv::ViewProxy proxy(m_pane, ratio); + layer->paint(&proxy, painter, image.rect()); + painter.end(); + return image; + } + + // The first and last rows drawn in a column, or (-1, -1) + static std::pair drawnRows(const QImage &image, int x) { + int first = -1, last = -1; + for (int y = 0; y < image.height(); ++y) { + if (image.pixel(x, y) != qRgb(255, 255, 255)) { + if (first < 0) first = y; + last = y; + } + } + return { first, last }; + } + + static int drawnHeight(const QImage &image, int x) { + auto rows = drawnRows(image, x); + return rows.first < 0 ? 0 : rows.second - rows.first + 1; + } + + // The middle of a logical column, in the pixels of an image at a + // ratio + static int column(int x, int ratio) { + return x * ratio + ratio / 2; + } + + // Whether the note tool, with the pointer at this point of the pane, + // would edit the note there: the pointer changes to the cursor of an + // edit, and to the plain arrow off the note + bool hitsNote(int x, int y) { + m_pane->setCursor(Qt::WaitCursor); + QMouseEvent e(QEvent::MouseMove, QPointF(x, y), QPointF(x, y), + Qt::NoButton, Qt::NoButton, Qt::NoModifier); + m_notes->mouseMoveEvent(m_pane, &e); + Qt::CursorShape shape = m_pane->cursor().shape(); + return shape != Qt::ArrowCursor && shape != Qt::WaitCursor; + } + + static void forgetSetting() { + QSettings settings; + settings.beginGroup("MainWindow"); + settings.remove("plotsize"); + settings.endGroup(); + } + +private slots: + void init() { + forgetSetting(); + + m_viewManager = new sv::ViewManager(); + m_pane = new sv::Pane(); + m_pane->setViewManager(m_viewManager); + m_pane->resize(kWidth, kHeight); + m_pane->setZoomLevel(sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, + kHop)); + m_pane->setStartFrame(0); + + // A second at 220 Hz and a second at 330 Hz, a pitch at every + // hop and a note for each; the scales run from 100 to 500 Hz, + // which keeps them clear of the edges + auto pitch = std::make_shared + (kRate, kHop, 100.f, 500.f, false); + pitch->setScaleUnits("Hz"); + for (sv::sv_frame_t f = 0; f < frames(2.0); f += kHop) { + pitch->add(sv::Event(f, f < frames(1.0) ? 220.f : 330.f, "")); + } + auto notes = std::make_shared + (kRate, kHop, 100.f, 500.f, false, sv::NoteModel::FLEXI_NOTE); + notes->setScaleUnits("Hz"); + notes->add(sv::Event(0, 220.f, frames(1.0), 1.f, "")); + notes->add(sv::Event(frames(1.0), 330.f, frames(1.0), 1.f, "")); + m_pitchModel = sv::ModelById::add(pitch); + m_noteModel = sv::ModelById::add(notes); + + m_pitch = new sv::TimeValueLayer(); + m_pitch->setModel(m_pitchModel); + m_pitch->setPlotStyle(sv::TimeValueLayer::PlotPoints); + m_pitch->setVerticalScale(sv::TimeValueLayer::LinearScale); + m_notes = new sv::FlexiNoteLayer(); + m_notes->setModel(m_noteModel); + // As in Tony, the notes take the scale of the pitch, which + // has one of its own + m_pane->addLayer(m_pitch); + m_pane->addLayer(m_notes); + } + + void cleanup() { + delete m_pane; + m_pane = nullptr; + delete m_pitch; + delete m_notes; + m_pitch = nullptr; + m_notes = nullptr; + sv::ModelById::release(m_pitchModel); + sv::ModelById::release(m_noteModel); + delete m_viewManager; + m_viewManager = nullptr; + forgetSetting(); + } + + // At ratio 1 and the normal size the layers draw what they drew + // before there was a plot scale, pixel for pixel: a point is a + // rectangle two pixels high in the view's pen, a note one sixteen + // high in a pen of a pixel. (A whole pane of Tony's, with its + // waveform, pitch and notes, was compared pixel for pixel before + // and after the change to the fork, once.) + void desktop_draws_as_before() { + sv::CoordinateScale pitchScale = + m_pane->getEffectiveVerticalExtentsForLayer(m_pitch); + sv::CoordinateScale noteScale = + m_pane->getEffectiveVerticalExtentsForLayer(m_notes); + + QImage pitch(kWidth, kHeight, QImage::Format_ARGB32_Premultiplied); + pitch.fill(Qt::white); + { + QPainter paint(&pitch); + QColor colour = colourOf(m_pitch); + QColor fill = colour; + fill.setAlpha(80); + paint.setPen(QPen(colour, m_pane->scalePenWidth(1.0))); + paint.setBrush(fill); + auto model = sv::ModelById::getAs + (m_pitchModel); + int w = m_pane->getXForFrame(kHop) - m_pane->getXForFrame(0); + if (w < 1) w = 1; + for (const auto &e : model->getAllEvents()) { + int x = m_pane->getXForFrame(e.getFrame()); + int y = pitchScale.getCoordForValueRounded(m_pane, e.getValue()); + paint.drawRect(x, y - 1, w, 2); + } + } + + QImage notes(kWidth, kHeight, QImage::Format_ARGB32_Premultiplied); + notes.fill(Qt::white); + { + QPainter paint(¬es); + QColor colour = colourOf(m_notes); + QColor fill = colour; + fill.setAlpha(80); + paint.setPen(colour); + paint.setBrush(fill); + auto model = sv::ModelById::getAs(m_noteModel); + for (const auto &e : model->getAllEvents()) { + int x = m_pane->getXForFrame(e.getFrame()); + int w = m_pane->getXForFrame(e.getFrame() + e.getDuration()) - x; + int y = noteScale.getCoordForValueRounded(m_pane, e.getValue()); + paint.drawRect(x, y - 8, w, 16); + } + } + + QCOMPARE(drawnHeight(pitch, kPitchX), 3); + QCOMPARE(drawnHeight(notes, kNoteX), 17); + QVERIFY(render(m_pitch, 1) == pitch); + QVERIFY(render(m_notes, 1) == notes); + } + + // What is drawn keeps its size in logical pixels: at a phone's ratio + // 3, three times the pixels of ratio 1; and the plot size makes it + // larger again, at either ratio + void sizes_follow_the_ratio_and_the_plot_size() { + for (double scale : { 1.0, 1.5, 2.0 }) { + m_viewManager->setPlotScale(scale); + for (int ratio : { 1, 3 }) { + double size = ratio * scale; + int point = drawnHeight(render(m_pitch, ratio), + column(kPitchX, ratio)); + int note = drawnHeight(render(m_notes, ratio), + column(kNoteX, ratio)); + QString where = QString("at ratio %1 and plot scale %2: a " + "point %3 pixels high, a note %4") + .arg(ratio).arg(scale).arg(point).arg(note); + // Two pixels and the pen's one, a note's sixteen and + // the pen's one, all times the size: to a pixel, which + // a thick pen may round either way + QVERIFY2(std::abs(point - 3 * size) <= 1, qPrintable(where)); + QVERIFY2(std::abs(note - 17 * size) <= 1, qPrintable(where)); + } + } + } + + // The note tool picks a note where it is drawn, in the pane's own + // (logical) coordinates, at either ratio and every plot size + void note_hit_area_is_what_is_drawn() { + for (double scale : { 1.0, 1.5, 2.0 }) { + m_viewManager->setPlotScale(scale); + + int hitTop = -1, hitBottom = -1; + for (int y = 0; y < kHeight; ++y) { + if (hitsNote(kNoteX, y)) { + if (hitTop < 0) hitTop = y; + hitBottom = y; + } + } + QVERIFY(hitTop >= 0); + + for (int ratio : { 1, 3 }) { + auto rows = drawnRows(render(m_notes, ratio), + column(kNoteX, ratio)); + // Edges in logical pixels; the drawing's are the pen's + // outer edges, half a pen outside the note + double drawnTop = double(rows.first) / ratio; + double drawnBottom = double(rows.second + 1) / ratio; + QString where = QString("at ratio %1 and plot scale %2: " + "drawn from %3 to %4, picked from " + "%5 to %6") + .arg(ratio).arg(scale).arg(drawnTop).arg(drawnBottom) + .arg(hitTop).arg(hitBottom + 1); + QVERIFY2(std::abs(hitTop - drawnTop) <= scale, + qPrintable(where)); + QVERIFY2(std::abs(hitBottom + 1 - drawnBottom) <= scale, + qPrintable(where)); + } + } + } + + // --- View > Plot Size ------------------------------------------------- + + void plot_size_starts_at_the_default() { + PlotSize plotSize(m_viewManager, nullptr); + QCOMPARE(PlotSize::getDefaultPercent(), 100); // 150 on Android + QCOMPARE(plotSize.getPercent(), 100); + QCOMPARE(m_viewManager->getPlotScale(), 1.0); + + QStringList texts, checked; + for (QAction *action : plotSize.getActions()) { + texts << action->text(); + if (action->isChecked()) checked << action->text(); + } + QCOMPARE(texts, QStringList({ "100%", "150%", "200%" })); + QCOMPARE(checked, QStringList({ "100%" })); + + // Nothing is stored until a step is chosen + QVERIFY(!QSettings().contains("MainWindow/plotsize")); + } + + void a_step_applies_at_once_and_is_remembered() { + PlotSize plotSize(m_viewManager, nullptr); + QSignalSpy changed(m_viewManager, &sv::ViewManager::plotScaleChanged); + + plotSize.getActions().at(1)->trigger(); + QCOMPARE(m_viewManager->getPlotScale(), 1.5); + QCOMPARE(changed.count(), 1); + QCOMPARE(plotSize.getPercent(), 150); + QVERIFY(plotSize.getActions().at(1)->isChecked()); + QVERIFY(!plotSize.getActions().at(0)->isChecked()); + QCOMPARE(QSettings().value("MainWindow/plotsize").toInt(), 150); + + // The next start takes it up + sv::ViewManager other; + PlotSize again(&other, nullptr); + QCOMPARE(again.getPercent(), 150); + QCOMPARE(other.getPlotScale(), 1.5); + QVERIFY(again.getActions().at(1)->isChecked()); + } + + void only_the_steps_are_taken() { + PlotSize plotSize(m_viewManager, nullptr); + plotSize.setPercent(175); + QCOMPARE(plotSize.getPercent(), 100); + QCOMPARE(m_viewManager->getPlotScale(), 1.0); + QVERIFY(!QSettings().contains("MainWindow/plotsize")); + + for (QVariant stored : { QVariant(175), QVariant("large") }) { + QSettings().setValue("MainWindow/plotsize", stored); + sv::ViewManager other; + PlotSize again(&other, nullptr); + QCOMPARE(again.getPercent(), 100); + QCOMPARE(other.getPlotScale(), 1.0); + } + } +}; + +#endif diff --git a/main/test/TestViewCache.h b/main/test/TestViewCache.h index d25f62c2..b40c6941 100644 --- a/main/test/TestViewCache.h +++ b/main/test/TestViewCache.h @@ -128,6 +128,15 @@ private slots: QCOMPARE(drawn(), std::vector({ 1, 1, 1 })); } + // A new plot size (View > Plot Size) has every layer drawn again, + // at once, at the new size + void a_plot_scale_change_draws_the_cache_again() { + settle(); + m_viewManager->setPlotScale(1.5); + m_pane->repaint(); + QCOMPARE(drawn(), std::vector({ 1, 1, 1 })); + } + // Kept out of the cache, a layer is drawn every time the pane is // painted, and so is every layer in front of it; a change to its // model leaves the layers behind it in the cache diff --git a/main/test/tony-app-test.cpp b/main/test/tony-app-test.cpp index e9567027..68685365 100644 --- a/main/test/tony-app-test.cpp +++ b/main/test/tony-app-test.cpp @@ -19,6 +19,7 @@ #include "TestCompactLayout.h" #include "TestTouchMenuStyle.h" #include "TestLyricsLayer.h" +#include "TestPlotSize.h" #include "TestUiChecks.h" #include "TestAudioCheck.h" @@ -93,6 +94,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestPlotSize t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestRecordWorkflow t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index 8b89eb2c..6cf0f1d8 100644 --- a/meson.build +++ b/meson.build @@ -1229,6 +1229,7 @@ tony_app_files = [ 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', 'main/PaneUtils.cpp', + 'main/PlotSize.cpp', 'main/TakeCommands.cpp', 'main/TakeLayers.cpp', 'main/TouchGestures.cpp', @@ -1258,6 +1259,7 @@ tony_app_moc_headers = [ 'main/CoverageStrip.h', 'main/LyricsTrack.h', 'main/LyricsEditor.h', + 'main/PlotSize.h', 'main/TouchGestures.h', 'main/TouchMenuStyle.h', ] @@ -1598,6 +1600,7 @@ if system != 'android' 'main/test/TestSingingAnalysis.h', 'main/test/TestRecordWorkflow.h', 'main/test/TestLyricsLayer.h', + 'main/test/TestPlotSize.h', 'main/test/TestUiChecks.h', 'main/test/TestAudioCheck.h', 'main/test/TestTouchGestures.h', diff --git a/repoint-lock.json b/repoint-lock.json index 35d53bfa..9f7f2533 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "c685b9726100448e015edb6342dcc67ad7932aa3" + "pin": "8286def6aa4044fbf52672ed199ac831f4c43623" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From de6b3e2053751d172c39f78f3934e777427301c0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:38:29 +0000 Subject: [PATCH 208/275] test: takes stopped at once record something first Three tests look at something during a take and stop it straight away. Under load the fake device had delivered nothing by then, the take was dropped, and stopTake() waited 30 s for an analysis that never came (lyrics_import_disabled_while_recording failed in 2 of 3 sharded runs, and in 1 of 12 runs alone under load). They now wait for half a second of recording first. testing.md also says that the live dots test can miss its 300 ms in a run of eight shards on Qt 6.4. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/testing.md | 9 ++++++++- main/test/TestRecordWorkflow.h | 11 +++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/testing.md b/docs/testing.md index 0040b54d..20a58688 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -72,7 +72,10 @@ Windows path would start an escape in the C string. for Linux. Do not combine shards with test names on the command line. - A sharded run is a whole run of the suites, but the tests that share a process are other ones. After a change to object lifetimes, threads or teardown (see "Timing and races"), - run the one-process run as well. + run the one-process run as well. It also loads the machine more: built against Ubuntu's + Qt 6.4, `TestUiChecks`' `live_dots_under_the_cursor` failed in both of two runs in eight + processes (the tracker itself 313 and 325 ms behind the cursor, over the test's 300 ms) + and passed with `-j 4`. Judge a failure of it there by running it alone. - **On Linux some tests fail whatever the change.** With the Qt of the cloud setup, conda-forge's 6.11 ([building.md](building.md#building-on-linux)), only `TestTakesFile`'s `takes_folder`, `relative_audio_path`, `resolve_audio_path` and @@ -269,6 +272,10 @@ it; the marker goes in the commit that fixes it. There are none at present. after it. A test that acts during that analysis calls `holdRangedMerges(true)` on its `TestMainWindow` before Stop (`Analyser::setRangedMergeHeld()`), and `false` before it waits for `analysed()`, or from a timer where a save's own wait has to let the merge go. +- A take stopped as soon as it started can have nothing in it under load: the fake device + has delivered nothing yet, the take is dropped, and no analysis comes for `stopTake()` + to wait for. A test that only looks at something during a take calls + `waitForSomethingRecorded()` before `stopTake()`. - The status bar is written by three base-class timers; a test that reads it must go through what `showTakeCountdown()` controls. - Deleting a derived layer does not stop its transform; only diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index dd8776e6..c3f082ae 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -178,6 +178,14 @@ class TestRecordWorkflow : public QObject QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); } + // For a test that stops a take as soon as it has looked at something: + // stopped at once, under load the take can have nothing in it, and + // then there is no take for stopTake() to wait for + void waitForSomethingRecorded() { + QTRY_VERIFY_WITH_TIMEOUT + (m_window->recordTarget()->getRecordDuration() > rate / 2, 5000); + } + void take(int ms) { startTake(); if (QTest::currentTestFailed()) return; @@ -5077,6 +5085,7 @@ private slots: m_window->doNewEmptyTake(); QCOMPARE(m_window->takes()->getTakeCount(), 1); + waitForSomethingRecorded(); stopTake(); if (QTest::currentTestFailed()) return; m_window->doUpdateMenuStates(); @@ -6002,6 +6011,7 @@ private slots: startTake(); if (QTest::currentTestFailed()) return; QVERIFY(!reference->isLayerDormant(pane)); + waitForSomethingRecorded(); stopTake(); } @@ -6434,6 +6444,7 @@ private slots: QVERIFY(!m_window->doImportLyricsFrom(path)); QVERIFY(!m_window->lyrics()->isShown()); + waitForSomethingRecorded(); stopTake(); if (QTest::currentTestFailed()) return; QTRY_VERIFY_WITH_TIMEOUT(import->isEnabled(), 2000); From 7ce97ab9d36ec9a9ea6dd061e2162bf10b0af7eb Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:38:29 +0000 Subject: [PATCH 209/275] test: qt 6.4's watchdog no longer ends the longest suite Built against Qt 6.4, QtTest's five-minute watchdog timed the whole suite rather than one test function. With default's lyrics tests TestRecordWorkflow runs for about 330 s, and a one-process run was ended 300 s in, one second into a test that passes; the executable then hung. runSuite() raises the limit to 30 minutes for a Qt older than 6.5, unless QTEST_FUNCTION_TIMEOUT is set. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/testing.md | 7 +++++++ main/test/RunSuite.h | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/docs/testing.md b/docs/testing.md index 20a58688..6e5c36ca 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -87,6 +87,13 @@ Windows path would start an escape in the C string. left running, or after the redo, with `analysedRangeStart()` already 0 — the same race. Five other tests failed so as well until they held the take's merge ("Timing and races"), which these two do not yet. +- **Qt 6.4's watchdog times the whole suite**, not one test function: with + `QTEST_FUNCTION_TIMEOUT=20000` it ended `TestRecordWorkflow` 20 s after the suite began, + 2.5 s into a test. That suite runs for longer than the five-minute default, so + `runSuite()` raises the limit to 30 minutes when built against a Qt older than 6.5. + After such a fatal error the executable does not exit: it spins, or waits for the gdb + that Qt starts for a backtrace. A run that has written nothing for minutes has + stopped; kill it. ## Design principles diff --git a/main/test/RunSuite.h b/main/test/RunSuite.h index ad2e7b4e..6dcdfe9b 100644 --- a/main/test/RunSuite.h +++ b/main/test/RunSuite.h @@ -91,6 +91,15 @@ runSuite(QObject *suite, int argc, char *argv[]) (QString("%1.txt").arg(suite->metaObject()->className())); args << "-o" << (file + ",txt") << "-o" << "-,txt"; } +#if QT_VERSION < QT_VERSION_CHECK(6, 5, 0) + // Qt 6.4's watchdog, meant to end a test function that runs for five + // minutes, can time the whole suite instead: it ended TestRecordWorkflow + // 300 s after the suite began, one second into a test that passes. + // That suite runs for longer, so on that Qt the limit goes beyond it + if (qEnvironmentVariableIsEmpty("QTEST_FUNCTION_TIMEOUT")) { + qputenv("QTEST_FUNCTION_TIMEOUT", "1800000"); + } +#endif return QTest::qExec(suite, args) == 0; } From 0c4f5fd2706e48f61da3b83619b323d1f776bba6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:46:17 +0000 Subject: [PATCH 210/275] test: a take is stopped only once it has some audio in it stopTake() stopped the take and waited for pYIN on it. a take stopped as soon as it started had no audio on a loaded machine, whose device had delivered nothing yet: "nothing to use: 0 frames recorded", no analysis, and a 30 s wait that failed. lyrics_import_disabled_while_ recording failed so on all three ci platforms, and here under load in two runs of three. stopTake() now waits first for a tenth of a second of audio, in testrecordworkflow and in testuichecks. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- main/test/TestRecordWorkflow.h | 7 ++++++- main/test/TestUiChecks.h | 3 +++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 62b530df..bf70f9ad 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -170,9 +170,14 @@ class TestRecordWorkflow : public QObject QVERIFY(m_window->recordTarget()->isRecording()); } - // Stop, then wait for pYIN on the take + // Stop, then wait for pYIN on the take. Not before the take has some + // audio in it: a take stopped as soon as it started has none on a + // loaded machine, whose device has delivered nothing yet, and then + // there is no analysis to wait for void stopTake() { QVERIFY(m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->getFramesReceived() + >= sv::sv_frame_t(0.1 * rate), 5000); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); diff --git a/main/test/TestUiChecks.h b/main/test/TestUiChecks.h index 0b9f78b5..bce1dcbf 100644 --- a/main/test/TestUiChecks.h +++ b/main/test/TestUiChecks.h @@ -151,8 +151,11 @@ class TestUiChecks : public QObject QVERIFY(m_window->recordTarget()->isRecording()); } + // Not before the take has some audio in it: see TestRecordWorkflow's void stopTake() { QVERIFY(m_window->recordTarget()->isRecording()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->getFramesReceived() + >= sv::sv_frame_t(0.1 * rate), 5000); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); From f09f920eb2080d7bade7ccc0095f59b7f1d68478 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:49:39 +0000 Subject: [PATCH 211/275] docs: work orders for a11 and a4c From the fifth phone test: the cursor races ahead while recording at 48 kHz, Save Log wrote an empty file, and a vertical zoom pushes a low pitch out of the pane. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 58 +++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index b2fcd7e1..5b85c589 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -179,6 +179,8 @@ builds happen in the container.) - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. - A9 — Live dots in real time on the phone. Done. - A10 — Plot elements sized for the screen. Done. +- A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log. +- A4c — Vertical zoom keeps the pitch in view. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -527,6 +529,62 @@ Wanted: - Tests: the sizes at ratio 1 and 3 and the setting's steps, in images rendered offscreen (`QT_SCALE_FACTOR` or a `QImage` with a device pixel ratio). +### A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log + +The fifth phone test (2026-09-26, APK at `1979ccc`): the live dots now keep up and sit +where they should, and the reference plays in time during a take; pitch and notes are +easier to see. Two faults: + +- **While recording, the play cursor races ahead** of the dots and of what is heard, further + and further as the take goes on. This is the known fault of section 4 (A1): svgui's + `ViewManager::getPlaybackFrame()` gives, while recording, `m_recordStartFrame + + m_recordTarget->getRecordDuration()`, and the duration counts the device's frames + (48 kHz on the phone) against a pane at the reference's rate (44.1 kHz): 8.8 % fast. + `QEXPECT_FAIL` in `takes_placed_from_a_device_at_48000` marks it. +- **Help > Save Log... saved an empty file.** The log was not empty (else "There is no log + to save"), and no error was shown. `MainWindow::saveLog()` writes to the document the + picker made through `AndroidStorage::openDocument(uri, "w")`, which takes the file + descriptor with `ParcelFileDescriptor.detachFd()` and later `::close()`s it. A provider + that opened the document with a close listener (the media store behind Downloads, a cloud + app) is then told the client detached, not that it finished, and may drop what was + written. Reading (`"r"`, the audio copied in by A7b) uses the same call. + +Wanted: + +- The cursor at the reference's frames while recording at any device rate: the smallest + change in the svgui fork, which you may edit (branch `feat/tonyandroid`, checked out; + `view: what`; committed there, not pushed; the lead pushes and pins). svcore's + `AudioRecordTarget` has no rate; Tony knows it once the take has started + (`TakeTiming::recordRate`), and already tells `ViewManager` where the take starts + (`setRecordStartFrame()` in `record()`), so it can tell it the ratio as well. The + `QEXPECT_FAIL` goes and the check passes. +- Documents written and read through the picker's grant are closed the way Android + expects: keep the `ParcelFileDescriptor` and `close()` it (or write through + `ContentResolver.openOutputStream()`), with `"wt"` for a write. After Save Log, check + what the document holds (its size, through the resolver) and say in the log, and in the + message if it differs, how many bytes went in and how many are there. Android-only code; + what is pure goes in `tony_core` with a test. +- Say in the report what the phone test should look for in the log. + +### A4c — Vertical zoom keeps the pitch in view + +The fifth phone test: pinching to zoom the frequency range zooms about the frequency under +the fingers (A4b). The Avi Kaplan song is low, and zooming in pushes the pitch below the +bottom edge; the user: "The smart thing to do would be to keep the plot centered somehow +when zooming in." + +- A vertical zoom anchors at the middle, on the pane's log scale, of the pitch that is on + show in the pane's time range: the reference's pitch track and notes, and the singing's, + whichever are shown. With none on show, it anchors under the fingers as now. The + horizontal part of a pinch and the two-finger drag (vertical scroll) stay as they are. +- The anchor's choice is pure (`VerticalZoom` in `tony_core`: given the values on show and + the current range, the anchor) and tested there; `MainWindow` gathers the values + (`TouchGestures::VerticalRange` is where the pane's range comes from now, see + `MainWindow::paneAdded`), and a test through synthetic touch events shows a low pitch + staying in view as the range narrows. +- Not asked, so not built: following the pitch vertically during playback, a "fit the + pitch" action. Say in the report if either looks needed. + ### A8 — Documentation pass - Bring the docs pages up to date from the code and the log: building.md (the container From 00d9339c20f0089ab8896f7026082b684ab3fd02 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 15:10:48 +0000 Subject: [PATCH 212/275] fix: the cursor keeps the reference's pace at 48 kHz; Save Log closes properly While recording, the cursor added the device's frames to the take's start, so on a phone at 48 kHz it ran 8.8 % ahead of the reference. record() now gives svgui's ViewManager the ratio of the two rates once the recording has started (pins svgui 049c6d9). Documents opened through the picker's grant were detached from their ParcelFileDescriptor, which tells the provider the client is done before anything is written, and Save Log's file came out empty. They are now closed through it, written with "wt", and Save Log checks the size the provider reports afterwards, logging it and warning when it differs. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 33 +++++++-- docs/forks.md | 4 +- docs/recording.md | 5 +- main/AndroidFiles.cpp | 11 +++ main/AndroidFiles.h | 19 +++++- main/AndroidStorage.cpp | 119 ++++++++++++++++++++++++++++----- main/AndroidStorage.h | 55 +++++++++++++-- main/MainWindow.cpp | 114 ++++++++++++++++++++++++++----- main/TakeTiming.cpp | 9 +++ main/TakeTiming.h | 4 ++ main/test/TestAndroidFiles.h | 32 ++++++++- main/test/TestRecordWorkflow.h | 28 ++++++-- main/test/TestTakeTiming.h | 4 ++ repoint-lock.json | 2 +- 14 files changed, 385 insertions(+), 54 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 5b85c589..22ffb07a 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -151,10 +151,10 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: frames received, the live tracker) and reference frames (`recordRate`, `recordedToReference()`, `referenceToRecorded()`). An Oboe backend at 48 kHz needs nothing more from Tony for placement. -- Known and not fixed: while recording at a device rate other than 44.1 kHz the play - cursor runs fast (svgui's `ViewManager::getPlaybackFrame()` adds device frames). A - `QEXPECT_FAIL` in `takes_placed_from_a_device_at_48000` marks it. It needs changes in - svcore, svgui and svapp; the lead has not made them. +- While recording at a device rate other than 44.1 kHz the play cursor ran fast (svgui's + `ViewManager::getPlaybackFrame()` added device frames). Fixed in A11 with svgui alone + (`ViewManager::setRecordFrameRatio()`, which `record()` sets once the recording has + started), not svcore and svapp as was thought; the `QEXPECT_FAIL` is gone. ## 5. Phases @@ -179,7 +179,7 @@ builds happen in the container.) - A7c — M4A/AAC and other formats through Android's decoders; no autosave of an incomplete session. Done. - A9 — Live dots in real time on the phone. Done. - A10 — Plot elements sized for the screen. Done. -- A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log. +- A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log. Done. - A4c — Vertical zoom keeps the pitch in view. - A8 — Documentation pass. @@ -949,3 +949,26 @@ strip and lyrics do not follow the plot size; forks.md not updated (A8). Not on Tests seen failing: ViewProxy without the ratio (the old sizes at 3): sizes, hit area; plot scale x1.1 at 100 %: desktop_draws_as_before; no connection: the cache test; PlotSize not applying: its step test and the menu test. + +### Phase A11 — 2026-09-26 +Built: svgui `049c6d9` (view, not pushed or pinned): `ViewManager::setRecordFrameRatio()` +(default 1); while recording the playback frame is the record start frame plus the duration +times it (`getRecordingFrame()`, both places). `TakeTiming::referenceFramesPerRecordedFrame()` +(tested). `record()` sets the ratio to 1 before the base call (the device's rate is unknown +then; a recording that becomes the session is at its own rate) and to the take's after it, +when the recording model is known. `takes_placed_from_a_device_at_48000`: `QEXPECT_FAIL` +gone; `preroll_and_punch_out_with_a_device_at_48000` checks the cursor after the lead-in. +The lyrics highlight follows the same frame, so it too was 8.8% fast in a 48 kHz take. +Save Log: `AndroidStorage::Document` replaces `openDocument()`: it keeps the +ParcelFileDescriptor, hands out `getFd()`, and `close()` asks `checkError()` (if it +`canDetectErrors()`: a reliable pipe) and closes through Java. The likely cause, from the +source: `detachFd()` sends the peer DETACHED at once, before anything is written, so a +provider that takes the file on its close listener took it empty. Writes "wt" (then "w" if +refused); `flush()` checked; the size behind the descriptor (`getStatSize()`) logged; +`_size` through the resolver (`sizeOf()`), asked up to 10 times 100 ms apart while it +differs (the provider's listener runs on its own thread). `AndroidFiles::savedSize()` +(core, tested). A warning box when the provider gives another size; the log line always. +The audio copy-in reads the same way and drops a copy whose close reports an error. +Tests seen failing: both cursor checks with the ratio left at 1; `savedSize()` taking "" as 0. +Left open: nothing run on a phone. forks.md (svgui) and recording.md "Start click" step 2 and +6 name the start frame plus the duration: for A8. diff --git a/docs/forks.md b/docs/forks.md index 3f8bc112..88b36a6f 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -92,7 +92,9 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` - `ViewManager::setRecordStartFrame()` / `getRecordStartFrame()`: while recording, the playback frame is this plus the recorded duration, not the duration alone. Without it a take recorded at P > 0 showed the cursor crawling from frame 0 and the pane scrolling - away from the dots. + away from the dots. `setRecordFrameRatio()` (branch `feat/tonyandroid`) scales the + duration, which the record target counts in the device's frames, to the timeline's: a + phone at 48 kHz against a reference at 44.1 kHz. - `RegionLayer::PlotStrip` plot style: the coverage strip. Saved through the existing `plotStyle` attribute. - `RegionLayer::PlotLyrics` plot style, after `PlotStrip` so saved numbers keep their diff --git a/docs/recording.md b/docs/recording.md index 20e0d278..cc730cfb 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -50,7 +50,10 @@ the reference at P is therefore at file frame **L + R**, and the splice reads fr 6. Record mode is switched to **`RecordCreateUnshownModel`** (svapp fork) around the base call: the recording becomes a model of the document with no pane, no layer and no "Import Recorded Audio" undo entry. `ViewManager::setRecordStartFrame(S)` (svgui fork) - makes the cursor run with the reference instead of crawling from frame 0. + makes the cursor run with the reference instead of crawling from frame 0; after the + base call, `setRecordFrameRatio()` gives it the reference's frames per recorded frame + (`TakeTiming::referenceFramesPerRecordedFrame()`), without which a device at 48 kHz ran + the cursor 8.8 % ahead of the reference. It is set back to 1 before every base call. 7. `recordStatusChanged(true)` → `recordingStarted()` fires *inside* the base call, before the model is in the document. It defers with `QTimer::singleShot(0)`: `setupRealtimePitchLayer()`, and, if Play Reference While Recording is on, the latency diff --git a/main/AndroidFiles.cpp b/main/AndroidFiles.cpp index d4c5cdad..09c6f970 100644 --- a/main/AndroidFiles.cpp +++ b/main/AndroidFiles.cpp @@ -473,3 +473,14 @@ AndroidFiles::removeIfEmpty(QString path) SVCERR << "AndroidFiles: removed the empty " << path << endl; return true; } + +AndroidFiles::SavedSize +AndroidFiles::savedSize(qint64 written, QString held) +{ + SavedSize saved; + saved.written = written; + bool ok = false; + qint64 size = held.trimmed().toLongLong(&ok); + if (ok && size >= 0) saved.held = size; + return saved; +} diff --git a/main/AndroidFiles.h b/main/AndroidFiles.h index 9e5884e8..3c0c8378 100644 --- a/main/AndroidFiles.h +++ b/main/AndroidFiles.h @@ -71,8 +71,8 @@ class AndroidFiles /** * The same, from in, open for reading: on Android a file descriptor - * the file's provider gave (AndroidStorage::openDocument()), which - * may be a pipe. sourceName says where it came from, for the log. + * the file's provider gave (AndroidStorage::Document), which may be + * a pipe. sourceName says where it came from, for the log. */ static QString copyIn(QIODevice &in, QString sourceName, QString name, QString dir, QString &error); @@ -198,6 +198,21 @@ class AndroidFiles * provider. True if it removed one. */ static bool removeIfEmpty(QString path); + + /** + * What a document holds after a save through its provider, against + * what was written to it: Save Log says both in the log, and tells + * the user when they differ. held is the document's _size column as + * ContentResolver.query() gives it, "" for a null: a provider that + * gives no size says nothing either way. + */ + struct SavedSize { + qint64 written = 0; + qint64 held = -1; // -1: the provider gives no size + bool known() const { return held >= 0; } + bool differs() const { return known() && held != written; } + }; + static SavedSize savedSize(qint64 written, QString held); }; #endif diff --git a/main/AndroidStorage.cpp b/main/AndroidStorage.cpp index 7be6c6a3..97b525dd 100644 --- a/main/AndroidStorage.cpp +++ b/main/AndroidStorage.cpp @@ -278,46 +278,135 @@ AndroidStorage::displayName(QString uri) return ""; } -int -AndroidStorage::openDocument(QString uri, QString mode, QString &error) +AndroidStorage::Document::Document() : + m_fd(-1) { +} + +AndroidStorage::Document::~Document() +{ + if (!m_descriptor.isValid()) return; + QString error; + if (!close(error)) { + cerr << "AndroidStorage: closing a document: " << error.toStdString() + << endl; + } +} + +bool +AndroidStorage::Document::open(QString uri, QString mode, QString &error) +{ + if (m_descriptor.isValid()) { + error = "a document is open already"; + return false; + } + QJniObject resolver = contentResolver(); QJniObject parsed = parseUri(uri); if (!resolver.isValid() || !parsed.isValid()) { error = "no content resolver"; - return -1; + return false; } QJniEnvironment env; jclass resolverClass = env->GetObjectClass(resolver.object()); - jmethodID open = env->GetMethodID + jmethodID openMethod = env->GetMethodID (resolverClass, "openFileDescriptor", "(Landroid/net/Uri;Ljava/lang/String;)" "Landroid/os/ParcelFileDescriptor;"); env->DeleteLocalRef(resolverClass); - if (!open) { + if (!openMethod) { error = takeException(env); - return -1; + return false; } QJniObject modeString = QJniObject::fromString(mode); jobject descriptor = env->CallObjectMethod - (resolver.object(), open, parsed.object(), modeString.object()); + (resolver.object(), openMethod, parsed.object(), modeString.object()); QString thrown = takeException(env); if (thrown != "") { error = thrown; - return -1; + return false; } if (!descriptor) { - error = "the provider gave nothing to read"; - return -1; + error = "the provider gave nothing to open"; + return false; } - // Ours to close from here - QJniObject pfd = QJniObject::fromLocalRef(descriptor); - int fd = pfd.callMethod("detachFd", "()I"); - if (fd < 0) error = "the provider gave no file descriptor"; - return fd; + // The descriptor stays the ParcelFileDescriptor's, and is closed + // through it + m_descriptor = QJniObject::fromLocalRef(descriptor); + m_fd = m_descriptor.callMethod("getFd", "()I"); + if (m_fd < 0) { + QString ignored; + close(ignored); + error = "the provider gave no file descriptor"; + return false; + } + return true; +} + +qint64 +AndroidStorage::Document::fileSize() const +{ + if (!m_descriptor.isValid()) return -1; + return qint64(m_descriptor.callMethod("getStatSize", "()J")); +} + +bool +AndroidStorage::Document::close(QString &error) +{ + if (!m_descriptor.isValid()) return true; + + QJniEnvironment env; + QString problem; + jclass descriptorClass = env->GetObjectClass(m_descriptor.object()); + // Each looked up with no exception pending, as JNI requires + auto method = [&](const char *name, const char *signature) { + jmethodID id = env->GetMethodID(descriptorClass, name, signature); + QString thrown = takeException(env); + if (problem == "") problem = thrown; + return id; + }; + jmethodID canDetect = method("canDetectErrors", "()Z"); + jmethodID checkError = method("checkError", "()V"); + jmethodID closeMethod = method("close", "()V"); + env->DeleteLocalRef(descriptorClass); + + // A provider that hands over a pipe says through it whether it went + // wrong at its end, which an end of file alone does not: asked + // before the close, which would not say + if (problem == "" && canDetect && checkError && + env->CallBooleanMethod(m_descriptor.object(), canDetect)) { + env->CallVoidMethod(m_descriptor.object(), checkError); + problem = takeException(env); + } + + if (closeMethod) { + env->CallVoidMethod(m_descriptor.object(), closeMethod); + QString thrown = takeException(env); + if (problem == "") problem = thrown; + } + + m_descriptor = QJniObject(); + m_fd = -1; + + if (problem != "") { + error = problem; + return false; + } + return true; +} + +QString +AndroidStorage::sizeOf(QString uri, QString &error) +{ + QList rows = query(uri, { "_size" }, "", {}, error); + if (rows.size() == 1 && rows[0].size() == 1) return rows[0][0]; + if (error == "") { + error = QString("the provider answered %1 rows").arg(rows.size()); + } + return ""; } bool diff --git a/main/AndroidStorage.h b/main/AndroidStorage.h index dfca1f79..00e04e58 100644 --- a/main/AndroidStorage.h +++ b/main/AndroidStorage.h @@ -16,6 +16,7 @@ #define TONY_ANDROID_STORAGE_H #include +#include #include class QWidget; @@ -57,12 +58,54 @@ class AndroidStorage // gives none static QString displayName(QString uri); - // A file descriptor for the document at uri, open to read ("r") or - // write ("w"), which the caller closes (QFile's AutoCloseHandle); -1 - // on failure, with error saying why. Called here rather than through - // QFile, whose content file engine rebuilds the URI, differently for - // names with parentheses, and then has no grant for it - static int openDocument(QString uri, QString mode, QString &error); + /** + * A document opened through its provider, from the URI as Android + * wrote it: not through QFile, whose content file engine rebuilds + * the URI, differently for names with parentheses, and then has no + * grant for it. fd() is read or written as it is (QFile's + * DontCloseHandle) and stays the provider's ParcelFileDescriptor's: + * close() closes it through that, which is what tells a provider + * that watches for the close (MediaStore behind Downloads, a cloud + * app) that Tony has finished with the document. A descriptor taken + * from it instead (detachFd()) tells the provider so at once, before + * anything is written, and a provider may then keep nothing. + */ + class Document + { + public: + Document(); + // Closes the document if it is still open + ~Document(); + + // Opens the document at uri to read ("r") or write ("wt", which + // leaves only what is written now); false on failure, with error + // saying why + bool open(QString uri, QString mode, QString &error); + + bool isOpen() const { return m_fd >= 0; } + int fd() const { return m_fd; } + + // The size of the file behind the descriptor, as fstat() gives + // it; -1 for a pipe, which a provider may give instead of a file + qint64 fileSize() const; + + // Closes the document through its provider. False if that failed, + // or if the provider has said through a pipe that it went wrong + // at its end, with error saying why + bool close(QString &error); + + private: + QJniObject m_descriptor; + int m_fd; + + Document(const Document &) = delete; + Document &operator=(const Document &) = delete; + }; + + // The size of the document at uri as its provider gives it (its + // _size column): "" if it gives none, or could not be asked, with + // error saying why (see AndroidFiles::savedSize()) + static QString sizeOf(QString uri, QString &error); // Removes the document at uri if it is empty: the one the picker // makes for a save, when the save is not made there. True if it did diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 4fe9c8af..3223c06b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -38,12 +38,12 @@ #include "OboeAudioIO.h" #include "data/fileio/AudioFileReaderFactory.h" #include +#include #include #include #include #include #include -#include #endif #ifdef TONY_DEV_CHECKS @@ -3120,15 +3120,28 @@ MainWindow::getOpenFileName(FileFinder::FileType type) (QStandardPaths::AppDataLocation) + "/imported"; QString error; QString copy; - int fd = AndroidStorage::openDocument(uri, "r", error); - if (fd >= 0) { + AndroidStorage::Document document; + if (document.open(uri, "r", error)) { QFile in; - if (in.open(fd, QIODevice::ReadOnly, QFileDevice::AutoCloseHandle)) { + if (in.open(document.fd(), QIODevice::ReadOnly, + QFileDevice::DontCloseHandle)) { copy = AndroidFiles::copyIn(in, uri, name, dir, error); + in.close(); } else { - ::close(fd); error = in.errorString(); } + // A provider that streams the file through a pipe says at the + // close whether all of it came: a copy of part of it is no copy + QString closeError; + if (!document.close(closeError)) { + cerr << "MainWindow::getOpenFileName: closing " << uri << ": " + << closeError << endl; + if (copy != "") { + QFile::remove(copy); + copy = ""; + error = closeError; + } + } } if (copy == "") { QMessageBox::critical @@ -3190,26 +3203,43 @@ MainWindow::saveLog() QString error; QFile out; bool opened = false; + AndroidStorage::Document document; if (urls[0].isLocalFile()) { out.setFileName(target); opened = out.open(QIODevice::WriteOnly | QIODevice::Truncate); if (!opened) error = out.errorString(); } else { - int fd = AndroidStorage::openDocument(target, "w", error); - if (fd >= 0) { - opened = out.open(fd, QIODevice::WriteOnly, - QFileDevice::AutoCloseHandle); - if (!opened) { - ::close(fd); - error = out.errorString(); - } + // "wt", so that a document written over holds only what is + // written now; "w" if the provider will not take that, which for + // the new, empty document the picker made comes to the same + if (!document.open(target, "wt", error)) { + cerr << "MainWindow::saveLog: " << target << " could not be " + << "opened with \"wt\": " << error << "; trying \"w\"" + << endl; + QString again; + if (!document.open(target, "w", again)) error = again; + } + if (document.isOpen()) { + opened = out.open(document.fd(), QIODevice::WriteOnly, + QFileDevice::DontCloseHandle); + if (!opened) error = out.errorString(); } } - bool written = opened && out.write(log) == log.size(); + bool written = opened && out.write(log) == log.size() && out.flush(); if (opened && !written) error = out.errorString(); if (opened) out.close(); + // What the file behind the descriptor holds before it goes back to + // the provider (-1 if the provider gave a pipe), and then the close, + // through the provider, which is when it takes what was written + qint64 inFile = document.fileSize(); + QString closeError; + if (!document.close(closeError) && written) { + written = false; + error = closeError; + } + if (!written) { cerr << "MainWindow::saveLog: could not write " << target << ": " << error << endl; @@ -3219,8 +3249,48 @@ MainWindow::saveLog() .arg(error.toHtmlEscaped()) + pickDetails(target, "", "")); return; } - cerr << "MainWindow::saveLog: saved " << log.size() << " bytes to " - << target << endl; + + if (urls[0].isLocalFile()) { + cerr << "MainWindow::saveLog: saved " << log.size() << " bytes to " + << target << endl; + return; + } + + // What the document holds now, as its provider tells anyone who asks. + // A provider hears of the close on a thread of its own and may say so + // a little later: asked again, for up to a second, while it gives + // another size than was written + QElapsedTimer waited; + waited.start(); + AndroidFiles::SavedSize saved; + QString sizeError; + for (int attempt = 1; ; ++attempt) { + sizeError = ""; + saved = AndroidFiles::savedSize + (log.size(), AndroidStorage::sizeOf(target, sizeError)); + if (!saved.differs() || attempt == 10) break; + QThread::msleep(100); + } + + cerr << "MainWindow::saveLog: wrote " << saved.written << " bytes to " + << target << "; the file behind the descriptor held " << inFile + << " before the close; "; + if (saved.known()) { + cerr << "its provider says the document holds " << saved.held + << " bytes"; + } else { + cerr << "its provider gives no size for the document"; + if (sizeError != "") cerr << " (" << sizeError << ")"; + } + cerr << ", " << waited.elapsed() << " ms after the close" << endl; + + if (saved.differs()) { + QMessageBox::warning + (this, tr("The log may be incomplete"), + tr("The log may not have been saved whole

%1 bytes were written to it, and the app that keeps it says it holds %2.

") + .arg(saved.written).arg(saved.held) + + pickDetails(target, "", "")); + } } bool @@ -5252,7 +5322,13 @@ MainWindow::record() // the start of the lead-in, so the cursor runs through the lead-in in // step with the reference. sv_frame_t playbackStart = currentTakeTiming().playbackStart(); - if (m_viewManager) m_viewManager->setRecordStartFrame(playbackStart); + if (m_viewManager) { + m_viewManager->setRecordStartFrame(playbackStart); + // What has been recorded is counted in the device's frames, and + // the device's rate is known only once it records: until then, + // and for a recording that becomes the session, one for one + m_viewManager->setRecordFrameRatio(1.0); + } MainWindowBase::record(); @@ -5276,6 +5352,10 @@ MainWindow::record() if (m_recordingAsSingingTrack && m_viewManager) { m_viewManager->setPlaybackFrame(playbackStart); m_viewManager->setGlobalCentreFrame(playbackStart); + // A device at 48 kHz against the reference at 44.1 would + // otherwise run the cursor 8.8% ahead of the reference + m_viewManager->setRecordFrameRatio + (currentTakeTiming().referenceFramesPerRecordedFrame()); } updateAlternatePitchForTake(); diff --git a/main/TakeTiming.cpp b/main/TakeTiming.cpp index 817e14e4..c5e77c99 100644 --- a/main/TakeTiming.cpp +++ b/main/TakeTiming.cpp @@ -50,6 +50,15 @@ TakeTiming::referenceToRecorded(sv_frame_t referenceFrames) const return sv_frame_t(std::llround(double(referenceFrames) * recordRate / rate)); } +double +TakeTiming::referenceFramesPerRecordedFrame() const +{ + if (rate <= 0 || recordRate <= 0 || recordRate == rate) { + return 1.0; + } + return double(rate) / double(recordRate); +} + sv_frame_t TakeTiming::preRollBefore(sv_frame_t position, sv_frame_t wanted) { diff --git a/main/TakeTiming.h b/main/TakeTiming.h index d6d508a1..1f427ff1 100644 --- a/main/TakeTiming.h +++ b/main/TakeTiming.h @@ -77,6 +77,10 @@ struct TakeTiming /// Frames of the reference's timeline as frames of the recording sv::sv_frame_t referenceToRecorded(sv::sv_frame_t referenceFrames) const; + /// How many of the reference's frames one frame of the recording + /// is: 1 unless both rates are known and differ + double referenceFramesPerRecordedFrame() const; + /** * The lead-in there is room for before position: the pre-roll * asked for, shortened near the start of the song and nothing at diff --git a/main/test/TestAndroidFiles.h b/main/test/TestAndroidFiles.h index c52a7a17..ceb12ba6 100644 --- a/main/test/TestAndroidFiles.h +++ b/main/test/TestAndroidFiles.h @@ -18,7 +18,8 @@ // links) and when a file is picked (the copy into app storage, the path // of a picked content:// URI or where to look it up, the URI string // Android granted, the files Tony opens, the names Save Session As -// suggests and accepts, the recent files that are still there), done +// suggests and accepts, the recent files that are still there, what Save +// Log makes of the size a provider gives for its document), done // here on plain files in a temporary directory and on URIs written as // Android writes them. What only a phone has -- a provider behind the // URI, MediaStore, the installed library directory -- is not here. @@ -594,6 +595,35 @@ private slots: QVERIFY(!AndroidFiles::removeIfEmpty("")); } + // Save Log compares what went into the document with what its + // provider says it holds afterwards, from the _size column + void a_saved_document_is_compared_with_what_was_written() { + auto whole = AndroidFiles::savedSize(51234, "51234"); + QVERIFY(whole.known()); + QVERIFY(!whole.differs()); + QCOMPARE(whole.written, qint64(51234)); + QCOMPARE(whole.held, qint64(51234)); + + // The fifth phone test: all of it written, nothing there + auto empty = AndroidFiles::savedSize(51234, "0"); + QVERIFY(empty.known()); + QVERIFY(empty.differs()); + QCOMPARE(empty.held, qint64(0)); + + QVERIFY(AndroidFiles::savedSize(51234, "4096").differs()); + QVERIFY(AndroidFiles::savedSize(0, "0").known()); + QVERIFY(!AndroidFiles::savedSize(0, "0").differs()); + + // A null column, as a provider that does not know the size gives + // it, is not a difference; nor is anything that is not a count + for (QString none : { "", " ", "-1", "big", "12 bytes" }) { + auto unknown = AndroidFiles::savedSize(51234, none); + QVERIFY2(!unknown.known(), qPrintable(none)); + QVERIFY2(!unknown.differs(), qPrintable(none)); + QCOMPARE(unknown.held, qint64(-1)); + } + } + // --- The Vamp plugin links --- void the_plugins_are_linked_under_their_own_names() { diff --git a/main/test/TestRecordWorkflow.h b/main/test/TestRecordWorkflow.h index 54403375..73cbd48d 100644 --- a/main/test/TestRecordWorkflow.h +++ b/main/test/TestRecordWorkflow.h @@ -1357,11 +1357,6 @@ class TestRecordWorkflow : public QObject m_window->recordTarget()->getRecordDuration(); sv::sv_frame_t cursorWant = P1 + sv::sv_frame_t (std::llround(double(recordedSoFar) * rate / deviceRate)); - if (deviceRate != int(rate)) { - QEXPECT_FAIL("", "ViewManager::getPlaybackFrame() adds the " - "recorded duration in the device's frames to the " - "record start frame (svgui fork)", Continue); - } QVERIFY2(std::llabs(during - cursorWant) <= 1, qPrintable(QString("%1 frames at %2 Hz into a take from " "frame %3, the cursor is at %4, not %5") @@ -2525,6 +2520,29 @@ private slots: path = recording->getLocation(); } + // Once the lead-in is over, the cursor has run through it from + // its start at the reference's pace, not at the device's faster + // one. The record target's duration is the GUI thread's, as the + // cursor's is, so the two are read at the same point + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->getRecordDuration() + > sv::sv_frame_t(leadIn * otherDeviceRate), + 5000); + QVERIFY(m_window->recordTarget()->isRecording()); + sv::sv_frame_t during = m_window->playbackFrame(); + sv::sv_frame_t recordedSoFar = + m_window->recordTarget()->getRecordDuration(); + sv::sv_frame_t cursorWant = P - m_window->takePreRoll() + + sv::sv_frame_t(std::llround(double(recordedSoFar) * rate / + otherDeviceRate)); + QVERIFY2(std::llabs(during - cursorWant) <= 1, + qPrintable(QString("%1 frames at %2 Hz into a take whose " + "lead-in starts at frame %3, the cursor " + "is at %4, not %5") + .arg(recordedSoFar).arg(otherDeviceRate) + .arg(P - m_window->takePreRoll()) + .arg(during).arg(cursorWant))); + QVERIFY(during >= P); + QTRY_VERIFY_WITH_TIMEOUT(!m_window->recordTarget()->isRecording(), 5000); QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); diff --git a/main/test/TestTakeTiming.h b/main/test/TestTakeTiming.h index a68813b0..0c4fcdac 100644 --- a/main/test/TestTakeTiming.h +++ b/main/test/TestTakeTiming.h @@ -179,6 +179,8 @@ private slots: QCOMPARE(t.recordedToReference(frame_t(deviceRate)), frame_t(kRate)); QCOMPARE(t.referenceToRecorded(frame_t(kRate)), frame_t(deviceRate)); QCOMPARE(t.recordedToReference(12345), frame_t(11342)); + // The same as a ratio, for the cursor while the take records + QCOMPARE(t.referenceFramesPerRecordedFrame(), kRate / deviceRate); // The splice reads the recording once it is at the reference's // rate: the latency is converted, the lead-in is not @@ -221,6 +223,8 @@ private slots: QCOMPARE(same.spliceOffset(), frame_t(4500)); QCOMPARE(take(5000, 3000, 1500).recordedToReference(12345), frame_t(12345)); + QCOMPARE(same.referenceFramesPerRecordedFrame(), 1.0); + QCOMPARE(take(5000, 3000, 1500).referenceFramesPerRecordedFrame(), 1.0); } // Recording into a selection: the one the playhead is in, else the diff --git a/repoint-lock.json b/repoint-lock.json index 9f7f2533..65a3c156 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "8286def6aa4044fbf52672ed199ac831f4c43623" + "pin": "049c6d9f53bf0c7a1f7273e89aa06f0dce05868a" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 60df16d58879a2eddba6ad601baa98758c084703 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 15:15:37 +0000 Subject: [PATCH 213/275] test: macos ci checks the app it builds, and packages nothing its tests all pass now, and the job failed at upstream's deploy step: that looked in /usr/local/opt, which is not where homebrew lives on the arm runner, ran deploy/macos/deploy.sh, which this fork has as deploy/osx, and those scripts need qmake's Makefile and a Tony.app, neither of which the meson build makes. the step goes, and the check after it looks at build/tony instead of the bundle. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/macos.yml | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/.github/workflows/macos.yml b/.github/workflows/macos.yml index 03d0c25a..0bada740 100644 --- a/.github/workflows/macos.yml +++ b/.github/workflows/macos.yml @@ -32,11 +32,9 @@ jobs: /^(FAIL!|XPASS|QFATAL)|Received signal/ { print; fail = 1; next } /^[A-Z!]+ *: / { fail = 0 } fail && /^ / { print }' build/meson-logs/testlog.txt - - name: deploy-app - run: | - ls -lR /usr/local/opt - QTDIR=/usr/local/opt/qt6 ./deploy/macos/deploy.sh "Tony" + # No app bundle: the packaging scripts in deploy/osx are qmake's, + # and this build makes none. The app it does build has to start - name: check run: | - otool -L ./"Tony.app/Contents/MacOS/Tony" - ./"Tony.app/Contents/MacOS/Tony" --version + otool -L build/tony + build/tony --version From 17e0a3561b42c00f3e9fbc77a4bc8d76b8eea18f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 15:33:03 +0000 Subject: [PATCH 214/275] feat: vertical zoom keeps the pitch in view A pinch up the pane zooms the frequency range about the middle of the pitch on show (the reference's and the singing's pitch and notes, in the pane's time range and on the range), which comes towards the middle of the pane as it zooms in; with none on show, about the fingers as before. A low voice no longer drops below the pane. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 28 +++- main/Analyser.cpp | 29 ++++ main/Analyser.h | 7 + main/MainWindow.cpp | 13 ++ main/TouchGestures.cpp | 31 +++- main/TouchGestures.h | 17 ++- main/VerticalZoom.cpp | 38 +++++ main/VerticalZoom.h | 27 +++- main/test/TestTouchGestures.h | 276 +++++++++++++++++++++++++++++----- main/test/TestVerticalZoom.h | 108 +++++++++++++ 10 files changed, 527 insertions(+), 47 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 22ffb07a..27f8f520 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -180,7 +180,7 @@ builds happen in the container.) - A9 — Live dots in real time on the phone. Done. - A10 — Plot elements sized for the screen. Done. - A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log. Done. -- A4c — Vertical zoom keeps the pitch in view. +- A4c — Vertical zoom keeps the pitch in view. Done. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -972,3 +972,29 @@ The audio copy-in reads the same way and drops a copy whose close reports an err Tests seen failing: both cursor checks with the ratio left at 1; `savedSize()` taking "" as 0. Left open: nothing run on a phone. forks.md (svgui) and recording.md "Start click" step 2 and 6 name the start frame plus the duration: for A8. + +### Phase A4c — 2026-09-26 +Built: `VerticalZoom` (core): `middleShown(values, range)`, the middle on the range's scale of +the values it shows, half way from lowest to highest less a twentieth at each end (an octave +jump, a breath); `towardsMiddle(y, height, factor)`, zooming in the held value's distance from +the pane's middle divided by the factor, zooming out where it was. `VerticalRange::drawn`; +`TouchGestures::beginPinch()` asks it once a pinch: with values on the start range the zoom is +about their middle, pulled towards the pane's middle; with none, about the fingers as before. +Scroll (travel) and re-anchoring at a limit as A4b. `Analyser::getPitchOnShow()`: pitch track +and notes, each if not dormant (a temporary hide counts), notes spanning the frames. +`paneAdded()` gathers both analysers in their pane over its start to end frame. +Choices / deviations: +- On show = in the pane's time range and on the range: pitch all off the range falls back to + the fingers. Wider than the zoomed range: still the middle of its extent, both ends cut + alike; a gap there (voices an octave apart) shows empty far in, two fingers scroll to + either. The median was not taken: it centres a skewed phrase badly while it still fits. +- The pull: the middle held where it was keeps a low voice's middle in view but loses its + lower half once it fills a third of the pane (B1-G2 from 40-1500: at 2.8x); pulled, all of + it stays until it fills about 90%. Not asked for in words; the user's "centered". +- Once a pinch (every two-finger touch): an event copy per point in view, ~700 per analyser + on a 4 s page. Not counted: the alternate pitch track, other takes' layers, live dots. +Tests: TestVerticalZoom +5; TestTouchGestures +3 (the low voice, checked at each step; the +fingers when nothing is on show; the singing counted); narrows/widens/diagonal now check the +pitch. Pitch checks allow for the whole-Hz range (~3 px at a bottom near 40 Hz). +Tests seen failing: anchor off (5); no pull (3 app, 2 core); singing not gathered; dormancy +ignored (2). Left open: not on a phone; no follow in playback, no "fit the pitch" action. diff --git a/main/Analyser.cpp b/main/Analyser.cpp index 54d8c252..cb71ea5e 100644 --- a/main/Analyser.cpp +++ b/main/Analyser.cpp @@ -333,6 +333,35 @@ Analyser::setDisplayFrequencyExtents(double min, double max) return true; } +void +Analyser::getPitchOnShow(sv_frame_t start, sv_frame_t end, + std::vector &values) const +{ + if (end <= start) return; + + if (isVisible(PitchTrack)) { + auto model = ModelById::getAs + (m_layers[PitchTrack]->getModel()); + if (model) { + for (const Event &e : model->getEventsSpanning(start, end - start)) { + values.push_back(e.getValue()); + } + } + } + + // A note that starts before the pane and ends in it is on show too: + // spanning, not within + if (isVisible(Notes)) { + auto model = ModelById::getAs + (m_layers[Notes]->getModel()); + if (model) { + for (const Event &e : model->getEventsSpanning(start, end - start)) { + values.push_back(e.getValue()); + } + } + } +} + int Analyser::getInitialAnalysisCompletion() { diff --git a/main/Analyser.h b/main/Analyser.h index 2e1fbd28..03e3d364 100644 --- a/main/Analyser.h +++ b/main/Analyser.h @@ -104,6 +104,13 @@ class Analyser : public QObject, bool getDisplayFrequencyExtents(double &min, double &max); bool setDisplayFrequencyExtents(double min, double max); + // Add to values, in Hz, those of the pitch track and the notes that + // are drawn between the frames start and end, of whichever of the + // two are on show: the pitch a zoom of the frequency range keeps in + // view (TouchGestures) + void getPitchOnShow(sv::sv_frame_t start, sv::sv_frame_t end, + std::vector &values) const; + // Return completion %age for initial analysis -- 100 means it's done int getInitialAnalysisCompletion(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index 3223c06b..943d952b 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -7692,6 +7692,19 @@ MainWindow::paneAdded(Pane *pane) }; range.limits = VerticalZoom::pitchLimits(); + // A zoom keeps in view the reference's pitch and notes and the + // singing's, those of them on show, in the time the pane shows + range.drawn = [this, pane]() { + std::vector values; + for (Analyser *a : { m_analyser, m_analyser2 }) { + if (a && a->getPane() == pane) { + a->getPitchOnShow(pane->getStartFrame(), pane->getEndFrame(), + values); + } + } + return values; + }; + TouchGestures *gestures = new TouchGestures(pane); // owned by the pane gestures->setVerticalRange(range); diff --git a/main/TouchGestures.cpp b/main/TouchGestures.cpp index e28cfc80..88ed8fc9 100644 --- a/main/TouchGestures.cpp +++ b/main/TouchGestures.cpp @@ -433,8 +433,23 @@ TouchGestures::beginPinch() m_shownRange = shown; m_startRange = VerticalZoom::limited (shown, m_verticalRange.limits, height, m_startCentre.y()); + m_anchorY = m_startCentre.y(); m_anchorValue = VerticalZoom::valueAtY - (m_startRange, height, m_startCentre.y()); + (m_startRange, height, m_anchorY); + m_anchorToMiddle = false; + + // Or rather the middle of the pitch on show, which a zoom about + // the fingers pushes out of the pane when it is far from them (a + // low voice). Asked for once a pinch: it is the same pitch + // however the pinch goes on + double middle = 0.0; + if (m_verticalRange.drawn && + VerticalZoom::middleShown(m_verticalRange.drawn(), m_startRange, + middle)) { + m_anchorValue = middle; + m_anchorY = VerticalZoom::yForValue(m_startRange, height, middle); + m_anchorToMiddle = true; + } } } @@ -508,16 +523,22 @@ TouchGestures::updateVerticalRange(double factor, double travel) double height = m_pane->height(); if (!(height > 0)) return; - // The value that was between the fingers goes where they are now, - // with the range narrowed by as much as they have spread - double y = m_startCentre.y() + travel; + // The value zoomed about goes as far up or down as the fingers have + // gone, with the range narrowed by as much as they have spread; the + // middle of the pitch towards the middle of the pane as it narrows, + // what was between the fingers with them + double y = m_anchorY; + if (m_anchorToMiddle) { + y = VerticalZoom::towardsMiddle(y, height, factor); + } + y += travel; VerticalZoom::Range wanted = VerticalZoom::zoomedAbout (m_startRange, factor, m_anchorValue, height, y); VerticalZoom::Range range = VerticalZoom::limited (wanted, m_verticalRange.limits, height, y); // Held at a limit, the range stays while the fingers go on: what is - // under them then is what they hold, as at the ends of the audio + // at y then is what is held, as at the ends of the audio if (range != wanted) { m_anchorValue = VerticalZoom::valueAtY(range, height, y); } diff --git a/main/TouchGestures.h b/main/TouchGestures.h index 987e849f..af4e256f 100644 --- a/main/TouchGestures.h +++ b/main/TouchGestures.h @@ -46,8 +46,9 @@ class Pane; * * The time axis zooms by the fingers' spread across the pane and * follows them across it. Given a vertical range (setVerticalRange()), - * the range zooms by their spread up the pane and follows them up and - * down, each axis only once the fingers plainly move along it + * the range zooms by their spread up the pane, about the pitch on show + * if there is any, and follows them up and down, each axis only once + * the fingers plainly move along it * (PinchZoom::AxisMovement): a pinch across the pane leaves the range * alone, and a pinch up the pane the zoom level. * @@ -95,6 +96,12 @@ class TouchGestures : public QObject /// Show another std::function set; VerticalZoom::Limits limits; + /// The values drawn on it in the pane's time range now (its + /// pitch), for a zoom to keep in view: it zooms about their + /// middle (VerticalZoom::middleShown()), which comes towards + /// the middle of the pane as it zooms in. Unset, or none on + /// the range: about what is between the fingers + std::function()> drawn; }; void setVerticalRange(const VerticalRange &range); @@ -176,12 +183,16 @@ class TouchGestures : public QObject PinchZoom::AxisMovement m_travelY; // The vertical range, as it was when they came down and as last - // shown, and the value that was under the point between them + // shown, and the value it zooms about: the middle of what was drawn + // on it, which goes towards the middle of the pane, or what was + // under the point between the fingers; and where that was VerticalRange m_verticalRange; bool m_haveRange = false; VerticalZoom::Range m_startRange; VerticalZoom::Range m_shownRange; double m_anchorValue = 0.0; + double m_anchorY = 0.0; + bool m_anchorToMiddle = false; }; #endif diff --git a/main/VerticalZoom.cpp b/main/VerticalZoom.cpp index 7f1783cb..86abfc21 100644 --- a/main/VerticalZoom.cpp +++ b/main/VerticalZoom.cpp @@ -144,4 +144,42 @@ limited(const Range &range, const Limits &limits, double height, double y) return result; } +bool +middleShown(const std::vector &values, const Range &range, + double &middle) +{ + if (!shows(range)) return false; + + std::vector shown; + shown.reserve(values.size()); + for (double value : values) { + // False for NaN as well + if (value >= range.min && value <= range.max) { + shown.push_back(map(range.log, value)); + } + } + if (shown.empty()) return false; + + // An octave jump at the start of a note, or a breath taken for a + // pitch, is not what is sung: a few points either way are left out + size_t stray = shown.size() / 20; + auto lowest = shown.begin() + stray; + auto highest = shown.end() - 1 - stray; + std::nth_element(shown.begin(), lowest, shown.end()); + double low = *lowest; + std::nth_element(shown.begin(), highest, shown.end()); + double high = *highest; + + middle = unmap(range.log, (low + high) / 2.0); + return true; +} + +double +towardsMiddle(double y, double height, double factor) +{ + if (!(factor > 1.0)) return y; + double middle = height / 2.0; + return middle + (y - middle) / factor; +} + } diff --git a/main/VerticalZoom.h b/main/VerticalZoom.h index e6556441..9b3ac32e 100644 --- a/main/VerticalZoom.h +++ b/main/VerticalZoom.h @@ -15,11 +15,14 @@ #ifndef TONY_VERTICAL_ZOOM_H #define TONY_VERTICAL_ZOOM_H +#include + /** * The arithmetic of two fingers on a pane's vertical axis: the value * at a height in the pane, the range that zooms about a value held at - * a height, and the limits a range is kept within. TouchGestures does - * as the answers say. + * a height, the limits a range is kept within, and the value a zoom is + * about when the pane has pitch to keep in view. TouchGestures does as + * the answers say. * * The mapping is svgui's CoordinateScale's for a vertical scale: y * from the top of the pane, the range's minimum at the bottom edge (y @@ -83,6 +86,26 @@ namespace VerticalZoom */ Range limited(const Range &range, const Limits &limits, double height, double y); + + /** + * What a zoom of range is anchored at to keep values in view (the + * pitch a pane draws, say): the middle, on range's scale, of those + * of them that range shows, half way between the lowest and the + * highest, less a twentieth of them at each end, so that a few + * that stray from the rest do not move it. False if range shows + * none of them, or shows nothing. + */ + bool middleShown(const std::vector &values, const Range &range, + double &middle); + + /** + * Where a value that was at y is held as a range zooms by factor + * about it: nearer the middle of the pane as it zooms in, its + * distance from there divided by the factor, so that what is about + * it comes into the middle as it grows; where it was as it zooms + * out. + */ + double towardsMiddle(double y, double height, double factor); } #endif diff --git a/main/test/TestTouchGestures.h b/main/test/TestTouchGestures.h index b7e55d94..7811b4b1 100644 --- a/main/test/TestTouchGestures.h +++ b/main/test/TestTouchGestures.h @@ -80,6 +80,7 @@ class TouchTestWindow : public MainWindow sv::PaneStack *paneStack() { return m_paneStack; } sv::ViewManager *viewManager() { return m_viewManager; } Analyser *analyser() { return m_analyser; } + Analyser *analyser2() { return m_analyser2; } AlternatePitchTrack *alternatePitch() { return m_alternatePitch; } void toggleAlternatePitch() { alternatePitchToggled(); } @@ -107,6 +108,10 @@ class TestTouchGestures : public QObject static constexpr double rate = 44100.0; + // The reference's pitch, and a low voice's (D2) + static constexpr double referenceHz = 220.5; + static constexpr double lowHz = 73.5; + // Where each test starts: frames per pixel, and the centre frame static constexpr int startLevel = 64; static constexpr sv::sv_frame_t startCentre = 3 * 44100; @@ -178,8 +183,7 @@ class TestTouchGestures : public QObject t.release(0, a1, p).release(1, b1, p).commit(); } - bool analysed() { - Analyser *a = m_window->analyser(); + bool analysed(Analyser *a) { return a && a->getLayer(Analyser::PitchTrack) && a->getLayer(Analyser::Notes) && a->getInitialAnalysisCompletion() >= 100 && @@ -188,25 +192,47 @@ class TestTouchGestures : public QObject ->haveRunningTransformers(); } - // The window on show with six seconds of reference, analysed, and - // the view at startLevel about startCentre - void openWindow() { + bool analysed() { return analysed(m_window->analyser()); } + + // A wav file in the test's directory, written the first time it is + // asked for; "" if it could not be + QString wavFile(QString name, const std::vector &data) { + QString path = m_dir.filePath(name); + if (QFile::exists(path)) return path; + sv::WavFileWriter writer(path, rate, 1, + sv::WavFileWriter::WriteToTarget); + const float *ptr = data.data(); + if (!writer.isOK() || + !writer.writeSamples(&ptr, sv::sv_frame_t(data.size())) || + !writer.close()) { + return ""; + } + return path; + } + + // Six seconds of a sine + QString sineFile(QString name, double hz) { + return wavFile(name, TestSignals::sine(hz, rate, int(6 * rate), 0.5)); + } + + // The view at startLevel about startCentre + void resetView() { + pane()->setZoomLevel + (sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, startLevel)); + pane()->setCentreFrame(startCentre); + } + + // The window on show with six seconds of reference, a sine at + // referenceHz unless another file is given, analysed, and the view at + // startLevel about startCentre + void openWindow(QString path = "") { m_window = new TouchTestWindow; m_window->resize(1000, 700); m_window->show(); QVERIFY(QTest::qWaitForWindowExposed(m_window)); - QString path = m_dir.filePath("reference.wav"); - if (!QFile::exists(path)) { - std::vector data = TestSignals::sine - (220.5, rate, int(6 * rate), 0.5); - sv::WavFileWriter writer(path, rate, 1, - sv::WavFileWriter::WriteToTarget); - const float *ptr = data.data(); - QVERIFY(writer.isOK()); - QVERIFY(writer.writeSamples(&ptr, sv::sv_frame_t(data.size()))); - QVERIFY(writer.close()); - } + if (path == "") path = sineFile("reference.wav", referenceHz); + QVERIFY(path != ""); m_window->discardModifications(); QCOMPARE(m_window->openPath(path, MainWindow::ReplaceSession), @@ -215,12 +241,34 @@ class TestTouchGestures : public QObject QVERIFY(pane()); QVERIFY(strip()); - pane()->setZoomLevel - (sv::ZoomLevel(sv::ZoomLevel::FramesPerPixel, startLevel)); - pane()->setCentreFrame(startCentre); + resetView(); QCOMPARE(framesPerPixel(), double(startLevel)); } + // An analyser's pitch and notes hidden or shown again, straight to + // the layers as the app does for a while: Analyser::setVisible() + // writes a setting that both analysers share + void showPitch(Analyser *a, bool shown) { + a->getLayer(Analyser::PitchTrack)->showLayer(pane(), shown); + a->getLayer(Analyser::Notes)->showLayer(pane(), shown); + } + + // Where a zoom by factor about a value at y puts it: towards the + // middle of the pane, its distance from there divided by the factor + double pulledTowardsMiddle(double y, double factor) { + double middle = pane()->height() / 2.0; + return middle + (y - middle) / factor; + } + + // How far, in pixels, a frequency on r may be from where the fingers + // put it: the range is whole Hz (the spectrogram's), so each end may + // be half a hertz from theirs, which low down is a few pixels + double slack(const VerticalZoom::Range &r) { + return 1.0 + pane()->height() * + (std::log2((r.min + 0.5) / r.min) + + std::log2((r.max + 0.5) / r.max)) / octaves(r); + } + void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); if (!modal) return; @@ -622,9 +670,11 @@ private slots: // Fingers spread up the pane: the frequency range narrows by as // much as they spread, less what the dead zone took, about the - // frequency between them, which stays there. The time axis stays as - // it was, the pitch layers in the pane are drawn on the new range, - // and nothing goes into the undo history + // pitch on show, the reference's sine, low in the pane and well + // below the fingers, which comes towards the middle of the pane by + // as much. The time axis stays as it was, the pitch layers in the + // pane are drawn on the new range, and nothing goes into the undo + // history void vertical_pinch_narrows_the_frequency_range() { openWindow(); if (QTest::currentTestFailed()) return; @@ -641,7 +691,11 @@ private slots: VerticalZoom::Range before = frequencyRange(); QCOMPARE(before.min, 40.0); QCOMPARE(before.max, 1500.0); - double held = frequencyAtY(y); + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(150, 1200)); + before = frequencyRange(); + double pitchY = yForFrequency(referenceHz); + QVERIFY2(pitchY > 0.75 * p->height(), + qPrintable(QString("the pitch is at %1").arg(pitchY))); int commands = 0; auto counted = connect(sv::CommandHistory::getInstance(), @@ -659,9 +713,12 @@ private slots: qPrintable(QString("%1, then %2: %3 times narrower, not %4") .arg(text(before)).arg(text(after)) .arg(narrowed).arg(factor))); - QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, - qPrintable(QString("%1 Hz was at %2, then at %3") - .arg(held).arg(y).arg(yForFrequency(held)))); + double expected = pulledTowardsMiddle(pitchY, factor); + QVERIFY2(std::fabs(yForFrequency(referenceHz) - expected) <= + slack(after), + qPrintable(QString("the pitch was at %1, then at %2, not %3") + .arg(pitchY).arg(yForFrequency(referenceHz)) + .arg(expected))); // As far as the centre goes, to the whole pixel the fingers hold QCOMPARE(framesPerPixel(), double(startLevel)); @@ -693,7 +750,7 @@ private slots: } // Fingers brought together up the pane: the range widens, about - // the frequency between them + // the pitch on show, which stays where it was void vertical_pinch_widens_the_frequency_range() { openWindow(); if (QTest::currentTestFailed()) return; @@ -705,7 +762,8 @@ private slots: int y = p->height() / 2 - 40; VerticalZoom::Range before = frequencyRange(); QCOMPARE(before.min, 100.0); - double held = frequencyAtY(y); + double pitchY = yForFrequency(referenceHz); + QVERIFY(pitchY > y + 20); twoFingers(p, QPoint(x, y - 100), QPoint(x, y + 100), QPoint(x, y - 50), QPoint(x, y + 50)); @@ -717,9 +775,9 @@ private slots: qPrintable(QString("%1, then %2: %3 times narrower, not %4") .arg(text(before)).arg(text(after)) .arg(narrowed).arg(factor))); - QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, - qPrintable(QString("%1 Hz was at %2, then at %3") - .arg(held).arg(y).arg(yForFrequency(held)))); + QVERIFY2(std::fabs(yForFrequency(referenceHz) - pitchY) <= 1.0, + qPrintable(QString("the pitch was at %1, then at %2") + .arg(pitchY).arg(yForFrequency(referenceHz)))); QCOMPARE(framesPerPixel(), double(startLevel)); } @@ -747,17 +805,20 @@ private slots: QCOMPARE(after.max, before.max); } - // Spread along both: both zoom, each about the fingers + // Spread along both: both zoom, time about the fingers and the + // frequency range about the pitch void diagonal_pinch_zooms_time_and_frequency() { openWindow(); if (QTest::currentTestFailed()) return; + QVERIFY(m_window->analyser()->setDisplayFrequencyExtents(100, 800)); + sv::Pane *p = pane(); int x = p->width() / 2 - 100; int y = p->height() / 2 - 30; sv::sv_frame_t frame = p->getFrameForX(x); VerticalZoom::Range before = frequencyRange(); - double held = frequencyAtY(y); + double pitchY = yForFrequency(referenceHz); twoFingers(p, QPoint(x - 50, y - 50), QPoint(x + 50, y + 50), QPoint(x - 100, y - 100), QPoint(x + 100, y + 100)); @@ -774,9 +835,152 @@ private slots: qPrintable(QString("%1, then %2: %3 times narrower, not %4") .arg(text(before)).arg(text(after)) .arg(narrowed).arg(factor))); - QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, - qPrintable(QString("%1 Hz was at %2, then at %3") - .arg(held).arg(y).arg(yForFrequency(held)))); + double expected = pulledTowardsMiddle(pitchY, factor); + QVERIFY2(std::fabs(yForFrequency(referenceHz) - expected) <= + slack(after), + qPrintable(QString("the pitch was at %1, then at %2, not %3") + .arg(pitchY).arg(yForFrequency(referenceHz)) + .arg(expected))); + } + + // With no pitch on show, the range zooms about the frequency between + // the fingers, which stays there: the reference's pitch and notes + // hidden, and then shown but below the range + void vertical_pinch_with_no_pitch_on_show_zooms_about_the_fingers() { + openWindow(); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int x = p->width() / 2 + 50; + int y = p->height() / 2 - 60; + Analyser *a = m_window->analyser(); + + for (bool hidden : { true, false }) { + showPitch(a, !hidden); + QVERIFY(a->setDisplayFrequencyExtents(hidden ? 100 : 300, 1200)); + VerticalZoom::Range before = frequencyRange(); + double held = frequencyAtY(y); + + twoFingers(p, QPoint(x, y - 50), QPoint(x, y + 50), + QPoint(x, y - 100), QPoint(x, y + 100)); + + QString what = QString(hidden ? "hidden" : "below the range"); + VerticalZoom::Range after = frequencyRange(); + double factor = (200.0 - deadZone()) / 100.0; + double narrowed = octaves(before) / octaves(after); + QVERIFY2(std::fabs(narrowed - factor) < 0.01 * factor, + qPrintable(what + QString(": %1 times narrower, not %2") + .arg(narrowed).arg(factor))); + QVERIFY2(std::fabs(yForFrequency(held) - y) <= 1.0, + qPrintable(what + QString(": %1 Hz was at %2, then at %3") + .arg(held).arg(y).arg(yForFrequency(held)))); + } + } + + // The fifth phone test: a low voice, C2 and G2 by turns, near the + // bottom of the range as it opens, and the fingers spread up the + // middle of the pane, far above it. At every step, as the range + // narrows, both notes stay in view, and they end nearer the middle. + // Zoomed about the fingers, G2 went below the pane half way through + void low_pitch_stays_in_view_as_the_range_narrows() { + std::vector voice; + for (int i = 0; i < 12; ++i) { + std::vector note = TestSignals::sine + (i % 2 ? 98.0 : 65.4, rate, int(rate / 2), 0.5); + voice.insert(voice.end(), note.begin(), note.end()); + } + QString path = wavFile("low-voice.wav", voice); + QVERIFY(path != ""); + openWindow(path); + if (QTest::currentTestFailed()) return; + + sv::Pane *p = pane(); + int height = p->height(); + int x = p->width() / 2; + int y = height / 2; + VerticalZoom::Range before = frequencyRange(); + QCOMPARE(before.min, 40.0); + QCOMPARE(before.max, 1500.0); + double lowY = yForFrequency(65.4); + double highY = yForFrequency(98.0); + QVERIFY2(highY > 0.7 * height, + qPrintable(QString("G2 at %1 of %2").arg(highY).arg(height))); + + int from = 30; + int to = height / 2 - 20; + Touch t = touch(); + t.press(0, QPoint(x, y - from), p).press(1, QPoint(x, y + from), p) + .commit(); + for (int i = 1; i <= 10; ++i) { + int d = from + (to - from) * i / 10; + t.move(0, QPoint(x, y - d), p).move(1, QPoint(x, y + d), p) + .commit(); + double low = yForFrequency(65.4); + double high = yForFrequency(98.0); + QVERIFY2(low <= height && high >= 0, + qPrintable(QString("step %1, %2: C2 at %3, G2 at %4 " + "of %5") + .arg(i).arg(text(frequencyRange())) + .arg(low).arg(high).arg(height))); + } + t.release(0, QPoint(x, y - to), p).release(1, QPoint(x, y + to), p) + .commit(); + + VerticalZoom::Range after = frequencyRange(); + QVERIFY2(octaves(after) < octaves(before) / 4, + qPrintable(text(before) + ", then " + text(after))); + double middle = height / 2.0; + double lowAfter = yForFrequency(65.4); + double highAfter = yForFrequency(98.0); + QVERIFY2(std::fabs((lowAfter + highAfter) / 2 - middle) < + std::fabs((lowY + highY) / 2 - middle) / 4, + qPrintable(QString("C2 and G2 at %1 and %2, then at %3 and " + "%4, of %5") + .arg(lowY).arg(highY).arg(lowAfter) + .arg(highAfter).arg(height))); + QCOMPARE(QGuiApplication::mouseButtons(), Qt::NoButton); + } + + // The singing's pitch counts as the reference's does. A low voice + // sung to a high reference: with the reference's pitch and notes + // hidden the zoom keeps the singing's in view, and with both on + // show it is about the middle between the two + void singing_pitch_on_show_is_kept_in_view() { + openWindow(); + if (QTest::currentTestFailed()) return; + + QString singing = sineFile("singing.wav", lowHz); + QVERIFY(singing != ""); + m_window->loadSingingTrack(singing); + QTRY_VERIFY_WITH_TIMEOUT(analysed(m_window->analyser2()), 30000); + QVERIFY(m_window->analyser2()->getPane() == pane()); + resetView(); + + sv::Pane *p = pane(); + int x = p->width() / 2; + int y = p->height() / 4; + Analyser *a = m_window->analyser(); + + for (bool reference : { false, true }) { + showPitch(a, reference); + QVERIFY(a->setDisplayFrequencyExtents(40, 1500)); + double kept = reference ? std::sqrt(lowHz * referenceHz) : lowHz; + double keptY = yForFrequency(kept); + + twoFingers(p, QPoint(x, y - 40), QPoint(x, y + 40), + QPoint(x, y - 80), QPoint(x, y + 80)); + + double factor = (160.0 - deadZone()) / 80.0; + double expected = pulledTowardsMiddle(keptY, factor); + VerticalZoom::Range after = frequencyRange(); + QVERIFY2(std::fabs(yForFrequency(kept) - expected) <= + slack(after), + qPrintable(QString("%1 Hz was at %2, then at %3, not " + "%4, on %5") + .arg(kept).arg(keptY) + .arg(yForFrequency(kept)).arg(expected) + .arg(text(after)))); + } } // Two fingers side by side dragged down the pane: what was between diff --git a/main/test/TestVerticalZoom.h b/main/test/TestVerticalZoom.h index f7bc8985..e2a35315 100644 --- a/main/test/TestVerticalZoom.h +++ b/main/test/TestVerticalZoom.h @@ -196,6 +196,114 @@ private slots: } } + // Half way between the lowest and the highest on the range's scale: + // the geometric mean on a log one, however many are in between + void middle_is_half_way_between_lowest_and_highest() { + std::vector values { 110.0, 400.0, 100.0, 105.0, 120.0 }; + double middle = 0.0; + QVERIFY(VerticalZoom::middleShown(values, range(40.0, 1500.0, true), + middle)); + QVERIFY(near(middle, 200.0, 1.0e-9)); + QVERIFY(VerticalZoom::middleShown(values, range(40.0, 1500.0, false), + middle)); + QVERIFY(near(middle, 250.0, 1.0e-9)); + + QVERIFY(VerticalZoom::middleShown({ 73.5 }, range(40.0, 1500.0, true), + middle)); + QVERIFY(near(middle, 73.5, 1.0e-9)); + } + + // Only what the range shows: values above or below it, and not + // values at all, are not on show. None on show: no middle + void middle_is_of_what_is_on_show() { + double nan = std::numeric_limits::quiet_NaN(); + std::vector values { 20.0, 100.0, nan, 400.0, 3000.0, -1.0 }; + double middle = 0.0; + QVERIFY(VerticalZoom::middleShown(values, range(50.0, 1000.0, true), + middle)); + QVERIFY(near(middle, 200.0, 1.0e-9)); + QVERIFY(VerticalZoom::middleShown(values, range(100.0, 400.0, true), + middle)); + QVERIFY(near(middle, 200.0, 1.0e-9)); + + middle = -5.0; + QVERIFY(!VerticalZoom::middleShown(values, range(500.0, 1000.0, true), + middle)); + QVERIFY(!VerticalZoom::middleShown({}, range(40.0, 1500.0, true), + middle)); + QVERIFY(!VerticalZoom::middleShown(values, range(0.0, 1500.0, true), + middle)); + QVERIFY(!VerticalZoom::middleShown(values, range(500.0, 50.0, false), + middle)); + QCOMPARE(middle, -5.0); + } + + // Two strays in forty, an octave jump or a breath, leave the middle + // where the rest put it; five are more than strays + void a_few_strays_do_not_move_the_middle() { + std::vector values; + for (int i = 0; i < 19; ++i) values.push_back(70.0); + for (int i = 0; i < 19; ++i) values.push_back(90.0); + values.push_back(700.0); + values.push_back(35.0); + double middle = 0.0; + QVERIFY(VerticalZoom::middleShown(values, range(30.0, 1500.0, true), + middle)); + QVERIFY2(near(middle, std::sqrt(70.0 * 90.0), 1.0e-9), + qPrintable(QString("%1").arg(middle))); + + values.clear(); + for (int i = 0; i < 17; ++i) values.push_back(70.0); + for (int i = 0; i < 18; ++i) values.push_back(90.0); + for (int i = 0; i < 5; ++i) values.push_back(700.0); + QVERIFY(VerticalZoom::middleShown(values, range(30.0, 1500.0, true), + middle)); + QVERIFY2(near(middle, std::sqrt(70.0 * 700.0), 1.0e-9), + qPrintable(QString("%1").arg(middle))); + } + + // Zoomed in, a value comes towards the middle of the pane, its + // distance from there divided by the factor; zoomed out it stays + void towards_the_middle_by_the_factor() { + QCOMPARE(VerticalZoom::towardsMiddle(380.0, 400.0, 2.0), 290.0); + QCOMPARE(VerticalZoom::towardsMiddle(380.0, 400.0, 4.0), 245.0); + QCOMPARE(VerticalZoom::towardsMiddle(0.0, 400.0, 2.0), 100.0); + QCOMPARE(VerticalZoom::towardsMiddle(200.0, 400.0, 8.0), 200.0); + QCOMPARE(VerticalZoom::towardsMiddle(380.0, 400.0, 1.0), 380.0); + QCOMPARE(VerticalZoom::towardsMiddle(380.0, 400.0, 0.5), 380.0); + } + + // A low voice, B1 to G2, near the bottom of the default range: zoomed + // about its middle, which comes towards the middle of the pane, all + // of it stays in view until it is nearly as tall as the pane. Zoomed + // about the middle of the pane, it is gone below by three times + void a_low_pitch_stays_in_view_as_the_range_narrows() { + const double height = 400.0; + Range start = range(40.0, 1500.0, true); + std::vector pitch { 61.7, 73.4, 82.4, 98.0, 65.4, 87.3 }; + double middle = 0.0; + QVERIFY(VerticalZoom::middleShown(pitch, start, middle)); + double y = VerticalZoom::yForValue(start, height, middle); + QVERIFY(y > 0.8 * height); + + for (double factor : { 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0 }) { + Range r = VerticalZoom::zoomedAbout + (start, factor, middle, height, + VerticalZoom::towardsMiddle(y, height, factor)); + double bottom = VerticalZoom::yForValue(r, height, 61.7); + double top = VerticalZoom::yForValue(r, height, 98.0); + QVERIFY2(bottom <= height && top >= 0.0, + qPrintable(QString("by %1: %2, the pitch from %3 to %4") + .arg(factor).arg(text(r)).arg(bottom) + .arg(top))); + } + + double centre = VerticalZoom::valueAtY(start, height, height / 2); + Range r = VerticalZoom::zoomedAbout(start, 3.0, centre, height, + height / 2); + QVERIFY(VerticalZoom::yForValue(r, height, 98.0) > height); + } + // Nothing within the dead zone; beyond it, all but the dead zone, // and then all the way back through it void movement_counts_beyond_the_dead_zone() { From 59f9e7653f755d4f2b959bf4ac74f9d89d0b3e52 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 15:40:36 +0000 Subject: [PATCH 215/275] docs: the driver project, WASAPI next to MME: plan and work orders What is built where: the fork chooses the host API, WASAPI's rate conversion and the latency; Tony gets a Driver menu; Calibrate Audio stops calling 48 kHz a mismatch. MME stays the default until measured. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/audio-drivers-work-orders.md | 118 +++++++++++++++++++++++++++ docs/audio-drivers.md | 129 ++++++++++++++++++++++++++++++ 2 files changed, 247 insertions(+) create mode 100644 docs/audio-drivers-work-orders.md create mode 100644 docs/audio-drivers.md diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md new file mode 100644 index 00000000..65bec243 --- /dev/null +++ b/docs/audio-drivers-work-orders.md @@ -0,0 +1,118 @@ +# Audio drivers: work orders for phase agents + +You are one of a line of agents, each building **one small phase** of the driver project +([audio-drivers.md](audio-drivers.md)). A lead reviews your work when you report back, +and commits it. You have no memory of earlier phases; what you need is here. This file is +working memory for the branch `feat/wasapi`, and the documentation phase removes it. + +**Your context is the budget.** Aim to finish well under 200k tokens. The rules below +say how. They are about not reading huge files whole and not maintaining big documents; +they are **not** a licence to skip what you need to understand. Careful, correct work +comes first. + +## 1. What to read, and what not to + +1. This file, all of it. +2. `AGENTS.md` at the repository root. Its rules apply, except that the **build and test + commands in section 2 below replace its Windows ones**. +3. `docs/audio-drivers.md` (the spec, short): all of it. +4. The docs your work order names, by section: search for the heading and read that + range. `docs/calibrate-audio.md` is about 600 lines; `docs/testing.md` about 350. +5. Code: + - `main/MainWindow.cpp` is over 6000 lines and `main/test/TestRecordWorkflow.h` over + 8000. **Never read them whole**: search for the function, then read that range. + - Before writing a test, read an existing test next to where yours will go and copy + its shape. + +## 2. Rules + +**Scope** + +- Build your phase only. Where the spec is silent, choose the simpler option and say so. + Where it is **wrong or impossible**, do not improvise another design: finish what can + be finished, leave the tree building and green, and report. +- **Do not edit** any top-level library directory (`svcore/`, `svgui/`, `svapp/`, + `bqaudioio/`, `bqaudiostream/`, `pyin/`, …). They are separate repositories, gitignored + here. If one needs a change, report exactly which; the lead makes it. +- Match the surrounding code: naming, comment density, idiom. Comments say why, in plain + words. Every new source file starts with the project's GPL header. +- **The user builds on Windows** with MinGW and Qt 6.11. Nothing specific to Linux; and + no identifiers named `near`, `far`, `min`, `max`, `ERROR`, `IN` or `OUT` (Windows + headers define them as macros). + +**Build and test.** This container builds in `build/` with Qt 6.11 (conda-forge). + +- Build, from the repository root: + + ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app test-tony-dev > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log + + Search the log for `error:`; never read it whole. No `.exe`. `pyin.so` must be built: + without it every app test that waits for an analysis hangs. +- Named tests while working, from `build/`: + + mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt + TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app some_test other_test > ../tmp/test.log 2>&1 + grep -a "^FAIL\|^XFAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt + + A name goes to every suite of the executable; the others report it unknown, so the + exit status of a run with names means nothing. Read results from the per-suite files. +- Whole suites, **once at the end**, from the repository root: + + deploy/linux/run-tests.sh test-tony-core + deploy/linux/run-tests.sh test-tony-app # about a minute and a half + deploy/linux/run-tests.sh test-tony-dev # about a minute + + Each runs eight processes and prints a summary with every failure. Never run two at + once, nor one while building: the app tests record in real time. +- Every behaviour gets a test that can fail. Show it for the two or three that matter by + breaking the code for a moment; undo the break **by hand**, never with `git checkout` + or `git restore`. +- Do not weaken or delete a test to get green. If one is wrong because the behaviour was + meant to change, change it and say so. +- App tests run in real time against `FakeAudioIO`: keep them to seconds. + +**Git.** Do not commit, push, or stage. The lead does, after review. Never `git add -A`. +Edit documents with the Edit and Write tools, not shell one-liners. + +**Docs.** Fix any statement in `docs/` your change makes false, in the page that makes +it. The big documentation pass is W4's; before that, only such fixes. + +**Report** (under 60 lines): what you built, by file; the three `run-tests.sh` summaries +verbatim; which tests you saw fail when you broke the code; decisions you took where the +spec was silent; anything fragile, unfinished, or needed from a library. + +## 3. State of the code + +- `feat/wasapi` starts from `feat/tonyandroid` with `default` merged in: Calibrate Audio + and the dev checks, the Android port, and a device at another rate than the + reference's handled (A1: the recording resampled before the splice, `TakeTiming` + converting; A11: the cursor keeps the reference's pace). +- All three suites green, sharded and in one process each. No expected failures left. + +## 4. Work orders + +### W1 — Calibrate Audio at 48 kHz + +Read: calibrate-audio.md §1, §3 (the verdicts), §8 (what the report says) and the part of +§10 or later that lists the device facts ("Device rate"); `main/AudioCheckRunner.h` and +`.cpp` around `rateMismatch`; `main/CalibrateAudioDialog.cpp` around `rateMismatch`; +`main/test/TestAudioCheck.h`, `check_flags_a_rate_mismatch` and the test that stores a +figure and checks again with it. + +- `AudioCheckResult::rateMismatch` and everything it decides go: a check on a device at + another rate than the reference's is judged, and its figure kept, like any other. +- The report still names the device's rate where it differs from the reference's (the + dialog's details, and `DevChecks.txt` if it prints the result), as a fact, not a fault. +- Tests, with `FakeAudioIO` at 48000 Hz as a loopback with a known delay: + - the check finds the round trip, its verdict is Ok, and the figure can be stored with + the key's rate 48000; + - a second check with the stored figure places the takes (the offsets near 0); + - `check_flags_a_rate_mismatch` becomes one of those; the details test that builds a + result at 48000 Hz changes to the new wording. +- Docs: calibrate-audio.md's statements about the device rate that A1 and this phase + made false (§1 "The device's sample rate is not checked", §10's driver project step 1, + the device facts on `TakeAudio::splice()` and on the check naming the mismatch). + +## 5. Log + +Capped at 25 lines per entry. Newest last. diff --git a/docs/audio-drivers.md b/docs/audio-drivers.md new file mode 100644 index 00000000..ac062193 --- /dev/null +++ b/docs/audio-drivers.md @@ -0,0 +1,129 @@ +# Audio drivers: WASAPI next to MME + +Plan and state of the driver project on the branch `feat/wasapi` (from `feat/tonyandroid`, +to be merged back into it). What is built is marked **Done** in §6; the reasons stay here. + +## 1. Why + +- **MME is slow and unsteady.** On the user's PC Calibrate Audio measured a round trip of + about 300 ms (bqaudioio opens every stream with `suggestedLatency = 0.2` on both sides), + and the offset between input and output moved by about 13 ms from one take to the next + (every take restarts the stream), 5 to 20 ms over three calibrations, while the sweeps + within one take agreed to 0.3 ms ([calibrate-audio.md](calibrate-audio.md), §10). No + one stored figure then places every take; the dev checks' items 1 and 2 fail on it. +- **WASAPI** is the Windows audio engine itself; MME and DirectSound are layers over it. + Its shared mode mixes with other programs as MME does, with buffers as small as the + engine's period (typically 10 ms). Whether it is steadier across stream restarts is what + the measurements in §6, W5 are for. +- **Today the device menus mix all host APIs.** PortAudio lists every device once per host + API (MME, DirectSound, WASAPI, WDM-KS); bqaudioio's `getDeviceIndex()` takes the first + name that matches, which is usually MME's, but MME cuts names to 31 characters, so a + long name picked from the menu can only match WASAPI's or WDM-KS's entry, which then + opens at the Windows mixer's rate (usually 48 kHz). That is where the "48 kHz device" + of the calibrate-audio work came from. + +## 2. Decisions + +| Decision | By, when | Why | +| --- | --- | --- | +| Work on `feat/wasapi`, branched from `feat/tonyandroid` after the calibrate-audio merge, merged back when done | user, 2026-09-26 | Another session works on `feat/tonyandroid` | +| **MME stays the default** until Calibrate Audio and a dev run on the user's PC show WASAPI better | user, 2026-09-26 | Not to change what works before it is measured | +| The driver changes are in the fork `jhhr/bqaudioio`, made by the lead; Tony pins it | user, 2026-09-26 | bqaudioio chooses the host API, the stream's rate and its latency; upstream is on sourcehut | +| **A driver is a bqaudioio implementation**: `mme`, `directsound`, `wasapi`, each PortAudio restricted to that host API; `port` stays as it is (all host APIs) | lead | Tony's device menus, the saved devices (`audio-*-device-`), svapp's `createAudioIO()` and the stored round trip (`LatencyCalibration::Key::implementation`) are already per implementation: no svapp change | +| WASAPI in **shared** mode, with `paWinWasapiAutoConvert` on both sides | lead | The input's and the output's mixers can run at different rates, and the stream opens at the output's; shared mode leaves other programs' sound alone. Exclusive mode is not in this project | +| A device at 48 kHz needs nothing more for placement | `feat/tonyandroid` A1 | The recording is resampled to the reference's rate before the splice, and `TakeTiming` converts device frames (§3). Calibrate Audio's "rate mismatch" verdict is out of date and goes (W1) | +| `suggestedLatency` settable per driver | lead; the Tony-side choice is open (§7) | At 0.2 s WASAPI would keep MME's buffers and gain nothing | + +## 3. Facts checked + +- **PortAudio on the Windows build** is MSYS2's `mingw-w64-x86_64-portaudio` 19.7.0, built + with CMake's defaults: `PA_USE_WMME`, `PA_USE_DS`, `PA_USE_WASAPI` and `PA_USE_WDMKS` are + all ON, and `pa_win_wasapi.h` is installed. Its `PaWasapiStreamInfo` has + `paWinWasapiAutoConvert` (`1 << 6`). (MSYS2's PKGBUILD and PortAudio's `v19.7.0` + `CMakeLists.txt`, 2026-09-26.) An ASIO build is a separate package, not used. +- **bqaudioio** (`src/PortAudioIO.cpp`, at the pin `017ab3ed3a33`, git `7ab6de9`, which is + also `jhhr/bqaudioio`'s `master`): + - `getDeviceNames()` lists every device of every host API with input (or output) + channels; `getDeviceIndex()` returns the first whose name matches, else PortAudio's + global default device (the default host API's, MME's on Windows). + - The stream opens at the output device's `defaultSampleRate`, as neither side of + svapp asks for a rate; `suggestedLatency = 0.2` and no host-API stream info on both + sides; `paFramesPerBufferUnspecified`, then 1024 if that fails, then 2×2 channels. + - `AudioFactory::getImplementationNames()` gives `pulse`, `port`, `jack` as built; + `createIO()` with no implementation named tries each in turn. +- **svapp** `MainWindowBase::createAudioIO()` reads `Preferences/audio-target` and the + devices `audio-record-device` / `audio-playback-device`, suffixed `-` + when one is named; `auto` means none. +- **Tony**: the Playback menu's "Audio Output Device" and "Audio Input Device" submenus + are rebuilt on `aboutToShow` (`rescanAudioDevices()`), from the implementation + `audioImplementationName()` gives: `audio-target` if set, else **the only + implementation built in**. With several PortAudio implementations that is none, and + the menus would be empty: the driver has to be named. +- **`LatencyCalibration::Key`** holds `implementation` (from `audio-target`), both device + names and the recording rate: a figure is kept per driver as it is. +- **Every take restarts the stream** (`MainWindowBase::stop()` suspends, `record()` + resumes: `Pa_StopStream` / `Pa_StartStream`). +- **At 48 kHz** (`feat/tonyandroid`, A1): `SingingTakes::spliceRecording()` resamples the + recording to the reference's rate, and `TakeTiming` keeps the latency and the frames + received in device frames (`recordRate`). Left: the play cursor runs fast during a take + (svgui's `ViewManager`), marked by a `QEXPECT_FAIL`; `feat/tonyandroid`'s A11 is about + it. + +## 4. Design + +### In the fork + +- `PortAudioIO` takes the host API it is restricted to (none: all, as today). Device + lists and `getDeviceIndex()` then look at that host API's devices only, and "no device + named" means **that host API's** default input and output device. +- `AudioFactory` reports `mme`, `directsound` and `wasapi` on Windows when PortAudio is + built in, and each opens a `PortAudioIO` restricted to its host API. `createIO()` with + no implementation named still tries only `port` (and the others built in), as today. +- WASAPI: a `PaWasapiStreamInfo` with `paWinWasapiAutoConvert` on both sides. +- `suggestedLatency` settable, per implementation, with 0.2 s as the default. +- Checked here by building Tony on Linux (the host API restriction with ALSA's devices is + the same code) and by cross-compiling `PortAudioIO.cpp` for Windows with MinGW-w64 + against PortAudio 19.7.0's headers; the rest only on the user's PC. + +### In Tony + +- A **Driver** submenu in the Playback menu, before the two device submenus, listing the + implementations the fork reports that are drivers (MME, DirectSound, WASAPI), when + there is more than one. Choosing one sets `audio-target`, recreates the audio IO + (not during a take or a check) and the device menus follow. +- When `audio-target` is empty and `mme` is built in, Tony names `mme` and carries the + devices saved without a suffix over to the `-mme` keys, once. +- Calibrate Audio: nothing to change for the key; its report names the driver. + +## 5. Phases + +Each leaves the tree building and all three suites green, committed and pushed to +`feat/wasapi`. + +- **W1 — Calibrate Audio at 48 kHz** (agent). The rate-mismatch verdict goes: a device at + another rate than the reference's is measured, its figure can be kept, and a second + check with it places the takes. Tests with `FakeAudioIO` at 48 kHz (a loopback with a + known delay); the old `check_flags_a_rate_mismatch` changes, as the behaviour does. + Docs: calibrate-audio.md §1, §10 and the test list. +- **W2 — The fork** (lead). As §4; then `repoint-project.json` takes bqaudioio from + `jhhr/bqaudioio` (git), `repoint-lock.json` pins it, `deploy/linux/container-setup.sh` + checks it out from there, and forks.md gets its row. +- **W3 — Driver menu** (agent). As §4, "In Tony", with app tests that name the driver + through the Preferences and read back what `createAudioIO()` was asked for. +- **W4 — Docs** (agent). recording.md, calibrate-audio.md, manual-checklist.md, + open-points.md, building.md from the code and the log. +- **W5 — Measure** (user). Calibrate Audio and a dev run on MME and on WASAPI, each at the + latencies offered; the report files back. Then the default is decided. + +## 6. State + +- Nothing built yet. + +## 7. Open + +- **How the latency is chosen in Tony**: a fixed smaller figure for WASAPI, or a submenu + of a few (10, 20, 50, 100, 200 ms). To settle before W3. +- WDM-KS (in PortAudio's build too) and WASAPI's exclusive mode would be lower still, but + take the device from every other program; not in this project. +- Keeping the stream running between takes would remove the restart from the take path + altogether (an svapp change); only if WASAPI's restarts are as unsteady as MME's. From 89b90186f7727e530994ac636744ab217bbda053 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 13:20:28 +0300 Subject: [PATCH 216/275] fix: tony-only defines stay off svcore and the plugins -DTONY_DEV_CHECKS was in general_defines, which every target gets, so adding it recompiled all of svcore and the plugins (about 620 steps) though only main/ reads it. It is now in tony_defines, which only Tony's own targets get: a change there leaves svcore, pyin and chp alone. Co-Authored-By: Claude Opus 5.5 --- docs/building.md | 4 ++++ meson.build | 13 ++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/building.md b/docs/building.md index 88ec5ac0..4a7bf84b 100644 --- a/docs/building.md +++ b/docs/building.md @@ -78,6 +78,10 @@ echo "exit:$?" >> tmp/build.log include directories for `opus`, `sord-0`, `serd-0`. - `-DHAVE_MEDIAFOUNDATION` with `-lmfplat -lmfreadwrite -lmfuuid -lpropsys`; needs the `bqaudiostream` fork. +- `general_defines` goes on every target, svcore and the plugins included, so a change to + it recompiles everything (about 620 steps, 20 minutes). A define that only Tony's code + reads goes in `tony_defines`, which only Tony's own targets get. `tony_app` compiles + svgui and svapp too, so a change there still recompiles those, but not svcore. - `tony_core` / `tony_app` static libraries and the test executables; see [architecture.md](architecture.md) for what goes where. A new source file goes into `tony_core_files` or `tony_app_files`, and its header into the matching `*_moc_files` diff --git a/meson.build b/meson.build index e37284a9..7de03c0b 100644 --- a/meson.build +++ b/meson.build @@ -71,6 +71,11 @@ elif buildtype.startswith('debug') ] endif # get_option('buildtype') +# Defines that only Tony's own code reads. They go on Tony's targets +# alone: general_defines reaches svcore too, and any change to it +# recompiles the whole of svcore for nothing +tony_defines = [] + # The development checks (main/dev/, docs/calibrate-audio.md) are in # every build but a release one: debug, debugoptimized, plain, minsize # and custom alike. Packages are release builds and have none of it. @@ -78,7 +83,7 @@ endif # get_option('buildtype') dev_checks = not buildtype.startswith('release') dev_moc_args = [] if dev_checks - general_defines += [ + tony_defines += [ '-DTONY_DEV_CHECKS', ] dev_moc_args += [ @@ -1237,6 +1242,7 @@ tony_core_lib = static_library( cpp_args: [ feature_defines, general_defines, + tony_defines, ], ) @@ -1261,6 +1267,7 @@ tony_app_lib = static_library( cpp_args: [ feature_defines, general_defines, + tony_defines, ], ) @@ -1290,6 +1297,7 @@ executable( cpp_args: [ feature_defines, general_defines, + tony_defines, ], link_args: [ feature_additional_libs, @@ -1432,6 +1440,7 @@ tony_core_test_exe = executable( cpp_args: [ feature_defines, general_defines, + tony_defines, '-DTONY_TEST_DATA_DIR="' + tony_test_data_dir + '"', ], link_args: [ @@ -1472,6 +1481,7 @@ tony_app_test_exe = executable( cpp_args: [ feature_defines, general_defines, + tony_defines, '-DTONY_TEST_DATA_DIR="' + tony_test_data_dir + '"', ], link_args: [ @@ -1514,6 +1524,7 @@ if dev_checks cpp_args: [ feature_defines, general_defines, + tony_defines, ], link_args: [ feature_additional_libs, From c409401d1e695bd0af3c8608e7e645c25b6df8bd Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 13:20:28 +0300 Subject: [PATCH 217/275] fix: svgui pinned at tony-customizations, where PR #1's work landed The pin was c685b97 on claude/nifty-goodall-4kocmz. That branch is now merged into tony-customizations as 3c8e3fc, with the same content, and that is what the checkout holds; the lock file follows the checkout. Co-Authored-By: Claude Opus 5.5 --- repoint-lock.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/repoint-lock.json b/repoint-lock.json index 35d53bfa..2f2b4b43 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "c685b9726100448e015edb6342dcc67ad7932aa3" + "pin": "3c8e3fc3c7ccdb69fb542aa349a09a6404b080f7" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From b022990c9eb71b65b81a2f29c3072c235d697e10 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 13:20:28 +0300 Subject: [PATCH 218/275] docs: the app suite takes about 12 minutes on Windows Measured today: 726 s, most of it TestRecordWorkflow (432 s), TestAudioCheck (166 s) and TestUiChecks (118 s). The 8 minutes were measured on Linux. The tool timeout goes to 20 minutes so that a run under load still fits. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 4 ++-- docs/testing.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 41c844a8..6723f624 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -44,7 +44,7 @@ Run tests from `build_mingw/` with the same environment: ```sh mkdir -p ../tmp/tl TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-core.exe > ../tmp/test.log 2>&1; echo "exit:$?" -TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe > ../tmp/test.log 2>&1; echo "exit:$?" # ~9 min, real time +TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe > ../tmp/test.log 2>&1; echo "exit:$?" # ~12 min, real time TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app.exe undo_two_takes_in_order > ../tmp/test.log 2>&1 grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt ``` @@ -58,7 +58,7 @@ grep -a "^FAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt the take path (`record()`, Stop, latency, pre-roll), `AudioCheckRunner`, `CalibrateAudioDialog` or `main/dev/`; "both whole suites" then means all three. - From PowerShell or cmd, `.\build.bat test` runs everything through `meson test`. -- Give the app suite a tool timeout of 15 minutes, or run it in the background. +- Give the app suite a tool timeout of 20 minutes, or run it in the background. In a **Linux cloud session** the commands are others. The session's hook has started a build into `build/` in the background; run `deploy/linux/cloud-session.sh wait` before the diff --git a/docs/testing.md b/docs/testing.md index 21b117a4..27fdbbce 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -7,7 +7,7 @@ commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | | `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`, `TestModelChangeThrottle`, `TestRunSuite` | seconds | -| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow`, `TestUiChecks`, `TestAudioCheck` | about 9 minutes in one process, a minute and a half in eight (measured 2026-09-26 on Linux), nearly all of it `TestRecordWorkflow`, `TestAudioCheck` and `TestUiChecks`: takes are recorded in real time | +| `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow`, `TestUiChecks`, `TestAudioCheck` | about 12 minutes on Windows; on Linux 9 in one process, a minute and a half in eight (measured 2026-09-26), nearly all of it `TestRecordWorkflow`, `TestAudioCheck` and `TestUiChecks`: takes are recorded in real time | | `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about 4 minutes in one process, a little over one in eight (2026-09-26, Linux): each test records a dev run's takes, or part of them, in real time | `meson test` / `build.bat test` runs these three (`test-tony-dev` where it is built) plus From 5ba14906df68c86dbe104e73ec46fec194688fc1 Mon Sep 17 00:00:00 2001 From: jhhr Date: Sat, 26 Sep 2026 13:22:21 +0300 Subject: [PATCH 219/275] docs: builds use -j 4, one job per core With Windows' page file back on, four compiles fit next to an editor and a browser; without it, Windows promises programs no more memory than the RAM and a large rebuild ran out of memory at higher -j. building.md now says that -j 4 relies on the page file. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 4 ++-- docs/building.md | 12 +++++++----- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6723f624..7fec81de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,12 +29,12 @@ from sh. A Linux cloud session builds otherwise: see the end of this section. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 +ninja -j 4 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log; tail -20 tmp/build.log ``` - Only `mingw64/bin` on PATH (never `/c/msys64/usr/bin`); always set `MINGW_PREFIX`, - spelled exactly so; always `-j 3`; always log to a file and never pipe ninja; targets + spelled exactly so; always `-j 4`; always log to a file and never pipe ninja; targets need `.exe`. The reasons are in [docs/building.md](docs/building.md). - Incremental builds take under a minute, a few minutes after `MainWindow.cpp`; a clean build up to 30 minutes. If `cc1plus.exe` runs out of memory, run the command again. diff --git a/docs/building.md b/docs/building.md index 4a7bf84b..0af3e88e 100644 --- a/docs/building.md +++ b/docs/building.md @@ -28,7 +28,7 @@ From PowerShell its output is safe to capture: `.\build.bat *> tmp\build.log`. ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -ninja -j 3 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 +ninja -j 4 -C build_mingw Tony.exe test-tony-core.exe test-tony-app.exe test-tony-dev.exe > tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log tail -20 tmp/build.log ``` @@ -46,9 +46,11 @@ Each part of that is there because of something that went wrong: - **Spell `MINGW_PREFIX` exactly `C:/msys64/mingw64`.** A reconfigure with a different spelling than the build directory was set up with changes the include flags and rebuilds everything (about 560 steps). -- **`-j 3`.** At ninja's default parallelism a large rebuild runs this machine out of - memory (`cc1plus.exe: out of memory`, bash cannot fork). If it happens, run the same - command again; ninja carries on where it stopped. +- **`-j 4`**, one job per core. It relies on Windows' page file being on. Without one, + Windows can promise programs no more memory than the RAM, and with an editor and a + browser open a large rebuild ran out (`cc1plus.exe: out of memory`, bash cannot fork) + while RAM was still free. If it happens, run the same command again; ninja carries on + where it stopped. - **Redirect to a log and never pipe ninja.** The output is large and can stall or time out the tool. `tmp/` is gitignored and is the place for logs. - **Write ninja's exit status into the log.** The status of a `ninja ...; tail ...` chain @@ -66,7 +68,7 @@ Reconfigure from scratch (rarely needed): ```sh export PATH="/c/msys64/mingw64/bin:$PATH" MINGW_PREFIX="C:/msys64/mingw64" -meson setup --wipe build_mingw > tmp/build.log 2>&1 && ninja -j 3 -C build_mingw Tony.exe >> tmp/build.log 2>&1 +meson setup --wipe build_mingw > tmp/build.log 2>&1 && ninja -j 4 -C build_mingw Tony.exe >> tmp/build.log 2>&1 echo "exit:$?" >> tmp/build.log ``` From 9589dfae9ab9417dc1f9a63ecf1aa82722eb7558 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:00:59 +0000 Subject: [PATCH 220/275] fix: calibrate audio measures a device at 48 kHz like any other Since a recording is converted to the reference's rate before the splice, a take from a 48 kHz device lands right, but Calibrate Audio still called the device a rate mismatch, refused its figure, and said takes could not line up. WASAPI opens at the mixer's rate, usually 48 kHz, so it could not have been calibrated. The rate is now a fact in the details; the tests measure at 48 kHz against the fake's delay in seconds, and check that the kept figure places a second check's takes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/calibrate-audio.md | 82 ++++++++++-------- docs/manual-checklist.md | 2 +- docs/takes.md | 5 -- docs/testing.md | 5 +- main/AudioCheckRunner.cpp | 9 +- main/AudioCheckRunner.h | 21 ++--- main/CalibrateAudioDialog.cpp | 154 ++++++++++++++++------------------ main/test/TestAudioCheck.h | 150 +++++++++++++++++++++++++++------ 8 files changed, 257 insertions(+), 171 deletions(-) diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index ffb11dfc..4f1765ea 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -24,11 +24,13 @@ Audacity's measurements) was a separate report, not kept in the repository. `Pa_GetStreamInfo()`, which on MME, DirectSound and WASAPI is buffer sizes only. Audacity measured it off by −5 to +155 ms. bqaudioio opens the stream with `suggestedLatency = 0.2` on both sides. -- **The device's sample rate is not checked.** The device opens at PortAudio's default +- **The device need not run at the reference's rate.** It opens at PortAudio's default rate: for "(System Default)" through MME most likely 44.1 kHz, for a device whose name - exists only under WASAPI or WDM-KS often 48 kHz. A take's first recording is written at - that rate and placed frame for frame on the 44.1 kHz reference, so at 48 kHz it lands - early by 8 % of its position. The check names the mismatch; it does not fix it (§10). + exists only under WASAPI or WDM-KS often 48 kHz. A recording is converted to the + reference's rate as it is spliced, and the round trip is counted in seconds and turned + into frames of the recording ([recording.md](recording.md#latency)), so a check on such + a device is judged, and its figure kept, like any other. The result names the device's + rate among its figures (§4). - **The manual checklist's device items had never been run.** Many ask whether logic the app suite proves on `FakeAudioIO` holds on a real device. A loopback run in the real app answers that without a person listening. @@ -84,9 +86,8 @@ would show how the run ended. Three pages: −12 dBFS it was made at, and the sonification silent. It stays so after the run; a session opened afterwards plays as before. 4. **Judged from the take's file** (`AudioCheckRunner::readTakeFile()`), mixed to one - channel at the rate it was recorded, never from the take's model: the model is - normalised to full scale as it is read, so every take would read as clipped, and - resampled to the session's rate. + channel at the file's rate (the reference's), never from the take's model: the model + is normalised to full scale as it is read, so every take would read as clipped. Under a minute in all. Every step has a limit and ends the run with a reason: 60 s for the reference's analysis, 30 s for a take's, a take's lead-in and range plus 10 s to stop @@ -150,17 +151,15 @@ kept. | Unsteady | they disagree by 5 to 15 ms | small enough: the measured round trip is the middle of it, and a take may land up to half the spread off | yes | | Ok | none of these | the sweeps came back steadily | yes | -Two findings stand beside the verdict: +One finding stands beside the verdict: **an echo**, a second peak at the same delay, +within 3 ms, after more than half of the sweeps heard and three at least, 20 ms late or +more and no more than 30 dB down: the input is played back out somewhere (Windows' "Listen +to this device", an interface's monitor) and heard again. A paragraph on the result page; +not a verdict. An echo under 20 ms is not seen: that is where the tail of a close +reflection lies. -- **A rate mismatch**, the recording's rate not the reference's. It replaces the verdict's - words, and the calibration is not usable whatever the verdict says. It comes from the - two rates, not from the sweeps: at 48 kHz a punch-in from about 10 s into the reference - lands further off than the finder searches, and the sweeps then read as Scattered. -- **An echo**: a second peak at the same delay, within 3 ms, after more than half of the - sweeps heard and three at least, 20 ms late or more and no more than 30 dB down: the - input is played back out somewhere (Windows' "Listen to this device", an interface's - monitor) and heard again. A paragraph on the result page; not a verdict. An echo under - 20 ms is not seen: that is where the tail of a close reflection lies. +A device at another rate than the reference's is judged like any other: no verdict comes +from the rates. **The calibrated round trip** is the one the takes were placed with plus the median offset. A take that landed late was spliced from too early a frame of its recording, so the round @@ -173,10 +172,11 @@ and 28 dB over it); none has been tuned on a real device yet. ## 4. The result page The verdict in one sentence and its fix; the echo, if one was heard; then a table: the -round trip measured (not for NoSignal or a rate mismatch) against the driver's, output plus -input; what the takes were placed with (measured before, or the driver's figure); where each -punch-in landed (+ is late); the spread; sweeps found of those judged; both rates; the input -peak in dBFS; the echo; the devices. A failed run shows why it ended instead. +round trip measured (not for NoSignal) against the driver's, output plus input; what the +takes were placed with (measured before, or the driver's figure); where each punch-in +landed (+ is late); the spread; sweeps found of those judged; both rates ("recorded at +48000 Hz, converted to the reference's 44100 Hz" where they differ); the input peak in +dBFS; the echo; the devices. A failed run shows why it ended instead. **Use this latency** keeps the calibrated round trip for the devices the check started on, not for those the Preferences name when it is pressed (the result stays on show for as long @@ -210,8 +210,10 @@ the round trip is exactly the old sum; a core test checks it over a grid of valu The menu line, Forget Measured Latency and the dialog's instructions use the rate of the last take placed with a round trip, or before any take the session's: the device's rate is -not known before a take (`AudioCallbackRecordTarget` has no getter for it), and the session's -is the only rate a usable check stores at. Choosing a device from the menu resets it. +not known before a take (`AudioCallbackRecordTarget` has no getter for it). Choosing a +device from the menu resets it. So on a device at another rate than the session's, the +three see a figure kept for it only once a take has been recorded since Tony started or +the device was chosen; takes are placed with it from the first. A dev run places its takes with the round trip the calibration before it measured, for the run only: nothing is stored, the menu line goes on describing the window's own figure, and @@ -262,7 +264,8 @@ Why so: - **The long song first**, so that the dev reference then replaces its session as a check's own, unsaved, without asking, and the run still ends on the saved session: no - saved session is ever replaced. Far into a song is also where a rate mismatch shows. + saved session is ever replaced. Far into a song is also where a take converted at the + wrong rate would show. - **Stage 2's ranges** leave room for the rest: the start (before 4.3 s) for stage 4; the held tones for stage 5; and each range holds two sweeps, so that stage 3 can start inside the second one past its first sweep and still judge the other. @@ -353,8 +356,8 @@ and the whole `DevChecks.txt`. What the numbers decide: wrong the driver is, and how far the offset moves from one stream start to the next (the Unsteady and Scattered thresholds, and the restart jitter of §10). - **Sweeps found, the input peak, the echo:** the finder's thresholds, NoSignal and Clipped. -- **The recording's rate:** whether the device runs at 44.1 kHz, which the rate fix of §10 - is for. +- **The recording's rate:** whether the device runs at the reference's 44.1 kHz or its + takes are converted; a figure is kept for each rate. - **The report's header:** the drivers built in and what the device reports. - **Items 1 and 2**, each sweep's offset and each start gap: the ±2 ms. **Item 3**, how far the dots trail the cursor. **Items 4 and 12**, the margin, the looks in the gaps and the @@ -433,12 +436,12 @@ the reasons. **Next: a lower-latency driver** (the user's decision, 2026-09-26, from the restart jitter below). In order: -1. **The device-rate mismatch.** A take recorded at 48 kHz is placed frame for frame into - a 44.1 kHz session. The robust fix converts when the take is spliced, whatever the - device's rate; the other way, the record target asking for the session's rate - (`AudioCallbackRecordTarget::getApplicationSampleRate()` in the svapp fork), fails where - the device runs only at its mixer's rate. It comes first because WASAPI opens at the - Windows mixer's rate, usually 48 kHz. The button then shows the fix working. +1. **The device's rate.** Done: a recording at another rate than the reference's is + converted as it is spliced, whatever the device's rate (the other way, the record + target asking for the session's rate through `getApplicationSampleRate()` in the svapp + fork, fails where the device runs only at its mixer's rate), and Calibrate Audio + measures such a device, and keeps its figure, like any other. It came first because + WASAPI opens at the Windows mixer's rate, usually 48 kHz. 2. **A `bqaudioio` fork**, `jhhr/bqaudioio` (created 2026-09-26; the remote `jhhr` in `bqaudioio/`). Nothing uses it yet: `repoint-project.json` still takes bqaudioio from sourcehut, and nothing is pinned to the fork. For choosing the host API, WASAPI's @@ -540,7 +543,9 @@ Item 10, by reading the code, does not fail for it (the join is a dip, below). - **`TestAudioCheck`** (`test-tony-app`): the check on the loopback fake, whose device reports 2 × 4096 frames out and 4096 in while the true round trip is 123 frames longer. The check measures the true one, and once it is stored a second check finds its takes - in place; a 48 kHz fake is a rate mismatch; cancel, a closed session, no device, a device + in place; the same on a 48 kHz fake, the figure kept under 48000 Hz and checked against + the fake's delay in seconds (bqaudioio's `ResamplerWrapper` adds about 1 ms, unreported, + that belongs to the round trip); cancel, a closed session, no device, a device that never calls back; the check's playback, and a session opened after it playing as before; the user's toggles and their settings untouched; plans refused; the plan's round trip and pre-roll; keeping the session; replacing a check's own session without asking, @@ -603,9 +608,14 @@ So that later work does not derive them again. opens at the output device's default rate. PortAudio's MME default is the first of 44100, 48000, … that the device accepts. - `ResamplerWrapper` resamples the play source to it; the record target records at it. - `MainWindow` sets `Preferences::setFixedSampleRate(44100)`. - - `TakeAudio::splice()` writes a take's first recording at the recording's rate without - converting positions, and refuses a later one whose rate differs from the take file's. + `MainWindow` sets `Preferences::setFixedSampleRate(44100)`. The wrapper pads with + silence what its resampler holds back, so at 48 kHz the reference goes out about 53 + frames (1.1 ms) later than the play source counts, which nothing reports; on the fake + the check measures it as part of the round trip. + - `TakeAudio::splice()` refuses a recording whose rate differs from the take file's; + `SingingTakes::spliceRecording()` converts one at another rate than the reference's + (`TakeAudio::resample()`) before it splices, so a take's file is at the reference's + rate, and so is what the check judges. - **The reported latencies** count frames at two rates: `getTargetPlayLatency()` at the play source's `getDeviceSampleRate()`, which is the session's when bqaudioio's `ResamplerWrapper` converted it, but the device's own when a device was opened before diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 721da91f..2abe4343 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -40,7 +40,7 @@ file, and compares the take before and after each punch-in. 4. **Playback > Calibrate Audio...**, with **Run the dev checks after calibrating** on (it is by default), then **Start**. A few minutes; leave the window alone meanwhile (Cancel stops the run). The dev checks run only after a calibration that can be used (verdict - Ok or Unsteady, recorded at the reference's rate, 44.1 kHz). No signal, or a fading one, + Ok or Unsteady, whatever the device's rate). No signal, or a fading one, means the microphone did not hear the sweeps, or Windows' audio enhancements took them out. The stored latency changes only through **Use this latency**. 5. The report is on the dialog's result page and in `DevChecks.txt` in Tony's application diff --git a/docs/takes.md b/docs/takes.md index eeea278b..85fcd2fb 100644 --- a/docs/takes.md +++ b/docs/takes.md @@ -277,11 +277,6 @@ Things to know, none of which stops the feature being used. See also user gets a dialog naming the file. - Playing a wave model with a **positive** start frame plays up to a block early and without an edge fade. This design avoids it: a take's file always starts at frame 0. -- A recording device whose sample rate differs from the reference's: a take's first - recording is written at the device's rate and placed frame for frame, so at 48 kHz it - lands early by 8 % of its position; a later recording at another rate than the take - file's is refused. Calibrate Audio names the mismatch; the fix is planned - ([calibrate-audio.md](calibrate-audio.md), §10). - Two recordings that meet at a frame J each fade over 5 ms against what the take held there, not into each other: where that was silence, the join is a 10 ms dip. The dev checks' join check reads it as no step, and a pitch gap of about one hop. diff --git a/docs/testing.md b/docs/testing.md index 27ba6412..ac886ffe 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -350,7 +350,10 @@ them: `loopback()`). The device reports 2 × 4096 frames out and 4096 in, and the true round trip (`inputDelay`) is 123 frames longer: a check that works measures the true one, and with it every sweep lands at 0 frames, while a run placed with the reported pair lands - 2.8 ms off. `TestDevChecks`' loopback also has `reportLevels` on, for the observer's + 2.8 ms off. With the fake at 48 kHz (`sampleRate`) the delay and the reported latencies + count its own frames, so work the true round trip out in seconds at that rate; bqaudioio's + `ResamplerWrapper` then adds about 1.1 ms that nothing reports, which the check measures + with the rest. `TestDevChecks`' loopback also has `reportLevels` on, for the observer's output levels. - **Keep them short.** Every run records in real time. - `TestAudioCheck`'s `shortPlan()` is two punch-ins of two sweeps on the calibration diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index f87d3355..64dabe7c 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -54,9 +54,7 @@ using namespace sv; bool AudioCheckResult::calibrationUsable() const { - // A take recorded at another rate is misplaced by an amount that - // grows with its position, so no one round trip places it right - if (failure != "" || rateMismatch) return false; + if (failure != "") return false; return summary.verdict == LatencyCheck::Verdict::Ok || summary.verdict == LatencyCheck::Verdict::Unsteady; } @@ -517,7 +515,7 @@ AudioCheckRunner::takeStopped() void AudioCheckRunner::judge() { - // The file holds what was recorded, at the rate it was recorded at + // The file holds what was recorded, at the reference's rate vector mono; sv_samplerate_t rate = 0; const QString error = @@ -703,9 +701,6 @@ AudioCheckRunner::end(QString failure) m_result.reportedInputLatency = first.reportedInput; m_result.recordingRate = first.recordingRate; m_result.key.rate = first.recordingRate; - m_result.rateMismatch = m_result.recordingRate > 0 && - m_result.referenceRate > 0 && - m_result.recordingRate != m_result.referenceRate; } if (failure == "") { m_result.calibratedRoundTrip = LatencyCheck::calibratedRoundTrip diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index 1f5df604..7a74b3ff 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -53,16 +53,13 @@ struct AudioCheckResult double reportedInputLatency; /// The rate the device recorded at, and the session's, which the - /// reference was made at + /// reference was made at. They may differ: a recording is converted + /// to the reference's rate before it is spliced, and the round trip + /// is counted in seconds, so a check at another rate is judged, and + /// its figure kept, like any other sv::sv_samplerate_t recordingRate; sv::sv_samplerate_t referenceRate; - /// The two rates differ. Set from the rates, whatever the sweeps - /// say: a take recorded at another rate is placed frame for frame - /// (a known bug), so it lands further off the further into the - /// reference it is, soon further than the finder looks - bool rateMismatch; - /// The round trip that would have placed the takes right /// (LatencyCheck::calibratedRoundTrip()); see calibrationUsable() double calibratedRoundTrip; @@ -75,13 +72,12 @@ struct AudioCheckResult LatencyCalibration::Key key; /// Whether calibratedRoundTrip means anything: the run was judged - /// Ok or Unsteady, at the reference's rate + /// Ok or Unsteady bool calibrationUsable() const; AudioCheckResult() : usedRoundTrip(0), reportedOutputLatency(0), reportedInputLatency(0), recordingRate(0), - referenceRate(0), rateMismatch(false), - calibratedRoundTrip(0) { } + referenceRate(0), calibratedRoundTrip(0) { } }; /** @@ -286,8 +282,9 @@ class AudioCheckRunner : public QObject static bool analysing(Analyser *analyser); /** - * A take's audio file, mixed to one channel, at the rate it was - * recorded at. The file, and not the take's model: the model is + * A take's audio file, mixed to one channel, at the file's rate + * (the reference's: a recording at another rate is converted before + * it is spliced). The file, and not the take's model: the model is * normalised to full scale as it is read (the "normalise audio" * preference), which would have every take clipped, and resampled * to the session's rate. "" on success, else what went wrong. diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index daabd8b7..b8ed509b 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -608,86 +608,73 @@ CalibrateAudioDialog::calibrationHtml() const const QString measured = milliseconds(r.calibratedRoundTrip); const QString timing = milliseconds(timingSpread(s)); - // The verdict in plain words, and what to do about it. A rate that - // differs comes first, whatever the sweeps say: they are misplaced - // by it, further the later they come + // The verdict in plain words, and what to do about it QString html; - if (r.rateMismatch) { - html += paragraph(bold(tr("The recording device runs at %1 Hz; takes " - "cannot line up until that is fixed.") - .arg(hertz(r.recordingRate)))); + switch (s.verdict) { + case Verdict::Ok: + html += paragraph(bold(tr("The test sounds came back steadily, " + "%1 after they were played.") + .arg(measured))); html += paragraph - (tr("The test reference runs at %1 Hz, as everything Tony plays " - "does. A take recorded at another rate lands further off the " - "later in the song it is, so no one latency places it right.") - .arg(hertz(r.referenceRate))); - } else { - switch (s.verdict) { - case Verdict::Ok: - html += paragraph(bold(tr("The test sounds came back steadily, " - "%1 after they were played.") - .arg(measured))); - html += paragraph - (tr("Press Use this latency to place your takes with it.")); - break; - case Verdict::NoSignal: - html += paragraph(bold(tr("Tony could not hear the test sounds: " - "it found %1 of %2.") - .arg(s.found).arg(s.judged))); - html += "
  • " + - tr("Turn the volume up, and hold the earcup right against " - "the microphone.") + "
  • " + - tr("Check that the microphone is not muted, and that it is " - "the one Tony records from (Playback ▸ Audio Input " - "Device).") + "
  • " + - tr("In Windows, turn off Sound settings ▸ your microphone ▸ " - "Audio enhancements.") + "
  • " + - tr("Do not record through a Bluetooth headset's " - "\"Hands-Free\" device: it records at telephone quality.") + - "
"; - break; - case Verdict::Clipped: - html += paragraph(bold(tr("The test sounds were too loud: the " - "recording reached full scale."))); - html += paragraph(tr("Turn the volume down, or hold the earcup a " - "little away from the microphone, and " - "check again.")); - break; - case Verdict::Fading: - html += paragraph(bold(tr("The test sounds got quieter as the " - "check went on, by %1 dB.") - .arg(QLocale().toString - (s.fadingDb, 'f', 0)))); - html += paragraph - (tr("Something is filtering the microphone, such as echo " - "cancellation or audio enhancements. In Windows, turn " - "off Sound settings ▸ your microphone ▸ Audio " - "enhancements, and check again.")); - break; - case Verdict::PositionDependent: - html += paragraph(bold(tr("The delay grew from one punch-in to " - "the next."))); - html += paragraph - (tr("The recording seems to run at another speed than the " - "playback, so no one latency places every take right.")); - break; - case Verdict::Scattered: - html += paragraph(bold(tr("The driver's timing varies from take " - "to take by %1.").arg(timing))); - html += paragraph - (tr("No one latency places every take right when it varies " - "that much. Close other programs that use sound, and " - "check again.")); - break; - case Verdict::Unsteady: - html += paragraph(bold(tr("The driver's timing varies from take " - "to take by %1.").arg(timing))); - html += paragraph - (tr("That is small enough to use: the measured round trip, " - "%1, is the middle of it. Press Use this latency to place " - "your takes with it.").arg(measured)); - break; - } + (tr("Press Use this latency to place your takes with it.")); + break; + case Verdict::NoSignal: + html += paragraph(bold(tr("Tony could not hear the test sounds: " + "it found %1 of %2.") + .arg(s.found).arg(s.judged))); + html += "
  • " + + tr("Turn the volume up, and hold the earcup right against " + "the microphone.") + "
  • " + + tr("Check that the microphone is not muted, and that it is " + "the one Tony records from (Playback ▸ Audio Input " + "Device).") + "
  • " + + tr("In Windows, turn off Sound settings ▸ your microphone ▸ " + "Audio enhancements.") + "
  • " + + tr("Do not record through a Bluetooth headset's " + "\"Hands-Free\" device: it records at telephone quality.") + + "
"; + break; + case Verdict::Clipped: + html += paragraph(bold(tr("The test sounds were too loud: the " + "recording reached full scale."))); + html += paragraph(tr("Turn the volume down, or hold the earcup a " + "little away from the microphone, and " + "check again.")); + break; + case Verdict::Fading: + html += paragraph(bold(tr("The test sounds got quieter as the " + "check went on, by %1 dB.") + .arg(QLocale().toString + (s.fadingDb, 'f', 0)))); + html += paragraph + (tr("Something is filtering the microphone, such as echo " + "cancellation or audio enhancements. In Windows, turn " + "off Sound settings ▸ your microphone ▸ Audio " + "enhancements, and check again.")); + break; + case Verdict::PositionDependent: + html += paragraph(bold(tr("The delay grew from one punch-in to " + "the next."))); + html += paragraph + (tr("The recording seems to run at another speed than the " + "playback, so no one latency places every take right.")); + break; + case Verdict::Scattered: + html += paragraph(bold(tr("The driver's timing varies from take " + "to take by %1.").arg(timing))); + html += paragraph + (tr("No one latency places every take right when it varies " + "that much. Close other programs that use sound, and " + "check again.")); + break; + case Verdict::Unsteady: + html += paragraph(bold(tr("The driver's timing varies from take " + "to take by %1.").arg(timing))); + html += paragraph + (tr("That is small enough to use: the measured round trip, " + "%1, is the middle of it. Press Use this latency to place " + "your takes with it.").arg(measured)); + break; } if (s.echo.heard) { @@ -711,9 +698,8 @@ CalibrateAudioDialog::calibrationHtml() const ""; }; // What the sweeps found says how far the driver's figure is out even - // when it cannot be used, as long as enough of them were found and - // at the right rate - const bool haveMeasurement = s.found > 0 && !r.rateMismatch && + // when it cannot be used, as long as enough of them were found + const bool haveMeasurement = s.found > 0 && s.verdict != Verdict::NoSignal; html += ""; @@ -742,8 +728,12 @@ CalibrateAudioDialog::calibrationHtml() const html += row(tr("Spread between punch-ins:"), milliseconds(s.spread)); html += row(tr("Test sounds found:"), tr("%1 of %2").arg(s.found).arg(s.judged)); + // A device at another rate than the reference's is a fact, not a + // fault: its recordings are converted as they are spliced html += row(tr("Sample rates:"), - tr("recorded at %1 Hz, reference at %2 Hz") + (r.recordingRate != r.referenceRate ? + tr("recorded at %1 Hz, converted to the reference's %2 Hz") : + tr("recorded at %1 Hz, reference at %2 Hz")) .arg(hertz(r.recordingRate)).arg(hertz(r.referenceRate))); html += row(tr("Input peak:"), s.inputPeak > 0.0 ? diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index 8a0236a1..aecd2df8 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -64,6 +64,11 @@ class TestAudioCheck : public QObject static constexpr double rate = 44100.0; + // What Windows' mixer and phones run at. The reference is made at + // rate whatever the device runs at, and a take recorded at this one + // is converted to it as it is spliced + static constexpr double otherRate = 48000.0; + // What the device reports, and what the round trip really is static constexpr int reportedOut = 2 * 4096; static constexpr int reportedIn = 4096; @@ -593,7 +598,6 @@ private slots: QCOMPARE(r.reportedInputLatency, reportedIn / rate); QCOMPARE(r.recordingRate, rate); QCOMPARE(r.referenceRate, rate); - QVERIFY(!r.rateMismatch); QVERIFY(r.calibrationUsable()); QVERIFY2(std::fabs(r.calibratedRoundTrip * rate - roundTrip) <= 4.0, describe(r).constData()); @@ -650,29 +654,118 @@ private slots: QCOMPARE(inUse.roundTrip, (reportedOut + reportedIn) / rate); } - // A device at 48 kHz: the takes are recorded at a rate other than the - // reference's, which is reported with both rates, whatever the sweeps - // say. What Tony does with such takes is a known bug of its own, so - // only what the check reports is asserted, and that it ends - void check_flags_a_rate_mismatch() { + // A device at 48 kHz against the reference at 44.1: the takes are + // recorded at the device's rate and converted as they are spliced, so + // the check is judged, and its figure kept, like any other. The fake's + // delay and the latencies it reports count its own frames: the round + // trip it really has is roundTrip frames at 48 kHz, worked out here in + // seconds. The figure is kept under the device's rate, not the + // reference's, and the Playback menu's line then shows it + void check_measures_the_round_trip_at_48000() { FakeAudioIO::Config config = loopback(); - config.sampleRate = 48000; + config.sampleRate = int(otherRate); makeWindow(config); runCheck(); if (QTest::currentTestFailed()) return; - qDebug() << "48 kHz:" << describe(m_result).constData(); - QVERIFY2(m_result.rateMismatch, describe(m_result).constData()); - QCOMPARE(m_result.recordingRate, 48000.0); - QCOMPARE(m_result.referenceRate, rate); - QVERIFY(!m_result.calibrationUsable()); + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QCOMPARE(r.summary.judged, 4); + QCOMPARE(r.summary.found, 4); + QCOMPARE(r.recordingRate, otherRate); + QCOMPARE(r.referenceRate, rate); + QCOMPARE(r.key.rate, otherRate); + + // Placed with the reported pair, in frames of the recording + QCOMPARE(int(r.takes.size()), 2); + for (const TakeLatency &t : r.takes) { + QCOMPARE(t.recordingRate, otherRate); + QCOMPARE(t.roundTrip, sv::sv_frame_t(reportedOut + reportedIn)); + } + QCOMPARE(r.usedRoundTrip, (reportedOut + reportedIn) / otherRate); + QCOMPARE(r.reportedInputLatency, reportedIn / otherRate); + // The play source has it at the reference's rate, to a frame + QVERIFY(std::fabs(r.reportedOutputLatency - reportedOut / otherRate) + < 1.0 / rate); + + // bqaudioio's ResamplerWrapper, which brings the reference to the + // device's rate, holds it back by about a millisecond that nothing + // reports (TestRecordWorkflow's latency_with_a_device_at_48000): + // part of the round trip the takes need, as on a real device at + // another rate, and never early. Counted at the wrong rate, the + // figure would be 8 % off, some 20 ms + QVERIFY(r.calibrationUsable()); + const double late = r.calibratedRoundTrip - roundTrip / otherRate; + QVERIFY2(late >= -0.0001 && late <= 0.0015, + qPrintable(QString("measured %1 ms, the fake's delay is " + "%2 ms: %3") + .arg(r.calibratedRoundTrip * 1000.0) + .arg(roundTrip / otherRate * 1000.0) + .arg(describe(r).constData()))); QVERIFY(!m_window->audioCheckTakes()); - // and such a figure is not kept - QVERIFY(!m_window->storeMeasuredLatency(m_result)); - QSettings settings; - QVERIFY(!settings.childGroups().contains("LatencyCalibration")); + QVERIFY(m_window->storeMeasuredLatency(r)); + LatencyCalibration::Key at48 = key("", ""); + at48.rate = otherRate; + LatencyCalibration::Figure figure; + { + QSettings settings; + QVERIFY(LatencyCalibration::load(settings, at48, figure)); + QCOMPARE(figure.roundTrip, r.calibratedRoundTrip); + QVERIFY(!LatencyCalibration::load(settings, key("", ""), figure)); + } + const LatencyCalibration::InUse inUse = m_window->latencyInUse(); + QVERIFY(inUse.source == LatencyCalibration::Source::Measured); + QCOMPARE(inUse.roundTrip, r.calibratedRoundTrip); + const QString line = latencyLine(); + QVERIFY2(line.startsWith("Latency: measured"), qPrintable(line)); + } + + // The round trip measured at 48 kHz, kept: a second check places its + // takes with it, turned into frames at the device's rate, and finds + // every one where it belongs. The first check is one punch-in, which + // is enough to measure with and keeps this short + void check_stored_round_trip_is_used_at_48000() { + FakeAudioIO::Config config = loopback(); + config.sampleRate = int(otherRate); + makeWindow(config); + + runCheck(onePunchIn()); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_result.calibrationUsable(), describe(m_result).constData()); + QVERIFY(m_window->storeMeasuredLatency(m_result)); + const double measured = m_result.calibratedRoundTrip; + + runCheck(); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QCOMPARE(r.summary.found, 4); + + // Near 0: the wrapper's hold-back moves by a frame or two from one + // stream start to the next, so not to 4 frames as at 44.1 kHz. A + // figure turned into frames at the wrong rate lands some 20 ms off, + // and one without the hold-back 1 ms + const double allowed = 0.0002; + QCOMPARE(int(r.summary.punchIns.size()), 2); + for (const LatencyCheck::PunchInResult &p : r.summary.punchIns) { + QVERIFY2(std::fabs(p.medianOffset) <= allowed, + describe(r).constData()); + } + QCOMPARE(int(r.takes.size()), 2); + for (const TakeLatency &t : r.takes) { + QVERIFY(t.measured); + QCOMPARE(t.roundTrip, + sv::sv_frame_t(std::llround(measured * otherRate))); + } + QVERIFY2(std::fabs(r.calibratedRoundTrip - measured) <= allowed, + describe(r).constData()); } // Cancel during a take stops it through the Stop path, clears the @@ -1432,16 +1525,18 @@ private slots: { "Use this latency" }); if (QTest::currentTestFailed()) return; - // A device at 48 kHz: what the sweeps say is not the point - AudioCheckResult fast = judgedResult(LatencyCheck::Verdict::Scattered); - fast.recordingRate = 48000; - fast.rateMismatch = true; - fast.summary.spread = 0.6; - verify(fast, false, - { "The recording device runs at 48000 Hz; takes cannot line " - "up until that is fixed.", - "recorded at 48000 Hz, reference at 44100 Hz" }, - { "varies from take to take" }); + // A device at 48 kHz is judged like any other: its rate is in the + // figures, as a fact + AudioCheckResult fast = judgedResult(LatencyCheck::Verdict::Ok); + fast.recordingRate = otherRate; + fast.key.rate = otherRate; + verify(fast, true, + { "The test sounds came back steadily", + "Press Use this latency", + "282 ms measured; the driver reports", + "recorded at 48000 Hz, converted to the reference's " + "44100 Hz" }, + { "cannot line up", "reference at 44100 Hz" }); if (QTest::currentTestFailed()) return; // Unsteady, but usable; and the microphone monitored @@ -1455,7 +1550,8 @@ private slots: { "The driver's timing varies from take to take by 8 ms.", "Your microphone is being played back somewhere", "Listen to this device", "45 ms later", - "45 ms after the sound, 12 dB quieter" }, + "45 ms after the sound, 12 dB quieter", + "recorded at 44100 Hz, reference at 44100 Hz" }, { "Kept." }); } From 9fc25b54b6aeebbb523fe6308afd8ee269566577 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:03:36 +0000 Subject: [PATCH 221/275] feat: lyrics at 65 % of their size on android On a phone the lyrics' desktop size left room for only three or four words in the pane unless it was zoomed far in, less than a verse. LyricsTrack asks svgui's lyrics layer for 65 % of it on Android (pins svgui adb3502, setLyricsTextScale()); the desktop is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/forks.md | 4 +++- main/LyricsTrack.cpp | 11 +++++++++++ main/LyricsTrack.h | 7 +++++++ main/test/TestLyricsLayer.h | 24 ++++++++++++++++++++++++ repoint-lock.json | 2 +- 5 files changed, 46 insertions(+), 2 deletions(-) diff --git a/docs/forks.md b/docs/forks.md index 1969717a..c6fc97f3 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -116,7 +116,9 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` line (where the value changes) is bold. The font (`getLyricsFontPixelSize()`) is twice the view's at the least, up to four times, and never more than an eighth of the view's height; it grows with the **square root** of the zoom, so that zooming in gives the - words room (their boxes grow with the zoom itself). No vertical scale, no feature + words room (their boxes grow with the zoom itself). `setLyricsTextScale()` (branch + `feat/tonyandroid`) draws the words at a share of that: `LyricsTrack` asks for 65 % on + Android, where the desktop's size left room for only a few words. No vertical scale, no feature description, and not editable by the pane's tools: Tony's `LyricsEditor` edits the model itself. `setHighlightFrame()` draws the region at that frame in amber (the latest to start, where diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index dcf678b7..5c70572a 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -54,6 +54,16 @@ LyricsTrack::~LyricsTrack() { } +double +LyricsTrack::textScale() +{ +#ifdef Q_OS_ANDROID + return 0.65; +#else + return 1.0; +#endif +} + QString LyricsTrack::layerName() { @@ -162,6 +172,7 @@ LyricsTrack::configureLayer() // scale to this layer m_layer->setVerticalScale(RegionLayer::EqualSpaced); m_layer->setPlotStyle(RegionLayer::PlotLyrics); + m_layer->setLyricsTextScale(textScale()); // The words are dark on light boxes whatever this is; grey rather // than the layer's default black wherever else its colour shows diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index 3c83c643..0cba15a1 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -110,6 +110,13 @@ class LyricsTrack : public QObject */ static QString layerName(); + /** + * The share of svgui's size the words are drawn at: 65% on a phone, + * where the desktop's size leaves room for only a few words in the + * pane and a verse wants to be on show at once; all of it elsewhere. + */ + static double textScale(); + private slots: void layerAboutToBeDeleted(sv::Layer *); diff --git a/main/test/TestLyricsLayer.h b/main/test/TestLyricsLayer.h index 8ca3a53f..2acb8ad9 100644 --- a/main/test/TestLyricsLayer.h +++ b/main/test/TestLyricsLayer.h @@ -582,6 +582,30 @@ private slots: QCOMPARE(row.width(), m_pane->getPaintWidth()); } + // A phone draws the words smaller (LyricsTrack::textScale()): the + // row of boxes, as high as a line of the font and a margin, comes + // down with the font + void the_words_are_drawn_at_the_text_scale() { + QCOMPARE(m_layer->getLyricsTextScale(), 1.0); + addSpacedWords(3); + render({ QRect(0, 0, kWidth, kHeight) }); + int margin = m_pane->scalePixelSize(4); + int whole = m_layer->getLyricsBoxRow(m_pane).height() - margin; + QVERIFY(whole > 10); + + m_layer->setLyricsTextScale(0.65); + render({ QRect(0, 0, kWidth, kHeight) }); + int scaled = m_layer->getLyricsBoxRow(m_pane).height() - margin; + QVERIFY2(std::abs(scaled - 0.65 * whole) <= 2.0, + qPrintable(QString("a line of the words is %1 px high at " + "65%, against %2 px at 100%") + .arg(scaled).arg(whole))); + + m_layer->setLyricsTextScale(1.0); + render({ QRect(0, 0, kWidth, kHeight) }); + QCOMPARE(m_layer->getLyricsBoxRow(m_pane).height() - margin, whole); + } + void painting_in_strips_matches_painting_whole() { // A view that scrolls repaints only the strip that comes into // sight; the labels in it must be where they were when the diff --git a/repoint-lock.json b/repoint-lock.json index 65a3c156..b45fdc7d 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "049c6d9f53bf0c7a1f7273e89aa06f0dce05868a" + "pin": "adb3502afb303f3caf6a04f77dcf0acb605d85b9" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From 806b28a59a8a50d5e56403b73e4e57ee08a84479 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:06:20 +0000 Subject: [PATCH 222/275] build: bqaudioio from the fork, with a PortAudio implementation per windows host api jhhr/bqaudioio's feat/wasapi: mme, directsound and wasapi are PortAudio restricted to one host API, each with its own devices and defaults; a WASAPI device converts rates; the suggested latency can be set. Linux builds none of the Windows part, and the suites are green against it. container-setup.sh now moves a checkout made from the old mirror over to the library's new home, so an environment's snapshot keeps working. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- deploy/linux/container-setup.sh | 11 +++++++---- docs/audio-drivers-work-orders.md | 17 +++++++++++++++++ docs/audio-drivers.md | 10 +++++++++- docs/building.md | 2 +- docs/calibrate-audio.md | 17 ++++++++++------- docs/forks.md | 27 ++++++++++++++++++++++----- docs/mobile-port.md | 9 +++++---- docs/open-points.md | 14 ++++++-------- repoint-lock.json | 2 +- repoint-project.json | 7 ++++--- 10 files changed, 82 insertions(+), 34 deletions(-) diff --git a/deploy/linux/container-setup.sh b/deploy/linux/container-setup.sh index 65b21c5a..74cd804f 100755 --- a/deploy/linux/container-setup.sh +++ b/deploy/linux/container-setup.sh @@ -149,9 +149,7 @@ done # carry, and a conversion back to Mercurial does not reproduce it. Each # pin was matched to a mirror commit by hand, by date and message: # -# - bqaudioio 017ab3ed3a33 is the merge of toggle-record-in-io into -# default: the mirror's "Merge from branch toggle-record-in-io". -# - The other pins were taken by upstream Tony's "Update Repoint +# - The pins were taken by upstream Tony's "Update Repoint # locations and revisions" (2024-06-25). For each, the commit is the # last one on the mirror's master before that date, and Sonic # Visualiser's repoint-lock.json took the same Mercurial pin shortly @@ -168,7 +166,6 @@ done mirror_commit() { case "$1 $2" in - "bqaudioio 017ab3ed3a33") echo 7ab6de96b44d2f0c8ce58a16ce1a9831724dd29f ;; "dataquay 79623fb778da") echo 2dbf1bed112c1a7eaaf43335abbe5ddb5c03d0ff ;; "bqvec 291cde50db9d") echo ddfcd1716576c6bb44218c5f5696bf24a248960a ;; "bqfft d41a117b8cbe") echo 68dc4c5735c1e0da099e8473fc4562acf2895cb8 ;; @@ -195,6 +192,12 @@ checkout() { warnings=1 return fi + # A library that has moved, as bqaudioio did from its mirror to + # the fork, is fetched from where it is now + if [ "$(git -C "$name" remote get-url origin 2>/dev/null)" != "$url" ]; then + echo " $name: origin is now $url" + git -C "$name" remote set-url origin "$url" + fi if ! git -C "$name" cat-file -e "$commit^{commit}" 2>/dev/null; then echo " $name: fetching from $url" git -C "$name" fetch -q origin diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md index 65bec243..61885c3b 100644 --- a/docs/audio-drivers-work-orders.md +++ b/docs/audio-drivers-work-orders.md @@ -88,6 +88,14 @@ spec was silent; anything fragile, unfinished, or needed from a library. reference's handled (A1: the recording resampled before the splice, `TakeTiming` converting; A11: the cursor keeps the reference's pace). - All three suites green, sharded and in one process each. No expected failures left. +- W1 and W2 are done (the spec's §6). bqaudioio is the fork, at its `feat/wasapi`: the + implementations `mme`, `directsound` and `wasapi` exist **on Windows only**; on Linux + `AudioFactory::getImplementationNames()` is as before (`pulse`, `port`, `jack`). So a + test of anything that lists or chooses a driver cannot get the list from the factory + here: give `MainWindow` a virtual that returns it, as `createAudioIO()` is virtual for + `FakeAudioIO`, and have `TestMainWindow` override it. +- `AudioFactory::setSuggestedLatency(seconds)` exists (0 or less: the default, 0.2 s), + for streams opened after the call. ## 4. Work orders @@ -116,3 +124,12 @@ figure and checks again with it. ## 5. Log Capped at 25 lines per entry. Newest last. + +**W1** (agent; lead reviewed and committed). The rate-mismatch verdict went; two 48 kHz +tests (measure, then place with the kept figure), both seen failing with the mismatch +blocking put back, and the second with the kept figure turned into frames at the +session's rate. A 48 kHz check measures about 1.1 ms over the fake's delay: bqaudioio's +`ResamplerWrapper` holds that back, unreported, and a real device has it too. + +**W2** (lead). The fork as the spec's §4; pinned. Tony's build unchanged on Linux; the +suites green against the fork. diff --git a/docs/audio-drivers.md b/docs/audio-drivers.md index ac062193..1b78f1d1 100644 --- a/docs/audio-drivers.md +++ b/docs/audio-drivers.md @@ -117,7 +117,15 @@ Each leaves the tree building and all three suites green, committed and pushed t ## 6. State -- Nothing built yet. +- **W1 Done.** Calibrate Audio judges a device at 48 kHz like any other and keeps its + figure; its details name both rates. Left: before a device's first take, the menu line, + Forget Measured Latency and the dialog look the figure up at the session's rate, so on a + 48 kHz device they show the driver's figure although a 48 kHz one is kept (takes use the + kept one). A fix needs the device's rate before the first take (svapp). +- **W2 Done.** `jhhr/bqaudioio` `feat/wasapi`, pinned; `repoint-project.json` takes it from + the fork, and `container-setup.sh` moves a checkout made from the mirror over to it. + Compiled for Linux in Tony's build and cross-compiled for Windows; run on no device yet + (the container has none). ## 7. Open diff --git a/docs/building.md b/docs/building.md index 88ec5ac0..856e2c1b 100644 --- a/docs/building.md +++ b/docs/building.md @@ -94,7 +94,7 @@ echo "exit:$?" >> tmp/build.log # Building on Linux For a cloud session: Ubuntu 24.04, 4 cores, 16 GB, root, no sound card, and no Windows. -repoint does not run there, and hg.sr.ht, where six of the libraries live, cannot be +repoint does not run there, and hg.sr.ht, where five of the libraries live, cannot be reached. Three scripts in `deploy/linux/` do the work: - **`cloud-environment.sh` is the cloud environment's setup script.** Its text is pasted diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 4f1765ea..f1d03683 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -442,10 +442,10 @@ below). In order: fork, fails where the device runs only at its mixer's rate), and Calibrate Audio measures such a device, and keeps its figure, like any other. It came first because WASAPI opens at the Windows mixer's rate, usually 48 kHz. -2. **A `bqaudioio` fork**, `jhhr/bqaudioio` (created 2026-09-26; the remote `jhhr` in - `bqaudioio/`). Nothing uses it yet: `repoint-project.json` still takes bqaudioio from - sourcehut, and nothing is pinned to the fork. For choosing the host API, WASAPI's - automatic rate conversion, and a `suggestedLatency` that can be set. +2. **A `bqaudioio` fork**, `jhhr/bqaudioio`, pinned: an implementation per Windows host + API, WASAPI's automatic rate conversion, and a `suggestedLatency` that can be set + ([forks.md](forks.md#bqaudioio)). Done; the plan from here on is in + [audio-drivers.md](audio-drivers.md). 3. **A driver type in Tony**: MME, DirectSound or WASAPI; the device menus list that type's devices only (today every host API's are listed, and `getDeviceIndex()` takes the first name that matches, which is MME's); the stored round trip kept per type. @@ -501,8 +501,9 @@ Item 10, by reading the code, does not fail for it (the join is a dip, below). - The calibration's result page does not give the microphone's channel or the noise floor: nothing in the calibration measures them (item 5 of the dev run finds the channel). - Windows' audio enhancements, echo cancellation or noise suppression can take the sweeps - out, and Tony cannot ask for raw capture (bqaudioio is upstream, and MME has no raw - mode): NoSignal and Fading tell the user to turn them off. + out, and Tony does not ask for raw capture (MME has no raw mode, and the bqaudioio + fork does not ask WASAPI for its own): NoSignal and Fading tell the user to turn them + off. - Not in the dev run, of what the retired `test-tony-device` did: a take with no lead-in and not into a selection, stopped by hand; "no dialog" over every take (item 14 watches stages 3 and 4); with no device at all, the "Couldn't open audio device" warning once for @@ -596,7 +597,9 @@ and the dev checks". So that later work does not derive them again. -- **bqaudioio's `PortAudioIO`** (upstream, not a fork): +- **bqaudioio's `PortAudioIO`**, as upstream has it and the fork's `port` still does + (the fork's per-host-API implementations and settable latency: + [audio-drivers.md](audio-drivers.md)): - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; - input goes to the record target **before** output is asked for, in the same callback; - `suspend()`/`resume()` are `Pa_StopStream`/`Pa_StartStream`. `MainWindowBase::stop()` diff --git a/docs/forks.md b/docs/forks.md index 1969717a..5722a159 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -10,12 +10,9 @@ forks under `github.com/jhhr` that exist only for this Tony fork: | `svgui/` | `jhhr/svgui` `tony-customizations` | See below. | | `svapp/` | `jhhr/svapp` `tony-customizations` | See below. | | `bqaudiostream/` | `jhhr/bqaudiostream` `master` | `` instead of `` under MinGW, needed for `-DHAVE_MEDIAFOUNDATION`. | +| `bqaudioio/` | `jhhr/bqaudioio` `master` | See below. Upstream is Mercurial on sourcehut; the fork started from its GitHub mirror. | -`pyin/` and the rest are upstream and must stay untouched. `bqaudioio/` too, for now: a -fork of it, `jhhr/bqaudioio`, was created on 2026-09-26 for the lower-latency driver work -([open-points.md](open-points.md)), and the checkout has it as the remote `jhhr`, but -`repoint-project.json` still takes bqaudioio from sourcehut and nothing is pinned to the -fork. It joins the table when that work first pins it. +`pyin/` and the rest are upstream and must stay untouched. ## Changing a fork @@ -175,6 +172,26 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` view drops its cache on a change. On a hi-DPI desktop (ratio 2) this doubles points and notes, and thickens the pens of every layer drawn through a `ViewProxy`. +### bqaudioio + +The driver project ([audio-drivers.md](audio-drivers.md)), on the fork branch +`feat/wasapi`: + +- **An implementation per Windows host API.** `mme`, `directsound` and `wasapi` are + PortAudio restricted to that host API: their device lists hold its devices only, and + with no device named its own default devices are used. `port` is as upstream has it: + every host API's devices, the first whose name matches, and PortAudio's default + devices. The three are reported only on Windows and only where PortAudio has the host + API; asking for them elsewhere would initialise PortAudio for nothing. +- **WASAPI converts rates.** A WASAPI device opens with `paWinWasapiAutoConvert`: in shared + mode each side runs at its own mixer's rate, and the stream opens at the output's. The + header comes from PortAudio (`pa_win_wasapi.h`, found with `__has_include`). +- **A settable latency.** `AudioFactory::setSuggestedLatency()` sets what both sides ask + for, for the streams opened after; 0.2 s, upstream's fixed figure, when unset. + +Only the Windows cross-compile (MinGW-w64 against PortAudio 19.7.0's headers) and the +user's PC see the Windows part; Linux builds none of it. + ## Known defects in the forks, not fixed - `svapp/audio/AudioCallbackRecordTarget.cpp` connects to `SIGNAL(aboutToBeDeleted())`, diff --git a/docs/mobile-port.md b/docs/mobile-port.md index aff3165f..bc82ba90 100644 --- a/docs/mobile-port.md +++ b/docs/mobile-port.md @@ -74,10 +74,11 @@ from a cloud session. ### Audio I/O -- **bqaudioio is upstream, not a fork.** It is on sourcehut (Mercurial), mirrored at - `github.com/breakfastquay/bqaudioio`. `AudioFactory.cpp` knows JACK, PulseAudio and - PortAudio only. A new backend, or any buffer change, means forking it by the procedure - in [forks.md](forks.md) and adding it to `repoint-project.json`. +- **bqaudioio is the fork `jhhr/bqaudioio`** since the driver project + ([audio-drivers.md](audio-drivers.md)); upstream is on sourcehut (Mercurial), mirrored + at `github.com/breakfastquay/bqaudioio`. `AudioFactory.cpp` knows JACK, PulseAudio and + PortAudio only (on Windows, PortAudio also restricted to one host API). A new backend, + or any buffer change, goes in the fork by the procedure in [forks.md](forks.md). - `PortAudioIO.cpp` (about 760 lines) is the model for a new backend: - one duplex stream, input and output in the same callback; - input and output latency from the stream info, handed to `setSystemRecordLatency()` diff --git a/docs/open-points.md b/docs/open-points.md index fef01967..18874133 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -33,14 +33,12 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Not built -- **A lower-latency driver**, the next project (the user's decision, 2026-09-26): first - the device-rate fix (a take recorded at 48 kHz is placed frame for frame into the - 44.1 kHz session; convert when it is spliced), then a `bqaudioio` fork for choosing the - host API, WASAPI's rate conversion and a settable `suggestedLatency` (`jhhr/bqaudioio` - exists, the remote `jhhr` in `bqaudioio/`, not pinned yet: [forks.md](forks.md)), then a - driver type in Tony with the stored round trip per type, then Calibrate Audio and a dev - run on each type. MME stays the default until a run shows another better - ([calibrate-audio.md](calibrate-audio.md), §10). +- **A lower-latency driver**, being built on `feat/wasapi` + ([audio-drivers.md](audio-drivers.md)). Done: a device at 48 kHz is placed and + calibrated like one at 44.1 kHz, and the pinned `bqaudioio` fork has an implementation + per Windows host API, WASAPI's rate conversion and a settable latency. Next: a driver + menu in Tony, then Calibrate Audio and a dev run on each driver. MME stays the default + until a run shows another better. - Showing two takes at once, or any comparison of takes other than switching. - Singing track gain and pan are not saved in the session. - Background music is not saved in the session; it is reloaded by hand. diff --git a/repoint-lock.json b/repoint-lock.json index 65a3c156..51ac07f4 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -31,7 +31,7 @@ "pin": "38c3e524416a" }, "bqaudioio": { - "pin": "017ab3ed3a33" + "pin": "e7c836fc1199340e55fabb40ce29a8f2a655be6a" }, "bqaudiostream": { "pin": "9b59e706f9dda8507c5e1b07ad619c060f65bd29" diff --git a/repoint-project.json b/repoint-project.json index 77997f54..fccaef0d 100644 --- a/repoint-project.json +++ b/repoint-project.json @@ -59,9 +59,10 @@ "owner": "breakfastquay" }, "bqaudioio": { - "vcs": "hg", - "service": "sourcehut", - "owner": "breakfastquay" + "vcs": "git", + "service": "github", + "owner": "jhhr", + "branch": "master" }, "bqaudiostream": { "vcs": "git", From 58b0974942adb4b04498bd34ebe1b00b45824aa5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:07:34 +0000 Subject: [PATCH 223/275] docs: the driver project's work order for w3, the driver and latency menus Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/audio-drivers-work-orders.md | 40 +++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md index 61885c3b..fa6825f2 100644 --- a/docs/audio-drivers-work-orders.md +++ b/docs/audio-drivers-work-orders.md @@ -121,6 +121,46 @@ figure and checks again with it. made false (§1 "The device's sample rate is not checked", §10's driver project step 1, the device facts on `TakeAudio::splice()` and on the check naming the mismatch). +### W3 — Driver and latency menus + +Read: the spec's §2–§4; recording.md "Latency"; calibrate-audio.md §5 (the key and the +figure in use) and §8 (what `DevChecks.txt` says); in `MainWindow.cpp`, +`audioDeviceSettingKey()`, `audioImplementationName()`, `buildAudioDeviceMenu()`, +`rescanAudioDevices()`, `audioDeviceSelected()` and where the Playback menu makes the two +device submenus; svapp's `MainWindowBase::createAudioIO()` and `recreateAudioIO()`, and +where `createAudioIO()` is first called. + +- **"Audio Driver" submenu** in the Playback menu, before the two device submenus: one + checkable entry per driver implementation the factory reports (`mme`, `directsound`, + `wasapi`, in that order, named by `AudioFactory::getImplementationDescription()`), only + when there are at least two. The list comes from a virtual of `MainWindow` (see §3), so + that tests can give it. Choosing one: `Preferences/audio-target` becomes its name, the + latency for it is applied, playback stops, the device rate is forgotten as + `audioDeviceSelected()` does, and the IO is recreated; the device submenus then list + that driver's devices under its own keys. Greyed out during a take and while a check + runs, as Calibrate Audio is. +- **The default.** Where `audio-target` is empty (or `auto`) and `mme` is reported, Tony + sets `mme` before the first IO is made, and copies the devices saved without a suffix + to the `-mme` keys where those are unset. It then does nothing again. Find out where + the first IO is made: this has to happen before it. +- **"Audio Latency" submenu**, next to it and shown with it: 10, 20, 50, 100 and 200 ms, + checkable, kept per driver (`Preferences/audio-latency-`, in seconds; + 200 ms when unset, which is what every stream asked for until now). Applied with + `AudioFactory::setSuggestedLatency()` before every IO is made, and choosing one + recreates the IO. Greyed out as the driver menu is. +- **Reports.** `DevChecks.txt` names the driver and the latency asked for, next to the + devices it names, so that runs on two drivers can be told apart. The Calibrate Audio + dialog's details name the driver where they name the devices. +- Tests (app suite; a few, short): the menu from a given list, and none from one entry; + choosing WASAPI writes the setting, recreates the IO and moves the device keys to the + `-wasapi` suffix; greyed out during a take; the default and the copy of the devices, and + that a set `audio-target` is left alone; the latency kept per driver and applied (make + what was applied readable); a figure measured under one driver is not the one in use + under another (the key has the implementation already: test it through the window). + Show the default's test and the per-driver latency's test fail by breaking the code. +- Docs: recording.md "Latency" (the latency asked for), calibrate-audio.md where it says + the reported pair comes from `suggestedLatency = 0.2`. + ## 5. Log Capped at 25 lines per entry. Newest last. From 88783c222e85c5bea3ec2671278084e75581da6e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:09:22 +0000 Subject: [PATCH 224/275] docs: work orders for a12 and a12b, calibrate audio and the dev run on the phone Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 69 +++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 27f8f520..7f373f6d 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -181,6 +181,10 @@ builds happen in the container.) - A10 — Plot elements sized for the screen. Done. - A11 — Fixes from the fifth phone test: the cursor at a 48 kHz device, Save Log. Done. - A4c — Vertical zoom keeps the pitch in view. Done. +- (Lead, 2026-09-26: lyrics at 65 % on Android, svgui `setLyricsTextScale()`; `feat/wasapi` + merged in for Calibrate Audio at 48 kHz.) +- A12 — Calibrate Audio on the phone. +- A12b — The dev run on the phone. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -585,6 +589,71 @@ when zooming in." - Not asked, so not built: following the pitch vertically during playback, a "fit the pitch" action. Say in the report if either looks needed. +### A12 — Calibrate Audio on the phone + +The user (2026-09-26): the calibration and dev-check framework that came from `default` +"should be developed for Android too, to set the latency variables and perform the +hardware dev-test for my phone too". It is described in `docs/calibrate-audio.md` (read +sections 1, 2, 5, 9 and 10; the rest as needed). This phase: Playback > Calibrate Audio on +the phone, up to a stored, usable figure that places takes. A12b: the dev run. + +Known before starting: + +- **48 kHz**: `feat/wasapi`'s fix is merged: a device at another rate than the reference + is measured like any other. Check the phone's path through it (Oboe records at 48 kHz + only). +- **The key** (`LatencyCalibration::currentKey()`) is the driver and the playback and + record devices as the Preferences name them. On Android `OboeAudioIO` opens whatever + route the phone has (speaker, wired or USB headset, Bluetooth; A6), and none of that is + in the settings: one figure would serve every route, and a Bluetooth route is 100-200 ms + longer than the speaker's. The key must name the route Oboe opened (the output and input + device: AAudio's device id, and its type and product name from `AudioManager`), so that + each route is calibrated and used on its own. A6 reopens the device when the route + changes: the menu line follows. +- **Staleness**: a stored figure is stale when a reported latency differs by more than + 1 ms from what was reported when it was measured. Oboe's latencies come from timestamps + and move between starts: the fifth phone test's log has output 252 then 401 frames + (5.2 then 8.4 ms), input 134, 154 and 222 frames. With 1 ms every figure would be stale + at the next take. On Android the fingerprint should be what the stream was opened with + (MMAP or not, sharing and performance mode, burst, buffer size and capacity, the + devices), or a tolerance the measurements justify: choose, and test it in `tony_core`. +- **The dialog on a phone**: reachable in the compact layout; fits a landscape phone + (about 923 x 411 logical px, the log's "popups within ... of 923x411"), touch-sized + buttons, text that scrolls; the result's text copyable, and on Android a **Save + Report...** through the picker as Help > Save Log... writes (share that code rather than + copy it; A11's `AndroidStorage::Document`). +- **Instructions for a phone**: the loopback is an earcup of wired headphones held to the + phone's microphone, or the phone's own speaker and microphone in a quiet room. A headset + with a microphone of its own moves the input to it; say which input and output are in use + (the route) on the instructions page. Android's input processing: Tony opens the input + with the VoicePerformance preset (A6); say if the check sees anything that suggests echo + cancellation or noise suppression. +- Timeouts that assume a desktop's speed (`AudioCheckRunner`'s 60 s for the reference's + analysis, 30 s for a take's): the calibration reference is short; say whether they hold + on a phone several times slower, and scale them if not. + +Tests on the desktop as far as they go (the fake device at 48 kHz, the key and staleness +rules in core); the JNI for the route compiles only for Android. Say what the phone test +should do and send back. + +### A12b — The dev run on the phone + +After A12: the dev checks (`main/dev/`, compiled in the Android build, which is +`debugoptimized`) run after a usable calibration on the phone and their report reaches the +user. Read `docs/calibrate-audio.md` sections 6 to 8. + +- The report, `DevChecks.txt` in the application data directory, cannot be reached on a + phone: the result page's Save Report... (A12) saves it as well, or with the calibration's + text, and the log names where it is. +- Timeouts that assume a desktop (the long song's 240 s reference analysed within 60 s, + 4 minutes a stage): scale them for a phone from what a phone takes (the log's pYIN + times, if any, or a margin stated in the report). +- Go through the stages and checks for what differs on a phone: one input channel (item + 5), the output and input levels through `OboeAudioIO`, the save and reopen of stage 6 in + the application data directory, anything that opens a dialog or a picker, the scratch + folders. +- Tests on the desktop where the change is not Android-only; `test-tony-dev` whole. + ### A8 — Documentation pass - Bring the docs pages up to date from the code and the log: building.md (the container From dc4d74f5ef912ca2b90f63fd525fd69f2d117bb9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:41:45 +0000 Subject: [PATCH 225/275] feat: calibrate audio on the phone, a figure per route Oboe reports the route it opened (AudioRoute): a measured round trip is kept for its output and input device, and is out of date when the streams open otherwise, not when Oboe's timestamp latencies move. The check counts each punch-in as placed with the first one's round trip, as the reported pair moves between starts. The dialog gets Copy, and on Android Save Report through Save Log's picker code, the route and a phone's loopback on its instructions, and asks for the microphone before it starts. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 +++- docs/calibrate-audio.md | 12 ++ main/AudioCheckRunner.cpp | 36 ++++- main/AudioCheckRunner.h | 10 +- main/AudioRoute.cpp | 71 +++++++++ main/AudioRoute.h | 98 ++++++++++++ main/CalibrateAudioDialog.cpp | 183 ++++++++++++++++++++--- main/CalibrateAudioDialog.h | 18 +++ main/LatencyCalibration.cpp | 76 +++++++++- main/LatencyCalibration.h | 55 ++++++- main/LatencyCheck.cpp | 8 +- main/LatencyCheck.h | 22 ++- main/LatencyUtils.h | 8 + main/MainWindow.cpp | 138 ++++++++++++----- main/MainWindow.h | 55 +++++-- main/OboeAudioIO.cpp | 126 ++++++++++++++-- main/OboeAudioIO.h | 16 +- main/dev/DevChecks.cpp | 6 +- main/test/FakeAudioIO.h | 24 ++- main/test/TestAudioCheck.h | 231 +++++++++++++++++++++++++++++ main/test/TestCompactLayout.h | 11 ++ main/test/TestLatencyCalibration.h | 160 ++++++++++++++++++++ main/test/TestLatencyCheck.h | 48 ++++++ main/test/TestMainWindow.h | 7 + meson.build | 1 + 25 files changed, 1339 insertions(+), 108 deletions(-) create mode 100644 main/AudioRoute.cpp create mode 100644 main/AudioRoute.h diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 7f373f6d..a95a2933 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -183,7 +183,8 @@ builds happen in the container.) - A4c — Vertical zoom keeps the pitch in view. Done. - (Lead, 2026-09-26: lyrics at 65 % on Android, svgui `setLyricsTextScale()`; `feat/wasapi` merged in for Calibrate Audio at 48 kHz.) -- A12 — Calibrate Audio on the phone. +- A12 — Calibrate Audio on the phone. Done (the dialog's size and layout left to A12c, by the + lead's change of scope). - A12b — The dev run on the phone. - A8 — Documentation pass. @@ -1067,3 +1068,27 @@ fingers when nothing is on show; the singing counted); narrows/widens/diagonal n pitch. Pitch checks allow for the whole-Hz range (~3 px at a bottom near 40 Hz). Tests seen failing: anchor off (5); no pull (3 app, 2 core); singing not gathered; dormancy ignored (2). Left open: not on a phone; no follow in playback, no "fit the pitch" action. + +### Phase A12 — 2026-09-26 +Built: `AudioRoute` (core): a route's devices (AudioDeviceInfo id, type, product name), its +rate and how each stream opened; `AudioRouteReporter`, which `OboeAudioIO` (JNI: +`AudioManager.getDevices()`, matched by `getDeviceId()`) and the tests' fake implement. +`LatencyCalibration`: `routeKey()` ("oboe", type and product name, never the id: a headset +gets a new one at each plug-in), `onlyRecordDevice()` (a phone is output-only until its first +take), a figure's `outputStreams`/`inputStreams`. `MainWindow::latencyKey()`, +`audioRoute()`; the runner keys a result by its first take's route (`TakeLatency::route`). +Staleness on a route: stale only if a stream it describes opened otherwise (API, MMAP, +sharing, mode, burst, buffer and capacity, preset); reported latencies ignored (they moved +4 ms take to take). Desktop: key and 1 ms rule as they were. `PunchIn::placedWith`: +`judgeTake()` counts each punch-in as if placed with the first's round trip; Oboe's reported +pair moves per start, which made the spread and the calibrated figure wrong. Dialog: Copy +(all platforms), Save Report... (Android, `saveTextThroughPicker()`, Save Log's code), the +route and the phone's loopback on the instructions, a phone's advice for NoSignal/Fading, a +Streams row; Start asks for the microphone first (the runner refuses a take without it: the +permission's answer would otherwise start a take of the user's after the check had ended). +Not changed: the dialog's size and layout (A12c), the timeouts (26 s reference ~1.3 s here, +a punch-in ~0.6 s: 60 s and 30 s leave 5-10x for a slower phone). +Tests seen failing: placement not counted (core Unsteady, app Unsteady 10 ms); streams not +passed (a route's figure unused once reported latency moved). +For A8: calibrate-audio.md §5 (route key, streams rule), recording.md "Latency", §3 judging. +Left open: none of it on a phone; which input an output-only device will open is a guess. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 4f1765ea..8965affe 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -202,6 +202,18 @@ changed and the round trip with them, and takes go back to the reported pair unt check is run again. A stale figure is not deleted: it applies again if the driver goes back to its old buffers. +**On Android** there are no device settings: `OboeAudioIO` opens whatever route the phone +has, and reports it (`AudioRoute`: each device's type and product name from +`AudioManager`, and how each stream was opened). The key is then `oboe` and the two +devices' type and product name, not their ids, which a headset changes each time it is +plugged in; so the speaker, a wired headset and a Bluetooth one each keep a figure. Before +the first take only the output is open, and the key takes the one input stored for it, if +there is exactly one. Oboe's latencies come from timestamps and move by several ms between +starts, so on a route a figure is stale when a stream it describes was **opened +otherwise** (API, MMAP, sharing and performance mode, burst, buffer, capacity, input +preset), not when the reported pair moves. For the same reason the check counts each +punch-in as placed with the first one's round trip (`LatencyCheck::PunchIn::placedWith`). + At every take, `MainWindow::roundTripAt()` gives the stored figure for the devices and the recording's rate if there is one and it is not stale, else the reported sum, each reported latency converted to seconds at the rate it is counted in. Then it is turned into frames of diff --git a/main/AudioCheckRunner.cpp b/main/AudioCheckRunner.cpp index 64dabe7c..08ebde5b 100644 --- a/main/AudioCheckRunner.cpp +++ b/main/AudioCheckRunner.cpp @@ -33,6 +33,7 @@ #include "view/ViewManager.h" #include "widgets/LevelPanToolButton.h" +#include #include #include #include @@ -206,11 +207,10 @@ AudioCheckRunner::start(const Plan &plan) m_reported = Progress(); // The devices the takes will be recorded on. The window's device - // menus are shut while the run lasts; the rate comes with the takes - { - QSettings settings; - m_result.key = LatencyCalibration::currentKey(settings, 0); - } + // menus are shut while the run lasts; the rate comes with the takes, + // and so does the route of a device that reports one, which may open + // its input only for the first of them + m_result.key = m_window->latencyKey(0); m_punchIns = punchIns; m_starts.clear(); m_ends.clear(); @@ -455,6 +455,17 @@ AudioCheckRunner::startPunchIn() return; } +#ifdef Q_OS_ANDROID + // Without the microphone, record() would ask for it and start a take + // of its own once it is given, long after this run had ended. The + // dialog asks for it before it starts a check + if (!m_window->microphoneAllowed()) { + end(tr("%1 may not use the microphone. Allow it, and check again.") + .arg(QCoreApplication::applicationName())); + return; + } +#endif + const sv_frame_t from = m_starts[m_punchIn]; const sv_frame_t to = m_ends[m_punchIn]; const Step step = m_step; @@ -525,6 +536,16 @@ AudioCheckRunner::judge() return; } + // Each punch-in was placed with the round trip in use when it started, + // which on a phone moves from one to the next with what Oboe measured + // (the reported pair; a stored figure does not move): judgeTake() + // takes that out + for (int i = 0; i < int(m_punchIns.size()) && + i < int(m_result.takes.size()); ++i) { + const TakeLatency &t = m_result.takes[i]; + m_punchIns[i].placedWith = t.recordingSeconds(t.roundTrip); + } + m_result.summary = LatencyCheck::judgeTake (m_plan.layout, mono.data(), sv_frame_t(mono.size()), rate, m_punchIns); @@ -701,6 +722,11 @@ AudioCheckRunner::end(QString failure) m_result.reportedInputLatency = first.reportedInput; m_result.recordingRate = first.recordingRate; m_result.key.rate = first.recordingRate; + if (first.route.driver != "") { + m_result.route = first.route; + m_result.key = LatencyCalibration::routeKey + (first.route, first.recordingRate); + } } if (failure == "") { m_result.calibratedRoundTrip = LatencyCheck::calibratedRoundTrip diff --git a/main/AudioCheckRunner.h b/main/AudioCheckRunner.h index 7a74b3ff..f934294e 100644 --- a/main/AudioCheckRunner.h +++ b/main/AudioCheckRunner.h @@ -65,12 +65,18 @@ struct AudioCheckResult double calibratedRoundTrip; /// What the figure is kept under (MainWindow::storeMeasuredLatency()): - /// the devices as the Preferences named them when the run started, - /// and the rate the takes were recorded at. Not the devices named + /// the devices as the Preferences named them when the run started + /// (MainWindow::latencyKey()), or the route the first take was + /// recorded through, and the rate the takes were recorded at. Not the devices named /// when the figure is kept: the result is on show for as long as the /// user likes, and another device may have been chosen by then LatencyCalibration::Key key; + /// The route the first punch-in was recorded through, for a device + /// that reports one (TakeLatency::route): the key names it, and its + /// streams are the figure's fingerprint. Its driver is "" otherwise + AudioRoute::Route route; + /// Whether calibratedRoundTrip means anything: the run was judged /// Ok or Unsteady bool calibrationUsable() const; diff --git a/main/AudioRoute.cpp b/main/AudioRoute.cpp new file mode 100644 index 00000000..2ed58ecf --- /dev/null +++ b/main/AudioRoute.cpp @@ -0,0 +1,71 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AudioRoute.h" + +namespace AudioRoute { + +QString +typeName(int type) +{ + // AudioDeviceInfo's TYPE_ constants, API 36 + switch (type) { + case 1: return "Earpiece"; + case 2: return "Built-in speaker"; + case 3: return "Wired headset"; + case 4: return "Wired headphones"; + case 5: return "Analog line"; + case 6: return "Digital line"; + case 7: return "Bluetooth headset (call)"; + case 8: return "Bluetooth"; + case 9: return "HDMI"; + case 10: return "HDMI ARC"; + case 11: return "USB device"; + case 12: return "USB accessory"; + case 13: return "Dock"; + case 14: return "FM"; + case 15: return "Built-in microphone"; + case 16: return "FM tuner"; + case 17: return "TV tuner"; + case 18: return "Telephony"; + case 19: return "Aux line"; + case 20: return "IP"; + case 21: return "Bus"; + case 22: return "USB headset"; + case 23: return "Hearing aid"; + case 24: return "Built-in speaker (safe)"; + case 25: return "Remote submix"; + case 26: return "Bluetooth LE headset"; + case 27: return "Bluetooth LE speaker"; + case 28: return "Echo reference"; + case 29: return "HDMI eARC"; + case 30: return "Bluetooth LE broadcast"; + case 31: return "Analog dock"; + case 32: return "Multichannel group"; + default: break; + } + return QString("Device of type %1").arg(type); +} + +QString +deviceName(const Device &device) +{ + // The id is left out: a headset gets another each time it is + // plugged in, and its figure would be lost with it + const QString type = (device.type > 0 ? typeName(device.type) : + QString("Unknown device")); + if (device.productName.trimmed() == "") return type; + return type + " (" + device.productName.trimmed() + ")"; +} + +} diff --git a/main/AudioRoute.h b/main/AudioRoute.h new file mode 100644 index 00000000..818b5405 --- /dev/null +++ b/main/AudioRoute.h @@ -0,0 +1,98 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_AUDIO_ROUTE_H +#define TONY_AUDIO_ROUTE_H + +#include "base/BaseTypes.h" + +#include + +/** + * The route an audio device opened, where the device can say, as + * Oboe's can on a phone: which output and input it plays and records + * through (the speaker, a wired or USB headset, Bluetooth) and how it + * opened its streams. The Preferences name no device on a phone, which + * opens whatever route it has, and a Bluetooth route is 100 to 200 ms + * longer than the speaker's: a measured round trip is kept for the + * route, and is out of date once the streams open otherwise + * (LatencyCalibration). + */ +namespace AudioRoute +{ + /// A device as Android's AudioManager describes it (AudioDeviceInfo) + struct Device { + /// What AAudio opens it by. A device plugged in again gets + /// another, so it is logged but names nothing + int id; + + /// One of AudioDeviceInfo's TYPE_ constants; 0 if not known + int type; + + /// Its product name: the phone's model for its own speaker and + /// microphone, a headset's name for a Bluetooth one + QString productName; + + Device() : id(0), type(0) { } + }; + + struct Route { + /// The driver that reports it; "" for a device that reports no + /// route, whose devices the Preferences name (the desktop's) + QString driver; + + Device output; + + /// The input, while it is open; hasInput is false for a device + /// opened for playback only + bool hasInput; + Device input; + + /// The rate both streams run at + sv::sv_samplerate_t rate; + + /// How each stream was opened (the audio API, shared or + /// exclusive, its burst and buffer), on which its latency + /// depends; "" for a stream not open + QString outputStreams; + QString inputStreams; + + Route() : hasInput(false), rate(0) { } + }; + + /// A type of device in a few words, as AudioDeviceInfo's TYPE_ + /// constant of that number names it; "device of type N" for one + /// this does not know + QString typeName(int type); + + /// A device as a figure is kept for it and the user is told of it: + /// its type, and its product name if it has one, never its id + QString deviceName(const Device &device); +} + +/** + * An audio device that knows its route (OboeAudioIO; the tests' fake, + * when told one). The window asks for it by casting the device it + * opened; a device that is not one reports no route. + */ +class AudioRouteReporter +{ +public: + virtual ~AudioRouteReporter() { } + + /// The route as it was opened: its driver is "" if the device + /// cannot say + virtual AudioRoute::Route getAudioRoute() const = 0; +}; + +#endif diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index b8ed509b..167656e7 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -16,7 +16,11 @@ #include "MainWindow.h" +#include +#include #include +#include +#include #include #include #include @@ -31,6 +35,10 @@ #include #endif +#ifdef Q_OS_ANDROID +#include +#endif + #include #include #include @@ -160,8 +168,14 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, m_startButton = new QPushButton(tr("Start")); m_cancelButton = new QPushButton(tr("Cancel")); m_closeButton = new QPushButton(tr("Close")); + m_copyButton = new QPushButton(tr("Copy")); buttons->addWidget(m_useButton); buttons->addStretch(1); + buttons->addWidget(m_copyButton); +#ifdef Q_OS_ANDROID + m_saveButton = new QPushButton(tr("Save Report...")); + buttons->addWidget(m_saveButton); +#endif buttons->addWidget(m_againButton); buttons->addWidget(m_startButton); buttons->addWidget(m_cancelButton); @@ -178,6 +192,12 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, this, &CalibrateAudioDialog::useLatency); connect(m_closeButton, &QPushButton::clicked, this, &CalibrateAudioDialog::reject); + connect(m_copyButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::copyReport); +#ifdef Q_OS_ANDROID + connect(m_saveButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::saveReport); +#endif // Direct: the runner's signals carry types with no metatype connect(m_runner, &AudioCheckRunner::progress, @@ -363,6 +383,20 @@ CalibrateAudioDialog::startCheck() { if (m_running) return; +#ifdef Q_OS_ANDROID + // The takes need the microphone, which Android asks the user for, and + // answers later: the check starts then, if this is still on show. + // A take started without it would ask, and start once it is given, + // long after the check had given up (AudioCheckRunner) + if (!m_window->microphoneAllowed()) { + QPointer dialog(this); + m_window->askForMicrophone([dialog]() { + if (dialog && dialog->isVisible()) dialog->startCheck(); + }); + return; + } +#endif + m_result = AudioCheckResult(); m_latencyKept = false; m_expectedSeconds = expectedSeconds(m_plan); @@ -430,6 +464,36 @@ CalibrateAudioDialog::useLatency() showPage(Page::Result); } +QString +CalibrateAudioDialog::reportText() const +{ + QTextDocument document; + document.setHtml(resultHtml()); + return tr("%1, Calibrate Audio, %2") + .arg(QCoreApplication::applicationName(), + QDateTime::currentDateTime().toString(Qt::ISODate)) + + "\n\n" + document.toPlainText() + "\n"; +} + +void +CalibrateAudioDialog::copyReport() +{ + QGuiApplication::clipboard()->setText(reportText()); + m_copyButton->setText(tr("Copied")); +} + +#ifdef Q_OS_ANDROID +void +CalibrateAudioDialog::saveReport() +{ + m_window->saveTextThroughPicker + (this, reportText().toUtf8(), tr("Save the report"), + QString("tony-calibration-%1.txt") + .arg(QDateTime::currentDateTime().toString("yyyyMMdd-HHmmss")), + tr("report")); +} +#endif + void CalibrateAudioDialog::showResult(const AudioCheckResult &result) { @@ -536,6 +600,11 @@ CalibrateAudioDialog::showPage(Page page) m_cancelButton->setVisible(page == Page::Progress); m_againButton->setVisible(page == Page::Result); m_closeButton->setVisible(page != Page::Progress); + m_copyButton->setVisible(page == Page::Result); + m_copyButton->setText(tr("Copy")); +#ifdef Q_OS_ANDROID + m_saveButton->setVisible(page == Page::Result); +#endif m_useButton->setVisible(page == Page::Result && m_result.calibrationUsable()); m_useButton->setEnabled(!m_latencyKept); @@ -547,30 +616,78 @@ CalibrateAudioDialog::showPage(Page page) QString CalibrateAudioDialog::instructionsHtml() const { - QSettings settings; - const LatencyCalibration::Key devices = - LatencyCalibration::currentKey(settings, 0); + // The devices the Preferences name, or on a phone the route it has + // open: the phone chooses it, and a figure is kept for each route + const LatencyCalibration::Key devices = m_window->latencyKey(0); + const AudioRoute::Route route = m_window->audioRoute(); +#ifdef Q_OS_ANDROID + // Before a file is opened no device is, and the route is not known + const bool phone = true; +#else + const bool phone = (route.driver != ""); +#endif // In fives of seconds: the analyses' share is a guess const int seconds = 5 * int(std::ceil(expectedSeconds(m_plan) / 5.0)); QString html; - html += paragraph - (tr("Tony plays short chirps and records them, to measure how late " - "recordings arrive through your devices. What it measures is " - "used to place your takes on the reference.")); - html += paragraph(bold(tr("Before you start:"))); - html += "
  • " + - tr("Hold one earcup of your headphones against the microphone, " - "%1: the chirps are sharp.").arg(bold(tr("off your ears"))) + - "
  • " + - tr("Set a moderate volume, and keep the room quiet.") + - "
"; + if (!phone) { + html += paragraph + (tr("Tony plays short chirps and records them, to measure how " + "late recordings arrive through your devices. What it " + "measures is used to place your takes on the reference.")); + html += paragraph(bold(tr("Before you start:"))); + html += "
  • " + + tr("Hold one earcup of your headphones against the microphone, " + "%1: the chirps are sharp.").arg(bold(tr("off your ears"))) + + "
  • " + + tr("Set a moderate volume, and keep the room quiet.") + + "
"; + } else { + html += paragraph + (tr("Tony plays short chirps and records them, to measure how " + "late recordings arrive through the phone's output and " + "input below. What it measures is used to place your takes " + "on the reference, whenever the phone has these two.")); + html += paragraph(bold(tr("Before you start:"))); + html += "
  • " + + tr("Plug in what you will sing with first: each output and " + "input is calibrated on its own, and a Bluetooth headset " + "is much later than the phone's speaker.") + + "
  • " + + tr("With wired headphones, hold one earcup against the phone's " + "microphone (usually at its bottom edge), %1: the chirps " + "are sharp.").arg(bold(tr("off your ears"))) + + "
  • " + + tr("With nothing plugged in, the phone's own speaker and " + "microphone make the loop: lay it down in a quiet room.") + + "
  • " + + tr("A headset with a microphone of its own records from that " + "microphone: hold the earcup against it.") + + "
  • " + + tr("Set a moderate volume, and keep the room quiet.") + + "
"; + } + // Open for playback only, a phone says which input it records from + // only when it records: the one calibrated with this output before, + // if there is one, else the phone's choice. With no device open, it + // says nothing yet + QString output = deviceName(devices.playbackDevice); + QString input = deviceName(devices.recordDevice); + if (phone && route.driver == "") { + output = tr("the phone chooses when the check starts"); + input = output; + } else if (phone && !route.hasInput) { + input = (devices.recordDevice != "" ? + tr("%1, as when it was calibrated; the phone chooses when " + "recording starts").arg(devices.recordDevice) : + tr("the phone chooses when recording starts")); + } html += "
"; html += ""; + output.toHtmlEscaped() + ""; html += ""; + input.toHtmlEscaped() + ""; html += ""; @@ -605,6 +722,9 @@ CalibrateAudioDialog::calibrationHtml() const const LatencyCheck::TakeSummary &s = r.summary; const double driver = r.reportedOutputLatency + r.reportedInputLatency; + + // Recorded through a route a phone reported: its advice is a phone's + const bool phone = (r.route.driver != ""); const QString measured = milliseconds(r.calibratedRoundTrip); const QString timing = milliseconds(timingSpread(s)); @@ -622,6 +742,20 @@ CalibrateAudioDialog::calibrationHtml() const html += paragraph(bold(tr("Tony could not hear the test sounds: " "it found %1 of %2.") .arg(s.found).arg(s.judged))); + if (phone) { + html += "
  • " + + tr("Turn the volume up, and hold the earcup right against " + "the phone's microphone, or with nothing plugged in, " + "lay the phone down in a quiet room.") + "
  • " + + tr("Check that %1 may use the microphone (the phone's " + "Settings, Apps, %1, Permissions).") + .arg(QCoreApplication::applicationName()) + "
  • " + + tr("The phone may be cancelling echo or suppressing noise " + "on the microphone, which takes out what it plays " + "itself, although Tony asks for the microphone " + "unprocessed.") + "
"; + break; + } html += "
  • " + tr("Turn the volume up, and hold the earcup right against " "the microphone.") + "
  • " + @@ -646,6 +780,14 @@ CalibrateAudioDialog::calibrationHtml() const "check went on, by %1 dB.") .arg(QLocale().toString (s.fadingDb, 'f', 0)))); + if (phone) { + html += paragraph + (tr("Something on the phone is filtering the microphone, " + "such as echo cancellation or noise suppression, " + "although Tony asks for the microphone unprocessed. " + "Check again; if it fades again, send the report.")); + break; + } html += paragraph (tr("Something is filtering the microphone, such as echo " "cancellation or audio enhancements. In Windows, turn " @@ -749,8 +891,13 @@ CalibrateAudioDialog::calibrationHtml() const tr("none heard")); html += row(tr("Devices:"), tr("output %1; input %2") - .arg(deviceName(r.key.playbackDevice)) - .arg(deviceName(r.key.recordDevice))); + .arg(deviceName(r.key.playbackDevice), + deviceName(r.key.recordDevice))); + // What a figure kept for the route is checked against + if (phone) { + html += row(tr("Streams:"), tr("output %1; input %2") + .arg(r.route.outputStreams, r.route.inputStreams)); + } html += "
" + tr("Output:") + "" + - deviceName(devices.playbackDevice).toHtmlEscaped() + "
" + tr("Input:") + "" + - deviceName(devices.recordDevice).toHtmlEscaped() + "
" + tr("Latency in use:") + "" + describeLatency(m_window->latencyInUse()).toHtmlEscaped() + "
"; return html; } diff --git a/main/CalibrateAudioDialog.h b/main/CalibrateAudioDialog.h index d0c18511..321a78a7 100644 --- a/main/CalibrateAudioDialog.h +++ b/main/CalibrateAudioDialog.h @@ -79,6 +79,10 @@ class CalibrateAudioDialog : public QDialog /// Whether Use this latency is offered, and not yet pressed bool canUseLatency() const; + /// The result as plain text, under a line saying when: what Copy + /// puts on the clipboard and Save Report... saves + QString reportText() const; + /// What a check started here records: the calibration /// (AudioCheckRunner::calibrationPlan()) unless set otherwise, as /// the tests set a shorter one @@ -119,6 +123,16 @@ public slots: /// Keep the round trip the check measured, for the devices it ran on void useLatency(); + /// The result page's Copy: reportText() on the clipboard, which on a + /// phone is how selected text would be copied, and cannot + void copyReport(); + +#ifdef Q_OS_ANDROID + /// The result page's Save Report...: reportText() through the save + /// picker, as Help > Save Log... saves the log + void saveReport(); +#endif + /// The result page for this result void showResult(const AudioCheckResult &result); @@ -154,6 +168,10 @@ public slots: QPushButton *m_useButton; QPushButton *m_againButton; QPushButton *m_closeButton; + QPushButton *m_copyButton; +#ifdef Q_OS_ANDROID + QPushButton *m_saveButton; +#endif void runnerProgress(const AudioCheckRunner::Progress &progress); void runnerFinished(const AudioCheckResult &result); diff --git a/main/LatencyCalibration.cpp b/main/LatencyCalibration.cpp index 8d9feaf5..968aa14a 100644 --- a/main/LatencyCalibration.cpp +++ b/main/LatencyCalibration.cpp @@ -40,6 +40,16 @@ QString encoded(QString name) return name; } +// The other way +QString decoded(QString name) +{ + name.replace("%7C", "|"); + name.replace("%5C", "\\"); + name.replace("%2F", "/"); + name.replace("%25", "%"); + return name; +} + QString devicesGroup(const Key &key) { return encoded(key.implementation) + "|" + @@ -82,6 +92,42 @@ currentKey(QSettings &settings, sv_samplerate_t recordingRate) return key; } +Key +routeKey(const AudioRoute::Route &route, sv_samplerate_t rate) +{ + Key key; + key.implementation = route.driver; + key.playbackDevice = AudioRoute::deviceName(route.output); + if (route.hasInput) { + key.recordDevice = AudioRoute::deviceName(route.input); + } + key.rate = rate; + return key; +} + +bool +onlyRecordDevice(QSettings &settings, const Key &key, QString &recordDevice) +{ + // The groups of the driver and playback device, whatever the record + // device: their names as devicesGroup() makes them, up to the record + // device's, which is encoded and so holds no "|" + const QString prefix = encoded(key.implementation) + "|" + + encoded(key.playbackDevice) + "|"; + QStringList found; + settings.beginGroup(settingsGroup); + for (const QString &group : settings.childGroups()) { + if (!group.startsWith(prefix)) continue; + settings.beginGroup(group); + const bool atRate = settings.childGroups().contains(rateGroup(key)); + settings.endGroup(); + if (atRate) found.push_back(decoded(group.mid(prefix.size()))); + } + settings.endGroup(); + if (found.size() != 1) return false; + recordDevice = found.front(); + return true; +} + void store(QSettings &settings, const Key &key, const Figure &figure) { @@ -94,6 +140,15 @@ store(QSettings &settings, const Key &key, const Figure &figure) figure.date.toUTC().toString(Qt::ISODateWithMs)); settings.setValue("reportedOutput", number(figure.reportedOutput)); settings.setValue("reportedInput", number(figure.reportedInput)); + // Only for a device that describes its streams, so that the desktop's + // figures are kept as they always were + if (figure.outputStreams != "" || figure.inputStreams != "") { + settings.setValue("outputStreams", figure.outputStreams); + settings.setValue("inputStreams", figure.inputStreams); + } else { + settings.remove("outputStreams"); + settings.remove("inputStreams"); + } settings.endGroup(); settings.endGroup(); settings.endGroup(); @@ -113,6 +168,8 @@ load(QSettings &settings, const Key &key, Figure &figure) Qt::ISODateWithMs); f.reportedOutput = numberFrom(settings.value("reportedOutput")); f.reportedInput = numberFrom(settings.value("reportedInput")); + f.outputStreams = settings.value("outputStreams").toString(); + f.inputStreams = settings.value("inputStreams").toString(); settings.endGroup(); settings.endGroup(); settings.endGroup(); @@ -139,8 +196,19 @@ forget(QSettings &settings, const Key &key) } bool -isStale(const Figure &figure, double reportedOutput, double reportedInput) +isStale(const Figure &figure, double reportedOutput, double reportedInput, + const QString &outputStreams, const QString &inputStreams) { + // Oboe's latencies come from timestamps and move by several ms from + // one start of the same streams to the next (5.2 then 8.4 then 4.4 ms + // out on the phone first tried), which the 1 ms below would take for + // new buffers at every take. What the round trip depends on there is + // how the streams were opened: MMAP or not, exclusive or shared, + // their burst and buffer + if (outputStreams != "" || inputStreams != "") { + return (outputStreams != "" && outputStreams != figure.outputStreams) || + (inputStreams != "" && inputStreams != figure.inputStreams); + } return std::fabs(figure.reportedOutput - reportedOutput) > kStaleToleranceSeconds || std::fabs(figure.reportedInput - reportedInput) > @@ -159,12 +227,14 @@ sourceName(Source source) InUse roundTripInUse(const Figure *stored, - double reportedOutput, double reportedInput) + double reportedOutput, double reportedInput, + const QString &outputStreams, const QString &inputStreams) { InUse inUse; inUse.reportedOutput = reportedOutput; inUse.reportedInput = reportedInput; - if (stored && !isStale(*stored, reportedOutput, reportedInput)) { + if (stored && !isStale(*stored, reportedOutput, reportedInput, + outputStreams, inputStreams)) { inUse.source = Source::Measured; inUse.roundTrip = stored->roundTrip; inUse.date = stored->date; diff --git a/main/LatencyCalibration.h b/main/LatencyCalibration.h index 4cd349cb..23c5ccf5 100644 --- a/main/LatencyCalibration.h +++ b/main/LatencyCalibration.h @@ -14,6 +14,8 @@ #ifndef TONY_LATENCY_CALIBRATION_H #define TONY_LATENCY_CALIBRATION_H +#include "AudioRoute.h" + #include "base/BaseTypes.h" #include @@ -35,6 +37,12 @@ class QSettings; * has changed with them, and the figure is stale: takes go back to the * reported pair until the check is run again. * + * A device that knows its route (AudioRouteReporter: Oboe's, on a + * phone) is keyed by that instead, as the Preferences name no device + * there, and its fingerprint is how it opened its streams: Oboe works + * its latencies out from timestamps, and they move by several ms from + * one start of the same streams to the next. + * * In the settings group "LatencyCalibration", one group for the driver * and devices, within it one for the rate. All times are in seconds. */ @@ -66,6 +74,22 @@ namespace LatencyCalibration */ Key currentKey(QSettings &settings, sv::sv_samplerate_t recordingRate); + /** + * The key for a route a device reports: its driver, and its output + * and input device as AudioRoute::deviceName() names them, the input + * "" while it is not open. + */ + Key routeKey(const AudioRoute::Route &route, sv::sv_samplerate_t rate); + + /** + * The record device of the one figure kept for the key's driver and + * playback device at its rate, whatever its record device: for a + * device open for playback only, which cannot say what it will + * record from. False if there is none, or more than one. + */ + bool onlyRecordDevice(QSettings &settings, const Key &key, + QString &recordDevice); + struct Figure { /// What the check measured double roundTrip; @@ -76,6 +100,11 @@ namespace LatencyCalibration double reportedOutput; double reportedInput; + /// How a device that knows its route had opened its streams + /// (AudioRoute::Route): its fingerprint instead; "" for others + QString outputStreams; + QString inputStreams; + Figure() : roundTrip(0), spread(0), reportedOutput(0), reportedInput(0) { } }; @@ -89,11 +118,20 @@ namespace LatencyCalibration /// Drop the figure kept for the key, if there is one void forget(QSettings &settings, const Key &key); - /// Whether either latency the device reports now differs from the - /// one it reported when the figure was measured by more than - /// kStaleToleranceSeconds + /** + * Whether the figure is out of date. For a device that describes + * its streams (outputStreams or inputStreams not ""), if either + * stream it describes was opened otherwise than when the figure was + * measured; a stream it does not describe (the input of a device + * open for playback only) is not compared, and the latencies it + * reports are not either. For any other device, if either latency + * it reports now differs from the one it reported then by more than + * kStaleToleranceSeconds. + */ bool isStale(const Figure &figure, - double reportedOutput, double reportedInput); + double reportedOutput, double reportedInput, + const QString &outputStreams = QString(), + const QString &inputStreams = QString()); enum class Source { Reported, ///< the device's reported output and input latency @@ -124,11 +162,14 @@ namespace LatencyCalibration }; /** - * The stored figure, if there is one and it is not stale; otherwise - * the sum of the two reported latencies. stored may be null. + * The stored figure, if there is one and it is not stale (isStale(), + * with the streams the device describes, if any); otherwise the sum + * of the two reported latencies. stored may be null. */ InUse roundTripInUse(const Figure *stored, - double reportedOutput, double reportedInput); + double reportedOutput, double reportedInput, + const QString &outputStreams = QString(), + const QString &inputStreams = QString()); /// A reported latency, counted in frames at the given rate, in /// seconds. A latency reported as zero or less, or a rate not yet diff --git a/main/LatencyCheck.cpp b/main/LatencyCheck.cpp index ba3069c3..12f1a75e 100644 --- a/main/LatencyCheck.cpp +++ b/main/LatencyCheck.cpp @@ -499,13 +499,17 @@ LatencyCheck::judgeTake(const Layout &layout, // Across punch-ins: each one that found anything counts once, at // its median, since what moves from one to the next is the stream - // start, which every punch-in makes once + // start, which every punch-in makes once. A punch-in placed with a + // longer round trip than the first lands that much earlier, which is + // no movement of the device's + const double firstPlaced = + punchIns.empty() ? 0.0 : punchIns.front().placedWith; vector positions, medians; double within = 0.0; for (const PunchInResult &r : summary.punchIns) { if (r.found == 0) continue; positions.push_back(r.range.start); - medians.push_back(r.medianOffset); + medians.push_back(r.medianOffset + (r.range.placedWith - firstPlaced)); within = std::max(within, r.spread); } summary.medianOffset = median(medians); diff --git a/main/LatencyCheck.h b/main/LatencyCheck.h index 8c65b5f3..e4204abd 100644 --- a/main/LatencyCheck.h +++ b/main/LatencyCheck.h @@ -329,8 +329,12 @@ namespace LatencyCheck double start; double end; - PunchIn() : start(0), end(0) { } - PunchIn(double s, double e) : start(s), end(e) { } + /// The round trip the punch-in was placed with, in seconds, if + /// known; see judgeTake() + double placedWith; + + PunchIn() : start(0), end(0), placedWith(0) { } + PunchIn(double s, double e) : start(s), end(e), placedWith(0) { } }; /// One judged event: which of the layout's, in which punch-in, and @@ -382,7 +386,10 @@ namespace LatencyCheck /// Across punch-ins, over those with an event found: the /// median of their median offsets, which weighs every stream - /// start alike, and the largest minus the smallest of them + /// start alike, and the largest minus the smallest of them. + /// Each counted as if its punch-in had been placed with the + /// first one's round trip (PunchIn::placedWith): what is left is + /// how the device moved, not how the placing did double medianOffset; double spread; @@ -426,6 +433,15 @@ namespace LatencyCheck * events were found, and also when none was, judged or not: there * is then nothing to measure. Unsteady and Scattered look at the * spread across punch-ins and within each, whichever is larger. + * + * Punch-ins placed with different round trips (a device whose + * reported latencies move between starts, as Oboe's do) land that + * much apart for that reason alone: across punch-ins, each one's + * offset is taken as if it had been placed with the first one's + * round trip, so that calibratedRoundTrip() of the first one's and + * the median offset is the round trip the device had, and the + * spread and the line are its own. Each PunchInResult keeps where + * its punch-in landed. */ TakeSummary judgeTake(const Layout &layout, const float *take, sv::sv_frame_t count, diff --git a/main/LatencyUtils.h b/main/LatencyUtils.h index 0ccfec92..dea9348f 100644 --- a/main/LatencyUtils.h +++ b/main/LatencyUtils.h @@ -14,6 +14,8 @@ #ifndef TONY_LATENCY_UTILS_H #define TONY_LATENCY_UTILS_H +#include "AudioRoute.h" + #include "base/BaseTypes.h" /** @@ -56,6 +58,11 @@ computeRecordingLatency(sv::sv_frame_t outputLatency, * started, and measured once the audio callback has handed the device * its first block; startGapMeasured says whether that happened, as it * does for every take that plays the reference. + * + * The route is the one the device reported when the take started, for + * a device that knows its route (AudioRouteReporter): what a figure + * measured on the take is kept under, with its streams as the figure's + * fingerprint. Its driver is "" for any other device. */ struct TakeLatency { @@ -66,6 +73,7 @@ struct TakeLatency sv::sv_samplerate_t recordingRate; sv::sv_frame_t startGap; bool startGapMeasured; + AudioRoute::Route route; TakeLatency() : roundTrip(0), measured(false), reportedOutput(0), reportedInput(0), recordingRate(0), startGap(0), diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index b5951099..af471cbd 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -3186,13 +3186,23 @@ MainWindow::saveLog() return; } - QFileDialog dialog(this, tr("Save the log")); + saveTextThroughPicker(this, log, tr("Save the log"), + QString("tony-log-%1.txt") + .arg(QDateTime::currentDateTime() + .toString("yyyyMMdd-HHmmss")), + tr("log")); +} + +void +MainWindow::saveTextThroughPicker(QWidget *parent, const QByteArray &text, + QString title, QString suggestedName, + QString what) +{ + QFileDialog dialog(parent, title); dialog.setAcceptMode(QFileDialog::AcceptSave); dialog.setFileMode(QFileDialog::AnyFile); dialog.setMimeTypeFilters({ "text/plain" }); - dialog.selectFile(QString("tony-log-%1.txt") - .arg(QDateTime::currentDateTime() - .toString("yyyyMMdd-HHmmss"))); + dialog.selectFile(suggestedName); if (!dialog.exec()) return; QList urls = dialog.selectedUrls(); if (urls.empty() || urls[0].isEmpty()) return; @@ -3214,9 +3224,9 @@ MainWindow::saveLog() // written now; "w" if the provider will not take that, which for // the new, empty document the picker made comes to the same if (!document.open(target, "wt", error)) { - cerr << "MainWindow::saveLog: " << target << " could not be " - << "opened with \"wt\": " << error << "; trying \"w\"" - << endl; + cerr << "MainWindow::saveTextThroughPicker: " << target + << " could not be opened with \"wt\": " << error + << "; trying \"w\"" << endl; QString again; if (!document.open(target, "w", again)) error = again; } @@ -3227,7 +3237,7 @@ MainWindow::saveLog() } } - bool written = opened && out.write(log) == log.size() && out.flush(); + bool written = opened && out.write(text) == text.size() && out.flush(); if (opened && !written) error = out.errorString(); if (opened) out.close(); @@ -3242,18 +3252,18 @@ MainWindow::saveLog() } if (!written) { - cerr << "MainWindow::saveLog: could not write " << target << ": " - << error << endl; + cerr << "MainWindow::saveTextThroughPicker: could not write the " + << what << " to " << target << ": " << error << endl; QMessageBox::critical - (this, tr("Failed to save the log"), - tr("The log was not saved

%1

") - .arg(error.toHtmlEscaped()) + pickDetails(target, "", "")); + (parent, tr("Failed to save the %1").arg(what), + tr("The %1 was not saved

%2

") + .arg(what, error.toHtmlEscaped()) + pickDetails(target, "", "")); return; } if (urls[0].isLocalFile()) { - cerr << "MainWindow::saveLog: saved " << log.size() << " bytes to " - << target << endl; + cerr << "MainWindow::saveTextThroughPicker: saved the " << what + << ", " << text.size() << " bytes, to " << target << endl; return; } @@ -3268,13 +3278,14 @@ MainWindow::saveLog() for (int attempt = 1; ; ++attempt) { sizeError = ""; saved = AndroidFiles::savedSize - (log.size(), AndroidStorage::sizeOf(target, sizeError)); + (text.size(), AndroidStorage::sizeOf(target, sizeError)); if (!saved.differs() || attempt == 10) break; QThread::msleep(100); } - cerr << "MainWindow::saveLog: wrote " << saved.written << " bytes to " - << target << "; the file behind the descriptor held " << inFile + cerr << "MainWindow::saveTextThroughPicker: wrote the " << what << ", " + << saved.written << " bytes, to " << target + << "; the file behind the descriptor held " << inFile << " before the close; "; if (saved.known()) { cerr << "its provider says the document holds " << saved.held @@ -3287,9 +3298,9 @@ MainWindow::saveLog() if (saved.differs()) { QMessageBox::warning - (this, tr("The log may be incomplete"), - tr("The log may not have been saved whole

%1 bytes were written to it, and the app that keeps it says it holds %2.

") - .arg(saved.written).arg(saved.held) + (parent, tr("The %1 may be incomplete").arg(what), + tr("The %1 may not have been saved whole

%2 bytes were written to it, and the app that keeps it says it holds %3.

") + .arg(what).arg(saved.written).arg(saved.held) + pickDetails(target, "", "")); } } @@ -3534,14 +3545,14 @@ MainWindow::microphoneAllowed() const } void -MainWindow::askForMicrophone() +MainWindow::askForMicrophone(std::function granted) { qApp->requestPermission (QMicrophonePermission(), this, - [this](const QPermission &permission) { + [this, granted](const QPermission &permission) { if (permission.status() == Qt::PermissionStatus::Granted) { - // The press of Record that asked, answered at last - if (microphoneAllowed()) record(); + // What asked, answered at last + if (microphoneAllowed() && granted) granted(); return; } QMessageBox::information @@ -3592,6 +3603,8 @@ MainWindow::checkAudioDevice() << "went away; opening it again" << endl; recreateAudioIO(); updateMenuStates(); + // Another route, perhaps, with a figure of its own or none + updateLatencyMenuLine(); } #endif @@ -5192,7 +5205,7 @@ MainWindow::record() // answer comes later: the take is started then, from the top if (!microphoneAllowed()) { if (m_recordAction) m_recordAction->setChecked(false); - askForMicrophone(); + askForMicrophone([this]() { record(); }); return; } #endif @@ -5658,6 +5671,7 @@ MainWindow::recordingStarted() (inUse.source == LatencyCalibration::Source::Measured); m_takeLatency.startGap = m_recordingStartGapEstimate; m_takeLatency.startGapMeasured = false; + deviceRoute(m_takeLatency.route); cerr << "MainWindow::recordingStarted: round trip " << roundTrip << " frames at " << recordingRate << " Hz (" << inUse.roundTrip * 1000.0 << " ms), "; @@ -5668,6 +5682,9 @@ MainWindow::recordingStarted() if (inUse.source == LatencyCalibration::Source::Measured) { cerr << " on " << inUse.date.toString(Qt::ISODate).toStdString(); + } else if (inUse.stale && m_takeLatency.route.driver != "") { + cerr << ": the measured one is stale, the streams were " + << "opened otherwise"; } else if (inUse.stale) { cerr << ": the measured one is stale, the device reports " << "other latencies now"; @@ -5755,18 +5772,67 @@ MainWindow::roundTripAt(sv_samplerate_t recordingRate) const LatencyCalibration::reportedSeconds (m_recordTarget->getSystemRecordLatency(), recordingRate) : 0.0; + // A device that knows its route is judged by how it opened its + // streams, not by the latencies it reports (LatencyCalibration) + AudioRoute::Route route; + deviceRoute(route); + QSettings settings; LatencyCalibration::Figure figure; bool stored = LatencyCalibration::load - (settings, LatencyCalibration::currentKey(settings, recordingRate), - figure); + (settings, latencyKey(recordingRate), figure); return LatencyCalibration::roundTripInUse - (stored ? &figure : nullptr, output, input); + (stored ? &figure : nullptr, output, input, + route.outputStreams, route.inputStreams); +} + +bool +MainWindow::deviceRoute(AudioRoute::Route &route) const +{ + route = AudioRoute::Route(); + breakfastquay::SystemPlaybackTarget *device = m_audioIO; + if (!device) device = m_playTarget; + auto reporter = dynamic_cast(device); + if (!reporter) return false; + route = reporter->getAudioRoute(); + return route.driver != ""; +} + +AudioRoute::Route +MainWindow::audioRoute() const +{ + AudioRoute::Route route; + deviceRoute(route); + return route; +} + +LatencyCalibration::Key +MainWindow::latencyKey(sv_samplerate_t rate) const +{ + QSettings settings; + AudioRoute::Route route; + if (!deviceRoute(route)) { + return LatencyCalibration::currentKey(settings, rate); + } + + // Opened for playback only, as the device is on a phone until the + // first take, it cannot say which input it will record from: the one + // a figure is kept with for this output, if only one is. How the + // input will open is not known either, and is not compared + LatencyCalibration::Key key = LatencyCalibration::routeKey(route, rate); + if (!route.hasInput) { + LatencyCalibration::onlyRecordDevice(settings, key, key.recordDevice); + } + return key; } sv_samplerate_t MainWindow::expectedRecordingRate() const { + // A device that knows its route knows its rate as soon as it is open, + // and records at it (OboeAudioIO opens its input at its output's) + AudioRoute::Route route; + if (deviceRoute(route) && route.rate > 0) return route.rate; return m_lastRecordingRate > 0 ? m_lastRecordingRate : sessionRate(); } @@ -5787,6 +5853,8 @@ MainWindow::storeMeasuredLatency(const AudioCheckResult &result) figure.date = QDateTime::currentDateTimeUtc(); figure.reportedOutput = result.reportedOutputLatency; figure.reportedInput = result.reportedInputLatency; + figure.outputStreams = result.route.outputStreams; + figure.inputStreams = result.route.inputStreams; // Under the devices the check ran on, which the Preferences may no // longer name: the result can be on show long after the run @@ -5798,8 +5866,13 @@ MainWindow::storeMeasuredLatency(const AudioCheckResult &result) cerr << "MainWindow::storeMeasuredLatency: round trip " << figure.roundTrip * 1000.0 << " ms at " << key.rate << " Hz, the device reporting " << figure.reportedOutput * 1000.0 - << " ms out and " << figure.reportedInput * 1000.0 << " ms in" - << endl; + << " ms out and " << figure.reportedInput * 1000.0 << " ms in"; + if (result.route.driver != "") { + cerr << "; for the route " << key.playbackDevice << " | " + << key.recordDevice << ", its streams: output " + << figure.outputStreams << "; input " << figure.inputStreams; + } + cerr << endl; updateLatencyMenuLine(); return true; } @@ -5809,8 +5882,7 @@ MainWindow::forgetMeasuredLatency() { QSettings settings; LatencyCalibration::forget - (settings, LatencyCalibration::currentKey(settings, - expectedRecordingRate())); + (settings, latencyKey(expectedRecordingRate())); cerr << "MainWindow::forgetMeasuredLatency: at " << expectedRecordingRate() << " Hz" << endl; updateLatencyMenuLine(); diff --git a/main/MainWindow.h b/main/MainWindow.h index 32ef36fa..ba728fcc 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -33,6 +33,7 @@ #include #include #include +#include #include "data/model/SparseTimeValueModel.h" @@ -132,10 +133,38 @@ class MainWindow : public sv::MainWindowBase // before the take starts: the device's rate is known only once it // has recorded, so until a take has been recorded on these devices // this assumes the session's rate, the only one a usable check - // stores a figure at. The reported pair is 0 until the device is - // open + // stores a figure at (a device that reports its route says its rate + // once it is open). The reported pair is 0 until the device is open LatencyCalibration::InUse latencyInUse() const; + // The devices a measured round trip is kept for, at the given rate: + // the route the open device reports, if it reports one (OboeAudioIO + // on a phone), else the devices the Preferences name. A device open + // for playback only names the input a figure is kept with for its + // output, if only one is (LatencyCalibration::onlyRecordDevice()) + LatencyCalibration::Key latencyKey(sv::sv_samplerate_t rate) const; + + // The route the open device reports; its driver is "" if it reports + // none, or there is no device open + AudioRoute::Route audioRoute() const; + +#ifdef Q_OS_ANDROID + // The microphone is asked for when it is first needed: Record starts + // the take once it is given, Calibrate Audio the check. granted is + // called then, and not if it is refused, which is said in a box + bool microphoneAllowed() const; + void askForMicrophone(std::function granted); + + // Text saved through the save picker, as Help > Save Log... and + // Calibrate Audio's Save Report... do: a new document the picker + // makes, written and closed through its provider, with what went + // wrong, and what the provider says it holds, said over parent. + // what names the text in the messages ("log") + void saveTextThroughPicker(QWidget *parent, const QByteArray &text, + QString title, QString suggestedName, + QString what); +#endif + signals: void canExportPitchTrack(bool); void canExportNotes(bool); @@ -1097,15 +1126,21 @@ protected slots: // file is read at before there is one sv::sv_samplerate_t sessionRate() const; - // The rate the next take is expected to record at: the last one's, - // or before there is one, the session's + // The rate the next take is expected to record at: the rate of a + // device that reports its route; else the last take's, or before + // there is one, the session's sv::sv_samplerate_t expectedRecordingRate() const; + // The route of the open device, if it is an AudioRouteReporter and + // reports one; false, with route cleared, if not + bool deviceRoute(AudioRoute::Route &route) const; + // The round trip for a take recorded at the given rate, in seconds, - // and where it came from: a stored figure for these devices and this - // rate, unless the latencies the device reports have changed since - // it was measured; otherwise the reported pair, each converted from - // the frames it counts + // and where it came from: a stored figure for these devices (or this + // route: latencyKey()) and this rate, unless the latencies the device + // reports, or for a route how it opened its streams, have changed + // since it was measured; otherwise the reported pair, each converted + // from the frames it counts LatencyCalibration::InUse roundTripAt(sv::sv_samplerate_t recordingRate) const; void refineRecordingLatency(); @@ -1203,10 +1238,6 @@ protected slots: // Android backend. Its input only once the microphone may be used void createAudioIO() override; - // The microphone is asked for when Record is first pressed, and the - // take is started once it is given - bool microphoneAllowed() const; - void askForMicrophone(); // A device that goes away (headphones in or out) leaves the device // failed: looked for on a timer, and the device opened afresh diff --git a/main/OboeAudioIO.cpp b/main/OboeAudioIO.cpp index 3c1e1983..115f4509 100644 --- a/main/OboeAudioIO.cpp +++ b/main/OboeAudioIO.cpp @@ -21,6 +21,10 @@ #include +#include +#include +#include + #include #include #include @@ -57,6 +61,84 @@ monotonicNanos() return int64_t(ts.tv_sec) * 1000000000 + ts.tv_nsec; } +// How a stream was opened, as the log has it and as a measured round +// trip's fingerprint keeps it: everything its latency depends on +std::string +describeStream(oboe::AudioStream *stream) +{ + // Only an AAudio stream can be asked about MMAP + bool mmap = (stream->getAudioApi() == oboe::AudioApi::AAudio && + oboe::OboeExtensions::isMMapUsed(stream)); + std::ostringstream os; + os << oboe::convertToText(stream->getAudioApi()) + << (mmap ? " (MMAP)" : "") + << ", " << stream->getSampleRate() << " Hz, " + << stream->getChannelCount() << " channel(s), " + << oboe::convertToText(stream->getFormat()) << ", " + << oboe::convertToText(stream->getPerformanceMode()) << ", " + << oboe::convertToText(stream->getSharingMode()) + << ", burst " << stream->getFramesPerBurst() + << ", buffer " << stream->getBufferSizeInFrames() + << " of " << stream->getBufferCapacityInFrames() << " frames"; + if (stream->getDirection() == oboe::Direction::Input) { + os << ", preset " << oboe::convertToText(stream->getInputPreset()); + } + return os.str(); +} + +// The device AAudio opened a stream on, as Android's AudioManager lists +// it. Only the id if it is not listed, or the stream cannot say +// (OpenSL ES: 0). On the GUI thread, as QJniObject clears and logs any +// exception a call throws +AudioRoute::Device +lookUpDevice(int id, bool input) +{ + AudioRoute::Device device; + device.id = id; + if (id <= 0) return device; + + QJniObject context = QNativeInterface::QAndroidApplication::context(); + if (!context.isValid()) return device; + QJniObject manager = context.callObjectMethod + ("getSystemService", "(Ljava/lang/String;)Ljava/lang/Object;", + QJniObject::fromString("audio").object()); + if (!manager.isValid()) return device; + + // AudioManager.GET_DEVICES_INPUTS and GET_DEVICES_OUTPUTS + QJniObject devices = manager.callObjectMethod + ("getDevices", "(I)[Landroid/media/AudioDeviceInfo;", + jint(input ? 1 : 2)); + if (!devices.isValid()) return device; + + QJniEnvironment env; + jobjectArray array = devices.object(); + const jsize count = env->GetArrayLength(array); + for (jsize i = 0; i < count; ++i) { + QJniObject info = QJniObject::fromLocalRef + (env->GetObjectArrayElement(array, i)); + if (!info.isValid()) continue; + if (info.callMethod("getId", "()I") != id) continue; + device.type = info.callMethod("getType", "()I"); + QJniObject name = info.callObjectMethod + ("getProductName", "()Ljava/lang/CharSequence;"); + if (name.isValid()) { + device.productName = name.callObjectMethod + ("toString", "()Ljava/lang/String;").toString(); + } + break; + } + return device; +} + +std::string +describeDevice(const AudioRoute::Device &device) +{ + std::ostringstream os; + os << AudioRoute::deviceName(device).toStdString() + << ", device " << device.id << ", type " << device.type; + return os.str(); +} + std::string describe(StreamLatency::Estimate latency, bool withInput, int rate) { @@ -284,6 +366,7 @@ OboeAudioIO::OboeAudioIO(ApplicationRecordTarget *target, logStream("output", m_output.get()); if (m_input) logStream("input", m_input.get()); + findRoute(); // Run the streams until they can say what their latency is, so that // the first take is compensated by a measurement too, and leave @@ -552,22 +635,33 @@ OboeAudioIO::report(StreamLatency::Estimate latency, bool withInput) void OboeAudioIO::logStream(std::string name, oboe::AudioStream *stream) const { - // Only an AAudio stream can be asked about MMAP - bool mmap = (stream->getAudioApi() == oboe::AudioApi::AAudio && - oboe::OboeExtensions::isMMapUsed(stream)); - cerr << "OboeAudioIO: " << name << ": " - << oboe::convertToText(stream->getAudioApi()) - << (mmap ? " (MMAP)" : "") - << ", " << stream->getSampleRate() << " Hz, " - << stream->getChannelCount() << " channel(s), " - << oboe::convertToText(stream->getFormat()) << ", " - << oboe::convertToText(stream->getPerformanceMode()) << ", " - << oboe::convertToText(stream->getSharingMode()) - << ", burst " << stream->getFramesPerBurst() - << ", buffer " << stream->getBufferSizeInFrames() - << " of " << stream->getBufferCapacityInFrames() << " frames"; - if (stream->getDirection() == oboe::Direction::Input) { - cerr << ", preset " << oboe::convertToText(stream->getInputPreset()); + cerr << "OboeAudioIO: " << name << ": " << describeStream(stream) << endl; +} + +void +OboeAudioIO::findRoute() +{ + // The devices AAudio chose: for an unspecified device, those of the + // route Android has now, which is what a measured round trip belongs + // to. Their ids change when a device is plugged in again, so the + // route is named by their types and product names + m_route = AudioRoute::Route(); + m_route.driver = "oboe"; + m_route.rate = m_rate; + m_route.output = lookUpDevice(m_output->getDeviceId(), false); + m_route.outputStreams = QString::fromStdString + (describeStream(m_output.get())); + if (m_input) { + m_route.hasInput = true; + m_route.input = lookUpDevice(m_input->getDeviceId(), true); + m_route.inputStreams = QString::fromStdString + (describeStream(m_input.get())); + } + cerr << "OboeAudioIO: route: output " << describeDevice(m_route.output); + if (m_input) { + cerr << "; input " << describeDevice(m_route.input); + } else { + cerr << "; no input"; } cerr << endl; } diff --git a/main/OboeAudioIO.h b/main/OboeAudioIO.h index 871fdfe1..aed9c14d 100644 --- a/main/OboeAudioIO.h +++ b/main/OboeAudioIO.h @@ -14,6 +14,7 @@ #ifndef TONY_OBOE_AUDIO_IO_H #define TONY_OBOE_AUDIO_IO_H +#include "AudioRoute.h" #include "StreamLatency.h" #include @@ -54,12 +55,18 @@ class AudioStream; * by what the device did last. Timestamps are read on the calling * thread, never in the callback. * + * The route, the devices Android opened (the speaker and the phone's + * microphone, a headset, Bluetooth) and how their streams were opened, + * is looked up once, when they are, and logged: a round trip measured + * through them is kept for that route (LatencyCalibration). + * * A stream that fails (a device disconnected: headphones plugged in * or out) is stopped by Oboe; hasFailed() then says so, and the owner * must delete this and open another. Every method but the callback's * is for the GUI thread, which is the only one that logs, to stderr. */ -class OboeAudioIO : public breakfastquay::SystemAudioIO +class OboeAudioIO : public breakfastquay::SystemAudioIO, + public AudioRouteReporter { public: /** @@ -88,6 +95,11 @@ class OboeAudioIO : public breakfastquay::SystemAudioIO /// Whether a stream has failed, so that this must be replaced bool hasFailed() const; + /// The devices the streams were opened on, as Android's AudioManager + /// names them, and how the streams were opened (the audio API, MMAP, + /// sharing and performance mode, burst, buffer, input preset) + AudioRoute::Route getAudioRoute() const override { return m_route; } + private: class Engine; class ErrorFlag; @@ -117,6 +129,7 @@ class OboeAudioIO : public breakfastquay::SystemAudioIO bool m_startFailed; StreamLatency::Estimate m_latency; int m_outputXRuns; + AudioRoute::Route m_route; // The callback: the input first, then the output friend class Engine; @@ -128,6 +141,7 @@ class OboeAudioIO : public breakfastquay::SystemAudioIO bool measureLatency(StreamLatency::Estimate &latency) const; void report(StreamLatency::Estimate latency, bool withInput); void logStream(std::string name, oboe::AudioStream *stream) const; + void findRoute(); OboeAudioIO(const OboeAudioIO &) = delete; OboeAudioIO &operator=(const OboeAudioIO &) = delete; diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index eacb52e7..96d0786d 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -490,10 +490,8 @@ DevChecks::start(const Options &options) m_sessionPath = QDir(scratch).filePath(kSessionFileName); m_saved = false; m_startedAt = QDateTime::currentDateTime(); - { - QSettings settings; - m_devices = LatencyCalibration::currentKey(settings, 0); - } + // As the Preferences name them, or the route a phone has open + m_devices = m_window->latencyKey(0); m_layout = LatencyCheck::devLayout(); m_observedPunchIn = 0; diff --git a/main/test/FakeAudioIO.h b/main/test/FakeAudioIO.h index b1c8a107..3021d529 100644 --- a/main/test/FakeAudioIO.h +++ b/main/test/FakeAudioIO.h @@ -23,6 +23,8 @@ // application computes a known compensation, and the input can be // made to arrive late by exactly that much. +#include "../AudioRoute.h" + #include #include #include @@ -36,7 +38,8 @@ #include #include -class FakeAudioIO : public breakfastquay::SystemAudioIO +class FakeAudioIO : public breakfastquay::SystemAudioIO, + public AudioRouteReporter { public: struct Config { @@ -49,6 +52,16 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO int recordLatency = 0; int playbackLatency = 0; + // Added to the reported record latency at every resume after the + // first: a device that measures its latencies at each start, as + // Oboe does from its timestamps, reports others every time. The + // input's real delay stays inputDelay + int recordLatencyStep = 0; + + // The route reported to the application, as OboeAudioIO reports + // the one Android opened; with no driver, none, as PortAudioIO + AudioRoute::Route route; + // Mono input, delivered once and followed by silence. The // input clock restarts whenever the device is resumed std::vector input; @@ -119,6 +132,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO m_target->setSystemRecordSampleRate(m_config.sampleRate); m_target->setSystemRecordChannelCount(m_config.channels); m_target->setSystemRecordLatency(m_config.recordLatency); + m_reportedRecordLatency = m_config.recordLatency; m_thread = std::thread([this]() { run(); }); } @@ -137,6 +151,8 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO void suppressRecordSide(bool) override { } + AudioRoute::Route getAudioRoute() const override { return m_config.route; } + // No callback is running, or will start, once this returns void suspend() override { std::lock_guard guard(m_mutex); @@ -152,6 +168,11 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO m_sinceResume = 0; m_framesBeforePlayStart = -1; ++m_resumeCount; + // No callback runs while suspended, and none has started yet + if (m_config.recordLatencyStep != 0 && m_resumeCount > 1) { + m_reportedRecordLatency += m_config.recordLatencyStep; + m_target->setSystemRecordLatency(m_reportedRecordLatency); + } } bool isSuspended() const { @@ -201,6 +222,7 @@ class FakeAudioIO : public breakfastquay::SystemAudioIO long m_sinceResume; long m_framesBeforePlayStart; int m_resumeCount; + int m_reportedRecordLatency = 0; std::vector m_captured; void run() { diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index aecd2df8..ee53a17b 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -46,7 +46,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -235,6 +237,28 @@ class TestAudioCheck : public QObject return key; } + // A phone's route as OboeAudioIO reports it: the speaker and the + // phone's own microphone, or a Bluetooth headset's output and the + // same microphone, each opened as AAudio opens such a device + static AudioRoute::Route phoneRoute(bool bluetooth = false) { + AudioRoute::Route route; + route.driver = "oboe"; + route.output.id = bluetooth ? 41 : 3; + route.output.type = bluetooth ? 8 : 2; + route.output.productName = bluetooth ? "Headset X" : "Pixel 7"; + route.hasInput = true; + route.input.id = 7; + route.input.type = 15; + route.input.productName = "Pixel 7"; + route.rate = rate; + route.outputStreams = bluetooth ? + "AAudio, 44100 Hz, Shared, burst 240, buffer 480 of 3840" : + "AAudio (MMAP), 44100 Hz, Exclusive, burst 96, buffer 192 of 1920"; + route.inputStreams = "AAudio (MMAP), 44100 Hz, Exclusive, burst 96, " + "buffer 11424 of 11520, preset VoicePerformance"; + return route; + } + // The Playback menu's line about the latency, as it reads when the // menu is opened QString latencyLine() { @@ -768,6 +792,146 @@ private slots: describe(r).constData()); } + // A device whose reported latencies move from one start to the next, + // as Oboe's do on a phone: the second punch-in is placed with 10 ms + // more than the first, and lands 10 ms earlier, while the path's + // round trip has not moved. The check measures that round trip, and + // finds it steady + void check_measures_the_round_trip_when_reports_move() { + FakeAudioIO::Config config = loopback(); + config.recordLatencyStep = 441; + makeWindow(config); + + runCheck(); + if (QTest::currentTestFailed()) return; + + const AudioCheckResult &r = m_result; + QVERIFY2(r.failure == "", describe(r).constData()); + QCOMPARE(int(r.takes.size()), 2); + QVERIFY2(r.takes[1].roundTrip - r.takes[0].roundTrip >= 441, + qPrintable(QString("placed with %1 and %2 frames") + .arg(r.takes[0].roundTrip) + .arg(r.takes[1].roundTrip))); + QCOMPARE(int(r.summary.punchIns.size()), 2); + QVERIFY2(r.summary.punchIns[0].medianOffset - + r.summary.punchIns[1].medianOffset > 0.009, + describe(r).constData()); + + QVERIFY2(r.summary.verdict == LatencyCheck::Verdict::Ok, + describe(r).constData()); + QVERIFY(r.calibrationUsable()); + QVERIFY2(std::fabs(r.calibratedRoundTrip * rate - roundTrip) <= 4.0, + describe(r).constData()); + } + + // A phone: the device reports the route it opened. The figure is + // kept for that route, whatever the Preferences name, with how its + // streams were opened; its takes are placed with it although the + // latencies the device reports move by more than a millisecond from + // take to take. Another route, a Bluetooth headset, has a figure of + // its own or none, and the menu's line follows the device as it is + // opened again for each; the same route opened otherwise has its + // figure out of date + void check_keeps_the_figure_for_the_route() { + setDevices("Speakers A", "Microphone A"); + FakeAudioIO::Config config = loopback(); + config.route = phoneRoute(); + config.recordLatencyStep = 441; + makeWindow(config); + + runCheck(onePunchIn()); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_result.calibrationUsable(), describe(m_result).constData()); + + const LatencyCalibration::Key speaker = + LatencyCalibration::routeKey(phoneRoute(), rate); + QCOMPARE(m_result.key.implementation, QString("oboe")); + QCOMPARE(m_result.key.playbackDevice, + QString("Built-in speaker (Pixel 7)")); + QCOMPARE(m_result.key.recordDevice, + QString("Built-in microphone (Pixel 7)")); + QCOMPARE(m_result.key.rate, rate); + QCOMPARE(m_result.route.outputStreams, phoneRoute().outputStreams); + QCOMPARE(m_result.route.inputStreams, phoneRoute().inputStreams); + + QVERIFY(m_window->storeMeasuredLatency(m_result)); + { + QSettings settings; + LatencyCalibration::Figure figure; + QVERIFY(!LatencyCalibration::load + (settings, key("Speakers A", "Microphone A"), figure)); + QVERIFY(LatencyCalibration::load(settings, speaker, figure)); + QCOMPARE(figure.roundTrip, m_result.calibratedRoundTrip); + QCOMPARE(figure.outputStreams, phoneRoute().outputStreams); + QCOMPARE(figure.inputStreams, phoneRoute().inputStreams); + } + QString line = latencyLine(); + QVERIFY2(line.startsWith("Latency: measured"), qPrintable(line)); + + // The next check's takes are placed with it, the reported input + // latency having moved by 10 ms since + const int reportedBefore = + int(m_window->recordTarget()->getSystemRecordLatency()); + runCheck(onePunchIn()); + if (QTest::currentTestFailed()) return; + QVERIFY2(m_window->recordTarget()->getSystemRecordLatency() - + reportedBefore >= 441, + qPrintable(QString("reported %1 frames, then %2") + .arg(reportedBefore) + .arg(m_window->recordTarget() + ->getSystemRecordLatency()))); + QCOMPARE(int(m_result.takes.size()), 1); + QVERIFY(m_result.takes[0].measured); + QVERIFY2(std::fabs(m_result.summary.medianOffset * rate) <= 4.0, + describe(m_result).constData()); + + // The headset: nothing kept for it + m_window->setFakeRoute(phoneRoute(true)); + m_window->doRecreateAudioIO(); + QCOMPARE(m_window->audioRoute().output.productName, + QString("Headset X")); + line = latencyLine(); + QVERIFY2(line.startsWith("Latency: driver's figure"), qPrintable(line)); + QVERIFY2(!line.contains("out of date"), qPrintable(line)); + QVERIFY(!m_window->forgetLatencyAction()->isEnabled()); + + // The speaker again + m_window->setFakeRoute(phoneRoute()); + m_window->doRecreateAudioIO(); + line = latencyLine(); + QVERIFY2(line.startsWith("Latency: measured"), qPrintable(line)); + QVERIFY(m_window->forgetLatencyAction()->isEnabled()); + + // Opened for playback only, as a phone's device is until its first + // take: the input calibrated with the speaker is taken for it + AudioRoute::Route playbackOnly = phoneRoute(); + playbackOnly.hasInput = false; + playbackOnly.input = AudioRoute::Device(); + playbackOnly.inputStreams = ""; + m_window->setFakeRoute(playbackOnly); + m_window->doRecreateAudioIO(); + QCOMPARE(m_window->latencyKey(rate).recordDevice, + QString("Built-in microphone (Pixel 7)")); + line = latencyLine(); + QVERIFY2(line.startsWith("Latency: measured"), qPrintable(line)); + + // The speaker, shared rather than exclusive: out of date + AudioRoute::Route shared = phoneRoute(); + shared.outputStreams = "AAudio, 44100 Hz, Shared, burst 96"; + m_window->setFakeRoute(shared); + m_window->doRecreateAudioIO(); + line = latencyLine(); + QVERIFY2(line.contains("(the measured one is out of date)"), + qPrintable(line)); + QVERIFY(m_window->forgetLatencyAction()->isEnabled()); + + // Forgotten for the route it is kept for + m_window->forgetLatencyAction()->trigger(); + QSettings settings; + LatencyCalibration::Figure figure; + QVERIFY(!LatencyCalibration::load(settings, speaker, figure)); + } + // Cancel during a take stops it through the Stop path, clears the // override, ends the run once and starts nothing more void check_cancelled_during_a_take() { @@ -1553,6 +1717,73 @@ private slots: "45 ms after the sound, 12 dB quieter", "recorded at 44100 Hz, reference at 44100 Hz" }, { "Kept." }); + if (QTest::currentTestFailed()) return; + + // Copy puts all of it on the clipboard, under when it was made + dialog->copyReport(); + const QString copied = QGuiApplication::clipboard()->text(); + QCOMPARE(copied, dialog->reportText()); + for (QString w : { "Calibrate Audio, ", + "The driver's timing varies from take to take", + "45 ms after the sound, 12 dB quieter", + "output (System Default); input (System Default)" }) { + QVERIFY2(copied.contains(w), qPrintable(w + " not in: " + copied)); + } + } + + // On a phone the instructions name the route the device has open and + // the phone's loopback, and the result's advice is a phone's, with + // the streams the figure is checked against + void calibrate_audio_on_a_phone() { + FakeAudioIO::Config config = loopback(); + config.route = phoneRoute(); + makeWindow(config); + // The device opens with the first file + QCOMPARE(m_window->audioRoute().driver, QString()); + openSong(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->audioRoute().driver, QString("oboe")); + + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Instructions); + QString words = dialog->pageText(); + for (QString w : { "Built-in speaker (Pixel 7)", + "Built-in microphone (Pixel 7)", + "phone's microphone", "off your ears", + "speaker and microphone make the loop", + "microphone of its own", "calibrated on its own", + "driver's figure" }) { + QVERIFY2(words.contains(w), qPrintable(w + " not in: " + words)); + } + QVERIFY2(!words.contains("System Default"), qPrintable(words)); + + AudioCheckResult silent = judgedResult(LatencyCheck::Verdict::NoSignal); + silent.summary.found = 1; + silent.route = phoneRoute(); + silent.key = LatencyCalibration::routeKey(phoneRoute(), rate); + dialog->showResult(silent); + words = dialog->pageText(); + for (QString w : { "could not hear the test sounds", + "Settings, Apps", "cancelling echo", + "output Built-in speaker (Pixel 7); input " + "Built-in microphone (Pixel 7)", + "Streams:", "Exclusive, burst 96, buffer 192" }) { + QVERIFY2(words.contains(w), qPrintable(w + " not in: " + words)); + } + for (QString w : { "Windows", "Hands-Free" }) { + QVERIFY2(!words.contains(w), qPrintable(w + " in: " + words)); + } + + AudioCheckResult fading = judgedResult(LatencyCheck::Verdict::Fading); + fading.summary.fadingDb = 12.0; + fading.route = phoneRoute(); + dialog->showResult(fading); + words = dialog->pageText(); + QVERIFY2(words.contains("echo cancellation or noise suppression"), + qPrintable(words)); + QVERIFY2(!words.contains("Windows"), qPrintable(words)); } // Not while an ordinary take is being recorded: the check records diff --git a/main/test/TestCompactLayout.h b/main/test/TestCompactLayout.h index 242e2ac4..933301ab 100644 --- a/main/test/TestCompactLayout.h +++ b/main/test/TestCompactLayout.h @@ -446,6 +446,17 @@ private slots: } QVERIFY(haveSwitch); + // Playback > Calibrate Audio among them, on show: the device menus + // are hidden, not the check + QAction *calibrate = nullptr; + for (QAction *action: menus) { + for (QAction *item: action->menu()->actions()) { + if (item->text() == "&Calibrate Audio...") calibrate = item; + } + } + QVERIFY(calibrate); + QVERIFY(calibrate->isVisible()); + // A tap opens it (and a timer closes it again) bool checked = false, shown = false; QTimer timer; diff --git a/main/test/TestLatencyCalibration.h b/main/test/TestLatencyCalibration.h index 37223691..4d526672 100644 --- a/main/test/TestLatencyCalibration.h +++ b/main/test/TestLatencyCalibration.h @@ -191,6 +191,166 @@ private slots: QCOMPARE(k.recordDevice, QString()); } + // A phone's route: its driver, and each device by its type and + // product name, never by its id, which a headset gets anew each time + // it is plugged in. A device open for playback only has no input yet. + // Routes stay apart, and a figure comes back under a name that holds + // the characters the settings take for subgroups + void route_key_names_the_devices() { + AudioRoute::Route route; + route.driver = "oboe"; + route.output.id = 3; + route.output.type = 2; + route.output.productName = "Pixel 7"; + route.hasInput = true; + route.input.id = 7; + route.input.type = 15; + route.input.productName = " Pixel 7 "; + route.rate = 48000; + + Key k = LatencyCalibration::routeKey(route, 48000); + QCOMPARE(k.implementation, QString("oboe")); + QCOMPARE(k.playbackDevice, QString("Built-in speaker (Pixel 7)")); + QCOMPARE(k.recordDevice, QString("Built-in microphone (Pixel 7)")); + QCOMPARE(k.rate, 48000.0); + + AudioRoute::Route again = route; + again.output.id = 31; + again.input.id = 32; + Key k2 = LatencyCalibration::routeKey(again, 48000); + QCOMPARE(k2.playbackDevice, k.playbackDevice); + QCOMPARE(k2.recordDevice, k.recordDevice); + + AudioRoute::Route playbackOnly = route; + playbackOnly.hasInput = false; + QCOMPARE(LatencyCalibration::routeKey(playbackOnly, 48000).recordDevice, + QString()); + + AudioRoute::Device unknown; + QCOMPARE(AudioRoute::deviceName(unknown), QString("Unknown device")); + unknown.type = 99; + QCOMPARE(AudioRoute::deviceName(unknown), QString("Device of type 99")); + AudioRoute::Device headset; + headset.type = 8; + headset.productName = "WH-1000XM4 / Ren's | 100%"; + QCOMPARE(AudioRoute::deviceName(headset), + QString("Bluetooth (WH-1000XM4 / Ren's | 100%)")); + + QSettings settings(m_path, QSettings::IniFormat); + AudioRoute::Route bluetooth = route; + bluetooth.output = headset; + const Key kb = LatencyCalibration::routeKey(bluetooth, 48000); + Figure f = figure(0.21, 0.004, 0.003); + f.outputStreams = "AAudio, 48000 Hz, Shared, burst 240"; + f.inputStreams = "AAudio (MMAP), 48000 Hz, Exclusive, burst 96"; + LatencyCalibration::store(settings, k, figure(0.03, 0.005, 0.003)); + LatencyCalibration::store(settings, kb, f); + QCOMPARE(loaded(settings, k), 0.03); + Figure back; + QVERIFY(LatencyCalibration::load(settings, kb, back)); + QCOMPARE(back.roundTrip, 0.21); + QCOMPARE(back.outputStreams, f.outputStreams); + QCOMPARE(back.inputStreams, f.inputStreams); + + // A desktop's figure keeps no streams + Figure desk; + QVERIFY(LatencyCalibration::load(settings, k, desk)); + QCOMPARE(desk.outputStreams, QString()); + QCOMPARE(desk.inputStreams, QString()); + } + + // With the input not open, the one input a figure is kept with for + // the driver and output at the rate, if there is only one; names that + // hold what is encoded in a group's name come back whole + void only_record_device_for_an_output() { + QSettings settings(m_path, QSettings::IniFormat); + const QString odd = QString::fromUtf8("Mic 1/2 | \\ 100%2F (\xc3\xa4)"); + LatencyCalibration::store(settings, key("Speaker", odd, 48000, "oboe"), + figure(0.03, 0.005, 0.003)); + LatencyCalibration::store(settings, + key("Headset", "Headset mic", 48000, "oboe"), + figure(0.04, 0.005, 0.003)); + LatencyCalibration::store(settings, + key("Speaker", "Other", 44100, "oboe"), + figure(0.05, 0.005, 0.003)); + LatencyCalibration::store(settings, key("Speaker", "Desk", 48000), + figure(0.06, 0.005, 0.003)); + + QString record; + QVERIFY(LatencyCalibration::onlyRecordDevice + (settings, key("Speaker", "", 48000, "oboe"), record)); + QCOMPARE(record, odd); + QVERIFY(LatencyCalibration::onlyRecordDevice + (settings, key("Headset", "", 48000, "oboe"), record)); + QCOMPARE(record, QString("Headset mic")); + QVERIFY(LatencyCalibration::onlyRecordDevice + (settings, key("Speaker", "", 44100, "oboe"), record)); + QCOMPARE(record, QString("Other")); + + record = "unchanged"; + QVERIFY(!LatencyCalibration::onlyRecordDevice + (settings, key("Speaker", "", 96000, "oboe"), record)); + QVERIFY(!LatencyCalibration::onlyRecordDevice + (settings, key("Speak", "", 48000, "oboe"), record)); + QVERIFY(!LatencyCalibration::onlyRecordDevice + (settings, key("Bluetooth", "", 48000, "oboe"), record)); + QCOMPARE(record, QString("unchanged")); + + // Two inputs calibrated with the speaker: which one is not known + LatencyCalibration::store(settings, + key("Speaker", "USB mic", 48000, "oboe"), + figure(0.07, 0.005, 0.003)); + QVERIFY(!LatencyCalibration::onlyRecordDevice + (settings, key("Speaker", "", 48000, "oboe"), record)); + QCOMPARE(record, QString("unchanged")); + } + + // A device that describes its streams is stale when it opened either + // otherwise, not when the latencies it reports move: Oboe's moved by + // 4 ms from one take to the next on the phone. A stream it does not + // describe (the input, before it is opened) is not compared. A figure + // kept without streams is stale for such a device; and a device that + // describes none is judged by its latencies as before + void stale_by_the_streams() { + Figure f = figure(0.030, 252 / 48000.0, 134 / 48000.0); + f.outputStreams = "AAudio (MMAP), 48000 Hz, Exclusive, burst 96"; + f.inputStreams = "AAudio (MMAP), 48000 Hz, Exclusive, preset " + "VoicePerformance"; + const double out = 401 / 48000.0; + const double in = 222 / 48000.0; + + QVERIFY(!LatencyCalibration::isStale(f, out, in, f.outputStreams, + f.inputStreams)); + QVERIFY(!LatencyCalibration::isStale(f, out, 0.0, f.outputStreams, "")); + QVERIFY(LatencyCalibration::isStale + (f, out, in, "AAudio, 48000 Hz, Shared, burst 96", + f.inputStreams)); + QVERIFY(LatencyCalibration::isStale + (f, out, in, f.outputStreams, "AAudio, 48000 Hz, Shared")); + QVERIFY(LatencyCalibration::isStale + (f, out, in, "", "AAudio, 48000 Hz, Shared")); + + Figure plain = figure(0.030, 252 / 48000.0, 134 / 48000.0); + QVERIFY(LatencyCalibration::isStale(plain, 252 / 48000.0, + 134 / 48000.0, f.outputStreams, + f.inputStreams)); + + // No streams now: the reported pair, to the millisecond + QVERIFY(LatencyCalibration::isStale(f, out, in)); + QVERIFY(!LatencyCalibration::isStale(f, 252 / 48000.0, 134 / 48000.0)); + + InUse measured = LatencyCalibration::roundTripInUse + (&f, out, in, f.outputStreams, f.inputStreams); + QVERIFY(measured.source == Source::Measured); + QCOMPARE(measured.roundTrip, 0.030); + QCOMPARE(measured.reportedOutput, out); + InUse other = LatencyCalibration::roundTripInUse + (&f, out, in, "OpenSLES, 48000 Hz", f.inputStreams); + QVERIFY(other.source == Source::Reported); + QVERIFY(other.stale); + QCOMPARE(other.roundTrip, out + in); + } + // Stale when either reported latency has moved by more than the // tolerance, either way; not when it has moved by less void stale_beyond_the_tolerance() { diff --git a/main/test/TestLatencyCheck.h b/main/test/TestLatencyCheck.h index c07c15dd..18df6bdf 100644 --- a/main/test/TestLatencyCheck.h +++ b/main/test/TestLatencyCheck.h @@ -1067,6 +1067,54 @@ private slots: QVERIFY(LatencyCheck::punchInsFor(layout, 0, 2).empty()); } + // Punch-ins placed with different round trips, as on a phone, whose + // reported latencies move from one stream start to the next (Oboe + // measures them each time), through a path whose round trip stays + // 0.12 s. Each lands late by what its round trip fell short: 20, 12, + // 17 and 9 ms. Counted as if placed with the first one's, they agree: + // Ok, and the round trip calibrated from the first one's is the + // path's. Not told how they were placed, the placing looks like the + // device moving: 11 ms apart, Unsteady, and 5.5 ms short + void judge_punch_ins_placed_differently() { + const LatencyCheck::Layout layout = LatencyCheck::calibrationLayout(); + const samples_t reference = LatencyCheck::generate(layout); + const double roundTrip = 0.120; + const double used[] = { 0.100, 0.108, 0.103, 0.111 }; + + punchins_t punchIns = fourPunchIns(); + std::vector shifts; + for (int p = 0; p < 4; ++p) { + shifts.push_back(framesOf(roundTrip - used[p])); + } + const samples_t take = spliced(reference, kRate, punchIns, shifts); + + LatencyCheck::TakeSummary s = judge(layout, take, kRate, punchIns); + QCOMPARE(s.found, 7); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Unsteady, + describe(s).constData()); + QVERIFY2(std::fabs(LatencyCheck::calibratedRoundTrip + (used[0], s.medianOffset) - roundTrip) > 0.005, + describe(s).constData()); + + for (int p = 0; p < 4; ++p) punchIns[p].placedWith = used[p]; + s = judge(layout, take, kRate, punchIns); + QCOMPARE(s.found, 7); + QVERIFY2(s.verdict == LatencyCheck::Verdict::Ok, describe(s).constData()); + QVERIFY2(s.spread < 1.5 / kRate && s.slopeResidual < 1.5 / kRate, + describe(s).constData()); + QVERIFY2(std::fabs(LatencyCheck::calibratedRoundTrip + (used[0], s.medianOffset) - roundTrip) < 1.0 / kRate, + describe(s).constData()); + + // Where each landed is kept as it was + for (int p = 0; p < 4; ++p) { + QVERIFY2(std::fabs(s.punchIns[p].medianOffset - + shifts[p] / kRate) < 1e-9, + describe(s).constData()); + QCOMPARE(s.punchIns[p].range.placedWith, used[p]); + } + } + // The calibration's arithmetic. A take that landed late was placed // with too small a round trip, and the new one is larger by the // offset; early, smaller. From takes placed with 0.2 and 0.31 s diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index 6a456a55..c8eeefbd 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -55,6 +55,13 @@ class TestMainWindow : public MainWindow FakeAudioIO *fake() { return dynamic_cast(m_audioIO); } + // The route the fake reports from the next time it is opened, and the + // device opened again, as a phone's is when its route changes + void setFakeRoute(const AudioRoute::Route &route) { + m_fakeConfig.route = route; + } + void doRecreateAudioIO() { recreateAudioIO(); } + void doRecord() { record(); } void doPlay() { play(); } // and again to stop void doAnalyseNow() { analyseNow(); } diff --git a/meson.build b/meson.build index b8337fb3..2325bd63 100644 --- a/meson.build +++ b/meson.build @@ -1186,6 +1186,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/AndroidFiles.cpp', + 'main/AudioRoute.cpp', 'main/Coverage.cpp', 'main/DecodedPcm.cpp', 'main/LogFile.cpp', From 8f1cfb3986a8fe4e78daa54687a60d58b279dc75 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:46:42 +0000 Subject: [PATCH 226/275] feat: playback > audio driver and audio latency, with mme named by default On Windows the fork offers MME, DirectSound and WASAPI; Tony shows them, and the latency asked of each (10 to 200 ms, 200 as every stream asked before), when there are two or more. A choice reopens the device; both are greyed out during a take and a check. Where no driver is named, MME is named before the first device opens, with the devices chosen before carried over: with several implementations the device menus would otherwise have none to list. DevChecks.txt and the Calibrate Audio dialog name the driver, so runs on two drivers can be told apart. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/audio-drivers-work-orders.md | 8 + docs/audio-drivers.md | 10 +- docs/calibrate-audio.md | 26 ++- docs/recording.md | 16 ++ docs/testing.md | 27 ++- main/AudioDriverMenus.cpp | 155 +++++++++++++ main/AudioDriverMenus.h | 92 ++++++++ main/AudioDriverSettings.cpp | 120 ++++++++++ main/AudioDriverSettings.h | 86 +++++++ main/CalibrateAudioDialog.cpp | 6 + main/MainWindow.cpp | 94 ++++++++ main/MainWindow.h | 26 +++ main/dev/DevChecks.cpp | 10 + main/test/TestAudioCheck.h | 336 +++++++++++++++++++++++++++- main/test/TestAudioDriverSettings.h | 143 ++++++++++++ main/test/TestDevChecks.h | 7 +- main/test/TestMainWindow.h | 34 ++- main/test/tony-core-test.cpp | 7 + meson.build | 4 + 19 files changed, 1177 insertions(+), 30 deletions(-) create mode 100644 main/AudioDriverMenus.cpp create mode 100644 main/AudioDriverMenus.h create mode 100644 main/AudioDriverSettings.cpp create mode 100644 main/AudioDriverSettings.h create mode 100644 main/test/TestAudioDriverSettings.h diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md index fa6825f2..fe3b340d 100644 --- a/docs/audio-drivers-work-orders.md +++ b/docs/audio-drivers-work-orders.md @@ -173,3 +173,11 @@ session's rate. A 48 kHz check measures about 1.1 ms over the fake's delay: bqau **W2** (lead). The fork as the spec's §4; pinned. Tony's build unchanged on Linux; the suites green against the fork. + +**W3** (agent; lead reviewed and committed). `AudioDriverSettings` (core) and +`AudioDriverMenus` (app); `MainWindow::createAudioIO()` on desktop names the default and +applies the latency, then `openAudioIO()`, which the tests override instead of +`createAudioIO()`. The first device opens lazily (first file, first take, recreate), never +in the constructor. Seen failing: the default, the per-driver latency (twice), the +greying. Not tested: that playback stops on a choice. For W4: the manual checklist has no +items for the menus yet; calibrate-audio.md §11's test list. diff --git a/docs/audio-drivers.md b/docs/audio-drivers.md index 1b78f1d1..1157dcb5 100644 --- a/docs/audio-drivers.md +++ b/docs/audio-drivers.md @@ -32,7 +32,7 @@ to be merged back into it). What is built is marked **Done** in §6; the reasons | **A driver is a bqaudioio implementation**: `mme`, `directsound`, `wasapi`, each PortAudio restricted to that host API; `port` stays as it is (all host APIs) | lead | Tony's device menus, the saved devices (`audio-*-device-`), svapp's `createAudioIO()` and the stored round trip (`LatencyCalibration::Key::implementation`) are already per implementation: no svapp change | | WASAPI in **shared** mode, with `paWinWasapiAutoConvert` on both sides | lead | The input's and the output's mixers can run at different rates, and the stream opens at the output's; shared mode leaves other programs' sound alone. Exclusive mode is not in this project | | A device at 48 kHz needs nothing more for placement | `feat/tonyandroid` A1 | The recording is resampled to the reference's rate before the splice, and `TakeTiming` converts device frames (§3). Calibrate Audio's "rate mismatch" verdict is out of date and goes (W1) | -| `suggestedLatency` settable per driver | lead; the Tony-side choice is open (§7) | At 0.2 s WASAPI would keep MME's buffers and gain nothing | +| `suggestedLatency` settable per driver, chosen in a Playback > Audio Latency submenu (10, 20, 50, 100, 200 ms; 200 when unset) | lead, 2026-09-26 | At 0.2 s WASAPI would keep MME's buffers and gain nothing; a choice lets W5 compare | ## 3. Facts checked @@ -126,11 +126,15 @@ Each leaves the tree building and all three suites green, committed and pushed t the fork, and `container-setup.sh` moves a checkout made from the mirror over to it. Compiled for Linux in Tony's build and cross-compiled for Windows; run on no device yet (the container has none). +- **W3 Done.** Playback > Audio Driver and Audio Latency (shown with two drivers or more, + so on Windows only), greyed out during a take and a check; MME named by default before + the first device opens, with the devices chosen before carried over. `DevChecks.txt` + and the Calibrate Audio dialog name the driver (and the report the latency). A figure + measured before is kept under no driver's name, so it is not carried over: calibrate + again on each driver. ## 7. Open -- **How the latency is chosen in Tony**: a fixed smaller figure for WASAPI, or a submenu - of a few (10, 20, 50, 100, 200 ms). To settle before W3. - WDM-KS (in PortAudio's build too) and WASAPI's exclusive mode would be lower still, but take the device from every other program; not in this project. - Keeping the stream running between takes would remove the restart from the take path diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index f1d03683..3600f126 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -22,8 +22,9 @@ Audacity's measurements) was a separate report, not kept in the repository. a measured figure the round trip is the sum of the output and input latency the device reports. The start gap is measured and right; the reported pair comes from `Pa_GetStreamInfo()`, which on MME, DirectSound and WASAPI is buffer sizes only. - Audacity measured it off by −5 to +155 ms. bqaudioio opens the stream with - `suggestedLatency = 0.2` on both sides. + Audacity measured it off by −5 to +155 ms. bqaudioio opens the stream with the + `suggestedLatency` chosen for the driver under Playback > Audio Latency on both sides, + 0.2 s unless another is chosen ([recording.md](recording.md#latency)). - **The device need not run at the reference's rate.** It opens at PortAudio's default rate: for "(System Default)" through MME most likely 44.1 kHz, for a device whose name exists only under WASAPI or WDM-KS often 48 kHz. A recording is converted to the @@ -54,9 +55,9 @@ meanwhile. Closing the dialog while its check runs cancels the check, since noth would show how the run ended. Three pages: 1. **Instructions:** one earcup against the microphone, off your ears; a moderate volume - and a quiet room; the output and input devices and the latency in use; how long it - takes. In development builds the checkbox **Run the dev checks after calibrating**, on - every time the page is shown and not remembered. **Start**. + and a quiet room; the driver, the output and input devices and the latency in use; how + long it takes. In development builds the checkbox **Run the dev checks after + calibrating**, on every time the page is shown and not remembered. **Start**. 2. **Progress:** the step and the punch-in, a bar and the time left (the recording to come, plus a guess of 3 s for each analysis), **Cancel**. During the dev checks, their stage. 3. **Result** (§4): **Use this latency** (only when the calibration is usable), **Check @@ -176,7 +177,7 @@ round trip measured (not for NoSignal) against the driver's, output plus input; takes were placed with (measured before, or the driver's figure); where each punch-in landed (+ is late); the spread; sweeps found of those judged; both rates ("recorded at 48000 Hz, converted to the reference's 44100 Hz" where they differ); the input peak in -dBFS; the echo; the devices. A failed run shows why it ended instead. +dBFS; the echo; the driver and the devices. A failed run shows why it ended instead. **Use this latency** keeps the calibrated round trip for the devices the check started on, not for those the Preferences name when it is pressed (the result stays on show for as long @@ -336,9 +337,10 @@ song was left out (`Options::longSeconds` of 0, for tests only). `DevChecks.txt` in `TONY_TEST_LOG_DIR` if that is set, else in the application data directory (`%APPDATA%\sonic-visualiser\Tony` on Windows), written over by each run. Its -header: the date, the output and input devices, the audio drivers built in, the playback and -record latencies the device reports (frames and ms), the round trip for the run, the -scratch folder, the session saved, and why the run ended early if it did. Then each check +header: the date, the output and input devices, the audio driver they were opened through +and the latency asked of it, the audio drivers built in, the playback and record latencies +the device reports (frames and ms), the round trip for the run, the scratch folder, the +session saved, and why the run ended early if it did. Then each check under "Item N", with its verdict, message and numbers, and last `Totals: N passed, N failed, N measured, N skipped`, as the suites end. @@ -446,9 +448,9 @@ below). In order: API, WASAPI's automatic rate conversion, and a `suggestedLatency` that can be set ([forks.md](forks.md#bqaudioio)). Done; the plan from here on is in [audio-drivers.md](audio-drivers.md). -3. **A driver type in Tony**: MME, DirectSound or WASAPI; the device menus list that type's - devices only (today every host API's are listed, and `getDeviceIndex()` takes the first - name that matches, which is MME's); the stored round trip kept per type. +3. **A driver type in Tony**: MME, DirectSound or WASAPI, and the latency asked of it, + under Playback > Audio Driver and Audio Latency; the device menus list that type's + devices only; the stored round trip kept per type. Done. 4. **Measure** with Calibrate Audio and a dev run on each type, on the user's PC. **MME stays the default** until such a run shows WASAPI, or another type, better. diff --git a/docs/recording.md b/docs/recording.md index e446606a..f1944122 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -145,6 +145,22 @@ L is the **round trip** plus the **start gap**, both in frames of the recording. - `m_takeLatency` keeps what the last take was placed with: the round trip, whether it was measured, the reported pair in seconds, the recording's rate, and the start gap and whether it was measured. The audio check reads it for each of its takes. +- **The latency asked for** is not part of L: it is what bqaudioio asks the driver for on + each side (PortAudio's `suggestedLatency`), which sets the driver's buffers, and the + round trip with them. It is chosen per driver under Playback > Audio Latency (10 to + 200 ms, `Preferences/audio-latency-`; 200 ms where none is chosen, what every + stream asked for before there was a choice), and `MainWindow::createAudioIO()` hands it + to bqaudioio before each device is opened; choosing one opens the device again. A + measured round trip is kept per driver, not per latency: a figure measured at another + latency is told apart only by the staleness fingerprint + ([calibrate-audio.md](calibrate-audio.md), §5), when the reported pair moves with the + buffers. +- **The driver** (Playback > Audio Driver: MME, DirectSound, WASAPI, shown where more than + one is built in, which is on Windows) is `Preferences/audio-target`. Where none is named + and MME is built in, `MainWindow::createAudioIO()` names MME before the first device is + opened, and the Playback menu before it shows the device menus, carrying the devices + chosen before over to MME's keys: bqaudioio lists no devices for no driver when it has + several. Choosing a driver or a latency is shut during a take and while a check runs. ## Pre-roll and Record into Selection diff --git a/docs/testing.md b/docs/testing.md index ac886ffe..5c27f21e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -6,7 +6,7 @@ commands are in [AGENTS.md](../AGENTS.md). | Executable | Links | Suites | Time | | --- | --- | --- | --- | -| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestTakeDiff`, `TestLiveDotsFeed`, `TestRunSuite` | seconds | +| `test-tony-core` | `tony_core`, svcore, pyin's `YinUtil.cpp` as the YIN reference. `QCoreApplication`, no GUI. | `TestRealtimeYin`, `TestRealtimePitchTracker`, `TestLatencyShift`, `TestCoverage`, `TestTakeAudio`, `TestTakeEvents`, `TestSingingTakes`, `TestTakesFile`, `TestTakeTiming`, `TestLyrics`, `TestLyricsTtml`, `TestLyricsEdit`, `TestLatencyCheck`, `TestLatencyCalibration`, `TestAudioDriverSettings`, `TestTakeDiff`, `TestLiveDotsFeed`, `TestRunSuite` | seconds | | `test-tony-app` | `tony_app` + `tony_core`, a real `MainWindow` on the offscreen platform, the real pYIN plugin, `FakeAudioIO`. | `TestSingingDocument`, `TestViewCache`, `TestSingingAnalysis`, `TestLyricsLayer`, `TestRecordWorkflow`, `TestUiChecks`, `TestAudioCheck` | about 9 minutes in one process, a minute and a half in eight (measured 2026-09-26 on Linux), nearly all of it `TestRecordWorkflow`, `TestAudioCheck` and `TestUiChecks`: takes are recorded in real time | | `test-tony-dev` | as `test-tony-app`; built only where the development checks are (any build type but `release`, `TONY_DEV_CHECKS`) | `TestDevChecks` | about 4 minutes in one process, a little over one in eight (2026-09-26, Linux): each test records a dev run's takes, or part of them, in real time | @@ -133,10 +133,14 @@ Windows path would start an escape in the C string. `TestDevChecks` in `test-tony-dev`): subclass of `MainWindow` that exposes protected operations as `doRecord()`, `doSwitchToTake()`, `seekTo()`, `selectRange()` and so on, and the audio check's parts (`audioCheck()`, `devChecks()`, `takeLatency()`, the - Playback menu's actions). It installs the fake device through `createAudioIO()`, or no - device at all when made with `installDevice` false, and **answers dialogs - through virtual seams**: `confirmRecordingOverTake()`, `confirmDeleteTake()`, - `askForTakeName()`, `askForLyricsFile()`, `askForLyricsExportFile()` (which also keeps + Playback menu's actions). It installs the fake device through `openAudioIO()`, which + `MainWindow::createAudioIO()` calls once it has named the driver and applied its + latency, or no device at all when made with `installDevice` false, and keeps what the + Preferences named for the last device opened (`audioIOOpenedFor()`). The drivers are + the ones a test gives with `setAudioImplementations()`, none by default, whatever the + platform has. It **answers dialogs through virtual seams**: `confirmRecordingOverTake()`, + `confirmDeleteTake()`, `askForTakeName()`, `askForLyricsFile()`, + `askForLyricsExportFile()` (which also keeps the path it was offered), each with a `set...Answer()` and a counter of questions asked. `askForLyricsWordText()` takes a queue of answers (`answerWordText()`, `cancelWordText()`; none left is Cancel) and can run something while the question is @@ -223,8 +227,12 @@ The rules of the edits themselves are tested without a window, in `TestLyricsEdi `setApplicationSessionExtension("ton")` and the record directory. - `QSignalSpy` connects directly; for a signal from another thread use a receiver object on the test thread. -- In `TestMainWindow` override only `createAudioIO()`: `~MainWindowBase` calls - `deleteAudioIO()` non-virtually. +- In `TestMainWindow` override only `openAudioIO()`: `~MainWindowBase` calls + `deleteAudioIO()` non-virtually, and `MainWindow::createAudioIO()` names the driver and + applies its latency before it calls `openAudioIO()`. +- A hidden `QAction` reads as disabled, whatever it was set to: a test of when the Audio + Driver and Audio Latency menus are greyed out gives the window drivers first + (`setAudioImplementations()`, `doRebuildAudioDriverMenus()`), so that they are shown. - The app and dev mains draw text without sub-pixel anti-aliasing. Ubuntu's fontconfig asks for it and Qt 6.4 follows it: the scale's labels then have orange fringes, which `TestUiChecks` takes for live dots. A new main that shows a window needs the same. @@ -343,8 +351,9 @@ What they cover is in [calibrate-audio.md](calibrate-audio.md), section 11. Both follow `TestRecordWorkflow`'s (a `TestMainWindow`, the dialog watchdog, the user's toggles reset in `init()`), and `TestDevChecks`' is a copy of `TestAudioCheck`'s, not shared: each class keeps its own. `cleanup()` also removes any round trip a test stored, which would -place the next test's takes. How they are built, and what to keep in mind when adding to -them: +place the next test's takes, and `TestAudioCheck`'s the driver, the devices kept per +driver and the latencies a test named. How they are built, and what to keep in mind when +adding to them: - **The loopback fake.** Both record through `FakeAudioIO` with `loopback` on (their `loopback()`). The device reports 2 × 4096 frames out and 4096 in, and the true round diff --git a/main/AudioDriverMenus.cpp b/main/AudioDriverMenus.cpp new file mode 100644 index 00000000..c5bc1a58 --- /dev/null +++ b/main/AudioDriverMenus.cpp @@ -0,0 +1,155 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AudioDriverMenus.h" +#include "AudioDriverSettings.h" + +#include + +#include +#include +#include +#include + +#include + +AudioDriverMenus::AudioDriverMenus(QMenu *menu, + std::function implementations, + QObject *parent) : + QObject(parent), + m_implementations(implementations), + m_appliedLatency(0.0) +{ + m_driverMenu = menu->addMenu(tr("Audio Dri&ver")); + m_driverMenu->setStatusTip + (tr("Choose which audio driver Tony plays and records through")); + m_driverGroup = new QActionGroup(this); + m_driverGroup->setExclusive(true); + + m_latencyMenu = menu->addMenu(tr("Audio &Latency")); + m_latencyMenu->setStatusTip + (tr("Choose how much latency Tony asks the audio driver for: less " + "answers sooner, more is safer from dropouts")); + m_latencyGroup = new QActionGroup(this); + m_latencyGroup->setExclusive(true); + + // The ticks follow the Preferences, which a choice in the other menu, + // or the driver named by default, may have changed since + for (QMenu *m : { m_driverMenu, m_latencyMenu }) { + connect(m, &QMenu::aboutToShow, this, &AudioDriverMenus::rebuild); + } + + rebuild(); +} + +AudioDriverMenus::~AudioDriverMenus() +{ +} + +void +AudioDriverMenus::rebuild() +{ + const QStringList drivers = + AudioDriverSettings::drivers(m_implementations()); + + for (QActionGroup *group : { m_driverGroup, m_latencyGroup }) { + for (QAction *a : group->actions()) group->removeAction(a); + } + m_driverMenu->clear(); + m_latencyMenu->clear(); + + // One driver is no choice: the menus are for Windows, where there + // are three, and elsewhere the device menus are all there is + const bool shown = (drivers.size() >= 2); + m_driverMenu->menuAction()->setVisible(shown); + m_latencyMenu->menuAction()->setVisible(shown); + if (!shown) return; + + QSettings settings; + const QString current = + AudioDriverSettings::currentImplementation(settings); + const double latency = AudioDriverSettings::latency(settings, current); + + for (const QString &name : drivers) { + QAction *action = m_driverMenu->addAction(driverName(name)); + action->setCheckable(true); + action->setChecked(name == current); + action->setData(name); + m_driverGroup->addAction(action); + connect(action, &QAction::triggered, + this, [this, name]() { driverTriggered(name); }); + } + + for (double seconds : AudioDriverSettings::latencyChoices()) { + QAction *action = m_latencyMenu->addAction + (tr("%1 ms").arg(int(std::lround(seconds * 1000.0)))); + action->setCheckable(true); + action->setChecked(AudioDriverSettings::sameLatency(seconds, latency)); + action->setData(seconds); + m_latencyGroup->addAction(action); + connect(action, &QAction::triggered, + this, [this, seconds]() { latencyTriggered(seconds); }); + } +} + +void +AudioDriverMenus::setEnabled(bool enabled) +{ + m_driverMenu->menuAction()->setEnabled(enabled); + m_latencyMenu->menuAction()->setEnabled(enabled); +} + +double +AudioDriverMenus::applyLatency() +{ + QSettings settings; + const double seconds = AudioDriverSettings::latency + (settings, AudioDriverSettings::currentImplementation(settings)); + breakfastquay::AudioFactory::setSuggestedLatency(seconds); + m_appliedLatency = seconds; + return seconds; +} + +QString +AudioDriverMenus::driverName(QString implementation) +{ + return QString::fromStdString + (breakfastquay::AudioFactory::getImplementationDescription + (implementation.toStdString())); +} + +void +AudioDriverMenus::driverTriggered(QString implementation) +{ + QSettings settings; + if (AudioDriverSettings::currentImplementation(settings) == + implementation) { + return; + } + AudioDriverSettings::setCurrentImplementation(settings, implementation); + emit driverChosen(implementation); +} + +void +AudioDriverMenus::latencyTriggered(double seconds) +{ + QSettings settings; + const QString current = + AudioDriverSettings::currentImplementation(settings); + if (AudioDriverSettings::sameLatency + (AudioDriverSettings::latency(settings, current), seconds)) { + return; + } + AudioDriverSettings::setLatency(settings, current, seconds); + emit latencyChosen(seconds); +} diff --git a/main/AudioDriverMenus.h b/main/AudioDriverMenus.h new file mode 100644 index 00000000..c7f4ee8a --- /dev/null +++ b/main/AudioDriverMenus.h @@ -0,0 +1,92 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_AUDIO_DRIVER_MENUS_H +#define TONY_AUDIO_DRIVER_MENUS_H + +#include +#include +#include + +#include + +class QMenu; +class QActionGroup; + +/** + * Playback > Audio Driver and Audio Latency: the driver the device is + * opened through (MME, DirectSound, WASAPI) and the latency asked of it, + * each kept in the Preferences (AudioDriverSettings). Both are shown + * only where more than one driver is built in, which is on Windows. + * + * A choice is written to the Preferences here, and then said with + * driverChosen() or latencyChosen(): MainWindow stops what is playing + * and opens the device again, which is where the choice takes effect. + * Before it opens one, it hands the latency chosen for the driver to + * bqaudioio with applyLatency(). + */ +class AudioDriverMenus : public QObject +{ + Q_OBJECT + +public: + /** + * Add the two submenus to the end of the menu given. The list gives + * the implementations bqaudioio has, whenever the menus are built. + */ + AudioDriverMenus(QMenu *menu, std::function implementations, + QObject *parent = nullptr); + virtual ~AudioDriverMenus(); + + QMenu *driverMenu() const { return m_driverMenu; } + QMenu *latencyMenu() const { return m_latencyMenu; } + + /** + * Build both again from the implementations there are and the + * Preferences: shown if there are two drivers or more, with the + * driver named and its latency ticked. Done as either opens. + */ + void rebuild(); + + /// Not while a take is being recorded or an audio check runs + void setEnabled(bool enabled); + + /** + * Hand bqaudioio the latency chosen for the driver the Preferences + * name, for the streams opened from now on. Returns it. + */ + double applyLatency(); + + /// What applyLatency() last handed over, in seconds; 0 before it has + double appliedLatency() const { return m_appliedLatency; } + + /// A driver's name as the user knows it: "WASAPI" for "wasapi" + static QString driverName(QString implementation); + +signals: + void driverChosen(QString implementation); + void latencyChosen(double seconds); + +private: + QMenu *m_driverMenu; + QActionGroup *m_driverGroup; + QMenu *m_latencyMenu; + QActionGroup *m_latencyGroup; + std::function m_implementations; + double m_appliedLatency; + + void driverTriggered(QString implementation); + void latencyTriggered(double seconds); +}; + +#endif diff --git a/main/AudioDriverSettings.cpp b/main/AudioDriverSettings.cpp new file mode 100644 index 00000000..96259af3 --- /dev/null +++ b/main/AudioDriverSettings.cpp @@ -0,0 +1,120 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AudioDriverSettings.h" + +#include + +#include + +namespace AudioDriverSettings { +namespace { + +const char *const preferencesGroup = "Preferences"; +const char *const targetKey = "audio-target"; + +} + +QStringList +drivers(const QStringList &implementations) +{ + QStringList result; + for (QString name : { QString("mme"), QString("directsound"), + QString("wasapi") }) { + if (implementations.contains(name)) result << name; + } + return result; +} + +std::vector +latencyChoices() +{ + return { 0.010, 0.020, 0.050, 0.100, 0.200 }; +} + +QString +currentImplementation(QSettings &settings) +{ + settings.beginGroup(preferencesGroup); + QString implementation = settings.value(targetKey, "").toString(); + settings.endGroup(); + if (implementation == "auto") return ""; + return implementation; +} + +void +setCurrentImplementation(QSettings &settings, QString implementation) +{ + settings.beginGroup(preferencesGroup); + settings.setValue(targetKey, implementation); + settings.endGroup(); +} + +QString +latencySettingKey(QString implementation) +{ + return "audio-latency-" + implementation; +} + +double +latency(QSettings &settings, QString implementation) +{ + if (implementation == "") return kDefaultLatency; + settings.beginGroup(preferencesGroup); + bool ok = false; + // As text, as LatencyCalibration keeps its figures: every settings + // format keeps it whole + double seconds = settings.value(latencySettingKey(implementation), "") + .toString().toDouble(&ok); + settings.endGroup(); + if (!ok || !(seconds > 0.0)) return kDefaultLatency; + return seconds; +} + +void +setLatency(QSettings &settings, QString implementation, double seconds) +{ + if (implementation == "") return; + settings.beginGroup(preferencesGroup); + settings.setValue(latencySettingKey(implementation), + QString::number(seconds, 'g', 17)); + settings.endGroup(); +} + +bool +sameLatency(double a, double b) +{ + // The choices are whole milliseconds apart + return std::fabs(a - b) < 0.0005; +} + +bool +nameDefaultDriver(QSettings &settings, const QStringList &implementations) +{ + if (currentImplementation(settings) != "") return false; + if (!implementations.contains(kDefaultDriver)) return false; + + settings.beginGroup(preferencesGroup); + settings.setValue(targetKey, QString(kDefaultDriver)); + const QString suffix = QString("-") + kDefaultDriver; + for (QString key : { QString("audio-playback-device"), + QString("audio-record-device") }) { + if (settings.contains(key) && !settings.contains(key + suffix)) { + settings.setValue(key + suffix, settings.value(key)); + } + } + settings.endGroup(); + return true; +} + +} diff --git a/main/AudioDriverSettings.h b/main/AudioDriverSettings.h new file mode 100644 index 00000000..2cc107f7 --- /dev/null +++ b/main/AudioDriverSettings.h @@ -0,0 +1,86 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_AUDIO_DRIVER_SETTINGS_H +#define TONY_AUDIO_DRIVER_SETTINGS_H + +#include +#include + +#include + +class QSettings; + +/** + * The audio driver Tony opens its device through, and the latency it + * asks that driver for, as the Preferences keep them. + * + * A driver is one of bqaudioio's implementations that is PortAudio + * restricted to one Windows host API: "mme", "directsound", "wasapi". + * The driver is "audio-target", which MainWindowBase::createAudioIO() + * reads, and with it the devices, which are kept per driver + * ("audio-playback-device-" and "audio-record-device-"). + * The latency is kept per driver as well, in seconds, as + * "audio-latency-". All in the group "Preferences". + */ +namespace AudioDriverSettings +{ + /// The driver named when none is, where it is built in: the one every + /// stream went through before there was a choice + constexpr const char *kDefaultDriver = "mme"; + + /// What a driver asks for when no latency has been chosen for it: + /// what every stream asked for before there was a choice + constexpr double kDefaultLatency = 0.2; + + /// Those of the implementations given that are drivers, in the order + /// they are offered in + QStringList drivers(const QStringList &implementations); + + /// The latencies offered, in seconds, shortest first + std::vector latencyChoices(); + + /// The implementation the Preferences name, "" when none is ("auto" + /// counts as none, as MainWindowBase::createAudioIO() has it) + QString currentImplementation(QSettings &settings); + + /// Name the implementation in the Preferences. The devices chosen for + /// it are then the ones read + void setCurrentImplementation(QSettings &settings, QString implementation); + + /// The key of the latency chosen for an implementation, in the group + /// "Preferences" + QString latencySettingKey(QString implementation); + + /// The latency chosen for an implementation, in seconds, or + /// kDefaultLatency where none has been (or none is named) + double latency(QSettings &settings, QString implementation); + + void setLatency(QSettings &settings, QString implementation, + double seconds); + + /// Whether two latencies are the same choice + bool sameLatency(double a, double b); + + /** + * Where no implementation is named and kDefaultDriver is among the + * ones given, name it, and give it the devices chosen before there + * were drivers (the keys without a suffix) where none are chosen for + * it. Once it is named this does nothing again. True if it named + * the driver. + */ + bool nameDefaultDriver(QSettings &settings, + const QStringList &implementations); +} + +#endif diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index b8ed509b..e4e3f64f 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -14,6 +14,7 @@ #include "CalibrateAudioDialog.h" +#include "AudioDriverMenus.h" #include "MainWindow.h" #include @@ -567,6 +568,9 @@ CalibrateAudioDialog::instructionsHtml() const tr("Set a moderate volume, and keep the room quiet.") + ""; html += ""; + html += ""; html += ""; html += "
" + tr("Driver:") + "" + + AudioDriverMenus::driverName(devices.implementation).toHtmlEscaped() + + "
" + tr("Output:") + "" + deviceName(devices.playbackDevice).toHtmlEscaped() + "
" + tr("Input:") + "" + @@ -747,6 +751,8 @@ CalibrateAudioDialog::calibrationHtml() const .arg(QLocale().toString(std::fabs(s.echo.levelDb), 'f', 0)) .arg(s.echo.levelDb <= 0.0 ? tr("quieter") : tr("louder")) : tr("none heard")); + html += row(tr("Driver:"), + AudioDriverMenus::driverName(r.key.implementation)); html += row(tr("Devices:"), tr("output %1; input %2") .arg(deviceName(r.key.playbackDevice)) diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index b5951099..6a76fe1c 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -21,6 +21,8 @@ #include "CompactLayout.h" #include "PlotSize.h" #include "AudioCheckRunner.h" +#include "AudioDriverMenus.h" +#include "AudioDriverSettings.h" #include "CalibrateAudioDialog.h" #include "LatencyUtils.h" #include "Lyrics.h" @@ -228,6 +230,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_audioDeviceGroup(0), m_audioInputDeviceMenu(0), m_audioInputDeviceGroup(0), + m_audioDriverMenus(nullptr), m_deleteSelectedAction(0), m_ffwdAction(0), m_rwdAction(0), @@ -1682,6 +1685,73 @@ MainWindow::audioDeviceSelected(QAction *action) recreateAudioIO(); } +QStringList +MainWindow::audioImplementationNames() const +{ + QStringList names; + for (const std::string &name : + breakfastquay::AudioFactory::getImplementationNames()) { + names << QString::fromStdString(name); + } + return names; +} + +void +MainWindow::nameDefaultAudioDriver() +{ + QSettings settings; + if (AudioDriverSettings::nameDefaultDriver + (settings, audioImplementationNames())) { + cerr << "MainWindow::nameDefaultAudioDriver: no audio driver was " + << "named; naming " << AudioDriverSettings::kDefaultDriver + << endl; + } +} + +void +MainWindow::audioDriverChosen(QString) +{ + // As for another device: the driver opens the devices it names, + // which may record at another rate. The latency chosen for it is + // applied as the device is opened + if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + m_lastRecordingRate = 0; + recreateAudioIO(); +} + +void +MainWindow::audioLatencyChosen(double) +{ + if (m_playSource && m_playSource->isPlaying()) { + stop(); + } + recreateAudioIO(); +} + +#ifndef Q_OS_ANDROID +void +MainWindow::createAudioIO() +{ + if (m_playTarget || m_audioIO) return; + + // The first device is opened lazily, with the first file or the + // first take, and svapp opens it for the driver the Preferences + // name, so a driver has to be named by then + nameDefaultAudioDriver(); + if (m_audioDriverMenus) m_audioDriverMenus->applyLatency(); + + openAudioIO(); +} + +void +MainWindow::openAudioIO() +{ + MainWindowBase::createAudioIO(); +} +#endif + void MainWindow::setupToolbars() { @@ -1859,6 +1929,20 @@ MainWindow::setupToolbars() menu->addAction(recordAction); menu->addSeparator(); + // The driver and the latency asked of it, before the devices, which + // are the driver's own + m_audioDriverMenus = new AudioDriverMenus + (menu, [this]() { return audioImplementationNames(); }, this); + connect(m_audioDriverMenus, &AudioDriverMenus::driverChosen, + this, &MainWindow::audioDriverChosen); + connect(m_audioDriverMenus, &AudioDriverMenus::latencyChosen, + this, &MainWindow::audioLatencyChosen); + + // Before anything in this menu reads the driver: the device menus + // list its devices, and the latency line looks its figure up + connect(menu, &QMenu::aboutToShow, + this, &MainWindow::nameDefaultAudioDriver); + m_audioDeviceMenu = menu->addMenu(tr("Audio Output &Device")); m_audioDeviceMenu->setStatusTip(tr("Choose which device Tony plays through")); m_audioDeviceGroup = new QActionGroup(this); @@ -2296,6 +2380,12 @@ MainWindow::setupCompactLayout() for (QMenu *menu: { m_audioDeviceMenu, m_audioInputDeviceMenu }) { if (menu) parts.hiddenActions.push_back(menu->menuAction()); } + if (m_audioDriverMenus) { + for (QMenu *menu: { m_audioDriverMenus->driverMenu(), + m_audioDriverMenus->latencyMenu() }) { + parts.hiddenActions.push_back(menu->menuAction()); + } + } // For room: the panes are what a phone's height is wanted for parts.hiddenWidgets = { m_overview }; @@ -2548,6 +2638,10 @@ MainWindow::updateMenuStates() if (m_calibrateAudioAction) { m_calibrateAudioAction->setEnabled(!inTake && !checking); } + // Choosing either opens the device afresh + if (m_audioDriverMenus) { + m_audioDriverMenus->setEnabled(!inTake && !checking); + } for (QMenu *m : { m_audioDeviceMenu, m_audioInputDeviceMenu }) { if (m) m->menuAction()->setEnabled(!checking); } diff --git a/main/MainWindow.h b/main/MainWindow.h index 32ef36fa..6615906a 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -50,6 +50,7 @@ class PlotSize; class AudioCheckRunner; struct AudioCheckResult; +class AudioDriverMenus; class CalibrateAudioDialog; #ifdef TONY_DEV_CHECKS class DevChecks; @@ -323,6 +324,11 @@ protected slots: virtual void rescanAudioDevices(); virtual void audioDeviceSelected(QAction *); + // Playback > Audio Driver or Audio Latency chosen, and written to the + // Preferences: the device is opened again, with it + void audioDriverChosen(QString implementation); + void audioLatencyChosen(double seconds); + // Playback > Calibrate Audio: the audio check's dialog, not modal virtual void calibrateAudio(); @@ -806,6 +812,9 @@ protected slots: QMenu *m_audioInputDeviceMenu; QActionGroup *m_audioInputDeviceGroup; + // Playback > Audio Driver and Audio Latency, before the device menus + AudioDriverMenus *m_audioDriverMenus; + QAction *m_deleteSelectedAction; QAction *m_ffwdAction; QAction *m_rwdAction; @@ -860,6 +869,16 @@ protected slots: const std::vector &names, QString settingKey); + // The implementations bqaudioio has, of which the drivers are offered + // in Playback > Audio Driver. Virtual so that the tests can give + // drivers the platform they run on does not have + virtual QStringList audioImplementationNames() const; + + // Where no driver is named and MME is built in, name it (and carry + // the devices chosen before over to it): before a device is opened, + // and before the Playback menu shows the device menus + void nameDefaultAudioDriver(); + // Helpers for the singing / second-track workflow. // deferAnalysis=true skips pYIN: swapSingingAudio() uses it, as the // pitch and notes layers it hands the new analyser are analysed @@ -1214,6 +1233,13 @@ protected slots: QTimer *m_audioDeviceCheck; QElapsedTimer m_audioDeviceReopened; int m_audioDeviceReopens; +#else + // Before each device is opened: a driver named where none is, and + // the latency chosen for it handed to bqaudioio. Then openAudioIO() + void createAudioIO() override; + + // Opens the device the Preferences name, as MainWindowBase does + virtual void openAudioIO(); #endif // A session must not be saved in the middle of the analysis of a diff --git a/main/dev/DevChecks.cpp b/main/dev/DevChecks.cpp index eacb52e7..61815579 100644 --- a/main/dev/DevChecks.cpp +++ b/main/dev/DevChecks.cpp @@ -16,6 +16,7 @@ #include "DevChecks.h" +#include "../AudioDriverMenus.h" #include "../MainWindow.h" #include "../RealtimePitchTracker.h" #include "../SingingTakes.h" @@ -2510,6 +2511,15 @@ DevChecks::writeReport(const DevReport &report) const out << "Output device: " << device(m_devices.playbackDevice) << "\n"; out << "Input device: " << device(m_devices.recordDevice) << "\n"; + // So that runs on two drivers, or at two latencies, can be told apart + out << "Audio driver: " + << AudioDriverMenus::driverName(m_devices.implementation) << "\n"; + const double latencyAsked = m_window->m_audioDriverMenus ? + m_window->m_audioDriverMenus->appliedLatency() : 0.0; + out << "Latency asked for: " + << (latencyAsked > 0.0 ? unsignedMs(latencyAsked) + : QString("none")) << "\n"; + // What the device says of itself, as the window has it now. The // output latency counts frames at the rate the play source was told // the device runs at, else the recording's (MainWindow::roundTripAt()) diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index aecd2df8..bb7b1367 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -227,6 +227,106 @@ class TestAudioCheck : public QObject settings.endGroup(); } + // The implementations bqaudioio has on Windows, whatever it has here + static QStringList windowsImplementations() { + return { "port", "mme", "directsound", "wasapi" }; + } + + // A Preference, as the driver and device menus write it + static void setPreference(QString name, QString value) { + QSettings settings; + settings.setValue("Preferences/" + name, value); + } + + static QString preference(QString name) { + QSettings settings; + return settings.value("Preferences/" + name).toString(); + } + + // The driver, the devices kept for each driver and the latency asked + // of each, as a test may have left them + static void forgetDriverPreferences() { + QSettings settings; + settings.beginGroup("Preferences"); + for (const QString &name : settings.childKeys()) { + if (name == "audio-target" || + name.startsWith("audio-latency-") || + name.startsWith("audio-playback-device-") || + name.startsWith("audio-record-device-")) { + settings.remove(name); + } + } + settings.endGroup(); + } + + // A menu's entries, the one ticked, and one by its text + static QStringList entries(QMenu *menu) { + QStringList texts; + for (QAction *a : menu->actions()) { + if (!a->isSeparator()) texts << a->text(); + } + return texts; + } + + static QString ticked(QMenu *menu) { + for (QAction *a : menu->actions()) { + if (a->isChecked()) return a->text(); + } + return {}; + } + + static QAction *entry(QMenu *menu, QString text) { + for (QAction *a : menu->actions()) { + if (a->text() == text) return a; + } + return nullptr; + } + + // Playback > Audio Driver or Audio Latency opened, and an entry of it + // chosen + void chooseDriver(QString text) { + QMenu *menu = m_window->audioDriverMenus()->driverMenu(); + emit menu->aboutToShow(); + QAction *action = entry(menu, text); + QVERIFY2(action, qPrintable(text + " not in: " + + entries(menu).join(", "))); + action->trigger(); + } + + void chooseLatency(QString text) { + QMenu *menu = m_window->audioDriverMenus()->latencyMenu(); + emit menu->aboutToShow(); + QAction *action = entry(menu, text); + QVERIFY2(action, qPrintable(text + " not in: " + + entries(menu).join(", "))); + action->trigger(); + } + + // Audio Driver and Audio Latency, which are greyed out together: + // both enabled, or both not + bool driverMenusEnabled() { + AudioDriverMenus *menus = m_window->audioDriverMenus(); + return menus->driverMenu()->menuAction()->isEnabled() && + menus->latencyMenu()->menuAction()->isEnabled(); + } + + bool driverMenusDisabled() { + AudioDriverMenus *menus = m_window->audioDriverMenus(); + return !menus->driverMenu()->menuAction()->isEnabled() && + !menus->latencyMenu()->menuAction()->isEnabled(); + } + + // Shown, as they are where there is more than one driver: a hidden + // action is disabled as well, whatever it was set to + void showDriverMenus(QStringList implementations) { + m_window->setAudioImplementations(implementations); + m_window->doRebuildAudioDriverMenus(); + } + + double latencyApplied() { + return m_window->audioDriverMenus()->appliedLatency(); + } + static LatencyCalibration::Key key(QString output, QString input) { LatencyCalibration::Key key; key.playbackDevice = output; @@ -532,9 +632,11 @@ private slots: } void cleanup() { - // A round trip a test stored would place the next test's takes. - // First, as the waits below return early when they fail + // A round trip a test stored would place the next test's takes, + // and a driver it named open the next test's device. First, as + // the waits below return early when they fail QSettings().remove("LatencyCalibration"); + forgetDriverPreferences(); if (m_window) { if (m_window->recordTarget()->isRecording()) { @@ -1374,6 +1476,9 @@ private slots: void calibrate_audio_from_the_menu() { setDevices("Speakers A", "Microphone A"); makeWindow(loopback()); + // Without MME, which would be named, and the devices then read + // from its keys and not from the ones setDevices() writes + showDriverMenus({ "directsound", "wasapi" }); m_window->calibrateAudioAction()->trigger(); CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); @@ -1410,6 +1515,7 @@ private slots: QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); QVERIFY(!m_window->audioOutputMenu()->menuAction()->isEnabled()); QVERIFY(!m_window->audioInputMenu()->menuAction()->isEnabled()); + QVERIFY(driverMenusDisabled()); QVERIFY2(dialog->pageText().contains("Recording punch-in 1 of 2"), qPrintable(dialog->pageText())); @@ -1419,6 +1525,7 @@ private slots: QVERIFY(m_window->calibrateAudioAction()->isEnabled()); QVERIFY(m_window->audioOutputMenu()->menuAction()->isEnabled()); QVERIFY(m_window->audioInputMenu()->menuAction()->isEnabled()); + QVERIFY(driverMenusEnabled()); // 12411 frames measured, 12288 reported, at 44.1 kHz QVERIFY2(m_result.calibrationUsable(), describe(m_result).constData()); @@ -1556,22 +1663,27 @@ private slots: } // Not while an ordinary take is being recorded: the check records - // takes of its own + // takes of its own. Nor can the driver or its latency be chosen, + // which would open the device again under the take void calibrate_audio_not_during_a_take() { makeWindow(FakeAudioIO::Config()); + showDriverMenus(windowsImplementations()); openSong(); if (QTest::currentTestFailed()) return; QVERIFY(m_window->calibrateAudioAction()->isEnabled()); + QVERIFY(driverMenusEnabled()); m_window->doRecord(); QVERIFY(m_window->recordTarget()->isRecording()); QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); + QVERIFY(driverMenusDisabled()); QTest::qWait(300); m_window->doRecord(); QVERIFY(!m_window->recordTarget()->isRecording()); QTRY_VERIFY_WITH_TIMEOUT (m_window->calibrateAudioAction()->isEnabled(), 10000); + QVERIFY(driverMenusEnabled()); } // Closing the dialog while its check runs cancels the check; opened @@ -1597,6 +1709,224 @@ private slots: QVERIFY(dialog->isVisible()); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Instructions); } + + // Playback > Audio Driver: the drivers among the implementations + // bqaudioio has, by the names the user knows, in their order, with + // the one named ticked, and Audio Latency with it, both before the + // device menus. One driver is no choice, and neither is shown + void driver_menu_lists_the_drivers() { + setPreference("audio-target", "wasapi"); + makeWindow(FakeAudioIO::Config()); + showDriverMenus({ "port", "wasapi", "directsound", "mme" }); + + QMenu *drivers = m_window->audioDriverMenus()->driverMenu(); + QMenu *latencies = m_window->audioDriverMenus()->latencyMenu(); + QVERIFY(drivers->menuAction()->isVisible()); + QVERIFY(latencies->menuAction()->isVisible()); + QCOMPARE(drivers->title(), QString("Audio Dri&ver")); + QCOMPARE(latencies->title(), QString("Audio &Latency")); + QCOMPARE(entries(drivers), + QStringList({ "MME", "DirectSound", "WASAPI" })); + QCOMPARE(ticked(drivers), QString("WASAPI")); + QCOMPARE(entries(latencies), + QStringList({ "10 ms", "20 ms", "50 ms", "100 ms", + "200 ms" })); + QCOMPARE(ticked(latencies), QString("200 ms")); + + const QList playback = m_window->playbackMenu()->actions(); + const qsizetype driverAt = playback.indexOf(drivers->menuAction()); + const qsizetype latencyAt = playback.indexOf(latencies->menuAction()); + QVERIFY(driverAt >= 0); + QCOMPARE(latencyAt, driverAt + 1); + QVERIFY(latencyAt < + playback.indexOf(m_window->audioOutputMenu()->menuAction())); + + // Ticked afresh as the menu opens + setPreference("audio-target", "directsound"); + setPreference("audio-latency-directsound", "0.05"); + emit drivers->aboutToShow(); + QCOMPARE(ticked(drivers), QString("DirectSound")); + QCOMPARE(ticked(latencies), QString("50 ms")); + + m_window->setAudioImplementations({ "port", "mme" }); + m_window->doRebuildAudioDriverMenus(); + QVERIFY(!drivers->menuAction()->isVisible()); + QVERIFY(!latencies->menuAction()->isVisible()); + } + + // Choosing WASAPI names it and opens the device again, WASAPI's + // devices this time; the device menus then write WASAPI's keys, and + // MME's are left as they were + void driver_chosen_from_the_menu() { + setPreference("audio-target", "mme"); + setPreference("audio-playback-device-mme", "Speakers (MME)"); + setPreference("audio-record-device-mme", "Mic (MME)"); + setPreference("audio-playback-device-wasapi", "Speakers (WASAPI)"); + setPreference("audio-record-device-wasapi", "Mic (WASAPI)"); + makeWindow(FakeAudioIO::Config()); + m_window->setAudioImplementations(windowsImplementations()); + m_window->recreateAudioIO(); + QCOMPARE(m_window->audioIOOpened(), 1); + QCOMPARE(m_window->audioIOOpenedFor().implementation, QString("mme")); + QCOMPARE(m_window->audioIOOpenedFor().playbackDevice, + QString("Speakers (MME)")); + + chooseDriver("WASAPI"); + if (QTest::currentTestFailed()) return; + QCOMPARE(preference("audio-target"), QString("wasapi")); + QCOMPARE(m_window->audioIOOpened(), 2); + QVERIFY(m_window->fake()); + const LatencyCalibration::Key opened = m_window->audioIOOpenedFor(); + QCOMPARE(opened.implementation, QString("wasapi")); + QCOMPARE(opened.playbackDevice, QString("Speakers (WASAPI)")); + QCOMPARE(opened.recordDevice, QString("Mic (WASAPI)")); + QCOMPARE(ticked(m_window->audioDriverMenus()->driverMenu()), + QString("WASAPI")); + + // The driver in use chosen again: nothing to open afresh + chooseDriver("WASAPI"); + QCOMPARE(m_window->audioIOOpened(), 2); + + // (System Default), from a menu that lists WASAPI's devices (none + // here, where there is no WASAPI) + m_window->doRescanAudioDevices(); + QAction *systemDefault = + m_window->audioOutputMenu()->actions().value(0); + QVERIFY(systemDefault); + QCOMPARE(systemDefault->text(), QString("(System Default)")); + systemDefault->trigger(); + QCOMPARE(preference("audio-playback-device-wasapi"), QString()); + QCOMPARE(preference("audio-playback-device-mme"), + QString("Speakers (MME)")); + QCOMPARE(m_window->audioIOOpenedFor().playbackDevice, QString()); + QCOMPARE(m_window->audioIOOpenedFor().recordDevice, + QString("Mic (WASAPI)")); + } + + // With no driver named and MME there, MME is named before the first + // device is opened, and the devices chosen before are MME's; or + // before the Playback menu shows its device menus. A driver named + // already is left alone + void driver_named_by_default() { + setDevices("Speakers", "Microphone"); + makeWindow(FakeAudioIO::Config()); + m_window->setAudioImplementations(windowsImplementations()); + QVERIFY(!m_window->fake()); + QCOMPARE(preference("audio-target"), QString()); + + m_window->recreateAudioIO(); + QCOMPARE(m_window->audioIOOpened(), 1); + LatencyCalibration::Key opened = m_window->audioIOOpenedFor(); + QCOMPARE(opened.implementation, QString("mme")); + QCOMPARE(opened.playbackDevice, QString("Speakers")); + QCOMPARE(opened.recordDevice, QString("Microphone")); + QCOMPARE(preference("audio-target"), QString("mme")); + QCOMPARE(preference("audio-playback-device-mme"), QString("Speakers")); + QCOMPARE(preference("audio-record-device-mme"), QString("Microphone")); + + forgetDriverPreferences(); + setPreference("audio-target", "auto"); + emit m_window->playbackMenu()->aboutToShow(); + QCOMPARE(preference("audio-target"), QString("mme")); + + setPreference("audio-target", "directsound"); + m_window->recreateAudioIO(); + emit m_window->playbackMenu()->aboutToShow(); + QCOMPARE(m_window->audioIOOpenedFor().implementation, + QString("directsound")); + QCOMPARE(preference("audio-target"), QString("directsound")); + + // Nor is anything named where there is no MME + forgetDriverPreferences(); + m_window->setAudioImplementations({ "pulse", "port", "jack" }); + m_window->recreateAudioIO(); + emit m_window->playbackMenu()->aboutToShow(); + QCOMPARE(m_window->audioIOOpenedFor().implementation, QString()); + QCOMPARE(preference("audio-target"), QString()); + } + + // The latency is chosen per driver, handed to bqaudioio as the device + // is opened, and choosing one opens it again; 200 ms where none has + // been chosen + void latency_kept_per_driver() { + setPreference("audio-target", "mme"); + makeWindow(FakeAudioIO::Config()); + m_window->setAudioImplementations(windowsImplementations()); + m_window->recreateAudioIO(); + QCOMPARE(latencyApplied(), 0.2); + + chooseLatency("20 ms"); + if (QTest::currentTestFailed()) return; + QCOMPARE(preference("audio-latency-mme").toDouble(), 0.02); + QCOMPARE(m_window->audioIOOpened(), 2); + QCOMPARE(latencyApplied(), 0.02); + + chooseDriver("WASAPI"); + if (QTest::currentTestFailed()) return; + QCOMPARE(latencyApplied(), 0.2); + emit m_window->audioDriverMenus()->latencyMenu()->aboutToShow(); + QCOMPARE(ticked(m_window->audioDriverMenus()->latencyMenu()), + QString("200 ms")); + + chooseLatency("10 ms"); + if (QTest::currentTestFailed()) return; + QCOMPARE(latencyApplied(), 0.01); + QCOMPARE(preference("audio-latency-mme").toDouble(), 0.02); + + chooseDriver("MME"); + if (QTest::currentTestFailed()) return; + QCOMPARE(latencyApplied(), 0.02); + emit m_window->audioDriverMenus()->latencyMenu()->aboutToShow(); + QCOMPARE(ticked(m_window->audioDriverMenus()->latencyMenu()), + QString("20 ms")); + QCOMPARE(m_window->audioIOOpened(), 5); + } + + // A round trip measured under one driver is not the one takes are + // placed with under another, and is again under the first. The + // check's instructions name the driver + void measured_latency_kept_per_driver() { + setPreference("audio-target", "mme"); + makeWindow(loopback()); + m_window->setAudioImplementations(windowsImplementations()); + m_window->recreateAudioIO(); + const LatencyCalibration::InUse reported = m_window->latencyInUse(); + QVERIFY(reported.source == LatencyCalibration::Source::Reported); + + LatencyCalibration::Figure figure; + figure.roundTrip = 0.3; + figure.date = QDateTime::currentDateTimeUtc(); + figure.reportedOutput = reported.reportedOutput; + figure.reportedInput = reported.reportedInput; + LatencyCalibration::Key mme = key("", ""); + mme.implementation = "mme"; + QSettings settings; + LatencyCalibration::store(settings, mme, figure); + QVERIFY(m_window->latencyInUse().source == + LatencyCalibration::Source::Measured); + + chooseDriver("WASAPI"); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->latencyInUse().source == + LatencyCalibration::Source::Reported); + QVERIFY2(latencyLine().startsWith("Latency: driver's figure"), + qPrintable(latencyLine())); + + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + for (QString words : { "Driver:", "WASAPI" }) { + QVERIFY2(dialog->pageText().contains(words), + qPrintable(words + " not in: " + dialog->pageText())); + } + dialog->close(); + + chooseDriver("MME"); + if (QTest::currentTestFailed()) return; + QVERIFY(m_window->latencyInUse().source == + LatencyCalibration::Source::Measured); + QCOMPARE(m_window->latencyInUse().roundTrip, 0.3); + } }; #endif diff --git a/main/test/TestAudioDriverSettings.h b/main/test/TestAudioDriverSettings.h new file mode 100644 index 00000000..78908d1d --- /dev/null +++ b/main/test/TestAudioDriverSettings.h @@ -0,0 +1,143 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TEST_AUDIO_DRIVER_SETTINGS_H +#define TEST_AUDIO_DRIVER_SETTINGS_H + +// Tier 2: the audio driver and the latency asked of it, as the +// Preferences keep them. No window and no device. The settings are an +// INI file of each test's own, as TestLatencyCalibration's are. + +#include "../AudioDriverSettings.h" + +#include +#include +#include +#include +#include + +class TestAudioDriverSettings : public QObject +{ + Q_OBJECT + + QTemporaryDir m_dir; + int m_counter = 0; + QString m_path; + +private slots: + void initTestCase() { + QVERIFY(m_dir.isValid()); + } + + void init() { + m_path = m_dir.filePath(QString("settings-%1.ini").arg(++m_counter)); + } + + void cleanup() { + QFile::remove(m_path); + } + + // The drivers among bqaudioio's implementations, in the order they + // are offered in, whatever order the factory gives them in + void drivers_in_their_order() { + QCOMPARE(AudioDriverSettings::drivers + ({ "pulse", "wasapi", "port", "directsound", "jack", "mme" }), + QStringList({ "mme", "directsound", "wasapi" })); + QCOMPARE(AudioDriverSettings::drivers({ "pulse", "port", "jack" }), + QStringList()); + QCOMPARE(AudioDriverSettings::drivers({ "port", "wasapi" }), + QStringList({ "wasapi" })); + } + + // MME is named where nothing is, or "auto", and the devices chosen + // before there were drivers become MME's, where MME has none of its + // own. Then, or where anything else is named, nothing happens + void default_driver_named_once() { + QSettings settings(m_path, QSettings::IniFormat); + const QStringList windows = { "port", "mme", "directsound", "wasapi" }; + + settings.setValue("Preferences/audio-playback-device", "Speakers"); + settings.setValue("Preferences/audio-record-device", "Mic"); + settings.setValue("Preferences/audio-record-device-mme", "Mic (MME)"); + QVERIFY(AudioDriverSettings::nameDefaultDriver(settings, windows)); + QCOMPARE(settings.value("Preferences/audio-target").toString(), + QString("mme")); + QCOMPARE(AudioDriverSettings::currentImplementation(settings), + QString("mme")); + QCOMPARE(settings.value("Preferences/audio-playback-device-mme") + .toString(), QString("Speakers")); + QCOMPARE(settings.value("Preferences/audio-record-device-mme") + .toString(), QString("Mic (MME)")); + + // Named now: another device without a suffix is not carried over + settings.setValue("Preferences/audio-playback-device", "Headphones"); + QVERIFY(!AudioDriverSettings::nameDefaultDriver(settings, windows)); + QCOMPARE(settings.value("Preferences/audio-playback-device-mme") + .toString(), QString("Speakers")); + + settings.setValue("Preferences/audio-target", "auto"); + QCOMPARE(AudioDriverSettings::currentImplementation(settings), + QString()); + QVERIFY(AudioDriverSettings::nameDefaultDriver(settings, windows)); + QCOMPARE(AudioDriverSettings::currentImplementation(settings), + QString("mme")); + + settings.setValue("Preferences/audio-target", "wasapi"); + QVERIFY(!AudioDriverSettings::nameDefaultDriver(settings, windows)); + QCOMPARE(AudioDriverSettings::currentImplementation(settings), + QString("wasapi")); + + // Without MME built in, nothing is named, and no device key made + settings.remove("Preferences"); + settings.setValue("Preferences/audio-playback-device", "Speakers"); + QVERIFY(!AudioDriverSettings::nameDefaultDriver + (settings, { "pulse", "port", "jack" })); + QVERIFY(!settings.contains("Preferences/audio-target")); + QVERIFY(!settings.contains("Preferences/audio-playback-device-mme")); + QVERIFY(!settings.contains("Preferences/audio-record-device-mme")); + } + + // Kept per driver, in seconds; 200 ms where none has been chosen, or + // no driver is named + void latency_kept_per_driver() { + std::vector choices = AudioDriverSettings::latencyChoices(); + QCOMPARE(int(choices.size()), 5); + QCOMPARE(choices.front(), 0.010); + QCOMPARE(choices.back(), AudioDriverSettings::kDefaultLatency); + QCOMPARE(AudioDriverSettings::kDefaultLatency, 0.2); + + QSettings settings(m_path, QSettings::IniFormat); + QCOMPARE(AudioDriverSettings::latency(settings, "wasapi"), 0.2); + AudioDriverSettings::setLatency(settings, "wasapi", 0.02); + AudioDriverSettings::setLatency(settings, "mme", 0.1); + settings.sync(); + + QSettings again(m_path, QSettings::IniFormat); + QCOMPARE(again.value("Preferences/audio-latency-wasapi").toString() + .toDouble(), 0.02); + QCOMPARE(AudioDriverSettings::latency(again, "wasapi"), 0.02); + QCOMPARE(AudioDriverSettings::latency(again, "mme"), 0.1); + QCOMPARE(AudioDriverSettings::latency(again, "directsound"), 0.2); + QCOMPARE(AudioDriverSettings::latency(again, ""), 0.2); + + again.setValue("Preferences/audio-latency-mme", "nonsense"); + QCOMPARE(AudioDriverSettings::latency(again, "mme"), 0.2); + again.setValue("Preferences/audio-latency-mme", "0"); + QCOMPARE(AudioDriverSettings::latency(again, "mme"), 0.2); + + QVERIFY(AudioDriverSettings::sameLatency(0.02, 0.0201)); + QVERIFY(!AudioDriverSettings::sameLatency(0.02, 0.01)); + } +}; + +#endif diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index 42944043..91abca88 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -588,8 +588,11 @@ private slots: describe()); QVERIFY2(mic->message.startsWith("Not applicable here"), describe()); - // What the device says of itself, at the head of the report - for (QString words : { QString("Audio drivers built in: "), + // What the device says of itself, at the head of the report, and + // the driver it was opened through with the latency asked of it + for (QString words : { QString("Audio driver: (auto)\n"), + QString("Latency asked for: 200.0 ms\n"), + QString("Audio drivers built in: "), QString("Playback latency reported: 8192 " "frames (185.8 ms)"), QString("Record latency reported: 4096 " diff --git a/main/test/TestMainWindow.h b/main/test/TestMainWindow.h index 6a456a55..1b8a983a 100644 --- a/main/test/TestMainWindow.h +++ b/main/test/TestMainWindow.h @@ -21,6 +21,7 @@ #include "../MainWindow.h" #include "../Analyser.h" +#include "../AudioDriverMenus.h" #include "../CoverageStrip.h" #include "../SingingTakes.h" @@ -37,6 +38,7 @@ #include #include #include +#include #include #include @@ -183,6 +185,23 @@ class TestMainWindow : public MainWindow QMenu *playbackMenu() { return m_playbackMenu; } QMenu *audioOutputMenu() { return m_audioDeviceMenu; } QMenu *audioInputMenu() { return m_audioInputDeviceMenu; } + + // Playback > Audio Driver and Audio Latency, from the implementations + // given here: none unless a test gives some, whatever the platform + // has. Rebuilt as the app rebuilds them when they open + void setAudioImplementations(QStringList names) { + m_implementations = names; + } + AudioDriverMenus *audioDriverMenus() { return m_audioDriverMenus; } + void doRebuildAudioDriverMenus() { m_audioDriverMenus->rebuild(); } + void doRescanAudioDevices() { rescanAudioDevices(); } + + // How often a device has been opened, and the driver and devices the + // Preferences named for the last one + int audioIOOpened() const { return m_audioIOOpened; } + LatencyCalibration::Key audioIOOpenedFor() const { + return m_audioIOOpenedFor; + } TakeLatency takeLatency() { return m_takeLatency; } QAction *playSingingAudioAction() { return m_playSingingAudio; } @@ -316,9 +335,15 @@ class TestMainWindow : public MainWindow MainWindow::onRealtimePitchDetected(estimates); } - void createAudioIO() override { + // MainWindow::createAudioIO() has named the driver and applied its + // latency by now; the fake is opened in place of the device svapp + // would open, and what that would have been asked for is kept + void openAudioIO() override { if (m_audioIO || m_playTarget) return; if (!m_installDevice) return; + ++m_audioIOOpened; + QSettings settings; + m_audioIOOpenedFor = LatencyCalibration::currentKey(settings, 0); m_fakeConfig.inputIsKept = [this]() { return m_recordTarget->isRecording(); }; @@ -328,6 +353,10 @@ class TestMainWindow : public MainWindow m_playSource->setSystemPlaybackTarget(m_audioIO); } + QStringList audioImplementationNames() const override { + return m_implementations; + } + bool confirmRecordingOverTake() override { ++m_recordOverQuestions; if (m_recordOverInDialog) { @@ -413,6 +442,9 @@ class TestMainWindow : public MainWindow private: FakeAudioIO::Config m_fakeConfig; bool m_installDevice; + QStringList m_implementations; + int m_audioIOOpened = 0; + LatencyCalibration::Key m_audioIOOpenedFor; int m_liveDotsDelayMs = 0; bool m_recordOverAnswer = true; bool m_recordOverInDialog = false; diff --git a/main/test/tony-core-test.cpp b/main/test/tony-core-test.cpp index 834ffaab..b9271c10 100644 --- a/main/test/tony-core-test.cpp +++ b/main/test/tony-core-test.cpp @@ -32,6 +32,7 @@ #include "TestLyricsEdit.h" #include "TestLatencyCheck.h" #include "TestLatencyCalibration.h" +#include "TestAudioDriverSettings.h" #include "TestTakeDiff.h" #include "TestLiveDotsFeed.h" #include "TestRunSuite.h" @@ -185,6 +186,12 @@ int main(int argc, char *argv[]) else ++bad; } + { + TestAudioDriverSettings t; + if (runSuite(&t, argc, argv)) ++good; + else ++bad; + } + { TestTakeDiff t; if (runSuite(&t, argc, argv)) ++good; diff --git a/meson.build b/meson.build index b8337fb3..eb6918e7 100644 --- a/meson.build +++ b/meson.build @@ -1186,6 +1186,7 @@ tony_entry_files = [ # No GUI dependencies: usable from a QCoreApplication test. tony_core_files = [ 'main/AndroidFiles.cpp', + 'main/AudioDriverSettings.cpp', 'main/Coverage.cpp', 'main/DecodedPcm.cpp', 'main/LogFile.cpp', @@ -1220,6 +1221,7 @@ endif tony_app_files = [ 'main/AlternatePitchTrack.cpp', 'main/AudioCheckRunner.cpp', + 'main/AudioDriverMenus.cpp', 'main/CalibrateAudioDialog.cpp', 'main/CompactLayout.cpp', 'main/CoverageStrip.cpp', @@ -1254,6 +1256,7 @@ tony_app_moc_headers = [ 'main/Analyser.h', 'main/AlternatePitchTrack.h', 'main/AudioCheckRunner.h', + 'main/AudioDriverMenus.h', 'main/CalibrateAudioDialog.h', 'main/CompactLayout.h', 'main/CoverageStrip.h', @@ -1561,6 +1564,7 @@ if system != 'android' 'main/test/TestLyricsEdit.h', 'main/test/TestLatencyCheck.h', 'main/test/TestLatencyCalibration.h', + 'main/test/TestAudioDriverSettings.h', 'main/test/TestTakeDiff.h', 'main/test/TestLiveDotsFeed.h', 'main/test/TestRunSuite.h', From 8bdd54c9e148fd86a1fa925e8dc5dbd6e0dfe1b9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:47:26 +0000 Subject: [PATCH 227/275] docs: the driver project's work order for w4, the docs pass Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- docs/audio-drivers-work-orders.md | 33 +++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md index fe3b340d..edaa45aa 100644 --- a/docs/audio-drivers-work-orders.md +++ b/docs/audio-drivers-work-orders.md @@ -161,6 +161,39 @@ where `createAudioIO()` is first called. - Docs: recording.md "Latency" (the latency asked for), calibrate-audio.md where it says the reported pair comes from `suggestedLatency = 0.2`. +### W4 — Docs + +Read: this file's log; the spec; `git log --stat bcfbfdd..HEAD` (the project's commits); +the docs named below, by section. Code only to check a statement. + +- `docs/audio-drivers.md` becomes the page for the drivers **as built**, as + calibrate-audio.md is for Calibrate Audio: why, the decisions, the facts, the design as + it is, the tests, what W5 is to measure and what to send back, open points. Plan + wording ("will", "phase W3 does") goes; §5's phases become a short "state" of what is + done and what W5 still is. +- `docs/manual-checklist.md`: items for the driver and latency menus on Windows (first + start: MME ticked, devices kept; WASAPI lists its own devices; a latency change reopens + the device; the report names driver and latency), and W5's runs: Calibrate Audio and a + dev run on MME at 200 ms and on WASAPI at 20 and 10 ms, the `DevChecks.txt` of each sent + back. Where the checklist's device section already has the dev run, extend it rather + than repeat it. +- `docs/calibrate-audio.md`: §11's test list (W1's and W3's tests); §10's driver project + pointing to audio-drivers.md instead of repeating it. +- `docs/open-points.md`, `docs/architecture.md` (the libraries' contents), + `docs/README.md` and `AGENTS.md`'s "Read / Before" table (a row for audio-drivers.md: + touching the driver or latency menus, `AudioDriverSettings`, `AudioDriverMenus`, or the + bqaudioio fork), the root `README.md`'s feature list (the two menus, one line). +- `docs/building.md`, "Building on Linux": how the fork's Windows-only code is checked + here — `apt-get install g++-mingw-w64-x86-64-posix`, PortAudio 19.7.0's headers + (`portaudio.h`, `pa_win_wasapi.h`, `pa_win_waveformat.h`) from + `raw.githubusercontent.com/PortAudio/portaudio/v19.7.0/include/`, and + `x86_64-w64-mingw32-g++ -std=c++17 -fsyntax-only -DHAVE_PORTAUDIO -I + -Ibqaudioio/bqaudioio -Ibqaudioio/src -Ibqvec bqaudioio/src/PortAudioIO.cpp` (and + `AudioFactory.cpp`); that is how W2 was checked (both clean, and the WASAPI block seen + in the preprocessed output). +- Remove this file (`docs/audio-drivers-work-orders.md`) and every link to it. +- No narratives of fixed bugs, no commit hashes, no lists of members. + ## 5. Log Capped at 25 lines per entry. Newest last. From a818119adb7c32342e2db24d1f23c2a7b8034751 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:48:33 +0000 Subject: [PATCH 228/275] feat: view > lyrics size, 50 % by default on android 65 % still left the words too large on a phone. View > Lyrics Size draws them at 35, 50, 65, 80 or 100 % of svgui's size, applied at once and remembered like Plot Size: 50 % by default on Android, 100 % elsewhere. The size the words come to is written to the log when it changes (pins svgui 23e1365). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/forks.md | 5 +- main/LyricsSize.cpp | 90 +++++++++++++++++++++++++++++++++++ main/LyricsSize.h | 68 ++++++++++++++++++++++++++ main/LyricsTrack.cpp | 17 ++++--- main/LyricsTrack.h | 12 +++-- main/MainWindow.cpp | 8 ++++ main/MainWindow.h | 4 ++ main/test/TestCompactLayout.h | 42 ++++++++++++++++ meson.build | 2 + repoint-lock.json | 2 +- 10 files changed, 234 insertions(+), 16 deletions(-) create mode 100644 main/LyricsSize.cpp create mode 100644 main/LyricsSize.h diff --git a/docs/forks.md b/docs/forks.md index c6fc97f3..4d1149e3 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -117,8 +117,9 @@ gitignored. Pass the directory as the search path explicitly, or use `grep -rn` the view's at the least, up to four times, and never more than an eighth of the view's height; it grows with the **square root** of the zoom, so that zooming in gives the words room (their boxes grow with the zoom itself). `setLyricsTextScale()` (branch - `feat/tonyandroid`) draws the words at a share of that: `LyricsTrack` asks for 65 % on - Android, where the desktop's size left room for only a few words. No vertical scale, no feature + `feat/tonyandroid`) draws the words at a share of that: View > Lyrics Size + (`LyricsSize`, 50 % by default on Android, where the desktop's size left room for only a + few words); each new size is written to the log. No vertical scale, no feature description, and not editable by the pane's tools: Tony's `LyricsEditor` edits the model itself. `setHighlightFrame()` draws the region at that frame in amber (the latest to start, where diff --git a/main/LyricsSize.cpp b/main/LyricsSize.cpp new file mode 100644 index 00000000..d485ae78 --- /dev/null +++ b/main/LyricsSize.cpp @@ -0,0 +1,90 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "LyricsSize.h" + +#include +#include +#include + +static const char *settingsGroup = "MainWindow"; +static const char *settingsKey = "lyricssize"; + +LyricsSize::LyricsSize(QObject *parent) : + QObject(parent), + m_group(new QActionGroup(this)), + m_percent(0) +{ + m_group->setExclusive(true); + for (int percent : getSteps()) { + QAction *action = new QAction(tr("%1%").arg(percent), m_group); + action->setCheckable(true); + action->setData(percent); + action->setStatusTip + (tr("Draw the words of the lyrics at %1% of their normal size") + .arg(percent)); + connect(action, &QAction::triggered, + this, [this, percent]() { setPercent(percent); }); + } + + QSettings settings; + settings.beginGroup(settingsGroup); + int percent = settings.value(settingsKey, getDefaultPercent()).toInt(); + settings.endGroup(); + + // A value from elsewhere that is not a step gives the default + if (!getSteps().contains(percent)) { + percent = getDefaultPercent(); + } + apply(percent); +} + +QList +LyricsSize::getActions() const +{ + return m_group->actions(); +} + +int +LyricsSize::getDefaultPercent() +{ +#ifdef Q_OS_ANDROID + return 50; +#else + return 100; +#endif +} + +void +LyricsSize::setPercent(int percent) +{ + if (!getSteps().contains(percent)) return; + apply(percent); + + QSettings settings; + settings.beginGroup(settingsGroup); + settings.setValue(settingsKey, percent); + settings.endGroup(); +} + +void +LyricsSize::apply(int percent) +{ + bool changed = (percent != m_percent); + m_percent = percent; + for (QAction *action : m_group->actions()) { + if (action->data().toInt() == percent) action->setChecked(true); + } + if (changed) emit textScaleChanged(getTextScale()); +} diff --git a/main/LyricsSize.h b/main/LyricsSize.h new file mode 100644 index 00000000..f6b82141 --- /dev/null +++ b/main/LyricsSize.h @@ -0,0 +1,68 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_LYRICS_SIZE_H +#define TONY_LYRICS_SIZE_H + +#include +#include + +class QAction; +class QActionGroup; + +/** + * How large the lyrics' words are drawn, as View > Lyrics Size sets + * it: a share of the size svgui's lyrics layer gives them, which + * grows with the zoom from twice the view's font to four times. + * Chosen, a step takes effect at once (textScaleChanged()) and is + * remembered in the settings. + * + * The default is 50% on Android, where the whole size left room for + * only three or four words in the pane, less than a verse, unless it + * was zoomed far in; 100% elsewhere. + */ +class LyricsSize : public QObject +{ + Q_OBJECT + +public: + /// Reads the setting; the scale is there from the start + explicit LyricsSize(QObject *parent); + + /// The checkable steps, one of them checked, for a submenu + QList getActions() const; + + int getPercent() const { return m_percent; } + double getTextScale() const { return m_percent / 100.0; } + + static QList getSteps() { return { 35, 50, 65, 80, 100 }; } + + /// 50 on Android, 100 elsewhere + static int getDefaultPercent(); + +signals: + void textScaleChanged(double scale); + +public slots: + /// Applies and remembers a step; anything else is ignored + void setPercent(int percent); + +private: + void apply(int percent); + + QActionGroup *m_group; + int m_percent; +}; + +#endif diff --git a/main/LyricsTrack.cpp b/main/LyricsTrack.cpp index 5c70572a..1b18b079 100644 --- a/main/LyricsTrack.cpp +++ b/main/LyricsTrack.cpp @@ -46,7 +46,8 @@ LyricsTrack::LyricsTrack(QObject *parent) : QObject(parent), m_document(nullptr), m_pane(nullptr), - m_layer(nullptr) + m_layer(nullptr), + m_textScale(1.0) { } @@ -54,14 +55,12 @@ LyricsTrack::~LyricsTrack() { } -double -LyricsTrack::textScale() +void +LyricsTrack::setTextScale(double scale) { -#ifdef Q_OS_ANDROID - return 0.65; -#else - return 1.0; -#endif + if (!(scale > 0.0)) return; + m_textScale = scale; + if (m_layer) m_layer->setLyricsTextScale(m_textScale); } QString @@ -172,7 +171,7 @@ LyricsTrack::configureLayer() // scale to this layer m_layer->setVerticalScale(RegionLayer::EqualSpaced); m_layer->setPlotStyle(RegionLayer::PlotLyrics); - m_layer->setLyricsTextScale(textScale()); + m_layer->setLyricsTextScale(m_textScale); // The words are dark on light boxes whatever this is; grey rather // than the layer's default black wherever else its colour shows diff --git a/main/LyricsTrack.h b/main/LyricsTrack.h index 0cba15a1..4c04d0dc 100644 --- a/main/LyricsTrack.h +++ b/main/LyricsTrack.h @@ -111,11 +111,14 @@ class LyricsTrack : public QObject static QString layerName(); /** - * The share of svgui's size the words are drawn at: 65% on a phone, - * where the desktop's size leaves room for only a few words in the - * pane and a verse wants to be on show at once; all of it elsewhere. + * The share of svgui's size the words are drawn at (View > Lyrics + * Size, LyricsSize), now and in any layer taken on later. 1 until + * set. */ - static double textScale(); + double getTextScale() const { return m_textScale; } + +public slots: + void setTextScale(double scale); private slots: void layerAboutToBeDeleted(sv::Layer *); @@ -124,6 +127,7 @@ private slots: sv::Document *m_document; sv::Pane *m_pane; sv::RegionLayer *m_layer; + double m_textScale; void takeLayer(sv::Document *, sv::Pane *, sv::RegionLayer *); void configureLayer(); diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index af471cbd..e929a447 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -20,6 +20,7 @@ #include "Analyser.h" #include "CompactLayout.h" #include "PlotSize.h" +#include "LyricsSize.h" #include "AudioCheckRunner.h" #include "CalibrateAudioDialog.h" #include "LatencyUtils.h" @@ -166,6 +167,7 @@ MainWindow::MainWindow(AudioMode audioMode, m_overview(0), m_compactLayout(nullptr), m_plotSize(nullptr), + m_lyricsSize(nullptr), m_playAction(nullptr), m_recordAction(nullptr), m_zoomInAction(nullptr), @@ -518,6 +520,10 @@ MainWindow::MainWindow(AudioMode audioMode, // Before the menus: their switches are in the View menu m_compactLayout = new CompactLayout(this); m_plotSize = new PlotSize(m_viewManager, this); + m_lyricsSize = new LyricsSize(this); + m_lyrics->setTextScale(m_lyricsSize->getTextScale()); + connect(m_lyricsSize, &LyricsSize::textScaleChanged, + m_lyrics, &LyricsTrack::setTextScale); setupMenus(); setupToolbars(); @@ -1165,6 +1171,8 @@ MainWindow::setupViewMenu() menu->addAction(m_compactLayout->getAction()); QMenu *plotSizeMenu = menu->addMenu(tr("Plot &Size")); plotSizeMenu->addActions(m_plotSize->getActions()); + QMenu *lyricsSizeMenu = menu->addMenu(tr("Lyrics Si&ze")); + lyricsSizeMenu->addActions(m_lyricsSize->getActions()); // Enabled and checked in updateLayerStatuses(). Not "Show &Lyrics": // Peek Left has the L m_showLyrics = new QAction(tr("Show L&yrics"), this); diff --git a/main/MainWindow.h b/main/MainWindow.h index ba728fcc..7f0514a2 100644 --- a/main/MainWindow.h +++ b/main/MainWindow.h @@ -48,6 +48,7 @@ class QActionGroup; class QToolBar; class CompactLayout; class PlotSize; +class LyricsSize; class AudioCheckRunner; struct AudioCheckResult; @@ -424,6 +425,9 @@ protected slots: // View > Plot Size: how large the panes draw pitch and notes PlotSize *m_plotSize; + // View > Lyrics Size: how large the lyrics' words are drawn + LyricsSize *m_lyricsSize; + QAction *m_playAction; QAction *m_recordAction; QAction *m_zoomInAction; diff --git a/main/test/TestCompactLayout.h b/main/test/TestCompactLayout.h index 933301ab..07c857db 100644 --- a/main/test/TestCompactLayout.h +++ b/main/test/TestCompactLayout.h @@ -24,6 +24,7 @@ #include "../MainWindow.h" #include "../Analyser.h" #include "../CompactLayout.h" +#include "../LyricsTrack.h" #include "version.h" @@ -63,6 +64,7 @@ class CompactTestWindow : public MainWindow QWidget *overview() { return m_overview; } SingingTakes *takes() { return m_takes; } sv::ViewManager *viewManager() { return m_viewManager; } + LyricsTrack *lyrics() { return m_lyrics; } Analyser *analyser() { return m_analyser; } // What the compact toolbar should hold, in order, but for the take @@ -736,6 +738,46 @@ private slots: if (QTest::currentTestFailed()) return; QCOMPARE(m_window->viewManager()->getPlotScale(), 2.0); } + + // View > Lyrics Size (LyricsSize): the same, for the scale the + // lyrics track gives its layer + void lyrics_size_in_the_view_menu() { + QSettings().remove("MainWindow/lyricssize"); + openWindow(); + if (QTest::currentTestFailed()) return; + + QMenu *lyricsSize = nullptr; + for (QAction *menu: m_window->menuBar()->actions()) { + if (!menu->menu() || !menu->menu()->actions() + .contains(m_window->compact()->getAction())) continue; + for (QAction *action: menu->menu()->actions()) { + if (action->menu() && action->text() == "Lyrics Si&ze") { + lyricsSize = action->menu(); + } + } + } + QVERIFY(lyricsSize); + + QStringList texts, checked; + for (QAction *action: lyricsSize->actions()) { + texts << action->text(); + if (action->isChecked()) checked << action->text(); + } + QCOMPARE(texts, QStringList({ "35%", "50%", "65%", "80%", "100%" })); + QCOMPARE(checked, QStringList({ "100%" })); // 50% on Android + QCOMPARE(m_window->lyrics()->getTextScale(), 1.0); + + lyricsSize->actions().at(1)->trigger(); + QCOMPARE(m_window->lyrics()->getTextScale(), 0.5); + + m_window->doCloseSession(); + delete m_window; + m_window = nullptr; + openWindow(); + if (QTest::currentTestFailed()) return; + QCOMPARE(m_window->lyrics()->getTextScale(), 0.5); + QSettings().remove("MainWindow/lyricssize"); + } }; #endif diff --git a/meson.build b/meson.build index 2325bd63..f827e793 100644 --- a/meson.build +++ b/meson.build @@ -1226,6 +1226,7 @@ tony_app_files = [ 'main/CoverageStrip.cpp', 'main/LyricsTrack.cpp', 'main/LyricsEditor.cpp', + 'main/LyricsSize.cpp', 'main/Analyser.cpp', 'main/MainWindow.cpp', 'main/NetworkPermissionTester.cpp', @@ -1260,6 +1261,7 @@ tony_app_moc_headers = [ 'main/CoverageStrip.h', 'main/LyricsTrack.h', 'main/LyricsEditor.h', + 'main/LyricsSize.h', 'main/PlotSize.h', 'main/TouchGestures.h', 'main/TouchMenuStyle.h', diff --git a/repoint-lock.json b/repoint-lock.json index b45fdc7d..448770a1 100644 --- a/repoint-lock.json +++ b/repoint-lock.json @@ -7,7 +7,7 @@ "pin": "959ea1a749a93dc0c9d01aec4a37671aff9e686f" }, "svgui": { - "pin": "adb3502afb303f3caf6a04f77dcf0acb605d85b9" + "pin": "23e1365ca62faf73fe99511f1920280897dc49a3" }, "svapp": { "pin": "f6da7b793f82d569472b284f97c3f7637a1c031f" From ae78793df87eca65e871c823a98cfa2b05bf58f5 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:49:26 +0000 Subject: [PATCH 229/275] docs: work order for a12c, the calibrate audio dialog small and out of the way Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 40 +++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index a95a2933..98d9cce6 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -185,6 +185,9 @@ builds happen in the container.) merged in for Calibrate Audio at 48 kHz.) - A12 — Calibrate Audio on the phone. Done (the dialog's size and layout left to A12c, by the lead's change of scope). +- (Lead, 2026-09-26: View > Lyrics Size, 35-100 %, 50 % by default on Android; the + size drawn is logged.) +- A12c — The Calibrate Audio dialog: small, and out of the way while a check runs. - A12b — The dev run on the phone. - A8 — Documentation pass. @@ -637,6 +640,43 @@ Tests on the desktop as far as they go (the fake device at 48 kHz, the key and s rules in core); the JNI for the route compiles only for Android. Say what the phone test should do and send back. +### A12c — The Calibrate Audio dialog: small, and out of the way while a check runs + +The user's phone test of Calibrate Audio (2026-09-26, the APK before A12): the text could +be selected but not copied (A12 added Copy); "the calibrate modal is too large and the +buttons on the bottom are off screen. The modals ought to be resized much smaller. Using +a smaller font-size would be acceptable too. I noticed this on desktop too ... the modal +blocks the view of the test happening. The modal could be made very small, just a small +progress bar and small status text could be shown in the corner. Tapping that could expand +to allow canceling an ongoing test so that possibility doesn't go away. Once the test +finishes the modal would expand again. This kind of change would be done for both desktop +and Android but it's mainly for Android." + +- **Smaller**: every page of `CalibrateAudioDialog` fits a landscape phone (the log's + 923 x 411 logical px, less the safe area margins 58,24,48,0) with all its buttons on + screen; text that does not fit scrolls. A smaller font on Android is acceptable. The + desktop dialog gets smaller too; keep it readable. +- **Out of the way during a check**: when a check starts, the dialog collapses to a small + indicator in a corner of the window (a progress bar and one line of status: the step, + the punch-in, the time left) that does not cover the pane where the takes are drawn. + Tapping or clicking it expands the dialog to the progress page, with Cancel, and a way + back to small. When the check ends (done, failed or cancelled), the dialog expands by + itself to the result page. The dev run (A12b) will use the same progress page and + indicator for its stages. +- **Behaviour kept**: not modal; closing it while a check runs cancels the check; Copy, + Save Report..., Use this latency, Check Again; the instructions and result texts A12 + wrote. A12 changed the dialog only by adding Copy and Save Report... on the result page, + the phone instructions, NoSignal and Fading advice for a phone, a "Streams:" row, and + asking for the microphone before Start. +- **The other dialogs**: open each of Tony's other dialogs in a window of the phone's size + with the compact layout (message boxes, the take name question, Edit Display Extents, + the lyrics dialogs, Preferences) and list in the report which do not fit. Fix only the + generic cause, if there is one (a font or margin that the compact layout could set for + all dialogs); the rest is for the user to choose. +- Tests: the pages' sizes against a phone-sized window; collapse at the start, expand on + a tap and at the end, Cancel reachable while collapsed by expanding; closing still + cancels. + ### A12b — The dev run on the phone After A12: the dev checks (`main/dev/`, compiled in the Android build, which is From 5734fe73af2c32765faf76f939e6d25a8e725245 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:57:04 +0000 Subject: [PATCH 230/275] test: android ci builds the apk with the scripts of deploy/android The job runs setup-toolchain, build-qt, build-deps, build-tony and build-apk as the cloud container does, and uploads the debug APK. Qt for Android and the C libraries are cached, each keyed on the script that builds it and the toolchain's, and saved right after it is built so that a later failure does not cost the build again. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/android.yml | 94 +++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 .github/workflows/android.yml diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml new file mode 100644 index 00000000..1271d875 --- /dev/null +++ b/.github/workflows/android.yml @@ -0,0 +1,94 @@ +name: Android CI + +on: [push, pull_request] + +# The APK, built by the scripts in deploy/android/ as the cloud container +# builds it (docs/building.md). Qt for Android and the C libraries are +# built from source there, which takes most of an hour: each is cached, +# keyed on the script that builds it and the toolchain's, and saved as +# soon as it is built, so that a later step's failure does not cost it + +env: + # The runner's own SDK and NDK are other versions; everything that + # looks for one in the environment is to find the scripts' + ANDROID_HOME: /opt/android/sdk + ANDROID_SDK_ROOT: /opt/android/sdk + ANDROID_NDK: /opt/android/sdk/ndk/27.2.12479018 + ANDROID_NDK_HOME: /opt/android/sdk/ndk/27.2.12479018 + ANDROID_NDK_ROOT: /opt/android/sdk/ndk/27.2.12479018 + ANDROID_NDK_LATEST_HOME: /opt/android/sdk/ndk/27.2.12479018 + +jobs: + build: + + runs-on: ubuntu-24.04 + + steps: + - uses: actions/checkout@v4 + with: + # build-apk.sh counts the commits for the version code + fetch-depth: 0 + # The runner's own Android SDK and .NET take disk the Qt build needs + - name: free-disk + run: | + df -h / + sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /usr/local/.ghcup + df -h / + - name: prepare + run: | + sudo mkdir -p /opt/android + sudo chown "$(id -u):$(id -g)" /opt/android + - name: install-packages + run: | + sudo apt-get update + sudo apt-get install -y git mercurial smlnj + - name: repoint + run: ./repoint install + - name: setup-toolchain + run: deploy/android/setup-toolchain.sh + - name: restore-qt + id: qt + uses: actions/cache/restore@v4 + with: + path: /opt/android/qt + key: android-qt-${{ hashFiles('deploy/android/build-qt.sh', 'deploy/android/setup-toolchain.sh') }} + - name: build-qt + run: deploy/android/build-qt.sh + - name: save-qt + if: steps.qt.outputs.cache-hit != 'true' + uses: actions/cache/save@v4 + with: + path: /opt/android/qt + key: ${{ steps.qt.outputs.cache-primary-key }} + - name: restore-deps + id: deps + uses: actions/cache/restore@v4 + with: + path: /opt/android/deps-arm64-v8a + key: android-deps-${{ hashFiles('deploy/android/build-deps.sh', 'deploy/android/setup-toolchain.sh') }} + - name: build-deps + run: deploy/android/build-deps.sh + - name: save-deps + if: steps.deps.outputs.cache-hit != 'true' + uses: actions/cache/save@v4 + with: + path: /opt/android/deps-arm64-v8a + key: ${{ steps.deps.outputs.cache-primary-key }} + - name: build-tony + run: deploy/android/build-tony.sh + - name: build-apk + run: deploy/android/build-apk.sh + - name: upload-apk + uses: actions/upload-artifact@v4 + with: + name: Tony-debug-apk + path: build-android/apk/Tony-debug.apk + if-no-files-found: error + # The scripts show the end of a failing log; the whole of each is here + - name: upload-logs + if: failure() + uses: actions/upload-artifact@v4 + with: + name: android-logs + path: /opt/android/logs + if-no-files-found: ignore From 2fe7ea26a2199e0185c5a78bc7ac785426f544bf Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 16:59:57 +0000 Subject: [PATCH 231/275] docs: the audio drivers as built, and what to try on windows audio-drivers.md is the page for the drivers as built, and what the runs on the user's PC are to settle. The manual checklist gets the menus and a Calibrate Audio and dev run per driver; building.md how the fork's Windows code is checked from Linux; forks.md how a bqaudioio cloned from the mirror moves to the fork. The work orders are gone. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01SDVZsDa83T6H9RKkbeBVzV --- AGENTS.md | 5 +- README.md | 3 + docs/README.md | 3 +- docs/architecture.md | 4 +- docs/audio-drivers-work-orders.md | 216 ----------------------- docs/audio-drivers.md | 278 ++++++++++++++++++------------ docs/building.md | 30 +++- docs/calibrate-audio.md | 88 +++++----- docs/forks.md | 15 +- docs/manual-checklist.md | 83 +++++++-- docs/open-points.md | 30 ++-- docs/recording.md | 3 +- 12 files changed, 344 insertions(+), 414 deletions(-) delete mode 100644 docs/audio-drivers-work-orders.md diff --git a/AGENTS.md b/AGENTS.md index e0239cd3..93ee2a23 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,7 +19,8 @@ code. | [docs/recording.md](docs/recording.md) | touching `record()`, the Stop path, latency, pre-roll, the live tracker | | [docs/takes.md](docs/takes.md) | touching takes, the audio swap, ranged analysis, undo, the coverage strip, save/restore | | [docs/calibrate-audio.md](docs/calibrate-audio.md) | touching Calibrate Audio (`AudioCheckRunner`, `CalibrateAudioDialog`), the dev checks (`main/dev/`) or the measured latency (`LatencyCheck`, `LatencyCalibration`) | -| [docs/forks.md](docs/forks.md) | needing a change in `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/` | +| [docs/audio-drivers.md](docs/audio-drivers.md) | touching the Audio Driver or Audio Latency menus (`AudioDriverSettings`, `AudioDriverMenus`), how the device is opened (`MainWindow::createAudioIO()`), or the bqaudioio fork | +| [docs/forks.md](docs/forks.md) | needing a change in `svcore/`, `svgui/`, `svapp/`, `bqaudiostream/`, `bqaudioio/` | | [docs/open-points.md](docs/open-points.md), [docs/manual-checklist.md](docs/manual-checklist.md) | choosing what to do next, or saying what the user should try by hand | | [docs/mobile-port.md](docs/mobile-port.md), then [docs/port-android.md](docs/port-android.md) or [docs/port-sailfish.md](docs/port-sailfish.md) | starting or working on a phone port | @@ -77,7 +78,7 @@ first). The rest is in [docs/building.md](docs/building.md#building-on-linux). Never read them whole: search, then read a range. - `svcore/`, `svgui/`, `svapp/`, `pyin/` and the other top-level library directories are **separate git repositories, gitignored here**, so ripgrep-based search tools skip them - unless given the directory explicitly. Four are forks that may be changed + unless given the directory explicitly. Five are forks that may be changed ([docs/forks.md](docs/forks.md)); the rest are upstream and stay untouched. A sub-agent does not edit a fork: it reports the change it needs. diff --git a/README.md b/README.md index c895027f..c8de0f57 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,9 @@ orange pitch track to compare with it. status bar counts you in, and nothing sung during the lead-in is kept * an optional "record into the selection only": select the phrase, and the recording starts and stops at the ends of the selection by itself + * on Windows, Playback -> Audio Driver chooses MME, DirectSound or WASAPI, and + Playback -> Audio Latency how much latency is asked of it (10 to 200 ms); + each driver keeps its own devices, latency and measured round trip * a take is one recording or many. A strip along the bottom of the pane shows where there is singing and where there is not; recording over a part of it replaces just that part, and only the part that changed is analysed again diff --git a/docs/README.md b/docs/README.md index 646a1ae0..b39f0288 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,7 +18,8 @@ methods, and they do not tell the story of fixed bugs. | [takes.md](takes.md) | Takes: decisions, the audio swap, ranged analysis and merge, undo, the coverage strip, files and sessions, limitations | | [forks.md](forks.md) | The `jhhr/*` library forks: how to change one, what each adds, known defects | | [open-points.md](open-points.md) | Decisions waiting for the user, things not built, weak spots | -| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes, starting with the device check (Calibrate Audio with the dev checks). Tried so far: Calibrate Audio and part of a dev run on the user's PC, and the looks from cloud screenshots; the rest not yet | +| [manual-checklist.md](manual-checklist.md) | What needs a real device, real ears or real eyes, starting with the device check (Calibrate Audio with the dev checks, on each driver) and the driver menus. Tried so far: Calibrate Audio and part of a dev run on the user's PC, and the looks from cloud screenshots; the rest not yet | | [mobile-port.md](mobile-port.md) | Porting to a phone: decisions, what in the code any port depends on, Android against Sailfish OS | | [port-android.md](port-android.md), [port-sailfish.md](port-sailfish.md) | Platform facts with sources, the work, and the first test port for each | | [calibrate-audio.md](calibrate-audio.md) | Calibrate Audio, which measures the round trip through an earcup held to the mic, and the dev checks that settle the checklist's device items: what they do, what each verdict and check means, which numbers to send back, design, tests, decisions, open points, the user's runs | +| [audio-drivers.md](audio-drivers.md) | Playback > Audio Driver and Audio Latency on Windows (MME, DirectSound, WASAPI): why, decisions, facts, the design in Tony, tests, what the measurements on the user's PC are to settle, open points | diff --git a/docs/architecture.md b/docs/architecture.md index 8fb0c6f9..05ddee54 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -32,8 +32,8 @@ only what they need: | Library | Rule | Contents | | --- | --- | --- | -| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `LiveDotsFeed`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LyricsTtml`, `LyricsEdit`, `LatencyUtils.h`, `LatencyCheck`, `LatencyCalibration`, `TakeDiff` | -| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `LyricsEditor`, `TakeCommands`, `TakeLayers`, `PaneUtils`, `AudioCheckRunner`, `CalibrateAudioDialog`; in development builds only, `main/dev/` (`DevChecks`, `TakeObserver`) | +| `tony_core` | No GUI, no document, no layers. Unit-tested without a window. | `RealtimePitchTracker`, `LiveDotsFeed`, `Coverage`, `TakeAudio`, `TakeEvents`, `SingingTakes`, `TakesFile`, `TakeTiming`, `Lyrics`, `LyricsTtml`, `LyricsEdit`, `LatencyUtils.h`, `LatencyCheck`, `LatencyCalibration`, `TakeDiff`, `AudioDriverSettings` | +| `tony_app` | Anything that touches a `Document`, a `Layer` or a window. | `MainWindow`, `Analyser`, `AlternatePitchTrack`, `CoverageStrip`, `LyricsTrack`, `LyricsEditor`, `TakeCommands`, `TakeLayers`, `PaneUtils`, `AudioCheckRunner`, `CalibrateAudioDialog`, `AudioDriverMenus`; in development builds only, `main/dev/` (`DevChecks`, `TakeObserver`) | Development builds are every build type but `release`: they define `TONY_DEV_CHECKS` and compile `main/dev/`, and everything that uses it elsewhere is inside `#ifdef diff --git a/docs/audio-drivers-work-orders.md b/docs/audio-drivers-work-orders.md deleted file mode 100644 index edaa45aa..00000000 --- a/docs/audio-drivers-work-orders.md +++ /dev/null @@ -1,216 +0,0 @@ -# Audio drivers: work orders for phase agents - -You are one of a line of agents, each building **one small phase** of the driver project -([audio-drivers.md](audio-drivers.md)). A lead reviews your work when you report back, -and commits it. You have no memory of earlier phases; what you need is here. This file is -working memory for the branch `feat/wasapi`, and the documentation phase removes it. - -**Your context is the budget.** Aim to finish well under 200k tokens. The rules below -say how. They are about not reading huge files whole and not maintaining big documents; -they are **not** a licence to skip what you need to understand. Careful, correct work -comes first. - -## 1. What to read, and what not to - -1. This file, all of it. -2. `AGENTS.md` at the repository root. Its rules apply, except that the **build and test - commands in section 2 below replace its Windows ones**. -3. `docs/audio-drivers.md` (the spec, short): all of it. -4. The docs your work order names, by section: search for the heading and read that - range. `docs/calibrate-audio.md` is about 600 lines; `docs/testing.md` about 350. -5. Code: - - `main/MainWindow.cpp` is over 6000 lines and `main/test/TestRecordWorkflow.h` over - 8000. **Never read them whole**: search for the function, then read that range. - - Before writing a test, read an existing test next to where yours will go and copy - its shape. - -## 2. Rules - -**Scope** - -- Build your phase only. Where the spec is silent, choose the simpler option and say so. - Where it is **wrong or impossible**, do not improvise another design: finish what can - be finished, leave the tree building and green, and report. -- **Do not edit** any top-level library directory (`svcore/`, `svgui/`, `svapp/`, - `bqaudioio/`, `bqaudiostream/`, `pyin/`, …). They are separate repositories, gitignored - here. If one needs a change, report exactly which; the lead makes it. -- Match the surrounding code: naming, comment density, idiom. Comments say why, in plain - words. Every new source file starts with the project's GPL header. -- **The user builds on Windows** with MinGW and Qt 6.11. Nothing specific to Linux; and - no identifiers named `near`, `far`, `min`, `max`, `ERROR`, `IN` or `OUT` (Windows - headers define them as macros). - -**Build and test.** This container builds in `build/` with Qt 6.11 (conda-forge). - -- Build, from the repository root: - - ninja -j 4 -C build tony pyin.so test-tony-core test-tony-app test-tony-dev > tmp/build.log 2>&1; echo "exit:$?" >> tmp/build.log; tail -5 tmp/build.log - - Search the log for `error:`; never read it whole. No `.exe`. `pyin.so` must be built: - without it every app test that waits for an analysis hangs. -- Named tests while working, from `build/`: - - mkdir -p ../tmp/tl && rm -f ../tmp/tl/*.txt - TONY_TEST_LOG_DIR=../tmp/tl ./test-tony-app some_test other_test > ../tmp/test.log 2>&1 - grep -a "^FAIL\|^XFAIL\|^ Loc\|^Totals" ../tmp/tl/*.txt - - A name goes to every suite of the executable; the others report it unknown, so the - exit status of a run with names means nothing. Read results from the per-suite files. -- Whole suites, **once at the end**, from the repository root: - - deploy/linux/run-tests.sh test-tony-core - deploy/linux/run-tests.sh test-tony-app # about a minute and a half - deploy/linux/run-tests.sh test-tony-dev # about a minute - - Each runs eight processes and prints a summary with every failure. Never run two at - once, nor one while building: the app tests record in real time. -- Every behaviour gets a test that can fail. Show it for the two or three that matter by - breaking the code for a moment; undo the break **by hand**, never with `git checkout` - or `git restore`. -- Do not weaken or delete a test to get green. If one is wrong because the behaviour was - meant to change, change it and say so. -- App tests run in real time against `FakeAudioIO`: keep them to seconds. - -**Git.** Do not commit, push, or stage. The lead does, after review. Never `git add -A`. -Edit documents with the Edit and Write tools, not shell one-liners. - -**Docs.** Fix any statement in `docs/` your change makes false, in the page that makes -it. The big documentation pass is W4's; before that, only such fixes. - -**Report** (under 60 lines): what you built, by file; the three `run-tests.sh` summaries -verbatim; which tests you saw fail when you broke the code; decisions you took where the -spec was silent; anything fragile, unfinished, or needed from a library. - -## 3. State of the code - -- `feat/wasapi` starts from `feat/tonyandroid` with `default` merged in: Calibrate Audio - and the dev checks, the Android port, and a device at another rate than the - reference's handled (A1: the recording resampled before the splice, `TakeTiming` - converting; A11: the cursor keeps the reference's pace). -- All three suites green, sharded and in one process each. No expected failures left. -- W1 and W2 are done (the spec's §6). bqaudioio is the fork, at its `feat/wasapi`: the - implementations `mme`, `directsound` and `wasapi` exist **on Windows only**; on Linux - `AudioFactory::getImplementationNames()` is as before (`pulse`, `port`, `jack`). So a - test of anything that lists or chooses a driver cannot get the list from the factory - here: give `MainWindow` a virtual that returns it, as `createAudioIO()` is virtual for - `FakeAudioIO`, and have `TestMainWindow` override it. -- `AudioFactory::setSuggestedLatency(seconds)` exists (0 or less: the default, 0.2 s), - for streams opened after the call. - -## 4. Work orders - -### W1 — Calibrate Audio at 48 kHz - -Read: calibrate-audio.md §1, §3 (the verdicts), §8 (what the report says) and the part of -§10 or later that lists the device facts ("Device rate"); `main/AudioCheckRunner.h` and -`.cpp` around `rateMismatch`; `main/CalibrateAudioDialog.cpp` around `rateMismatch`; -`main/test/TestAudioCheck.h`, `check_flags_a_rate_mismatch` and the test that stores a -figure and checks again with it. - -- `AudioCheckResult::rateMismatch` and everything it decides go: a check on a device at - another rate than the reference's is judged, and its figure kept, like any other. -- The report still names the device's rate where it differs from the reference's (the - dialog's details, and `DevChecks.txt` if it prints the result), as a fact, not a fault. -- Tests, with `FakeAudioIO` at 48000 Hz as a loopback with a known delay: - - the check finds the round trip, its verdict is Ok, and the figure can be stored with - the key's rate 48000; - - a second check with the stored figure places the takes (the offsets near 0); - - `check_flags_a_rate_mismatch` becomes one of those; the details test that builds a - result at 48000 Hz changes to the new wording. -- Docs: calibrate-audio.md's statements about the device rate that A1 and this phase - made false (§1 "The device's sample rate is not checked", §10's driver project step 1, - the device facts on `TakeAudio::splice()` and on the check naming the mismatch). - -### W3 — Driver and latency menus - -Read: the spec's §2–§4; recording.md "Latency"; calibrate-audio.md §5 (the key and the -figure in use) and §8 (what `DevChecks.txt` says); in `MainWindow.cpp`, -`audioDeviceSettingKey()`, `audioImplementationName()`, `buildAudioDeviceMenu()`, -`rescanAudioDevices()`, `audioDeviceSelected()` and where the Playback menu makes the two -device submenus; svapp's `MainWindowBase::createAudioIO()` and `recreateAudioIO()`, and -where `createAudioIO()` is first called. - -- **"Audio Driver" submenu** in the Playback menu, before the two device submenus: one - checkable entry per driver implementation the factory reports (`mme`, `directsound`, - `wasapi`, in that order, named by `AudioFactory::getImplementationDescription()`), only - when there are at least two. The list comes from a virtual of `MainWindow` (see §3), so - that tests can give it. Choosing one: `Preferences/audio-target` becomes its name, the - latency for it is applied, playback stops, the device rate is forgotten as - `audioDeviceSelected()` does, and the IO is recreated; the device submenus then list - that driver's devices under its own keys. Greyed out during a take and while a check - runs, as Calibrate Audio is. -- **The default.** Where `audio-target` is empty (or `auto`) and `mme` is reported, Tony - sets `mme` before the first IO is made, and copies the devices saved without a suffix - to the `-mme` keys where those are unset. It then does nothing again. Find out where - the first IO is made: this has to happen before it. -- **"Audio Latency" submenu**, next to it and shown with it: 10, 20, 50, 100 and 200 ms, - checkable, kept per driver (`Preferences/audio-latency-`, in seconds; - 200 ms when unset, which is what every stream asked for until now). Applied with - `AudioFactory::setSuggestedLatency()` before every IO is made, and choosing one - recreates the IO. Greyed out as the driver menu is. -- **Reports.** `DevChecks.txt` names the driver and the latency asked for, next to the - devices it names, so that runs on two drivers can be told apart. The Calibrate Audio - dialog's details name the driver where they name the devices. -- Tests (app suite; a few, short): the menu from a given list, and none from one entry; - choosing WASAPI writes the setting, recreates the IO and moves the device keys to the - `-wasapi` suffix; greyed out during a take; the default and the copy of the devices, and - that a set `audio-target` is left alone; the latency kept per driver and applied (make - what was applied readable); a figure measured under one driver is not the one in use - under another (the key has the implementation already: test it through the window). - Show the default's test and the per-driver latency's test fail by breaking the code. -- Docs: recording.md "Latency" (the latency asked for), calibrate-audio.md where it says - the reported pair comes from `suggestedLatency = 0.2`. - -### W4 — Docs - -Read: this file's log; the spec; `git log --stat bcfbfdd..HEAD` (the project's commits); -the docs named below, by section. Code only to check a statement. - -- `docs/audio-drivers.md` becomes the page for the drivers **as built**, as - calibrate-audio.md is for Calibrate Audio: why, the decisions, the facts, the design as - it is, the tests, what W5 is to measure and what to send back, open points. Plan - wording ("will", "phase W3 does") goes; §5's phases become a short "state" of what is - done and what W5 still is. -- `docs/manual-checklist.md`: items for the driver and latency menus on Windows (first - start: MME ticked, devices kept; WASAPI lists its own devices; a latency change reopens - the device; the report names driver and latency), and W5's runs: Calibrate Audio and a - dev run on MME at 200 ms and on WASAPI at 20 and 10 ms, the `DevChecks.txt` of each sent - back. Where the checklist's device section already has the dev run, extend it rather - than repeat it. -- `docs/calibrate-audio.md`: §11's test list (W1's and W3's tests); §10's driver project - pointing to audio-drivers.md instead of repeating it. -- `docs/open-points.md`, `docs/architecture.md` (the libraries' contents), - `docs/README.md` and `AGENTS.md`'s "Read / Before" table (a row for audio-drivers.md: - touching the driver or latency menus, `AudioDriverSettings`, `AudioDriverMenus`, or the - bqaudioio fork), the root `README.md`'s feature list (the two menus, one line). -- `docs/building.md`, "Building on Linux": how the fork's Windows-only code is checked - here — `apt-get install g++-mingw-w64-x86-64-posix`, PortAudio 19.7.0's headers - (`portaudio.h`, `pa_win_wasapi.h`, `pa_win_waveformat.h`) from - `raw.githubusercontent.com/PortAudio/portaudio/v19.7.0/include/`, and - `x86_64-w64-mingw32-g++ -std=c++17 -fsyntax-only -DHAVE_PORTAUDIO -I - -Ibqaudioio/bqaudioio -Ibqaudioio/src -Ibqvec bqaudioio/src/PortAudioIO.cpp` (and - `AudioFactory.cpp`); that is how W2 was checked (both clean, and the WASAPI block seen - in the preprocessed output). -- Remove this file (`docs/audio-drivers-work-orders.md`) and every link to it. -- No narratives of fixed bugs, no commit hashes, no lists of members. - -## 5. Log - -Capped at 25 lines per entry. Newest last. - -**W1** (agent; lead reviewed and committed). The rate-mismatch verdict went; two 48 kHz -tests (measure, then place with the kept figure), both seen failing with the mismatch -blocking put back, and the second with the kept figure turned into frames at the -session's rate. A 48 kHz check measures about 1.1 ms over the fake's delay: bqaudioio's -`ResamplerWrapper` holds that back, unreported, and a real device has it too. - -**W2** (lead). The fork as the spec's §4; pinned. Tony's build unchanged on Linux; the -suites green against the fork. - -**W3** (agent; lead reviewed and committed). `AudioDriverSettings` (core) and -`AudioDriverMenus` (app); `MainWindow::createAudioIO()` on desktop names the default and -applies the latency, then `openAudioIO()`, which the tests override instead of -`createAudioIO()`. The first device opens lazily (first file, first take, recreate), never -in the constructor. Seen failing: the default, the per-driver latency (twice), the -greying. Not tested: that playback stops on a choice. For W4: the manual checklist has no -items for the menus yet; calibrate-audio.md §11's test list. diff --git a/docs/audio-drivers.md b/docs/audio-drivers.md index 1157dcb5..37dd2c28 100644 --- a/docs/audio-drivers.md +++ b/docs/audio-drivers.md @@ -1,141 +1,195 @@ # Audio drivers: WASAPI next to MME -Plan and state of the driver project on the branch `feat/wasapi` (from `feat/tonyandroid`, -to be merged back into it). What is built is marked **Done** in §6; the reasons stay here. +On Windows, **Playback > Audio Driver** chooses which of Windows' audio APIs Tony plays and +records through (MME, DirectSound or WASAPI), and **Playback > Audio Latency** how much +latency it asks that driver for (10 to 200 ms). Each driver keeps its own devices, its own +latency and its own measured round trip. MME, which every stream went through before there +was a choice, is the default until measurements on the user's PC show another better. + +This page is the design as built: why, the decisions, the facts it rests on, the design in +Tony, the tests, what the measurements on the user's PC are to settle, and what is open. +The bqaudioio fork's side is in [forks.md](forks.md#bqaudioio); how the latency asked for +differs from the one takes are placed with, in [recording.md](recording.md#latency); what +to try by hand, in [manual-checklist.md](manual-checklist.md), sections 1 and 2. ## 1. Why - **MME is slow and unsteady.** On the user's PC Calibrate Audio measured a round trip of - about 300 ms (bqaudioio opens every stream with `suggestedLatency = 0.2` on both sides), - and the offset between input and output moved by about 13 ms from one take to the next - (every take restarts the stream), 5 to 20 ms over three calibrations, while the sweeps - within one take agreed to 0.3 ms ([calibrate-audio.md](calibrate-audio.md), §10). No - one stored figure then places every take; the dev checks' items 1 and 2 fail on it. + about 300 ms, with every stream asking for 0.2 s on both sides, and the offset between + input and output moved by about 13 ms from one take to the next (every take restarts the + stream), 5 to 20 ms over three calibrations, while the sweeps within one take agreed to + 0.3 ms ([calibrate-audio.md](calibrate-audio.md), §10). No one stored figure then places + every take; the dev checks' items 1 and 2 fail on it. - **WASAPI** is the Windows audio engine itself; MME and DirectSound are layers over it. Its shared mode mixes with other programs as MME does, with buffers as small as the engine's period (typically 10 ms). Whether it is steadier across stream restarts is what - the measurements in §6, W5 are for. -- **Today the device menus mix all host APIs.** PortAudio lists every device once per host - API (MME, DirectSound, WASAPI, WDM-KS); bqaudioio's `getDeviceIndex()` takes the first - name that matches, which is usually MME's, but MME cuts names to 31 characters, so a - long name picked from the menu can only match WASAPI's or WDM-KS's entry, which then - opens at the Windows mixer's rate (usually 48 kHz). That is where the "48 kHz device" - of the calibrate-audio work came from. - -## 2. Decisions + the measurements of §7 are for. +- **The latency asked for was fixed at 0.2 s.** At that figure WASAPI would keep MME's + buffers and gain nothing. +- **The device menus mixed every host API.** PortAudio lists every device once per host + API (MME, DirectSound, WASAPI, WDM-KS). bqaudioio's `port` takes the first name that + matches, usually MME's, but MME cuts names to 31 characters, so a long name picked from + the menu could only match WASAPI's or WDM-KS's entry, which then opened at the Windows + mixer's rate (usually 48 kHz). That is where the "48 kHz device" of the calibrate-audio + work came from. Each driver now lists its own devices only. + +## 2. What the user sees + +- **Audio Driver**, in the Playback menu before the two device submenus: MME, DirectSound, + WASAPI, in that order, the one in use ticked. **Audio Latency** next to it: 10, 20, 50, + 100 and 200 ms, the one chosen for that driver ticked, 200 ms where none has been. Both + are shown only where two drivers or more are built in, which is on Windows; on Linux and + Android nothing changed. +- **Choosing a driver** stops playback and opens the device again through it, with the + devices chosen under it (the driver's own default devices where none are) and its + latency. The device submenus then list that driver's devices and write its keys; going + back to another driver finds its devices as they were left. **Choosing a latency** stops + playback and opens the device again. Choosing what is ticked does nothing. +- Both are **greyed out during a take and while a check runs**: either would open the + device again under the take. +- **The first start with the menus** names MME and carries the devices chosen before over + to it. A device name MME does not list (a long one, which only WASAPI or WDM-KS had) + shows as "(not connected)", ticked, and MME's own default device opens instead. +- **A round trip measured before the menus** was kept under no driver, and is not carried + over as the devices are: the Playback menu's latency line reads "driver's figure" until + Calibrate Audio is run, and its figure kept, on each driver used. +- **The reports** name the driver: Calibrate Audio's instructions and result page + ("Driver:"), and `DevChecks.txt`'s header ("Audio driver:", and "Latency asked for:"), + so that runs on two drivers, or at two latencies, can be told apart. + +## 3. Decisions | Decision | By, when | Why | | --- | --- | --- | | Work on `feat/wasapi`, branched from `feat/tonyandroid` after the calibrate-audio merge, merged back when done | user, 2026-09-26 | Another session works on `feat/tonyandroid` | | **MME stays the default** until Calibrate Audio and a dev run on the user's PC show WASAPI better | user, 2026-09-26 | Not to change what works before it is measured | -| The driver changes are in the fork `jhhr/bqaudioio`, made by the lead; Tony pins it | user, 2026-09-26 | bqaudioio chooses the host API, the stream's rate and its latency; upstream is on sourcehut | -| **A driver is a bqaudioio implementation**: `mme`, `directsound`, `wasapi`, each PortAudio restricted to that host API; `port` stays as it is (all host APIs) | lead | Tony's device menus, the saved devices (`audio-*-device-`), svapp's `createAudioIO()` and the stored round trip (`LatencyCalibration::Key::implementation`) are already per implementation: no svapp change | -| WASAPI in **shared** mode, with `paWinWasapiAutoConvert` on both sides | lead | The input's and the output's mixers can run at different rates, and the stream opens at the output's; shared mode leaves other programs' sound alone. Exclusive mode is not in this project | -| A device at 48 kHz needs nothing more for placement | `feat/tonyandroid` A1 | The recording is resampled to the reference's rate before the splice, and `TakeTiming` converts device frames (§3). Calibrate Audio's "rate mismatch" verdict is out of date and goes (W1) | -| `suggestedLatency` settable per driver, chosen in a Playback > Audio Latency submenu (10, 20, 50, 100, 200 ms; 200 when unset) | lead, 2026-09-26 | At 0.2 s WASAPI would keep MME's buffers and gain nothing; a choice lets W5 compare | +| The driver changes are in the fork `jhhr/bqaudioio`; Tony pins it | user, 2026-09-26 | bqaudioio chooses the host API, the stream's rate and its latency; upstream is on sourcehut | +| **A driver is a bqaudioio implementation**: `mme`, `directsound`, `wasapi`, each PortAudio restricted to that host API; `port` stays as it was (all host APIs) and is not offered | lead | Tony's device menus, the saved devices, svapp's `createAudioIO()` and the stored round trip (`LatencyCalibration::Key::implementation`) were already per implementation: no svapp change | +| WASAPI in **shared** mode, with `paWinWasapiAutoConvert` on both sides | lead | The input's and the output's mixers can run at different rates, and the stream opens at the output's; shared mode leaves other programs' sound alone | +| A device at 48 kHz needs nothing more for placement | `feat/tonyandroid` | The recording is resampled to the reference's rate before the splice, and `TakeTiming` converts device frames; Calibrate Audio judges such a device like any other ([calibrate-audio.md](calibrate-audio.md), §1) | +| The latency chosen per driver from 10, 20, 50, 100 and 200 ms; 200 when unset | lead, 2026-09-26 | 200 ms is what every stream asked for before; a choice lets the measurements compare | +| The menus only with two drivers or more | lead | One driver is no choice; elsewhere the device menus are all there is | +| MME named where no driver is, before the first device opens and before the Playback menu shows the device menus, not at start-up | lead | With four PortAudio implementations the device menus have no driver to list the devices of; the first device opens lazily, with the first file or take | -## 3. Facts checked +## 4. Facts checked - **PortAudio on the Windows build** is MSYS2's `mingw-w64-x86_64-portaudio` 19.7.0, built with CMake's defaults: `PA_USE_WMME`, `PA_USE_DS`, `PA_USE_WASAPI` and `PA_USE_WDMKS` are all ON, and `pa_win_wasapi.h` is installed. Its `PaWasapiStreamInfo` has `paWinWasapiAutoConvert` (`1 << 6`). (MSYS2's PKGBUILD and PortAudio's `v19.7.0` `CMakeLists.txt`, 2026-09-26.) An ASIO build is a separate package, not used. -- **bqaudioio** (`src/PortAudioIO.cpp`, at the pin `017ab3ed3a33`, git `7ab6de9`, which is - also `jhhr/bqaudioio`'s `master`): - - `getDeviceNames()` lists every device of every host API with input (or output) - channels; `getDeviceIndex()` returns the first whose name matches, else PortAudio's - global default device (the default host API's, MME's on Windows). - - The stream opens at the output device's `defaultSampleRate`, as neither side of - svapp asks for a rate; `suggestedLatency = 0.2` and no host-API stream info on both - sides; `paFramesPerBufferUnspecified`, then 1024 if that fails, then 2×2 channels. - - `AudioFactory::getImplementationNames()` gives `pulse`, `port`, `jack` as built; - `createIO()` with no implementation named tries each in turn. +- **bqaudioio's `port`**, as upstream has it: its device lists hold every host API's + devices; `getDeviceIndex()` returns the first whose name matches, else PortAudio's + global default device (the default host API's, MME's on Windows). The stream opens at + the output device's `defaultSampleRate`, as neither side of svapp asks for a rate; + `paFramesPerBufferUnspecified`, then 1024 if that fails, then 2×2 channels. +- **The fork's drivers**: a device name their host API does not list opens that host + API's default device, as no name does. - **svapp** `MainWindowBase::createAudioIO()` reads `Preferences/audio-target` and the devices `audio-record-device` / `audio-playback-device`, suffixed `-` when one is named; `auto` means none. -- **Tony**: the Playback menu's "Audio Output Device" and "Audio Input Device" submenus - are rebuilt on `aboutToShow` (`rescanAudioDevices()`), from the implementation - `audioImplementationName()` gives: `audio-target` if set, else **the only - implementation built in**. With several PortAudio implementations that is none, and - the menus would be empty: the driver has to be named. +- **Tony's device submenus** are rebuilt on `aboutToShow`, from the implementation + `audioImplementationName()` gives: `audio-target` if set, else the only implementation + built in. On Windows there are four (`port` and the three drivers), so with no driver + named the menus would be empty: hence the default. - **`LatencyCalibration::Key`** holds `implementation` (from `audio-target`), both device - names and the recording rate: a figure is kept per driver as it is. + names and the recording rate: a figure is kept per driver, and not per latency. - **Every take restarts the stream** (`MainWindowBase::stop()` suspends, `record()` resumes: `Pa_StopStream` / `Pa_StartStream`). -- **At 48 kHz** (`feat/tonyandroid`, A1): `SingingTakes::spliceRecording()` resamples the - recording to the reference's rate, and `TakeTiming` keeps the latency and the frames - received in device frames (`recordRate`). Left: the play cursor runs fast during a take - (svgui's `ViewManager`), marked by a `QEXPECT_FAIL`; `feat/tonyandroid`'s A11 is about - it. - -## 4. Design - -### In the fork - -- `PortAudioIO` takes the host API it is restricted to (none: all, as today). Device - lists and `getDeviceIndex()` then look at that host API's devices only, and "no device - named" means **that host API's** default input and output device. -- `AudioFactory` reports `mme`, `directsound` and `wasapi` on Windows when PortAudio is - built in, and each opens a `PortAudioIO` restricted to its host API. `createIO()` with - no implementation named still tries only `port` (and the others built in), as today. -- WASAPI: a `PaWasapiStreamInfo` with `paWinWasapiAutoConvert` on both sides. -- `suggestedLatency` settable, per implementation, with 0.2 s as the default. -- Checked here by building Tony on Linux (the host API restriction with ALSA's devices is - the same code) and by cross-compiling `PortAudioIO.cpp` for Windows with MinGW-w64 - against PortAudio 19.7.0's headers; the rest only on the user's PC. - -### In Tony - -- A **Driver** submenu in the Playback menu, before the two device submenus, listing the - implementations the fork reports that are drivers (MME, DirectSound, WASAPI), when - there is more than one. Choosing one sets `audio-target`, recreates the audio IO - (not during a take or a check) and the device menus follow. -- When `audio-target` is empty and `mme` is built in, Tony names `mme` and carries the - devices saved without a suffix over to the `-mme` keys, once. -- Calibrate Audio: nothing to change for the key; its report names the driver. - -## 5. Phases - -Each leaves the tree building and all three suites green, committed and pushed to -`feat/wasapi`. - -- **W1 — Calibrate Audio at 48 kHz** (agent). The rate-mismatch verdict goes: a device at - another rate than the reference's is measured, its figure can be kept, and a second - check with it places the takes. Tests with `FakeAudioIO` at 48 kHz (a loopback with a - known delay); the old `check_flags_a_rate_mismatch` changes, as the behaviour does. - Docs: calibrate-audio.md §1, §10 and the test list. -- **W2 — The fork** (lead). As §4; then `repoint-project.json` takes bqaudioio from - `jhhr/bqaudioio` (git), `repoint-lock.json` pins it, `deploy/linux/container-setup.sh` - checks it out from there, and forks.md gets its row. -- **W3 — Driver menu** (agent). As §4, "In Tony", with app tests that name the driver - through the Preferences and read back what `createAudioIO()` was asked for. -- **W4 — Docs** (agent). recording.md, calibrate-audio.md, manual-checklist.md, - open-points.md, building.md from the code and the log. -- **W5 — Measure** (user). Calibrate Audio and a dev run on MME and on WASAPI, each at the - latencies offered; the report files back. Then the default is decided. - -## 6. State - -- **W1 Done.** Calibrate Audio judges a device at 48 kHz like any other and keeps its - figure; its details name both rates. Left: before a device's first take, the menu line, - Forget Measured Latency and the dialog look the figure up at the session's rate, so on a - 48 kHz device they show the driver's figure although a 48 kHz one is kept (takes use the - kept one). A fix needs the device's rate before the first take (svapp). -- **W2 Done.** `jhhr/bqaudioio` `feat/wasapi`, pinned; `repoint-project.json` takes it from - the fork, and `container-setup.sh` moves a checkout made from the mirror over to it. - Compiled for Linux in Tony's build and cross-compiled for Windows; run on no device yet - (the container has none). -- **W3 Done.** Playback > Audio Driver and Audio Latency (shown with two drivers or more, - so on Windows only), greyed out during a take and a check; MME named by default before - the first device opens, with the devices chosen before carried over. `DevChecks.txt` - and the Calibrate Audio dialog name the driver (and the report the latency). A figure - measured before is kept under no driver's name, so it is not carried over: calibrate - again on each driver. - -## 7. Open - +- **At 48 kHz**: `SingingTakes::spliceRecording()` resamples the recording to the + reference's rate, `TakeTiming` keeps the latency and the frames received in device + frames, and the cursor keeps the reference's pace during a take (the svgui fork). + +## 5. Design in Tony + +The fork's part (an implementation per host API, WASAPI's rate conversion, the settable +latency) is in [forks.md](forks.md#bqaudioio). Linux builds none of its Windows code; it +is checked by cross-compiling ([building.md](building.md#checking-the-forks-windows-code)). + +- **`AudioDriverSettings`** (`tony_core`): the Preferences, as plain functions: which + implementations are drivers and in what order, the latencies offered, the default's + naming and the carry-over of the devices. Its keys, all in `Preferences`: + `audio-target` (the driver), `audio-playback-device-` and + `audio-record-device-`, and `audio-latency-` in seconds, as text, as + `LatencyCalibration` keeps its figures. +- **`AudioDriverMenus`** (`tony_app`): the two submenus. Rebuilt as either opens, so that + the ticks follow the Preferences, which the other menu or the default may have changed. + A choice is written to the Preferences, then signalled; `MainWindow` does the rest. +- **`MainWindow::createAudioIO()`** (desktop): names the default driver, hands the + latency chosen for the driver to bqaudioio (`AudioFactory::setSuggestedLatency()`, for + the streams opened after), then `openAudioIO()`, which opens the device as svapp does. + Every device is opened through it: the first, and each recreate. The Playback menu names + the default too, as it opens, before the device menus read the driver. +- **A choice** stops playback, forgets the device's rate when the driver changed (another + driver may record at another rate, as another device may), and recreates the audio IO. +- **The report's latency** is the one last handed to bqaudioio, not the Preferences', so + that it says what the device was opened with. + +## 6. Tests + +- **Core** (`TestAudioDriverSettings`): the drivers picked out of any list in their order; + MME named where nothing or `auto` is, the unsuffixed devices carried over only where MME + has none, and nothing named again, nor where another driver is named or MME is not + built in; the latency kept per driver, read back from a file, 200 ms where unset or + unreadable. +- **App** (`TestAudioCheck`): the menus from a given list, in order, with the ticks, before + the device menus, and hidden with one driver; WASAPI chosen, its setting written, the + device opened again with WASAPI's devices, MME's kept, and a device then chosen written + to WASAPI's key; the default named before the first device and as the Playback menu + opens, and a named driver left alone; the latency kept per driver and handed over at + each opening; a round trip kept for MME not used under WASAPI, and used again on the way + back, with the dialog naming the driver; both menus greyed out during a take and a + check. The drivers are given through `TestMainWindow` ([testing.md](testing.md)), as + Linux's bqaudioio has none. +- **Dev** (`TestDevChecks`): the report's header names the driver and the latency. +- Seen failing with the code broken: the default, the latency per driver, the greying. +- Not tested: that a choice stops playback. Nothing here runs the fork's Windows code. + +## 7. State, and the measurements on the user's PC + +**Built**, on `feat/wasapi`: the fork, pinned; Calibrate Audio at any device rate; the two +menus, the default and the reports. The fork's Windows part is compiled by the +cross-compile only: nothing of it has run on Windows yet, and no figure has been measured +through DirectSound or WASAPI. + +**To do: the user's runs on Windows** (W5), wired headphones with one earcup against the +microphone, as in [manual-checklist.md](manual-checklist.md), section 1: + +1. The menus themselves (section 2 of the checklist). +2. Calibrate Audio, then a whole dev run, on **MME at 200 ms**: the same setting as the + earlier runs, now with every item. +3. The same on **WASAPI at 20 ms**, with WASAPI's own devices chosen. +4. The same on **WASAPI at 10 ms**. + +For each: the result page's text and that run's `DevChecks.txt`, copied aside before the +next run writes over it; and whether anything crackled or dropped out. DirectSound is +offered but is not among these runs. + +What they settle: + +- **Steadiness:** whether WASAPI's offset moves less from one take to the next than MME's + 13 ms: Calibrate Audio's verdict (Ok or Unsteady), and items 1 and 2 within ±2 ms. +- **The round trip** at each latency, measured against the driver's figure. +- **Whether 10 ms holds** without dropouts, or 20 ms is needed. +- **The rate** WASAPI records at, which is its mixer's, and that takes still line up. + +Then the user decides the default. If WASAPI's restarts are as unsteady as MME's, the next +step is keeping the stream running between takes (§8). + +## 8. Open points + +- **The default driver**: MME until the runs of §7. +- **A round trip is kept per driver, not per latency.** After a latency change the kept + figure is used unless the latencies the device reports moved by more than 1 ms (then it + is stale, and the menu line says so): calibrate again after changing the latency. +- **Before a device's first take** the menu line, Forget Measured Latency and the dialog + look the figure up at the session's rate, and choosing a driver forgets the device's rate + as choosing a device does. On a device at 48 kHz they then show the driver's figure + although a 48 kHz one is kept; takes use the kept one. A fix needs the device's rate + before the first take, from svapp ([calibrate-audio.md](calibrate-audio.md), §5). - WDM-KS (in PortAudio's build too) and WASAPI's exclusive mode would be lower still, but - take the device from every other program; not in this project. -- Keeping the stream running between takes would remove the restart from the take path + take the device from every other program; not built. +- Keeping the stream running between takes would take the restart out of the take path altogether (an svapp change); only if WASAPI's restarts are as unsteady as MME's. +- Tony does not ask WASAPI for raw capture, so Windows' enhancements apply on every + driver ([calibrate-audio.md](calibrate-audio.md), §10). diff --git a/docs/building.md b/docs/building.md index 856e2c1b..49d19294 100644 --- a/docs/building.md +++ b/docs/building.md @@ -104,8 +104,9 @@ reached. Three scripts in `deploy/linux/` do the work: or about a week has passed. It installs the packages, Qt, ccache and mold, the Android SDK and NDK when `dl.google.com` is reachable, and spends what is left of four minutes filling ccache from a build of the libraries. It also writes an `autoMode` entry to - `/root/.claude/settings.json` by which auto mode trusts the four library forks as it does - Tony's own repository ([forks.md](forks.md#changing-a-fork)): auto mode reads that from + `/root/.claude/settings.json` by which auto mode trusts four of the library forks (all but + `bqaudioio`) as it does Tony's own repository ([forks.md](forks.md#changing-a-fork)): + auto mode reads that from the user's settings, never from the repository's `.claude/settings.json`. The snapshot is kept only when the script ends within about five minutes, so any change to it has to keep to that. Its logs are in `/var/log/tony-environment/`. @@ -188,3 +189,28 @@ Why each part is as it is: - Measured and left alone: `-g1` compiles svcore in 19 % less time than `-g`, but Windows builds `debugoptimized`, with full debug information; clang is no faster than GCC; and a unity build fails in the libraries, which define the same names in several files. + +## Checking the fork's Windows code + +The bqaudioio fork's drivers ([forks.md](forks.md#bqaudioio)) are under `#ifdef _WIN32`, +so the Linux build compiles none of them. Here they are checked by compiling the two files +for Windows with MinGW-w64, against the headers of PortAudio 19.7.0, the version of MSYS2's +package: + +```sh +apt-get install -y g++-mingw-w64-x86-64-posix +mkdir -p tmp/pa197 +for h in portaudio.h pa_win_wasapi.h pa_win_waveformat.h; do + curl -sSfo tmp/pa197/$h https://raw.githubusercontent.com/PortAudio/portaudio/v19.7.0/include/$h +done +for f in PortAudioIO AudioFactory; do + x86_64-w64-mingw32-g++ -std=c++17 -fsyntax-only -DHAVE_PORTAUDIO -Itmp/pa197 \ + -Ibqaudioio/bqaudioio -Ibqaudioio/src -Ibqvec bqaudioio/src/$f.cpp +done +``` + +Both must compile without a word. WASAPI's stream info is included only where +`__has_include` finds `pa_win_wasapi.h`, and a missing header drops it without an error: +run the line for `PortAudioIO` with `-E` in place of `-fsyntax-only` and look for +`wasapiInfo.flags` in the output. Nothing more of the Windows part can be tried here: it +runs only on the user's PC. diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 3600f126..8c1e5187 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -26,8 +26,8 @@ Audacity's measurements) was a separate report, not kept in the repository. `suggestedLatency` chosen for the driver under Playback > Audio Latency on both sides, 0.2 s unless another is chosen ([recording.md](recording.md#latency)). - **The device need not run at the reference's rate.** It opens at PortAudio's default - rate: for "(System Default)" through MME most likely 44.1 kHz, for a device whose name - exists only under WASAPI or WDM-KS often 48 kHz. A recording is converted to the + rate: through MME most likely 44.1 kHz, through WASAPI the rate of Windows' mixer, often + 48 kHz ([audio-drivers.md](audio-drivers.md)). A recording is converted to the reference's rate as it is spliced, and the round trip is counted in seconds and turned into frames of the recording ([recording.md](recording.md#latency)), so a check on such a device is judged, and its figure kept, like any other. The result names the device's @@ -212,9 +212,9 @@ the round trip is exactly the old sum; a core test checks it over a grid of valu The menu line, Forget Measured Latency and the dialog's instructions use the rate of the last take placed with a round trip, or before any take the session's: the device's rate is not known before a take (`AudioCallbackRecordTarget` has no getter for it). Choosing a -device from the menu resets it. So on a device at another rate than the session's, the -three see a figure kept for it only once a take has been recorded since Tony started or -the device was chosen; takes are placed with it from the first. +device or a driver from the menu resets it. So on a device at another rate than the +session's, the three see a figure kept for it only once a take has been recorded since +Tony started or the device or driver was chosen; takes are placed with it from the first. A dev run places its takes with the round trip the calibration before it measured, for the run only: nothing is stored, the menu line goes on describing the window's own figure, and @@ -360,7 +360,9 @@ and the whole `DevChecks.txt`. What the numbers decide: - **Sweeps found, the input peak, the echo:** the finder's thresholds, NoSignal and Clipped. - **The recording's rate:** whether the device runs at the reference's 44.1 kHz or its takes are converted; a figure is kept for each rate. -- **The report's header:** the drivers built in and what the device reports. +- **The report's header:** the driver and the latency asked of it, the drivers built in, + and what the device reports: which run on which driver is which + ([audio-drivers.md](audio-drivers.md), §7). - **Items 1 and 2**, each sweep's offset and each start gap: the ±2 ms. **Item 3**, how far the dots trail the cursor. **Items 4 and 12**, the margin, the looks in the gaps and the longest wait: how the gap check fares with a real device's blocks. **Item 5**, each @@ -435,25 +437,15 @@ What it rests on: The open points are also in [open-points.md](open-points.md), briefly; this section has the reasons. -**Next: a lower-latency driver** (the user's decision, 2026-09-26, from the restart jitter -below). In order: - -1. **The device's rate.** Done: a recording at another rate than the reference's is - converted as it is spliced, whatever the device's rate (the other way, the record - target asking for the session's rate through `getApplicationSampleRate()` in the svapp - fork, fails where the device runs only at its mixer's rate), and Calibrate Audio - measures such a device, and keeps its figure, like any other. It came first because - WASAPI opens at the Windows mixer's rate, usually 48 kHz. -2. **A `bqaudioio` fork**, `jhhr/bqaudioio`, pinned: an implementation per Windows host - API, WASAPI's automatic rate conversion, and a `suggestedLatency` that can be set - ([forks.md](forks.md#bqaudioio)). Done; the plan from here on is in - [audio-drivers.md](audio-drivers.md). -3. **A driver type in Tony**: MME, DirectSound or WASAPI, and the latency asked of it, - under Playback > Audio Driver and Audio Latency; the device menus list that type's - devices only; the stored round trip kept per type. Done. -4. **Measure** with Calibrate Audio and a dev run on each type, on the user's PC. - -**MME stays the default** until such a run shows WASAPI, or another type, better. +**The driver project** (the user's decision, 2026-09-26, from the restart jitter below) is +in [audio-drivers.md](audio-drivers.md): Playback > Audio Driver and Audio Latency put +WASAPI and DirectSound next to MME, each with its own devices, latency and stored round +trip. Left of it: Calibrate Audio and a dev run on each driver, on the user's PC. **MME +stays the default** until such a run shows another better. The device's rate came first, +as WASAPI opens at the Windows mixer's rate. A recording is converted as it is spliced, +whatever the device's rate; the other way, the record target asking for the session's +rate through `getApplicationSampleRate()` in the svapp fork, fails where the device runs +only at its mixer's rate. **Restart jitter on MME.** Every take restarts the stream. On the user's PC the offset between input and output moved by about 13 ms between two takes and by 5 to 20 ms over three @@ -461,9 +453,10 @@ calibrations, while the sweeps within one take agreed to 0.3 ms; the start gap, 0 frames both times, does not see it. No one stored figure then places every take. Considered: widening items 1 and 2 to ±15 ms; keeping the stream running between takes (an svapp change, which would make one session's takes agree with each other but not with the -reference). Chosen: the driver project. Until then items 1 and 2 keep ±2 ms and fail on MME, -the true reading, and so do items 7 and 13 whenever their punch-in lands more than 2 ms off. -Item 10, by reading the code, does not fail for it (the join is a dip, below). +reference). Chosen: the driver project, whose runs on WASAPI are still to come. Items 1 +and 2 keep ±2 ms and fail on MME, the true reading, and so do items 7 and 13 whenever +their punch-in lands more than 2 ms off. Item 10, by reading the code, does not fail for +it (the join is a dip, below). **For the user to decide:** @@ -538,7 +531,8 @@ Item 10, by reading the code, does not fail for it (the join is a dip, below). as the Preferences give it, staleness either side of the tolerance, the round trip in use and its frames at the recording's rate. `TestTakeDiff`: each comparison passing and failing on purpose, and the real `splice()` and `erase()` through files, whose fades lie - inside the range. + inside the range. `TestAudioDriverSettings`: the drivers, the default and the latency + kept per driver ([audio-drivers.md](audio-drivers.md), §6). - **The round trip in the take path** (`TestRecordWorkflow`, `latency_*`): a stored figure lines a take up where the reported pair does not, a stale one is ignored, and the reported pair is converted at the device's rate, also when the device was opened before @@ -552,19 +546,24 @@ Item 10, by reading the code, does not fail for it (the join is a dip, below). that never calls back; the check's playback, and a session opened after it playing as before; the user's toggles and their settings untouched; plans refused; the plan's round trip and pre-roll; keeping the session; replacing a check's own session without asking, - and asking before the user's; Record ignored; the menu and the dialog. Its runs are two - punch-ins of two sweeps on the first 11 s of the calibration reference, about 13 s each. + and asking before the user's; Record ignored; the menu and the dialog, the dialog naming + the driver. The driver and latency menus are tested here too, as they open the device + again: a driver chosen, MME named by default, the latency and the round trip kept per + driver, both greyed out during a take and a check ([audio-drivers.md](audio-drivers.md), + §6). Its runs are two punch-ins of two sweeps on the first 11 s of the calibration + reference, about 13 s each. - **`TestDevChecks`** (`test-tony-dev`, development builds only): whole dev runs on the loopback fake. Passing, with the fake's true round trip and a long song of 60 s (240 s - would add most of a minute to every passing run). Failing: the round trip 20 ms off (items - 1, 2, 7 and 13; item 10 still passes, both punch-ins moved alike); an echo tap, with the - microphone on input 2 (item 4 fails, item 5 judged on input 2); the take made audible - during the re-record's lead-in (items 4 and 12, also through a stall); a stall of the GUI - thread in the lead-in (item 12 still judged, or not judged, never failed). Also cancel, a - closed session, the dev checks deleted during a run, the scratch folders, and the dialog - carrying on into them. Parts that no fault run makes fail (among them item 3's dots on - the tones, item 9, and item 10's step) were seen failing with the code broken for a - moment. About 4 minutes. + would add most of a minute to every passing run). Failing: the round trip 20 ms off + (items 1, 2, 7 and 13; item 10 still passes, both punch-ins moved alike); an echo tap, + with the microphone on input 2 (item 4 fails, item 5 judged on input 2); the take made + audible during the re-record's lead-in (items 4 and 12, also through a stall); a stall + of the GUI thread in the lead-in (item 12 still judged, or not judged, never failed). + Also cancel, a closed session, the dev checks deleted during a run, the scratch folders, + the report's header with the driver and the latency asked for, and the dialog carrying + on into them. Parts that no fault run makes fail (among them item 3's dots on the tones, + item 9, and item 10's step) were seen failing with the code broken for a moment. About 4 + minutes. How the tests are built, and what to watch for: [testing.md](testing.md), "The audio check and the dev checks". @@ -592,7 +591,7 @@ and the dev checks". | How runs are driven | Polling timers and signals, never a nested event loop | | `test-tony-device` (from `default`) | Its checks moved into the dev run, and it is retired | | Where the dev checks' tests run | `test-tony-dev`, a third executable in development builds, run when a change touches what the checks drive (`AGENTS.md`) | -| Restart jitter on MME (about 13 ms) | Not tuned away: a lower-latency driver is the next project; MME stays the default until a run shows another type better | +| Restart jitter on MME (about 13 ms) | Not tuned away: WASAPI was put next to MME to be measured against it ([audio-drivers.md](audio-drivers.md)); MME stays the default until a run shows another driver better | | The notes merge at a join inside a held note | Fixed on this branch: one note across the join ([takes.md](takes.md)) | ## 13. Facts checked in the code @@ -601,7 +600,7 @@ So that later work does not derive them again. - **bqaudioio's `PortAudioIO`**, as upstream has it and the fork's `port` still does (the fork's per-host-API implementations and settable latency: - [audio-drivers.md](audio-drivers.md)): + [forks.md](forks.md#bqaudioio)): - one duplex `Pa_OpenStream`, `suggestedLatency = 0.2`, no host-API stream info; - input goes to the record target **before** output is asked for, in the same callback; - `suspend()`/`resume()` are `Pa_StopStream`/`Pa_StartStream`. `MainWindowBase::stop()` @@ -627,8 +626,9 @@ So that later work does not derive them again. any file (the wrapper then passes the figure through and tells the play source 0); `getSystemRecordLatency()` at the device's. They differ only when the device is not at 44.1 kHz. -- **Device choice.** `getDeviceIndex()` takes the first PortAudio device with the given - name, across host APIs. MME names are cut to 31 characters. +- **Device choice.** Under `port`, `getDeviceIndex()` takes the first PortAudio device + with the given name, across host APIs; under a driver, the first of its host API, else + that host API's default device. MME names are cut to 31 characters. - **Settings a check must not write:** the toggles `m_recordIntoSelection` (`MainWindow/recordintoselection`), `m_playRefWhileRecording` and `m_preRoll` write QSettings when toggled; `wantedPreRollFrames()` reads `MainWindow/prerollseconds`. diff --git a/docs/forks.md b/docs/forks.md index 5722a159..09dd2dec 100644 --- a/docs/forks.md +++ b/docs/forks.md @@ -1,7 +1,7 @@ # Dependency forks Tony's libraries are separate repositories checked out into the top-level directories by -[repoint](../repoint-project.json) and **gitignored in this repository**. Four of them are +[repoint](../repoint-project.json) and **gitignored in this repository**. Five of them are forks under `github.com/jhhr` that exist only for this Tony fork: | Directory | Fork branch | Why it is forked | @@ -26,15 +26,17 @@ library over a workaround in `main/`. which reaches the fork branch when the Tony branch is merged. Commit messages there follow that repository's style: `area: what`. 2. Push to the remote named **`jhhr`**. In `svcore`, `svgui` and `svapp`, `origin` is - upstream sonic-visualiser — do not push there. In a cloud session the checkouts are + upstream sonic-visualiser, and in a `bqaudioio` cloned from its mirror it is + breakfastquay's — do not push there. In a cloud session the checkouts are `container-setup.sh`'s, whose `origin` is the fork, and two checks stand in the way: - The session's git proxy refuses a push to a repository not attached to the session, a new branch included (HTTP 403). The session's add-repository tool attaches it, with push access. - Auto mode trusts only the repository the session started in and its remotes, and so blocks committing in a fork's checkout, attaching the fork and pushing to it. The - environment's setup script names the four forks as trusted as well, which the user - chose ([building.md](building.md#building-on-linux)); `claude auto-mode config` + environment's setup script names four of the forks, all but `bqaudioio`, as trusted + as well, which the user chose ([building.md](building.md#building-on-linux)); + `claude auto-mode config` shows whether a session has that entry. Without it, the user's own message has to ask for the action, naming the fork and the branch. After a denial, stop and tell the user what is blocked: trying again another way counts as getting round the check, and @@ -192,6 +194,11 @@ The driver project ([audio-drivers.md](audio-drivers.md)), on the fork branch Only the Windows cross-compile (MinGW-w64 against PortAudio 19.7.0's headers) and the user's PC see the Windows part; Linux builds none of it. +A `bqaudioio/` cloned from the mirror before the fork was pinned does not have the pin: +`git remote add jhhr https://github.com/jhhr/bqaudioio`, `git fetch jhhr`, then +`git checkout -B feat/wasapi jhhr/feat/wasapi` (the branch that goes with Tony's +`feat/wasapi`). `container-setup.sh` does the equivalent by itself in a cloud session. + ## Known defects in the forks, not fixed - `svapp/audio/AudioCallbackRecordTarget.cpp` connects to `SIGNAL(aboutToBeDeleted())`, diff --git a/docs/manual-checklist.md b/docs/manual-checklist.md index 2abe4343..e0f387d9 100644 --- a/docs/manual-checklist.md +++ b/docs/manual-checklist.md @@ -17,13 +17,13 @@ area are what to ask the user to try. Launch with `.\build.bat run`. ## 1. The device check: Calibrate Audio with the dev checks -Once per machine, and again for each output device sung with (Bluetooth headphones have a -latency of their own). It covers: latency on this machine; several recordings in one take, -each in time; nothing of the take coming back out of the speakers; how long Stop takes on a -four-minute song; live dots from whichever input the microphone is on; and a device that -records nothing. Besides those: recording over part of a take, the lead-in, the pre-roll -near the start, two takes meeting inside a note, and Record into Selection stopping by -itself. +Once per machine and driver, and again for each output device sung with (Bluetooth +headphones have a latency of their own). It covers: latency on this machine; several +recordings in one take, each in time; nothing of the take coming back out of the speakers; +how long Stop takes on a four-minute song; live dots from whichever input the microphone +is on; and a device that records nothing. Besides those: recording over part of a take, +the lead-in, the pre-roll near the start, two takes meeting inside a note, and Record into +Selection stopping by itself. It calibrates, then records test references of sweeps and tones through the air, placing the takes with the round trip it has just measured, for the run only: a four-minute song @@ -33,8 +33,10 @@ file, and compares the take before and after each punch-in. 1. A development build: any build type but `release` (`build.bat` builds `debugoptimized`). -2. Choose the devices under **Playback > Audio Output Device** and **Audio Input Device** - (or leave the system default): the run records with them, as any take does. +2. On Windows, choose the driver and the latency under **Playback > Audio Driver** and + **Audio Latency** (section 2). Then the devices under **Playback > Audio Output + Device** and **Audio Input Device**, which list that driver's (or leave the system + default): the run records with them, as any take does. 3. With wired headphones, hold one earcup against the microphone, off your ears; with speakers, a moderate volume and the microphone where it hears them. A quiet room. 4. **Playback > Calibrate Audio...**, with **Run the dev checks after calibrating** on (it @@ -47,8 +49,9 @@ file, and compares the take before and after each punch-in. data folder (`%APPDATA%\sonic-visualiser\Tony` on Windows). The test session is saved beside it in a `dev-checks-` folder and left open, to be looked at and played. -The report's header names the devices, the audio drivers built in, the playback and record -latencies the device reports, and the round trip used. Then, item by item: +The report's header names the devices, the driver and the latency asked of it, the audio +drivers built in, the playback and record latencies the device reports, and the round trip +used. Then, item by item: - **1** `latency_on_this_machine`: where each sweep of each punch-in landed against the reference, in ms (+ is late), within ±2 ms; the same after saving and reopening. @@ -83,8 +86,24 @@ the sweeps within one take agree to 0.3 ms; the start gap does not see it. No on trip then places every take within ±2 ms: expect items 1 and 2 to fail, and items 7 and 13 whenever their punch-in lands more than 2 ms off. Item 10 shows two punch-ins that landed apart only in its number "second punch-in against the first": the join is a 10 ms dip, which -hides a jump. That is the true reading, not a fault of the check: the remedy is a -lower-latency driver, the next project ([calibrate-audio.md](calibrate-audio.md), §10). +hides a jump. That is the true reading, not a fault of the check: WASAPI is there to be +measured against it, below. + +**On each driver (Windows).** The driver project waits for these runs to decide the +default ([audio-drivers.md](audio-drivers.md), §7). The same setup each time, wired +headphones with one earcup against the microphone: + +1. **MME at 200 ms**: what Tony has always asked for. +2. **WASAPI at 20 ms.** +3. **WASAPI at 10 ms.** + +Each is steps 2 to 5 above, with that driver and latency and the devices chosen under it: +Calibrate Audio carrying on into the dev checks. After each, select and copy the result +page's text, and copy `DevChecks.txt` to a name that says which run it was +(`DevChecks-wasapi-20.txt`, say): the next run writes over it. Send back the three pairs, +and whether anything crackled or dropped out during a run. Press **Use this latency** on +the driver and latency you will sing with: the figure is kept for that driver only. +Not yet done. A device that opens but delivers nothing ends the run with "The audio device delivered no input" once the take's lead-in and range and 2 s more have gone by without one frame, and @@ -101,7 +120,39 @@ no device at all Record does no harm and the next file is analysed (the "Couldn' audio device" warning comes back once per file opened); with a device that opens but delivers nothing the take is dropped quietly, no harm. -## 2. Still by hand +## 2. The audio driver and latency menus (Windows) + +Only on Windows, where the menus are shown. The design is in +[audio-drivers.md](audio-drivers.md). + +1. **The first start** with a build that has them: **Playback > Audio Driver** lists MME, + DirectSound and WASAPI with MME ticked, and **Audio Latency** lists 10 to 200 ms with + 200 ms ticked. **Audio Output Device** and **Audio Input Device** tick the devices + chosen before. One shown as "(not connected)" is a name MME does not have (MME cuts + names to 31 characters): MME's default device is used instead, until the device is + chosen again from the list. The line under Calibrate Audio reads "Latency: driver's + figure, …": a round trip measured before is not carried over, so calibrate again. +2. **WASAPI lists its own devices.** Choose WASAPI while the reference plays: playback + stops. The device menus now list WASAPI's names, whole, with (System Default) ticked. + Choose the headphones and the microphone, play and record a short take: sound comes + out, and the take sits in time by ear. Back to MME: MME's choices are ticked again. +3. **A latency change opens the device again.** On WASAPI choose 20 ms, then 10 ms: each + time what was playing stops, and the next play and take work. Note the figure in + "Latency: driver's figure, …" at each latency, and whether 10 ms crackles. +4. **Greyed out during a take**: while recording, and while Calibrate Audio runs, Audio + Driver and Audio Latency cannot be opened. +5. **The reports name the driver.** Calibrate Audio's first page and its result page + show "Driver: WASAPI"; `DevChecks.txt` has "Audio driver: WASAPI" and "Latency asked + for: 20.0 ms" under the devices at its head. +6. **After Use this latency on WASAPI** the latency line gives the measured figure. Once + Tony is started again, or the driver chosen again, it may read "driver's figure" until + the first take: the device's rate, often 48 kHz there, is not known before. Takes use + the kept figure from the first; a known limit ([audio-drivers.md](audio-drivers.md), + §8). + +Not tried yet. + +## 3. Still by hand 1. **A device in use**: another program holding the microphone exclusively. Record does nothing harmful, and the next file opened is analysed as usual. @@ -128,7 +179,7 @@ delivers nothing the take is dropped quietly, no harm. 7. **Live dots on this machine**: during a take the dots keep up with the cursor and grow smoothly, and neither they nor the cursor stutter, in a maximised window. -## 3. Lyrics +## 4. Lyrics 1. **Legibility**: the words, dark on light boxes along the bottom of the pane, are readable over the waveform, the pitch tracks, the alternate pitch track and the live @@ -188,7 +239,7 @@ delivers nothing the take is dropped quietly, no harm. through the take from its position; after Stop it is back on the word at the take's position. The same with Play Reference While Recording off. -## 4. Lyrics: import and export dialogs, editing +## 5. Lyrics: import and export dialogs, editing 1. **The file dialogs on Windows.** Import Lyrics opens beside the reference and offers "Lyrics (*.ttml *.lrc)" first, then TTML, LRC and all files. Export Lyrics offers diff --git a/docs/open-points.md b/docs/open-points.md index 18874133..2eba9233 100644 --- a/docs/open-points.md +++ b/docs/open-points.md @@ -18,11 +18,14 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. and Edit Lyrics living in the Edit menu. - **The alternate pitch track at ±3 octaves** of a 220 Hz reference (28 Hz, 1.8 kHz) is outside the range the pane shows, and nothing scrolls to it; ±2 is in view. +- **The default audio driver**: MME, until Calibrate Audio and a dev run on WASAPI show it + better ([audio-drivers.md](audio-drivers.md), §7). - Of the [manual checklist](manual-checklist.md), the device check (Calibrate Audio with the dev checks) has been run on real hardware only in part: Calibrate Audio, and a dev run of an early build with items 1 and 2 only (the user's PC, MME, 2026-09-26). The whole - dev run not yet. Of section 2, only the looks, from cloud screenshots (2026-09-25). None - of the lyrics items. + dev run not yet, nor anything on WASAPI or DirectSound, nor the driver menus (section + 2). Of section 3, only the looks, from cloud screenshots (2026-09-25). None of the lyrics + items. - **The dev checks' "not judged" reads Pass.** Items 4 and 12 pass when no look at the output lay in a silent gap, with a message that says so, and the report's Totals then overstate. Should it count otherwise, as Measured say? @@ -33,12 +36,10 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. ## Not built -- **A lower-latency driver**, being built on `feat/wasapi` - ([audio-drivers.md](audio-drivers.md)). Done: a device at 48 kHz is placed and - calibrated like one at 44.1 kHz, and the pinned `bqaudioio` fork has an implementation - per Windows host API, WASAPI's rate conversion and a settable latency. Next: a driver - menu in Tony, then Calibrate Audio and a dev run on each driver. MME stays the default - until a run shows another better. +- **Beyond the three drivers** ([audio-drivers.md](audio-drivers.md), §8): WASAPI's + exclusive mode and WDM-KS, lower still but taking the device from every other program; + keeping the stream running between takes (an svapp change), if WASAPI's restarts turn + out as unsteady as MME's. - Showing two takes at once, or any comparison of takes other than switching. - Singing track gain and pan are not saved in the session. - Background music is not saved in the session; it is reloaded by hand. @@ -70,10 +71,6 @@ library forks are in [forks.md](forks.md). Remove an item when it is dealt with. is not known. `TestUiChecks::grabPaneRedrawn()` works around it. - With no audio device at all, "Couldn't open audio device" is shown again for every file opened (`MainWindowBase::createAudioIO()` tries each time). -- **A recording device not at 44.1 kHz** probably places take audio at the wrong scale: - recordings are written at the device's rate, take timing uses the reference's 44.1 kHz, - and the splice does not resample. Unverified; see - [mobile-port.md](mobile-port.md#sample-rate). - **If pYIN fails part-way, the live dots wait for ever**: they are removed on `initialAnalysisCompleted`, which then never comes. - **`Analyser::newFileLoaded()` error path for the singing track** (pYIN plugin missing): @@ -149,8 +146,13 @@ The reasons are in [calibrate-audio.md](calibrate-audio.md), §10. - **Restart jitter on MME.** Every take restarts the stream, and on the user's PC the offset between input and output moved by about 13 ms from one start to the next. No one round trip then places every take: the dev checks' items 1 and 2, and 7 and 13 whenever - their punch-in lands more than 2 ms off, fail on MME today. The remedy is the driver - project above. + their punch-in lands more than 2 ms off, fail on MME today. WASAPI is there to be + measured against it ([audio-drivers.md](audio-drivers.md), §7). +- **A round trip is kept per driver, not per latency**: after a latency change the kept + figure is used unless the latencies the device reports moved by more than 1 ms. And + before a device's first take, the menu line, Forget Measured Latency and the dialog + look the figure up at the session's rate, so on a 48 kHz device they show the driver's + figure although one is kept ([audio-drivers.md](audio-drivers.md), §8). - The runner allows the reference's analysis 60 s (`kReferenceTimeoutMs`); the 4-minute song's took 10.4 s on the cloud machine, so a PC six times slower ends the dev run at its first stage. diff --git a/docs/recording.md b/docs/recording.md index f1944122..2e4cc4be 100644 --- a/docs/recording.md +++ b/docs/recording.md @@ -160,7 +160,8 @@ L is the **round trip** plus the **start gap**, both in frames of the recording. and MME is built in, `MainWindow::createAudioIO()` names MME before the first device is opened, and the Playback menu before it shows the device menus, carrying the devices chosen before over to MME's keys: bqaudioio lists no devices for no driver when it has - several. Choosing a driver or a latency is shut during a take and while a check runs. + several. Choosing a driver or a latency is shut during a take and while a check runs + ([audio-drivers.md](audio-drivers.md)). ## Pre-roll and Record into Selection From 6f9697959af40d023cbd3463309a42d3e018f23c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 17:25:39 +0000 Subject: [PATCH 232/275] feat: calibrate audio dialog small, and a corner indicator while a check runs Every page fits the window less a phone's bars, its text scrolling, centred there. A started check hides the dialog behind a bar and a line at the right end of the status bar, clear of the pane; a tap brings back the progress page with Make Small and Cancel, and the end of the run brings back the result. Android: text at 85 %, finger-sized buttons, one-finger scrolling. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 25 ++- docs/calibrate-audio.md | 10 +- main/AudioCheckIndicator.cpp | 140 +++++++++++++++++ main/AudioCheckIndicator.h | 71 +++++++++ main/CalibrateAudioDialog.cpp | 283 ++++++++++++++++++++++++++++++---- main/CalibrateAudioDialog.h | 68 +++++++- main/MainWindow.cpp | 4 + main/PopupArea.cpp | 11 ++ main/PopupArea.h | 9 ++ main/test/TestAudioCheck.h | 253 ++++++++++++++++++++++++++++++ main/test/TestDevChecks.h | 12 ++ main/test/TestPopupArea.h | 28 ++++ meson.build | 2 + 13 files changed, 884 insertions(+), 32 deletions(-) create mode 100644 main/AudioCheckIndicator.cpp create mode 100644 main/AudioCheckIndicator.h diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 98d9cce6..5ddb3a97 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -187,7 +187,7 @@ builds happen in the container.) lead's change of scope). - (Lead, 2026-09-26: View > Lyrics Size, 35-100 %, 50 % by default on Android; the size drawn is logged.) -- A12c — The Calibrate Audio dialog: small, and out of the way while a check runs. +- A12c — The Calibrate Audio dialog: small, and out of the way while a check runs. Done. - A12b — The dev run on the phone. - A8 — Documentation pass. @@ -1132,3 +1132,26 @@ Tests seen failing: placement not counted (core Unsteady, app Unsteady 10 ms); s passed (a route's figure unused once reported latency moved). For A8: calibrate-audio.md §5 (route key, streams rule), recording.md "Latency", §3 judging. Left open: none of it on a phone; which input an output-only device will open is a guess. + +### Phase A12c — 2026-09-26 +Built: `CalibrateAudioDialog` sized by `fitToWindow()` at each page and show: 64 average +characters wide (wider if the buttons need it), as tall as the page's text, never more than +`windowArea()` (the window less its safe area margins, within the screen), centred there when +shown (`PopupArea::place()`, core, tested), kept where it is when on show. The texts in scroll +areas that report the text's height for a width; the stack is not asked (it gives the tallest +page's). Android: font at 85 %, buttons 3/4 of a finger high, one-finger scroll (QScroller), +the result not selectable (Copy). `AudioCheckIndicator` (app): a bar and one elided line +("Recording punch-in 2 of 4, 25 s left", "Dev checks, stage 1 of 6: ..."); `MainWindow:: +calibrateAudio()` puts it at the right end of the status bar. A started check collapses the +dialog to it; a tap expands to the progress page (Make Small, Cancel); the run's end expands. +Choices: the corner is the status bar's right end: the pane fills all between toolbar and status +bar, the status line (countdown, sung note) is at the left; it grows the bar (a phone: 2/3 of a +finger), covers nothing. Closing still cancels, but a small dialog is hidden: expand first. +Dialogs at 817x387, compact, fonts 12/15/17 px: message boxes, take name, Open Location, Edit +Display Extents, lyrics word and shift fit. What's New does not (minimum 520x450 scaled by font, +~624x540 at a phone's); About at 17 px (537x393); Key Reference is sized from the screen +(600x274 here) but has no parent (placed by Qt). No Preferences dialog. No generic cause: none +fixed. +Tests seen failing: fit off (fits_a_phone); no collapse (3); no expand at the end (from_the_menu). +For A8: calibrate-audio.md §2 (small, Make Small, not selectable on Android), §9, §11. +Left open: not on a phone (the font, the finger scroll, the dialog's place under the bars). diff --git a/docs/calibrate-audio.md b/docs/calibrate-audio.md index 8965affe..47a88d9f 100644 --- a/docs/calibrate-audio.md +++ b/docs/calibrate-audio.md @@ -50,8 +50,14 @@ Audacity's measurements) was a separate report, not kept in the repository. While a check runs, Record and both device submenus are disabled as well. **The dialog** is not modal: the check's session is in the window and can be looked at -meanwhile. Closing the dialog while its check runs cancels the check, since nothing else -would show how the run ended. Three pages: +meanwhile. It is as small as its page allows and fits the window less a phone's bars, its +text scrolling. **Once a check starts it hides**, and `AudioCheckIndicator`, a bar and one +line of status at the right end of the status bar, clear of the pane, stands in for it; a +tap or click brings back the progress page, which has **Make Small** as well as Cancel, and +the end of the run brings the dialog back on the result page. Closing the dialog while its +check runs cancels the check, since nothing else would show how the run ended. On Android +its text is 85 % of the phone's font, its buttons finger-sized, and it scrolls with one +finger. Three pages: 1. **Instructions:** one earcup against the microphone, off your ears; a moderate volume and a quiet room; the output and input devices and the latency in use; how long it diff --git a/main/AudioCheckIndicator.cpp b/main/AudioCheckIndicator.cpp new file mode 100644 index 00000000..9b8383f0 --- /dev/null +++ b/main/AudioCheckIndicator.cpp @@ -0,0 +1,140 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#include "AudioCheckIndicator.h" + +#include +#include +#include +#include +#include + +#include + +#ifdef Q_OS_ANDROID +#include "PopupArea.h" +#include +#include +#endif + +AudioCheckIndicator::AudioCheckIndicator(QWidget *parent) : + QWidget(parent), + m_pressed(false) +{ + QHBoxLayout *layout = new QHBoxLayout; + layout->setContentsMargins(4, 0, 4, 0); + setLayout(layout); + + m_bar = new QProgressBar; + m_bar->setRange(0, 1000); + m_bar->setValue(0); + m_bar->setTextVisible(false); + m_label = new QLabel; + layout->addWidget(m_bar); + layout->addWidget(m_label); + + // The whole of it is the one thing to press + m_bar->setAttribute(Qt::WA_TransparentForMouseEvents); + m_label->setAttribute(Qt::WA_TransparentForMouseEvents); + setCursor(Qt::PointingHandCursor); + +#ifdef Q_OS_ANDROID + // Tall enough for a finger at the bottom of the screen: the status + // bar grows by it while the check runs + if (QScreen *screen = QGuiApplication::primaryScreen()) { + setMinimumHeight(PopupArea::fingerWidth + (screen->physicalDotsPerInchY()) * 2 / 3); + } +#endif + + updateSizes(); +} + +AudioCheckIndicator::~AudioCheckIndicator() +{ +} + +void +AudioCheckIndicator::setText(QString text) +{ + m_text = text; + setToolTip(tr("%1. Show the check, with Cancel").arg(text)); + updateLabel(); +} + +void +AudioCheckIndicator::setProgress(int permille) +{ + if (permille < 0) { + m_bar->setRange(0, 0); + return; + } + m_bar->setRange(0, 1000); + m_bar->setValue(std::min(permille, 1000)); +} + +int +AudioCheckIndicator::progress() const +{ + if (m_bar->maximum() == 0) return -1; + return m_bar->value(); +} + +void +AudioCheckIndicator::mousePressEvent(QMouseEvent *e) +{ + if (e->button() != Qt::LeftButton) { + QWidget::mousePressEvent(e); + return; + } + m_pressed = true; + e->accept(); +} + +void +AudioCheckIndicator::mouseReleaseEvent(QMouseEvent *e) +{ + if (e->button() != Qt::LeftButton || !m_pressed) { + QWidget::mouseReleaseEvent(e); + return; + } + m_pressed = false; + e->accept(); + // As a button: a press that slides off and lifts elsewhere is not one + if (rect().contains(e->position().toPoint())) emit clicked(); +} + +void +AudioCheckIndicator::changeEvent(QEvent *e) +{ + QWidget::changeEvent(e); + if (e->type() == QEvent::FontChange) updateSizes(); +} + +void +AudioCheckIndicator::updateSizes() +{ + // The label's font, which a phone may give labels of their own + const int character = m_label->fontMetrics().averageCharWidth(); + m_bar->setFixedWidth(character * 10); + m_label->setFixedWidth(character * textWidth); + updateLabel(); +} + +void +AudioCheckIndicator::updateLabel() +{ + m_label->setText(m_label->fontMetrics().elidedText + (m_text, Qt::ElideRight, m_label->width())); +} diff --git a/main/AudioCheckIndicator.h b/main/AudioCheckIndicator.h new file mode 100644 index 00000000..568df8c3 --- /dev/null +++ b/main/AudioCheckIndicator.h @@ -0,0 +1,71 @@ +/* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */ + +/* + Tony + An intonation analysis and annotation tool + Centre for Digital Music, Queen Mary, University of London. + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. See the file + COPYING included with this distribution for more information. +*/ + +#ifndef TONY_AUDIO_CHECK_INDICATOR_H +#define TONY_AUDIO_CHECK_INDICATOR_H + +#include + +class QLabel; +class QProgressBar; + +/** + * The Calibrate Audio dialog made small while its check runs: a bar and + * one line of text (the step, the punch-in, the time left), which the + * window puts at the right end of its status bar, below the panes, so + * that the check's takes can be watched as they are drawn. A tap or a + * click on it says clicked(), and the dialog comes back with Cancel. + * + * The line keeps one width whatever it says, so that the status bar + * does not jump at each step, and is cut short with an ellipsis where + * it is longer; the whole of it is in the tooltip. + */ +class AudioCheckIndicator : public QWidget +{ + Q_OBJECT + +public: + explicit AudioCheckIndicator(QWidget *parent = nullptr); + virtual ~AudioCheckIndicator(); + + void setText(QString text); + QString text() const { return m_text; } + + /// How far the run has got, from 0 to 1000; -1 while that is not + /// known, for a bar that says only that something is going on + void setProgress(int permille); + int progress() const; + + /// The line's width, in average characters of the font + static const int textWidth = 40; + +signals: + void clicked(); + +protected: + void mousePressEvent(QMouseEvent *e) override; + void mouseReleaseEvent(QMouseEvent *e) override; + void changeEvent(QEvent *e) override; + +private: + QString m_text; + QLabel *m_label; + QProgressBar *m_bar; + bool m_pressed; + + void updateSizes(); + void updateLabel(); +}; + +#endif diff --git a/main/CalibrateAudioDialog.cpp b/main/CalibrateAudioDialog.cpp index 167656e7..743b5eb4 100644 --- a/main/CalibrateAudioDialog.cpp +++ b/main/CalibrateAudioDialog.cpp @@ -14,7 +14,9 @@ #include "CalibrateAudioDialog.h" +#include "AudioCheckIndicator.h" #include "MainWindow.h" +#include "PopupArea.h" #include #include @@ -26,17 +28,20 @@ #include #include #include +#include +#include #include #include #include #include +#include #ifdef TONY_DEV_CHECKS #include #endif #ifdef Q_OS_ANDROID -#include +#include #endif #include @@ -92,6 +97,41 @@ bold(QString html) return "" + html + ""; } +// A page's text, scrolling where it is longer than the dialog can be +// tall. It asks for the text's height at the width it is given, so that +// the dialog is no taller than its text needs, and for a few lines at +// the least, so that the dialog can be as short as a phone's window +class TextArea : public QScrollArea +{ +public: + TextArea(QLabel *text) { + setWidget(text); + setWidgetResizable(true); + setFrameShape(QFrame::NoFrame); + setHorizontalScrollBarPolicy(Qt::ScrollBarAlwaysOff); +#ifdef Q_OS_ANDROID + // A finger dragged over the text scrolls it, as on a phone; Qt's + // own gesture on the viewport wants two + QScroller::grabGesture(viewport(), QScroller::LeftMouseButtonGesture); +#endif + } + + bool hasHeightForWidth() const override { + return true; + } + + int heightForWidth(int width) const override { + const int frame = 2 * frameWidth(); + return widget()->heightForWidth(width - frame) + frame; + } + + QSize minimumSizeHint() const override { + const QFontMetrics metrics(widget()->font()); + return QSize(metrics.averageCharWidth() * 20, + metrics.lineSpacing() * 3 + 2 * frameWidth()); + } +}; + } CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, @@ -103,7 +143,8 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, m_running(false), m_latencyKept(false), m_expectedSeconds(0), - m_shownPermille(0) + m_shownPermille(0), + m_collapsed(false) #ifdef TONY_DEV_CHECKS , m_devChecksBox(nullptr), @@ -114,11 +155,28 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, setWindowTitle(tr("Calibrate Audio")); setModal(false); +#ifdef Q_OS_ANDROID + // A phone's widget font is sized for text read on its own: in a + // window 400 px high the pages show more of themselves at a little + // less, and scroll less + QFont smaller = font(); + if (smaller.pixelSize() > 0) { + smaller.setPixelSize + (std::max(10, int(std::lround(smaller.pixelSize() * 0.85)))); + } else if (smaller.pointSizeF() > 0) { + smaller.setPointSizeF(smaller.pointSizeF() * 0.85); + } + setFont(smaller); + cerr << "CalibrateAudioDialog: text " << QFontInfo(font()).pixelSize() + << " px, the window's " << QFontInfo(window->font()).pixelSize() + << " px" << endl; +#endif + QVBoxLayout *layout = new QVBoxLayout; setLayout(layout); m_pages = new QStackedWidget; - layout->addWidget(m_pages); + layout->addWidget(m_pages, 1); auto textLabel = []() { QLabel *label = new QLabel; @@ -133,17 +191,18 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, instructionsLayout->setContentsMargins(0, 0, 0, 0); instructions->setLayout(instructionsLayout); m_instructions = textLabel(); - instructionsLayout->addWidget(m_instructions); + m_instructionsArea = new TextArea(m_instructions); + instructionsLayout->addWidget(m_instructionsArea, 1); #ifdef TONY_DEV_CHECKS m_devChecksBox = new QCheckBox(tr("Run the dev checks after calibrating")); m_devChecksBox->setChecked(true); instructionsLayout->addWidget(m_devChecksBox); #endif - instructionsLayout->addStretch(1); m_pages->addWidget(instructions); QWidget *progress = new QWidget; QVBoxLayout *progressLayout = new QVBoxLayout; + progressLayout->setContentsMargins(0, 0, 0, 0); progress->setLayout(progressLayout); m_step = new QLabel; m_step->setWordWrap(true); @@ -157,30 +216,58 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, progressLayout->addStretch(1); m_pages->addWidget(progress); - // Selectable, so that the figures can be copied and passed on + // Selectable, so that the figures can be copied and passed on. Not + // on a phone, whose selection cannot be copied (Copy does it), and + // where a finger dragged over the text scrolls it m_resultText = textLabel(); +#ifndef Q_OS_ANDROID m_resultText->setTextInteractionFlags(Qt::TextSelectableByMouse); - m_pages->addWidget(m_resultText); +#endif + m_resultArea = new TextArea(m_resultText); + m_pages->addWidget(m_resultArea); - QHBoxLayout *buttons = new QHBoxLayout; + m_buttons = new QHBoxLayout; m_useButton = new QPushButton(tr("Use this latency")); m_againButton = new QPushButton(tr("Check Again")); m_startButton = new QPushButton(tr("Start")); + m_smallButton = new QPushButton(tr("Make Small")); m_cancelButton = new QPushButton(tr("Cancel")); m_closeButton = new QPushButton(tr("Close")); m_copyButton = new QPushButton(tr("Copy")); - buttons->addWidget(m_useButton); - buttons->addStretch(1); - buttons->addWidget(m_copyButton); + m_buttons->addWidget(m_useButton); + m_buttons->addStretch(1); + m_buttons->addWidget(m_copyButton); #ifdef Q_OS_ANDROID m_saveButton = new QPushButton(tr("Save Report...")); - buttons->addWidget(m_saveButton); + m_buttons->addWidget(m_saveButton); #endif - buttons->addWidget(m_againButton); - buttons->addWidget(m_startButton); - buttons->addWidget(m_cancelButton); - buttons->addWidget(m_closeButton); - layout->addLayout(buttons); + m_buttons->addWidget(m_againButton); + m_buttons->addWidget(m_startButton); + m_buttons->addWidget(m_smallButton); + m_buttons->addWidget(m_cancelButton); + m_buttons->addWidget(m_closeButton); + layout->addLayout(m_buttons); + +#ifdef Q_OS_ANDROID + // Tall enough for a finger, whatever the font + if (QScreen *screen = QGuiApplication::primaryScreen()) { + const int height = + PopupArea::fingerWidth(screen->physicalDotsPerInchY()) * 3 / 4; + for (QPushButton *button : { m_useButton, m_againButton, + m_startButton, m_smallButton, + m_cancelButton, m_closeButton, + m_copyButton, m_saveButton }) { + button->setMinimumHeight(height); + } + } +#endif + + // Put in the window's status bar by the window, and shown there while + // the check runs with the dialog hidden + m_indicator = new AudioCheckIndicator; + m_indicator->hide(); + connect(m_indicator, &AudioCheckIndicator::clicked, + this, &CalibrateAudioDialog::expand); connect(m_startButton, &QPushButton::clicked, this, &CalibrateAudioDialog::startCheck); @@ -188,6 +275,8 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, this, &CalibrateAudioDialog::startCheck); connect(m_cancelButton, &QPushButton::clicked, this, &CalibrateAudioDialog::cancelCheck); + connect(m_smallButton, &QPushButton::clicked, + this, &CalibrateAudioDialog::collapse); connect(m_useButton, &QPushButton::clicked, this, &CalibrateAudioDialog::useLatency); connect(m_closeButton, &QPushButton::clicked, @@ -205,12 +294,13 @@ CalibrateAudioDialog::CalibrateAudioDialog(MainWindow *window, connect(m_runner, &AudioCheckRunner::finished, this, &CalibrateAudioDialog::runnerFinished); - setMinimumWidth(520); showPage(Page::Instructions); } CalibrateAudioDialog::~CalibrateAudioDialog() { + // The window's status bar holds it, unless the window is gone first + delete m_indicator; } void @@ -277,6 +367,7 @@ CalibrateAudioDialog::startDevChecks(const AudioCheckResult &calibration) m_step->setText(tr("Calibrated. Starting the dev checks...")); m_timeLeft->setText(QString()); m_bar->setRange(0, 0); + indicate(tr("Calibrated. Starting the dev checks"), -1); return true; } @@ -287,6 +378,8 @@ CalibrateAudioDialog::devProgress(QString stage, int stageNumber, int stages) m_step->setText(tr("Dev checks, stage %1 of %2: %3...") .arg(stageNumber).arg(stages).arg(stage)); m_timeLeft->setText(QString()); + indicate(tr("Dev checks, stage %1 of %2: %3") + .arg(stageNumber).arg(stages).arg(stage), -1); } void @@ -295,11 +388,10 @@ CalibrateAudioDialog::devFinished(const DevReport &report) // Only those that carried on from a calibration of this dialog's if (!m_devRunning) return; m_devRunning = false; - m_running = false; m_devReport = report; m_haveDevReport = true; m_bar->setRange(0, 1000); - showResultPage(); + runEnded(); } QString @@ -373,11 +465,40 @@ CalibrateAudioDialog::present() #endif showPage(Page::Instructions); } + expand(); +} + +void +CalibrateAudioDialog::collapse() +{ + if (!m_running || !m_indicator) return; + m_collapsed = true; + m_indicator->show(); + hide(); +} + +void +CalibrateAudioDialog::expand() +{ + m_collapsed = false; + if (m_indicator) m_indicator->hide(); + fitToWindow(); show(); + // Again, with the frame a desktop gives it when shown, if it is known + // by now: the dialog stays where it is, made smaller if it must be + fitToWindow(); raise(); activateWindow(); } +void +CalibrateAudioDialog::indicate(QString text, int permille) +{ + if (!m_indicator) return; + m_indicator->setText(text); + m_indicator->setProgress(permille); +} + void CalibrateAudioDialog::startCheck() { @@ -411,6 +532,7 @@ CalibrateAudioDialog::startCheck() #endif m_step->setText(tr("Starting the check...")); m_timeLeft->setText(QString()); + indicate(tr("Starting the check"), 0); // The runner says nothing from inside start(), so the page is set // for what comes after it either way @@ -424,7 +546,12 @@ CalibrateAudioDialog::startCheck() "is being recorded. Stop the recording, then " "try again."); showResult(refused); + return; } + + // Out of the way of the takes it records, which are the thing to + // watch while it runs + collapse(); } void @@ -438,8 +565,7 @@ CalibrateAudioDialog::cancelCheck() } else { // Gone with nothing said (the window is going) m_devRunning = false; - m_running = false; - showResultPage(); + runEnded(); } return; } @@ -514,6 +640,15 @@ CalibrateAudioDialog::showResultPage() showPage(Page::Result); } +void +CalibrateAudioDialog::runEnded() +{ + m_running = false; + showResultPage(); + // Back from small by itself, to say how the run went + if (m_collapsed) expand(); +} + void CalibrateAudioDialog::reject() { @@ -532,23 +667,31 @@ CalibrateAudioDialog::runnerProgress(const AudioCheckRunner::Progress &p) if (m_devRunning) return; #endif - QString step; + // The step in full for the progress page, and in a few words for the + // indicator, whose line has the time left too + QString step, brief; switch (p.step) { case AudioCheckRunner::Step::Idle: return; case AudioCheckRunner::Step::OpeningReference: step = tr("Opening the test session..."); + brief = tr("Opening the test session"); break; case AudioCheckRunner::Step::AnalysingReference: step = tr("Getting the test reference ready..."); + brief = tr("Getting the test reference ready"); break; case AudioCheckRunner::Step::Recording: step = tr("Recording punch-in %1 of %2. Keep the earcup against the " "microphone.").arg(p.punchIn).arg(p.punchIns); + brief = tr("Recording punch-in %1 of %2") + .arg(p.punchIn).arg(p.punchIns); break; case AudioCheckRunner::Step::AnalysingTake: step = tr("Analysing punch-in %1 of %2...") .arg(p.punchIn).arg(p.punchIns); + brief = tr("Analysing punch-in %1 of %2") + .arg(p.punchIn).arg(p.punchIns); break; } m_step->setText(step); @@ -568,8 +711,9 @@ CalibrateAudioDialog::runnerProgress(const AudioCheckRunner::Progress &p) std::max(m_shownPermille, std::min(1000, std::max(0, permille))); m_bar->setValue(m_shownPermille); } - m_timeLeft->setText(tr("About %1 seconds left") - .arg(int(std::ceil(left)))); + const int seconds = int(std::ceil(left)); + m_timeLeft->setText(tr("About %1 seconds left").arg(seconds)); + indicate(tr("%1, %2 s left").arg(brief).arg(seconds), m_shownPermille); } void @@ -587,8 +731,7 @@ CalibrateAudioDialog::runnerFinished(const AudioCheckResult &result) #else m_result = result; #endif - m_running = false; - showResultPage(); + runEnded(); } void @@ -597,6 +740,7 @@ CalibrateAudioDialog::showPage(Page page) m_pages->setCurrentIndex(int(page)); m_startButton->setVisible(page == Page::Instructions); + m_smallButton->setVisible(page == Page::Progress); m_cancelButton->setVisible(page == Page::Progress); m_againButton->setVisible(page == Page::Result); m_closeButton->setVisible(page != Page::Progress); @@ -611,6 +755,91 @@ CalibrateAudioDialog::showPage(Page page) if (page == Page::Instructions) m_startButton->setDefault(true); if (page == Page::Result) m_closeButton->setDefault(true); + + fitToWindow(); +} + +void +CalibrateAudioDialog::fitToWindow() +{ + ensurePolished(); + + const QRect area = windowArea(); + if (!area.isValid()) return; + + // The layouts keep the text areas' heights for a width until told + // that they may have changed, which a new text does not tell them + m_instructionsArea->updateGeometry(); + m_resultArea->updateGeometry(); + m_buttons->invalidate(); + + // A desktop's window frame, once the dialog has been shown in one; a + // phone draws none + const QSize frame = + (frameGeometry().size() - size()).expandedTo(QSize(0, 0)); + const QSize room = area.size() - frame; + + const int wanted = std::max(fontMetrics().averageCharWidth() * + preferredWidth, + minimumSizeHint().width()); + const int width = std::max(1, std::min(wanted, room.width())); + const int height = std::max(1, std::min(heightFor(width), room.height())); + resize(width, height); + + // Centred when it is about to be shown, as Qt would centre it, but + // clear of a phone's bars, which Qt's centring is not; where the user + // put it when it is on show + const QRect placed = + PopupArea::place(size() + frame, area, pos(), !isVisible()); + move(placed.topLeft()); + + // Said when it changes, for a report from a phone + const QRect now(pos(), size()); + if (now != m_fitted) { + m_fitted = now; + cerr << "CalibrateAudioDialog: " << now.width() << "x" + << now.height() << " at " << now.x() << "," << now.y() + << " within " << area.x() << "," << area.y() << " " + << area.width() << "x" << area.height() << endl; + } +} + +QRect +CalibrateAudioDialog::windowArea() const +{ + QScreen *screen = m_window->screen(); + if (!screen) screen = QGuiApplication::primaryScreen(); + if (!screen) return QRect(); + + // As TouchMenuStyle keeps menus inside it: on a phone the available + // geometry is the whole screen, bars and all, and the window's safe + // area margins are the bars + QRect window; + QMargins safeArea; + if (m_window->isVisible()) { + window = QRect(m_window->mapToGlobal(QPoint(0, 0)), m_window->size()); +#if QT_VERSION >= QT_VERSION_CHECK(6, 9, 0) + if (QWindow *handle = m_window->windowHandle()) { + safeArea = handle->safeAreaMargins(); + } +#endif + } + return PopupArea::usable(screen->availableGeometry(), window, safeArea, 0); +} + +int +CalibrateAudioDialog::heightFor(int width) const +{ + const QMargins margins = layout()->contentsMargins(); + const int inner = width - margins.left() - margins.right(); + + // The page on show only: the stack would give the tallest page's + QWidget *page = m_pages->currentWidget(); + const int pageHeight = page->hasHeightForWidth() ? + page->heightForWidth(inner) : page->sizeHint().height(); + + return margins.top() + pageHeight + std::max(0, layout()->spacing()) + + m_buttons->sizeHint().height() + margins.bottom(); } QString diff --git a/main/CalibrateAudioDialog.h b/main/CalibrateAudioDialog.h index 321a78a7..d788a8bb 100644 --- a/main/CalibrateAudioDialog.h +++ b/main/CalibrateAudioDialog.h @@ -15,6 +15,7 @@ #ifndef TONY_CALIBRATE_AUDIO_DIALOG_H #define TONY_CALIBRATE_AUDIO_DIALOG_H +#include "AudioCheckIndicator.h" #include "AudioCheckRunner.h" #include "LatencyCalibration.h" @@ -25,11 +26,14 @@ class QCheckBox; #endif #include +#include class MainWindow; +class QHBoxLayout; class QLabel; class QProgressBar; class QPushButton; +class QScrollArea; class QStackedWidget; /** @@ -48,7 +52,17 @@ class QStackedWidget; * take. Closing it while its check runs cancels the check, since * nothing else would show how the run ended. * - * The window owns it, and makes it the first time it is asked for. + * Small, to fit a phone held in landscape: never larger than the part + * of the window clear of the phone's bars, the pages' text scrolling + * where it is longer, and centred in that part when it is shown. While + * its check runs it is smaller still: it hides, and its indicator (a + * bar and a line of text in the window's status bar) says how far the + * check has got, out of the way of the takes being drawn. A tap on the + * indicator brings it back on the progress page, with Cancel and Make + * Small; when the run ends it comes back by itself, with the result. + * + * The window owns it, and makes it the first time it is asked for; it + * puts the indicator in its status bar. * * In development builds the instructions page has a checkbox, on by * default and not remembered, to carry on into the dev checks @@ -79,6 +93,18 @@ class CalibrateAudioDialog : public QDialog /// Whether Use this latency is offered, and not yet pressed bool canUseLatency() const; + /// What the dialog becomes while its check runs, for the window's + /// status bar. Hidden until then; the dialog deletes it + AudioCheckIndicator *indicator() const { return m_indicator; } + + /// Whether the dialog is small: its check running, the dialog hidden + /// and the indicator on show + bool isCollapsed() const { return m_collapsed; } + + /// The width the dialog takes when the window has room, in average + /// characters of its font: about half a phone's width in landscape + static const int preferredWidth = 64; + /// The result as plain text, under a line saying when: what Copy /// puts on the clipboard and Save Report... saves QString reportText() const; @@ -113,9 +139,18 @@ public slots: /// check is running void present(); - /// Start, and Check Again + /// Start, and Check Again. The dialog goes small once the check + /// has started void startCheck(); + /// Make Small: while its check runs, the dialog hides and its + /// indicator shows how far the check has got + void collapse(); + + /// The dialog on show again, and its indicator hidden: a tap on the + /// indicator, and the end of a run + void expand(); + /// Cancel: the run ends, and how it ended is the result shown. In /// its dev checks, they end, and the calibration is shown with them void cancelCheck(); @@ -156,15 +191,24 @@ public slots: double m_expectedSeconds; int m_shownPermille; + /// Small while the check runs + bool m_collapsed; + QPointer m_indicator; + QStackedWidget *m_pages; QLabel *m_instructions; + QScrollArea *m_instructionsArea; QLabel *m_step; QProgressBar *m_bar; QLabel *m_timeLeft; QLabel *m_resultText; + QScrollArea *m_resultArea; + QHBoxLayout *m_buttons; + QRect m_fitted; QPushButton *m_startButton; QPushButton *m_cancelButton; + QPushButton *m_smallButton; QPushButton *m_useButton; QPushButton *m_againButton; QPushButton *m_closeButton; @@ -181,6 +225,26 @@ public slots: /// The result page for m_result as it stands void showResultPage(); + /// The end of a run this dialog started: the result page, on show + void runEnded(); + + /// The indicator's line and bar (-1 for one that says only that + /// something is going on) + void indicate(QString text, int permille); + + /// Sized for the page on show, within windowArea(), and placed there: + /// centred if the dialog is about to be shown, else kept where it is + void fitToWindow(); + + /// The part of the screen the dialog may cover: the window, less its + /// safe area margins (a phone's bars and camera cutout), within the + /// screen's available geometry; the screen's, while the window is + /// not on show + QRect windowArea() const; + + /// The dialog's height at width, all of the page's text on show + int heightFor(int width) const; + QString instructionsHtml() const; QString resultHtml() const; QString calibrationHtml() const; diff --git a/main/MainWindow.cpp b/main/MainWindow.cpp index e929a447..ad7a93d6 100644 --- a/main/MainWindow.cpp +++ b/main/MainWindow.cpp @@ -5918,6 +5918,10 @@ MainWindow::calibrateAudio() #ifdef TONY_DEV_CHECKS m_calibrateAudioDialog->setDevChecks(m_devChecks); #endif + // Where the dialog goes while its check runs: the right end of + // the status bar, a corner the panes never reach, clear of the + // status line at its left + statusBar()->addPermanentWidget(m_calibrateAudioDialog->indicator()); } m_calibrateAudioDialog->present(); } diff --git a/main/PopupArea.cpp b/main/PopupArea.cpp index 64e322bc..fd9d26db 100644 --- a/main/PopupArea.cpp +++ b/main/PopupArea.cpp @@ -64,6 +64,17 @@ PopupArea::fit(QRect rect, QRect usable) return rect; } +QRect +PopupArea::place(QSize size, QRect usable, QPoint topLeft, bool centre) +{ + QRect rect(topLeft, size); + if (centre && usable.isValid()) { + rect.moveTopLeft(QPoint(usable.left() + (usable.width() - size.width()) / 2, + usable.top() + (usable.height() - size.height()) / 2)); + } + return fit(rect, usable); +} + int PopupArea::pixels(double mm, double dotsPerInch) { diff --git a/main/PopupArea.h b/main/PopupArea.h index a94317cc..28c6f11c 100644 --- a/main/PopupArea.h +++ b/main/PopupArea.h @@ -62,6 +62,15 @@ class PopupArea */ static QRect fit(QRect rect, QRect usable); + /** + * Where a dialog of size goes inside usable: centred in it when it + * is being shown afresh (centre), else where topLeft puts it, as the + * user may have moved it; and then fit() into usable. Qt centres a + * dialog over its parent inside the screen's available geometry, + * which on a phone takes in the system bars. + */ + static QRect place(QSize size, QRect usable, QPoint topLeft, bool centre); + /** * A length in pixels: mm millimetres at dotsPerInch. */ diff --git a/main/test/TestAudioCheck.h b/main/test/TestAudioCheck.h index ee53a17b..b6b07d99 100644 --- a/main/test/TestAudioCheck.h +++ b/main/test/TestAudioCheck.h @@ -51,8 +51,14 @@ #include #include #include +#include +#include +#include +#include +#include #include #include +#include #include #include @@ -81,6 +87,9 @@ class TestAudioCheck : public QObject QTimer m_watchdog; QStringList m_dialogs; + // The application's font, which a test may make a phone's + QFont m_font; + // What the runner said when the run ended, and how often it said it; // and every progress it reported AudioCheckResult m_result; @@ -488,6 +497,88 @@ class TestAudioCheck : public QObject return ""; } + // The window as a phone shows it: the compact layout, in the part of + // the phone's window clear of its bars (817 by 387 of 923 by 411, as + // the log of the user's phone has it; this platform has no bars) + void showAsAPhone() { + m_window->setCompactLayout(true); + m_window->resize(817, 387); + m_window->show(); + QVERIFY(QTest::qWaitForWindowExposed(m_window)); + settle(); + } + + // Let the windows lay themselves out again + void settle() { + QCoreApplication::sendPostedEvents(); + QTest::qWait(20); + } + + static QString describe(QRect r) { + return QString("%1x%2 at %3,%4").arg(r.width()).arg(r.height()) + .arg(r.x()).arg(r.y()); + } + + static QRect onScreen(QWidget *widget) { + return QRect(widget->mapToGlobal(QPoint(0, 0)), widget->size()); + } + + // The dialog inside the window, with every button it shows inside + // both, and its text, if it has more than there is room for, + // scrolling to its end; scrolls says whether it had to + void verifyFits(CalibrateAudioDialog *dialog, QString page, + bool *scrolls = nullptr) { + settle(); + const QRect window = onScreen(m_window); + QVERIFY2(dialog->isVisible(), qPrintable(page)); + QVERIFY2(window.contains(dialog->frameGeometry()), + qPrintable(QString("%1 page %2 is not inside the window, %3") + .arg(page).arg(describe(dialog->frameGeometry())) + .arg(describe(window)))); + int buttons = 0; + for (QPushButton *button : dialog->findChildren()) { + if (!button->isVisible()) continue; + ++buttons; + const QRect r = onScreen(button); + QVERIFY2(window.contains(r) && onScreen(dialog).contains(r), + qPrintable(QString("%1 page: %2 at %3 is off the dialog, " + "%4, or the window") + .arg(page).arg(button->text()).arg(describe(r)) + .arg(describe(onScreen(dialog))))); + } + QVERIFY2(buttons > 0, qPrintable(page)); + + if (scrolls) *scrolls = false; + for (QScrollArea *area : dialog->findChildren()) { + if (!area->isVisible()) continue; + QWidget *text = area->widget(); + const int beyond = + std::max(0, text->heightForWidth(text->width()) - + area->viewport()->height()); + QVERIFY2(text->height() >= text->heightForWidth(text->width()), + qPrintable(page + ": the text is cut short")); + QCOMPARE(area->verticalScrollBar()->maximum(), beyond); + if (beyond > 0 && scrolls) *scrolls = true; + } + } + + // A finger's tap, which Qt makes into a mouse press and release when + // the widget under it takes no touch, as on a phone + void tap(QWidget *widget) { + QPointingDevice *finger = QTest::createTouchDevice(); + const QPoint centre = widget->rect().center(); + QTest::touchEvent(widget->window(), finger).press(0, centre, widget); + QTest::touchEvent(widget->window(), finger).release(0, centre, widget); + settle(); + } + + static QPushButton *button(QWidget *dialog, QString text) { + for (QPushButton *b : dialog->findChildren()) { + if (b->text() == text) return b; + } + return nullptr; + } + // Not a slot: QtTest would run it as a test. As TestRecordWorkflow's void dismissDialog() { QWidget *modal = QApplication::activeModalWidget(); @@ -514,6 +605,7 @@ class TestAudioCheck : public QObject private slots: void initTestCase() { QVERIFY(m_dir.isValid()); + m_font = QApplication::font(); QSettings().clear(); @@ -571,6 +663,7 @@ private slots: delete m_window; m_window = nullptr; } + QApplication::setFont(m_font); QVERIFY2(m_dialogs.isEmpty(), qPrintable("unexpected dialog: " + m_dialogs.join(" | "))); } @@ -1569,6 +1662,15 @@ private slots: QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); setDevices("Speakers B", "Microphone B"); + // Small while the check runs: the dialog hidden, and its + // indicator, which the window's status bar holds, on show there + AudioCheckIndicator *indicator = dialog->indicator(); + QVERIFY(dialog->isCollapsed()); + QVERIFY(!dialog->isVisible()); + QCOMPARE(indicator->parentWidget(), + static_cast(m_window->statusBar())); + QVERIFY(!indicator->isHidden()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); @@ -1576,9 +1678,18 @@ private slots: QVERIFY(!m_window->audioInputMenu()->menuAction()->isEnabled()); QVERIFY2(dialog->pageText().contains("Recording punch-in 1 of 2"), qPrintable(dialog->pageText())); + // The step, the punch-in and the time left, on one line + QVERIFY2(QRegularExpression("^Recording punch-in 1 of 2, \\d+ s " + "left$").match(indicator->text()) + .hasMatch(), qPrintable(indicator->text())); + QVERIFY(indicator->progress() >= 0); + // Back by itself at the end, with the result QTRY_VERIFY_WITH_TIMEOUT (dialog->page() == CalibrateAudioDialog::Page::Result, 60000); + QVERIFY(dialog->isVisible()); + QVERIFY(!dialog->isCollapsed()); + QVERIFY(indicator->isHidden()); QCOMPARE(m_finished, 1); QVERIFY(m_window->calibrateAudioAction()->isEnabled()); QVERIFY(m_window->audioOutputMenu()->menuAction()->isEnabled()); @@ -1815,8 +1926,15 @@ private slots: QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); + // Small, it has nothing to close: brought back as a tap on its + // indicator brings it, then closed + QVERIFY(!dialog->isVisible()); + emit dialog->indicator()->clicked(); + QVERIFY(dialog->isVisible()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); QVERIFY(dialog->close()); QVERIFY(!dialog->isVisible()); + QVERIFY(dialog->indicator()->isHidden()); QVERIFY(!m_window->audioCheck()->isRunning()); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(!m_window->audioCheckTakes()); @@ -1828,6 +1946,141 @@ private slots: QVERIFY(dialog->isVisible()); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Instructions); } + + // Every page fits a phone held in landscape, all its buttons on + // screen and its text scrolling where it is longer: at this desktop's + // font and at a phone's size of font, which scrolls + void calibrate_audio_fits_a_phone() { + for (int pixels : { 0, 17 }) { + if (pixels > 0) { + QFont font = m_font; + font.setPixelSize(pixels); + QApplication::setFont(font); + } + const QString size = (pixels > 0 ? QString("%1 px").arg(pixels) : + QString("the desktop's font")); + if (m_window) { + QTRY_VERIFY_WITH_TIMEOUT + (!sv::ModelTransformerFactory::getInstance() + ->haveRunningTransformers(), 30000); + m_window->doCloseSession(); + } + // A phone's route, and its instructions, once a file is open + FakeAudioIO::Config config = loopback(); + config.route = phoneRoute(); + makeWindow(config); + showAsAPhone(); + if (QTest::currentTestFailed()) return; + openSong(); + if (QTest::currentTestFailed()) return; + + m_window->calibrateAudioAction()->trigger(); + CalibrateAudioDialog *dialog = m_window->calibrateAudioDialog(); + QVERIFY(dialog); + verifyFits(dialog, "instructions, " + size); + if (QTest::currentTestFailed()) return; + + // Started, and brought back from small + dialog->setPlan(shortPlan()); + m_window->discardModifications(); + dialog->startCheck(); + QVERIFY(dialog->isCollapsed()); + emit dialog->indicator()->clicked(); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); + verifyFits(dialog, "progress, " + size); + if (QTest::currentTestFailed()) return; + dialog->cancelCheck(); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + verifyFits(dialog, "result of a cancelled check, " + size); + if (QTest::currentTestFailed()) return; + + // As long as a result gets: a phone's, every paragraph there is + AudioCheckResult result = + judgedResult(LatencyCheck::Verdict::Unsteady); + result.summary.spread = 0.008; + result.summary.echo.heard = true; + result.summary.echo.delaySeconds = 0.045; + result.summary.echo.levelDb = -12.0; + result.route = phoneRoute(); + result.key = LatencyCalibration::routeKey(phoneRoute(), rate); + dialog->showResult(result); + dialog->useLatency(); + bool scrolls = false; + verifyFits(dialog, "result, " + size, &scrolls); + if (QTest::currentTestFailed()) return; + if (pixels > 0) { + QVERIFY2(scrolls, "the result fitted without scrolling: the " + "test shows nothing"); + } + + m_window->discardModifications(); + } + } + + // While the check runs the dialog is a bar and a line of text at the + // right end of the status bar, clear of the pane its takes are drawn + // in. A tap brings the dialog back on the progress page, with Cancel + // and Make Small; Make Small hides it again, the check going on; and + // after Cancel it shows the result, the indicator gone + void calibrate_audio_small_during_a_check() { + makeWindow(loopback()); + showAsAPhone(); + if (QTest::currentTestFailed()) return; + CalibrateAudioDialog *dialog = startCheckFromMenu(); + QVERIFY(dialog); + QVERIFY(m_window->audioCheck()->isRunning()); + QVERIFY(dialog->isCollapsed()); + QVERIFY(!dialog->isVisible()); + + AudioCheckIndicator *indicator = dialog->indicator(); + settle(); + QVERIFY(indicator->isVisible()); + QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), + 30000); + QVERIFY2(indicator->text().startsWith("Recording punch-in 1 of 2"), + qPrintable(indicator->text())); + + // In the window's bottom right corner, in its status bar, clear + // of the pane; and not so wide as to leave the status line no room + const QRect window = onScreen(m_window); + const QRect corner = onScreen(indicator); + QVERIFY2(onScreen(m_window->statusBar()).contains(corner), + qPrintable(describe(corner))); + QVERIFY2(!corner.intersects(onScreen(m_window->paneStack())), + qPrintable(describe(corner) + " and " + + describe(onScreen(m_window->paneStack())))); + QVERIFY2(corner.left() > window.center().x() && + corner.bottom() >= onScreen(m_window->paneStack()).bottom(), + qPrintable(describe(corner) + " in " + describe(window))); + + tap(indicator); + QVERIFY(dialog->isVisible()); + QVERIFY(!dialog->isCollapsed()); + QVERIFY(indicator->isHidden()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); + QPushButton *cancel = button(dialog, "Cancel"); + QPushButton *small = button(dialog, "Make Small"); + QVERIFY(cancel && cancel->isVisible()); + QVERIFY(small && small->isVisible()); + + QTest::mouseClick(small, Qt::LeftButton); + settle(); + QVERIFY(!dialog->isVisible()); + QVERIFY(dialog->isCollapsed()); + QVERIFY(indicator->isVisible()); + QVERIFY(m_window->audioCheck()->isRunning()); + QCOMPARE(m_finished, 0); + + tap(indicator); + QVERIFY(dialog->isVisible()); + QTest::mouseClick(cancel, Qt::LeftButton); + QCOMPARE(m_finished, 1); + QVERIFY(!m_window->audioCheck()->isRunning()); + QVERIFY(!m_window->recordTarget()->isRecording()); + QVERIFY(dialog->isVisible()); + QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + QVERIFY(indicator->isHidden()); + } }; #endif diff --git a/main/test/TestDevChecks.h b/main/test/TestDevChecks.h index 42944043..0619d5a2 100644 --- a/main/test/TestDevChecks.h +++ b/main/test/TestDevChecks.h @@ -1122,6 +1122,8 @@ private slots: 30000); dialog->cancelCheck(); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + QVERIFY(dialog->isVisible()); + QVERIFY(!dialog->isCollapsed()); QVERIFY2(dialog->pageText().contains("The dev checks did not run"), qPrintable(dialog->pageText())); QVERIFY(!m_window->devChecks()->isRunning()); @@ -1135,6 +1137,14 @@ private slots: QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Progress); QTRY_VERIFY_WITH_TIMEOUT (dialog->pageText().contains("Dev checks, stage 1 of 6"), 10000); + // Still small, carrying on from the calibration, the indicator + // following the stages + QVERIFY(dialog->isCollapsed()); + QVERIFY(!dialog->isVisible()); + QVERIFY2(dialog->indicator()->text() + .startsWith("Dev checks, stage 1 of 6: "), + qPrintable(dialog->indicator()->text())); + QCOMPARE(dialog->indicator()->progress(), -1); QTRY_VERIFY_WITH_TIMEOUT(m_window->recordTarget()->isRecording(), 30000); QVERIFY(!m_window->calibrateAudioAction()->isEnabled()); @@ -1145,6 +1155,8 @@ private slots: QVERIFY(!m_window->devChecks()->isRunning()); QVERIFY(!m_window->recordTarget()->isRecording()); QVERIFY(dialog->page() == CalibrateAudioDialog::Page::Result); + QVERIFY(dialog->isVisible()); + QVERIFY(dialog->indicator()->isHidden()); const QString words = dialog->pageText(); for (QString w : { QString("came back steadily"), diff --git a/main/test/TestPopupArea.h b/main/test/TestPopupArea.h index c94a9439..f3f85dfc 100644 --- a/main/test/TestPopupArea.h +++ b/main/test/TestPopupArea.h @@ -127,6 +127,34 @@ private slots: QCOMPARE(PopupArea::fit(QRect(1, 2, 3, 4), QRect()), QRect(1, 2, 3, 4)); } + // The phone of the Calibrate Audio test, as its log has it: a window + // of 923 by 411, the camera's cutout at the left, the status bar at + // the top and the navigation buttons at the right + void a_dialog_is_centred_in_the_safe_area_and_kept_inside() { + const QRect window(0, 0, 923, 411); + QRect usable = PopupArea::usable + (window, window, QMargins(58, 24, 48, 0), 0); + QCOMPARE(usable, QRect(58, 24, 817, 387)); + + // Shown afresh: centred in the safe area, not over the bars, as + // Qt's own centring on the window would put it + QCOMPARE(PopupArea::place(QSize(517, 301), usable, QPoint(0, 0), true), + QRect(208, 67, 517, 301)); + // Taller than the safe area: as tall as it, below the status bar + QCOMPARE(PopupArea::place(QSize(517, 450), usable, QPoint(0, 0), true), + QRect(208, 24, 517, 387)); + // On show: left where it is, unless it sticks out + QCOMPARE(PopupArea::place(QSize(300, 200), usable, QPoint(100, 150), + false), + QRect(100, 150, 300, 200)); + QCOMPARE(PopupArea::place(QSize(300, 200), usable, QPoint(700, 300), + false), + QRect(575, 211, 300, 200)); + // Nowhere to go + QCOMPARE(PopupArea::place(QSize(3, 4), QRect(), QPoint(1, 2), true), + QRect(1, 2, 3, 4)); + } + void millimetres_become_pixels() { QCOMPARE(PopupArea::pixels(8, 160), 50); QCOMPARE(PopupArea::pixels(25.4, 96), 96); diff --git a/meson.build b/meson.build index f827e793..a5ef8219 100644 --- a/meson.build +++ b/meson.build @@ -1220,6 +1220,7 @@ endif tony_app_files = [ 'main/AlternatePitchTrack.cpp', + 'main/AudioCheckIndicator.cpp', 'main/AudioCheckRunner.cpp', 'main/CalibrateAudioDialog.cpp', 'main/CompactLayout.cpp', @@ -1255,6 +1256,7 @@ tony_app_moc_headers = [ 'main/MainWindow.h', 'main/Analyser.h', 'main/AlternatePitchTrack.h', + 'main/AudioCheckIndicator.h', 'main/AudioCheckRunner.h', 'main/CalibrateAudioDialog.h', 'main/CompactLayout.h', From c26f2427baa42f84785e01e26ebc3c36ec4a50e2 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 17:25:48 +0000 Subject: [PATCH 233/275] fix: the host qt for android is built without zstd On CI's runner, which has libzstd-dev, the host Qt found zstd and its rcc compressed Tony's resources with it; Qt for Android has no zstd, and linking the app failed on qt_resourceFeatureZstd. The container has no libzstd-dev, so its builds never met this. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- deploy/android/build-qt.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/deploy/android/build-qt.sh b/deploy/android/build-qt.sh index 991c3548..e0f669c3 100755 --- a/deploy/android/build-qt.sh +++ b/deploy/android/build-qt.sh @@ -33,6 +33,9 @@ # The host Qt is built only for its tools, so it leaves out what they # do not need (SQL, printing, D-Bus, OpenGL, ICU) and uses Qt's bundled # third-party libraries, to depend on nothing the container may lack. +# Nor on what a machine happens to have: with zstd found (libzstd-dev, +# which CI's runner has), rcc compresses Tony's resources with it, and +# Qt for Android, which has no zstd, cannot link or read them. # Qt for Android leaves out SQL and printing, which Tony does not use, # and has no OpenSSL: Qt Network works, without TLS. # @@ -142,7 +145,7 @@ else -prefix "$qt_host" -release -nomake tests -nomake examples \ -qt-zlib -qt-pcre -qt-doubleconversion -qt-freetype -qt-harfbuzz \ -qt-libpng -qt-libjpeg -no-icu -no-glib -no-dbus -no-opengl \ - -no-xcb -no-gtk -no-feature-sql -no-feature-printsupport + -no-xcb -no-gtk -no-zstd -no-feature-sql -no-feature-printsupport echo " done in $(( (SECONDS - start) / 60 )) min" fi From 743e83b204b9e83a70188641cfd6edb6924306c6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 17:31:08 +0000 Subject: [PATCH 234/275] docs: the android ci job, what it builds and what it caches Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- docs/android-work-orders.md | 4 ++-- docs/building.md | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 98d9cce6..4ff655de 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -160,8 +160,8 @@ report, list the files to stage and propose a message (`feat:` / `fix:` / `test: Order: A0, A1, A2, A3a, A3b, then A4 and A5 while the user tries the APK on the phone. A6 needs the result of that phone test. A8 is last. (Since 2026-09-25 `download.qt.io` and -`dl.google.com` are reachable from the container. GitHub workflows are turned off: all -builds happen in the container.) +`dl.google.com` are reachable from the container. The builds happen in the container; +since 2026-09-26 `.github/workflows/android.yml` builds the APK on CI as well.) - A0 — Desktop build and tests in the container. Done. - A1 — Sample rate: a device that is not at 44.1 kHz. Done. diff --git a/docs/building.md b/docs/building.md index f5a9ef9b..31c346cd 100644 --- a/docs/building.md +++ b/docs/building.md @@ -5,7 +5,11 @@ The development machine builds with meson + ninja under MSYS2's `mingw64` toolch agent in a cloud session builds on Linux instead, into `build/`: see [Building on Linux](#building-on-linux) at the end. The Windows CI workflow in `.github/workflows/` builds as the development machine does, from MSYS2's packages; the -Linux and macOS ones build the upstream way. +Linux and macOS ones build the upstream way. The Android one builds the APK with the +scripts in `deploy/android/`, as a cloud session does, and keeps it as the run's artifact +`Tony-debug-apk`. It caches Qt for Android and the C libraries, each under a key made from +the script that builds it and `setup-toolchain.sh`: a change to one of those builds it +again, Qt in about twenty minutes. # Building on Windows (MSYS2 MinGW-w64) From 16f602b9dc2a1c33d0c28060b8a117f0966bae28 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 17:54:20 +0000 Subject: [PATCH 235/275] test: android ci shows what repoint install changed in the tree The APK built on CI was named as built from a changed tree ("+" after the commit). repoint writes the lock file afresh from what it checked out; this shows whether that is the change. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0175FSQFemK6yNDepHnCX4Nq --- .github/workflows/android.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml index 1271d875..3e99fc43 100644 --- a/.github/workflows/android.yml +++ b/.github/workflows/android.yml @@ -43,7 +43,10 @@ jobs: sudo apt-get update sudo apt-get install -y git mercurial smlnj - name: repoint - run: ./repoint install + run: | + ./repoint install + git status --short + git diff - name: setup-toolchain run: deploy/android/setup-toolchain.sh - name: restore-qt From af06892f97fd5c492edb279ff1e8ec1b69533f4b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 18:05:31 +0000 Subject: [PATCH 236/275] feat: the dev run on the phone Save Report... and Copy carry DevChecks.txt, which a phone keeps where only Tony can read it. Analysis limits and the dev stages' backstops are scaled 4x on Android, and each analysis's time is logged. The screen is kept on while a check runs. Item 14 is worked out at the device's and the reference's rates (it failed every take at 48 kHz) and counts a dialog Android draws itself; item 5 reads Measured on a one-channel input; the report names the route. A dev run in the phone's shape in test-tony-dev. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01V9EHVCNZ4EXP662cDczuon --- docs/android-work-orders.md | 27 +++++- docs/calibrate-audio.md | 12 ++- main/AndroidScreen.cpp | 63 +++++++++++++ main/AndroidScreen.h | 34 +++++++ main/AudioCheckRunner.cpp | 13 +++ main/AudioCheckRunner.h | 21 ++++- main/CalibrateAudioDialog.cpp | 54 ++++++++++- main/CalibrateAudioDialog.h | 11 ++- main/dev/DevChecks.cpp | 33 ++++++- main/dev/DevChecks.h | 16 +++- main/dev/TakeObserver.cpp | 8 +- main/dev/TakeObserver.h | 10 ++- main/test/FakeAudioIO.h | 21 +++-- main/test/TestDevChecks.h | 165 +++++++++++++++++++++++++++++++++- meson.build | 1 + 15 files changed, 462 insertions(+), 27 deletions(-) create mode 100644 main/AndroidScreen.cpp create mode 100644 main/AndroidScreen.h diff --git a/docs/android-work-orders.md b/docs/android-work-orders.md index 5ddb3a97..7e781677 100644 --- a/docs/android-work-orders.md +++ b/docs/android-work-orders.md @@ -188,7 +188,7 @@ builds happen in the container.) - (Lead, 2026-09-26: View > Lyrics Size, 35-100 %, 50 % by default on Android; the size drawn is logged.) - A12c — The Calibrate Audio dialog: small, and out of the way while a check runs. Done. -- A12b — The dev run on the phone. +- A12b — The dev run on the phone. Done. - A8 — Documentation pass. ### A0 — Desktop build and tests in the container @@ -1155,3 +1155,28 @@ fixed. Tests seen failing: fit off (fits_a_phone); no collapse (3); no expand at the end (from_the_menu). For A8: calibrate-audio.md §2 (small, Make Small, not selectable on Android), §9, §11. Left open: not on a phone (the font, the finger scroll, the dialog's place under the bars). + +### Phase A12b — 2026-09-26 +Built: `CalibrateAudioDialog::reportText()` (Copy, and Save Report... on Android) ends with +DevChecks.txt whole when a dev run wrote one; Save Report... then suggests +`tony-dev-checks-