feat: add senselab VAD, eGeMAPS, and embedding processors (#566)#1169
feat: add senselab VAD, eGeMAPS, and embedding processors (#566)#1169Chelsy-AI wants to merge 2 commits into
Conversation
|
Hey, do you mind giving more context for what this is for? |
Hi @jhnwu3 — happy to clarify! This PR implements issue #566, which proposed integrating senselab into PyHealth as a set of audio processor wrappers. Senselab is an open-source package for standardized voice and speech analysis in health and clinical research contexts. The three processors I added each wrap a different senselab capability: SenselabVADProcessor — runs Voice Activity Detection to strip silence and noise, returning only the speech segments as a waveform tensor. Useful as a preprocessing step before feature extraction. SenselabEGeMAPSProcessor — extracts the eGeMAPS feature set (88 hand-crafted acoustic features) widely used in clinical voice research for depression detection, Parkinson's monitoring, and cognitive assessment. SenselabEmbeddingProcessor — extracts deep neural speaker embeddings using WavLM or ECAPA-TDNN, which capture paralinguistic features useful for voice biomarker tasks. The motivation from the original issue was to extend PyHealth's existing sensor/EHR pipeline to support end-to-end voice biomarker workflows — so clinicians could run speech-based assessments (e.g. depression screening from voice) using the same processor interface as the rest of PyHealth. All three follow the existing AudioProcessor pattern in the codebase — lazy imports, optional dependency, process() takes a file path and returns a torch.Tensor. Let me know if you'd like me to adjust the scope, API design, or add a usage example to the docstrings. |
|
Ahh, that's super cool. Can you follow the documentation requirements to add: docs/.rst files for each processor Tbh, I think I'll need to update the website later to include easier to read documentation on contribution styling, etc. |
Done — added individual .rst files for each processor under docs/api/processors/, updated the toctree and Available Processors list in docs/api/processors.rst, and added examples/senselab_processor_tutorial.py showing initialization and processing for all three. Tests were already included in the original commit. Let me know if anything needs adjusting. |
Closes #566
Summary
Adds three
FeatureProcessorsubclasses wrapping senselab for voice/speech analysis inpyhealth/processors/senselab_processor.py:SenselabVADProcessor— removes non-speech segments, returns speech-only waveform tensorSenselabEGeMAPSProcessor— extracts 88 hand-crafted eGeMAPS acoustic featuresSenselabEmbeddingProcessor— extracts deep neural embeddings via WavLM or ECAPA-TDNNAll three follow the existing
AudioProcessorpattern: lazy imports,ImportErrorwith install hint,process()takes a file path, returns atorch.Tensor. Processors are registered in the processor registry and exported from__init__.py.Tests are in
tests/processors/test_senselab_processor.pyusingpytest.importorskip("senselab")so they skip gracefully in environments where senselab is unavailable.Note on dev environment
senselab>=1.3.0requirestorch>=2.8.0with CUDA libraries (~3GB), which conflicts with available disk space on GitHub Codespaces alongside PyHealth's own dependencies. Local macOS testing is blocked byllvmlite 0.47requiring LLVM 20 while Homebrew ships LLVM 22. Happy to test in a larger environment if maintainers can advise on the recommended dev setup for audio dependencies.