Repository navigation
Feat/speech service - #25
Conversation
…ckages Add two new Robonix packages forming the voice interaction pipeline: - speech_service (robonix/srv/speech): gRPC service with ASR (Whisper + FunASR Paraformer), TTS (Edge TTS), and Dialog session management. Includes audio_utils for auto-adapting any input format to 16kHz mono pcm_s16le, eliminating the need for callers to pre-process audio. - audio_driver (robonix/prm/audio): ALSA auto-detection and streaming audio I/O via arecord/aplay subprocesses. Designed as a reference template for future prm driver packages (driver registry pattern, tiago_bridge startup flow, separate gRPC servers per interface). Also adds 7 contract TOMLs (2 prm audio + 5 srv speech), 19 ROS IDL message/service definitions (asr, tts, speech packages), generated proto files, and integrates both packages into the run.sh orchestration script with START_SPEECH_SERVICE / START_AUDIO_DRIVER env toggles. Smoke tested: Atlas registration, contract discovery via QueryNodes, and ASR/TTS RPCs all pass end-to-end.
… additions in robonix_msg.proto The previous commit accidentally included unrelated deletions from slam.proto, prm_base.proto, and robonix_msg.proto that were pre-existing local changes. This restores those files to their dev state and only adds the new audio/speech messages (AudioChunk, AudioConfig, DialogEvent, DialogState, VadEvent) to robonix_msg.proto.
… ID as default Replace hardcoded /home/zzb/... path with openai/whisper-large-v3 so the model auto-downloads on first run via HuggingFace Hub.
Each backend (Whisper, FunASR, Edge TTS) is now initialized independently with try/except: - Failure → clear boxed error message with actionable fix options (download model, set env var, or use CI mode) - Service still starts with whatever backends ARE available - Unavailable backends return gRPC UNAVAILABLE with fix instructions - If ALL backends fail, exit(1) with guidance No silent degradation — callers get explicit errors telling them exactly what's missing and how to fix it.
Review — blocking issues before mergeNice architecture split (ASR / TTS / Dialog, CI mode fallback, ALSA auto-discovery). Before we can merge, four things need addressing: 1.
|
| stream_response = "asr/msg/RecognizeStreamEvent" | ||
|
|
||
| [mode] | ||
| type = "rpc_bidirectional_stream" |
There was a problem hiding this comment.
robonix codegen (input: toml+ROS IDL, output: proto, mcp types, ...) currently does not support "rpc_bidirectional_stream" communication model
There was a problem hiding this comment.
please do not commit anything in rust/crates/robonix-interfaces/robonix_proto because it's scheduled to be removed in repo
these protos should only be in the "generated stuffs" under a robonix package
There was a problem hiding this comment.
Please use English documentation and comments in this repo, thanks
There was a problem hiding this comment.
as discussed before, we should do the "checkpoints/models/..." downloading here instead of downloading it at "first time running the service".
…h docs
1. Switch to rbnx codegen for proto stubs
- Delete hand-written speech_service.proto / audio_driver.proto
- Keep speech_service.proto as docs/speech_service.proto.ref
- build.sh now only calls `rbnx codegen`, no grpc_tools.protoc
- service.py / node.py import from codegen-generated stubs
(robonix_contracts_pb2_grpc, asr_pb2, tts_pb2, etc.)
2. Fix ROS IDL .srv files
- Move --- separator to correct position (request fields before ---)
- This fixes empty Request messages in generated protos
3. Add rpc_bidirectional_stream to codegen
- New mode supported in contract_gen.rs
- Uses stream_request / stream_response from contract TOML
- Generates: rpc Stream(stream InType) returns (stream OutType)
4. Add speech/audio services to robonix_contracts.proto
- SrvSpeechAsr, SrvSpeechAsrStream, SrvSpeechTts,
SrvSpeechTtsStream, SrvSpeechDialog
- PrmAudioMic, PrmAudioSpeaker
5. Model download at build time, not runtime
- build.sh downloads Whisper + FunASR models
- service.py uses local_files_only=True — fail fast if missing
- No silent network downloads on boot
6. Translate all Chinese to English
- audio_driver/README.md, speech_service/README.md
- All code comments in English
Add asr, tts, speech modules to contract_proto_modules.rs so tonic::include_proto! resolves the new message types referenced by robonix_contracts.proto services.
|
LGTM |
fix : modify the files context & delete the test files
add system speech service + example audio primitive driver support on dev branch. new package spec and examples refactor are scheduled later (currently WIP in `dev-packaging` branch)
feat(speech): 语音交互服务 — speech_service + audio_driver
Summary
为 Robonix 新增两个松耦合的 srv/prm 层包，提供完整的语音交互基础能力：
gRPC 接口
对外暴露
主要改动
新增包
TTS（语音合成）
模式），不崩服务
接口定义 & 契约
消息
启动集成
环境变量控制，默认启用
文件统计
52 files changed, +3544 / -19 行
协作者须知
SpeechBridge（TODO）消费