Skip to content

Add Daft audio stability benchmark for Ray - #22

Merged
FANNG1 merged 1 commit into
mainfrom
benchmark-audio-stability
Jul 15, 2026
Merged

FANNG1 merged 1 commit into
mainfrom
benchmark-audio-stability

Conversation

@FANNG1

@FANNG1 FANNG1 commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Summary

  • add a standalone benchmark CLI for data generation, Mock LLM serving, direct Ray execution, Ray Jobs submission, and report generation
  • exercise the full MinIO audio download, SenseVoice ASR, Mock LLM, and Lance Blob v2 path with per-row failure status
  • collect Daft event logs and per-node resource samples, apply memory SLO reporting, partition manifests for distributed execution, and preserve reports on a configurable shared path
  • document local usage, one-hour sizing, and scaling to a 4-worker Ray cluster without starting Docker or MinIO

Validation

  • uv run pytest -q: 114 passed, 1 deselected
  • real local-smoke --count 4 against the configured MinIO and local Ray: 4/4 rows succeeded and Blob v2 validation passed
  • smoke processed 297.39 audio seconds in 146.39 wall seconds; ASR P95 was 4.55s and Ray process RSS peaked at about 5.11GB

Notes

  • the real end-to-end run has been validated locally; the documented 4-worker cluster configuration has not yet been exercised on a live multi-node cluster
  • benchmark data and Lance outputs use an isolated s3://benchmark/audio/<run-id>/ prefix

@FANNG1
FANNG1 merged commit 85d8d7b into main Jul 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant