Skip to content

fix(helpers): await async system results during evaluation - #36

Open
Yash1hi wants to merge 2 commits into
mainfrom
fix/async-run-and-evaluate
Open

Yash1hi wants to merge 2 commits into
mainfrom
fix/async-run-and-evaluate

Conversation

@Yash1hi

@Yash1hi Yash1hi commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Summary

async_run_and_evaluate passed awaitable system results directly to record creation, producing coroutine outputs or errors. It now awaits those results while continuing to support synchronous systems and concurrent trials. The synchronous helper raises a clear error directing callers to the async helper when a system returns an awaitable.

Changes

  • Await system results before creating records.
  • Update async system typing and helper docstrings.
  • Reject awaitable results in the sync helper and close rejected native coroutines.
  • Add mocked-client regression coverage without network access.

Testing

  • .venv/bin/python -m pytest -q tests/lib/test_helpers.py — 24 passed.
  • .venv/bin/ruff check src/scorecard_ai/lib/_helpers.py tests/lib/test_helpers.py — passed.
  • .venv/bin/ruff format --check src/scorecard_ai/lib/_helpers.py tests/lib/test_helpers.py — passed.
  • .venv/bin/pyright src/scorecard_ai/lib/_helpers.py tests/lib/test_helpers.py — 0 errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant