Skip to content

fix(runs): resolve run URLs for custom API base URLs - #37

Open
Yash1hi wants to merge 1 commit into
mainfrom
fix/run-url-custom-base-url
Open

Yash1hi wants to merge 1 commit into
mainfrom
fix/run-url-custom-base-url

Conversation

@Yash1hi

@Yash1hi Yash1hi commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Summary

run_and_evaluate, async_run_and_evaluate, and the other run helpers returned a staging app link whenever the API base URL didn't exactly match one of the built-in environments, for example a proxy in front of production or a local backend on another port. Run links now match the production, staging, and local environments after normalizing the URL, fall back to the production app for unrecognized hosts, and honor a new SCORECARD_APP_URL override for dedicated deployments.

Changes

  • src/scorecard_ai/lib/_helpers.py: resolve the app URL from the three ENVIRONMENTS entries (trailing slashes and scheme/host case ignored); unknown hosts use https://app.scorecard.io; SCORECARD_APP_URL overrides it, minus trailing slashes.
  • tests/lib/test_run_url.py: offline tests for each environment, normalization, unknown hosts, the override, and every run helper.
  • The generated client's base_app_url property is unchanged; it still falls back to staging for unknown hosts and lives in Stainless-managed code.

The TypeScript SDK gets the same rules in scorecard-ai/scorecard-node#51.

Testing

  • .venv/bin/python -m pytest -q tests/lib/test_run_url.py: 55 passed.
  • ruff check, ruff format --check, and pyright on the touched files: passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant