Prosus Vending Bench is released from the main branch. Fixes are applied to
the latest release only.
Please do not report security vulnerabilities through public GitHub issues.
Use one of these private channels instead:
- GitHub private vulnerability reporting (preferred): the repository's Security tab → Report a vulnerability.
- Email:
cyber@prosus.com
Please include:
- A description of the vulnerability and its impact
- Steps to reproduce or a proof of concept
- Affected version or commit
- Any suggested mitigation
We will acknowledge receipt within 5 business days and aim to give an assessment and a remediation timeline within 15 business days. Please give us a reasonable opportunity to address the issue before public disclosure.
This repository is a benchmark, not a hosted service. A few design points worth knowing when assessing a report:
- The simulator is not multi-tenant.
sim-serverholds one world in one process with no authentication, and it is meant to run on a private network inside a single trial's Docker Compose project. Exposing it to a shared network is a deployment mistake, not a supported configuration. Isolate every trial. - The agent under test is untrusted by design. The scoring contract assumes
it will probe. There is no live
/stateor/scoreendpoint;POST /verifier/finalizeirreversibly seals a run before returning the score, so an agent that reaches it forfeits the run rather than gaining an advantage. Sealing is serialized with tool calls. A way to read or change the score without sealing, or to keep operating after sealing, is a real finding. - Configuration is arbitrary-code-adjacent.
VENDING_CONFIG,VENDING_SETand theVENDING_*variables load and override a TOML file from disk. Only point them at files you control. - No credentials live here.
scripts/run_openrouter.pyreads onlyOPENROUTER_API_KEYfrom an env file you pass explicitly, andscripts/run_models.pyuses your already-authenticated Claude CLI. Keys are never written into results. A path that leaks a key into a result directory, a log or a trajectory is a real finding. - Published results contain model output. Trajectories and conversations
under
results/are raw model text. Treat them as untrusted data, not as instructions.
If you find a way to read or alter a score without sealing the run, escape the simulated tool surface, or leak credentials into results, we want to hear about it.