Skip to content

fix: answer jobs with a TRUST_REMOTE_CODE hint instead of crash-looping - #361

Merged
velaraptor-runpod merged 2 commits into
runpod-workers:mainfrom
ashwinsreedhar28:fix/remote-code-startup-error
Oct 21, 2026
Merged

velaraptor-runpod merged 2 commits into
runpod-workers:mainfrom
ashwinsreedhar28:fix/remote-code-startup-error

Conversation

@ashwinsreedhar28

Copy link
Copy Markdown
Contributor

When loading a model requires code from its own repository and TRUST_REMOTE_CODE isn't set, vLLM fails config validation with a ValidationError asking for trust_remote_code=True. The worker doesn't recognise that error, so it exits and is restarted into the same failure, and the job never gets an answer. I hit this with IFM/K2-Horizon-7B on v2.27.2 and v2.28.0: the worker failed config validation on every start, and the submitted job had no result after each 15-minute observation window (log, runs).

This adds it to startup_errors.classify, so the worker stays up and answers jobs with the fix: review the repository's code, then set TRUST_REMOTE_CODE=true. It's checked before the unsupported-architecture match on purpose. With remote code allowed, vLLM can serve an architecture it has no native class for through its Transformers backend: K2-Horizon loaded that way on v2.27.2 (vLLM 0.28.0, no K2-Horizon class).

How often this can come up: 8 of the 25 new-architecture repos in my sample map AutoModel* to repository code in config.json (data). That's potential exposure, not 8 reproduced failures; transformers can still use a built-in implementation when one exists.

Evidence:

check result
python -m pytest tests 144 pass (142 on main + 2 new in tests/test_startup_errors.py, using the error text from the K2-Horizon log)
image from this branch at c1b457f (FROM vllm/vllm-openai:v0.32.0, build; the later commit only edits README.md) on Serverless, IFM/K2-Horizon-7B, 24 GB class, no TRUST_REMOTE_CODE worker stays up; job FAILED with startup_error 2.8 min after endpoint creation: "IFM/K2-Horizon-7B ships its own modeling code on Hugging Face (an auto_map in its config.json), and vLLM only runs that code with TRUST_REMOTE_CODE=true. Review the repository's code, then set TRUST_REMOTE_CODE=true if you trust it. …" (result)
same image with TRUST_REMOTE_CODE=true first request COMPLETED 4.6 min after endpoint creation (result)

The secret-gated jobs (dev image build, serverless-model-tests) don't run on a fork PR; happy for them to be mirrored like #352.

Study, build logs and results: https://github.com/ashwinsreedhar28/day0-ready

ashwinsreedhar28 and others added 2 commits October 21, 2026 19:49
When loading a model requires code from its repository and TRUST_REMOTE_CODE
isn't set, vLLM fails config validation with a ValidationError asking for
trust_remote_code=True. The worker didn't recognise it, so it exited and was
restarted into the same error, and the job got no answer. Seen with
IFM/K2-Horizon-7B on v2.27.2 and v2.28.0: the worker failed the same way on
every start, and the submitted job had no result after 15 minutes.

Classify it like the other fatal startup failures and tell the user to set
TRUST_REMOTE_CODE=true after reviewing the code. It is checked before the
unsupported-architecture match because with remote code allowed, vLLM can
serve an architecture it has no native class for through its Transformers
backend (K2-Horizon loaded that way on vLLM 0.28.0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxoD6sb3tVrig5ddmB8f1
Only a model that requires repository-provided code fails without
TRUST_REMOTE_CODE, and its failure happens at config validation, before
any weights load, with the error in the logs. Say what holds for every
case in the list: the startup repeats and jobs get no result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxoD6sb3tVrig5ddmB8f1
@velaraptor-runpod
velaraptor-runpod self-requested a review October 21, 2026 22:15
@velaraptor-runpod
velaraptor-runpod merged commit 71d503e into runpod-workers:main Oct 21, 2026
5 of 8 checks passed
@promptless

promptless Bot commented Oct 21, 2026

Copy link
Copy Markdown

Promptless documentation updates

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants