Repository navigation
fix: answer jobs with a TRUST_REMOTE_CODE hint instead of crash-looping - #361
Merged
velaraptor-runpod merged 2 commits intoOct 21, 2026
Conversation
When loading a model requires code from its repository and TRUST_REMOTE_CODE isn't set, vLLM fails config validation with a ValidationError asking for trust_remote_code=True. The worker didn't recognise it, so it exited and was restarted into the same error, and the job got no answer. Seen with IFM/K2-Horizon-7B on v2.27.2 and v2.28.0: the worker failed the same way on every start, and the submitted job had no result after 15 minutes. Classify it like the other fatal startup failures and tell the user to set TRUST_REMOTE_CODE=true after reviewing the code. It is checked before the unsupported-architecture match because with remote code allowed, vLLM can serve an architecture it has no native class for through its Transformers backend (K2-Horizon loaded that way on vLLM 0.28.0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016kxoD6sb3tVrig5ddmB8f1
Only a model that requires repository-provided code fails without TRUST_REMOTE_CODE, and its failure happens at config validation, before any weights load, with the error in the logs. Say what holds for every case in the list: the startup repeats and jobs get no result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016kxoD6sb3tVrig5ddmB8f1
velaraptor-runpod
self-requested a review
October 21, 2026 22:15
velaraptor-runpod
approved these changes
Oct 21, 2026
Promptless documentation updates
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When loading a model requires code from its own repository and
TRUST_REMOTE_CODEisn't set, vLLM fails config validation with a ValidationError asking fortrust_remote_code=True. The worker doesn't recognise that error, so it exits and is restarted into the same failure, and the job never gets an answer. I hit this with IFM/K2-Horizon-7B on v2.27.2 and v2.28.0: the worker failed config validation on every start, and the submitted job had no result after each 15-minute observation window (log, runs).This adds it to
startup_errors.classify, so the worker stays up and answers jobs with the fix: review the repository's code, then setTRUST_REMOTE_CODE=true. It's checked before the unsupported-architecture match on purpose. With remote code allowed, vLLM can serve an architecture it has no native class for through its Transformers backend: K2-Horizon loaded that way on v2.27.2 (vLLM 0.28.0, no K2-Horizon class).How often this can come up: 8 of the 25 new-architecture repos in my sample map
AutoModel*to repository code inconfig.json(data). That's potential exposure, not 8 reproduced failures; transformers can still use a built-in implementation when one exists.Evidence:
python -m pytest teststests/test_startup_errors.py, using the error text from the K2-Horizon log)FROM vllm/vllm-openai:v0.32.0, build; the later commit only edits README.md) on Serverless, IFM/K2-Horizon-7B, 24 GB class, noTRUST_REMOTE_CODEstartup_error2.8 min after endpoint creation: "IFM/K2-Horizon-7B ships its own modeling code on Hugging Face (an auto_map in its config.json), and vLLM only runs that code with TRUST_REMOTE_CODE=true. Review the repository's code, then set TRUST_REMOTE_CODE=true if you trust it. …" (result)TRUST_REMOTE_CODE=trueThe secret-gated jobs (dev image build,
serverless-model-tests) don't run on a fork PR; happy for them to be mirrored like #352.Study, build logs and results: https://github.com/ashwinsreedhar28/day0-ready