Send the provider a combinator-free planner schema - #2052
Merged
Merged
Conversation
Since #1854 expanded the planner acceptance's per-kind oneOf branches, every planner call to the hosted vLLM gateway has failed with an empty HTTP 500 (followed by 429s), the planner node parked on SupervisorInfraPark, and the live planner gates measured nothing on every main run. The same model answers every other caller, and heads without #1854 got 200s. vLLM compiles the forced tool's schema into a decoding grammar, and its error names no keyword. The request now carries an optional WireJsonSchema that both provider clients send as the tool schema, falling back to JsonSchema. The planner sends PlannerSchema.WireSchema: ResponseSchema with oneOf, anyOf, allOf, not, if, then and else stripped at every schema position, which leaves the flat typed acceptance the gateway answered before #1849 while still declaring every acceptance field. Dropping a combinator only removes a constraint, so the wire schema accepts every reply the contract does; validation, the bounded re-ask and the schema quoted in the prompt all keep the full ResponseSchema. A guard pins every schema the code sends to a model as combinator-free on the wire. The park log and the E2E ride's unresolved-park message now carry the fault text, so the next outage shows the gateway's words.
3 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
litellm.InternalServerError: InternalServerError: Hosted_vllmException - {"error":{"message":"","type":"InternalServerError","param":null,"code":500}}), followed by 429s. The planner node parks onSupervisorInfraPark, and the live planner gates have been dark on main since. vLLM compiles the forced tool's schema into a decoding grammar, and Expose complete oracle shapes to structured planner models #1854's per-kind acceptanceoneOfbranches break that compile. Other callers of the same model get 200s.StructuredLLMCompletionRequest.WireJsonSchemais a new optional field.AnthropicClientsends it astools[0].input_schema, andOpenAiClient(whichCustomClientdelegates to) sends it asfunction.parameters. When it is null, both fall back toJsonSchema. The planner sendsPlannerSchema.WireSchema, which isResponseSchemapassed throughJsonSchemaCombinators.Strip.StripdropsoneOf/anyOf/allOf/not/if/then/elseat every schema position and keeps property names and literals. The result is the flat typed acceptance the gateway answered before Express oracle payload requirements in the model schema #1849. It still declaresformatVersion,kind,argv,artifactPaths,oraclePaths,rubricandschema.ResponseSchema(StructuredResponseValidation.cs:137is unchanged), so the per-kind requirements are still enforced. The wire schema is the full schema minus one conjunct, so it accepts everything the full schema accepts.InfraParknow logs the fault message. When a park outlives the E2E ride,InfraParkRide's skip message now appends theerrorstored on the park marker.Test plan
StructuredResponseContractTests.The_planner_request_sends_the_provider_a_combinator_free_schema_that_still_declares_every_acceptance_fieldfailed for Anthropic and OpenAI with 6 combinator paths (acceptance.oneOfplus 5not).JsonSchemaCombinatorsTests.Every_schema_the_code_sends_to_a_model_is_combinator_free_on_the_wirecovers 14 wire forms. A reflection check fails if Core gains a static*Schemafield that is not listed.TypedModelSchemaBranchTestschecks that the wire schema equals the full schema minus the acceptanceoneOf, and that every valid oracle reply is also valid on the wire.A_reply_the_wire_schema_admits_but_the_contract_rejects_still_earns_the_reask_and_the_faultchecks that validation is unchanged.WireSchema = ResponseSchema: the capture test (both providers), the guard and 2 wire tests go red.WireJsonSchema: the capture test goes red (both providers).TapeSummarySchemaplus an unlisted*Schemafield: the guard and the completeness check go red.PlannerCassetteDriftTests.ExpectedPlannerKeyis unchanged because the cassette key hashesJsonSchema, not the wire schema. PR CI is green. The unit lane's first attempt failed once inLocalProcessDurableRunnerTests.A_controllers_deadline_stop_is_timed_out_whatever_the_clock_or_the_oom_counter_reads(IOException: No such process: '/proc/<pid>/stat'), a liveness race inNativeProcess.IsRunningthat this PR does not touch; the rerun passed.ok 2 0 0 RealModelPlannerandok 1 0 0 Live_planner_keeps_executable_content_checks_separate_from_file_obligations_and_the_real_oracle_rejects_wrong_content. Main run 36565851194 at 2eae0e5 hadUNMEASURED 0 0 2andUNMEASURED 0 0 1.POST /v1/messages): the acceptance arm got 200 (4429 ms) plus a re-ask 200 (3390 ms); the whole-loop planner got 200 (5646 ms) plus a re-ask 200 (4088 ms).plan.authordropped all 4 acceptances (TestsPass withoutargv, LlmJudge withoutartifactPaths). The report-only acceptance arm logged the same miss asINFORMATIONAL-FAIL: "content-equals-accepted (TestsPass): Planner acceptance requires a non-empty argv array.; report-file-present (ArtifactPresent): Planner acceptance requires a non-empty artifactPaths array." That arm's class doc already records this model-quality gap; it is now measured instead of dark.claude-code, which is rejected on turns 2–9 as naming no active persona. The run then stopped for no progress, and arbitration turned Success into Failure. This PR does not change that path: supervisor requests set noWireJsonSchema, so their tool schema is byte-identical. The same rejection appears 4–7 times in each of main's last three supervisor runs.