Skip to content

fix(grpc): default each unset sampling field independently - #2

Merged
autoantohaki merged 1 commit into
mainfrom
fix/grpc-partial-sampling
Oct 22, 2026
Merged

autoantohaki merged 1 commit into
mainfrom
fix/grpc-partial-sampling

Conversation

@autoantohaki

Copy link
Copy Markdown
Member

Requested by mau

Before: a gRPC request that set only some Sampling fields was rejected. With proto3 scalars, an unset top_p arrives as 0 and an unset max_tokens as 0, so {"sampling":{"maxTokens":32}} failed with INVALID_ARGUMENT: top_p must be greater than 0, and {"sampling":{"seed":7}} failed on max_tokens. Only "no sampling at all" or "every field set" worked.

After: each unset field falls back to the same default HTTP uses (max_tokens 256, temperature 0.8, top_p 0.95), so callers can set just what they need. An explicit temperature: 0 still means greedy decoding.

How: max_tokens, temperature, and top_p are now proto3 optional (wire-compatible: existing clients that send values are unaffected), and sampling() in grpc.rs defaults each field independently. A unit test covers partial, greedy, and absent sampling; the gRPC API doc notes the defaults.

CI needs #1 merged first (it restores the privacy-scan and supply-chain checks on main).

@autoantohaki
autoantohaki force-pushed the fix/grpc-partial-sampling branch from b50e70c to 6a948d7 Compare October 22, 2026 01:27
@autoantohaki
autoantohaki merged commit c5654a9 into main Oct 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant