Large top-k values overflow int32 in sampler preparation #789

What happened

A non-greedy request with top_k=2**31 fails in Sampler.prepare() when the accepted Python integer is converted to int32. 2**80 also fails, and one such row prevents preparation of a mixed batch.

Keeping at least the vocabulary size should be equivalent to disabling top-k. Bound it to the vocabulary before constructing the int32 tensor.

Reproduction

Source checkout 546c5fe2e4468e2f3f8a574c64021a47016c35f7 (0.2.0). No checkpoint is needed: the failure occurs during sampler argument preparation.

Save as reproduce.py:

"""Reproduce top-k integer overflow without loading a model."""
import torch
from freetoken.core import Batch, Req, SamplingParams
from freetoken.engine.sample import Sampler

params = SamplingParams(temperature=0.7, top_k=2**31)
req = Req(torch.tensor([0], dtype=torch.int32), 0, 0, 4, 0, params, None)
args = Sampler(torch.device('cuda'), vocab_size=17).prepare(Batch([req], phase='decode'))
print('prepared top_k:', args.top_k)

From the checkout (PowerShell):

$env:PYTHONPATH = "$PWD/python"
python -B reproduce.py

Full output on main (local directory prefixes sanitized as <repro> and <repo>):

Traceback (most recent call last):
  File "<repro>\reproduce.py", line 8, in <module>
    args = Sampler(torch.device('cuda'), vocab_size=17).prepare(Batch([req], phase='decode'))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>\python\freetoken\engine\sample.py", line 93, in prepare
    top_k = make_device_tensor(top_ks, torch.int32, self.device)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<repo>\python\freetoken\engine\sample.py", line 23, in make_device_tensor
    return torch.tensor(data, dtype=dtype, pin_memory=True).to(device, non_blocking=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: value cannot be converted to type int32 without overflow

Environment

Windows 11 build 26200; Python 3.12.10; Torch 2.14.1+cu130; Triton Windows 3.8.0.post29; RTX 5070 Ti 16 GB, driver 616.56; Ryzen 7 9700X, approximately 64 GB RAM. No model weights loaded.

FAQ, roadmap and existing issues/PRs checked. #223 addresses API parameter validation; this is the sampler's integer bound. AI-assisted reproduction, executed through development tooling.