5080deploy: generic fixes from 1080deploy (PLE per-fill readback events, host-tier probes, GDN workspace reserve, server fixes, --embed-weights host off) by gdevenyi · Pull Request #77 · gdevenyi/FreeToken

perf(ple): skip the device n-gram hash for the disk table, which reads only its shape

gdevenyi

committed

2896393

fix(ple): give each graph fill its own readback event and keep every pending fill

gdevenyi

committed

2657a34

fix(ple): recycle the fill readback events instead of freeing them on the filler

gdevenyi

committed

d892dc6

fix(kvcache): refuse the host kv-page tier on a pool whose index slab it cannot restore

gdevenyi

committed

81c6137

perf(kvcache): keep one gpu gdn snapshot slot when the host snapshot tier is on

gdevenyi

committed

1c736fb

perf(scheduler): probe the host tier for the exact tombstone prefix

gdevenyi

committed

23b110b

fix(scheduler): re-probe the host entry after making room to rehydrate it

gdevenyi

committed

b8de896

test(kvcache): cover the engine-stream waits of the prefix tier's kv page copies

gdevenyi

committed

ebaa599

fix(engine): reserve the GDN prefill workspace in the cache budget

fangyuan-3149

authored and

gdevenyi

committed

081e508

fix(engine): reserve the gdn prefill workspace in the rebuild fit-check too

gdevenyi

committed

5286dd0

fix(engine): count min_tokens progress from the device length

gdevenyi

committed

7d8614f

test(engine): import pytest for the parametrized min_tokens test

gdevenyi

committed

219cc98

fix(server): let protocol-level thinking requests win over --default-thinking-mode

gdevenyi

committed

8c61084

fix(server): type allOf-wrapped refs and untyped enum/const tool parameters

gdevenyi

committed

298e7df

fix(server): keep /v1/models/ listing the model after the retrieve route

gdevenyi

committed

86674ac

fix(server): open the streamed thinking block with the signature field

gdevenyi

committed

05a5540

test(server): exercise detector SDK and lifecycle contracts

gdevenyi

committed

10451a7

fix(server): drop /v1/completions from the --enable-metrics-report help, which it does not serve

gdevenyi

committed

e2c1de4

fix(server): keep developer ahead of system and reject images when merging system messages

gdevenyi

committed

7909266

test(server): ask for n=0 in the invalid-request sdk test, since n=2 is served here

gdevenyi

committed

21261f6

fix(moe): keep unpinned layers off the small-prefill path and take it in every layer

gdevenyi

committed

e5e9582

test(moe): let the small-prefill ensure fake take the eviction policy kwargs

gdevenyi

committed

9be869b

test(qwen4_exp): compare chunked prefill to one-shot at bf16 tolerance

gdevenyi

committed

a4fe795

test(qwen4_exp): compare the chunked prefill's compressed slab rows to the one-shot run

gdevenyi

committed

c95f299

fix(kernel): keep the triton sampling softmax finite over a fully masked prefix

gdevenyi

committed

f1dd670

feat(engine): keep the token embedding in pinned host ram (--embed-weights host)

gdevenyi

committed

f5fdc70

fix(layers): gather fp32 host embeddings and refuse other dtypes at placement

gdevenyi

committed

d4d9e52

fix(kernel): compute the host gather output offset in int64

gdevenyi

committed

243c70b

fix(engine): charge the host embedding's pinned bytes to the pin budget

gdevenyi

committed

1140b36

fix(engine): refuse --embed-weights host for a tied embedding or tp > 1 at config time

gdevenyi

committed

764c357

test(engine): fake cached_load_hf_config with the overrides argument #573 passes

gdevenyi

committed

99f01b2

test(ple): drive overlapped graph fills against an in-order fake stream so a post-launch fence fails as a hang

gdevenyi

committed

2224380

Navigation Menu

Repository navigation

5080deploy: generic fixes from 1080deploy (PLE per-fill readback events, host-tier probes, GDN workspace reserve, server fixes, --embed-weights host off) - #77

5080deploy: generic fixes from 1080deploy (PLE per-fill readback events, host-tier probes, GDN workspace reserve, server fixes, --embed-weights host off)#77
gdevenyi wants to merge 32 commits into
5080/34-upstream-prsfrom
5080/35-1080deploy-fixes

Commits on Oct 2, 2026

perf(ple): skip the device n-gram hash for the disk table, which reads only its shape

fix(ple): give each graph fill its own readback event and keep every pending fill

fix(ple): recycle the fill readback events instead of freeing them on the filler

fix(kvcache): refuse the host kv-page tier on a pool whose index slab it cannot restore

perf(kvcache): keep one gpu gdn snapshot slot when the host snapshot tier is on

perf(scheduler): probe the host tier for the exact tombstone prefix

fix(scheduler): re-probe the host entry after making room to rehydrate it

test(kvcache): cover the engine-stream waits of the prefix tier's kv page copies

fix(engine): reserve the GDN prefill workspace in the cache budget

fix(engine): reserve the gdn prefill workspace in the rebuild fit-check too

fix(engine): count min_tokens progress from the device length

test(engine): import pytest for the parametrized min_tokens test

fix(server): let protocol-level thinking requests win over --default-thinking-mode

fix(server): type allOf-wrapped refs and untyped enum/const tool parameters

fix(server): keep /v1/models/ listing the model after the retrieve route

fix(server): open the streamed thinking block with the signature field

test(server): exercise detector SDK and lifecycle contracts

fix(server): drop /v1/completions from the --enable-metrics-report help, which it does not serve

fix(server): keep developer ahead of system and reject images when merging system messages

test(server): ask for n=0 in the invalid-request sdk test, since n=2 is served here

fix(moe): keep unpinned layers off the small-prefill path and take it in every layer

test(moe): let the small-prefill ensure fake take the eviction policy kwargs

test(qwen4_exp): compare chunked prefill to one-shot at bf16 tolerance

test(qwen4_exp): compare the chunked prefill's compressed slab rows to the one-shot run

fix(kernel): keep the triton sampling softmax finite over a fully masked prefix

feat(engine): keep the token embedding in pinned host ram (--embed-weights host)

fix(layers): gather fp32 host embeddings and refuse other dtypes at placement

fix(kernel): compute the host gather output offset in int64

fix(engine): charge the host embedding's pinned bytes to the pin budget

fix(engine): refuse --embed-weights host for a tied embedding or tp > 1 at config time

test(engine): fake cached_load_hf_config with the overrides argument #573 passes

test(ple): drive overlapped graph fills against an in-order fake stream so a post-launch fence fails as a hang