Skip to content

5080deploy: opt-in on-demand experts for short prefills (FREETOKEN_MOE_SMALL_PREFILL_TOKENS, off) - #68

Open
gdevenyi wants to merge 3 commits into
5080/27-ple-fill-before-launchfrom
5080/28-small-prefill-on-demand
Open

5080deploy: opt-in on-demand experts for short prefills (FREETOKEN_MOE_SMALL_PREFILL_TOKENS, off)#68
gdevenyi wants to merge 3 commits into
5080/27-ple-fill-before-launchfrom
5080/28-small-prefill-on-demand

fix(moe): ensure a small prefill's experts in decode-width chunks

283cb2e
Select commit
Loading
Failed to load commit list.

There are no checks for this commit