Skip to content

API: Report prompt cache and speculative decoding counters in usage stats - #452

Open
DenysAshikhin wants to merge 1 commit into
theroyallab:mainfrom
DenysAshikhin:feat/usage-cache-and-draft-counters
Open

API: Report prompt cache and speculative decoding counters in usage stats#452
DenysAshikhin wants to merge 1 commit into
theroyallab:mainfrom
DenysAshikhin:feat/usage-cache-and-draft-counters

API: Report prompt-cache and speculative-decoding counters in usage s…

973b22b
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs