Repository navigation

fix: O(n log n) byte pair merge and one shared Tokenizer declaration - #15

Open

ThomasK33 wants to merge 10 commits into

mainfrom

fix/bpe-quadratic-and-typing

Open

fix: O(n log n) byte pair merge and one shared Tokenizer declaration#15
ThomasK33 wants to merge 10 commits into
mainfrom
fix/bpe-quadratic-and-typing

Commits on Nov 3, 2026
- test: add timing test for long single-piece inputs
  ThomasK33
  committed
  7b589df
- perf: merge byte pairs in O(n log n)
  ThomasK33
  committed
  a704ff0
- ci: typecheck the README example against the packed tarball
  ThomasK33
  committed
  b3241f0
- build: share one Tokenizer declaration between entry points
  ThomasK33
  committed
  2eafae8
- ci: fail the package types check on any TypeScript diagnostic
  ThomasK33
  committed
  a93d793
- perf: assert the merge heap never overflows
  ThomasK33
  committed
  9bca535
- Merge origin/main into fix/bpe-quadratic-and-typing
  ThomasK33
  committed
  6f4a413
- test: compare pieces that contain U+FEFF in the differential test
  ThomasK33
  committed
  0d28d1b
Commits on Nov 4, 2026
- Merge origin/main into fix/bpe-quadratic-and-typing
  ThomasK33
  committed
  6a96d8b
- ci: typecheck a claude_v5 tokenizer through the packed types
  ThomasK33
  committed
  1125a60