Issues

• Replace custom HF tokenizer with C++ bindings for HF tokenizer

  meta-pytorch/tokenizers#153 · Filter by author jackzhxng opened on Oct 28, 2025
  

• Use prompt template from HF tokenizer

  enhancement
  meta-pytorch/tokenizers#131 · Filter by author metascroy opened on Oct 2, 2025
  

• Package tokenizers into swift PM

  meta-pytorch/tokenizers#83 · Filter by author larryliu0820 opened on Jun 11, 2025
  1

  

• Distinguish between public and internal headers

  meta-pytorch/tokenizers#52 · Filter by author kirklandsign opened on Apr 17, 2025
  1

• [Feature request] Java binding for tokenizers

  meta-pytorch/tokenizers#40 · Filter by author kirklandsign opened on Mar 31, 2025
  1