refactor(tokenizer): drop inert __slots__, inline single-use num_embeddings - #9

refactor(tokenizer): drop inert __slots__, inline single-use num_embeddings#9

AmitMY merged 1 commit into

mainsign/utf8-tokenizer:mainfrom

simplify/inert-slots-and-encode-mergesign/utf8-tokenizer:simplify/inert-slots-and-encode-mergeCopy head branch name to clipboard