Metadata-Version: 2.1
Name: quicktok
Version: 0.1.1
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Summary: Minimal, fast, multi-threaded implementation of the Byte Pair Encoding (BPE) for LLM tokenization
License: MIT
Requires-Python: >=3.8
