Metadata-Version: 2.1
Name: lexikanon
Version: 0.1.4
Summary: A Python Library for Tokenizers
Home-page: https://lexikanon.entelecheia.ai
License: MIT
Author: Young Joon Lee
Author-email: entelecheia@hotmail.com
Requires-Python: >=3.8.1,<3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: click (>=8.1.3,<9.0.0)
Requires-Dist: hyfi (>=1.8.3,<2.0.0)
Project-URL: Repository, https://github.com/entelecheia/lexikanon
Description-Content-Type: text/markdown

# Lexikanon

[![pypi-image]][pypi-url]
[![version-image]][release-url]
[![release-date-image]][release-url]
[![license-image]][license-url]
[![codecov][codecov-image]][codecov-url]
[![jupyter-book-image]][docs-url]

<!-- Links: -->
[codecov-image]: https://codecov.io/gh/entelecheia/lexikanon/branch/main/graph/badge.svg?token=KGST5XVW3F
[codecov-url]: https://codecov.io/gh/entelecheia/lexikanon
[pypi-image]: https://img.shields.io/pypi/v/lexikanon
[license-image]: https://img.shields.io/github/license/entelecheia/lexikanon
[license-url]: https://github.com/entelecheia/lexikanon/blob/main/LICENSE
[version-image]: https://img.shields.io/github/v/release/entelecheia/lexikanon?sort=semver
[release-date-image]: https://img.shields.io/github/release-date/entelecheia/lexikanon
[release-url]: https://github.com/entelecheia/lexikanon/releases
[jupyter-book-image]: https://jupyterbook.org/en/stable/_images/badge.svg

[repo-url]: https://github.com/entelecheia/lexikanon
[pypi-url]: https://pypi.org/project/lexikanon
[docs-url]: https://lexikanon.entelecheia.ai
[changelog]: https://github.com/entelecheia/lexikanon/blob/main/CHANGELOG.md
[contributing guidelines]: https://github.com/entelecheia/lexikanon/blob/main/CONTRIBUTING.md
<!-- Links: -->

A Python Library for Tokenizers

- Documentation: [https://lexikanon.entelecheia.ai][docs-url]
- GitHub: [https://github.com/entelecheia/lexikanon][repo-url]
- PyPI: [https://pypi.org/project/lexikanon][pypi-url]

Lexikanon is a robust and efficient Python library designed for creating, training, and deploying tokenizers, an essential component in natural language processing (NLP) and artificial intelligence (AI) applications. The name Lexikanon originates from the Greek words λέξη (word) and κάνων (maker), reflecting the library's purpose in enabling users to build powerful tokenizers for various languages and tasks.

## Changelog

See the [CHANGELOG] for more information.

## Contributing

Contributions are welcome! Please see the [contributing guidelines] for more information.

## License

This project is released under the [MIT License][license-url].

