Metadata-Version: 2.1
Name: llama-cpp-cffi
Version: 0.0.3
Summary: Python binding for llama.cpp using cffi
Home-page: https://github.com/mtasic85/llama-cpp-cffi
License: MIT
Author: Marko Tasic
Author-email: mtasic85@gmail.com
Requires-Python: >=3.10,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: attrs (>=23.2.0,<24.0.0)
Requires-Dist: cffi (>=1.16.0,<2.0.0)
Requires-Dist: huggingface-hub (>=0.23.4,<0.24.0)
Requires-Dist: jinja2 (>=3.1.4,<4.0.0)
Requires-Dist: protobuf (>=5.27.2,<6.0.0)
Requires-Dist: psutil (>=6.0.0,<7.0.0)
Requires-Dist: sentencepiece (>=0.2.0,<0.3.0)
Requires-Dist: setuptools (>=70.2.0,<71.0.0)
Requires-Dist: transformers (>=4.42.4,<5.0.0)
Project-URL: Repository, https://github.com/mtasic85/llama-cpp-cffi
Description-Content-Type: text/markdown

# llama-cpp-cffi

Python binding for llama.cpp using cffi

## Build

```bash
#
# setup venv
#
python -m venv venv
source venv/bin/activate
pip install poetry

#
# build
#

# x86_64
poetry run cibuildwheel --output-dir wheelhouse --platform linux --arch x86_64 .

# aarch64
docker run --rm --privileged linuxkit/binfmt:v0.8
poetry run cibuildwheel --output-dir wheelhouse --platform linux --arch aarch64 .

# pyodide, pyscript, wasm (NOTE: cannot be published to PyPI)
# poetry run cibuildwheel --output-dir wheelhouse --platform pyodide .

#
# publish
#
poetry publish --dist-dir wheelhouse

#
# run demos
#
python -B examples/demo_cffi.py
python -B examples/demo_ctypes.py
python -m http.server -d examples/demo_pyonide -b "0.0.0.0" 5000
```

```bash
make -j llama-cli-shared llama-cli-static GGML_NO_OPENMP=1 GGML_NO_LLAMAFILE=1
```
