Metadata-Version: 2.1
Name: fasttext-parallel
Version: 0.1.3
Classifier: Programming Language :: Rust
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: numpy >= 1.20.0
Requires-Dist: fasttext; extra == 'tests'
Provides-Extra: tests
License-File: LICENSE
Summary: FastText Multithreading Inference
Keywords: fasttext
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

# Multithreading For FastText

An easy to use tool for multithreading fasttext batch inference.

## Features
- Fast inference using multithreading utilizing all cpu cores.
- Predicted labels are encoded as i16 numpy array for small memory footprint and easy serialization.

## Performance

On macbook Air M2 chip, the performance is about 4 times faster,
which is reasonable since it has 4 performance cores.

```
❯ python -m unittest discover test
fasttext 1.7130832079565153
fasttext-parallel 6.7440170829650015
```

## Usage

```
pip install fasttext-parallel
```

```python
import fasttext_parallel as ft
model = ft.load_model("./model/lid.176.bin")

# this uses multiple threads
labels, probabilities = model.batch(["你好", "how are you"])

# labels are in a format of numpy.ndarray (i16) format
# to get actual label, call get_label_by_id
assert model.get_label_by_id(labels[0][0]) == "__label__zh"
assert model.get_label_by_id(labels[1][0]) == "__label__en"

# to view all labels (a dict from label_id to label)
print(model.get_labels())
```

