Metadata-Version: 2.1
Name: optimum-furiosa
Version: 0.1.0
Summary: Optimum Furiosa is the interface between the 🤗 Transformers library and Furiosa NPUs such as Furiosa Warboy. It provides a set of tools enabling easy model loading and inference for different downstream tasks for Furiosa NPU.
Home-page: https://huggingface.co/hardware
Author: HuggingFace Inc. Special Ops Team
Author-email: hardware@huggingface.co
License: Apache
Keywords: transformers,quantization,pruning,knowledge distillation,optimization,training
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: optimum (>=1.8.0)
Requires-Dist: transformers (>=4.20.0)
Requires-Dist: datasets (>=1.4.0)
Requires-Dist: furiosa-optimizer
Requires-Dist: furiosa-quantizer (==0.9.0)
Requires-Dist: furiosa-quantizer-impl (==0.9.1)
Requires-Dist: furiosa-sdk
Requires-Dist: onnx (>=1.12.0)
Requires-Dist: sentencepiece
Requires-Dist: scipy
Provides-Extra: quality
Requires-Dist: black (~=23.1) ; extra == 'quality'
Requires-Dist: ruff (>=0.0.241) ; extra == 'quality'
Provides-Extra: testing
Requires-Dist: filelock ; extra == 'testing'
Requires-Dist: GitPython ; extra == 'testing'
Requires-Dist: parameterized ; extra == 'testing'
Requires-Dist: psutil ; extra == 'testing'
Requires-Dist: pytest ; extra == 'testing'
Requires-Dist: pytest-pythonpath ; extra == 'testing'
Requires-Dist: pytest-xdist ; extra == 'testing'
Requires-Dist: Pillow ; extra == 'testing'
Requires-Dist: librosa ; extra == 'testing'
Requires-Dist: soundfile ; extra == 'testing'

[![Test](https://github.com/huggingface/optimum-furiosa/actions/workflows/test.yml/badge.svg)](https://github.com/huggingface/optimum-furiosa/actions/workflows/test.yml)


# optimum-furiosa
Accelerated inference of 🤗 models using FuriosaAI NPU chips.

## Furiosa SDK setup
A Furiosa SDK environment needs to be enabled to use this library. Please refer to Furiosa's [Installation](https://furiosa-ai.github.io/docs/latest/en/software/installation.html) guide.

## Install
To install the latest release of this package:

```bash
pip install optimum[furiosa]
```

Optimum Furiosa is a fast-moving project, and you may want to install from source.

`pip install git+https://github.com/huggingface/optimum-furiosa.git`

### Installing in developer mode

If you are working on the `optimum-furiosa` code then you should use an editable install
by cloning and installing `optimum` and `optimum-furiosa`:

```
git clone https://github.com/huggingface/optimum
git clone https://github.com/huggingface/optimum-furiosa
pip install -e optimum -e optimum-furiosa
```

Now whenever you change the code, you'll be able to run with those changes instantly.


## How to use it?
To load a model and run inference with Furiosa NPU, you can just replace your `AutoModelForXxx` class with the corresponding `FuriosaAIModelForXxx` class. 

```diff
import requests
from PIL import Image

- from transformers import AutoModelForImageClassification
+ from optimum.furiosa import FuriosaAIModelForImageClassification
from transformers import AutoFeatureExtractor, pipeline

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

model_id = "microsoft/resnet-50"
- model = AutoModelForImageClassification.from_pretrained(model_id)
+ model = FuriosaAIModelForImageClassification.from_pretrained(model_id, export=True, input_shape_dict={"pixel_values": [1, 3, 224, 224]}, output_shape_dict={"logits": [1, 1000]},)
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)
cls_pipe = pipeline("image-classification", model=model, feature_extractor=feature_extractor)
outputs = cls_pipe(image)
```

If you find any issue while using those, please open an issue or a pull request.
