Metadata-Version: 2.1
Name: greenformer
Version: 0.2.2
Summary: Automatic Factorization package for PyTorch modules
Home-page: https://github.com/SamuelCahyawijaya/greenformer
Author: Samuel Cahyawijaya
Author-email: samuel.cahyawijaya@gmail.com
License: UNKNOWN
Project-URL: Bug Tracker, https://github.com/SamuelCahyawijaya/greenformer/issues
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.3
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: matrix-fact (==1.1.2)
Requires-Dist: transformers (==4.8.2)
Requires-Dist: torch (>=1.5.0)
Requires-Dist: scipy
Requires-Dist: cvxopt (==1.2.6)
Requires-Dist: matplotlib (==3.4.2)
Requires-Dist: seaborn (==0.11.1)
Requires-Dist: sklearn

# Greenformer
### Automatic factorization library for pytorch

<img src="img/pytorch-logo-dark.png" width="10%"> [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) 

If you use any source codes included in this toolkit in your work, please cite the following paper.
- Winata, G. I., Cahyawijaya, S., Lin, Z., Liu, Z., & Fung, P. (2020, May). Lightweight and efficient end-to-end speech recognition using low-rank transformer. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6144-6148). IEEE.

### What is Greenformer
Greenformer is a library to convert `Linear`, `Conv1d`, `Conv2d`, `Conv3d` layers into its own variant which called `LED`.
Greenformer seeks over your PyTorch module, replace all `Linear` layers into `LED` layers and all `Conv1d`, `Conv2d`, `Conv3d` layers into `CED` layers with the specified rank.

### How to Install
```
pip install greenformer
```

### Usage
##### BERT Model
```
from transformers import BertModel, BertConfig
from greenformer import auto_fact

config = BertConfig.from_pretrained('bert-base-uncased', pretrained=False)
model = BertModel(config=config)

model = auto_fact(model, rank=100, deepcopy=False, ignore_lower_equal_dim=True, fact_led_unit=False)
```

##### VGG Model
```
import torch
from torchvision import models
from greenformer import auto_fact

model = models.vgg16()
model = auto_fact(model, rank=64, deepcopy=False, ignore_lower_equal_dim=True, fact_led_unit=False)
```

### Why Use GreenFormer
- Improve the speed of you model significantly, check our [Example Notebook](https://github.com/SamuelCahyawijaya/py_auto_fact/blob/main/examples/factorize_bert.ipynb)
- Mantain model performance with appropriate choice of rank, check our [ICASSP 2020 Paper](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9053878)
- Easy to use and works on any kind of model!


