Metadata-Version: 2.4
Name: semanticlens
Version: 0.1.0
Summary: A package for mechanistic understanding and validation of large  AI model with SemanticLens
Author-email: Jim Berend <jim.berend@hhi.fraunhofer.de>, Maximilian Dreyer <maximilian.dreyer@hhi.fraunhofer.de>
License-Expression: BSD-3-Clause
Keywords: deep learning,foundation model,mechanistic interpretability,semantic analysis
Requires-Python: >=3.11
Requires-Dist: einops>=0.8.0
Requires-Dist: open-clip-torch>=2.30.0
Requires-Dist: scikit-learn>=1.6.1
Requires-Dist: timm>=1.0.13
Requires-Dist: torch>=2.5.1
Requires-Dist: transformers>=4.48.0
Requires-Dist: zennit-crp>=0.6.0
Description-Content-Type: text/markdown

<div align="center">
  <img src="static/images/logo-with-name_big.svg" width="350"/>
  <p>An open-source pytorch implementation of SemanticLens.
  </p>
</div>

[![arXiv](https://img.shields.io/badge/arXiv-2501.05398-b31b1b.svg)](https://arxiv.org/abs/2501.05398)



<!-- ![Tests status](https://img.shields.io/badge/...) -->

<!-- ## Overview

...

<div align="center">
  <img src="./static/images/overview.svg" width="1000"/>å
  <p>
  Tools:
  <a href="./test.txt">Search</a>,
  <a href="./test.txt">Describe</a>,
  <a href="./test.txt">Compare</a>,
  <a href="./test.txt">Audit</a>,
  <a href="./test.txt">Assess Interpretability</a>
  </p>
</div>



Examples -->

## Installation
```bash
pip install semanticlens
```

## Quickstart

```python
import semanticlens as sl

# step 1 - preprocess component-visualizations

act_cv = sl.component_visualization.ActivationComponentVisualizer(
    model,
    dataset,
    layer_names,
)
act_cv.run(batch_size=128, num_workers=16)

# step 2 - compute semantic embeddings

fm = sl.foundation_models.OpenClip("hf-hub:apple/MobileCLIP-S2-OpenCLIP")
lens = sl.Lens(
    dataset=dataset,
    component_visualizer=act_cv,
    foundation_model=fm,
)
lens.compute_semantic_embeddigs(layer_names)

# step 3 - inspect, search, label, evaluate, ...

lens.label(
  ["wall", "small", "sky", "floor", "red",...],
  templates=["a natural image showing {}"],
) # ("layer3", 95): "brown vegetation" ,...

lens.search("watermark") # ("layer4", [63, 21, 362]), ...

lens.evaluate_clarity() # {"layer4" : [[0.64, 0.32, ... ]]}

```

## Project status

> **Note**  
> The project is currently under active development.  🛠️  
> Please expect interfaces to change.
 
The state of development will be updated here.



## Contributing

We adhere to the [PEP8](https://www.python.org/dev/peps/pep-0008) standard with a maximum line width of 120 characters.  
For linting and style checks, we use `ruff`, configured to enforce PEP8 compliance along with additional rules.  
Our basic tests are implemented using `pytest`.


## License

BSD 3-Clause Clear License


### Citation
```
@article{dreyer2025mechanistic,
  title={Mechanistic understanding and validation of large AI models with SemanticLens},
  author={Dreyer, Maximilian and Berend, Jim and Labarta, Tobias and Vielhaben, Johanna and Wiegand, Thomas and Lapuschkin, Sebastian and Samek, Wojciech},
  journal={arXiv preprint arXiv:2501.05398},
  year={2025}
}
```
