Metadata-Version: 2.2
Name: freamon
Version: 0.2.0
Summary: A package to make data science projects on tabular data easier
Author: Freamon Team
License: MIT License
        
        Copyright (c) 2025 Stephen Oates
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: scikit-learn
Requires-Dist: matplotlib
Requires-Dist: seaborn
Requires-Dist: joblib
Requires-Dist: category_encoders
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Provides-Extra: all
Requires-Dist: lightgbm; extra == "all"
Requires-Dist: xgboost; extra == "all"
Requires-Dist: catboost; extra == "all"
Requires-Dist: spacy; extra == "all"
Requires-Dist: polars; extra == "all"
Requires-Dist: dask[dataframe]; extra == "all"
Requires-Dist: shapiq; extra == "all"
Requires-Dist: shap; extra == "all"
Provides-Extra: lightgbm
Requires-Dist: lightgbm; extra == "lightgbm"
Provides-Extra: xgboost
Requires-Dist: xgboost; extra == "xgboost"
Provides-Extra: catboost
Requires-Dist: catboost; extra == "catboost"
Provides-Extra: nlp
Requires-Dist: spacy; extra == "nlp"
Provides-Extra: polars
Requires-Dist: polars; extra == "polars"
Provides-Extra: dask
Requires-Dist: dask[dataframe]; extra == "dask"
Provides-Extra: explainability
Requires-Dist: shapiq; extra == "explainability"
Requires-Dist: shap; extra == "explainability"
Dynamic: requires-python

# freamon

<p align="center">
  <img src="package_logo.webp" alt="Freamon Logo" width="250"/>
</p>

A package to make data science projects on tabular data easier. Named after the great character from The Wire played by Clarke Peters.

## Features

- **Data Quality Assessment:** Missing values, outliers, data types, duplicates
- **Exploratory Data Analysis (EDA):** Statistical analysis and visualizations
- **Feature Engineering:** 
  - **Standard Features:** Polynomial, interaction, datetime, binned features
  - **Automatic Interaction Detection:** ShapIQ-based automatic feature engineering
- **Categorical Encoding:** 
  - **Basic Encoders:** One-hot, ordinal, target encoding
  - **Advanced Encoders:** Binary, hashing, weight of evidence (WOE) encoding
- **Text Processing:** Basic NLP with optional spaCy integration
- **Model Selection:** Train/test splitting with time-series awareness
- **Modeling:** Training, evaluation, and validation
  - **Support for Multiple Libraries:** scikit-learn, LightGBM, XGBoost, CatBoost
- **Explainability:** 
  - **SHAP Support:** Feature importance and explanations
  - **ShapIQ Integration:** Feature interactions detection and visualization
  - **Interactive Reports:** HTML reports for explainability findings
- **Multiple DataFrame Backends:** 
  - **Pandas:** Standard interface
  - **Polars:** High-performance alternative
  - **Dask:** Out-of-core processing for large datasets

## Installation

```bash
# Basic installation
pip install freamon

# With all optional dependencies
pip install freamon[all]

# With specific optional dependencies
pip install freamon[lightgbm]        # For LightGBM support
pip install freamon[xgboost]         # For XGBoost support
pip install freamon[catboost]        # For CatBoost support
pip install freamon[nlp]             # For NLP capabilities with spaCy
pip install freamon[polars]          # For Polars support
pip install freamon[dask]            # For Dask support
pip install freamon[explainability]  # For SHAP and ShapIQ integration

# Development installation
git clone https://github.com/yourusername/freamon.git
cd freamon
pip install -e ".[dev,all]"
```

## Quick Start

```python
import pandas as pd
from freamon.data_quality import DataQualityAnalyzer
from freamon.modeling import ModelTrainer
from freamon.model_selection import train_test_split
from freamon.utils import OneHotEncoderWrapper
from freamon.utils.dataframe_utils import detect_datetime_columns

# Load your data
df = pd.read_csv("your_data.csv")

# Automatically detect and convert datetime columns
df = detect_datetime_columns(df)

# Analyze data quality
analyzer = DataQualityAnalyzer(df)
analyzer.generate_report("data_quality_report.html")

# Handle missing values
from freamon.data_quality import handle_missing_values
df_clean = handle_missing_values(df, strategy="mean")

# Encode categorical features
encoder = OneHotEncoderWrapper()
df_encoded = encoder.fit_transform(df_clean)

# Split data
train_df, test_df = train_test_split(df_encoded, test_size=0.2, random_state=42)

# Train a model
feature_cols = [col for col in train_df.columns if col != "target"]
trainer = ModelTrainer(
    model_type="lightgbm",
    model_name="LGBMClassifier",
    problem_type="classification",
)
metrics = trainer.train(
    train_df[feature_cols],
    train_df["target"],
    X_val=test_df[feature_cols],
    y_val=test_df["target"],
)

# View the results
print(f"Validation metrics: {metrics}")
```

### Using with Polars

```python
import polars as pl
from freamon.utils.dataframe_utils import detect_datetime_columns, convert_dataframe

# Load data with Polars
df = pl.read_csv("your_data.csv")

# Detect and convert datetime columns
df = detect_datetime_columns(df)

# Convert to pandas for operations that require it
pandas_df = convert_dataframe(df, "pandas")

# ... perform operations ...

# Convert back to polars
result = convert_dataframe(pandas_df, "polars")
```

## Module Overview

- **data_quality:** Tools for assessing and improving data quality
- **utils:** Utility functions for working with dataframes and encoders
  - **dataframe_utils:** Tools for different dataframe backends and date detection
  - **encoders:** Categorical variable encoding tools
  - **text_utils:** Text processing utilities
- **model_selection:** Methods for splitting data and cross-validation
- **modeling:** Model training, evaluation, and comparison

Check out the [ROADMAP.md](ROADMAP.md) file for information on planned features and development phases.

## Development

To contribute to freamon, install the development dependencies:

```bash
pip install -e ".[dev]"
```

Run tests:

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=freamon
```

## License

MIT License
