Metadata-Version: 2.1
Name: polars
Version: 0.16.2
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering
Requires-Dist: typing_extensions >= 4.0.0; python_version < '3.10'
Requires-Dist: deltalake >= 0.6.2; extra == 'deltalake'
Requires-Dist: connectorx; extra == 'connectorx'
Requires-Dist: pyarrow>=4.0.0; extra == 'pyarrow'
Requires-Dist: xlsx2csv >= 0.8.0; extra == 'xlsx2csv'
Requires-Dist: backports.zoneinfo; (python_version < '3.9') and extra == 'timezone'
Requires-Dist: tzdata; (platform_system == 'Windows') and extra == 'timezone'
Requires-Dist: matplotlib; extra == 'matplotlib'
Requires-Dist: polars[pyarrow,pandas,numpy,fsspec,connectorx,xlsx2csv,deltalake,timezone,matplotlib]; extra == 'all'
Requires-Dist: pyarrow>=4.0.0; extra == 'pandas'
Requires-Dist: pandas; extra == 'pandas'
Requires-Dist: numpy >= 1.16.0; extra == 'numpy'
Requires-Dist: fsspec; extra == 'fsspec'
Provides-Extra: deltalake
Provides-Extra: connectorx
Provides-Extra: pyarrow
Provides-Extra: xlsx2csv
Provides-Extra: timezone
Provides-Extra: matplotlib
Provides-Extra: all
Provides-Extra: pandas
Provides-Extra: numpy
Provides-Extra: fsspec
License-File: LICENSE
Summary: Blazingly fast DataFrame library
Keywords: dataframe,arrow,out-of-core
Author-email: Ritchie Vink <ritchie46@gmail.com>
Requires-Python: >=3.7
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://www.pola.rs/
Project-URL: Repository, https://github.com/pola-rs/polars
Project-URL: Documentation, https://pola-rs.github.io/polars/py-polars/html/reference/
Project-URL: Changelog, https://github.com/pola-rs/polars/releases

<h1 align="center">
  <img src="https://raw.githubusercontent.com/pola-rs/polars-static/master/logos/polars_github_logo_rect_dark_name.svg">
  <br>
</h1>

<div align="center">
  <a href="https://docs.rs/polars/latest/polars/">
    <img src="https://docs.rs/polars/badge.svg" alt="rust docs"/>
  </a>
  <a href="https://github.com/pola-rs/polars/actions">
    <img src="https://github.com/pola-rs/polars/workflows/Build%20and%20test/badge.svg" alt="Build and test"/>
  </a>
  <a href="https://crates.io/crates/polars">
    <img src="https://img.shields.io/crates/v/polars.svg"/>
  </a>
  <a href="https://pypi.org/project/polars/">
    <img src="https://img.shields.io/pypi/v/polars.svg" alt="PyPi Latest Release"/>
  </a>
  <a href="https://www.npmjs.com/package/nodejs-polars">
    <img src="https://img.shields.io/npm/v/nodejs-polars.svg" alt="NPM Latest Release"/>
  </a>
</div>

<p align="center">
  <b>Documentation</b>:
  <a href="https://pola-rs.github.io/polars/py-polars/html/reference/index.html">Python</a>
  -
  <a href="https://pola-rs.github.io/polars/polars/index.html">Rust</a>
  -
  <a href="https://pola-rs.github.io/nodejs-polars/index.html">Node.js</a>
  |
  <b>StackOverflow</b>:
  <a href="https://stackoverflow.com/questions/tagged/python-polars">Python</a>
  -
  <a href="https://stackoverflow.com/questions/tagged/rust-polars">Rust</a>
  -
  <a href="https://stackoverflow.com/questions/tagged/nodejs-polars">Node.js</a>
  |
  <a href="https://pola-rs.github.io/polars-book/">User Guide</a>
  |
  <a href="https://discord.gg/4UfP5cfBE7">Discord</a>
</p>

## Polars: Blazingly fast DataFrames in Rust, Python & Node.js

Polars is a blazingly fast DataFrames library implemented in Rust using
[Apache Arrow Columnar Format](https://arrow.apache.org/docs/format/Columnar.html) as the memory model.

- Lazy | eager execution
- Multi-threaded
- SIMD
- Query optimization
- Powerful expression API
- Hybrid Streaming (larger than RAM datasets)
- Rust | Python | NodeJS | ...

To learn more, read the [User Guide](https://pola-rs.github.io/polars-book/).

```python
>>> import polars as pl
>>> df = pl.DataFrame(
...     {
...         "A": [1, 2, 3, 4, 5],
...         "fruits": ["banana", "banana", "apple", "apple", "banana"],
...         "B": [5, 4, 3, 2, 1],
...         "cars": ["beetle", "audi", "beetle", "beetle", "beetle"],
...     }
... )

# embarrassingly parallel execution & very expressive query language
>>> df.sort("fruits").select(
...     [
...         "fruits",
...         "cars",
...         pl.lit("fruits").alias("literal_string_fruits"),
...         pl.col("B").filter(pl.col("cars") == "beetle").sum(),
...         pl.col("A").filter(pl.col("B") > 2).sum().over("cars").alias("sum_A_by_cars"),
...         pl.col("A").sum().over("fruits").alias("sum_A_by_fruits"),
...         pl.col("A").reverse().over("fruits").alias("rev_A_by_fruits"),
...         pl.col("A").sort_by("B").over("fruits").alias("sort_A_by_B_by_fruits"),
...     ]
... )
shape: (5, 8)
┌──────────┬──────────┬──────────────┬─────┬─────────────┬─────────────┬─────────────┬─────────────┐
│ fruits   ┆ cars     ┆ literal_stri ┆ B   ┆ sum_A_by_ca ┆ sum_A_by_fr ┆ rev_A_by_fr ┆ sort_A_by_B │
│ ---      ┆ ---      ┆ ng_fruits    ┆ --- ┆ rs          ┆ uits        ┆ uits        ┆ _by_fruits  │
│ str      ┆ str      ┆ ---          ┆ i64 ┆ ---         ┆ ---         ┆ ---         ┆ ---         │
│          ┆          ┆ str          ┆     ┆ i64         ┆ i64         ┆ i64         ┆ i64         │
╞══════════╪══════════╪══════════════╪═════╪═════════════╪═════════════╪═════════════╪═════════════╡
│ "apple"  ┆ "beetle" ┆ "fruits"     ┆ 11  ┆ 4           ┆ 7           ┆ 4           ┆ 4           │
│ "apple"  ┆ "beetle" ┆ "fruits"     ┆ 11  ┆ 4           ┆ 7           ┆ 3           ┆ 3           │
│ "banana" ┆ "beetle" ┆ "fruits"     ┆ 11  ┆ 4           ┆ 8           ┆ 5           ┆ 5           │
│ "banana" ┆ "audi"   ┆ "fruits"     ┆ 11  ┆ 2           ┆ 8           ┆ 2           ┆ 2           │
│ "banana" ┆ "beetle" ┆ "fruits"     ┆ 11  ┆ 4           ┆ 8           ┆ 1           ┆ 1           │
└──────────┴──────────┴──────────────┴─────┴─────────────┴─────────────┴─────────────┴─────────────┘
```

## Performance 🚀🚀

### Blazingly fast

Polars is very fast. In fact, it is one of the best performing solutions available.
See the results in [h2oai's db-benchmark](https://h2oai.github.io/db-benchmark/).

In the [TPCH benchmarks](https://www.pola.rs/benchmarks.html) polars is orders of magnitudes faster than pandas, dask, modin and vaex
on full queries (including IO).

### Lightweight

Polars is also very lightweight. It comes with zero required dependencies, and this shows in the import times:

- polars: 70ms
- numpy: 104ms
- pandas: 520ms

### Handles larger than RAM data

If you have data that does not fit into memory, polars lazy is able to process your query (or parts of your query) in a
streaming fashion, this drastically reduces memory requirements so you might be able to process your 250GB dataset on your
laptop. Collect with `collect(streaming=True)` to run the query streaming. (This might be a little slower, but
it is still very fast!)

## Setup

### Python

Install the latest polars version with:

```sh
pip install polars
```

We also have a conda package (`conda install polars`), however pip is the preferred way to install Polars.

Install Polars with all optional dependencies.

```sh
pip install 'polars[all]'
pip install 'polars[numpy,pandas,pyarrow]'  # install a subset of all optional dependencies
```

You can also install the dependencies directly.

| Tag        | Description                                                                                                                           |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| all        | Install all optional dependencies (all of the following)                                                                              |
| pandas     | Install with Pandas for converting data to and from Pandas Dataframes/Series                                                          |
| numpy      | Install with numpy for converting data to and from numpy arrays                                                                       |
| pyarrow    | Reading data formats using PyArrow                                                                                                    |
| fsspec     | Support for reading from remote file systems                                                                                          |
| connectorx | Support for reading from SQL databases                                                                                                |
| xlsx2csv   | Support for reading from Excel files                                                                                                  |
| deltalake  | Support for reading from Delta Lake Tables                                                                                            |
| timezone   | Timezone support, only needed if 1. you are on Python < 3.9 and/or 2. you are on Windows, otherwise no dependencies will be installed |

Releases happen quite often (weekly / every few days) at the moment, so updating polars regularly to get the latest bugfixes / features might not be a bad idea.

### Rust

You can take latest release from `crates.io`, or if you want to use the latest features / performance improvements
point to the `master` branch of this repo.

```toml
polars = { git = "https://github.com/pola-rs/polars", rev = "<optional git tag>" }
```

Required Rust version `>=1.58`

## Contributing

Want to contribute? Read our [contribution guideline](https://github.com/pola-rs/polars/blob/master/CONTRIBUTING.md).

## Python: compile polars from source

If you want a bleeding edge release or maximal performance you should compile **polars** from source.

This can be done by going through the following steps in sequence:

1. Install the latest [Rust compiler](https://www.rust-lang.org/tools/install)
2. Install [maturin](https://maturin.rs/): `pip install maturin`
3. Choose any of:
   - Fastest binary, very long compile times:
     ```sh
     $ cd py-polars && maturin develop --release -- -C target-cpu=native
     ```
   - Fast binary, Shorter compile times:
     ```sh
     $ cd py-polars && maturin develop --release -- -C codegen-units=16 -C lto=thin -C target-cpu=native
     ```

Note that the Rust crate implementing the Python bindings is called `py-polars` to distinguish from the wrapped
Rust crate `polars` itself. However, both the Python package and the Python module are named `polars`, so you
can `pip install polars` and `import polars`.

## Use custom Rust function in python?

Extending polars with UDFs compiled in Rust is easy. We expose pyo3 extensions for `DataFrame` and `Series`
data structures. See more in https://github.com/pola-rs/pyo3-polars.

## Going big...

Do you expect more than `2^32` ~4,2 billion rows? Compile polars with the `bigidx` feature flag.

Or for python users install `pip install polars-u64-idx`.

Don't use this unless you hit the row boundary as the default polars is faster and consumes less memory.

## Legacy

Do you want polars to run on an old CPU (e.g. dating from before 2011)? Install `pip polars-lts-cpu`. This polars project is
compiled without [avx](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) target features.

## Acknowledgements

Development of Polars is proudly powered by

[![Xomnia](https://raw.githubusercontent.com/pola-rs/polars-static/master/sponsors/xomnia.png)](https://www.xomnia.com/)

## Sponsors

[<img src="https://raw.githubusercontent.com/pola-rs/polars-static/master/sponsors/xomnia.png" height="40" />](https://www.xomnia.com/) &emsp; [<img src="https://www.jetbrains.com/company/brand/img/jetbrains_logo.png" height="50" />](https://www.jetbrains.com)

