Metadata-Version: 2.1
Name: openlineage-python
Version: 0.26.0
Summary: OpenLineage Python Client
Author-email: OpenLineage <info@openlineage.io>
Keywords: openlineage
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.7
Requires-Dist: attrs>=23.1
Requires-Dist: python-dateutil>=2.8.2
Requires-Dist: pyyaml>=6
Requires-Dist: requests>=2.30
Provides-Extra: kafka
Requires-Dist: confluent-kafka>=2.1.1; extra == 'kafka'
Provides-Extra: test
Requires-Dist: covdefaults>=2.3; extra == 'test'
Requires-Dist: pytest-cov>=4; extra == 'test'
Requires-Dist: pytest>=7.3.1; extra == 'test'
Requires-Dist: pyyaml>=6; extra == 'test'
Description-Content-Type: text/markdown

# OpenLineage-python

To install from source, run:

```bash
$ python -m pip install .
```

## Configuration
### Config file

The most common way to configure the OpenLineage Client is by `.yaml` file, which contains all the
details of how to connect to your OpenLineage backend.

The config file is located by:
1) looking at the `OPENLINEAGE_CONFIG` environment variable
2) looking for the `openlineage.yml` file in the current working directory
3) looking for the `openlineage.yml` file in the `$HOME/.openlineage` directory.

Different ways of connecting to OpenLineage backend are supported
by the standarized `Transport` interface.
This is an example config for specifying `http` transport:

```yaml
transport:
  type: "http"
  url: "https://backend:5000"
  auth:
    type: "api_key"
    api_key: "f048521b-dfe8-47cd-9c65-0cb07d57591e"
```

The `type` property is required. It can be one of the built-in transports or a custom one.
There are three built-in transports, `http`, `kafka` and `console`.
Custom transports `type` is a fully qualified class name that can be imported.

The rest of the properties are defined by the particular transport.
Specifications for the built-in options are below.

#### HTTP

* `url` - required string parameter.
* `endpoint` - optional string parameter specifying the endpoint to which events are sent. By default `api/v1/lineage`.
* `timeout` - optional float parameter specifying the timeout when sending events. By default 5 seconds.
* `verify` optional boolean attribute specifying if the client should verify TLS certificates of the backend. By default true.
* `auth` - optional dictionary specifying authentication options. The type property is required.
    * `type`: required property if an auth dictionary is set. Set to `api_key` or to the fully qualified class name of your TokenProvider.
    * `api_key`: if `api_key` type is set, it sets value at `Authentication` HTTP header as `Bearer`.

#### Kafka

For KafkaTransport, `confluent-kafka` needs to be installed.
You can also install `pip install openlineage-python[kafka]`

* `config` - required string parameter. A dictionary that contains a Kafka producer config as in [Kafka producer config](https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#kafka-client-configuration).
* `topic` - required string parameter. The topic on what events will be sent.
* `flush` - optional boolean parameter. If set to True, Kafka will flush after each event. By default true.


### Custom transport

To implement a custom transport, follow the instructions in the `openlineage/client/transport/transport.py` file.

### Config as env vars

If there is no config file found, the OpenLineage client looks at environment variables.
This way of configuring the client supports only `http` transport, and only a subset of its config.

* `OPENLINEAGE_URL` - point to the service that will consume OpenLineage events.
* `OPENLINEAGE_API_KEY` - set if the consumer of OpenLineage events requires a `Bearer` authentication key.

`OPENLINEAGE_URL` and `OPENLINEAGE_API_KEY` can also be set up manually when creating a client instance.
