Metadata-Version: 2.3
Name: waf_downloader
Version: 0.1.1
Summary: Web Application Firewall logs downloader.
Project-URL: Bug reports, https://github.com/MihaiBojin/waf-downloader/issues/new
Project-URL: Documentation, https://github.com/MihaiBojin/waf-downloader/blob/main/python/README.md
Project-URL: Source, https://github.com/MihaiBojin/waf-downloader
Author-email: Mihai Bojin <557584+MihaiBojin@users.noreply.github.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: application,cloudflare,download,firewall,logs,waf,web
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.11
Requires-Dist: boto3==1.34.129
Requires-Dist: more-itertools==10.2.0
Requires-Dist: psycopg2-binary==2.9.9
Requires-Dist: requests==2.32.3
Provides-Extra: cli
Requires-Dist: argparse==1.4.0; extra == 'cli'
Requires-Dist: python-dotenv==1.0.1; extra == 'cli'
Provides-Extra: dev
Requires-Dist: build==1.2.1; extra == 'dev'
Requires-Dist: pre-commit==3.7.1; extra == 'dev'
Requires-Dist: pytest==8.2.2; extra == 'dev'
Requires-Dist: toml==0.10.2; extra == 'dev'
Requires-Dist: twine==5.1.0; extra == 'dev'
Requires-Dist: wheel-inspect==1.7.1; extra == 'dev'
Requires-Dist: wheel==0.43.0; extra == 'dev'
Description-Content-Type: text/markdown

# Cloudflare Web Application Firewall downloader

![Build Status](https://github.com/MihaiBojin/waf-downloader/actions/workflows/python-tests.yml/badge.svg)
[![PyPI version](https://badge.fury.io/py/waf-downloader.svg)](https://badge.fury.io/py/waf-downloader)
[![Python Versions](https://img.shields.io/pypi/pyversions/waf-downloader.svg)](https://pypi.org/project/waf-downloader/)
[![License](https://img.shields.io/github/license/waf-downloader/waf-downloader.svg)](LICENSE)

A library, CLI, and docker image that downloads Cloudflare WAF logs for a specified zone_id and time frame.

## Schema

See [src/waf_logs/resources/db/](./src/waf_logs/resources/db) for a list of schemas that are auto-applied at start. This can be disabled by passing `--ensure_schema False`.

## Quickstart

The project is published to <https://pypi.org/project/waf-downloader/>.
Install it via:

```shell
pip install waf-downloader

# or alternatively, directly from git
pip install "git+https://github.com/MihaiBojin/waf-downloader@main"
```

### Build and run with Docker

Define secrets in an `.env` file (do not quote values):

```properties
CLOUDFLARE_TOKEN=...
DB_CONN_STR=...
```

The Cloudflare token is required, but the connection string is optional.
If skipped, it will result in logs being printed to stdout.

Build and run:

```shell
# Build
make docker

# Load all logs in zone, starting 5 minutes prior
make docker-run ARGS="--zone_id [CLOUDFLARE_ZONE_ID] --start_minutes_ago -5"

# And alternatively, only output the logs
make docker-run ARGS="--zone_id [CLOUDFLARE_ZONE_ID] --start_minutes_ago -5" 2>/dev/null
```

## Publishing to PyPI

### GitHub-based version publishing

The simplest way to publish a new version (if you have committer rights) is to tag a commit and push it to the repo:

```shell
# At a certain commit, ideally after merging a PR to main
git tag v0.1.x
git push origin v0.1.x
```

A [GitHub Action](https://github.com/MihaiBojin/waf-downloader/actions) will run, build the library and publish it to the PyPI repositories.

### Manual publish

These steps can also be performed locally. For these commands to work, you will need to export two environment variables (or define them in `.env`):

```shell
export TESTPYPI_PASSWORD=... # token for https://test.pypi.org/legacy/
export PYPI_PASSWORD=... # token for https://upload.pypi.org/legacy/
```

First, publish to the test repo and inspect the package:

```shell
make publish-test
```

If correct, distribute the wheel to the PyPI index:

```shell
make publish
```

Verify the distributed code

```shell
make publish-verify
```

## Cloudflare WAF documentation

- <https://developers.cloudflare.com/analytics/graphql-api/tutorials/querying-firewall-events>
- <https://developers.cloudflare.com/analytics/graphql-api/tutorials/export-graphql-to-csv/>
