Metadata-Version: 2.1
Name: purpleair-data-logger
Version: 1.2.0a2
Home-page: https://github.com/carlkid1499/purpleair_data_logger
Author: Carlos Santos
Author-email: 27721404+carlkid1499@users.noreply.github.com
License: MIT
Keywords: purpleair_data_logger,purple air,purple air data logger,PurpleAirPSQLDataLogger,PurpleAirCSVDataLogger,purple air api,PurpleAirSQLiteDataLogger
Platform: Windows 32/64
Platform: Linux 32/64
Platform: MacOS 32/64
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

# purple_air_data_logger

A set of data logger(s) that will query purple air sensor(s) for data. That data will then be ingested into a TimeScaleDB PostGreSQL database, CSV files, or a SQLite3 database. To use these tools a PurpleAPI key is required. You can get API keys by sending an email to `contact@purpleair.com` with a first and last name to assign them to.

| [![Behave Tests](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/behave_tests.yml/badge.svg?branch=main)](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/behave_tests.yml) | [![PyPI Distributions](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/build_and_publish_to_pypi.yml/badge.svg?branch=main)](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/build_and_publish_to_pypi.yml) | [![TestPyPI Distributions](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/build_and_publish_to_test_pypi.yml/badge.svg?branch=main)](https://github.com/carlkid1499/purpleair_data_logger/actions/workflows/build_and_publish_to_test_pypi.yml) |
| --------------- | --------------- | --------------- |

## How to Support This Project

<a href="https://www.buymeacoffee.com/carlkid1499" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="41" width="174"></a>

## Installation

You can install the PurpleAir Data Logger via pip.

```bash
python3 -m pip install purpleair_data_logger
```

You can install PurpleAir Data Logger by cloning down this repo.

```bash
git clone https://github.com/carlkid1499/purpleair_data_logger.git
cd purpleair_data_logger
python3 setup.py install 
```

## Usage PurpleAirPSQLDataLogger.py

```bash
usage: PurpleAirPSQLDataLogger.py [-h] [-db_drop_all_tables] -db_usr DB_USR [-db_host DB_HOST] -db DB
                                  [-db_port DB_PORT] [-db_pwd DB_PWD] -paa_read_key PAA_READ_KEY
                                  [-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
                                  [-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]

Collect data from PurpleAir sensors and insert into a database!

optional arguments:
  -h, --help            show this help message and exit
  -db_drop_all_tables   Set this flag if you wish to drop all tables before loading in new data. Useful if a database
                        change has happened. Note: Make sure to provide a db_usr with DROP rights. WARNING: ALL
                        COLLECTED DATA WILL BE LOST!
  -db_usr DB_USR        The PSQL database user
  -db_host DB_HOST      The PSQL database host
  -db DB                The PSQL database name
  -db_port DB_PORT      The PSQL database port number
  -db_pwd DB_PWD        The PSQL database password
  -paa_read_key PAA_READ_KEY
                        The PurpleAirAPI Read key
  -paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a single sensor request.
  -paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a multiple sensor request.
```

### High Level Design

![PAA_Data_Logger_Software_Stack.drawio.png](/diagrams/PAA_Data_Logger_Software_Stack.drawio.png)

### Getting Started

1. Grab and install Postgresql for your platform. <https://www.postgresql.org/download/>
2. Create two database users. One for Grafana with select only privileges. The other for the data logger with only insert/create privileges. <https://medium.com/coding-blocks/creating-user-database-and-adding-access-on-postgresql-8bfcd2f4a91e>, <https://www.techonthenet.com/postgresql/grant_revoke.php>
3. Install and configure TimescaleDB. <https://docs.timescale.com/install/latest/self-hosted/>
4. Install and configure Grafana. <https://grafana.com/docs/grafana/latest/setup-grafana/installation/>
5. Import into your local Grafana instance the dashboard file found [here](./grafana_dashboards/PurpleAirAPI%20(PAA)%20Data%20Logger%20Grafana%20Dashboard-1660355898051.json)

## Usage PurpleAirCSVDataLogger.py

```bash
usage: PurpleAirCSVDataLogger.py [-h] -save_file_path SAVE_FILE_PATH -paa_read_key PAA_READ_KEY
                                 [-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
                                 [-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]

Collect data from PurpleAir sensors and store it in CSV files!

optional arguments:
  -h, --help            show this help message and exit
  -save_file_path SAVE_FILE_PATH
                        The path to save CSV files in.
  -paa_read_key PAA_READ_KEY
                        The PurpleAirAPI Read key
  -paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a single sensor request.
  -paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a multiple sensor request.
```

## Usage PurpleAirSQLiteDataLogger.py

```bash
usage: PurpleAirSQLiteDataLogger.py [-h] -db_name DB_NAME -paa_read_key PAA_READ_KEY
                                    [-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
                                    [-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]

Collect data from PurpleAir sensors and store it a SQLite3 database file!

optional arguments:
  -h, --help            show this help message and exit
  -db_name DB_NAME      The path and name for the SQLite3 database file! i.e database_name.db
  -paa_read_key PAA_READ_KEY
                        The PurpleAirAPI Read key
  -paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a single sensor request.
  -paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
                        The path to a json file containing the parameters to send a multiple sensor request.
```

## Sample JSON Configuration File(s)

The following sample json configuration files can be used with any of the data loggers.

### PAA_SINGLE_SENSOR_REQUEST_JSON_FILE Example

Out of the parameters listed below only "sensor_index" is required. The others are all optional according to PurpleAirAPI (PAA) documentation. If a field is not being used, mark it 'null' without the single quotes.

```json
{
    "sensor_index": 53,
    "read_key": null,
    "fields": null
}
```

> Note: Refer to the PurpleAirAPI (PAA) documentation for more information. <https://api.purpleair.com/#api-sensors-get-sensor-data>

### PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE Example

Out of the parameters listed below only "fields" is required. The others are all optional according to PurpleAirAPI (PAA) documentation. If a field is not being used, mark it 'null' without the single quotes.

```json
{
    "fields": "A string comma delimited list of fields to retrieve",
    "location_type": null,
    "read_keys": null,
    "show_only": null,
    "modified_since": null,
    "max_age": null,
    "nwlng" : null,
    "nwlat": null,
    "selng": null,
    "selat": null
}
```

> Note: Refer to the PurpleAirAPI (PAA) documentation for more information. <https://api.purpleair.com/#api-sensors-get-sensors-data>

> Taken From the PurpleAirAPI documentation:

    Field Type Description
    fields String 
    The 'Fields' parameter specifies which 'sensor data fields' to include in the response. It is a comma separated list with one or more of the following:

    Station information and status fields:
    name, icon, model, hardware, location_type, private, latitude, longitude, altitude, position_rating, led_brightness, firmware_version, firmware_upgrade, rssi, uptime, pa_latency, memory, last_seen, last_modified, date_created, channel_state, channel_flags, channel_flags_manual, channel_flags_auto, confidence, confidence_manual, confidence_auto

    Environmental fields:
    humidity, humidity_a, humidity_b, temperature, temperature_a, temperature_b, pressure, pressure_a, pressure_b

    Miscellaneous fields:
    voc, voc_a, voc_b, ozone1, analog_input

    PM1.0 fields:
    pm1.0, pm1.0_a, pm1.0_b, pm1.0_atm, pm1.0_atm_a, pm1.0_atm_b, pm1.0_cf_1, pm1.0_cf_1_a, pm1.0_cf_1_b

    PM2.5 fields:
    pm2.5_alt, pm2.5_alt_a, pm2.5_alt_b, pm2.5, pm2.5_a, pm2.5_b, pm2.5_atm, pm2.5_atm_a, pm2.5_atm_b, pm2.5_cf_1, pm2.5_cf_1_a, pm2.5_cf_1_b

    PM2.5 pseudo (simple running) average fields:
    pm2.5_10minute, pm2.5_10minute_a, pm2.5_10minute_b, pm2.5_30minute, pm2.5_30minute_a, pm2.5_30minute_b, pm2.5_60minute, pm2.5_60minute_a, pm2.5_60minute_b, pm2.5_6hour, pm2.5_6hour_a, pm2.5_6hour_b, pm2.5_24hour, pm2.5_24hour_a, pm2.5_24hour_b, pm2.5_1week, pm2.5_1week_a, pm2.5_1week_b

    PM10.0 fields:
    pm10.0, pm10.0_a, pm10.0_b, pm10.0_atm, pm10.0_atm_a, pm10.0_atm_b, pm10.0_cf_1, pm10.0_cf_1_a, pm10.0_cf_1_b

    Visibility fields:
    scattering_coefficient, scattering_coefficient_a, scattering_coefficient_b, deciviews, deciviews_a, deciviews_b, visual_range, visual_range_a, visual_range_b

    Particle count fields:
    0.3_um_count, 0.3_um_count_a, 0.3_um_count_b, 0.5_um_count, 0.5_um_count_a, 0.5_um_count_b, 1.0_um_count, 1.0_um_count_a, 1.0_um_count_b, 2.5_um_count, 2.5_um_count_a, 2.5_um_count_b, 5.0_um_count, 5.0_um_count_a, 5.0_um_count_b, 10.0_um_count 10.0_um_count_a, 10.0_um_count_b

    ThingSpeak fields, used to retrieve data from api.thingspeak.com:
    primary_id_a, primary_key_a, secondary_id_a, secondary_key_a, primary_id_b, primary_key_b, secondary_id_b, secondary_key_b

    For field descriptions, please see the 'sensor data fields'. section.

    location_type optional Number 
    The location_type of the sensors.
    Possible values are: 0 = Outside or 1 = Inside.

    read_keys optional String 
    A read_key is required for private devices. It is separate to the api_key and each sensor has its own read_key. Submit multiple keys by separating them with a comma (,) character for example: key-one,key-two,key-three.

    show_only optional String 
    A comma (,) separated list of sensor_index values. When provided, the results are limited only to the sensors included in this list.

    modified_since optional long 
    The modified_since parameter causes only sensors modified after the provided time stamp to be included in the results. Using the time_stamp value from a previous call (recommended) will limit results to those with new values since the last request. Using a value of 0 will match sensors modified at any time.

    max_age optional Integer 
    Filter results to only include sensors modified or updated within the last number of seconds. Using a value of 0 will match sensors of any age.

    Default value: 604800

    nwlng optional Number 
    A north west longitude for the bounding box.

    Use a bounding box to limit the sensors returned to a specific geographic area. The bounding box is defined by two points, a north west latitude/longitude and a south east latitude/longitude.

    nwlat optional Number 
    A north west latitude for the bounding box.

    selng optional Number 
    A south east longitude for the bounding box.

    selat optional Number 
    A south east latitude for the bounding box.
