Metadata-Version: 2.1
Name: etk
Version: 2.0.8
Summary: extraction toolkit
Home-page: https://github.com/usc-isi-i2/etk
Author: Amandeep Singh
Author-email: amandeep.s.saggu@gmail.com
License: UNKNOWN
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
Requires-Dist: chardet (==3.0.4)
Requires-Dist: cssselect (==1.0.3)
Requires-Dist: cymem (==1.31.2)
Requires-Dist: cytoolz (==0.8.2)
Requires-Dist: decorator (==4.2.1)
Requires-Dist: dill (==0.2.7.1)
Requires-Dist: extruct (==0.4.0)
Requires-Dist: ftfy (==4.4.3)
Requires-Dist: html5lib (==1.0b8)
Requires-Dist: idna (==2.6)
Requires-Dist: isodate (==0.6.0)
Requires-Dist: landmark-extractor (==0.1.8.0)
Requires-Dist: langdetect (==1.0.7)
Requires-Dist: langid (==1.1.6)
Requires-Dist: lxml (==4.1.1)
Requires-Dist: msgpack-numpy (==0.4.1)
Requires-Dist: msgpack-python (==0.5.4)
Requires-Dist: murmurhash (==0.28.0)
Requires-Dist: numpy (==1.14.1)
Requires-Dist: pandas (==0.22.0)
Requires-Dist: pathlib (==1.0.1)
Requires-Dist: plac (==0.9.6)
Requires-Dist: ply (==3.11)
Requires-Dist: preshed (==1.0.0)
Requires-Dist: pygtrie (==2.2)
Requires-Dist: pyparsing (==2.2.0)
Requires-Dist: python-dateutil (==2.7.2)
Requires-Dist: pytz (==2018.3)
Requires-Dist: rdflib (==4.2.2)
Requires-Dist: rdflib-jsonld (==0.4.0)
Requires-Dist: requests (==2.18.4)
Requires-Dist: six (==1.11.0)
Requires-Dist: spacy (==2.0.11)
Requires-Dist: termcolor (==1.1.0)
Requires-Dist: thinc (==6.10.2)
Requires-Dist: toolz (==0.9.0)
Requires-Dist: tqdm (==4.19.5)
Requires-Dist: tzlocal (==1.5.1)
Requires-Dist: ujson (==1.35)
Requires-Dist: urllib3 (==1.22)
Requires-Dist: wcwidth (==0.1.7)
Requires-Dist: wrapt (==1.10.11)
Requires-Dist: jsonpath-ng (==1.4.3)
Requires-Dist: dateparser (==0.7.0)
Requires-Dist: jdcal (==1.3)
Requires-Dist: lml (==0.0.1)
Requires-Dist: demjson (==2.2.4)
Requires-Dist: beautifulsoup4 (==4.6.0)
Requires-Dist: pyexcel (==0.5.7)
Requires-Dist: pyexcel-io (==0.5.6)
Requires-Dist: pyexcel-xls (==0.5.6)
Requires-Dist: pyexcel-xlsx (==0.5.6)
Requires-Dist: SPARQLWrapper (==1.8.2)

# ETK: Information Extraction Toolkit

ETK is a Python library for high precision information extraction from many document formats.
It proivdes a flexible framework of **composable extractors** that enables you to combine a host of **predefined extractors** provided in ETK with custom extractors that you may need to develop for your application.
It supports extraction from HTML pages, text documents, CSV and Excel files and JSON documents.
ETK is open-source software, released under the MIT license.



![MIT License](https://img.shields.io/badge/license-MIT-blue.svg) ![travis ci](https://travis-ci.org/usc-isi-i2/etk.svg?branch=etk2)

## Documentation


## Features

* Extraction from HTML, text, CSV, Excel, JSON
* High-precision predefined extractors for common entities (dates, phones, email, cities, ...)
* Extraction of microdata, schema.org and RDFa markup
* Integration with [spaCy](https://github.com/explosion/spaCy) for text processing
* Automatic identification and extraction of HTML tables containing data
* Automatic identification and extraction of time series
* Semi-automatic generation of Web wrappers
* Scalable execution and management of extraction pipelines
* Automatic provenance recording

# Releases

- [Source code](https://github.com/usc-isi-i2/etk/releases)
- [Docker images](https://hub.docker.com/r/uscisii2/etk/tags/)

## Installation

<table>
  <tr><td><b>Operating system:</td><td>macOS / OS X, Linux, Windows</td></tr>
  <tr><td><b>Python version:</td><td>Python 3.6+</td></tr>
<table>

1. Clone or fork this repository, open a terminal window and in the directory where you downloaded ETK type the following commands:
```
python3 -m venv etk2_env
source etk2_env/bin/activate
pip install -r requirements.txt
```

2. Install using pip
```
pip install etk
```

Load the spacy modules
```
python -m spacy download en_core_web_sm
python -m spacy download en_core_web_lg
```
To deactivate this virtual environment
```
deactivate
```

## Run Tests

`python -m unittest discover`

## Docker

Build image

`docker build -t etk:test .`

Run container

`docker run -it etk:dev /bin/bash`

Mount local volume for test

`docker run -it -v $(pwd):/app/etk etk:dev /bin/bash`



