Metadata-Version: 2.1
Name: selenium-crawler-template
Version: 0.2.0
Summary: Boilerplate for developing crawler with Selenium.
Home-page: https://github.com/yeongbin-jo/python-selenium-crawler-template
Author: Yeongbin Jo
Author-email: yeongbin.jo@pylab.co
License: MIT
Keywords: chromedriver chrome selenium splinter
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: System :: Installation/Setup
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3
Description-Content-Type: text/markdown

# Selenium Crawler Template
Boilerplate for developing crawler with Selenium.

## Installation

```bash
pip install selenium-crawler-template
```

## Usage
```python
from selenium_crawler_template import Crawler

class MyCrawler(Crawler):
    @Crawler.open_url_in_new_tab
    def _get_email_from_profile(self, _):
        return self.find_element('a#email').get_attribute('href')

    def crawl(self, **kwargs):
        self.driver.get(kwargs['url'])

        for profile in self.find_elements('ul > .profile'):
            _ = self._get_email_from_profile(profile.get_attribute('href'))

        self._scroll_to_bottom()
```


