Metadata-Version: 1.2
Name: iterfilesystem
Version: 1.1.0
Summary: Threaded directory iteration via os.scandir() with progress indicator and resume function.
Home-page: https://github.com/jedie/iterfilesystem
Author: Jens Diemer
Author-email: python@jensdiemer.de
License: GNU General Public License v3
Description: --------------
        IterFilesystem
        --------------
        
        Multiprocess directory iteration via ``os.scandir()``:
        
        * "stats" processes:
        
            * only counts up all directories and files.
        
            * accumulates the sizes of all files.
        
        * "worker" process:
        
            * Filesystem walk and process the real action with dir/files
        
        among other things these packages are used:
        
        * progress bar `tqdm <https://pypi.org/project/tqdm/>`_
        
        Requirement:
        
        * Python 3.6 or newer.
        
        * `Pipenv <https://pipenv.readthedocs.io/en/latest/install/#installing-pipenv>`_. Packages and virtual environment manager.
        
        Please: try, fork and contribute! ;)
        
        +--------------------------------------+----------------------------------------------------------+
        | |Build Status on travis-ci.org|      | `travis-ci.org/jedie/IterFilesystem`_                    |
        +--------------------------------------+----------------------------------------------------------+
        | |Build Status on appveyor.com|       | `ci.appveyor.com/project/jedie/IterFilesystem`_          |
        +--------------------------------------+----------------------------------------------------------+
        | |Coverage Status on codecov.io|      | `codecov.io/gh/jedie/IterFilesystem`_                    |
        +--------------------------------------+----------------------------------------------------------+
        | |Coverage Status on coveralls.io|    | `coveralls.io/r/jedie/IterFilesystem`_                   |
        +--------------------------------------+----------------------------------------------------------+
        | |Requirements Status on requires.io| | `requires.io/github/jedie/IterFilesystem/requirements/`_ |
        +--------------------------------------+----------------------------------------------------------+
        
        .. |Build Status on travis-ci.org| image:: https://travis-ci.org/jedie/IterFilesystem.svg
        .. _travis-ci.org/jedie/IterFilesystem: https://travis-ci.org/jedie/IterFilesystem/
        .. |Build Status on appveyor.com| image:: https://ci.appveyor.com/api/projects/status/py5sl38ql3xciafc?svg=true
        .. _ci.appveyor.com/project/jedie/IterFilesystem: https://ci.appveyor.com/project/jedie/IterFilesystem/history
        .. |Coverage Status on codecov.io| image:: https://codecov.io/gh/jedie/IterFilesystem/branch/master/graph/badge.svg
        .. _codecov.io/gh/jedie/IterFilesystem: https://codecov.io/gh/jedie/IterFilesystem
        .. |Coverage Status on coveralls.io| image:: https://coveralls.io/repos/jedie/IterFilesystem/badge.svg
        .. _coveralls.io/r/jedie/IterFilesystem: https://coveralls.io/r/jedie/IterFilesystem
        .. |Requirements Status on requires.io| image:: https://requires.io/github/jedie/IterFilesystem/requirements.svg?branch=master
        .. _requires.io/github/jedie/IterFilesystem/requirements/: https://requires.io/github/jedie/IterFilesystem/requirements/
        
        -------
        Example
        -------
        
        Use example CLI, e.g.:
        
        ::
        
            ~$ git clone https://github.com/jedie/IterFilesystem.git
            ~$ cd IterFilesystem
            ~/IterFilesystem$ pipenv install
            ~/IterFilesystem$ pipenv shell
            (IterFilesystem) ~/IterFilesystem$ print_fs_stats --help
            (IterFilesystem) ~/IterFilesystem$ pip install -e .
            ...
            Successfully installed iterfilesystem
            
            (IterFilesystem) ~/IterFilesystem$ $ print_fs_stats --help
            usage: print_fs_stats.py [-h] [-v] [--debug] [--path PATH]
                                     [--skip_dir_patterns [SKIP_DIR_PATTERNS [SKIP_DIR_PATTERNS ...]]]
                                     [--skip_file_patterns [SKIP_FILE_PATTERNS [SKIP_FILE_PATTERNS ...]]]
            
            Scan filesystem and print some information
            
            optional arguments:
              -h, --help            show this help message and exit
              -v, --version         show program's version number and exit
              --debug               enable DEBUG
              --path PATH           The file path that should be scanned e.g.: "~/foobar/"
                                    default is "~"
              --skip_dir_patterns [SKIP_DIR_PATTERNS [SKIP_DIR_PATTERNS ...]]
                                    Directory names to exclude from scan.
              --skip_file_patterns [SKIP_FILE_PATTERNS [SKIP_FILE_PATTERNS ...]]
                                    File names to ignore.
        
        example output looks like this:
        
        ::
        
            (IterFilesystem) ~/IterFilesystem$ $ print_fs_stats --path ~/IterFilesystem --skip_dir_patterns ".*" "*.egg-info" --skip_file_patterns ".*"
            Read/process: '~/IterFilesystem'...
            Skip directory patterns:
            	* .*
            	* *.egg-info
            
            Skip file patterns:
            	* .*
            
            Filesystem items..:Read/process: '~/IterFilesystem'...
            
            ...
            
            Filesystem items..: 100%|█████████████████████████████████████████|135/135 13737.14entries/s [00:00<00:00, 13737.14entries/s]
            File sizes........: 100%|██████████████████████████████████████████████████████████████|843k/843k [00:00<00:00, 88.5MBytes/s]
            Average progress..: 100%|███████████████████████████████████████████████████████████████████████████████████████|00:00<00:00
            Current File......:, /home/jens/repos/IterFilesystem/Pipfile
            
            
            Processed 135 filesystem items in 0.02 sec
            SHA515 hash calculated over all file content: 10f9475b21977f5aea1d4657a0e09ad153a594ab30abc2383bf107dbc60c430928596e368ebefab3e78ede61dcc101cb638a845348fe908786cb8754393439ef
            File count: 109
            Total file size: 843.5 KB
            6 directories skipped.
            6 files skipped.
        
        -------
        History
        -------
        
        * **dev** - `compare v1.1.0...master <https://github.com/jedie/IterFilesystem/compare/v1.1.0...master>`_ 
        
            * TBC
        
        * 12.10.2019 - `compare v1.0.0...v1.1.0 <https://github.com/jedie/IterFilesystem/compare/v1.0.0...v1.1.0>`_ 
        
            * don't create separate process for worker: Just do the work in main process
        
            * dir/file filter uses now ``fnmatch``
        
        * 12.10.2019 - `compare v0.2.0...v1.0.0 <https://github.com/jedie/IterFilesystem/compare/v0.2.0...v1.0.0>`_ 
        
            * refactoring:
        
                * don't use ``persist-queue``
        
                * switch from threading to multiprocessing
        
                * enhance progress display with multiple ``tqdm`` process bars
        
        * 15.09.2019 - `compare v0.1.0...v0.2.0 <https://github.com/jedie/IterFilesystem/compare/v0.1.0...v0.2.0>`_ 
        
            * store persist queue in temp directory
        
            * Don't catch ``process_path_item`` errors, this should be made in child class
        
        * 15.09.2019 - `compare v0.0.1...v0.1.0 <https://github.com/jedie/IterFilesystem/compare/v0.0.1...v0.1.0>`_ 
        
            * add some project meta files and tests
        
            * setup CI
        
            * fix tests
        
        * 15.09.2019 - v0.0.1
        
            * first Release on PyPi
        
        -----
        Links
        -----
        
        * `https://pypi.python.org/pypi/IterFilesystem/ <https://pypi.python.org/pypi/IterFilesystem/>`_
        
        * `https://github.com/jedie/IterFilesystem/ <https://github.com/jedie/IterFilesystem/>`_
        
        --------
        Donating
        --------
        
        * `paypal.me/JensDiemer <https://www.paypal.me/JensDiemer>`_
        
        * `Flattr This! <https://flattr.com/submit/auto?uid=jedie&url=https%3A%2F%2Fgithub.com%2Fjedie%2FIterFilesystem%2F>`_
        
        * Send `Bitcoins <http://www.bitcoin.org/>`_ to `1823RZ5Md1Q2X5aSXRC5LRPcYdveCiVX6F <https://blockexplorer.com/address/1823RZ5Md1Q2X5aSXRC5LRPcYdveCiVX6F>`_
Keywords: iterfilesystem
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Requires-Python: >=3.6
