Metadata-Version: 2.1
Name: odoo-addon-fs_storage
Version: 17.0.2.0.2.1
Requires-Python: >=3.10
Requires-Dist: fsspec>=2024.5.0
Requires-Dist: odoo-addon-server_environment>=17.0dev,<17.1dev
Requires-Dist: odoo>=17.0a,<17.1dev
Summary: Implement the concept of Storage with amazon S3, sftp...
Home-page: https://github.com/OCA/storage
License: LGPL-3
Author: ACSONE SA/NV, Odoo Community Association (OCA)
Author-email: support@odoo-community.org
Classifier: Programming Language :: Python
Classifier: Framework :: Odoo
Classifier: Framework :: Odoo :: 17.0
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
Classifier: Development Status :: 4 - Beta

==========================
Filesystem Storage Backend
==========================

.. 
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !! This file is generated by oca-gen-addon-readme !!
   !! changes will be overwritten.                   !!
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   !! source digest: sha256:a630b91d2413452b356bd0d3c33130b762f40a96ff38eeb4b4a36a7f3c51aad8
   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
    :target: https://odoo-community.org/page/development-status
    :alt: Beta
.. |badge2| image:: https://img.shields.io/badge/licence-LGPL--3-blue.png
    :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html
    :alt: License: LGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fstorage-lightgray.png?logo=github
    :target: https://github.com/OCA/storage/tree/17.0/fs_storage
    :alt: OCA/storage
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
    :target: https://translation.odoo-community.org/projects/storage-17-0/storage-17-0-fs_storage
    :alt: Translate me on Weblate
.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png
    :target: https://runboat.odoo-community.org/builds?repo=OCA/storage&target_branch=17.0
    :alt: Try me on Runboat

|badge1| |badge2| |badge3| |badge4| |badge5|

This addon is a technical addon that allows you to define filesystem
like storage for your data. It's used by other addons to store their
data in a transparent way into different kind of storages.

Through the fs.storage record, you get access to an object that
implements the
`fsspec.spec.AbstractFileSystem <https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem>`__
interface and therefore give you an unified interface to access your
data whatever the storage protocol you decide to use.

The list of supported protocols depends on the installed fsspec
implementations. By default, the addon will install the following
protocols:

- LocalFileSystem
- MemoryFileSystem
- ZipFileSystem
- TarFileSystem
- FTPFileSystem
- CachingFileSystem
- WholeFileSystem
- SimplCacheFileSystem
- ReferenceFileSystem
- GenericFileSystem
- DirFileSystem
- DatabricksFileSystem
- GitHubFileSystem
- JupiterFileSystem
- OdooFileSystem

The OdooFileSystem is the one that allows you to store your data into a
directory mounted into your Odoo's storage directory. This is the
default FS Storage when creating a new fs.storage record.

Others protocols are available through the installation of additional
python packages:

- DropboxDriveFileSystem -> pip install fsspec[dropbox]
- HTTPFileSystem -> pip install fsspec[http]
- HTTPSFileSystem -> pip install fsspec[http]
- GCSFileSystem -> pip install fsspec[gcs]
- GSFileSystem -> pip install fsspec[gs]
- GoogleDriveFileSystem -> pip install gdrivefs
- SFTPFileSystem -> pip install fsspec[sftp]
- HaddoopFileSystem -> pip install fsspec[hdfs]
- S3FileSystem -> pip install fsspec[s3]
- WandbFS -> pip install wandbfs
- OCIFileSystem -> pip install fsspec[oci]
- AsyncLocalFileSystem -> pip install 'morefs[asynclocalfs]
- AzureDatalakeFileSystem -> pip install fsspec[adl]
- AzureBlobFileSystem -> pip install fsspec[abfs]
- DaskWorkerFileSystem -> pip install fsspec[dask]
- GitFileSystem -> pip install fsspec[git]
- SMBFileSystem -> pip install fsspec[smb]
- LibArchiveFileSystem -> pip install fsspec[libarchive]
- OSSFileSystem -> pip install ossfs
- WebdavFileSystem -> pip install webdav4
- DVCFileSystem -> pip install dvc
- XRootDFileSystem -> pip install fsspec-xrootd

This list of supported protocols is not exhaustive or could change in
the future depending on the fsspec releases. You can find more
information about the supported protocols on the `fsspec
documentation <https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem>`__.

**Table of contents**

.. contents::
   :local:

Usage
=====

Configuration
-------------

When you create a new backend, you must specify the following:

- The name of the backend. This is the name that will be used to
  identify the backend into Odoo

- The code of the backend. This code will identify the backend into the
  store_fname field of the ir.attachment model. This code must be
  unique. It will be used as scheme. example of the store_fname field:
  ``odoofs://abs34Tg11``.

- The protocol used by the backend. The protocol refers to the supported
  protocols of the fsspec python package.

- A directory path. This is a root directory from which the filesystem
  will be mounted. This directory must exist.

- The protocol options. These are the options that will be passed to the
  fsspec python package when creating the filesystem. These options
  depend on the protocol used and are described in the fsspec
  documentation.

- Resolve env vars. This options resolves the protocol options values
  starting with $ from environment variables

- Check Connection Method. If set, Odoo will always check the connection
  before using a storage and it will remove the fs connection from the
  cache if the check fails.

  - ``Create Marker file``: create a hidden file on remote and then
    check it exists with Use it if you have write access to the remote
    and if it is not an issue to leave the marker file in the root
    directory.
  - ``List file``: list all files from the root directory. You can use
    it if the directory path does not contain a big list of files (for
    performance reasons)

Some protocols defined in the fsspec package are wrappers around other
protocols. For example, the SimpleCacheFileSystem protocol is a wrapper
around any local filesystem protocol. In such cases, you must specify
into the protocol options the protocol to be wrapped and the options to
be passed to the wrapped protocol.

For example, if you want to create a backend that uses the
SimpleCacheFileSystem protocol, after selecting the
SimpleCacheFileSystem protocol, you must specify the protocol options as
follows:

.. code:: python

   {
       "directory_path": "/tmp/my_backend",
       "target_protocol": "odoofs",
       "target_options": {...},
   }

In this example, the SimpleCacheFileSystem protocol will be used as a
wrapper around the odoofs protocol.

Server Environment
------------------

To ease the management of the filesystem storages configuration accross
the different environments, the configuration of the filesystem storages
can be defined in environment files or directly in the main
configuration file. For example, the configuration of a filesystem
storage with the code fsprod can be provided in the main configuration
file as follows:

.. code:: ini

   [fs_storage.fsprod]
   protocol=s3
   options={"endpoint_url": "https://my_s3_server/", "key": "KEY", "secret": "SECRET"}
   directory_path=my_bucket

To work, a storage.backend record must exist with the code fsprod into
the database. In your configuration section, you can specify the value
for the following fields:

- protocol
- options
- directory_path

Migration from storage_backend
------------------------------

The fs_storage addon can be used to replace the storage_backend addon.
(It has been designed to be a drop-in replacement for the
storage_backend addon). To ease the migration, the fs.storage model
defines the high-level methods available in the storage_backend model.
These methods are:

- add
- get
- list_files
- find_files
- move_files
- delete

These methods are wrappers around the methods of the
fsspec.AbstractFileSystem class (see
https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.spec.AbstractFileSystem).
These methods are marked as deprecated and will be removed in a future
version (V18) of the addon. You should use the methods of the
fsspec.AbstractFileSystem class instead since they are more flexible and
powerful. You can access the instance of the fsspec.AbstractFileSystem
class using the fs property of a fs.storage record.

Known issues / Roadmap
======================

- Transactions: fsspec comes with a transactional mechanism that once
  started, gathers all the files created during the transaction, and if
  the transaction is committed, moves them to their final locations. It
  would be useful to bridge this with the transactional mechanism of
  odoo. This would allow to ensure that all the files created during a
  transaction are either all moved to their final locations, or all
  deleted if the transaction is rolled back. This mechanism is only
  valid for files created during the transaction by a call to the open
  method of the file system. It is not valid for others operations, such
  as rm, mv_file, ... .

Changelog
=========

17.0.2.0.0 (2024-10-07)
-----------------------

Features
~~~~~~~~

- Invalidate FS filesystem object cache when the connection fails,
  forcing a reconnection.
  (`#320 <https://github.com/OCA/storage/issues/320>`__)

16.0.1.1.0 (2023-12-22)
-----------------------

**Features**

- Add parameter on storage backend to resolve protocol options values
  starting with $ from environment variables
  (`#303 <https://github.com/OCA/storage/issues/303>`__)

16.0.1.0.3 (2023-10-17)
-----------------------

**Bugfixes**

- Fix access to technical models to be able to upload attachments for
  users with basic access
  (`#289 <https://github.com/OCA/storage/issues/289>`__)

16.0.1.0.2 (2023-10-09)
-----------------------

**Bugfixes**

- Avoid config error when using the webdav protocol. The auth option is
  expected to be a tuple not a list. Since our config is loaded from a
  json file, we cannot use tuples. The fix converts the list to a tuple
  when the config is related to a webdav protocol and the auth option is
  into the confix.
  (`#285 <https://github.com/OCA/storage/issues/285>`__)

Bug Tracker
===========

Bugs are tracked on `GitHub Issues <https://github.com/OCA/storage/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us to smash it by providing a detailed and welcomed
`feedback <https://github.com/OCA/storage/issues/new?body=module:%20fs_storage%0Aversion:%2017.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.

Do not contact contributors directly about support or help with technical issues.

Credits
=======

Authors
-------

* ACSONE SA/NV

Contributors
------------

- Laurent Mignon <laurent.mignon@acsone.eu>
- Sébastien BEAU <sebastien.beau@akretion.com>

Maintainers
-----------

This module is maintained by the OCA.

.. image:: https://odoo-community.org/logo.png
   :alt: Odoo Community Association
   :target: https://odoo-community.org

OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.

This module is part of the `OCA/storage <https://github.com/OCA/storage/tree/17.0/fs_storage>`_ project on GitHub.

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.
