| Metadata-Version: 2.4 |
| Name: fsspec |
| Version: 2025.12.0 |
| Summary: File-system specification |
| Project-URL: Changelog, https://filesystem-spec.readthedocs.io/en/latest/changelog.html |
| Project-URL: Documentation, https://filesystem-spec.readthedocs.io/en/latest/ |
| Project-URL: Homepage, https://github.com/fsspec/filesystem_spec |
| Maintainer-email: Martin Durant <mdurant@anaconda.com> |
| License-Expression: BSD-3-Clause |
| License-File: LICENSE |
| Keywords: file |
| Classifier: Development Status :: 4 - Beta |
| Classifier: Intended Audience :: Developers |
| Classifier: Operating System :: OS Independent |
| Classifier: Programming Language :: Python :: 3.10 |
| Classifier: Programming Language :: Python :: 3.11 |
| Classifier: Programming Language :: Python :: 3.12 |
| Classifier: Programming Language :: Python :: 3.13 |
| Classifier: Programming Language :: Python :: 3.14 |
| Requires-Python: >=3.10 |
| Provides-Extra: abfs |
| Requires-Dist: adlfs; extra == 'abfs' |
| Provides-Extra: adl |
| Requires-Dist: adlfs; extra == 'adl' |
| Provides-Extra: arrow |
| Requires-Dist: pyarrow>=1; extra == 'arrow' |
| Provides-Extra: dask |
| Requires-Dist: dask; extra == 'dask' |
| Requires-Dist: distributed; extra == 'dask' |
| Provides-Extra: dev |
| Requires-Dist: pre-commit; extra == 'dev' |
| Requires-Dist: ruff>=0.5; extra == 'dev' |
| Provides-Extra: doc |
| Requires-Dist: numpydoc; extra == 'doc' |
| Requires-Dist: sphinx; extra == 'doc' |
| Requires-Dist: sphinx-design; extra == 'doc' |
| Requires-Dist: sphinx-rtd-theme; extra == 'doc' |
| Requires-Dist: yarl; extra == 'doc' |
| Provides-Extra: dropbox |
| Requires-Dist: dropbox; extra == 'dropbox' |
| Requires-Dist: dropboxdrivefs; extra == 'dropbox' |
| Requires-Dist: requests; extra == 'dropbox' |
| Provides-Extra: entrypoints |
| Provides-Extra: full |
| Requires-Dist: adlfs; extra == 'full' |
| Requires-Dist: aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'full' |
| Requires-Dist: dask; extra == 'full' |
| Requires-Dist: distributed; extra == 'full' |
| Requires-Dist: dropbox; extra == 'full' |
| Requires-Dist: dropboxdrivefs; extra == 'full' |
| Requires-Dist: fusepy; extra == 'full' |
| Requires-Dist: gcsfs; extra == 'full' |
| Requires-Dist: libarchive-c; extra == 'full' |
| Requires-Dist: ocifs; extra == 'full' |
| Requires-Dist: panel; extra == 'full' |
| Requires-Dist: paramiko; extra == 'full' |
| Requires-Dist: pyarrow>=1; extra == 'full' |
| Requires-Dist: pygit2; extra == 'full' |
| Requires-Dist: requests; extra == 'full' |
| Requires-Dist: s3fs; extra == 'full' |
| Requires-Dist: smbprotocol; extra == 'full' |
| Requires-Dist: tqdm; extra == 'full' |
| Provides-Extra: fuse |
| Requires-Dist: fusepy; extra == 'fuse' |
| Provides-Extra: gcs |
| Requires-Dist: gcsfs; extra == 'gcs' |
| Provides-Extra: git |
| Requires-Dist: pygit2; extra == 'git' |
| Provides-Extra: github |
| Requires-Dist: requests; extra == 'github' |
| Provides-Extra: gs |
| Requires-Dist: gcsfs; extra == 'gs' |
| Provides-Extra: gui |
| Requires-Dist: panel; extra == 'gui' |
| Provides-Extra: hdfs |
| Requires-Dist: pyarrow>=1; extra == 'hdfs' |
| Provides-Extra: http |
| Requires-Dist: aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'http' |
| Provides-Extra: libarchive |
| Requires-Dist: libarchive-c; extra == 'libarchive' |
| Provides-Extra: oci |
| Requires-Dist: ocifs; extra == 'oci' |
| Provides-Extra: s3 |
| Requires-Dist: s3fs; extra == 's3' |
| Provides-Extra: sftp |
| Requires-Dist: paramiko; extra == 'sftp' |
| Provides-Extra: smb |
| Requires-Dist: smbprotocol; extra == 'smb' |
| Provides-Extra: ssh |
| Requires-Dist: paramiko; extra == 'ssh' |
| Provides-Extra: test |
| Requires-Dist: aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'test' |
| Requires-Dist: numpy; extra == 'test' |
| Requires-Dist: pytest; extra == 'test' |
| Requires-Dist: pytest-asyncio!=0.22.0; extra == 'test' |
| Requires-Dist: pytest-benchmark; extra == 'test' |
| Requires-Dist: pytest-cov; extra == 'test' |
| Requires-Dist: pytest-mock; extra == 'test' |
| Requires-Dist: pytest-recording; extra == 'test' |
| Requires-Dist: pytest-rerunfailures; extra == 'test' |
| Requires-Dist: requests; extra == 'test' |
| Provides-Extra: test-downstream |
| Requires-Dist: aiobotocore<3.0.0,>=2.5.4; extra == 'test-downstream' |
| Requires-Dist: dask[dataframe,test]; extra == 'test-downstream' |
| Requires-Dist: moto[server]<5,>4; extra == 'test-downstream' |
| Requires-Dist: pytest-timeout; extra == 'test-downstream' |
| Requires-Dist: xarray; extra == 'test-downstream' |
| Provides-Extra: test-full |
| Requires-Dist: adlfs; extra == 'test-full' |
| Requires-Dist: aiohttp!=4.0.0a0,!=4.0.0a1; extra == 'test-full' |
| Requires-Dist: cloudpickle; extra == 'test-full' |
| Requires-Dist: dask; extra == 'test-full' |
| Requires-Dist: distributed; extra == 'test-full' |
| Requires-Dist: dropbox; extra == 'test-full' |
| Requires-Dist: dropboxdrivefs; extra == 'test-full' |
| Requires-Dist: fastparquet; extra == 'test-full' |
| Requires-Dist: fusepy; extra == 'test-full' |
| Requires-Dist: gcsfs; extra == 'test-full' |
| Requires-Dist: jinja2; extra == 'test-full' |
| Requires-Dist: kerchunk; extra == 'test-full' |
| Requires-Dist: libarchive-c; extra == 'test-full' |
| Requires-Dist: lz4; extra == 'test-full' |
| Requires-Dist: notebook; extra == 'test-full' |
| Requires-Dist: numpy; extra == 'test-full' |
| Requires-Dist: ocifs; extra == 'test-full' |
| Requires-Dist: pandas; extra == 'test-full' |
| Requires-Dist: panel; extra == 'test-full' |
| Requires-Dist: paramiko; extra == 'test-full' |
| Requires-Dist: pyarrow; extra == 'test-full' |
| Requires-Dist: pyarrow>=1; extra == 'test-full' |
| Requires-Dist: pyftpdlib; extra == 'test-full' |
| Requires-Dist: pygit2; extra == 'test-full' |
| Requires-Dist: pytest; extra == 'test-full' |
| Requires-Dist: pytest-asyncio!=0.22.0; extra == 'test-full' |
| Requires-Dist: pytest-benchmark; extra == 'test-full' |
| Requires-Dist: pytest-cov; extra == 'test-full' |
| Requires-Dist: pytest-mock; extra == 'test-full' |
| Requires-Dist: pytest-recording; extra == 'test-full' |
| Requires-Dist: pytest-rerunfailures; extra == 'test-full' |
| Requires-Dist: python-snappy; extra == 'test-full' |
| Requires-Dist: requests; extra == 'test-full' |
| Requires-Dist: smbprotocol; extra == 'test-full' |
| Requires-Dist: tqdm; extra == 'test-full' |
| Requires-Dist: urllib3; extra == 'test-full' |
| Requires-Dist: zarr; extra == 'test-full' |
| Requires-Dist: zstandard; (python_version < '3.14') and extra == 'test-full' |
| Provides-Extra: tqdm |
| Requires-Dist: tqdm; extra == 'tqdm' |
| Description-Content-Type: text/markdown |
|
|
| |
|
|
| [](https://pypi.python.org/pypi/fsspec/) |
| [](https://anaconda.org/conda-forge/fsspec) |
|  |
| [](https://filesystem-spec.readthedocs.io/en/latest/?badge=latest) |
|
|
| A specification for pythonic filesystems. |
|
|
| |
|
|
| ```bash |
| pip install fsspec |
| ``` |
|
|
| would install the base fsspec. Various optionally supported features might require specification of custom |
| extra require, e.g. `pip install fsspec[ssh]` will install dependencies for `ssh` backends support. |
| Use `pip install fsspec[full]` for installation of all known extra dependencies. |
|
|
| Up-to-date package also provided through conda-forge distribution: |
|
|
| ```bash |
| conda install -c conda-forge fsspec |
| ``` |
|
|
|
|
| |
|
|
| To produce a template or specification for a file-system interface, that specific implementations should follow, |
| so that applications making use of them can rely on a common behaviour and not have to worry about the specific |
| internal implementation decisions with any given backend. Many such implementations are included in this package, |
| or in sister projects such as `s3fs` and `gcsfs`. |
|
|
| In addition, if this is well-designed, then additional functionality, such as a key-value store or FUSE |
| mounting of the file-system implementation may be available for all implementations "for free". |
|
|
| |
|
|
| Please refer to [RTD](https://filesystem-spec.readthedocs.io/en/latest/?badge=latest) |
|
|
| |
|
|
| fsspec uses GitHub Actions for CI. Environment files can be found |
| in the "ci/" directory. Note that the main environment is called "py38", |
| but it is expected that the version of python installed be adjustable at |
| CI runtime. For local use, pick a version suitable for you. |
|
|
| ```bash |
| |
| mamba create -n fsspec -c conda-forge python=3.10 -y |
| conda activate fsspec |
|
|
| |
| pip install -e ".[dev,doc,test]" |
|
|
| |
| pip install s3fs |
| pip uninstall s3fs |
| pip install -e .[dev,doc,test_full] |
| pip install s3fs --no-deps |
| pytest -v |
|
|
| |
| sh install_s3fs.sh |
| |
| install_s3fs.sh |
| ``` |
|
|
| |
|
|
| Tests can be run in the dev environment, if activated, via ``pytest fsspec``. |
|
|
| The full fsspec suite requires a system-level docker, docker-compose, and fuse |
| installation. If only making changes to one backend implementation, it is |
| not generally necessary to run all tests locally. |
|
|
| It is expected that contributors ensure that any change to fsspec does not |
| cause issues or regressions for either other fsspec-related packages such |
| as gcsfs and s3fs, nor for downstream users of fsspec. The "downstream" CI |
| run and corresponding environment file run a set of tests from the dask |
| test suite, and very minimal tests against pandas and zarr from the |
| test_downstream.py module in this repo. |
|
|
| |
|
|
| fsspec uses [Black](https://black.readthedocs.io/en/stable) to ensure |
| a consistent code format throughout the project. |
| Run ``black fsspec`` from the root of the filesystem_spec repository to |
| auto-format your code. Additionally, many editors have plugins that will apply |
| ``black`` as you edit files. ``black`` is included in the ``tox`` environments. |
|
|
| Optionally, you may wish to setup [pre-commit hooks](https://pre-commit.com) to |
| automatically run ``black`` when you make a git commit. |
| Run ``pre-commit install --install-hooks`` from the root of the |
| filesystem_spec repository to setup pre-commit hooks. ``black`` will now be run |
| before you commit, reformatting any changed files. You can format without |
| committing via ``pre-commit run`` or skip these checks with ``git commit |
| --no-verify``. |
|
|
| |
|
|
| Work on this repository is supported in part by: |
|
|
| "Anaconda, Inc. - Advancing AI through open source." |
|
|
| <a href="https://anaconda.com/"><img src="https://camo.githubusercontent.com/b8555ef2222598ed37ce38ac86955febbd25de7619931bb7dd3c58432181d3b6/68747470733a2f2f626565776172652e6f72672f636f6d6d756e6974792f6d656d626572732f616e61636f6e64612f616e61636f6e64612d6c617267652e706e67" alt="anaconda logo" width="40%"/></a> |
|
|