File size: 7,601 Bytes
3bb804c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
Metadata-Version: 2.1
Name: lazy_loader
Version: 0.4
Summary: Makes it easy to load subpackages and functions on demand.
Author: Scientific Python Developers
License: BSD 3-Clause License
        
        Copyright (c) 2022--2023, Scientific Python project
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
        
Project-URL: Home, https://scientific-python.org/specs/spec-0001/
Project-URL: Source, https://github.com/scientific-python/lazy_loader
Classifier: Development Status :: 4 - Beta
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: packaging
Requires-Dist: importlib-metadata ; python_version < "3.8"
Provides-Extra: dev
Requires-Dist: changelist ==0.5 ; extra == 'dev'
Provides-Extra: lint
Requires-Dist: pre-commit ==3.7.0 ; extra == 'lint'
Provides-Extra: test
Requires-Dist: pytest >=7.4 ; extra == 'test'
Requires-Dist: pytest-cov >=4.1 ; extra == 'test'

[![PyPI](https://img.shields.io/pypi/v/lazy_loader)](https://pypi.org/project/lazy_loader/)
[![Test status](https://github.com/scientific-python/lazy_loader/workflows/test/badge.svg?branch=main)](https://github.com/scientific-python/lazy_loader/actions?query=workflow%3A%22test%22)
[![Test coverage](https://codecov.io/gh/scientific-python/lazy_loader/branch/main/graph/badge.svg)](https://app.codecov.io/gh/scientific-python/lazy_loader/branch/main)

`lazy_loader` makes it easy to load subpackages and functions on demand.

## Motivation

1. Allow subpackages to be made visible to users without incurring import costs.
2. Allow external libraries to be imported only when used, improving import times.

For a more detailed discussion, see [the SPEC](https://scientific-python.org/specs/spec-0001/).

## Installation

```
pip install -U lazy_loader
```

We recommend using `lazy_loader` with Python >= 3.11.
If using Python 3.11, please upgrade to 3.11.9 or later.
If using Python 3.12, please upgrade to 3.12.3 or later.
These versions [avoid](https://github.com/python/cpython/pull/114781) a [known race condition](https://github.com/python/cpython/issues/114763).

## Usage

### Lazily load subpackages

Consider the `__init__.py` from [scikit-image](https://scikit-image.org):

```python
subpackages = [
    ...,
    'filters',
    ...
]

import lazy_loader as lazy
__getattr__, __dir__, _ = lazy.attach(__name__, subpackages)
```

You can now do:

```python
import skimage as ski
ski.filters.gaussian(...)
```

The `filters` subpackages will only be loaded once accessed.

### Lazily load subpackages and functions

Consider `skimage/filters/__init__.py`:

```python
from ..util import lazy

__getattr__, __dir__, __all__ = lazy.attach(
    __name__,
    submodules=['rank'],
    submod_attrs={
        '_gaussian': ['gaussian', 'difference_of_gaussians'],
        'edges': ['sobel', 'scharr', 'prewitt', 'roberts',
                  'laplace', 'farid']
    }
)
```

The above is equivalent to:

```python
from . import rank
from ._gaussian import gaussian, difference_of_gaussians
from .edges import (sobel, scharr, prewitt, roberts,
                    laplace, farid)
```

Except that all subpackages (such as `rank`) and functions (such as `sobel`) are loaded upon access.

### Type checkers

Static type checkers and IDEs cannot infer type information from
lazily loaded imports. As a workaround you can load [type
stubs](https://mypy.readthedocs.io/en/stable/stubs.html) (`.pyi`
files) with `lazy.attach_stub`:

```python
import lazy_loader as lazy
__getattr__, __dir__, _ = lazy.attach_stub(__name__, "subpackages.pyi")
```

Note that, since imports are now defined in `.pyi` files, those
are not only necessary for type checking but also at runtime.

The SPEC [describes this workaround in more
detail](https://scientific-python.org/specs/spec-0001/#type-checkers).

### Early failure

With lazy loading, missing imports no longer fail upon loading the
library. During development and testing, you can set the `EAGER_IMPORT`
environment variable to disable lazy loading.

### External libraries

The `lazy.attach` function discussed above is used to set up package
internal imports.

Use `lazy.load` to lazily import external libraries:

```python
sp = lazy.load('scipy')  # `sp` will only be loaded when accessed
sp.linalg.norm(...)
```

_Note that lazily importing *sub*packages,
i.e. `load('scipy.linalg')` will cause the package containing the
subpackage to be imported immediately; thus, this usage is
discouraged._

You can ask `lazy.load` to raise import errors as soon as it is called:

```python
linalg = lazy.load('scipy.linalg', error_on_import=True)
```

#### Optional requirements

One use for lazy loading is for loading optional dependencies, with
`ImportErrors` only arising when optional functionality is accessed. If optional
functionality depends on a specific version, a version requirement can
be set:

```python
np = lazy.load("numpy", require="numpy >=1.24")
```

In this case, if `numpy` is installed, but the version is less than 1.24,
the `np` module returned will raise an error on attribute access. Using
this feature is not all-or-nothing: One module may rely on one version of
numpy, while another module may not set any requirement.

_Note that the requirement must use the package [distribution name][] instead
of the module [import name][]. For example, the `pyyaml` distribution provides
the `yaml` module for import._

[distribution name]: https://packaging.python.org/en/latest/glossary/#term-Distribution-Package
[import name]: https://packaging.python.org/en/latest/glossary/#term-Import-Package