opendataset uses cftime, not datetime64, when calendar attribute is "Gregorian" <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: I used to open a NetCDF file whose coordinate had the attribute set to . All dates were within the Timestamp-valid range. The resulting dataset represented the co-ordinate as a . What you expected to happen: I expected the dataset to represent the co-ordinate as a , as documented and . Minimal Complete Verifiable Example: Anything else we need to know?: The says: In practice, we are getting some s even for times which are interpretable and representable as s. In particular, we have some NetCDF files in which the variable has a attribute with a value of (with a capital ‘G’). CF conventions : However, xarray regards as a non-standard calendar and falls back to . If (as in the example) is to , the times are read as s. Suggested fix: in , change ‘’ to ‘’.


Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.2 (default, Feb 21 2021, 05:02:46) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.8.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.17.1.dev39+g45b4436b pandas: 1.2.3 numpy: 1.20.2 scipy: None netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: None IPython: None sphinx: None <!-- Paste the output here xr.show_versions() here --</details


bfill behavior dask arrays with small chunk size bfill methods seems to miss nans when dask array chunk size is small. Resulting array still has nan present (see 'small chunk' section of code) absence of nans INSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.15.0-43-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: en_CA.UTF-8 xarray: 0.11.0 pandas: 0.23.4 numpy: 1.15.4 scipy: None netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: 1.0.0 distributed: 1.25.2 matplotlib: None cartopy: None seaborn: None setuptools: 40.6.3 pip: 18.1 conda: None pytest: None IPython: None sphinx: None


Should weighted operations raise an error when dimensions don't exist? <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Weighted operations don't raise an error when the dimensions passed don't exist. What you expected to happen: This is not really a bug, but I find it a bit confusing because it's not consistent with the same "unweighted" operation.


Minimal Complete Verifiable Example: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.9.4 (default, May 10 2021, 22:13:33) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1062.18.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: ('enGB', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.18.1.dev30+g2578fc3 pandas: 1.2.4 numpy: 1.20.2 scipy: 1.6.3 netCDF4: 1.5.6 pydap: installed h5netcdf: 0.11.0 h5py: 3.2.1 Nio: None zarr: 2.8.1 cftime: 1.4.1 nctimeaxis: 1.2.0 PseudoNetCDF: None rasterio: 1.2.3 cfgrib: 0.9.9.0 iris: None bottleneck: 1.3.2 dask: 2021.05.0 distributed: 2021.05.0 matplotlib: 3.4.2 cartopy: 0.19.0.post1 seaborn: 0.11.1 numbagg: installed pint: None setuptools: 49.6.0.post20210108 pip: 21.1.1 conda: None pytest: None IPython: 7.23.1 sphinx: None </details


Feature request: vector cross product xarray currently has the function for calculating arbitrary dot products which is indeed very handy. Sometimes, especially for physical applications I also need a vector cross product. I' wondering whether you would be interested in having as a wrapper of I currently use the following implementation: Do you want such a function (and possibly associated methods) in the namespace, or should it be in some other package? I didn't find a package which would be a good fit as this is close to core numpy functionality and isn't as domain specific as some geo packages. I'm not aware of some "xrphysics" package. I could make a PR if you'd want to have it in directly. <detailsINSTALLED VERSIONS commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.9.0-9-amd64 machine: x8664 processor: byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.3 pandas: 0.24.2 numpy: 1.16.4 scipy: 1.3.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.7.4 h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: 2.1.0 distributed: 2.1.0 matplotlib: 3.1.0 cartopy: None seaborn: 0.9.0 numbagg: None setuptools: 41.0.1 pip: 19.1.1 conda: 4.7.11 pytest: 5.0.1 IPython: 7.6.1 sphinx: 2.1.2 </details


Increase default This must have been introduced into somewhere around version 0.17. First of all this limit breaks backwards compatibility in the output format with or something similar on the console. Second, the default of 12 is much too low imo and makes no sense, in particular since terminals usually have a scrollback buffer and notebook cells can be made scrollable. I use frequently to check that all variables made it into the data set correctly, which is meaningless when lines are skipped with this default limit. And it broke my doctests that I wrote to do exactly that (thanks for that btw.). So if not removed, could the default be at least increased to a sensible number like 100 or 1000 or ? (I'd personally prefer much higher or even no limit, but I guess that is not an option.) Cheers.


Limit number of displayed dimensions in repr <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Dimension doesn't seem to be limited when there are too many of them. See example below. This slows down the repr significantly and is quite unreadable to me. What you expected to happen: To be limited so that it aligns with whatever the maximum line length is for variables. It's also fine if it continues on a couple of rows below in similar fashion to variables. Minimal Complete Verifiable Example: This is probably a bit of an edge case. My real datasets usually have around 12 "dimensions" and coords, +2000 variables, 50 attrs. Anything else we need to know?


: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.8 (default, Feb 20 2021, 15:50:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 byteorder: little LCALL: None LANG: en libhdf5: 1.10.6 libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: 3.2.1 Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.05.0 distributed: 2021.05.0 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 49.6.0.post20210108 pip: 21.1.2 conda: 4.10.1 pytest: 6.2.4 IPython: 7.24.1 sphinx: 4.0.2 </details


Complex LaTeX expressions in s aren't rendered correctly when plotting <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: When I try to give a variable a that's a complex latex expression and then plot that variable the expression doesn't get rendered by latex What you expected to happen: I expected the name to get rendered by latex Minimal Complete Verifiable Example: In the example below I'm plotting a variable with a complex via xarray and then plotting it again (in a separate figure) using only matplotlib and manually setting . The matplotlib-only version works fine (right), but the xarray version doesn't render (left). ! Anything else we need to know?


: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.2 (default, Mar 3 2021, 20:02:32) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.10.53-1-MANJARO machine: x8664 processor: byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.6.1 xarray: 0.17.0 pandas: 1.2.3 numpy: 1.19.2 scipy: 1.5.3 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.04.0 distributed: 2021.04.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: None numbagg: None pint: 0.17 setuptools: 52.0.0.post20210125 pip: 21.0.1 conda: None pytest: None IPython: 7.22.0 sphinx: None </details


Dask error on What happened: When I use on two Dataarrays I get a error. Notice that dask is not installed in my environement. What you expected to happen: Obtain the correlation values without dask interfering (as it should be optional in my understanding) Minimal Complete Verifiable Example: Results in: Environment:


Dataset groupby returning DataArray broken in some cases Got a TypeError when resampling a dataset along a dimension, mapping a function to each group. The function returns a DataArray. Failed with : This worked before the merging of . A DataArray was returned as expected. In the docstring of it is not made clear that the passed function should return a dataset, but the opposite is also not said. This worked before and I think the issues comes from , which introduced different signatures for (which is called in my case) and (which is expected by the new ). If the function passed to should only return s then I believe a more explicit error is needed, as well as some notice in the docs and a breaking change entry in the changelog. If s should be accepted, then we have a regression here. I may have time to help on this.


<detailsINSTALLED VERSIONS commit: None python: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.16.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: ('frCA', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.1.dev16+g3ead17ea pandas: 1.4.0 numpy: 1.20.3 scipy: 1.7.1 netCDF4: 1.5.7 pydap: None h5netcdf: 0.11.0 h5py: 3.4.0 Nio: None zarr: 2.10.0 cftime: 1.5.0 nctime_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.08.0 distributed: 2021.08.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2021.07.0 cupy: None pint: 0.18 sparse: None setuptools: 57.4.0 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 8.0.1 sphinx: 4.1.2 </details


DataArray groupby returning Dataset broken in some cases This is a the reverse problem of , the method seems broken when the mapped function returns a Dataset (which worked before ). No response I guess the same solution as could be used! <detailsINSTALLED VERSIONS commit: None python: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.16.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: ('frCA', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.1.dev16+g3ead17ea pandas: 1.4.0 numpy: 1.20.3 scipy: 1.7.1 netCDF4: 1.5.7 pydap: None h5netcdf: 0.11.0 h5py: 3.4.0 Nio: None zarr: 2.10.0 cftime: 1.5.0 nctime_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.08.0 distributed: 2021.08.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2021.07.0 cupy: None pint: 0.18 sparse: None setuptools: 57.4.0 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 8.0.1 sphinx: 4.1.2 </details


Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details


with scalar as second argument fails with keepattrs=True fails with The workaround is to pass No response No response No response No response xarray 2022.3.0


xr.polyval first arg requires name attribute I have some polynomial coefficients and want to evaluate them at some values using . As described in the docstring/docu I created a 1D coordinate DataArray and pass it to but it raises a KeyError (see example). I expected that the polynomial would be evaluated at the given points. No response I assume that the "standard" workflow is to obtain the argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow. It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future. <detailsINSTALLED VERSIONS commit: None python: 3.9.10 (main, Mar 15 2022, 15:56:56) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: None netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 58.1.0 pip: 22.0.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details


xarray improperly decodes times from a NetCDF when it is a uint improperly decodes times from a NetCDF when it is a . The generates a NetCDF file with the right time ('goodtime') and the wrong time ('time') (use ) to be properly decoded (see ). [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate.


No response <detailsINSTALLED VERSIONS commit: None python: 3.8.13 (default, Mar 25 2022, 06:04:10) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.62.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.8.0 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.7.0 netCDF4: 1.5.7 pydap: None h5netcdf: 1.0.0 h5py: 3.3.0 Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: 0.9.10.1 iris: None bottleneck: 1.3.4 dask: 2022.04.1 distributed: 2022.4.1 matplotlib: 3.5.1 cartopy: None seaborn: 0.11.2 numbagg: None fsspec: 2021.06.1 cupy: None pint: 0.19.1 sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: 4.12.0 pytest: 7.1.1 IPython: 8.2.0 sphinx: 4.5.0 </details


with timedelta64 coordinates produces wrong results I'm not sure if this is a bug or an expected breaking change, but I'm not able to reproduce the results generated by using a timedelta64 coordinate. The results are correct in , whereas they are wrong in the latest unreleased version (, ). Both the stable and latest functions should return the same results. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 or 2022.3.1.dev102+g6bb2b855 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.0 distributed: 2022.5.0 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.2.0 pip: 22.1 conda: None pytest: 7.1.2 IPython: None sphinx: None </details


returns objects with different dimension order I noticed that the dimension order of the object returned by the latest (, unreleased) is different compared to . For example, the following code returns different results. Is this the expected behaviour? If yes, is it worth mentioning this change in what's new/breaking changes? cc:


Accessing chunks on zarr backed xarray seems to load entire array into memory When running the following example it appears the entire dataset is loaded into memory when accessing the attribute: According to accessing the chunks attribute should simply inspect the attribute on the underlying DataArrays. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.9.12 (main, Mar 24 2022, 23:24:38) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.2.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.21.2 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.1 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.04.0 distributed: 2022.4.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.0.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details


"center" kwarg ignored when manually iterating over DataArrayRolling <div type='discussions-op-text'<supOriginally posted by ckingdon95 June 29, 2022</supHello, I am trying to manually iterate over a DataArrayRolling object, as described in the documentation. I am confused why the following two code chunks do not produce the same sequence of values. I would like to be able to manually iterate over a DataArrayRolling object, and still be given center-justified windows. Is there a way to do this? This returns the following values, as expected: Whereas when I do it manually, it is not equivalent: This returns Is this an issue with the window iterator? If it is not an issue, then is there a way for me to get the center-justified windows in the manual iteration? </div


Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything.


No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details


Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks


RuntimeError when formatting sparse-backed DataArray in f-string On upgrading from xarray 2022.3.0 to 2022.6.0, f-string formatting of sparse-backed DataArray raises an exception. Code does not error, or A breaking change is listed in the section of the docs.[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. Along with the versions below, I have confirmed the error occurs with both sparse 0.12 and sparse 0.13.


<detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: ('enCA', 'UTF-8') libhdf5: 1.10.7 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: 0.12.0 h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2022.01.0+dfsg distributed: 2022.01.0+ds.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.01.0 cupy: None pint: 0.18 sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 62.1.0 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 7.31.1 sphinx: 4.5.0 </details


Testing DataArray equality using built-in '==' operator leads to mutilated dictionary In previous versions of xarray, testing numerical equivalence of two DataArrays was possible using the built-in operator '==' and without side affects. Now in version 2022.6.0, when one DataArray lacks an attribute that the other DataArray has, the DataArray with the attribute is mutilated during comparison leading to an empty attrs dictionary. DataArray1 == DataArray2 should not have side affects. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.9.13 (tags/v3.9.13:, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: ('EnglishUnited States', '1252') libhdf5: None libnetcdf: None xarray: 2022.6.0 pandas: 1.4.3 numpy: 1.23.1 scipy: 1.9.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.2 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 63.2.0 pip: 22.2.1 conda: None pytest: 7.1.2 IPython: 8.4.0 sphinx: None </details


Cannot open NetCDF file if dimension with time coordinate has length 0 ( when decoding CF datetime) If a data set has a zero-sized coordinate that is a time index, reading fails. A is triggered when xarray tries to decode the array, as shown below:


Alignment of dataset with MultiIndex fails after applying After applying the function to a dataset with a Multiindex, a lot of functions related to indexing are broken. For example, it is not possible to apply to itself anymore. The error is raised in the alignment module. It seems that the function does not find indexes that belong to the same dimension. I expected the alignment to be functional and that these basic functions work. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate.


No response <detailsINSTALLED VERSIONS commit: None python: 3.10.5 (main, Jun 14 2022, 07:04:59) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.21.6 scipy: 1.8.1 netCDF4: 1.6.0 pydap: None h5netcdf: None h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.1.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.6.1 distributed: 2022.6.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: 0.13.0 flox: None numpygroupies: None setuptools: 61.2.0 pip: 22.1.2 conda: 4.13.0 pytest: 7.1.2 IPython: 7.33.0 sphinx: 5.0.2 /home/fabian/.miniconda3/lib/python3.10/site- UserWarning: Setuptools is replacing distutils. ("Setuptools is replacing distutils.") </details


can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details


index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details


[Bug]: renamevars to dimension coordinate does not create an index We used to make a coordinate a dimension coordinate (instead of ). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like fail with a strange . I expect one of two things to be true: does not allow setting dimension coordinates (raises Error and tells you to use setindex) checks for this occasion and sets the index correctly No response No response INSTALLED VERSIONS commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None


Indexes.getunique() TypeError with pandas indexes I also just tested the method that you mentioned and maybe noticed a related issue here, which I'm not sure is wanted / expected. Taking the above dataset , accessing this function results in an error: However, for it works: Originally posted by in


Broken state when using assigncoords with multiindex I was trying to assign coordinates on a dataset that had been created by using stack. After assigning the coordinates, the dataset was in a state where its length was coming out as less than zero, which caused all sorts of issues. I think the issue is with the updating of , perhaps in I expected to just be able to assign the coords and then print the array to see the result. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. Here's a test (I put it in but maybe there is a better spot) <detailsINSTALLED VERSIONS commit: python: 3.8.13 (default, Mar 25 2022, 06:04:14) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.5.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enAU.UTF-8 LOCALE: ('enAU', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 0.1.dev4312+ge678a1d.


pandas: 1.5.0 numpy: 1.22.4 scipy: 1.9.1 netCDF4: 1.6.1 pydap: installed h5netcdf: 1.0.2 h5py: 3.7.0 Nio: None zarr: 2.13.2 cftime: 1.6.2 nctimeaxis: 1.4.1 PseudoNetCDF: 3.2.2 rasterio: 1.3.1 cfgrib: 0.9.10.1 iris: 3.3.0 bottleneck: 1.3.5 dask: 2022.9.1 distributed: 2022.9.1 matplotlib: 3.6.0 cartopy: 0.21.0 seaborn: 0.12.0 numbagg: 0.2.1 fsspec: 2022.8.2 cupy: None pint: 0.19.2 sparse: 0.13.0 flox: 0.5.9 numpy_groupies: 0.9.19 setuptools: 65.4.0 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: None sphinx: None </details


Raise nicer error if passing a list of dimension names to transpose Hello, in xarray 0.20.1, I am getting the following error The expected result is No response No response No response <detailsINSTALLED VERSIONS commit: None python: 3.9.12 (main, Apr 5 2022, 06:56:58) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.42.2.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS LOCALE: ('enUS', 'ISO8859-1') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.20.1 pandas: 1.4.1 numpy: 1.21.5 scipy: 1.8.0 netCDF4: 1.5.7 pydap: None h5netcdf: 999 h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.1.1 nctimeaxis: 1.4.0 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.02.1 distributed: 2022.2.1 matplotlib: 3.5.1 cartopy: 0.18.0 seaborn: 0.11.2 numbagg: None fsspec: 2022.02.0 cupy: None pint: 0.18 sparse: 0.13.0 setuptools: 61.2.0 pip: 21.2.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details


Time decoding error message does not include the problematic variable's name If any variable in a Dataset has times that cannot be represented as objects, an error message will be raised. However, this error message will not indicate the problematic variable's name. It would be nice if it did, because it would make it easier for users to determine the source of the error. cc: xref: Unidata/cftime This is a minimal example of the issue. The error message gives no indication that is the problem:


xarray.opendataset has issues if the dataset returned by the backend contains a multiindex As a follow up of this comment: I'm currently trying to implement a custom backend that allows me to also handle multiindices when loading a NetCDF dataset using . I'm using the following two functions to convert the dataset to a NetCDF compatible version and back again: Here is a small code example: While the manual patching for saving is currently still required, I tried to at least work around the function call in by creating a custom NetCDF Backend: but this works: So I'm guessing is performing some operation on the dataset returned by the backend, and one of those leads to a failure if there is a multiindex already contained. I expected that it doesn't matter wheter is called inside the backend or afterwards, and the same dataset will be returned each time. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response I'm also open to other suggestions how I could simplify the usage of multiindices, maybe there is an approach that doesn't require a custom backend at all?


<detailsINSTALLED VERSIONS commit: None python: 3.8.10 (default, Jan 28 2022, 09:41:12) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.10.102.1-microsoft-standard-WSL2 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: C.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.10.5 libnetcdf: 4.6.3 xarray: 2022.9.0 pandas: 1.5.0 numpy: 1.23.3 scipy: 1.9.1 netCDF4: 1.5.4 pydap: None h5netcdf: None h5py: 3.7.0 Nio: None zarr: None cftime: 1.6.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.3.2 cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.6.0 cartopy: 0.19.0.post1 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 65.3.0 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: 8.5.0 sphinx: 4.5.0 </details


Avoid loading any data for reprs For "small" datasets, we load in to memory when displaying the repr. For cloud backed datasets with large number of "small" variables, this can use a lot of time sequentially loading O(100) variables just for a repr. Fast reprs! This dataset has 48 "small" variables On this repr takes 36.4s If I comment the condition I get 6μs. [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [x] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [x] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.5.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.2 distributed: None matplotlib: 3.5.2 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.5.0 cupy: None pint: None sparse: None setuptools: 62.3.2 pip: 22.1.2 conda: None pytest: None IPython: 8.4.0 sphinx: 4.5.0 </details


overwrites coordinate attributes had some unintended consequences for , where coordinate attributes are getting overwritten by variable attributes. I guess this has been broken since . Coordinate attributes should be preserved. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. I'm struggling to figure out how the simple change in brought this about. I tried tracing my way through the various merge functions but there are a lot of layers. Happy to submit a PR if someone has an idea for an obvious fix.


<detailsINSTALLED VERSIONS commit: None python: 3.9.13 (main, May 27 2022, 16:56:21) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-52-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 2022.10.0 pandas: 1.4.3 numpy: 1.23.4 scipy: 1.9.3 netCDF4: 1.6.1 pydap: None h5netcdf: 1.0.2 h5py: 3.7.0 Nio: None zarr: 2.13.3 cftime: 1.6.2 nctimeaxis: 1.4.1 PseudoNetCDF: None rasterio: 1.3.3 cfgrib: 0.9.10.2 iris: None bottleneck: 1.3.5 dask: 2022.10.0 distributed: 2022.10.0 matplotlib: 3.6.1 cartopy: 0.21.0 seaborn: None numbagg: None fsspec: 2022.10.0 cupy: None pint: 0.19.2 sparse: 0.13.0 flox: 0.6.1 numpy_groupies: 0.9.19 setuptools: 65.5.0 pip: 22.3 conda: None pytest: 7.1.3 IPython: 8.5.0 sphinx: None </details


ds.Coarsen.construct demotes non-dimensional coordinates to variables demotes non-dimensional coordinates to variables All variables that were coordinates before the coarsen.construct stay as coordinates afterwards. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response_


assigncoords reset all dimension coords to default (pandas) index See should preserve the index of coordinates that are not updated or not part of a dropped multi-coordinate index. See [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response_ <detailsXarray version 2022.11.0 </details


binary ops ignore option When doing arithmetic operations on two Dataset operands, the option is ignored and therefore attributes not kept. Minimal example: This is not true for DataArrays/Variables which do take into account. Datasets to behave the same as DataArray/Variables, and keep attributes during binary operations when option is set. PR is inbound.


stack casts int32 dtype coordinate to int64 The code example below results in , because the data type of the coordinate is changed from 'i4' to 'i8'. I expect the result to be . Creating a MultiIndex should not change the data type of the Indexes from which it is built. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)] python-bits: 64 OS: Darwin OS-release: 21.6.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.9.0 xarray: 2022.10.0 pandas: 1.5.1 numpy: 1.23.4 scipy: 1.9.3 netCDF4: 1.6.1 pydap: None h5netcdf: None h5py: 3.7.0 Nio: None zarr: None cftime: 1.6.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.10.2 distributed: None matplotlib: 3.6.1 cartopy: 0.21.0 seaborn: None numbagg: None fsspec: 2022.10.0 cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 65.5.0 pip: 22.1.2 conda: None pytest: None IPython: 8.6.0 sphinx: None ("Setuptools is replacing distutils.") </details


Add defaults during concat 508 [x] [x] Tests [x] Passes [x] Fully documented, including for all changes and for new API Contined on issue by removing exception when concat two datasets with disjoint variables and instead add the missing variable with


Pylint 2.7.0 seems to ignore the min-similarity-lines setting <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --two Python source files that share 8 common lines min-similarity-lines=40 in the pylint config pylint 2.7.0 on the source files Before pylint 2.7.0, the min-similarity-lines setting was honored and caused shorter similar lines to be accepted. Starting with pylint 2.7.0, the min-similarity-lines setting seems to be ignored and the common lines are always reported as an issue R0801, even when the min-similarity-lines setting is significantly larger than the number of common lines. The min-similarity-lines setting should be respected again as it was before pylint 2.7.0. pylint 2.7.0 astroid 2.5 Python 3.9.1 (default, Feb 1 2021, 20:41:56) [Clang 12.0.0 (clang-1200.0.32.29)] Pylint 2.7.0 seems to ignore the min-similarity-lines setting <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --two Python source files that share 8 common lines min-similarity-lines=40 in the pylint config pylint 2.7.0 on the source files Before pylint 2.7.0, the min-similarity-lines setting was honored and caused shorter similar lines to be accepted. Starting with pylint 2.7.0, the min-similarity-lines setting seems to be ignored and the common lines are always reported as an issue R0801, even when the min-similarity-lines setting is significantly larger than the number of common lines.


The min-similarity-lines setting should be respected again as it was before pylint 2.7.0. pylint 2.7.0 astroid 2.5 Python 3.9.1 (default, Feb 1 2021, 20:41:56) [Clang 12.0.0 (clang-1200.0.32.29)]


Support output to a file to support tox on CI servers Currently , which means when running Pylint from within there seems to be no way to output the results to a single (for example HTML) file that a CI server can store for display. used to be close to what is needed, but it , and also created a different file for each module which is difficult to navigate. Here is the that relates to this. this feature for the same reason:


Add --fail-on option to always return error code if specific issues (or issue types) are found We're using pylint in CI with two primary goals: there are no error-category issues - this can be achieved by looking at the exit code the overall linting score doesn't deteriorate too much - this can be achieved by using However if is used, and the score is above that, then it passes even if there are error-category issues detected. Essentially it's acting as a "only throw fail (non-zero) codes if under this", instead of a "fail if under this, otherwise change nothing". Two possible solutions I can think of here are: a configuration option to prevent from consuming other exit statuses. I.e. a way to say "if score is under N, return error code X, regardless of all else, otherwise change nothing". a new option like which means that if there are any code (error-category) issues, or any issues, then fail, otherwise change nothing.


Pylint 2.8.2 broke pylint-quotes <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --See See in attached Running this configuration on a file containing: results in an exception: should not throw exceptions, and instead generate report complaining about the use of double quotes () and illegal variable name . Result of output: Additional dependencies: seems to have just changed versioning scheme: broke this line:


Allow parseable output in parallel of human-readable output Currently, when selecting a format like JSON using the argument, it will replace the "normal" output. It would be nice to have it generate an output-file instead so the normal human-readable output is still there. This makes sens in a CI environment like travis or gitlab where you can see the stdout and stderr of each job but where you also want to parse the output. Allowing multiple output formats would solve this. Maybe with a new CLI argument? Or by adding a separator to the current value? For example: pylint -f # Would redirect the JSON data to and still write the normal report to stdout pylint 1.8.1, astroid 1.6.0 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 ]


Ignore clause not ignoring directories This is a different issue to . a directory and within that a directory . files and . Put syntax errors in both. , run . Pylint does not ignore , producing the message Pylint ignores the file . ignore-patterns does not skip non-top-level directories. <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --create a (where c.py will generate a pylint message, so that we get output) (along with the appropriate \init\.py files) Run pylint: pylint --ignore-patterns=b Run pylint: pylint --ignore-patterns=a c.py is skipped for ignore-patterns=a, but not for ignore-patterns=b c.py should be skipped for both pylint 2.1.1 astroid 2.1.0-dev Python 3.6.3 (v3.6.3:, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)]


Use Python type hints for UML generation It seems that pyreverse does not read python type hints (as defined by ), and this does not help when you use as a default value : Output of pyreverse : ! I would like to see something like : in the output. pylint- 1.6.5, astroid 1.4.9 Python 3.6.0 (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]


unused-import false positive for a module used in a type comment should not be emitted. Result of output: This is a follow up to .


Make pylint XDG Base Directory Specification compliant I have this really annoying directory in my home folder. From what I can tell (I don't do C or C++), this directory is storing data. The problem with this is, quite simply, that data storage has a designated spot. The folder. This is a part of the . A system that designates the folders for specific things like cached files (), configuration files (), and data files (), among other things. The point is to keep user home directories clean and the user sane. This should be pretty easy to implement. Simply change the variables/constants for where these files are made and stored to the appropriate directory. Simple as that, even for a large codebase (if it was done right).


Similarities checker with "ignore-signatures" option enabled ignores functions with docstring-only bodies follow-up. Given multiple files: : : Result of : error shouldn't be reported in this case. Note: if functions have bodies everything works correctly. You could add to the example functions above and no error would be reported. Result of output:


Imports within TYPECHECKING can induce "cyclic-import" Suppose you have two modules: a.py and b.py. imports , but needs in order to do type annotations, then this is usually done by importing within a block. Unfortunately, this causes pylint to report . Possibly related to


Config File Does not Expand Environment Variables If a config file is provided that has an environment variable, such as "%AppData%", it fails. This can be fixed by changing from to


Ignore class methods signatures with similarities "ignore-signatures" option enabled Initial already closed issue . One more related closed question with examples . Similarities "ignore-signatures" option currently only ignores functions' signatures. Class methods won't be ignored with the current implementation which would be very useful to avoid false-negative errors for classes. With similarities "ignore-signatures" option enabled, class methods' signatures are ignored the same way as functions'.


Setting to should stop pylint from checking duplicate code Setting to in the rcfile doesn't disable checking for duplicate code, it instead treats every line of code as duplicate and raises many errors. Setting to should disable the duplicate code check. It works that way in many other linters (like flake8). Setting a numerical value in flake8 to (e.g. ) disables that check. requests being able to disable , but it is still open


args is considered as missing in documentation Hi, I met an issue with the following code: ` Return this warning: W9015: "args" missing in parameter documentation (missing-param-doc) No warning must be returned as there is a documentation concerning args pylint 2.5.3 astroid 2.4.2 Python 3.7.2 (default, Feb 25 2019, 14:07:05) [GCC 4.8.5 (Red Hat 4.8.5-36)]


False positive missing-param-doc on numpy style when omitted Would not have expected either or checks to have failed. VS Code 1.61.2 Windows 10 Pro 21H1 No response False positive missing-param-doc on numpy style when omitted Would not have expected either or checks to have failed. VS Code 1.61.2 Windows 10 Pro 21H1 No response


The duplicate-code (R0801) can't be disabled Originally reported by: Anonymous It's seems like it's not possible to disable the duplicate code check on portions of a file. Looking at the source, I can see why as it's not a trivial thing to do (if you want to maintain the same scope semantics as other #pylint:enable/disable comments. This would be nice to have though (or I guess I could just cleanup my duplicate code). Bitbucket:


The negative score is not very beginner friendly For the past months I've been under the impression that the negative score is not very friendly for beginners. It somewhat demotivating to run over your project just to get a as a score, which would make look like a super drastic tool, that's super nitpicky about everything. I think we could do a better job here at welcoming everyone to using . Instead of having a score, we could check if other tools are doing something different and see what works best in general. The negative score is not very beginner friendly For the past months I've been under the impression that the negative score is not very friendly for beginners. It somewhat demotivating to run over your project just to get a as a score, which would make look like a super drastic tool, that's super nitpicky about everything. I think we could do a better job here at welcoming everyone to using . Instead of having a score, we could check if other tools are doing something different and see what works best in general.


Please provide a better entry point to pyreverse Currently, I have to do: I would like to do something like:


[crash] doc params extension: --accept-no-param-doc: conflicting option string(s) There is a crash when using the doc_params extension. No crash.


Pylint needs to forbid to re-use the msgid or symbol of old deleted msgid/symbol Right now it's possible to reuse old msgid that were removed, for example, everything from the python 3 checker (the problem was detected with `), or mixed-indentation, bad-whitespace, bad-continuation... maybe other that I don't remember that we deleted. We have a mechanism in place for renamed message with old_name, but nothing for removed message. Related to and This is going to cause occasional bug, but more than that confusion and inconsistencies when searching for the msgid online. Impossible to use an old message id or symbol.


"--notes" option ignores note tags that are entirely punctuation If a note tag specified with the option is entirely punctuation, pylint won't report a fixme warning (W0511). will return a fixme warning (W0511) for the first line, but not the second. macOS 11.6.1 No response


DOC: pyreverse supported format not recognized missing puml, plantuml, mmd, etc Some supported formats are not listed in the error messaging: Here, I have made the mistake of prepending a to . The error output should list (and and and ) as . Without the preceding , the command runs as expected. All supported formats are listed when user makes a typo in the format argument OpenSUSE Tumbleweed


Is required As noted by ,  is not currently covered. If this required then we should add a unit test, otherwise we can remove this method. N/A No response No response


Use, misuse and confusion of decorator The work on showed that specifically enabling just a single message is sometimes not working as expected. This is due to either missing message declaration in a checkers decorator. Ideally I would like to implement a new, optional checker aimed at Pylint core development and custom checker development. This checker would focus on correct message declaration and handling. I already started working on it, but noticed that there is especially quite some problems with the decorator. I totally understand that, because while being extensively used across all checker classes, there is not a single word on its purpose and effect in the documentation. Its docstring is also not really helpful, you have to dig deeper into the codebase to understand what it does. Before continuing my work on the new checker, I want to address and clarify some questions about the intended use of. After that this issue can be closed with a PR that extends the documentation, and if necessary some code changes. A brief explanation on what does: It helps the decide whether a callback for the current node type should be called or not. This is done by adding the list of messages passed in as a attribute on method object. When a checker is to the attribute: if the attribute does not exist, this method will be to the list of callbacks if it exists and at least one message of the list is enabled, this method will be to the list of callbacks otherwise, it will not be to the list of callbacks and will therefore not be executed Essentially this means: main (and only) intent of the decorator is to improve performance the does not have any effect to apply the decorator has no negative functional consequences, it only affects performance (which, of course, is always an issue) the decorator but forgetting to add messages that are emitted from this callback can lead to false negatives: if none of the messages actually included in the decorator are enabled, it will not be called the decorator to a callback that not only emits messages but also does other things can have nasty side effects (false positives, false negatives, crashes) What I want to gain a consensus on: we want to keep the name ? I don't think it really conveys the effect it has on the decorated method. shall we regard as best practice for this decorator?


Should every callback be decorated and the list of messages must always be correct and consistent, or should we rather advise to use it only in cases where we expect a significant speedup when this callback is omitted? we want to keep the restriction that this decorator only has an effect on the "top level callbacks"? Seeing that a lot of those callbacks can emit quite a lot of messages and the logic for the checks is often split up into several subfunctions, it would be more flexible and also more robust (as it is easier to keep track of the messages that need to be included in the decorator) if one could apply it on the subfunctions rather than on the top level callback. Looking over the code base there are quite a lot of places where the decorator is not used correctly: used with simply passing in all messages (which has the same effect as just omitting it): used on methods that are not a callback (which has no effect) Used, but not including all messages possibly emitted by the callback No response_


Crash When parsing the following file: <!-- If sharing the code is not an option, please state so, but providing only the stacktrace would still be helpful. --pylint crashed with a and with the following stacktrace: If I remove the comment, things work as expected. If I call without , things work as expected. If I downgrade (and ) to astroid-2.9.3 pylint-2.12.2, things work as expected. No response no astroid error ;-) Fedora 35 astroid==2.11.2 dill==0.3.4 flake8==4.0.1 isort==4.3.21 lazy-object-proxy==1.7.1 mccabe==0.6.1 platformdirs==2.5.1 pycodestyle==2.8.0 pyflakes==2.4.0 pylint==2.13.5 toml==0.10.2 tomli==2.0.1 wrapt==1.13.3


option ignored The option is ignored in 2.14. Place two files that both read like this (ideally in their own dir so you can just run that dir): Proper output in 2.12: Caused in No response No response


Argument expected for short verbose option The short option of the option expects an argument. Also, the help message for the option suggests a value should be provided. The long option works ok & doesn't expect an argument: Similar behaviour to the long option.


Support isinstance checks through ABCMeta for checkers and reporters Currently the PyLinter categorizes checkers through the class attribute. This is not very standard, and is one more barrier for others to write plugins. I propose Changing the interfaces to have a metaclass Change PyLinter to do isinstance checks to categorize checkers/reporters in addition to keeping the old checks


Traceback printed for unrecognized option A traceback is printed when an unrecognized option is passed to pylint. No response The top part of the current output is handy: The traceback I don't think is expected & not user-friendly. A usage tip, for example: No response No response


Pylint runs unexpectedly pass if in pylintrc Runs unexpectedly pass in 2.14 if a pylintrc file has . (Default pylintrc files have . 's own config was fixed in to comment it out, but this might bite existing projects.) No response No response


How does pylint decide where it should put stat files? I am from the VS Code python extension team, working on building an extension for VS Code using pylint. We have pre-release version out. An issue was reported on the extension that it creates pylint stat files in the project directory. We are not specifically passing anything to change the stat file location (see issue here ). Generally, the stat files go into pre-set directory like %USERPROFILE%/AppData/Local/pylint on windows. What can cause these to be written to the current working directory. We use pass the source code into pylint via stdin, and the only two arguments we use are , . Any clue to debug this would be helpful. I could not find fall back location for the stat files and when would the fallback location be used. We use to run pylint, and we send source to pylint via stdin. Relevant code here:


Pylint does not respect ignores in mode Pylint does not respect the , , or setting when running in recursive mode. This contradicts the documentation and seriously compromises the usefulness of recursive mode. No response All of these commands give the same output: should be ignored by all of the above commands, because it is in an ignored directory (even the first command with no ignore setting should skip it, since the default value of is . For reference, the docs for the various ignore settings from : No response No response


Can't disable bad-option-value code on a computer with a somewhat new pylint (2.4.3 in my example). Get a warning like that I want to ignore, as I'm writing code compatible with python2 and python3. said warning with . a "Bad option value" when other people run their pylint version (example: 2.3.1; and by people, sometimes I mean docker instances ran from Jenkins that I would rather not rebuild or that depend on other people and I can't modify) to disable said error with a global is ignored is ignored To be able to write code that works on several versions of pylint and not having to make sure every computer in the company and every docker container has the same pylint version.


Using in CLI silently fail to load the plugin See Warning that load-plugin is not the right argument.


Running pylint in Kubernetes Pod with --jobs=0 fails I run pylint in multiple parallel stages with Jenkins at a Kubernets agent with . The newly introduced function is called to determine the number of cpus to use and returns 0 in this case. This leads to a crash of pylint because the multiprocessing needs a value 0. I checked the function and found out the following values from the files that are read in above mentioned function: This leads to the calculation then in line which is cast to an and therefore 0 then. No response I expect pylint to not crash if the number of available cpu is misscalculated in this special case. The calculated number should never be 0. A possible solution would be to append a at the end of this line. I'm not sure if the same can happen for the calculation in line though, as I don't know the exact backgrounds of that files. Ubuntu 20.04 Kubernetes Version: v1.18.6 Python 3.9.12 No response


--disable ignored when --errors-only specified a python file () with a single line import notfoundpackage --disable=E0401 --errors-only test.py_ reports *** Module test E0401: Unable to import 'notfoundpackage' (import-error) Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) 11.5


ignores When running recursively, it seems in my settings in is completely ignored src\gen* should not be checked Windows 10.0. No response


Linting fails if module contains module of the same name Given multiple files: Which are all empty, running fails: However, if I rename , succeeds: Alternatively, I can also , but that shouldn't be necessary anymore. Running if is present fails while searching for an file. Running if is present should succeed. Result of output: This also has some side-effects in module resolution. For example, if I create another file : With the content: Running will run fine, but will fail. Not just for module a, but for module r as well. Again, if I rename to , will work perfectly.


rxg include '\p{Han}' will throw error config rxg in pylintrc with \p{Han} will throw err .pylintrc: not throw error macOS 11.6.7


removes first item from when running from . This is the line where the first item from is removed. I think there should be a check to ensure that the first item is , or before removing. No response Check if , or before removing the first item from No response No response


Using custom braces in message template does not work Have any list of errors: On pylint 1.7 w/ python3.6 - I am able to use this as my message template However, on Python3.9 with Pylint 2.12.2, I get the following: Is this intentional or a bug? No response Expect the dictionary to print out with as the key. No response No response


false positive 'useless-import-alias' error for mypy-compatible explicit re-exports Suppose a package has the following layout: Since the submodules here implement public APIs, imports and re-exports them, expecting users to import them from the public, top-level package, e.g. . Since the implementations of and are complex, they are split into and for better maintainability and separation of concerns. So looks like this: The reason for the aliases here is to be explicit that these imports are for the purpose of re-export (without having to resort to defining , which is error-prone). Without the aliases, popular linters such as will raise an "implicit re-export" error ( -- part of ). However, pylint does not currently understand this usage, and raises "useless-import-alias" errors. Example real-world code triggering pylint false positive errors: No response No "useless-import-alias" errors should be flagged. No response No response_


False positive when importing from ` option Simply importing exceptions from the  library is giving this error. Here's an example of how we import them: Pycharm can find the exception classes just fine. I know they exist. It could have something to do with how the library is using , but I don't know too much about how that works to draw that conclusion. Also, note that we're using version 1.95.1 of . We use it in some critical paths, so we can't update it to the latest version quite yet. The configuration written below is what I've tried, but it seems based on googling that that doesn't stop all errors from being ignored regarding those modules. So I'm still getting the issue. No error to be reported Intel based 2019 Mac Book Pro. Mac OS 13.1 (Ventura). Fish shell. ccxt==1.95.1


Support globbing patterns for source-roots It's quite inconvenient having to specify every source root for complex multi-package projects like For complex multi-package projects it would be nice to be able to specify source roots as instead of listing every one of them. IMHO, it's better to go with globbing patterns rather than with regexp patterns since those give better support for path-specific matching. No response


Support globbing pattern for input specification As mentioned in and , lets consider supporting cc Add globbing support right in this line No response_


fail/warn on using parallel execution with custom plugins According to documentation: Actually, it is possible but silently broken. then Pylint should inform a user about it in such cases. As for now, I could run: without any warning or error. Unfortunately, linting results are not the same as a single process linting, but Pylint silently pass. So, results are not predictable. Proposal: emit a warning or better explicitly fail on using parallel execution with custom Pylint plugins, because people usually don't read the documentation while things works.


Different output with --jobs=1 and --jobs=2 Originally reported by: Robert Spier (BitBucket: ) Parallelism (--jobs) changes the output of pylint. It's not just the order of the tests, --jobs=2 outputs lines of output compared to only 21 for --jobs=1. pylint 1.3.1 reports no lint errors. PWD/pylintrc app/codein app/melange app/soc app/summerofcode tests 2&1 | head \ Module codein.callback W: 17, 0: import missing (no-absolute-import) W: 18, 0: import missing (no-absolute-import) W: 19, 0: import missing (no-absolute-import) W: 20, 0: import missing (no-absolute-import) \ Module W: 17, 0: import missing (no-absolute-import) W: 18, 0: import missing (no-absolute-import) W: 20, 0: import missing (no-absolute-import) W: 21, 0: import missing (no-absolute-import) PWD/pylintrc app/codein app/melange app/soc app/summerofcode tests 2&1 | head \ Module main E: 46, 2: print statement used (print-statement) E: 47, 2: print statement used (print-statement) E: 48, 2: print statement used (print-statement) E: 49, 2: print statement used (print-statement) E: 50, 2: print statement used (print-statement) \ Module tests.test_utils E:658, 8: print statement used (print-statement) E:662,10: print statement used (print-statement) E:667, 8: print statement used (print-statement) This is with the current head of the pylint repository. (1889: tip) The source code being linted is Bitbucket:


Short circuit if all checks disabled Running "pylint --disable=all" takes more than 3s! Running pylint without "disable=all" on a little project (150-lines telegram bot) takes more than 8s. It is non-usable. No response It is unacceptable that even on an empty file pylint runs for at least 3 seconds. I use the VS Code extension in my project, which for a small example with a 150-line project reacts to changes in 8 (!) seconds. This is literally impossible to use. Windows 11 No response


enable/disable options in rcfile should not depend on the order with which they are specified Hello, i'm running ; i kinda understand why behaves differently than (where the first command enables everything and then disable , while the second command the disable option is overwritten by ) but i dont think it should be the same in the rcfile: the section is (by default) before the section, so if i want to have the same effect of command1 i need to swap the sections around. on the cli i can specify multiple enable/disable options, but that's not allowed in the rcfile, so the current result is extremely counter-intuitive; and rcfile with is clear what result the user wants: to enable all checks except for some, and i shouldnt need to move config options around. can you please get that fixed? Thanks!


bad-names-rgxs mangles regular expressions with commas Since pylint splits on commas in this option, instead of taking a list of strings, if there are any commas in the regular expression, the result is mangled before being parsed. The config below demonstrates this clearly by causing pylint to crash immediately. I would expect any valid regular expression to be expressible in this option. If not directly, adding some way to escape commas so that this issue can be worked around. Pop! OS 22.04 No response


Exporting to JSON does not honor score option <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --pylint on some random Python file or module: As you can see this outputs some warnings/scoring: run the same command but with to export it to JSON: The output doesn't contain the scores now anymore: execute it with again but also supply the option: The score is not outputted when exporting to JSON, not even when is activated. The score is to the JSON, at least when is activated.


caplog.getrecords and conflict gets decoupled from actual caplog records when is called. As a result, after is called, is frozen: it does not get cleared, nor does it get new records. During test set up it is as , but the latter gets in , which diverges the two objects. Arch Linux, Python 3.9.10:


unittest.TestCase.tearDown executed for classes marked with when running --pdb <!-- Thanks for submitting an issue! Quick check-list while reporting bugs: --- [x] a detailed description of the bug or problem you are having [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible Running will run the of classes that are decorated with on the class level. Identical to , but with the on the class level rather than on the function level. Minimal test (adapted from ), : Some versions (full below): Test is properly skipped normally: but when running with , the teardown seems executed: Full versions:


Get rid of atomicwrites (unmaintained) PyPI has started enforcing 2-factor-auth for maintainers of various popular packages: For context, here is the mail I got: project. Unfortunately, this has caused the maintainer of to go on what I can only describe as a rampage, first deleting the project from PyPI (and then finding out it's not possible to restore it): then simply declare the project as unmaintained outright: No matter what the outcome of this will be, IMHO, given those actions I do not feel comfortable with trusting this dependency for something as popular as pytest. The library itself , and we only use it on Windows. It's MIT-licensed. Should we just copy the parts we need into pytest instead?


Old-style hookimpl warning has no location information The old-style hookimpl deprecation warning from has no way to see where the culprit is implemented. I'm now getting: with no easy way to figure out what the problem is. I have 12 plugins installed, all of which might have a , and I'd rather not have to find out manually which one is the culprit. The error message should show either the plugin that's coming from, or at least the Python file it's in. Old-style hookimpl warning has no location information The old-style hookimpl deprecation warning from has no way to see where the culprit is implemented. I'm now getting: with no easy way to figure out what the problem is. I have 12 plugins installed, all of which might have a , and I'd rather not have to find out manually which one is the culprit. The error message should show either the plugin that's coming from, or at least the Python file it's in.


Consider MRO when obtaining marks for classes When using pytest markers in two baseclasses and , inheriting from both of those baseclasses will lose the markers of one of those classes. This behavior is present in pytest 3-6, and I think it may as well have been intended. I am still filing it as a bug because I am not sure if this edge case was ever explicitly considered. If it is widely understood that all markers are part of a single attribute, I guess you could say that this is just expected behavior as per MRO. However, I'd argue that it would be more intuitive to attempt to merge marker values into one, possibly deduplicating marker names by MRO. I'd expect and to be markers for , but this only actually is the case with this metaclass. Please note that the repro case is Python 2/3 compatible excluding how metaclasses are to (this needs to be taken care of to repro this issue on pytest 6) Consider MRO when obtaining marks for classes When using pytest markers in two baseclasses and , inheriting from both of those baseclasses will lose the markers of one of those classes. This behavior is present in pytest 3-6, and I think it may as well have been intended. I am still filing it as a bug because I am not sure if this edge case was ever explicitly considered. If it is widely understood that all markers are part of a single attribute, I guess you could say that this is just expected behavior as per MRO. However, I'd argue that it would be more intuitive to attempt to merge marker values into one, possibly deduplicating marker names by MRO. I'd expect and to be markers for , but this only actually is the case with this metaclass. Please note that the repro case is Python 2/3 compatible excluding how metaclasses are to (this needs to be taken care of to repro this issue on pytest 6) Fix missing marks when inheritance from multiple classes <!-- Thanks for submitting a PR, your contribution is really appreciated! Here is a quick checklist that should be present in PRs. [] Include documentation when adding new features. [ ] Include new tests or update existing tests when applicable. [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.


If this change fixes an issue, please: for more information. Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please: for details. Write sentences in the past or present tense, examples: Improved verbose diff output with sequences. Terminal summary statistics now use multiple colors. Also make sure to end the sentence with a . [x] Add yourself to in alphabetical order. --


RFE: allow to selectively disable loggers from command-line A common debugging strategy is to study the logs. But sometimes the program issues excessive logging messages, necessitating the selective disabling of babbler loggers. This SO captures the crux & solution of this Request For Enhancement: Although the proposed SO solution of a new option works perfectly fine, it is annoying to have to patch every new project, plus, it does not support auto-completion e.g. on bash. Would it make sense to include such capability into core code? In any case, I'm suggesting the new option to be called, to fit with the existing option names starting with `.


Keep temporary directory for failing tests only Currently pytest will keep the last 3 root temporary directories for each testing session, for all tests. This proposal is to change this behavior to only keep the directories for failed tests, instead of for all tests. This would save significant disk space for test suites which manipulate large amounts of data in the temporary directories. The behavior of keeping the last 3 root temporary directories would be kept unchanged. From: EDIT: after some discussion, the full proposal is defined in


Short test summary doesn't show the test name when skipping from a fixture I'm using Pytest 7.0.1on Ubuntu 18.04 with Python 3.6.9. Consider a test: and a fixture defined in that will skip a test based on some conditional check. Then running the test with pytest shows something like: The summary shows that some test was skipped but there's no indication which test was skipped. Instead, it should show the test name rather than the location in the fixture where the was called from. If there are multiple tests that are skipped from various locations, matching a test with its skip condition becomes impossible. There are some similar issues in , , which may be related.


Inconsistent support for staticmethod/classmethod Pytest discovery & running of staticmethods/classmethods is inconsistent. Here's an example: The most worrysome cases are and , which are not discovered by pytest. I think that there should at least be a warning or error to alert the user that their test code cannot be run. <details<summaryHere's the full output from running : </summary</detailspython v3.9.15, pytest v7.2.0, ubuntu 20.04 <details<summaryOutput of : </summary</details


when is dict containing zero value results in ZeroDivisionError Pytest behaves differently when comparing dictionaries containing zero values compared to lists containing same values pytest==7.2.0 Ubuntu 22.04


Walrus operator causes different behavior in PyTest. I am currently testing the following function but find that the function passes the test in the normal Python terminal but fails in a PyTest run. The walrus operator is relatively new and not many people use it. I think that there may be an inconsistency in the execution environment.


Use of causes on Python 3.12 alpha The current Python 3.12 alpha has made a change to , deprecating the parameter and replacing it with an parameter. Something in Pytest's temp path fixtures calls an function which calls with the parameter. When warnings are treated as errors, this makes pytest fail any test using temp paths. Add to cpython 3.12 (alpha) has an API to test results (python/cpython). This would not be an issue, except it was designed to trigger a warning if the test result doesn't have such a method (hence e.g. python/cpython). This means when using pytest as runner for unittest tests and running an error is triggered as pytest's test result (which seems to be ) does not support this protocol. Now obviously this should be non-blocking, as hopefully nobody is running CI which blocks on testing 3.12, but there you go, I figure an early forewarning can't hurt.


Pytest trying to check if custom argument is a file crashes due to filename being too long I have a custom flag defined in , and when I try to assign it to a value that is too long pytest crashes before ever getting to my code. This reproduces even if the flag isn't defined, and even if the current working directory is . Failing example: If I reduce the length of the flag, I get the expected behavior for my project, and this different and expected error from my pytest MVP: I did a little digging into my version of pytest (7.0.0) to make sure I wasn't doing something wrong, but it looks like there is a blind call to with a path constructed from the argument in : It seems to me like there should be a try or something here, since in cases like mine the argument may not be a file at all, and that can cause OS level errors. Operating System: Ubuntu 20.04 LTS [x] a detailed description of the bug or problem you are having [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible Pytest trying to check if custom argument is a file crashes due to filename being too long I have a custom flag defined in , and when I try to assign it to a value that is too long pytest crashes before ever getting to my code. This reproduces even if the flag isn't defined, and even if the current working directory is . Failing example: If I reduce the length of the flag, I get the expected behavior for my project, and this different and expected error from my pytest MVP: I did a little digging into my version of pytest (7.0.0) to make sure I wasn't doing something wrong, but it looks like there is a blind call to with a path constructed from the argument in : It seems to me like there should be a try or something here, since in cases like mine the argument may not be a file at all, and that can cause OS level errors. Operating System: Ubuntu 20.04 LTS [x] a detailed description of the bug or problem you are having [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible


UnboundLocalError: cannot access local variable 'x' where it is not associated with a value There seems to be a regression in pytest version when a walrus operator is used in an assert line. Code: Fails the test with error: in pytest version , whereas with pytest version it passes successfully. My Python version is . Looks like it has to do with PR .


Issue warning/error if 'testpaths' does not match any files/folders We should issue a warning (or even an error?) if does not match any files or folders. I think an error is reasonable, even if it might break some incorrectly-configured suite out there. Originally posted by in This is not really a bug, but an intended (albeit questionable) behavior: The values of are actually globs, so globbing for in the root yields nothing. Given it finds nothing, pytest will behave as if called from the command-line without any parameters, which makes it search recursively from the current directory looking for to collect. If you create the directory, then pytest will correctly search in that directory only. I agree those 2 facts are surprising: fact that is a glob. This is but easy to overlook, probably we should add a glob to the example there. silently not finding anything, and then proceeding as usual. I don't think we can do anything more for 1, but for 2 seems like we should at least emit a warning if is defined but does not match anything.


support sub-second granularity/precision in (strftime ) tl;dr pytest processing strftime specifier from arguments would allow me to accurately merge log messages from disparate sub-systems Tests I run have pytest log messages that print at the second granularity for the datetimestamp, e.g. . At the same time, other log file messages not generated by pytest print sub-second datetimestamps, e.g. . When reviewing the various logs, there are many message from other system components that are printing many log messages per second. Because pytest log messages are lacking sub-second precision, I am unable to align pytest log messages within other system log messages. For example, the system-under-test generates a log file like: and the driver of tests, pytest, generates a log file like: The pytest log messages cannot be precisely ordered among the other log messages that occurred during the datetime second , there were many things that occurred in the other system components within that second. Given the following pytest code To add sub-second granularity, it seems sensible to add within the but then I see the confusing output of pytest logging is ignoring the strftime specifier! I want pytest log messages to print sub-second granularity, e.g. process strftime within settings. <!-- A clear and concise description of what you want to happen. --Support strftime specifier in the various settings for date-format, e.g. , , . <!-- Provide examples of real-world use cases that this would enable and how it solves the problem described above. --In my complex testing system, this means all log messages would be printed to millisecond precision. This allows engineers investigating issues to more accurately merge disparate testing system logs by their natural ordering mechanism of a datetimestamp. <!-- Have you tried to workaround the problem using a pytest plugin or other tools? Or a different approach to solving this issue? Please elaborate here. --I can set the format to include . However, it's a little confusing to have to manipulate log datetimestamps by two different mechanisms, and . On the command-line run: This prints datetimestamps with millisecond precision <br /It is more intuitive for pytest to process the Python strftime specifier within all options.


Pytest 7.3.2 changes in behaviour regarding and In , we have two test suites - the unit tests at and the integration test suite at . Both and are listed in testpaths- We then run either or using /. Each / dir contains a file, which adds some options using . One option that is common to both test suites is . Before 7.3.2, this setup seemed to work, we could run both unit tests and integration tests without issue. But on 7.3.2 (perhaps since ?) we get the following error: Is this an issue in our configuration, or a bug? Should we no longer use testpaths to list all the test suites? <details<summarypip list output</summary</detailsXref


Rewrite fails when first expression of file is a number and mistaken as docstring <!-- Thanks for submitting an issue! Quick check-list while reporting bugs: --- [x] a detailed description of the bug or problem you are having [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible use report an error it seems a error in collecting testcase


Module imported twice under import-mode=importlib In pmxbot/pmxbot I'm attempting to switch pmxbot off of pkgresources style namespace packaging to PEP 420 namespace packages. To do so, I've needed to switch to for the and re-organize the tests to avoid import errors on the tests. Yet even after working around these issues, the tests are failing when the effect of doesn't seem to have had any effect. Investigating deeper, I see that initializer is executed and performs its actions (setting a class variable ), but when that happens, there are two different versions of present, one in and another found in : I haven't yet made a minimal reproducer, but I wanted to first capture this condition.


() improperly matches warning When trying to pop a specific warning from a WarningsRecorder instance, the wrong warning is returned. I believe the issue is that pop uses issubclass I believe the correct comparison should be: Here is a minimum working example that triggers the buggy behavior: The test output is pytest 7.2.1 on archlinux. virtual environment is a clean conda environment with only python and pytest (and their dependencies installed from conda-forge). If this is indeed a bug, I'm happy to open a PR with my proposed solution. () improperly matches warning When trying to pop a specific warning from a WarningsRecorder instance, the wrong warning is returned. I believe the issue is that pop uses issubclass I believe the correct comparison should be: Here is a minimum working example that triggers the buggy behavior: The test output is pytest 7.2.1 on archlinux. virtual environment is a clean conda environment with only python and pytest (and their dependencies installed from conda-forge). If this is indeed a bug, I'm happy to open a PR with my proposed solution.


fails with When using to test float and one of the objects in the statement contain I see the following TypeError: TypeError: unsupported operand type(s) for -: 'float' and 'NoneType'. macOS 12.6.3 python 3.10.4 pytest 7.2.1


fixtures: show test as skip location if skipped from an xunit setup function PR made it so that if a fixture calls , the skip location is shown as the test function, not the fixture. But it excluded xunit setup fixtures from this. I suspect this was done to make a pre-existing test pass, however I think that the same reason for fixtures applies to xunit fixtures just as well, so we shouldn't exclude it. Would also remove a string-hack that was used to implement this exclusion...


Unroll the iterable for all/any calls to get better reports Sometime I need to assert some predicate on all of an iterable, and for that the builtin functions / are great - but the failure messages aren't useful at all! For example - the same test written in three ways: A generator expression A list comprehension A for loop The only one that gives a meaningful report is the for loop - but it's way more wordy, and asserts don't translate to a for loop nicely (I'll have to write a or a helper function - yuck) I propose the assertion re-writer "unrolls" the iterator to the third form, and then uses the already existing reports. [x] Include a detailed description of the bug or suggestion [x] of the virtual environment you are using [x] pytest and operating system versions [x] Minimal example if possible


Invalid XML schema for <propertiestags in JUnit reports The problem: JUnit breaks when it reads an XML generated by pytest if plugins make use of . This behavior happens with newer versions of hypothesis (). In fact, as already mentioned in , is adding inside which seems to be wrong (it should be inside ). See: . It happens with all junit families. Reproducing: I was trying to create a PR to fix this, but when I saw code I realized that what is needed to do could have more implications that I though. I think that nobody uses this feature with JUnit (as it breaks) and removing that is something to think about.


Display fixture scope with It would be useful to show fixture scopes with ; currently the only way to learn the scope of a fixture is look at the docs (when that is documented) or at the source code.


Improve default logging format Currently it is: I think (module name) would be very useful here, instead of just the base filename. (It might also be good to have the relative path there (maybe at the end), but it is usually still very long (but e.g. could be substituted therein)) Currently it would look like this: Using instead:


does not correctly hide fixtures of the same name (it misses its dependencies) From This works: the parameter in the test correctly hides the fixture so the unique pytest node has id (instead of there being two nodes because of the fixture). However if the fixture that is hidden by the parameter depends on another fixture, that other fixture is mistakenly kept in the fixtures closure, even if it is not needed anymore. Therefore the test fails:


pytest.capture.EncodedFile mode should not include (binary) <!-- Thanks for submitting an issue! Here's a quick checklist for what to provide: --- [x] a detailed description of the bug or suggestion Exception when youtube-dl logs to pytest captured output. Youtube-dl looks for in to decide whether to writes or . incorrectly advertises , the mode of the underlying stream. Its method raises an exception when passed . [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible


Invalid unicode in with Jython With: Jython 2.7.1 and pytest 4.5.0, I'm having the error below: The error happens because has the following unicode literal: . I'm not sure about the proper fix... in that function seems to be doing a and later does , so, I'm not even sure if we're dealing with or there on Python 2 because it's mixed (my guess is bytes, but I'm not really sure). If it's bytes, doing it an actual literal instead of a literal -- i.e.: remove the -- does make Jython happy (would that be an appropriate solution?)


Version 4.6.0 skips tests without apparent reason Since version 4.6.0 pytest skips tests without apparent reason: https://travis- [x] pytest and operating system versions: pytest 4.6.0 on Ubuntu Xenial @ Travis CI [x] minimal example if possible: link above I can't understand why it happens, so I had to immediately rollback to 4.5.0 and fix this version in dev dependencies.


KeyError: 'wrapped' with from xxx import yyy [X] Include a detailed description of the bug or suggestion [ ] of the virtual environment you are using [X] pytest and operating system versions [X] Minimal example if possible I implemented and for automated testing. When I run the , I get a when I perform a . In the example I included, I do not get the KeyError if I comment out the line: ... from sty import fg ... No virtual environment used. Python: 3.6.5 & 3.7.1 PyTest: 4.4.0 OS: MacOS Mojave 10.14.4 I created a very simple . If I clone the repo and call either or in the root directory, I get the error. My apologies if this bug is obvious or I am doing something wrong.


str() on the context variable doesn't behave same as normal exception catch Pytest 4.6.2, macOS 10.14.5 prints But prints In order to get the full error message, one must do , which is documented, but this is a different interaction. Any chance the behavior could be changed to eliminate this gotcha? Pip list gives


Confusing assertion rewriting message with byte strings The comparison with assertion rewriting for byte strings is confusing: 52 is the ASCII ordinal of "4" here. It became clear to me when using another example: Not sure what should/could be done here.


xunit2 format does not contain a root <testsuitestag junitfamily=xunit2 option still generates an old xml format tests report. junitfamily=xunit2 creates the .xml file in the legacy formatOS: MAC OS Mojave == 10.14.5 Sample output from the tests by running:


pytest stepwise doesn't work with xfail strict failures recommended pytest-runner in setuprequires means packages fail to install often The recommendation to add to means that all users of that package end up with an unnecessary pytest-runner package installed. This is bad because it bypasses pip hashes and


ValueError when collecting tests that patch an array <!-- Thanks for submitting an issue! Here's a quick checklist for what to provide: --I'm trying to run pytest with a test file that contains patch where "new" is an array, for example: from import patch ([-5.5, 3.0])) ... This works fine with pytest 3.1.3, but when using pytest 3.6.0 the following error is received upon collection: Seems like a bug, that was introduced by the following fix: when using like: ([-5.5, 3.0])), p.new is an array and the check: "p.new in sentinels" returns an array of booleans instead of a boolean which causes the ValueError.


Hostname and timestamp properties in generated JUnit XML reports Pytest enables generating JUnit XML reports of the tests. However, there are some properties missing, specifically and from the XML element. Is there an option to include them? Example of a pytest XML report: Example of a junit XML report:


exception serialization should include chained exceptions given some simple tests: when run without xdist it displays whole exception trace nicely : but when run with xdist (), it just displays the last one: my setup:


Lexer "python3" in --pastebin feature causes HTTP errors The option currently submits the output of to using : For some , this will raise a "HTTP Error 400: Bad Request". As an example: with the attached . This is the underlying cause for the problems mentioned in . The call goes through fine if is changed from to . This would seem like the right thing to do in any case: the console output of a run that is being uploaded is not Python code, but arbitrary text.


2 ImportError while loading conftest (windows import folder casing issues) 1 works fine. after upgrade to 5.1.2, the path was converted to lower case


Provide an alternative to --result-log After discussion in , mentions he would like to provide a replacement to in the core before removing it (). This potentially is an easy contribution given that we have the plugin already which can be used as a starting point. I would like for us to discuss how that "log file" will look like in this issue. I understand the rationale is to provide a line-based log file, which can be parsed using standard tools. I have used a log file in the past where each line was a JSON object, something like: pytest would then write each line to the file during , like does now. I suppose we also want to add an option to replay the tests in a log file, so users can reproduce a previous run that was saved to a log? you also mentioned that would not be an appropriate replacement, can you elaborate on why?