Limit number of data variables shown in repr <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: xarray feels very unresponsive when using datasets with 2000 data variables because it has to print all the 2000 variables everytime you print something to console. What you expected to happen: xarray should limit the number of variables printed to console. Maximum maybe 25? Same idea probably apply to dimensions, coordinates and attributes as well, pandas only shows 2 for reference, the first and last variables. Minimal Complete Verifiable Example: Anything else we need to know?: Out of scope brainstorming: Though printing 2000 variables is probably madness for most people it is kind of nice to show all variables because you sometimes want to know what happened to a few other variables as well. Is there already an easy and fast way to create subgroup of the dataset, so we don' have to rely on the dataset printing everything to the console everytime? Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryxr.showversions() INSTALLED VERSIONS commit: None python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 libhdf5: 1.10.4 libnetcdf: None xarray: 0.16.2 pandas: 1.1.5 numpy: 1.17.5 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 51.0.0.post20201207 pip: 20.3.3 conda: 4.9.2 pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.0 </details


CFTime offsets missing for milli- and micro-seconds <!-- A short summary of the issue, if appropriate --The smallest cftime offset defined in is "second" (S), but the precision of cftime objects goes down to the millisecond (L) and microsecond (U). They should be easily . PR adds a that supports the two, but they are currently untested as cannot generate an index. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --An error gets raised : . <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.6.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: frCA.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.4 xarray: 0.15.2.dev9+g6378a711. pandas: 1.0.3 numpy: 1.18.4 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.1.1.2 nctime_axis: 1.2.0 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None pint: 0.11 setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.2 </details


DataArray transpose inconsistent with Dataset Ellipsis usage This works: This doesn't (subset air): The error message is a bit inaccurate too since I do have Ellipsis included; might be related to two calls of:


Decodecf fails when scalefactor is a length-1 list Some datasets I work with have and encoded as length-1 lists. The following code worked as of Xarray 0.16.1 In 0.16.2 (just released) and current master, it fails with this error I'm very confused, because this feels quite similar to , and I thought it was resolved . However, the behavior is different with . That works fine--no error. How might I end up with a dataset with as a python list? It happens when I open a netcdf file using the engine (documented by in ) and then write it to zarr. The numpy array gets encoded as a list in the zarr json metadata. This problem would go away if we could resolve the discrepancies between the two engines' treatment of scalar attributes.


dropsel indices in dimension that doesn't have coordinates? <!-- Please do a quick search of existing issues to make sure that this has not been asked before. --Is your feature request related to a problem? Please describe. I am trying to drop particular indices from a dimension that doesn't have coordinates. Following: , but leaving out the coordinate labels: gives me an error. Describe the solution you'd like I would think should exist and work in analogy to as does to . Describe alternatives you've considered As far as I know, I could either create coordinates especially to in order to drop, or rebuild a new dataset. Both are not congenial. (I'd be grateful to know if there is actually a straightforward way to do this I've overlooked.


Option for combineattrs with conflicting values silently dropped currently supports four options for merging : It would be nice to have an option to combine attrs from all objects like "noconflicts", but that drops attributes with conflicting values rather than raising an error. We might call this or . This is similar to how xarray currently handles conflicting values for and would be more suitable to consider for the default behavior of and other functions/methods that merge coordinates (e.g., applyufunc, concat, where, binary arithmetic). cc


jupyter repr caching deleted netcdf file What happened: Testing xarray data storage in a jupyter notebook with varying data sizes and storing to a netcdf, i noticed that opendataset/array (both show this behaviour) continue to return data from the first testing run, ignoring the fact that each run deletes the previously created netcdf file. This only happens once the was used to display the xarray object. But once in error mode, even the previously fine printed objects are then showing the wrong data. This was hard to track down as it depends on the precise sequence in jupyter. What you expected to happen: when i use , the resulting object should reflect reality on disk. Minimal Complete Verifiable Example: When executed in a cell with print for display, all is fine: but as soon as one cell used the jupyter repr: all future file reads, even after executing the test function again and even using and not , show the data from the last repr use. Anything else we need to know?: Here's a notebook showing the issue: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jun 1 2020, 18:57:50) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 5.4.0-40-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.21.0 distributed: 2.21.0 matplotlib: 3.3.0 cartopy: 0.18.0 seaborn: 0.10.1 numbagg: None pint: None setuptools: 49.2.0.post20200712 pip: 20.1.1 conda: installed pytest: 6.0.0rc1 IPython: 7.16.1 sphinx: 3.1.2 </details


Sum and prod with mincount forces evaluation If I use the method on a lazy array with then evaluation is forced. If there is some limitation of the implementation which means it cannot be to the computation graph for lazy evaluation then this should be mentioned in the docs. Minimal Complete Verifiable Example: What happened: ` What you expected to happen: No output or exceptions, as the result1 and result2 arrays are not printed or saved. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.1 (default, Feb 6 2021, 06:49:13) [GCC 10.2.0] python-bits: 64 OS: Linux OS-release: 5.10.15-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: enNZ.UTF-8 LOCALE: enNZ.UTF-8 libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.16.2 pandas: 1.2.1 numpy: 1.20.0 scipy: 1.6.0 netCDF4: 1.5.5.1 pydap: None h5netcdf: 0.9.0 h5py: 3.1.0 Nio: None zarr: None cftime: 1.4.1 nctime_axis: None PseudoNetCDF: None rasterio: 1.2.0 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 53.0.0 pip: 20.3.1 conda: None pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.3 </details


DataArrayCoarsen does not have a map or reduce function I'm trying to count unique samples when resampling to a square kilometre from a 5x5m input grid. I'd like to be able to apply the function with to give me a new dimension with the original integer values and their counts. In order to resample along spatial dimensions I assume I need to use , unfortunately the object does not yet implement either a or function for applying an arbitrary function when coarsening. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --outputs; N.B. has both and while has . Would it make sense for all three to have the same interface? <detailsINSTALLED VERSIONS commit: None python: 3.7.4 (default, Sep 7 2019, 18:27:02) [Clang 10.0.1 (clang-1001.0.46.4)] python-bits: 64 OS: Darwin OS-release: 19.3.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enAU.UTF-8 LOCALE: enAU.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.0 pandas: 1.0.0 numpy: 1.18.1 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: None numbagg: None setuptools: 40.8.0 pip: 19.0.3 conda: None pytest: 5.3.5 IPython: 7.11.1 sphinx: None </details


Calling () drops coordinates This is a similar issue to bug . <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --Output (with extra stuff snipped for brevity): It's also worth mentioning that the data arrays contained in the dataset also loose their coordinates during this operation. I.E: <detailsINSTALLED VERSIONS commit: None python: 3.7.4 (tags/v3.7.4:, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 11, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.2 xarray: 0.14.0 pandas: 0.25.2 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.7.0 distributed: None matplotlib: 3.1.1 cartopy: None seaborn: None numbagg: None setuptools: 40.8.0 pip: 19.3 conda: None pytest: None IPython: 7.8.0 sphinx: None None </details


Handling of signed bytes from OPeNDAP via pydap netCDF3 only knows signed bytes, but there's of adding an attribute to the variable to be able to store unsigned bytes non the less. This convention is handled by xarray. OPeNDAP only knows unsigned bytes, but there's which is used by the thredds server and the netCDF-c library of adding an attribute to the variable to be able to store signed bytes non the less. This hack is not handled by xarray, but maybe should be handled symmetrically at the same place (i.e. ). As descibed in the "hack", netCDF-c handles this internally, but pydap doesn't. This is why the variant returns (correctly according to the hack) negative values and the variant doesn't. However, as returns a warning at exactly the location referenced above, I think that this is the place where it should be fixed. If you agree, I could prepare a PR to implement the fix. Handling of signed bytes from OPeNDAP via pydap netCDF3 only knows signed bytes, but there's of adding an attribute to the variable to be able to store unsigned bytes non the less. This convention is handled by xarray. OPeNDAP only knows unsigned bytes, but there's which is used by the thredds server and the netCDF-c library of adding an attribute to the variable to be able to store signed bytes non the less. This hack is not handled by xarray, but maybe should be handled symmetrically at the same place (i.e. ). As descibed in the "hack", netCDF-c handles this internally, but pydap doesn't. This is why the variant returns (correctly according to the hack) negative values and the variant doesn't. However, as returns a warning at exactly the location referenced above, I think that this is the place where it should be fixed. If you agree, I could prepare a PR to implement the fix.


Date missing in datetime accessor What happened: I wonder if there is a reason, why there is no attribute in the datetime accessor. What you expected to happen: As the attribute is supported I would expect the same for the attribute Minimal Complete Verifiable Example: Suggestion: A simple addition of in should do the trick. Happy to do a PR. Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.6 (default, Oct 7 2020, 19:08:05) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 2.6.32-754.33.1.el6.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.17.0 pandas: 1.2.1 numpy: 1.20.0 scipy: 1.6.0 netCDF4: 1.5.5.1 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.6.1 cftime: 1.4.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.02.0 distributed: 2021.02.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: 0.11.1 numbagg: None pint: 0.16.1 setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: None IPython: 7.20.0 sphinx: None </details


Simplify adding custom backends <!-- Please do a quick search of existing issues to make sure that this has not been asked before. --Is your feature request related to a problem? Please describe. I've been working on opening custom hdf formats in xarray, reading up on the apiv2 it is currently only possible to declare a new external plugin in but that doesn't seem easy or intuitive to me. Describe the solution you'd like Why can't we simply be allowed to add functions to the engine parameter? Example: This seems like a small function change to me from my initial quick look because there's mainly a bunch of string checks in the normal case until we get to the registered backend functions, if we send in a function instead in the engine-parameter we can just bypass those checks.


Trailing whitespace in DatasetGroupBy text representation When displaying a DatasetGroupBy in an interactive Python session, the first line of output contains a trailing whitespace. The first example in the documentation demonstrate this: There is a trailing whitespace in the first line of output which is "DatasetGroupBy, grouped over 'letters' ". This can be seen more clearly by converting the object to a string (note the whitespace before ): While this isn't a problem in itself, it causes an issue for us because we use flake8 in continuous integration to verify that our code is correctly formatted and we also have doctests that rely on DatasetGroupBy textual representation. Flake8 reports a violation on the trailing whitespaces in our docstrings. If we remove the trailing whitespaces, our doctests fail because the expected output doesn't match the actual output. So we have conflicting constraints coming from our tools which both seem reasonable. Trailing whitespaces are forbidden by flake8 because, among other reasons, they lead to noisy git diffs. Doctest want the expected output to be exactly the same as the actual output and considers a trailing whitespace to be a significant difference. We could configure flake8 to ignore this particular violation for the files in which we have these doctests, but this may cause other trailing whitespaces to creep in our code, which we don't want. Unfortunately it's not possible to just add comments to get flake8 to ignore the violation only for specific lines because that creates a difference between expected and actual output from doctest point of view. Flake8 doesn't allow to disable checks for blocks of code either. Is there a reason for having this trailing whitespace in DatasetGroupBy representation? Whould it be OK to remove it? If so please let me know and I can make a pull request.


opendataset uses cftime, not datetime64, when calendar attribute is "Gregorian" <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: I used to open a NetCDF file whose coordinate had the attribute set to . All dates were within the Timestamp-valid range. The resulting dataset represented the co-ordinate as a . What you expected to happen: I expected the dataset to represent the co-ordinate as a , as documented and . Minimal Complete Verifiable Example: Anything else we need to know?: The says: In practice, we are getting some s even for times which are interpretable and representable as s. In particular, we have some NetCDF files in which the variable has a attribute with a value of (with a capital ‘G’). CF conventions : However, xarray regards as a non-standard calendar and falls back to . If (as in the example) is to , the times are read as s. Suggested fix: in , change ‘’ to ‘’. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.2 (default, Feb 21 2021, 05:02:46) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.8.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.17.1.dev39+g45b4436b pandas: 1.2.3 numpy: 1.20.2 scipy: None netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: None IPython: None sphinx: None <!-- Paste the output here xr.show_versions() here --</details


bfill behavior dask arrays with small chunk size bfill methods seems to miss nans when dask array chunk size is small. Resulting array still has nan present (see 'small chunk' section of code) absence of nans INSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.15.0-43-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: en_CA.UTF-8 xarray: 0.11.0 pandas: 0.23.4 numpy: 1.15.4 scipy: None netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: 1.0.0 distributed: 1.25.2 matplotlib: None cartopy: None seaborn: None setuptools: 40.6.3 pip: 18.1 conda: None pytest: None IPython: None sphinx: None


Should weighted operations raise an error when dimensions don't exist? <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Weighted operations don't raise an error when the dimensions passed don't exist. What you expected to happen: This is not really a bug, but I find it a bit confusing because it's not consistent with the same "unweighted" operation. Minimal Complete Verifiable Example: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.9.4 (default, May 10 2021, 22:13:33) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1062.18.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: ('enGB', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.18.1.dev30+g2578fc3 pandas: 1.2.4 numpy: 1.20.2 scipy: 1.6.3 netCDF4: 1.5.6 pydap: installed h5netcdf: 0.11.0 h5py: 3.2.1 Nio: None zarr: 2.8.1 cftime: 1.4.1 nctimeaxis: 1.2.0 PseudoNetCDF: None rasterio: 1.2.3 cfgrib: 0.9.9.0 iris: None bottleneck: 1.3.2 dask: 2021.05.0 distributed: 2021.05.0 matplotlib: 3.4.2 cartopy: 0.19.0.post1 seaborn: 0.11.1 numbagg: installed pint: None setuptools: 49.6.0.post20210108 pip: 21.1.1 conda: None pytest: None IPython: 7.23.1 sphinx: None </details


Feature request: vector cross product xarray currently has the function for calculating arbitrary dot products which is indeed very handy. Sometimes, especially for physical applications I also need a vector cross product. I' wondering whether you would be interested in having as a wrapper of I currently use the following implementation: Do you want such a function (and possibly associated methods) in the namespace, or should it be in some other package? I didn't find a package which would be a good fit as this is close to core numpy functionality and isn't as domain specific as some geo packages. I'm not aware of some "xrphysics" package. I could make a PR if you'd want to have it in directly. <detailsINSTALLED VERSIONS commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.9.0-9-amd64 machine: x8664 processor: byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.3 pandas: 0.24.2 numpy: 1.16.4 scipy: 1.3.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.7.4 h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: 2.1.0 distributed: 2.1.0 matplotlib: 3.1.0 cartopy: None seaborn: 0.9.0 numbagg: None setuptools: 41.0.1 pip: 19.1.1 conda: 4.7.11 pytest: 5.0.1 IPython: 7.6.1 sphinx: 2.1.2 </details


Increase default This must have been introduced into somewhere around version 0.17. First of all this limit breaks backwards compatibility in the output format with or something similar on the console. Second, the default of 12 is much too low imo and makes no sense, in particular since terminals usually have a scrollback buffer and notebook cells can be made scrollable. I use frequently to check that all variables made it into the data set correctly, which is meaningless when lines are skipped with this default limit. And it broke my doctests that I wrote to do exactly that (thanks for that btw.). So if not removed, could the default be at least increased to a sensible number like 100 or 1000 or ? (I'd personally prefer much higher or even no limit, but I guess that is not an option.) Cheers.


Limit number of displayed dimensions in repr <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Dimension doesn't seem to be limited when there are too many of them. See example below. This slows down the repr significantly and is quite unreadable to me. What you expected to happen: To be limited so that it aligns with whatever the maximum line length is for variables. It's also fine if it continues on a couple of rows below in similar fashion to variables. Minimal Complete Verifiable Example: This is probably a bit of an edge case. My real datasets usually have around 12 "dimensions" and coords, +2000 variables, 50 attrs. Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.8 (default, Feb 20 2021, 15:50:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 byteorder: little LCALL: None LANG: en libhdf5: 1.10.6 libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: 3.2.1 Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.05.0 distributed: 2021.05.0 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 49.6.0.post20210108 pip: 21.1.2 conda: 4.10.1 pytest: 6.2.4 IPython: 7.24.1 sphinx: 4.0.2 </details


Complex LaTeX expressions in s aren't rendered correctly when plotting <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: When I try to give a variable a that's a complex latex expression and then plot that variable the expression doesn't get rendered by latex What you expected to happen: I expected the name to get rendered by latex Minimal Complete Verifiable Example: In the example below I'm plotting a variable with a complex via xarray and then plotting it again (in a separate figure) using only matplotlib and manually setting . The matplotlib-only version works fine (right), but the xarray version doesn't render (left). ! Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.2 (default, Mar 3 2021, 20:02:32) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.10.53-1-MANJARO machine: x8664 processor: byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.6.1 xarray: 0.17.0 pandas: 1.2.3 numpy: 1.19.2 scipy: 1.5.3 netCDF4: 1.5.6 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.04.0 distributed: 2021.04.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: None numbagg: None pint: 0.17 setuptools: 52.0.0.post20210125 pip: 21.0.1 conda: None pytest: None IPython: 7.22.0 sphinx: None </details


Dask error on What happened: When I use on two Dataarrays I get a error. Notice that dask is not installed in my environement. What you expected to happen: Obtain the correlation values without dask interfering (as it should be optional in my understanding) Minimal Complete Verifiable Example: Results in: Environment:


Dataset groupby returning DataArray broken in some cases Got a TypeError when resampling a dataset along a dimension, mapping a function to each group. The function returns a DataArray. Failed with : This worked before the merging of . A DataArray was returned as expected. In the docstring of it is not made clear that the passed function should return a dataset, but the opposite is also not said. This worked before and I think the issues comes from , which introduced different signatures for (which is called in my case) and (which is expected by the new ). If the function passed to should only return s then I believe a more explicit error is needed, as well as some notice in the docs and a breaking change entry in the changelog. If s should be accepted, then we have a regression here. I may have time to help on this. <detailsINSTALLED VERSIONS commit: None python: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.16.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: ('frCA', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.1.dev16+g3ead17ea pandas: 1.4.0 numpy: 1.20.3 scipy: 1.7.1 netCDF4: 1.5.7 pydap: None h5netcdf: 0.11.0 h5py: 3.4.0 Nio: None zarr: 2.10.0 cftime: 1.5.0 nctime_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.08.0 distributed: 2021.08.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2021.07.0 cupy: None pint: 0.18 sparse: None setuptools: 57.4.0 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 8.0.1 sphinx: 4.1.2 </details


DataArray groupby returning Dataset broken in some cases This is a the reverse problem of , the method seems broken when the mapped function returns a Dataset (which worked before ). No response I guess the same solution as could be used! <detailsINSTALLED VERSIONS commit: None python: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.16.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: ('frCA', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.1.dev16+g3ead17ea pandas: 1.4.0 numpy: 1.20.3 scipy: 1.7.1 netCDF4: 1.5.7 pydap: None h5netcdf: 0.11.0 h5py: 3.4.0 Nio: None zarr: 2.10.0 cftime: 1.5.0 nctime_axis: 1.3.1 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.08.0 distributed: 2021.08.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2021.07.0 cupy: None pint: 0.18 sparse: None setuptools: 57.4.0 pip: 21.2.4 conda: None pytest: 6.2.5 IPython: 8.0.1 sphinx: 4.1.2 </details


Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details


with scalar as second argument fails with keepattrs=True fails with The workaround is to pass No response No response No response No response xarray 2022.3.0


xr.polyval first arg requires name attribute I have some polynomial coefficients and want to evaluate them at some values using . As described in the docstring/docu I created a 1D coordinate DataArray and pass it to but it raises a KeyError (see example). I expected that the polynomial would be evaluated at the given points. No response I assume that the "standard" workflow is to obtain the argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow. It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future. <detailsINSTALLED VERSIONS commit: None python: 3.9.10 (main, Mar 15 2022, 15:56:56) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: None netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 58.1.0 pip: 22.0.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details


xarray improperly decodes times from a NetCDF when it is a uint improperly decodes times from a NetCDF when it is a . The generates a NetCDF file with the right time ('goodtime') and the wrong time ('time') (use ) to be properly decoded (see ). [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.8.13 (default, Mar 25 2022, 06:04:10) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.62.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.8.0 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.7.0 netCDF4: 1.5.7 pydap: None h5netcdf: 1.0.0 h5py: 3.3.0 Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: 0.9.10.1 iris: None bottleneck: 1.3.4 dask: 2022.04.1 distributed: 2022.4.1 matplotlib: 3.5.1 cartopy: None seaborn: 0.11.2 numbagg: None fsspec: 2021.06.1 cupy: None pint: 0.19.1 sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: 4.12.0 pytest: 7.1.1 IPython: 8.2.0 sphinx: 4.5.0 </details


with timedelta64 coordinates produces wrong results I'm not sure if this is a bug or an expected breaking change, but I'm not able to reproduce the results generated by using a timedelta64 coordinate. The results are correct in , whereas they are wrong in the latest unreleased version (, ). Both the stable and latest functions should return the same results. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 or 2022.3.1.dev102+g6bb2b855 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.0 distributed: 2022.5.0 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.2.0 pip: 22.1 conda: None pytest: 7.1.2 IPython: None sphinx: None </details


returns objects with different dimension order I noticed that the dimension order of the object returned by the latest (, unreleased) is different compared to . For example, the following code returns different results. Is this the expected behaviour? If yes, is it worth mentioning this change in what's new/breaking changes? cc:


Accessing chunks on zarr backed xarray seems to load entire array into memory When running the following example it appears the entire dataset is loaded into memory when accessing the attribute: According to accessing the chunks attribute should simply inspect the attribute on the underlying DataArrays. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.9.12 (main, Mar 24 2022, 23:24:38) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.2.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.21.2 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.1 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.04.0 distributed: 2022.4.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.0.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details


"center" kwarg ignored when manually iterating over DataArrayRolling <div type='discussions-op-text'<supOriginally posted by ckingdon95 June 29, 2022</supHello, I am trying to manually iterate over a DataArrayRolling object, as described in the documentation. I am confused why the following two code chunks do not produce the same sequence of values. I would like to be able to manually iterate over a DataArrayRolling object, and still be given center-justified windows. Is there a way to do this? This returns the following values, as expected: Whereas when I do it manually, it is not equivalent: This returns Is this an issue with the window iterator? If it is not an issue, then is there a way for me to get the center-justified windows in the manual iteration? </div


Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details


Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks


RuntimeError when formatting sparse-backed DataArray in f-string On upgrading from xarray 2022.3.0 to 2022.6.0, f-string formatting of sparse-backed DataArray raises an exception. Code does not error, or A breaking change is listed in the section of the docs.[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. Along with the versions below, I have confirmed the error occurs with both sparse 0.12 and sparse 0.13. <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: ('enCA', 'UTF-8') libhdf5: 1.10.7 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: 0.12.0 h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2022.01.0+dfsg distributed: 2022.01.0+ds.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.01.0 cupy: None pint: 0.18 sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 62.1.0 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 7.31.1 sphinx: 4.5.0 </details


Testing DataArray equality using built-in '==' operator leads to mutilated dictionary In previous versions of xarray, testing numerical equivalence of two DataArrays was possible using the built-in operator '==' and without side affects. Now in version 2022.6.0, when one DataArray lacks an attribute that the other DataArray has, the DataArray with the attribute is mutilated during comparison leading to an empty attrs dictionary. DataArray1 == DataArray2 should not have side affects. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.9.13 (tags/v3.9.13:, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: ('EnglishUnited States', '1252') libhdf5: None libnetcdf: None xarray: 2022.6.0 pandas: 1.4.3 numpy: 1.23.1 scipy: 1.9.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.2 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 63.2.0 pip: 22.2.1 conda: None pytest: 7.1.2 IPython: 8.4.0 sphinx: None </details


Cannot open NetCDF file if dimension with time coordinate has length 0 ( when decoding CF datetime) If a data set has a zero-sized coordinate that is a time index, reading fails. A is triggered when xarray tries to decode the array, as shown below:


Alignment of dataset with MultiIndex fails after applying After applying the function to a dataset with a Multiindex, a lot of functions related to indexing are broken. For example, it is not possible to apply to itself anymore. The error is raised in the alignment module. It seems that the function does not find indexes that belong to the same dimension. I expected the alignment to be functional and that these basic functions work. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.5 (main, Jun 14 2022, 07:04:59) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.21.6 scipy: 1.8.1 netCDF4: 1.6.0 pydap: None h5netcdf: None h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.1.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.6.1 distributed: 2022.6.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: 0.13.0 flox: None numpygroupies: None setuptools: 61.2.0 pip: 22.1.2 conda: 4.13.0 pytest: 7.1.2 IPython: 7.33.0 sphinx: 5.0.2 /home/fabian/.miniconda3/lib/python3.10/site- UserWarning: Setuptools is replacing distutils. ("Setuptools is replacing distutils.") </details


can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details


index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details


[Bug]: renamevars to dimension coordinate does not create an index We used to make a coordinate a dimension coordinate (instead of ). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like fail with a strange . I expect one of two things to be true: does not allow setting dimension coordinates (raises Error and tells you to use setindex) checks for this occasion and sets the index correctly No response No response INSTALLED VERSIONS commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None


Indexes.getunique() TypeError with pandas indexes I also just tested the method that you mentioned and maybe noticed a related issue here, which I'm not sure is wanted / expected. Taking the above dataset , accessing this function results in an error: However, for it works: Originally posted by in


Broken state when using assigncoords with multiindex I was trying to assign coordinates on a dataset that had been created by using stack. After assigning the coordinates, the dataset was in a state where its length was coming out as less than zero, which caused all sorts of issues. I think the issue is with the updating of , perhaps in I expected to just be able to assign the coords and then print the array to see the result. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. Here's a test (I put it in but maybe there is a better spot) <detailsINSTALLED VERSIONS commit: python: 3.8.13 (default, Mar 25 2022, 06:04:14) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.5.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enAU.UTF-8 LOCALE: ('enAU', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 0.1.dev4312+ge678a1d. pandas: 1.5.0 numpy: 1.22.4 scipy: 1.9.1 netCDF4: 1.6.1 pydap: installed h5netcdf: 1.0.2 h5py: 3.7.0 Nio: None zarr: 2.13.2 cftime: 1.6.2 nctimeaxis: 1.4.1 PseudoNetCDF: 3.2.2 rasterio: 1.3.1 cfgrib: 0.9.10.1 iris: 3.3.0 bottleneck: 1.3.5 dask: 2022.9.1 distributed: 2022.9.1 matplotlib: 3.6.0 cartopy: 0.21.0 seaborn: 0.12.0 numbagg: 0.2.1 fsspec: 2022.8.2 cupy: None pint: 0.19.2 sparse: 0.13.0 flox: 0.5.9 numpy_groupies: 0.9.19 setuptools: 65.4.0 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: None sphinx: None </details


Raise nicer error if passing a list of dimension names to transpose Hello, in xarray 0.20.1, I am getting the following error The expected result is No response No response No response <detailsINSTALLED VERSIONS commit: None python: 3.9.12 (main, Apr 5 2022, 06:56:58) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.42.2.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS LOCALE: ('enUS', 'ISO8859-1') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.20.1 pandas: 1.4.1 numpy: 1.21.5 scipy: 1.8.0 netCDF4: 1.5.7 pydap: None h5netcdf: 999 h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.1.1 nctimeaxis: 1.4.0 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.02.1 distributed: 2022.2.1 matplotlib: 3.5.1 cartopy: 0.18.0 seaborn: 0.11.2 numbagg: None fsspec: 2022.02.0 cupy: None pint: 0.18 sparse: 0.13.0 setuptools: 61.2.0 pip: 21.2.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details


Time decoding error message does not include the problematic variable's name If any variable in a Dataset has times that cannot be represented as objects, an error message will be raised. However, this error message will not indicate the problematic variable's name. It would be nice if it did, because it would make it easier for users to determine the source of the error. cc: xref: Unidata/cftime This is a minimal example of the issue. The error message gives no indication that is the problem:


xarray.opendataset has issues if the dataset returned by the backend contains a multiindex As a follow up of this comment: I'm currently trying to implement a custom backend that allows me to also handle multiindices when loading a NetCDF dataset using . I'm using the following two functions to convert the dataset to a NetCDF compatible version and back again: Here is a small code example: While the manual patching for saving is currently still required, I tried to at least work around the function call in by creating a custom NetCDF Backend: but this works: So I'm guessing is performing some operation on the dataset returned by the backend, and one of those leads to a failure if there is a multiindex already contained. I expected that it doesn't matter wheter is called inside the backend or afterwards, and the same dataset will be returned each time. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response I'm also open to other suggestions how I could simplify the usage of multiindices, maybe there is an approach that doesn't require a custom backend at all? <detailsINSTALLED VERSIONS commit: None python: 3.8.10 (default, Jan 28 2022, 09:41:12) [GCC 9.3.0] python-bits: 64 OS: Linux OS-release: 5.10.102.1-microsoft-standard-WSL2 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: C.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.10.5 libnetcdf: 4.6.3 xarray: 2022.9.0 pandas: 1.5.0 numpy: 1.23.3 scipy: 1.9.1 netCDF4: 1.5.4 pydap: None h5netcdf: None h5py: 3.7.0 Nio: None zarr: None cftime: 1.6.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.3.2 cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.6.0 cartopy: 0.19.0.post1 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 65.3.0 pip: 22.2.2 conda: None pytest: 7.1.3 IPython: 8.5.0 sphinx: 4.5.0 </details


Avoid loading any data for reprs For "small" datasets, we load in to memory when displaying the repr. For cloud backed datasets with large number of "small" variables, this can use a lot of time sequentially loading O(100) variables just for a repr. Fast reprs! This dataset has 48 "small" variables On this repr takes 36.4s If I comment the condition I get 6μs. [x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [x] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [x] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.5.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.2 distributed: None matplotlib: 3.5.2 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.5.0 cupy: None pint: None sparse: None setuptools: 62.3.2 pip: 22.1.2 conda: None pytest: None IPython: 8.4.0 sphinx: 4.5.0 </details


overwrites coordinate attributes had some unintended consequences for , where coordinate attributes are getting overwritten by variable attributes. I guess this has been broken since . Coordinate attributes should be preserved. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. I'm struggling to figure out how the simple change in brought this about. I tried tracing my way through the various merge functions but there are a lot of layers. Happy to submit a PR if someone has an idea for an obvious fix. <detailsINSTALLED VERSIONS commit: None python: 3.9.13 (main, May 27 2022, 16:56:21) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-52-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.8.1 xarray: 2022.10.0 pandas: 1.4.3 numpy: 1.23.4 scipy: 1.9.3 netCDF4: 1.6.1 pydap: None h5netcdf: 1.0.2 h5py: 3.7.0 Nio: None zarr: 2.13.3 cftime: 1.6.2 nctimeaxis: 1.4.1 PseudoNetCDF: None rasterio: 1.3.3 cfgrib: 0.9.10.2 iris: None bottleneck: 1.3.5 dask: 2022.10.0 distributed: 2022.10.0 matplotlib: 3.6.1 cartopy: 0.21.0 seaborn: None numbagg: None fsspec: 2022.10.0 cupy: None pint: 0.19.2 sparse: 0.13.0 flox: 0.6.1 numpy_groupies: 0.9.19 setuptools: 65.5.0 pip: 22.3 conda: None pytest: 7.1.3 IPython: 8.5.0 sphinx: None </details


ds.Coarsen.construct demotes non-dimensional coordinates to variables demotes non-dimensional coordinates to variables All variables that were coordinates before the coarsen.construct stay as coordinates afterwards. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response_


assigncoords reset all dimension coords to default (pandas) index See should preserve the index of coordinates that are not updated or not part of a dropped multi-coordinate index. See [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response_ <detailsXarray version 2022.11.0 </details


binary ops ignore option When doing arithmetic operations on two Dataset operands, the option is ignored and therefore attributes not kept. Minimal example: This is not true for DataArrays/Variables which do take into account. Datasets to behave the same as DataArray/Variables, and keep attributes during binary operations when option is set. PR is inbound.


stack casts int32 dtype coordinate to int64 The code example below results in , because the data type of the coordinate is changed from 'i4' to 'i8'. I expect the result to be . Creating a MultiIndex should not change the data type of the Indexes from which it is built. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)] python-bits: 64 OS: Darwin OS-release: 21.6.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.9.0 xarray: 2022.10.0 pandas: 1.5.1 numpy: 1.23.4 scipy: 1.9.3 netCDF4: 1.6.1 pydap: None h5netcdf: None h5py: 3.7.0 Nio: None zarr: None cftime: 1.6.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.10.2 distributed: None matplotlib: 3.6.1 cartopy: 0.21.0 seaborn: None numbagg: None fsspec: 2022.10.0 cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 65.5.0 pip: 22.1.2 conda: None pytest: None IPython: 8.6.0 sphinx: None ("Setuptools is replacing distutils.") </details


Add defaults during concat 508 [x] [x] Tests [x] Passes [x] Fully documented, including for all changes and for new API Contined on issue by removing exception when concat two datasets with disjoint variables and instead add the missing variable with


Pylint 2.7.0 seems to ignore the min-similarity-lines setting <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --two Python source files that share 8 common lines min-similarity-lines=40 in the pylint config pylint 2.7.0 on the source files Before pylint 2.7.0, the min-similarity-lines setting was honored and caused shorter similar lines to be accepted. Starting with pylint 2.7.0, the min-similarity-lines setting seems to be ignored and the common lines are always reported as an issue R0801, even when the min-similarity-lines setting is significantly larger than the number of common lines. The min-similarity-lines setting should be respected again as it was before pylint 2.7.0. pylint 2.7.0 astroid 2.5 Python 3.9.1 (default, Feb 1 2021, 20:41:56) [Clang 12.0.0 (clang-1200.0.32.29)] Pylint 2.7.0 seems to ignore the min-similarity-lines setting <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --two Python source files that share 8 common lines min-similarity-lines=40 in the pylint config pylint 2.7.0 on the source files Before pylint 2.7.0, the min-similarity-lines setting was honored and caused shorter similar lines to be accepted. Starting with pylint 2.7.0, the min-similarity-lines setting seems to be ignored and the common lines are always reported as an issue R0801, even when the min-similarity-lines setting is significantly larger than the number of common lines. The min-similarity-lines setting should be respected again as it was before pylint 2.7.0. pylint 2.7.0 astroid 2.5 Python 3.9.1 (default, Feb 1 2021, 20:41:56) [Clang 12.0.0 (clang-1200.0.32.29)]


Support output to a file to support tox on CI servers Currently , which means when running Pylint from within there seems to be no way to output the results to a single (for example HTML) file that a CI server can store for display. used to be close to what is needed, but it , and also created a different file for each module which is difficult to navigate. Here is the that relates to this. this feature for the same reason:


Add --fail-on option to always return error code if specific issues (or issue types) are found We're using pylint in CI with two primary goals: there are no error-category issues - this can be achieved by looking at the exit code the overall linting score doesn't deteriorate too much - this can be achieved by using However if is used, and the score is above that, then it passes even if there are error-category issues detected. Essentially it's acting as a "only throw fail (non-zero) codes if under this", instead of a "fail if under this, otherwise change nothing". Two possible solutions I can think of here are: a configuration option to prevent from consuming other exit statuses. I.e. a way to say "if score is under N, return error code X, regardless of all else, otherwise change nothing". a new option like which means that if there are any code (error-category) issues, or any issues, then fail, otherwise change nothing.


Pylint 2.8.2 broke pylint-quotes <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --See See in attached Running this configuration on a file containing: results in an exception: should not throw exceptions, and instead generate report complaining about the use of double quotes () and illegal variable name . Result of output: Additional dependencies: seems to have just changed versioning scheme: broke this line:


Allow parseable output in parallel of human-readable output Currently, when selecting a format like JSON using the argument, it will replace the "normal" output. It would be nice to have it generate an output-file instead so the normal human-readable output is still there. This makes sens in a CI environment like travis or gitlab where you can see the stdout and stderr of each job but where you also want to parse the output. Allowing multiple output formats would solve this. Maybe with a new CLI argument? Or by adding a separator to the current value? For example: pylint -f # Would redirect the JSON data to and still write the normal report to stdout pylint 1.8.1, astroid 1.6.0 Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 ]


Ignore clause not ignoring directories This is a different issue to . a directory and within that a directory . files and . Put syntax errors in both. , run . Pylint does not ignore , producing the message Pylint ignores the file . ignore-patterns does not skip non-top-level directories. <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --create a (where c.py will generate a pylint message, so that we get output) (along with the appropriate \init\.py files) Run pylint: pylint --ignore-patterns=b Run pylint: pylint --ignore-patterns=a c.py is skipped for ignore-patterns=a, but not for ignore-patterns=b c.py should be skipped for both pylint 2.1.1 astroid 2.1.0-dev Python 3.6.3 (v3.6.3:, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)]


Use Python type hints for UML generation It seems that pyreverse does not read python type hints (as defined by ), and this does not help when you use as a default value : Output of pyreverse : ! I would like to see something like : in the output. pylint- 1.6.5, astroid 1.4.9 Python 3.6.0 (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]


unused-import false positive for a module used in a type comment should not be emitted. Result of output: This is a follow up to .


Make pylint XDG Base Directory Specification compliant I have this really annoying directory in my home folder. From what I can tell (I don't do C or C++), this directory is storing data. The problem with this is, quite simply, that data storage has a designated spot. The folder. This is a part of the . A system that designates the folders for specific things like cached files (), configuration files (), and data files (), among other things. The point is to keep user home directories clean and the user sane. This should be pretty easy to implement. Simply change the variables/constants for where these files are made and stored to the appropriate directory. Simple as that, even for a large codebase (if it was done right).


Similarities checker with "ignore-signatures" option enabled ignores functions with docstring-only bodies follow-up. Given multiple files: : : Result of : error shouldn't be reported in this case. Note: if functions have bodies everything works correctly. You could add to the example functions above and no error would be reported. Result of output:


Imports within TYPECHECKING can induce "cyclic-import" Suppose you have two modules: a.py and b.py. imports , but needs in order to do type annotations, then this is usually done by importing within a block. Unfortunately, this causes pylint to report . Possibly related to


Config File Does not Expand Environment Variables If a config file is provided that has an environment variable, such as "%AppData%", it fails. This can be fixed by changing from to


Ignore class methods signatures with similarities "ignore-signatures" option enabled Initial already closed issue . One more related closed question with examples . Similarities "ignore-signatures" option currently only ignores functions' signatures. Class methods won't be ignored with the current implementation which would be very useful to avoid false-negative errors for classes. With similarities "ignore-signatures" option enabled, class methods' signatures are ignored the same way as functions'.


Setting to should stop pylint from checking duplicate code Setting to in the rcfile doesn't disable checking for duplicate code, it instead treats every line of code as duplicate and raises many errors. Setting to should disable the duplicate code check. It works that way in many other linters (like flake8). Setting a numerical value in flake8 to (e.g. ) disables that check. requests being able to disable , but it is still open


args is considered as missing in documentation Hi, I met an issue with the following code: ` Return this warning: W9015: "args" missing in parameter documentation (missing-param-doc) No warning must be returned as there is a documentation concerning args pylint 2.5.3 astroid 2.4.2 Python 3.7.2 (default, Feb 25 2019, 14:07:05) [GCC 4.8.5 (Red Hat 4.8.5-36)]


False positive missing-param-doc on numpy style when omitted Would not have expected either or checks to have failed. VS Code 1.61.2 Windows 10 Pro 21H1 No response False positive missing-param-doc on numpy style when omitted Would not have expected either or checks to have failed. VS Code 1.61.2 Windows 10 Pro 21H1 No response


The duplicate-code (R0801) can't be disabled Originally reported by: Anonymous It's seems like it's not possible to disable the duplicate code check on portions of a file. Looking at the source, I can see why as it's not a trivial thing to do (if you want to maintain the same scope semantics as other #pylint:enable/disable comments. This would be nice to have though (or I guess I could just cleanup my duplicate code). Bitbucket:


The negative score is not very beginner friendly For the past months I've been under the impression that the negative score is not very friendly for beginners. It somewhat demotivating to run over your project just to get a as a score, which would make look like a super drastic tool, that's super nitpicky about everything. I think we could do a better job here at welcoming everyone to using . Instead of having a score, we could check if other tools are doing something different and see what works best in general. The negative score is not very beginner friendly For the past months I've been under the impression that the negative score is not very friendly for beginners. It somewhat demotivating to run over your project just to get a as a score, which would make look like a super drastic tool, that's super nitpicky about everything. I think we could do a better job here at welcoming everyone to using . Instead of having a score, we could check if other tools are doing something different and see what works best in general.


Please provide a better entry point to pyreverse Currently, I have to do: I would like to do something like:


[crash] doc params extension: --accept-no-param-doc: conflicting option string(s) There is a crash when using the doc_params extension. No crash.


Pylint needs to forbid to re-use the msgid or symbol of old deleted msgid/symbol Right now it's possible to reuse old msgid that were removed, for example, everything from the python 3 checker (the problem was detected with `), or mixed-indentation, bad-whitespace, bad-continuation... maybe other that I don't remember that we deleted. We have a mechanism in place for renamed message with old_name, but nothing for removed message. Related to and This is going to cause occasional bug, but more than that confusion and inconsistencies when searching for the msgid online. Impossible to use an old message id or symbol.


"--notes" option ignores note tags that are entirely punctuation If a note tag specified with the option is entirely punctuation, pylint won't report a fixme warning (W0511). will return a fixme warning (W0511) for the first line, but not the second. macOS 11.6.1 No response


DOC: pyreverse supported format not recognized missing puml, plantuml, mmd, etc Some supported formats are not listed in the error messaging: Here, I have made the mistake of prepending a to . The error output should list (and and and ) as . Without the preceding , the command runs as expected. All supported formats are listed when user makes a typo in the format argument OpenSUSE Tumbleweed


Is required As noted by ,  is not currently covered. If this required then we should add a unit test, otherwise we can remove this method. N/A No response No response


Use, misuse and confusion of decorator The work on showed that specifically enabling just a single message is sometimes not working as expected. This is due to either missing message declaration in a checkers decorator. Ideally I would like to implement a new, optional checker aimed at Pylint core development and custom checker development. This checker would focus on correct message declaration and handling. I already started working on it, but noticed that there is especially quite some problems with the decorator. I totally understand that, because while being extensively used across all checker classes, there is not a single word on its purpose and effect in the documentation. Its docstring is also not really helpful, you have to dig deeper into the codebase to understand what it does. Before continuing my work on the new checker, I want to address and clarify some questions about the intended use of. After that this issue can be closed with a PR that extends the documentation, and if necessary some code changes. A brief explanation on what does: It helps the decide whether a callback for the current node type should be called or not. This is done by adding the list of messages passed in as a attribute on method object. When a checker is to the attribute: if the attribute does not exist, this method will be to the list of callbacks if it exists and at least one message of the list is enabled, this method will be to the list of callbacks otherwise, it will not be to the list of callbacks and will therefore not be executed Essentially this means: main (and only) intent of the decorator is to improve performance the does not have any effect to apply the decorator has no negative functional consequences, it only affects performance (which, of course, is always an issue) the decorator but forgetting to add messages that are emitted from this callback can lead to false negatives: if none of the messages actually included in the decorator are enabled, it will not be called the decorator to a callback that not only emits messages but also does other things can have nasty side effects (false positives, false negatives, crashes) What I want to gain a consensus on: we want to keep the name ? I don't think it really conveys the effect it has on the decorated method. shall we regard as best practice for this decorator? Should every callback be decorated and the list of messages must always be correct and consistent, or should we rather advise to use it only in cases where we expect a significant speedup when this callback is omitted? we want to keep the restriction that this decorator only has an effect on the "top level callbacks"? Seeing that a lot of those callbacks can emit quite a lot of messages and the logic for the checks is often split up into several subfunctions, it would be more flexible and also more robust (as it is easier to keep track of the messages that need to be included in the decorator) if one could apply it on the subfunctions rather than on the top level callback. Looking over the code base there are quite a lot of places where the decorator is not used correctly: used with simply passing in all messages (which has the same effect as just omitting it): used on methods that are not a callback (which has no effect) Used, but not including all messages possibly emitted by the callback No response_


Crash When parsing the following file: <!-- If sharing the code is not an option, please state so, but providing only the stacktrace would still be helpful. --pylint crashed with a and with the following stacktrace: If I remove the comment, things work as expected. If I call without , things work as expected. If I downgrade (and ) to astroid-2.9.3 pylint-2.12.2, things work as expected. No response no astroid error ;-) Fedora 35 astroid==2.11.2 dill==0.3.4 flake8==4.0.1 isort==4.3.21 lazy-object-proxy==1.7.1 mccabe==0.6.1 platformdirs==2.5.1 pycodestyle==2.8.0 pyflakes==2.4.0 pylint==2.13.5 toml==0.10.2 tomli==2.0.1 wrapt==1.13.3


option ignored The option is ignored in 2.14. Place two files that both read like this (ideally in their own dir so you can just run that dir): Proper output in 2.12: Caused in No response No response


Argument expected for short verbose option The short option of the option expects an argument. Also, the help message for the option suggests a value should be provided. The long option works ok & doesn't expect an argument: Similar behaviour to the long option.


Support isinstance checks through ABCMeta for checkers and reporters Currently the PyLinter categorizes checkers through the class attribute. This is not very standard, and is one more barrier for others to write plugins. I propose Changing the interfaces to have a metaclass Change PyLinter to do isinstance checks to categorize checkers/reporters in addition to keeping the old checks


Traceback printed for unrecognized option A traceback is printed when an unrecognized option is passed to pylint. No response The top part of the current output is handy: The traceback I don't think is expected & not user-friendly. A usage tip, for example: No response No response


Pylint runs unexpectedly pass if in pylintrc Runs unexpectedly pass in 2.14 if a pylintrc file has . (Default pylintrc files have . 's own config was fixed in to comment it out, but this might bite existing projects.) No response No response


How does pylint decide where it should put stat files? I am from the VS Code python extension team, working on building an extension for VS Code using pylint. We have pre-release version out. An issue was reported on the extension that it creates pylint stat files in the project directory. We are not specifically passing anything to change the stat file location (see issue here ). Generally, the stat files go into pre-set directory like %USERPROFILE%/AppData/Local/pylint on windows. What can cause these to be written to the current working directory. We use pass the source code into pylint via stdin, and the only two arguments we use are , . Any clue to debug this would be helpful. I could not find fall back location for the stat files and when would the fallback location be used. We use to run pylint, and we send source to pylint via stdin. Relevant code here:


Pylint does not respect ignores in mode Pylint does not respect the , , or setting when running in recursive mode. This contradicts the documentation and seriously compromises the usefulness of recursive mode. No response All of these commands give the same output: should be ignored by all of the above commands, because it is in an ignored directory (even the first command with no ignore setting should skip it, since the default value of is . For reference, the docs for the various ignore settings from : No response No response


Can't disable bad-option-value code on a computer with a somewhat new pylint (2.4.3 in my example). Get a warning like that I want to ignore, as I'm writing code compatible with python2 and python3. said warning with . a "Bad option value" when other people run their pylint version (example: 2.3.1; and by people, sometimes I mean docker instances ran from Jenkins that I would rather not rebuild or that depend on other people and I can't modify) to disable said error with a global is ignored is ignored To be able to write code that works on several versions of pylint and not having to make sure every computer in the company and every docker container has the same pylint version.


Using in CLI silently fail to load the plugin See Warning that load-plugin is not the right argument.


Running pylint in Kubernetes Pod with --jobs=0 fails I run pylint in multiple parallel stages with Jenkins at a Kubernets agent with . The newly introduced function is called to determine the number of cpus to use and returns 0 in this case. This leads to a crash of pylint because the multiprocessing needs a value 0. I checked the function and found out the following values from the files that are read in above mentioned function: This leads to the calculation then in line which is cast to an and therefore 0 then. No response I expect pylint to not crash if the number of available cpu is misscalculated in this special case. The calculated number should never be 0. A possible solution would be to append a at the end of this line. I'm not sure if the same can happen for the calculation in line though, as I don't know the exact backgrounds of that files. Ubuntu 20.04 Kubernetes Version: v1.18.6 Python 3.9.12 No response


--disable ignored when --errors-only specified a python file () with a single line import notfoundpackage --disable=E0401 --errors-only test.py_ reports *** Module test E0401: Unable to import 'notfoundpackage' (import-error) Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) 11.5


ignores When running recursively, it seems in my settings in is completely ignored src\gen* should not be checked Windows 10.0. No response


Linting fails if module contains module of the same name Given multiple files: Which are all empty, running fails: However, if I rename , succeeds: Alternatively, I can also , but that shouldn't be necessary anymore. Running if is present fails while searching for an file. Running if is present should succeed. Result of output: This also has some side-effects in module resolution. For example, if I create another file : With the content: Running will run fine, but will fail. Not just for module a, but for module r as well. Again, if I rename to , will work perfectly.


rxg include '\p{Han}' will throw error config rxg in pylintrc with \p{Han} will throw err .pylintrc: not throw error macOS 11.6.7


removes first item from when running from . This is the line where the first item from is removed. I think there should be a check to ensure that the first item is , or before removing. No response Check if , or before removing the first item from No response No response


Using custom braces in message template does not work Have any list of errors: On pylint 1.7 w/ python3.6 - I am able to use this as my message template However, on Python3.9 with Pylint 2.12.2, I get the following: Is this intentional or a bug? No response Expect the dictionary to print out with as the key. No response No response


false positive 'useless-import-alias' error for mypy-compatible explicit re-exports Suppose a package has the following layout: Since the submodules here implement public APIs, imports and re-exports them, expecting users to import them from the public, top-level package, e.g. . Since the implementations of and are complex, they are split into and for better maintainability and separation of concerns. So looks like this: The reason for the aliases here is to be explicit that these imports are for the purpose of re-export (without having to resort to defining , which is error-prone). Without the aliases, popular linters such as will raise an "implicit re-export" error ( -- part of ). However, pylint does not currently understand this usage, and raises "useless-import-alias" errors. Example real-world code triggering pylint false positive errors: No response No "useless-import-alias" errors should be flagged. No response No response_


False positive when importing from ` option Simply importing exceptions from the  library is giving this error. Here's an example of how we import them: Pycharm can find the exception classes just fine. I know they exist. It could have something to do with how the library is using , but I don't know too much about how that works to draw that conclusion. Also, note that we're using version 1.95.1 of . We use it in some critical paths, so we can't update it to the latest version quite yet. The configuration written below is what I've tried, but it seems based on googling that that doesn't stop all errors from being ignored regarding those modules. So I'm still getting the issue. No error to be reported Intel based 2019 Mac Book Pro. Mac OS 13.1 (Ventura). Fish shell. ccxt==1.95.1


Support globbing patterns for source-roots It's quite inconvenient having to specify every source root for complex multi-package projects like For complex multi-package projects it would be nice to be able to specify source roots as instead of listing every one of them. IMHO, it's better to go with globbing patterns rather than with regexp patterns since those give better support for path-specific matching. No response


Support globbing pattern for input specification As mentioned in and , lets consider supporting cc Add globbing support right in this line No response_


fail/warn on using parallel execution with custom plugins According to documentation: Actually, it is possible but silently broken. then Pylint should inform a user about it in such cases. As for now, I could run: without any warning or error. Unfortunately, linting results are not the same as a single process linting, but Pylint silently pass. So, results are not predictable. Proposal: emit a warning or better explicitly fail on using parallel execution with custom Pylint plugins, because people usually don't read the documentation while things works.


Different output with --jobs=1 and --jobs=2 Originally reported by: Robert Spier (BitBucket: ) Parallelism (--jobs) changes the output of pylint. It's not just the order of the tests, --jobs=2 outputs lines of output compared to only 21 for --jobs=1. pylint 1.3.1 reports no lint errors. PWD/pylintrc app/codein app/melange app/soc app/summerofcode tests 2&1 | head \ Module codein.callback W: 17, 0: import missing (no-absolute-import) W: 18, 0: import missing (no-absolute-import) W: 19, 0: import missing (no-absolute-import) W: 20, 0: import missing (no-absolute-import) \ Module W: 17, 0: import missing (no-absolute-import) W: 18, 0: import missing (no-absolute-import) W: 20, 0: import missing (no-absolute-import) W: 21, 0: import missing (no-absolute-import) PWD/pylintrc app/codein app/melange app/soc app/summerofcode tests 2&1 | head \ Module main E: 46, 2: print statement used (print-statement) E: 47, 2: print statement used (print-statement) E: 48, 2: print statement used (print-statement) E: 49, 2: print statement used (print-statement) E: 50, 2: print statement used (print-statement) \ Module tests.test_utils E:658, 8: print statement used (print-statement) E:662,10: print statement used (print-statement) E:667, 8: print statement used (print-statement) This is with the current head of the pylint repository. (1889: tip) The source code being linted is Bitbucket:


Short circuit if all checks disabled Running "pylint --disable=all" takes more than 3s! Running pylint without "disable=all" on a little project (150-lines telegram bot) takes more than 8s. It is non-usable. No response It is unacceptable that even on an empty file pylint runs for at least 3 seconds. I use the VS Code extension in my project, which for a small example with a 150-line project reacts to changes in 8 (!) seconds. This is literally impossible to use. Windows 11 No response


enable/disable options in rcfile should not depend on the order with which they are specified Hello, i'm running ; i kinda understand why behaves differently than (where the first command enables everything and then disable , while the second command the disable option is overwritten by ) but i dont think it should be the same in the rcfile: the section is (by default) before the section, so if i want to have the same effect of command1 i need to swap the sections around. on the cli i can specify multiple enable/disable options, but that's not allowed in the rcfile, so the current result is extremely counter-intuitive; and rcfile with is clear what result the user wants: to enable all checks except for some, and i shouldnt need to move config options around. can you please get that fixed? Thanks!


bad-names-rgxs mangles regular expressions with commas Since pylint splits on commas in this option, instead of taking a list of strings, if there are any commas in the regular expression, the result is mangled before being parsed. The config below demonstrates this clearly by causing pylint to crash immediately. I would expect any valid regular expression to be expressible in this option. If not directly, adding some way to escape commas so that this issue can be worked around. Pop! OS 22.04 No response


Exporting to JSON does not honor score option <!-- Hi there! Thank you for discovering and submitting an issue. Before you submit this, make sure that the issue doesn't already exist or if it is not closed. Is your issue fixed on the preview release?: pip install pylint astroid --pre -U --pylint on some random Python file or module: As you can see this outputs some warnings/scoring: run the same command but with to export it to JSON: The output doesn't contain the scores now anymore: execute it with again but also supply the option: The score is not outputted when exporting to JSON, not even when is activated. The score is to the JSON, at least when is activated.


caplog.getrecords and conflict gets decoupled from actual caplog records when is called. As a result, after is called, is frozen: it does not get cleared, nor does it get new records. During test set up it is as , but the latter gets in , which diverges the two objects. Arch Linux, Python 3.9.10:


unittest.TestCase.tearDown executed for classes marked with when running --pdb <!-- Thanks for submitting an issue! Quick check-list while reporting bugs: --- [x] a detailed description of the bug or problem you are having [x] output of from the virtual environment you are using [x] pytest and operating system versions [x] minimal example if possible Running will run the of classes that are decorated with on the class level. Identical to , but with the on the class level rather than on the function level. Minimal test (adapted from ), : Some versions (full below): Test is properly skipped normally: but when running with , the teardown seems executed: Full versions:


Get rid of atomicwrites (unmaintained) PyPI has started enforcing 2-factor-auth for maintainers of various popular packages: For context, here is the mail I got: project. Unfortunately, this has caused the maintainer of to go on what I can only describe as a rampage, first deleting the project from PyPI (and then finding out it's not possible to restore it): then simply declare the project as unmaintained outright: No matter what the outcome of this will be, IMHO, given those actions I do not feel comfortable with trusting this dependency for something as popular as pytest. The library itself , and we only use it on Windows. It's MIT-licensed. Should we just copy the parts we need into pytest instead?


Old-style hookimpl warning has no location information The old-style hookimpl deprecation warning from has no way to see where the culprit is implemented. I'm now getting: with no easy way to figure out what the problem is. I have 12 plugins installed, all of which might have a , and I'd rather not have to find out manually which one is the culprit. The error message should show either the plugin that's coming from, or at least the Python file it's in. Old-style hookimpl warning has no location information The old-style hookimpl deprecation warning from has no way to see where the culprit is implemented. I'm now getting: with no easy way to figure out what the problem is. I have 12 plugins installed, all of which might have a , and I'd rather not have to find out manually which one is the culprit. The error message should show either the plugin that's coming from, or at least the Python file it's in.


Consider MRO when obtaining marks for classes When using pytest markers in two baseclasses and , inheriting from both of those baseclasses will lose the markers of one of those classes. This behavior is present in pytest 3-6, and I think it may as well have been intended. I am still filing it as a bug because I am not sure if this edge case was ever explicitly considered. If it is widely understood that all markers are part of a single attribute, I guess you could say that this is just expected behavior as per MRO. However, I'd argue that it would be more intuitive to attempt to merge marker values into one, possibly deduplicating marker names by MRO. I'd expect and to be markers for , but this only actually is the case with this metaclass. Please note that the repro case is Python 2/3 compatible excluding how metaclasses are to (this needs to be taken care of to repro this issue on pytest 6) Consider MRO when obtaining marks for classes When using pytest markers in two baseclasses and , inheriting from both of those baseclasses will lose the markers of one of those classes. This behavior is present in pytest 3-6, and I think it may as well have been intended. I am still filing it as a bug because I am not sure if this edge case was ever explicitly considered. If it is widely understood that all markers are part of a single attribute, I guess you could say that this is just expected behavior as per MRO. However, I'd argue that it would be more intuitive to attempt to merge marker values into one, possibly deduplicating marker names by MRO. I'd expect and to be markers for , but this only actually is the case with this metaclass. Please note that the repro case is Python 2/3 compatible excluding how metaclasses are to (this needs to be taken care of to repro this issue on pytest 6) Fix missing marks when inheritance from multiple classes <!-- Thanks for submitting a PR, your contribution is really appreciated! Here is a quick checklist that should be present in PRs. [] Include documentation when adding new features. [ ] Include new tests or update existing tests when applicable. [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself. If this change fixes an issue, please: for more information. Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please: for details. Write sentences in the past or present tense, examples: Improved verbose diff output with sequences. Terminal summary statistics now use multiple colors. Also make sure to end the sentence with a . [x] Add yourself to in alphabetical order. --


RFE: allow to selectively disable loggers from command-line A common debugging strategy is to study the logs. But sometimes the program issues excessive logging messages, necessitating the selective disabling of babbler loggers. This SO captures the crux & solution of this Request For Enhancement: Although the proposed SO solution of a new option works perfectly fine, it is annoying to have to patch every new project, plus, it does not support auto-completion e.g. on bash. Would it make sense to include such capability into core code? In any case, I'm suggesting the new option to be called, to fit with the existing option names starting with `.


Keep temporary directory for failing tests only Currently pytest will keep the last 3 root temporary directories for each testing session, for all tests. This proposal is to change this behavior to only keep the directories for failed tests, instead of for all tests. This would save significant disk space for test suites which manipulate large amounts of data in the temporary directories. The behavior of keeping the last 3 root temporary directories would be kept unchanged. From: EDIT: after some discussion, the full proposal is defined in


Short test summary doesn't show the test name when skipping from a fixture I'm using Pytest 7.0.1on Ubuntu 18.04 with Python 3.6.9. Consider a test: and a fixture defined in that will skip a test based on some conditional check. Then running the test with pytest shows something like: The summary shows that some test was skipped but there's no indication which test was skipped. Instead, it should show the test name rather than the location in the fixture where the was called from. If there are multiple tests that are skipped from various locations, matching a test with its skip condition becomes impossible. There are some similar issues in , , which may be related.


Inconsistent support for staticmethod/classmethod Pytest discovery & running of staticmethods/classmethods is inconsistent. Here's an example: The most worrysome cases are and , which are not discovered by pytest. I think that there should at least be a warning or error to alert the user that their test code cannot be run. <details<summaryHere's the full output from running : </summary</detailspython v3.9.15, pytest v7.2.0, ubuntu 20.04 <details<summaryOutput of : </summary</details


when is dict containing zero value results in ZeroDivisionError Pytest behaves differently when comparing dictionaries containing zero values compared to lists containing same values pytest==7.2.0 Ubuntu 22.04


Walrus operator causes different behavior in PyTest. I am currently testing the following function but find that the function passes the test in the normal Python terminal but fails in a PyTest run. The walrus operator is relatively new and not many people use it. I think that there may be an inconsistency in the execution environment.


Use of causes on Python 3.12 alpha The current Python 3.12 alpha has made a change to , deprecating the parameter and replacing it with an parameter. Something in Pytest's temp path fixtures calls an function which calls with the parameter. When warnings are treated as errors, this makes pytest fail any test using temp paths. Add to cpython 3.12 (alpha) has an API to test results (python/cpython). This would not be an issue, except it was designed to trigger a warning if the test result doesn't have such a method (hence e.g. python/cpython). This means when using pytest as runner for unittest tests and running an error is triggered as pytest's test result (which seems to be ) does not support this protocol. Now obviously this should be non-blocking, as hopefully nobody is running CI which blocks on testing 3.12, but there you go, I figure an early forewarning can't hurt.