[ENH]: ContourSet.set_paths To get contour labelling working with its special transforms, Cartopy has a where it replaces all the paths on the with transformed versions. This currently looks like which doesn’t smell very good. The above would smell better as


[Bug]: ContourSet.antialiased attribute not present The new does not have an attribute. This causes failures in . With v3.7.1, I can access this attribute. Marking as release critical, as this is a regression. RHEL7 main QtAgg 11.4 N/A git checkout


Updating an array passed as the xy parameter to annotate updates the anottation Bug summary When an array is used as the xy kwarg for an annotation that includes arrows, changing the array after calling the function changes the arrow position. It is very likely that the same array is kept instead of a copy. Code for reproduction Actual outcome ! Expected outcome Both arrows should be horizontal Matplotlib version Operating system: Debian 9 Matplotlib version: '3.0.3' Matplotlib backend: Qt5Agg Python version:'3.5.3' Jupyter version (if applicable): Other libraries: Numpy 1.17.3 Matplotlib was installed using pip


Switching to inline backend closes GUI windows <!--To help us understand and resolve your issue, please fill out the form to the best of your ability.--<!--You can feel free to delete the sections that do not apply.--Bug summary When a plot is made with the backend on and then the backend is changed to , the active plot window(s) are closed. This makes it very difficult to switch cleanly between backends within a notebook. The two mediocre workarounds I can see are: always specify the backend before plotting (since you can't automatically "switch back") use and switch back to after the user is done with the GUI figure. This has the downside of locking the Notebook execution while the figure is live. Code for reproduction The linked gist shows what I'd like to accomplish which is a context manager that enables matplotlib plotting in a GUI window from within a Notebook that is otherwise using the backend. Basically I want the notebook to use inline (for a variety of reasons), but I occasionally wish I could interact with the data in a separate figure. As a side note, easy switching between and / backends would also scratch part of this itch, but the interactive notebook backends still don't enable full screen usage or easily throwing a plot on another monitor. has full screen figured out, so that seems doable. Here's the context manager I wish worked: Actual outcome The outcome of the above code is that a plot window flashes into existence for a split second, then is closed when the call is processed. Expected outcome It would be great if plots created with the qt5 backend could stay visible while other plots with the inline backend were also being created. If use (to establish a reliable event loop) and create a bunch of Qt windows by other means, they live concurrently with the Notebook as long as the kernel is alive. I'd like matplotlib GUI figures to be able to live on regardless of the current backend. Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: Matplotlib version: 3.1.1 Matplotlib backend (): and Python version: 3.7.3 Jupyter version (if applicable): <!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda--<!--If you installed from conda, please specify which channel you used if not the default--


Postscript backend gives wrong page sizes Bug summary When creating a Figure of exactly A4 size, the PS backend chooses "letter" as document type, leading to additional borders of the output in x direction and undesired cropping in y direction. Code for reproduction Actual outcome When not specifying the papertype explicitly, the PS backend chooses "letter" format as can be seen from the resulting postscript output. It should, instead, choose A4 format. When specifying the papertype explicitly, the output looks fine. Expected outcome The PS backend should choose A4 format if the Figure is exactly this size. Anyway, nothing should ever be cropped off the Figure. If the Figure does not fit one papertype, the next larger one should be chosen. I also do not understand why the output of the PS backend is restricted to a handfull of explicit paper sizes in the first place. Postscript does well support arbitrary page sizes. Can someone explain why matplotlib chose to only support the given sizes? This is not transparent to the user, who expects to get output of exactly the size of his/her desired Figure object. Matplotlib version Operating system: Ubuntu 19.04 Matplotlib version: 3.1.1 Matplotlib backend: PS Python version: 3.7.4 Jupyter version: 1.0.0 Other libraries: Matplotlib was installed via conda.


Typo in Poly3DCollection constructor There is a typo in that causes a exception whenever the function is called with . should be


ValueError: fill value must be in categories In the preprocesscolors function, there is the code to replace na's with background color as the comment said, using , however, if the original colors do not contain the 'white' category, this line would raise the Pandas ValueError:fill value must be in categories in .


lmplot(sharey=False) not working The following code behaves as if . (edit: actually, it does not behave the same, but it is still not rescaling the plots individually the way it should) If you do this, it suddenly works: Versions of seaborn and matplotlib: ! lmplot(sharey=False) not working The following code behaves as if . (edit: actually, it does not behave the same, but it is still not rescaling the plots individually the way it should) If you do this, it suddenly works: Versions of seaborn and matplotlib: ! Allow xlim as parameter for lmplot Seaborn versions: latest dev version and 0.11.1 doesn't seem to accept the parameter, although FacetGrid does. Use case: when , the regression lines are extrapolated until they touch the current xlims. If one afterwards want to extend these xlims, the regression lines are floating again. A workaround is either to call FacetGrid and regplot separately, or to set very wide xmargins via the rcParams. Example code.


histplot stat=count does not count all data points import as plt import seaborn as sns import numpy as np (style="whitegrid") dataa = [1, 2, 3] datab = [2.4, 2.5, 2.6] sns.histplot((dataa), color="red", binwidth=0.01, stat="count") sns.histplot((datab), color="blue", binwidth=0.01, stat="count") This produces The data point 2.6 is omitted in the output produced by histplot. The problem also exists, if the first sns.histplot command is removed. Interestingly, it has been pointed out to me that the following command works: but as I said, the single command also does not work.


relplot does not handle numpy-types for dimensional variables Test case: ! Note how it handles fine\, but does not create two columns (or error/warn in any useful way). handles this better: ! and solve the problem of initializing a from vector data differently. ~I do not remember if this ever worked in and is a regression (quite possibly not) and, if not, whether that was a failure of implementation or the result of a decision to punt on a particularly tricky issue. If the latter, it should at least give feedback about why it is not working.~ It looks like this never worked. the internal name used for here, , shows up in the y label, which I'm also not sure that we want. Note that there is no y axis label for the , although the internal name for the column variable is used because titles include that in their title template.


PairGrid errors with assigned in In seaborn version 0.9.0 I was able to use the following Code to plot scatterplots across a PairGrid with categorical hue. The reason I am not using the "hue" keyword in creating the PairGrid is, that I want one regression line (with regplot) and not one regression per hue-category. However, since I updated to searbon 0.11.1 the following Error message occurs: My further observations are: the error does not occur when using the "hue" keyword when creating PairGrid the error does not occur for numerical values for hue changing the dtype to "categorical" does not help Edit: I tried all versions between 0.9.0 and the current release (0.11.1) and the error only occurs in the current release. If I use 0.11.0, the plot seems to work.


xestimator bars now inherit scatterkws alpha x_estimator error bars were previously always opaque, but now inherit alpha parameter from scatterplot settings (if present), since the error bars replace the scatterplot.


Regression: and raise when using a custom estimator This may be related to . According to (and ) documentation, the can be any callable that maps vector to scalar. However, the following example raises with on my Windows and Mac machines (and doesn't raise with ): <details<summaryException</summary</detailsIt does work, however, when changing the function to use the builtin func: The same error is raised when changing the example code to use , or when using a different numpy aggregator within the custom function.


Visibility of internal axis labels is wrong with wrapped pair plot ! The top two subplots should have distinct x labels. Visibility of internal axis labels is wrong with wrapped pair plot ! The top two subplots should have distinct x labels.


Rename layout(algo=) to layout(engine=) Matplotlib has settled on this term with the new method in 3.6 so might as well be consistent with them. The new API also ha some implications for how the parameter should be documented / typed.


PolyFit is not robust to missing data <details<summaryTraceback</summary</details


Nominal scale should be drawn the same way as categorical scales Three distinctive things happen on the categorical axis in seaborn's categorical plots: scale is drawn to +/- 0.5 from the first and last tick, rather than using the normal margin logic grid is not shown, even when it otherwise would be with the active style on the y axis, the axis is inverted It probably makes sense to have scales (including inferred ones) do this too. Some comments on implementation: is actually trickier than you'd think; I may have posted an issue over in matplotlib about this at one point, or just discussed on their gitter. I believe the suggested approach is to add an invisible artist with sticky edges and set the margin to 0. Feels like a hack! I might have looked into setting the sticky edges on the spine artist at one point? straightforward to do in . Always a good idea? How do we defer to the theme if the user wants to force a grid? Should the grid be something that is set in the scale object itself straightforward to implement but I am not exactly sure where would be best.


Wrong legend values of large ranges As of 0.12.1, legends describing large numbers that were created using with an offset are formatted without their multiplicative offset value. An example: The code creates the following plot: ! which is wrong because is in the order of 1E6. The issue also reproduces if you create the mentioned plot using . I believe the issue stems from not using the offset value of the used to generate the tick labels: Examining the code of suggests the issue also depends on the following rcParam settings: However, I did not test it. The offset value can be safely retrieved from all formatters and based on that it can be used to create the legend title and/or labels.


Objects interface raises if faceting on partially-crossed row and column In the objects interface, one can facet two variables using rows and columns. When the faceted categories are not fully crossed, it raises: <details<summaryThe trace</summary</detailsI expect a behavior that is similar to , where the facets that contain no data are empty: !


Figure title being removed by seaborn objects API when plotting on subfigures I recently came across an odd behaviour with the seaborn objects API when using subfigures. Here is a minimal example : which results in the title missing from the image : ! Commenting the results in the title reappearing. I have done a bit of digging and found that changing line 186 from the file from to seems to solve the issue. Is there a specific reason to why it fetches the parent figure currently, since Subfigure is supposed to be a drop-in replacement for Figure ? I also expect this will not have the intended behaviour if we deal with subfigures of subfigures.


Width computation after histogram slightly wrong with log scale Note the slight overlap here: ! It becomes nearly imperceptible with more bins: ! This is not about ; has it too: !


has different behaviour than without specifying the argument Hi, I'm noticing a strange behaviour when passing to a heatmap and I believe it's due to these lines: Specifically, if I use I get something like this: ! but when I use , and are lost: ! I'm happy to send a PR if this issue isn't addressed anywhere. Ștefan


pd.NA reverses axis ordering When plotting with , axis ordering get reversed into descending. does not produce this issue NAs should be excluded without reversing axis order !


pairplot raises KeyError with MultiIndex DataFrame When trying to pairplot a MultiIndex DataFrame, raises a : MRE: Output: A workaround is to "flatten" the columns:


Raise error when blueprint name contains a dot This is required since every dot is now significant since blueprints can be nested. An error was already for endpoint names in 1.0, but should have been for this as well.


handle Decimal in json encoder The removal () decreased the flask encoding capabilities as the built-in doesn't cover cases like types. The solution seems to be: overwrite the flask app encoder with from , but this incorporates a problem for users that relies on both and as doesn't handle , while flask encoder does. The solution I found is to build a custom encoder that tests if the value is and gives it to , otherwise, handles it with the default flask app encoder. My suggestion is to incorporate a simple test in the flask encoder to add coverage, that would remove any residual dependency on . The str(decimal) was taken from: The custom encoder: The expected behavior is to work with both and as it used to work on Flask version 1.1.2 Environment: Python version: 3.8.10 Flask version: 2.0.1 and 1.1.2 Simplejson(Optional) version: 3.17.2


Exceptions are sometimes replaced with "TypeError: exceptions must derive from BaseException" I expected something about a RuntimeError. here worked for me Environment: Python version: 3.9.5 Flask version: 2.0.1


rejects When trying the development server, I find that works fine, but fails with: Isn't this somewhat counterintuitive? In ,  function insists the certificate file should have been specified, and it feels too restrictive. But I'm not familiar with Click to contribute a pretty PR... Environment: Python version: 3.9.10 Flask version: 2.0.3


Move to the app object Add a method to the app object. Similar to functions like , should look for a and call its method. This will allow applications to override the redirect behavior.


Nested blueprints are not respected when mounted on subdomains Hello, and thanks for all your work Nested blueprints work perfectly fine when using . However, when mounting the parent blueprint using a subdomain, the child routes are not accessible. The index route works as expected: So does the parent route in the subdomain: But the child responds with a 404: If we change the for when registering the blueprint however everything works as expected: This was surprising to me as I expected the same nesting to apply regardless of whether the parent is mounted using a subdomain or a URL prefix. Am I missing something? Environment: Python version: 3.10 Flask version: 2.2.2


Add a file mode parameter to flask.Config.fromfile() Python 3.11 introduced native TOML support with the package. This could work nicely with the method as an easy way to load TOML config files: However, takes an object readable in binary mode, while opens a file in text mode, resulting in this error: We can get around this with a more verbose expression, like loading from a file opened with the built-in function and passing the to : But adding a file mode parameter to would enable the use of a simpler expression. E.g.:


Require a non-empty name for Blueprints Things do not work correctly if a Blueprint is given an empty name (e.g. ). It would be helpful if a was raised when trying to do that.


Flask routes to return domain/sub-domains information Currently when checking flask routes it provides all routes but it is no way to see which routes are assigned to which subdomain. Default server name: SERVERNAME: '' Domains (sub-domains): Adding blueprints: app.registerblueprint(adminblueprint,urlprefix='',subdomain='admin') app.registerblueprint(testsubdomainblueprint,urlprefix='',subdomain='test') Feature request It will be good to see something like below (that will make more clear which route for which subdomain, because now need to go and check configuration). If it is not possible to fix routes, can you add or tell which method(s) should be used to get below information from flask?


is ALWAYS sending content length Hi, It seems like that always adds 'content-length' header to the request. I think that the right behavior is not to add this header automatically in GET requests or add the possibility to not send it. For example returns 503 for every get request that contains 'content-length' header. Thanks, Oren


Transport adapters don't work as advertised Having: We cannot currently reliably do:: We should, this is documented: Instead, because of the following might or might not work as expected: This is because there is already a default adapter for in the form of . Depending on the (seemingly random) order of keys in the dictionary, for some combinations of keys it will work, for others it won't. There are a couple. As usual, a compromise between backwards compatibility, clarity and performance. Change so that it sorts the keys every time it fetches an adapter. The advantage is that is still a vanilla dictionary, if you consider it a public API. Something like Django's which would sort the keys on insertion. The advantage is that is still a mapping and now is faster. The disadvantage is the maintenance cost of a custom data structure. Also, mild backwards incompatibility if someone somewhere used the fact that is exactly of the type (unlikely, isn't it?). Having and as the public API for adapters, we might just as well implement the sorting while mounting and just maintain as a list of tuples. The advantage is that it's the fastest solution with no additional maintenance burden. The disadvantage is backwards incompatibility. This is another design issue. It appears that sorting first from the longest key to shortest, and then alphabetically (for keys of the same length), is the way. What should we do about it? By the way, this test adapter in the example above .


CaseInsensitiveDict setitem faulty for differing-case keys I think I figured out the cause of the issue described here: It appears that if one isn't careful to use a common case convention when assigning a header after it has been assigned before, the header will fail to work. The request will use the old header value... I think I know why, too. Here's an example: The output: Notice how in both passes, only the first assignment actually "works". It stores the data for all three assignments, but will only return the value set by the initial key. I believe the only change necessary would be to the setitem method on CaseInsensitiveDict: Broken: Fixed: When I use the corrected module, the CaseInsensitiveDict behaves as I believe it should: Note that the value for that key is the last one assigned, instead of the first. If needed, I can put together a pull request for this, but it might take some time since I've never done it before :)


multipart/form-data and datetime data I raise an bug that you already fix in the past on this issue : or I tried the same methodology with that code : With the 1.2.0 version, no error is raised. With 1.2.3 version, I have that traceback : My original problem was with a python datetime in the data dict Thanks,


Cookies not persisted when set via functional API. Cookies set as part of a call to (or any of the top level methods that call it) are not persisted, including on redirects. Expected behaviour: Actual behaviour: And, a super extra bonus bug: even though we definitely sent the cookie on the first request.


Session hooks broken Request hooks are being with session hooks; since both hook dicts have a list as the value, one simply overwrites the other.


Problem POST'ing png file because of UnicodeError Here is the code I'm using: This raises the error: This problem is caused by the fact that the content-length header is actually a unicode object. When the actual body of the request is being constructed, python attempts to coerce the entire request into unicode resulting in the decode error. After tracing it, the cause is the following lines: where is declared in


Regression 2.0.1: Using MozillaCookieJar does not work Could not find an issue raised for this, not sure if this was an expected change either. This is reproducible on master. Existing code fails on update to . The cause seems to be triggered by the change at The parameter expects either or . Treating as a dict triggers the error in this instance. The following code highlights the issue: The following is thrown:


Unicode method names cause UnicodeDecodeError for some requests in Python 2.7.2 The following example works fine: But the following example (using instead of ) produces a UnicodeDecodeError: My guess is that is infecting the header with unicode when it should be a string. This is because is simply: My requests version is 1.2.3, but I see the same being used in the current source. Unicode method names cause UnicodeDecodeError for some requests in Python 2.7.2 The following example works fine: But the following example (using instead of ) produces a UnicodeDecodeError: My guess is that is infecting the header with unicode when it should be a string. This is because is simply: My requests version is 1.2.3, but I see the same being used in the current source. Unicode method names cause UnicodeDecodeError for some requests in Python 2.7.2 The following example works fine: But the following example (using instead of ) produces a UnicodeDecodeError: My guess is that is infecting the header with unicode when it should be a string. This is because is simply: My requests version is 1.2.3, but I see the same being used in the current source.


quote qop options in Digest Auth Based on RFC2617 (), the value of 'qop-options' directive should be quoted with double quotes: curl comamnd-line tool also appends these quotes. You can see this by . Unfortunately, some minor server-side implementations seem to be sensitive on this difference.


HTTP Basic Auth credentials extracted from URL are %-encoded I was relying on credential auto-extraction from the hostname [1] to perform HTTP Basic Auth but I was getting "401 Unauthorized": the spaces in the password were substituted by . Manually extracting them with and passing them to the parameter as a tuple works. [1] (second paragraph)


Request cookies should not be persisted to session After the fix for , cookies sent with a request are now incorrectly persisted to the session. Specifically, problem lies here: Removing that breaks the test case for though, still investigating a solution.


301 redirect broken with latest pyopenssl/SNI With the latest pyopenssl on Windows 64bit: I get an exception raised when ing a response to a HTTPS request. I see that after the redirect is received the returned URL is . Then requests passes the response to which uses the url to make a new request. This leads to a Unicode string being passed to urllib3 and eventually pyopenssl. And because in pyopenssl they now check that the data is of type bytes, an exception is thrown. I Wrote this test: and this is the result of


Removing a default header of a session say that you can prevent sending a session header by setting the headers value to None in the method's arguments. You would expect (as ) that this would work for session's default headers, too: What happens is that "None" gets sent as the value of header. For the reference, here is a way that works:


Why decode the response body of a redirect? Requests fails on the URL , which is a 301 redirect to . The issue seems to be that the server's initial 301 response has a header falsely claiming that the response body (a simple HTML page) is gzipped, when it's actually uncompressed. When resolving redirects, Requests does (in ): which attempts to decode One could legitimately say this is the server's problem. However, conceptually, why decode the response body of a redirect, which won't get returned? Other programs (Chromium, Firefox, ) don't do this. For example, gives an error, as expected, when not following redirects: whereas it works if you add the flag (follow redirects).


request.history can be either a list or a tuple IMHO r.history should always be a list for least surprise. In some cases, it is returned as a tuple: Thanks!


copies the original request for all subsequent requests, can cause incorrect method selection Consider the following redirection chain: The intermediate 303 See Other has caused the POST to be converted to a GET. The subsequent 307 should preserve the GET. However, because starts each iteration by copying the original request object, Requests will issue a POST!


exception not caught/wrapped in a requests exception (ConnectionError perhaps?) I just noticed a case where I had a socket reset on me, and was raised to me as a raw socket error as opposed to something like a requests.exceptions.ConnectionError: Not sure if this is by accident or design... in general, I guess I'd expect a requests exception when using requests, but I can start looking for socket errors and the like as well.


Uncaught socket.timeout during post Hallo requests devs (and thanks for an awesome lib) During a specific I most of the time get a (on timeouts) but I also sometimes get a exception. Since there is a requests exception for it, I assume that the socket exception was supposed to be caught and the requests one raise instead. The full stack trace is: The development is for a plugin for the music service Wimp for the project, which means that I could post code to reproduce, but you will not be able to run it without a Sonos speaker and a Wimp subscription. I understand if this difficulty to reproduce may mean that you cannot work with this issue. Thanks in advance Kenneth


method = builtinstr(method) problem In is a command: method = builtinstr(method) Converts method from b’GET’ to "b'GET’" Which is the literal string, no longer a binary string. When requests tries to use the method "b'GET’”, it gets a 404 Not Found response. I am using python3.4 and python-neutronclient (2.3.9) with requests (2.4.3). neutronclient is broken because it uses this "args = utils.safeencodelist(args)" command which converts all the values to binary string, including method. I'm not sure if this is a bug with neutronclient or a bug with requests, but I'm starting here. Seems if requests handled the method value being a binary string, we wouldn't have any problem. Also, I tried in python2.6 and this bug doesn't exist there. Some difference between 2.6 and 3.4 makes this not work right.


Requests unable to follow/retrieve links with percent in url A simple (url) doesn't work for the following: !;&placement=%pplacement=!;


Requests 2.5.1 doesn't recognize unicode filenames for uploads After merge of , to allow filenames to be types, unicode filenames are no longer recognized under Python 2. This checks that the filename is a , which has different behaviour on Python 2 and Python 3: In , is defines as , which is non-unicode in Python 2 and unicode in Python 3. Perhaps the check should be against basestring, or is this change in behaviour intended?


Issue with pyinstaller Can't seem to wrap my head around this. having a problem compiling a windows exe in python 2.7.9 that uses the requests library and can't find anything on google about the specific error. My script runs fine from the interpreter but when i use pyinstaller, i get :ImportError: No module named '' I can also compile windows executables that don't use requests just fine.


Prepared requests containing binary files will not send when unicodeliterals is imported The above program works perfectly in python3, and in python2 when is not imported. If the request isn't prepared it works without a problem unfortunately, I require both prepared requests and in my project. The exception raised is: Traceback (most recent call last): File "", line 15, in (main()) File "", line 12, in main requests.Session().send(prepared) File "", line 573, in send r = (request, kwargs) File "", line 370, in send timeout=timeout File "", line 544, in urlopen body=body, headers=headers) File "", line 349, in makerequest conn.request(method, url, httplibrequestkw) File "", line 1001, in request self.sendrequest(method, url, body, headers) File "", line 1035, in sendrequest self.endheaders(body) File "", line 997, in endheaders self.sendoutput(messagebody) File "", line 848, in sendoutput msg += message_body UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 109: ordinal not in range(128)


urllib3 exceptions passing through requests API I don't know if it's a design goal of requests to hide urllib3's exceptions and wrap them around requests.exceptions types. (If it's not IMHO it should be, but that's another discussion) If it is, I have at least two of them passing through that I have to catch in addition to requests' exceptions. They are requests.packages.urllib3.exceptions.DecodeError and requests.packages.urllib3.exceptions.TimeoutError (this one I get when a proxy timeouts) Thanks!


.htaccesss redirect to non ASCII folder does not work Hello, I have the following setup on a shared hoster: Apache 2.2.15 A Japanese language . (.minna; xn--q9jyb4c) IDN domain. A blog which is in the subfolder (blog) A redirect in the .htaccess file like this: So I usually open the domain http://test. and the server redirects to http://test./. This works fine in Firefox etc. With requests, I get the following error (Python 3.4 with Requests 2.7.0 on a Japanese Ubuntu 15.04): So I guess the request lib gets a redirect from a server with Japanese characters, but then fails to convert the characters correctly. If I do directly it works, only the redirect does not.


TypeError after 2.7.0 -2.8.0 upgrade (cannot make memory view...) I'm running into this traceback after upgrading to 2.8.0 from 2.7.0: The problem goes away after downgrading to 2.7.0. A full traceback can be found . Here are the versions of relevant packages: And a full list of packages: Any ideas?


Post request hangs in certain cases when body is a StringIO This is related to a report for the . The following hangs: After a cursory look, it looks like the request isn't fully formed so the server doesn't attempt to send a response which leaves the client hanging. If we call , this works. A bit more counterintuitively, this also works:


Request with binary payload fails due to calling tonativestring Introduced with This works with 2.8.1, but not with 2.9.


Uncertain about content/text vs itercontent(decodeunicode=True/False) When requesting an application/json document, I'm seeing returning bytes, whereas returns unicode. My understanding was that both should return a unicode object. In essence, I thought "itercontent" was equivalent to "itertext" when decode_unicode was True. Have I misunderstood something? I can provide an example if needed. For reference, I'm using python 3.5.1 and requests 2.10.0. Thanks!


AttributeError: 'NoneType' object has no attribute 'read' Hello :) After a recent upgrade for our project to 2.12.1 we encounter an exception in our test suites which seems to be caused by . Build: Relevant part: happens on Windows and Linux. Thanks in advance :)


Requests v2.12.2 does not add parameters to URLs with schemes it does not understand. Follows from . When we patched to ignore all unrecognised schemes instead of just those that didn't begin , we stopped handling parameters for those URLs. This may break some more benign uses such as , which wanted to add parameters to their URLs. While this is inline with our intended policies (we do not understand URLs that have schemes we don't know anything about), this, along with the IDNA behaviour in v2.12.{0,1} will probably have broken a whole bunch of people using these non-standard URL schemes. We should consider whether Session objects should have a registry of places to look for URL preparation based on scheme. This will allow people to opt-in to the HTTP-like processing of URLs, as well as to register their own.


AttributeError: module 'requests.packages' has no attribute 'urllib3' This seems to have broken requests.packages. requests.packages.urllib3 to be the urllib3 package AttributeError: module 'requests.packages' has no attribute 'urllib3' $ python -m


Misleading exception with invalid protocol in proxy variable When the value of or variable(s) accidentally miss one '/' in the protocol, a traceback is thrown to the user which doesn't pin point that the issue is with the proxy configuration. A better exception An exception which doesn't pin point exactly what went wrong. I am not sure what is the correct place to fix this. Should the fix/check be in requests, urllib3, or urlparse?


Should Authorization header be cleared in https -http redirect? This may be considered intentional behaviour (in which case feel free to close this), but if a request is made to an https endpoint with authorization and it redirects to http on the same host, the Authorization header is not stripped and will be exposed on the wire. rebuild_auth would strip the Authorization header if the scheme is changed from https to http. The credentials that were intended to be sent over TLS were transmitted in plaintext with the redirected request. Run an HTTPS server on localhost:4443 that replies with a 302 redirect to , and a plain HTTP server (or netcat) on localhost:8000. Then run The basic auth credentials are sent in plaintext to (the is just because I had a self-signed cert). Here's the code I used for the SSL server:


Accessing response.content twice removes forgets read error I had a hard debugging time today because an error in the response stream is only reported when accessing for the first time. This is especially irritating when running code in a debugger. If accessing the first time raises an exception I would expect that accessing again would also raise an exception (ideally the same). Instead after raising on the first get, getting again returns an empty string. Here is a patch with a new test case for this: . Basically, it boils down to this: Output of my test case: $ python -m Edit: Oops, I used which actually called into system python 2.7. Here comes the real data: Thanks for looking into this!


Getting raises UnicodeError Attempting to get e.g. results in a . It seems like the intention so far has been to raise instead (see e.g. ). I see there was some hesitation in fixing a similar issue () and would like to add that even catching the error just to rethrow as a requests exception would be beneficial. Based on PR : $ python -m


Proxy authentication bug <!-- Summary. --When using proxies in python 3.8.12, I get an error 407. Using any other version of python works fine. I am assuming it could be to do with this <!-- What you expected. --I should get a status of 200. <!-- What happened instead. --I get a status code of 407.


Catch UnicodeError coming from Here's a Python 2.6 Requests 0.13.1 traceback. Looks like the call to needs a try/except.


Allow lists in the dict values of the hooks argument Currently the Request class has a .registerhook() method but it parses the dictionary it expects from it's hooks argument weirdly: the argument can only specify one hook function per hook. If you pass in a list of hook functions per hook the code in () will wrap the list in a list which then fails when the hooks are consumed (since a list is not callable). This is especially annoying since you can not use multiple hooks from a session. The only way to get multiple hooks now is to create the request object without sending it, then call .registerhook() multiple times and then finally call .send(). This would all be much easier if () parsed the hooks parameter in a way that it accepts lists as it's values.


Variable.setitem coercing types on objects with a values property The current behavior prevents storing objects inside arrays of even when only performing non-broadcasted assignments if the RHS has a property. Many libraries produce objects with a property that gets coerced as a result. The use case I had in prior versions was to store instances from the curve fitting library , when fitting had be performed over an axis of a or . Ideally: Breaking changed introduced going from -as a result of , namely the change on line <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Darwin OS-release: 16.7.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 xarray: 0.10.1 pandas: 0.20.3 numpy: 1.13.1 scipy: 0.19.1 netCDF4: 1.3.0 h5netcdf: None h5py: 2.7.0 Nio: None zarr: None bottleneck: None cyordereddict: None dask: 0.15.2 distributed: None matplotlib: 2.0.2 cartopy: None seaborn: 0.8.1 setuptools: 38.4.0 pip: 9.0.1 conda: None pytest: 3.3.2 IPython: 6.1.0 sphinx: None </detailsThank you for your help! If I can be brought to better understand any constraints to adjacent issues, I can consider drafting a fix for this.


add average function It would be nice to be able to do to compute weighted averages (e.g. for geo data). Of course this would require the axes to be in a predictable order. Or to give a weight per dimension...


REGRESSION: copy(deep=True) casts unicode indices to object (deep=True) and (deep=True/False) accidentally cast IndexVariable's with dtype='<U*' to object. Same applies to () and copy.deepcopy(). This is a regression in xarray = 0.12.2. xarray 0.12.1 and earlier are unaffected.


expanddims() modifies to write only, upon manually reverting this flag back, attempting to set a single inner value using .loc will instead set all of the inner array values I am using the newly updated expanddims API that was recently updated with this PR . However the flag setting behaviour can also be observed using the old API syntax. When applying the operation 'expanddims({'z': 3}, -1)' on a DataArray the underlying Numpy array flags are changed. 'CCONTIGUOUS' is set to False, and 'WRITEABLE' is set to False, and 'OWNDATA' is set to False. Upon changing 'WRITEABLE' back to True, when I try to set a single value in the DataArray using the '.loc' operator it will instead set all the values in that selected inner array. I am new to Xarray so I can't be entirely sure if this expected behaviour. Regardless I would expect that adding a new dimension to the array would not make that array 'read-only'. I would also not expect the '.loc' method to work differently to how it would otherwise. It's also not congruent with the Numpy 'expanddims' operation. Because when I call the operation 'np.expanddims(nparr, axis=-1)' the 'CCONTIGUOUS ' and 'WRITEABLE ' flags will not be modified.


Here is a similar flow of operations that demonstrates the behaviour I would expect from the DataArray after applying 'expanddims': <detailsINSTALLED VERSIONS commit: None python: 3.6.7 (default, Dec 29 2018, 12:05:36) [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] python-bits: 64 OS: Darwin OS-release: 18.2.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enAU.UTF-8 LOCALE: enAU.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.4.1.1 xarray: 0.12.1 pandas: 0.24.2 numpy: 1.16.2 scipy: 1.2.1 netCDF4: 1.5.0 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nctime_axis: None PseudonetCDF: None rasterio: None cfgrib: 0.9.6.1.post1 iris: None bottleneck: None dask: None distributed: None matplotlib: 3.0.3 cartopy: 0.17.0 seaborn: None setuptools: 39.0.1 pip: 10.0.1 conda: None pytest: None IPython: None sphinx: None </details


xr.combinebycoords raises ValueError if identical coordinates are non-monotonic <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --should return without error. Running the example with raises an error: The documentation for says that "Non-coordinate dimensions will be ignored, as will any coordinate dimensions which do not vary between each dataset". This is not the case with the current implementation, since identical coordinate dimensions are still required to be monotonic. <detailsINSTALLED VERSIONS commit: None python: 3.7.1 (v3.7.1:, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: None.None libhdf5: None libnetcdf: None xarray: 0.12.3 pandas: 0.24.2 numpy: 1.16.4 scipy: 1.3.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.1.1 cartopy: None seaborn: 0.9.0 numbagg: None setuptools: 39.0.1 pip: 10.0.1 conda: None pytest: None IPython: 7.1.1 sphinx: None </details


groupby on empty DataArray raises StopIteration This seems similar to and so apologies if it is a duplicate, but I have a minimal example where it happens on an empty DataArray: Using groupby on an empty DataArray or Dataset raises . It should raise a more meaningful error. In particular, I had this issue in a function I was calling inside of a generator, so the StopIteration just broke out of the generator and it took some digging to figure out what was going wrong in my code. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 3.10.0-862.14.4.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.utf8 LOCALE: enUS.UTF-8 xarray: 0.10.7 pandas: 0.23.0 numpy: 1.14.3 scipy: 1.1.0 netCDF4: 1.4.0 h5netcdf: 0.6.1 h5py: 2.7.1 Nio: None zarr: None bottleneck: 1.2.1 cyordereddict: None dask: 0.17.5 distributed: 1.21.8 matplotlib: 2.2.2 cartopy: None seaborn: 0.8.1 setuptools: 39.1.0 pip: 10.0.1 conda: None pytest: 3.5.1 IPython: 6.4.0 sphinx: 1.7.4 </details


Allow passing a default value (instead of ndarray) for data argument for DataArray Hi, First of all, thanks a lot for the amazing module. It seems when I create a DataArray, I have to pass a numpy.ndarray with a correct size for the argument. It works well when I already have some data, but sometimes I want to create an "empty" DataArray with known coordinates and fill up the data later. For these cases, it would be great if xarray allows passing just a value for the argument, and fill up all the elements of the array with the value. For example, with pandas, I can do: and the resulting would be: Thanks a lot!


DataArray.quantile does not honor <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --returns <detailsINSTALLED VERSIONS commit: python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-60-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: enCA.UTF-8 libhdf5: 1.10.2 libnetcdf: 4.6.1 xarray: pandas: 0.23.4 numpy: 1.16.1 scipy: 1.1.0 netCDF4: 1.3.1 pydap: installed h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: 0.19.0 distributed: 1.23.0 matplotlib: 3.0.2 cartopy: 0.17.0 seaborn: None numbagg: None setuptools: 41.0.0 pip: 9.0.1 conda: None pytest: 4.4.0 IPython: 7.0.1 sphinx: 1.7.1 </details


Ignore missing variables when concatenating datasets? Several users ( now myself) have wondered about how to concatenate xray Datasets with different variables. With the current , you need to awkwardly create dummy variables filled with in datasets that don't have them (or drop mismatched variables entirely). Neither of these are great options -- should have an option (the default?) to take care of this for the user. This would also be more consistent with , which takes a more relaxed approach to matching dataframes with different variables (it does an outer join).


groupby().apply() on variable with NaNs raises IndexError This results in an IndexError. This is mildly confusing, it took me a little while to figure out the NaN's were to blame. I'm guessing the NaN doesn't get filtered out everywhere. The traceback: My assumption was that it would throw out the values that fall within the NaN group, like: Out: <details</details


selection from MultiIndex does not work properly Should select the array <detailsINSTALLED VERSIONS commit: None python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-957.10.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.14.0 pandas: 0.24.2 numpy: 1.15.4 scipy: 1.2.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.2 cartopy: None seaborn: 0.9.0 numbagg: None setuptools: 40.8.0 pip: 19.0.3 conda: None pytest: 5.0.0 IPython: 7.3.0 sphinx: None </detailsSorry for being quiet for a long time. I hope I could send a fix for this in a few days...


DatasetGroupBy does not implement quantile The docs claim works on grouped datasets, but that does not seem to be the case: this was found while trying to silence the nit-picky sphinx warnings in


"ValueError: Percentiles must be in the range [0, 100]" <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --By wrapping (xref: ) we also get the numpy error. However, the error message is wrong as xarray needs it to be in 0..1. BTW: thanks for , makes my life easier! Edit: uses internally. <detailsINSTALLED VERSIONS commit: None python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.12.14-lp151.28.36-default machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.1 xarray: 0.14.1+28.gf2b2f9f6 (current master) pandas: 0.25.2 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: 1.2.0 PseudoNetCDF: None rasterio: 1.1.1 cfgrib: None iris: None bottleneck: 1.2.1 dask: 2.6.0 distributed: 2.6.0 matplotlib: 3.1.2 cartopy: 0.17.0 seaborn: 0.9.0 numbagg: None setuptools: 41.4.0 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.9.0 sphinx: 2.2.1 </details


Confusing error message when attribute not equal during concat If two or more s are concatentated and they have list attributes that are not equal an exception is raised about arrays not being truth values. I guess the expected result would be that the list attribute is not included in the resulting DataArray's attributes. <details</detailsconcat fails with attrs that are dictionaries with ndarrays Fails with the following error: This is a problem because the following code actually executes properly Equivalence for an array within a nested dictionary as an attribute is evaluated differently than an array attribute, which is non-intuitive. This bug is related to but is additionally pointing out a difference in evaluation for more complex attributes. The output of the code sample should concatenate successfully with the nested dictionary attribute, or a more easily interpretable error should be thrown telling me I'm dumb for using dictionaries in attributes. (See ) <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.15.0-23-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.3 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: None cyordereddict: None dask: None distributed: None matplotlib: 3.0.0 cartopy: None seaborn: None setuptools: 40.4.3 pip: 9.0.1 conda: None pytest: None IPython: 7.0.1 sphinx: None </detailsconcat fails with attrs that are dictionaries with ndarrays Fails with the following error: This is a problem because the following code actually executes properly Equivalence for an array within a nested dictionary as an attribute is evaluated differently than an array attribute, which is non-intuitive. This bug is related to but is additionally pointing out a difference in evaluation for more complex attributes. The output of the code sample should concatenate successfully with the nested dictionary attribute, or a more easily interpretable error should be thrown telling me I'm dumb for using dictionaries in attributes.


(See ) <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.15.0-23-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.3 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: None cyordereddict: None dask: None distributed: None matplotlib: 3.0.0 cartopy: None seaborn: None setuptools: 40.4.3 pip: 9.0.1 conda: None pytest: None IPython: 7.0.1 sphinx: None </details


combinebycoords should allow for missing panels in hypercube Currently, it throws the following error: This is b/c calls , which mandates that the passed datasets form a complete hypercube. This check functiono also serves the purpose of mandating that the dimension depths are the same. Could we pull that part out as a separate function and, for , only call this first part but NOT mandate that the hypercube is complete? The expected behavior, in my mind, should be to simply replace the missing tiles of the hypercube with . I'll file a PR to this effect and welcome comments. <detailsINSTALLED VERSIONS commit: None python: 3.7.3 (default, Dec 6 2019, 08:54:18) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.14.150+ machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.1 xarray: 0.14.1 pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.2 netCDF4: 1.5.3 pydap: None h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: 2.3.2 cftime: 1.0.4.2 nctime_axis: 1.2.0 PseudoNetCDF: None rasterio: 1.1.1 cfgrib: None iris: 2.2.0 bottleneck: None dask: 2.8.1 distributed: 2.8.1 matplotlib: 3.1.2 cartopy: 0.17.0 seaborn: 0.9.0 numbagg: None setuptools: 42.0.2.post20191201 pip: 19.3.1 conda: 4.8.0 pytest: 5.3.1 IPython: 7.10.1 sphinx: 2.2.2 </details


Merging dataArray into dataset using dataset method fails While it's possible to merge a dataset and a dataarray object using the top-level function, if you try the same thing with the method it fails. Output:


Implement polyfit? Fitting a line (or curve) to data along a specified axis is a long-standing need of xarray users. There are many blog posts and SO questions about how to do it: The main use case in my domain is finding the temporal trend on a 3D variable (e.g. temperature in time, lon, lat). Yes, you can do it with applyufunc, but applyufunc is inaccessibly complex for many users. Much of our existing API could be removed and replaced with applyufunc calls, but that doesn't mean we should do it. I am proposing we add a Dataarray method called . It would work like this: Numpy's function is already vectorized in the sense that it accepts 1D x and 2D y, performing the fit independently over each column of y. To extend this to ND, we would just need to reshape the data going in and out of the function. We do this already in . For dask, we could simply require that the dimension over which the fit is calculated be contiguous, and then call mapblocks. Thoughts?


Turn on reprhtml by default? I just wanted to open this to discuss turning the reprhtml on by default. This PR it as a style option, but I suspect that more people will use if it is on by default. Does that seem like a reasonable change?


Truncate array repr based on line count <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --I thought we might have had an issue (and maybe solved it?) but couldn't find it anywhere. Forgive me if I'm duplicating. With larger arrays, it's much more reasonable: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --Something like 40 lines is probably a reasonable place to truncate? <detailsINSTALLED VERSIONS commit: None python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: [...] machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.utf8 LOCALE: enUS.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.1 xarray: 0.15.0 pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.2 netCDF4: 1.5.3 pydap: None h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: 2.7.0 distributed: 2.7.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.9.0 numbagg: None setuptools: 41.6.0.post20191101 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.9.0 sphinx: 2.2.1 </details


In-place addition of arrays with the same coords but in a different order I have two DataArrays with the same dimension, but the index is in a different order. Adding them with works fine, but the in-place addition fails. is working fine. I would expect to do the same. The in-place addition fails: <details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.8.2 (default, Mar 26 2020, 15:53:00) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.19.112-1-MANJARO machine: x8664 processor: byteorder: little LCALL: None LANG: frFR.UTF-8 LOCALE: frFR.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.0 pandas: 1.0.3 numpy: 1.18.1 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: None cartopy: None seaborn: None numbagg: None setuptools: 46.1.1.post20200323 pip: 20.0.2 conda: None pytest: None IPython: None sphinx: None </details


xr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data. The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ?


<details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </detailsxr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data.


The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ?


<details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </details


DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?


[bug] when passing boolean weights to weighted mean <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --Returns Passing a boolean array as weights to the weighted mean returns the wrong result because the are not properly normalized (in this case). Internally the is calculated as i.e. the dot product of two boolean arrays. This yields: We'll need to convert it to int or float: which is correct <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Mar 23 2020, 23:03:20) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.3.0-51-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.1.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: None numbagg: None setuptools: 46.1.3.post20200325 pip: 20.1 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.3 </details


tounstackeddataset broken for single-dim variables <!-- A short summary of the issue, if appropriate --A working roundtrip. I need to stack a bunch of variables and later unstack them again, however this doesn't work if the variables only have a single dimension. <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-96-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.1 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 41.0.0 pip: 19.0.3 conda: 4.8.3 pytest: 5.3.5 IPython: 7.9.0 sphinx: None </details


Pre-expand data and attributes in DataArray/Variable HTML repr? Given that a major purpose for plotting an array is to look at data or attributes, I wonder if we should expand these sections by default? I worry that clicking on icons to expand sections may not be easy to discover This would also be consistent with the text repr, which shows these sections by default (the Dataset repr is already consistent by default between text and HTML already) Currently the HTML repr for DataArray/Variable looks like this: ! To see array data, you have to click on the ! icon: ! (thanks to for making this a little bit more manageably sized in !) There's also a really nice repr for nested dask arrays: !


Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted.


See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details


Feature request: show units in dataset overview Here's a hypothetical dataset: It would be really nice if the units of the coordinates and of the data variables were shown in the repr, for example as:


sum: mincount is not available for reduction with more than one dimensions Is your feature request related to a problem? Please describe. with errors when passing more than one dim: Describe the solution you'd like The logic to calculate the number of valid elements is here: I think this can be fixed by replacing with Additional context Potentially relevant for


concat changes variable order A "Minimal, Complete and Verifiable Example" will make it much easier for maintainers to help you: Case 1: Creation of Dataset without Coordinates yields (assumed correct) output of: Case 2: Creation of Dataset with Coordinates yields (assumed false) output of: changes the dimension order for as well as to an alphanumerically sorted representation. should not change the dimension order in any case. <detailsINSTALLED VERSIONS commit: None python: 3.7.1 (default, Nov 13 2018, 18:33:04) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.12.14-lp150.12.48-default machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: deDE.UTF-8 LOCALE: deDE.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.11.3 pandas: 0.24.1 numpy: 1.16.1 scipy: 1.2.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.6.2 h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 3.0.2 cartopy: 0.17.0 seaborn: None setuptools: 40.8.0 pip: 19.0.2 conda: None pytest: 4.2.0 IPython: 7.2.0 sphinx: None </details[BUG] inverts coordinates order <!-- A short summary of the issue, if appropriate --Following the issue Merging two datasets using inverts the coordinates order. The concat function should not invert the coordinates but maintain the original order.


<details<summaryINSTALLED VERSIONS </summarycommit: None python: 3.6.8 (default, May 7 2019, 14:58:50) [GCC 8.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-88-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: C.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.2 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.2.0 cartopy: None seaborn: None numbagg: None setuptools: 46.1.3 pip: 9.0.1 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.3</details


(mincount=1) errors for integer data <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: returns a if has an integer dtype. Of course mincount is not necessary for integer data as it cannot contain . What you expected to happen: should be ignored Minimal Complete Verifiable Example: Anything else we need to know?


: Full traceback <details</detailsEnvironment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-42-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.2.dev64+g2542a63f pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.4 pydap: installed h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: 2.3.2 cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: installed rasterio: 1.1.0 cfgrib: 0.9.5.4 iris: None bottleneck: 1.2.1 dask: 2.6.0 distributed: 2.6.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: 0.9.0 numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.17.0 sphinx: None </details


todataframe/toseries fails when one out of more than one dims are stacked / multiindex When a dataarray has one multiindex dimension, as produced by stack, and has other dimesnions as well, toseries fails to create an combined multiindex. I would expect a series/dataframe with a multiindex with names a,b,c. Instead I get <detailslib/python2.7/site- in isnanew(obj) 115 # hack (for now) because MI registers as ndarray 116 elif isinstance(obj, ABCMultiIndex): --117 raise NotImplementedError("isna is not defined for MultiIndex") 118 elif isinstance(obj, (ABCSeries, np.ndarray, ABCIndexClass, 119 ABCExtensionArray)): NotImplementedError: isna is not defined for MultiIndex </detailsOn the other hand, when there is only one dimension, which is stacked, toseries and todataframe work <detailsINSTALLED VERSIONS commit: None python: 2.7.15 (default, May 1 2018, 23:32:55) [GCC 7.2.0] python-bits: 64 OS: Linux OS-release: 3.13.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: en_GB.UTF-8 LOCALE: None.None libhdf5: 1.8.17 libnetcdf: 4.4.1 xarray: 0.11.3 pandas: 0.23.4 numpy: 1.12.1 scipy: 0.19.1 netCDF4: 1.2.8 pydap: None h5netcdf: None h5py: 2.6.0 Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.0 cyordereddict: None dask: 0.17.3 distributed: 1.21.0 matplotlib: 2.2.2 cartopy: None seaborn: 0.7.1 setuptools: 0.6 pip: 19.0.1 conda: None pytest: 3.0.5 IPython: 5.8.0 sphinx: 1.5.1 </details


causes chunked dask DataArray to evalute its values eagerly What happened: Used to update a chunked dask DataArray, but the DataArray is no longer chunked after the update. What you expected to happen: The chunked DataArray should still be chunked after the update Minimal Complete Verifiable Example: Environment: <details<summaryOutput of <ttxr.show_versions()</tt</summary</detailsDataset constructor with DataArray triggers computation Is it intentional that creating a Dataset with a DataArray and dimension names for a single variable causes computation of that variable? In other words, why does cause the dask array to compute? A longer example:


merge(combineattrs='override') does not copy attrs but instead references attrs from the first object <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: After a merge, an attribute value change in the merged product is reflected in the first source. What you expected to happen: After a merge, the attrs of the merged product should be able to be changed without having any effect on the sources. Minimal Complete Verifiable Example: Anything else we need to know?: I believe the issue is with the line for combineattrs == "override": . This should be changed to , like it is for the other combineattrs cases.


Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.6.12 (default, Sep 15 2020, 12:49:50) [GCC 4.8.5 (Red Hat 4.8.5-37)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.6.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.16.1 pandas: 1.1.4 numpy: 1.19.4 scipy: 1.5.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.5.0 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.30.0 distributed: 2.30.0 matplotlib: 3.3.2 cartopy: None seaborn: None numbagg: None pint: None setuptools: 50.3.2 pip: 20.2.4 conda: None pytest: None IPython: None sphinx: 3.3.0 </details


astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.


Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details


not preserving attributes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Using on a DataArray with attributes results in a new DataArray without attributes. What you expected to happen: Attributes should be preserved or at least there should be a choice (e.g. pass kwargs to so can be passed). Minimal Complete Verifiable Example: Anything else we need to know?: Apart from loosing attributes the dtype is not conserved. In the example the resulting DataArray has dtype np.int64 instead of np.int8. As far as I can see this might not be an xarray but a numpy problem.


Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.8 (default, Jul 31 2020, 02:25:08) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.14.11--generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.3 conda: None pytest: 6.0.1 IPython: 7.18.1 sphinx: 3.2.1 () drops attributes <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --I would expect the attributes to remain in the data array.


<details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-33-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.4 numpy: 1.18.4 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.4 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: None matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None setuptools: 46.2.0 pip: 20.1 conda: None pytest: None IPython: 7.14.0 sphinx: 3.0.4 </details


Naming a dimension "method" throws error when calling ".loc" The name of the dimension should be irrelevant. The error message suggests that at some point the are given to another method in unsanitized form. Edit: Updated to xarray 0.12 from conda-forge channel. The bug is still present. <detailsINSTALLED VERSIONS commit: None python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.18.0-16-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.0 pandas: 0.24.2 numpy: 1.16.2 scipy: 1.2.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.3 cartopy: None seaborn: None setuptools: 40.8.0 pip: 19.0.3 conda: 4.6.8 pytest: None IPython: 7.3.0 sphinx: 1.8.5 </detailsNaming a dimension "method" throws error when calling ".loc" The name of the dimension should be irrelevant. The error message suggests that at some point the are given to another method in unsanitized form. Edit: Updated to xarray 0.12 from conda-forge channel. The bug is still present.


<detailsINSTALLED VERSIONS commit: None python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.18.0-16-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.0 pandas: 0.24.2 numpy: 1.16.2 scipy: 1.2.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.3 cartopy: None seaborn: None setuptools: 40.8.0 pip: 19.0.3 conda: 4.6.8 pytest: None IPython: 7.3.0 sphinx: 1.8.5 </details


Limit number of data variables shown in repr <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: xarray feels very unresponsive when using datasets with 2000 data variables because it has to print all the 2000 variables everytime you print something to console. What you expected to happen: xarray should limit the number of variables printed to console. Maximum maybe 25? Same idea probably apply to dimensions, coordinates and attributes as well, pandas only shows 2 for reference, the first and last variables. Minimal Complete Verifiable Example: Anything else we need to know?: Out of scope brainstorming: Though printing 2000 variables is probably madness for most people it is kind of nice to show all variables because you sometimes want to know what happened to a few other variables as well. Is there already an easy and fast way to create subgroup of the dataset, so we don' have to rely on the dataset printing everything to the console everytime?


Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryxr.showversions() INSTALLED VERSIONS commit: None python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 libhdf5: 1.10.4 libnetcdf: None xarray: 0.16.2 pandas: 1.1.5 numpy: 1.17.5 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 51.0.0.post20201207 pip: 20.3.3 conda: 4.9.2 pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.0 </details


CFTime offsets missing for milli- and micro-seconds <!-- A short summary of the issue, if appropriate --The smallest cftime offset defined in is "second" (S), but the precision of cftime objects goes down to the millisecond (L) and microsecond (U). They should be easily . PR adds a that supports the two, but they are currently untested as cannot generate an index. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --An error gets raised : . <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.6.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: frCA.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.4 xarray: 0.15.2.dev9+g6378a711.


pandas: 1.0.3 numpy: 1.18.4 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.1.1.2 nctime_axis: 1.2.0 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None pint: 0.11 setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.2 </details


DataArray transpose inconsistent with Dataset Ellipsis usage This works: This doesn't (subset air): The error message is a bit inaccurate too since I do have Ellipsis included; might be related to two calls of:


Decodecf fails when scalefactor is a length-1 list Some datasets I work with have and encoded as length-1 lists. The following code worked as of Xarray 0.16.1 In 0.16.2 (just released) and current master, it fails with this error I'm very confused, because this feels quite similar to , and I thought it was resolved . However, the behavior is different with . That works fine--no error. How might I end up with a dataset with as a python list? It happens when I open a netcdf file using the engine (documented by in ) and then write it to zarr. The numpy array gets encoded as a list in the zarr json metadata. This problem would go away if we could resolve the discrepancies between the two engines' treatment of scalar attributes.


dropsel indices in dimension that doesn't have coordinates? <!-- Please do a quick search of existing issues to make sure that this has not been asked before. --Is your feature request related to a problem? Please describe. I am trying to drop particular indices from a dimension that doesn't have coordinates. Following: , but leaving out the coordinate labels: gives me an error. Describe the solution you'd like I would think should exist and work in analogy to as does to . Describe alternatives you've considered As far as I know, I could either create coordinates especially to in order to drop, or rebuild a new dataset. Both are not congenial. (I'd be grateful to know if there is actually a straightforward way to do this I've overlooked.


Option for combineattrs with conflicting values silently dropped currently supports four options for merging : It would be nice to have an option to combine attrs from all objects like "noconflicts", but that drops attributes with conflicting values rather than raising an error. We might call this or . This is similar to how xarray currently handles conflicting values for and would be more suitable to consider for the default behavior of and other functions/methods that merge coordinates (e.g., applyufunc, concat, where, binary arithmetic). cc


jupyter repr caching deleted netcdf file What happened: Testing xarray data storage in a jupyter notebook with varying data sizes and storing to a netcdf, i noticed that opendataset/array (both show this behaviour) continue to return data from the first testing run, ignoring the fact that each run deletes the previously created netcdf file. This only happens once the was used to display the xarray object. But once in error mode, even the previously fine printed objects are then showing the wrong data. This was hard to track down as it depends on the precise sequence in jupyter. What you expected to happen: when i use , the resulting object should reflect reality on disk. Minimal Complete Verifiable Example: When executed in a cell with print for display, all is fine: but as soon as one cell used the jupyter repr: all future file reads, even after executing the test function again and even using and not , show the data from the last repr use. Anything else we need to know?


: Here's a notebook showing the issue: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jun 1 2020, 18:57:50) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 5.4.0-40-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.21.0 distributed: 2.21.0 matplotlib: 3.3.0 cartopy: 0.18.0 seaborn: 0.10.1 numbagg: None pint: None setuptools: 49.2.0.post20200712 pip: 20.1.1 conda: installed pytest: 6.0.0rc1 IPython: 7.16.1 sphinx: 3.1.2 </details


Sum and prod with mincount forces evaluation If I use the method on a lazy array with then evaluation is forced. If there is some limitation of the implementation which means it cannot be to the computation graph for lazy evaluation then this should be mentioned in the docs. Minimal Complete Verifiable Example: What happened: ` What you expected to happen: No output or exceptions, as the result1 and result2 arrays are not printed or saved. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.9.1 (default, Feb 6 2021, 06:49:13) [GCC 10.2.0] python-bits: 64 OS: Linux OS-release: 5.10.15-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: enNZ.UTF-8 LOCALE: enNZ.UTF-8 libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.16.2 pandas: 1.2.1 numpy: 1.20.0 scipy: 1.6.0 netCDF4: 1.5.5.1 pydap: None h5netcdf: 0.9.0 h5py: 3.1.0 Nio: None zarr: None cftime: 1.4.1 nctime_axis: None PseudoNetCDF: None rasterio: 1.2.0 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 53.0.0 pip: 20.3.1 conda: None pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.3 </details


DataArrayCoarsen does not have a map or reduce function I'm trying to count unique samples when resampling to a square kilometre from a 5x5m input grid. I'd like to be able to apply the function with to give me a new dimension with the original integer values and their counts. In order to resample along spatial dimensions I assume I need to use , unfortunately the object does not yet implement either a or function for applying an arbitrary function when coarsening. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --outputs; N.B. has both and while has . Would it make sense for all three to have the same interface? <detailsINSTALLED VERSIONS commit: None python: 3.7.4 (default, Sep 7 2019, 18:27:02) [Clang 10.0.1 (clang-1001.0.46.4)] python-bits: 64 OS: Darwin OS-release: 19.3.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enAU.UTF-8 LOCALE: enAU.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.0 pandas: 1.0.0 numpy: 1.18.1 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: None numbagg: None setuptools: 40.8.0 pip: 19.0.3 conda: None pytest: 5.3.5 IPython: 7.11.1 sphinx: None </details


Calling () drops coordinates This is a similar issue to bug . <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --Output (with extra stuff snipped for brevity): It's also worth mentioning that the data arrays contained in the dataset also loose their coordinates during this operation. I.E: <detailsINSTALLED VERSIONS commit: None python: 3.7.4 (tags/v3.7.4:, Jul 8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 11, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.2 xarray: 0.14.0 pandas: 0.25.2 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.7.0 distributed: None matplotlib: 3.1.1 cartopy: None seaborn: None numbagg: None setuptools: 40.8.0 pip: 19.3 conda: None pytest: None IPython: 7.8.0 sphinx: None None </details


Handling of signed bytes from OPeNDAP via pydap netCDF3 only knows signed bytes, but there's of adding an attribute to the variable to be able to store unsigned bytes non the less. This convention is handled by xarray. OPeNDAP only knows unsigned bytes, but there's which is used by the thredds server and the netCDF-c library of adding an attribute to the variable to be able to store signed bytes non the less. This hack is not handled by xarray, but maybe should be handled symmetrically at the same place (i.e. ). As descibed in the "hack", netCDF-c handles this internally, but pydap doesn't. This is why the variant returns (correctly according to the hack) negative values and the variant doesn't. However, as returns a warning at exactly the location referenced above, I think that this is the place where it should be fixed. If you agree, I could prepare a PR to implement the fix. Handling of signed bytes from OPeNDAP via pydap netCDF3 only knows signed bytes, but there's of adding an attribute to the variable to be able to store unsigned bytes non the less. This convention is handled by xarray. OPeNDAP only knows unsigned bytes, but there's which is used by the thredds server and the netCDF-c library of adding an attribute to the variable to be able to store signed bytes non the less. This hack is not handled by xarray, but maybe should be handled symmetrically at the same place (i.e. ). As descibed in the "hack", netCDF-c handles this internally, but pydap doesn't. This is why the variant returns (correctly according to the hack) negative values and the variant doesn't. However, as returns a warning at exactly the location referenced above, I think that this is the place where it should be fixed. If you agree, I could prepare a PR to implement the fix.


Date missing in datetime accessor What happened: I wonder if there is a reason, why there is no attribute in the datetime accessor. What you expected to happen: As the attribute is supported I would expect the same for the attribute Minimal Complete Verifiable Example: Suggestion: A simple addition of in should do the trick. Happy to do a PR. Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.6 (default, Oct 7 2020, 19:08:05) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 2.6.32-754.33.1.el6.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.17.0 pandas: 1.2.1 numpy: 1.20.0 scipy: 1.6.0 netCDF4: 1.5.5.1 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.6.1 cftime: 1.4.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2021.02.0 distributed: 2021.02.0 matplotlib: 3.3.4 cartopy: 0.18.0 seaborn: 0.11.1 numbagg: None pint: 0.16.1 setuptools: 49.6.0.post20210108 pip: 21.0.1 conda: None pytest: None IPython: 7.20.0 sphinx: None </details


Simplify adding custom backends <!-- Please do a quick search of existing issues to make sure that this has not been asked before. --Is your feature request related to a problem? Please describe. I've been working on opening custom hdf formats in xarray, reading up on the apiv2 it is currently only possible to declare a new external plugin in but that doesn't seem easy or intuitive to me. Describe the solution you'd like Why can't we simply be allowed to add functions to the engine parameter? Example: This seems like a small function change to me from my initial quick look because there's mainly a bunch of string checks in the normal case until we get to the registered backend functions, if we send in a function instead in the engine-parameter we can just bypass those checks.


Trailing whitespace in DatasetGroupBy text representation When displaying a DatasetGroupBy in an interactive Python session, the first line of output contains a trailing whitespace. The first example in the documentation demonstrate this: There is a trailing whitespace in the first line of output which is "DatasetGroupBy, grouped over 'letters' ". This can be seen more clearly by converting the object to a string (note the whitespace before ): While this isn't a problem in itself, it causes an issue for us because we use flake8 in continuous integration to verify that our code is correctly formatted and we also have doctests that rely on DatasetGroupBy textual representation. Flake8 reports a violation on the trailing whitespaces in our docstrings. If we remove the trailing whitespaces, our doctests fail because the expected output doesn't match the actual output. So we have conflicting constraints coming from our tools which both seem reasonable. Trailing whitespaces are forbidden by flake8 because, among other reasons, they lead to noisy git diffs. Doctest want the expected output to be exactly the same as the actual output and considers a trailing whitespace to be a significant difference. We could configure flake8 to ignore this particular violation for the files in which we have these doctests, but this may cause other trailing whitespaces to creep in our code, which we don't want. Unfortunately it's not possible to just add comments to get flake8 to ignore the violation only for specific lines because that creates a difference between expected and actual output from doctest point of view. Flake8 doesn't allow to disable checks for blocks of code either. Is there a reason for having this trailing whitespace in DatasetGroupBy representation? Whould it be OK to remove it? If so please let me know and I can make a pull request.