Formsets' addfields() method fails in some circumstances if the argument index is None. Description Formsets' addfields() method fails in some circumstances if the argument index is None. When a FormSet has the attributes self.candelete == True and self.candeleteextra == False, calling the addfields() method on that FormSet fails if the argument index is None. This occurs for example when calling FormSet.emptyform(). The result is that the method raises the exception TypeError: '<' not supported between instances of 'NoneType' and 'int'. Code example: MyFormSet = forms.formsetfactory( form=MyForm, candelete=True, candeleteextra=False, ) myformset = MyFormSet( initial=None, ) print(myformset.emptyform) The reason this happens is that in in line 493 of index is compared to initialformcount: if self.candelete and (self.candeleteextra or index < initialformcount): Checking for index not None should fix the issue: if self.candelete and (self.candeleteextra or (index is not None and index < initialformcount)): How to Reproduce A self-contained example to reproduce this bug is as follows: #!/usr/bin/env python3 import os import django from django import forms class MyForm(forms.Form): myfield = forms.CharField() if name == "main": settingsfile = ((file))[0] ( DEBUG=True, MIDDLEWARECLASSES=[], ROOTURLCONF=settingsfile, ) () MyFormSet = forms.formsetfactory( form=MyForm, candelete=True, candeleteextra=False, ) myformset = MyFormSet( initial=None, ) print(myformset.emptyform)


Make BaseConstraint arguments keyword-only Description As suggested in  I think we should change the signature of BaseConstraint to use keyword-only arguments as a separate cleanup name and violationerrormessage are already keyword-only in all the BaseConstraint subclasses in Django code base.


floatformat() crashes on "0.0000" Description Similar to the current Django code (both 4.2b1 and latest git) crash on using the floatformat template filter with 0 values. from decimal import Decimal from django.template.defaultfilters import floatformat floatformat('0.0000', 2) floatformat(Decimal('0.0000'), 2) Both throw ValueError: valid range for prec is [1, MAXPREC] Using git bisect I tracked the bug to commit . When one uses "0.0000": floatformat:2 the current code results in a precision of 0 which is not allowed: tupl = d.astuple() # with d being "0.0000" returns (sign=0,digits=(0,),exponent=-4) units = len(tupl[1]) # equals 1 units += -tupl[2] if m else tupl[2] # 1 + (-4) prec = abs(p) + units + 1 # 2 (number of requested decimals) + (-3) + 1 equals 0 roundedd = d.quantize(exp, ROUNDHALF_UP, Context(prec=prec))


Migration optimizer does not reduce multiple AlterField Description Let's consider the following operations: operations = [ migrations.AddField( modelname="book", name="title", field=models.CharField(maxlength=256, null=True), ), migrations.AlterField( modelname="book", name="title", field=models.CharField(maxlength=128, null=True), ), migrations.AlterField( modelname="book", name="title", field=models.CharField(maxlength=128, null=True, helptext="help"), ), migrations.AlterField( modelname="book", name="title", field=models.CharField(maxlength=128, null=True, helptext="help", default=None), ), ] If I run the optimizer, I get only the AddField, as we could expect. However, if the AddField model is separated from the AlterField (e.g. because of a non-elidable migration, or inside a non-squashed migration), none of the AlterField are reduced: optimizer.optimize(operations[1:], "books") [<AlterField modelname='book', name='title', field=, <AlterField modelname='book', name='title', field=, <AlterField modelname='book', name='title', field=] Indeed, the does not consider the the case where operation is also an AlterField. Is this behaviour intended? If so, could it be documented? Otherwise, would it make sense to add something like if isinstance(operation, AlterField) and self.issamefieldoperation( operation ): return [operation]


Subquery referencing WINDOW expression breaks query compilation Description introduced the ability to filter on WINDOW expressions but when a WINDOW expression is referred to in a subquery, the query compiler fails: ValueError: Need either using or connection I assume it has to do with self.connection not being passed on to innerquery.getcompiler in getqualifysql introduced here:  Below is the full traceback. A dummy test is available here:  Traceback (most recent call last): File "", line 57, in testPartExecutor yield File "", line 623, in run self.callTestMethod(testMethod) File "", line 579, in callTestMethod if method() is not None: File "", line 1025, in testfiltersubquery list( File "", line 398, in iter self.fetchall() File "", line 1881, in fetchall self.resultcache = list(self.iterableclass(self)) File "", line 91, in iter results = compiler.executesql( File "", line 1545, in executesql sql, params = () File "", line 732, in assql extraselect, orderby, groupby = self.presqlsetup( File "", line 84, in presqlsetup self.setupquery(withcolaliases=withcolaliases) File "", line 73, in setupquery , self.klassinfo, self.annotationcolmap = self.getselect( File "", line 296, in getselect sql, params = self.compile(col) File "", line 542, in compile sql, params = (self, self.connection) File "", line 1536, in assql subquerysql, sqlparams = (compiler, connection) File "", line 1150, in assql sql, params = self.getcompiler(connection=connection).assql() File "", line 751, in assql result, params = self.getqualifysql() File "", line 676, in getqualifysql innerquerycompiler = innerquery.getcompiler( File "", line 298, in get_compiler raise ValueError("Need either using or connection") ValueError: Need either using or connection


Querying with an integer larger than SQLite supports crashes with OverflowError Description Hi, i don't know, you accept this behavior Overflow on SQLite or there is max length here? Step : http://127.0.0.1:8000/admin/auth/user//change/ Error : OverflowError at /admin/auth/user//change/ Python int too large to convert to SQLite INTEGER Request Method: GET Request URL: http://127.0.0.1:8000/admin/auth/user//change/ Django Version: 1.10.2 Exception Type: OverflowError Exception Value: Python int too large to convert to SQLite INTEGER Exception Location: C:\Python27\lib\site-packages\django\db\backends\sqlite3\ in quoteparamsforlastexecutedquery, line 129 Python Executable: C:\Python27\ Python Version: 2.7.11 Python Path: ['E:/Programmer Language/NodeJS/untitled1', 'C:\Users\RaminFP\.IntelliJIdea14\config\plugins\python\helpers\pydev', 'C:\Python27\lib\site-packages\six-1.10.0-', 'C:\Python27\lib\site-packages\pisa-3.0.33-', 'C:\Python27\lib\site-packages\sqlacodegen-1.1.6-', 'C:\Python27\lib\site-packages\inflect-0.2.5-', 'C:\Python27\lib\site-packages\0x10c_asm-0.0.2-', 'C:\Python27\lib\site-packages\jpype1-0.6.1-py2.7-', 'C:\Python27\lib\site-packages\simpleaes-1.0-', 'C:\Python27\lib\site-packages\celery-3.1.23-', 'C:\Python27\lib\site-packages\kombu-3.0.35-', 'C:\Python27\lib\site-packages\anyjson-0.3.3-', 'C:\Python27\lib\site-packages\pythonnet-2.1.0.dev1-py2.7-', 'E:\Programmer Language\NodeJS\untitled1', 'C:\Windows\SYSTEM32\', 'C:\Python27\DLLs', 'C:\Python27\lib', 'C:\Python27\lib\plat-win', 'C:\Python27\lib\lib-tk', 'C:\Python27', 'C:\Python27\lib\site-packages', 'c:\python27\lib\site-packages'] Server time: Fri, 28 Oct 2016 11:43:05 +0000


Field position reference for aggregate ends up in group-by clause Description Changeset introduced a regression. Aggregate queries are rejected by the database due to the aggregated field being to the GROUP BY clause. It was difficult for me to pin down, especially because it looks like the error only occurs on the second evaluation of the query. The first query is executed just fine and doesn't contain the position reference to the aggregated field in the GROUP BY, only the second one. Below is a test to reproduce the behaviour: diff --git index .. a b class AggregationTests(TestCase): lambda b: (b.name, b.authorCount), ) 1324 def testquotingaggregateorderbyf(self): 1325 author = (name="Peter Norvig") 1326 qs = ( 1327 () 1328 .annotate(num=Count("authors")) 1329 .orderby(F("num").desc()) 1330 ) 1331 list(qs.iterator()) 1332 list(qs.iterator()) 1333 def test_stddev(self): self.assertEqual( Book.objects.aggregate(StdDev("pages")),


ASGI http.disconnect not handled on requests with body. Description Noticed whilst reviewing PR for , we're not handling the ASGI http.disconnect message correctly. Since it's only dealt with whilst reading the request body, http.disconnect is not processed on a request that includes a body.  async def readbody(self, receive): """Reads an HTTP body from an ASGI connection.""" # Use the tempfile that auto rolls-over to a disk file as it fills up. bodyfile = tempfile.SpooledTemporaryFile( maxsize=settings.FILEUPLOADMAXMEMORYSIZE, mode="w+b" ) while True: message = await receive() if message["type"] == "http.disconnect": # This is the only place is handled. () # Early client disconnect. raise RequestAborted() # Add a body chunk from the message, if provided. if "body" in message: (message["body"]) # Quit out if that's the end. if not ("morebody", False): break (0) return bodyfile http.disconnect is designed for long-polling — so we imagine a client opening a request, with a request body, and then disconnecting before the response is generated. The protocol server (Daphne/uvicorn/...) will send the http.diconnect message, but it's not handled. This test fails on main (at — 4.2 pre-alpha) diff --git index .. --- +++ -188,6 +188,18 class ASGITest(SimpleTestCase): with self.assertRaises(asyncio.TimeoutError): await communicator.receiveoutput() async def testdisconnectwithbody(self): application = getasgiapplication() scope = self.asyncrequestfactory.basescope(path="/") communicator = ApplicationCommunicator(application, scope) await communicator.sendinput({ "type": "http.request", "body": b"some body", }) await communicator.sendinput({"type": "http.disconnect"}) with self.assertRaises(asyncio.TimeoutError): await communicator.receiveoutput() async def testwrongconnectiontype(self): application = getasgiapplication() scope = self.asyncrequestfactory.basescope(path="/", type="other") To handle this correctly it looks like we'd need something like Channel's  to keep receiving from the input queue whilst dispatching the request.


AdminSite.catchallview() drops query string in redirects Description introduced AdminSite.catchallview(). However, in the process it broke the ability to redirect with settings.APPENDSLASH = True when there are query strings. Provided URL: http://127.0.0.1:8000/admin/auth/foo?id=123 Expected redirect: http://127.0.0.1:8000/admin/auth/foo/?id=123 Actual redirect: http://127.0.0.1:8000/admin/auth/foo/ This seems to be because the redirect in question does not include the query strings (such as via request.META['QUERYSTRING']): return HttpResponsePermanentRedirect("%s/" % )


Make URLField assume "https". Description In the assumption is made that the http (no S) is a good default scheme for URLs that do not specify a scheme when submitted. Entering in a URLField will give as cleaned data. Ref:  I think URLField should assume the safe option https. I've notified the security team, and they didn't see this as a security issue.


Added support for direct usage of Choices classes on model fields Description (last modified by T. Franzel) Hi, I would like to propose a feature addition on how Choices are handled when used on model fields. Currently, Field.choices only accepts iterables. This has 2 shortcommings imho: Rejecting a Choices class as argument to Field(choices=...) seems counter-intuitive. Providing the class directly currently results in a fields.E005 error. To make this more pythonic, the field should also accept the Choice class directly and deal with the variation internally. I really can't come up with a scenario where a user would want a different behavior or rather that a user would be surprised by the implicit resolution. forcing the user to expand the Choices class manually, essentially all meta information is lost. Downstream packages may benefit from this lost information. Specifically, as maintainer of drf-spectcular (OpenAPI generator for DRF), I am interested in the name of the choice set (e.g. Suit, Vehicle, Gender) and potentially also the docstring. This would greatly improve OpenAPI generation of choice sets and take out the unnecessary guesswork to find a proper name. (And if anyone wonders, the model field name is not a good candidate for a choice set name.) This PR allows to use Choices classes directly as argument, while being transparent. No behavioral changes otherwise. I marked this as dev, but it would be awesome if it could still slip into 4.2. Not sure if the feature window is still open, but this is more or less a trivial and backwards-compatible change with little risk. PR is still missing some docs, which I will write if this is considered. class Suit(models.IntegerChoices): """ All possible card categories in a deck """ DIAMOND = 1, ("Diamond") SPADE = 2, ("Spade") HEART = 3, ("Heart") CLUB = 4, ("Club") class Choiceful(models.Model): # CURRENTLY: fromenumold = models.IntegerField(choices=Suit.choices) # NEW: raised an fields.E005 prior to proposed PR. Now, retains reference # to class and transparently resolves via implicit call from_new = models.IntegerField(choices=Suit)


SECRETKEYFALLBACKS is not used for sessions Description I recently rotated my secret key, made the old one available in SECRETKEYFALLBACKS and I'm pretty sure everyone on our site is logged out now. I think the docs for SECRETKEYFALLBACKS may be incorrect when stating the following: In order to rotate your secret keys, set a new SECRETKEY and move the previous value to the beginning of SECRETKEYFALLBACKS. Then remove the old values from the end of the SECRETKEYFALLBACKS when you are ready to expire the sessions, password reset tokens, and so on, that make use of them. When looking at the Django source code, I see that the saltedhmac function uses the SECRETKEY by default and the AbstractBaseUser.getsessionauthhash method does not call salted_hmac with a value for the secret keyword argument.


Migrations tries to add constraint before adding a foreign key. Description (last modified by Raphael Beekmann) Hello, I have a model, already created through previous migrations, and in a new migration I a new field with an UniqueConstraint. The migrations script try to create the constraint first and then the new field, resulting an error : NewModel has no field named 'category' To reproduce the bug : Create a project with two models linked together with a One-to-Many relation and an unique constraint : class Type(models.Model): name = models.CharField(maxlength=10) class Model(models.Model): name = models.CharField(maxlength=10) type = models.ForeignKey(Type, ondelete=models.SETNULL, null=True) date = models.DateField(autonow=True) class Meta: constraints = ( models.UniqueConstraint(fields=('date', 'type'), name='uniquetypefordate'), ) Create a migration file with makemigrations Add a new model with another One-to-Many relation and unique constraint. The models looks like this : class Type(models.Model): name = models.CharField(maxlength=10) class Category(models.Model): name = models.CharField(maxlength=10) class Model(models.Model): name = models.CharField(maxlength=10) type = models.ForeignKey(Type, ondelete=models.SETNULL, null=True) category = models.ForeignKey(Category, ondelete=models.SETNULL, null=True) date = models.DateField(autonow=True) class Meta: constraints = ( models.UniqueConstraint(fields=('date', 'type'), name='uniquetypefordate'), models.UniqueConstraint(fields=('date', 'category'), name='uniquecategoryfordate'), ) Create a new migration file. The order of the migration's steps are incorrect and the migration crash : class Migration(migrations.Migration): dependencies = [ ('app', '0001initial'), ] operations = [ migrations.CreateModel( name='Category', fields=[ ('id', models.BigAutoField(autocreated=True, primarykey=True, serialize=False, verbosename='ID')), ('name', models.CharField(maxlength=10)), ], ), migrations.AddConstraint( modelname='model', constraint=models.UniqueConstraint(fields=('date', 'category'), name='uniquecategoryfordate'), ), migrations.AddField( modelname='model', name='category', field=models.ForeignKey(null=True, , to='app.category'), ), ]


Improper guessing of Mime Type for "br" and "Z" file types Description Below FileResponse will set the content type as text/html, even if the last file extension is "Z' or "br". FileResponse(open('', 'rb')) FileResponse(open('', 'rb'))


Querysets: annotate() columns are forced into a certain position which may disrupt union() Description (Reporting possible issue found by a user on #django) Using values() to force selection of certain columns in a certain order proved useful unioning querysets with union() for the aforementioned user. The positioning of columns with annotate() is not controllable with values() and has the potential to disrupt union() unless this fact is known and the ordering done in a certain way to accommodate it. I'm reporting this mainly for posterity but also as a highlight that perhaps this should be mentioned in the documentation. I'm sure there are reasons why the annotations are appended to the select but if someone feels that this is changeable then it would be a bonus outcome.


() on querysets in descending order by annotations. Description When I execute Model.objects.annotate(messagelength=Length('message')).orderby('-messagelength').update(text="Can I come on board?") I get the error FieldError: Cannot resolve keyword 'messagelength' into field. Choices are: message, id, text, x because the code ignores the descending case.


ModelAdmin.lookupallowed() incorrectly raises DisallowedModelAdminLookup lookup with foreign key as primary key Description (last modified by Tim Graham) Wrote a failing test for to demonstrate - same test/code passes on 1.8 def testlookupallowedforeignprimary(self): class Country(models.Model): name = models.CharField(maxlength=256) class Place(models.Model): country = models.ForeignKey(Country, models.CASCADE) class Restaurant(models.Model): place = models.OneToOneField(Place, models.CASCADE, primarykey=True) class Waiter(models.Model): restaurant = models.ForeignKey(Restaurant, models.CASCADE) class WaiterAdmin(ModelAdmin): listfilter = [ 'restaurantplacecountry', ] ma = WaiterAdmin(Waiter, ) self.assertIs(ma.lookupallowed('restaurantplacecountry', 'testvalue'), True) I think this is caused by the admin thinking that having a foreign key field as a primary key is the same as concrete inheritance. So when you try and check lookups for restaurantplacecountry it thinks 'place' is the concrete parent of 'restaurant' and shortcuts it to restaurantcountry which isn't in 'listfilter'. And you can't add restaurantcountry to list_filter because country isn't actually on restaurant.


Migration import ordering violates coding style and isort defaults Description New migration files are generated with imports sorted by module, independent of import style. For example: import datetime from import migrations, models import time The Django coding style specifies: Place all import module statements before from module import objects in each section. This guidance is the same as what isort does by default, as documented here. Newly generated migrations can fail isort for this reason. This would mean migration files should instead be generated like this: import datetime import time from import migrations, models For reference, previous issues related to migration import sorting: , .


SelectDateWidget can crash with OverflowError. Description Given a relatively common view like this: from django import forms from import SelectDateWidget from import HttpResponse class ReproForm(forms.Form): mydate = forms.DateField(widget=SelectDateWidget()) def reproview(request): form = ReproForm(request.GET) # for ease of reproducibility if form.isvalid(): return HttpResponse("ok") else: return HttpResponse("not ok") urlpatterns = [path('repro/', views.reproview, name='repro')] A user can trigger a server crash, reproducible by running locally and visiting http://127.0.0.1:8000/repro/?mydateday=1&mydatemonth=1&mydateyear=, which results in [...] - ERROR - django.request: Internal Server Error: /repro/ Traceback (most recent call last): [...] File "[...]/site-", line 1160, in valuefromdatadict datevalue = (int(y), int(m), int(d)) OverflowError: signed integer is greater than maximum This can be triggered similarly for a post request. The issue happens as part of the validation logic run in form.isvalid, specifically, when calling the SelectDateWidget.valuefromdatadict, where the user-controlled value is converted into a date without guarding against a possible OverflowError. Specifically, y, m and d are user controlled, and the code does this: date_value = (int(y), int(m), int(d)) When large integers (larger than sys.maxsize) are supplied to date's constructor it will throw an OverflowError: import datetime, sys (sys.maxsize+1, 3, 4) Traceback (most recent call last): File "<stdin", line 1, in <moduleOverflowError: Python int too large to convert to C long


ASGIStaticFilesHandler raises warning. Description (last modified by Carlton Gibson) async support to StreamingHttpResponse for Django 4.2. With Django 4.2rc1 and Daphne, ASGIStaticFilesHandler raises a warning about the incorrect iterator type being used: Warning: StreamingHttpResponse must consume synchronous iterators in order to serve them asynchronously. Use an asynchronous iterator instead. This is because FileResponse is not async compatible (nor is that likely, since neither is open()). A new project with: INSTALLEDAPPS = [ "daphne", ... ] ASGIAPPLICATION = "" runserver, and then visit any page serving static files such as /admin/. The fix is to have ASGIStaticFilesHandler adapt the iterator in the same way StreamingHttpResponse does.


values() doesn't mention annotations as valid choices on resolving error Description When doing Book.objects.annotate(annotation=Value(1)).values("annotationtype") The resulting FieldError doesn't mention annotation as a valid choice FieldError: Cannot resolve keyword 'annotationtypo' into field. Choices are: age, book, bookcontactset, friends, id, name


Improve error message for incorrect context processors Description If you write a template context processor but return the wrong type, for example None by missing the return statement: def data(request): data = {"something": True} Then currently the error message is very mysterious: Internal Server Error: / Traceback (most recent call last): ... File "/..../site-", line 61, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/..../site-", line 173, in render with context.bindtemplate(self): File "", line 137, in enter return next() ^^^^^^^^^^^^^^ File "/..../site-", line 254, in bindtemplate (processor(self.request)) TypeError: 'NoneType' object is not iterable If a view returns the wrong type, Django raises a nice error message: The view didn't return an HttpResponse object. It returned None instead. I suggest we do the same for context processors. If we use try/except around the () line, it will not slow down normal processing any noticeable amount, thanks to Python 3.11's “zero-cost” exception handling:


DateField.inputformats cannot be printed Description I am trying to display the input formats for This seems to fail at the moment when using str, while repr works. Example code: from import DateField print(repr(DateField().inputformats)) print(DateField().input_formats) This will result in the following error: Traceback (most recent call last): File "<console", line 1, in <moduleTypeError: str returned non-string (type list) I would have expected the string representation to be available as well instead of failing with an internal Python error.


i18npatterns() not respecting prefixdefaultlanguage=False Description (last modified by Oussama Jarrousse) In my django project file I have the following setup: from import i18npatterns from django.contrib import admin from import include from import path urlpatterns = [] urlpatterns += i18npatterns( path("admin/", ), prefixdefaultlanguage=False ) In versions Django==4.1.7 (or prior), I was able to navigating to /admin/ without having to add the language prefix. Django==4.2.0, navigating to /admin/ will cause a HTTP 302 and only /en/admin/ works... although prefixdefault_language=False is explicitly defined. This change broke my API upon backend packages upgrade from 4.1.7 to 4.2.0


StepValueValidator does not take into account minvalue Description If you define a number input with <input type="number" min=1 step=2, client side this will only allow positive odd numbers. We could generate the same input in a Django form with IntegerField(minvalue=1, stepsize=2) and Field.localize is False, which would then use MinValueValidator and StepValueValidator. We then get into a problem as StepValueValidator always uses 0 as the base, so stepsize=2 only even numbers are allowed. This then conflicts with the client side validation, and the user cannot submit any value for the input. I'm unsure if this is a bug or whether this is just a configuration problem, apologies if so, but the behaviour does seem to me to conflict with how min and step is handled by browsers.


Error messages customization and internationalization in Paginator class Description There is no ability to change error messages raised by Paginator.validate_number method.


ASGIRequest doesn't respect settings.FORCESCRIPTNAME. Description For example, I have settings.FORCESCRIPTNAME = '/some-prefix' I start a django server with command: daphne And I navigate to the http://localhost:8000/admin/login, and see the login form action url is "/admin/login" which is wrong, which should be "/some-prefix/admin/login"


Annotating with Chr raises ValueError Description (last modified by Jacob Walls) Author.objects.annotate(firstletter=Chr(Value(77))) File "", line 369, in <lambdaelse int(value) ValueError: invalid literal for int() with base 10: 'M' I'm suggesting in Chr we can set outputfield = CharField() to prevent this.


Admin check for reversed foreign key used in "listdisplay" Description Currently the admin site checks and reports an admin.E109 system check error when a ManyToManyField is declared within the listdisplay values. But, when a reversed foreign key is used there is no system error reported: System check identified no issues (0 silenced). April 10, 2023 - 19:04:29 Django version 5.0.dev20230410064954, using settings 'projectfromrepo.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. and visiting the admin site for the relevant model results in the following error: TypeError: createreversemanytoonemanager.() missing 1 required keyword-only argument: 'manager' [10/Apr/2023 19:30:40] "GET /admin/testapp/question/ HTTP/1.1" 500 Ideally, using a reversed foreign key would also result in a system check error instead of a 500 response. To reproduce, follow the Question and Choice models from the Django Tutorial and add the following to the QuestionAdmin: listdisplay = ["questiontext", "choiceset"] Then, visit /admin/testapp/question/ and the following traceback is returned: Internal Server Error: /admin/testapp/question/ Traceback (most recent call last): File "", line 681, in getfield return self.fieldsmap[fieldname] ~~~^^^^^^^^^^^^ KeyError: 'choiceset' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 288, in lookupfield f = getnongfkfield(opts, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 319, in getnongfkfield field = opts.getfield(name) ^^^^^^^^^^^^^^^^^^^^ File "", line 683, in getfield raise FieldDoesNotExist( Question has no field named 'choiceset' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 55, in inner response = getresponse(request) ^^^^^^^^^^^^^^^^^^^^^ File "", line 220, in getresponse response = () ^^^^^^^^^^^^^^^^^ File "", line 111, in render self.content = self.renderedcontent ^^^^^^^^^^^^^^^^^^^^^ File "", line 89, in renderedcontent return (context, self.request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 61, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 171, in render return (context) ^^^^^^^^^^^^^^^^^^^^^ File "", line 163, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in render return SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 962, in renderannotated return (context) ^^^^^^^^^^^^^^^^^^^^ File "", line 157, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 163, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in render return SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 962, in renderannotated return (context) ^^^^^^^^^^^^^^^^^^^^ File "", line 157, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 163, in render return (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in render return SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 962, in renderannotated return (context) ^^^^^^^^^^^^^^^^^^^^ File "", line 63, in render result = (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in render return SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 962, in renderannotated return (context) ^^^^^^^^^^^^^^^^^^^^ File "", line 63, in render result = (context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in render return SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1001, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 962, in renderannotated return (context) ^^^^^^^^^^^^^^^^^^^^ File "", line 45, in render return super().render(context) ^^^^^^^^^^^^^^^^^^^^^^^ File "", line 258, in render dict = (resolvedargs, resolvedkwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 340, in resultlist "results": list(results(cl)), ^^^^^^^^^^^^^^^^^ File "", line 316, in results yield ResultList(None, itemsforresult(cl, res, None)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 307, in init super().init(items) File "", line 217, in itemsforresult f, attr, value = lookupfield(fieldname, result, cl.modeladmin) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 301, in lookupfield value = attr() ^^^^^^ TypeError: createreversemanytoonemanager.() missing 1 required keyword-only argument: 'manager' [10/Apr/2023 19:30:40] "GET /admin/testapp/question/ HTTP/1.1" 500


lookupallowed fails to consider dynamic listfilter Description Currently, lookupallowed iterates over self.listfilter to determine valid params. This is technically incorrect since the introduction of getlistfilter() on ModelAdmin in 1.5, because it is possible to define a ModelAdmin such that self.listfilter is () but getlistfilter yields SimpleListFilter classes. To correct it, the above code would need to change from: for filteritem in self.listfilter: to for filteritem in self.getlistfilter(request): The problem is that now lookupallowed needs to accept request so that it can pass it back to getlistfilter In Django itself, that's actually reasonably acceptable as a change, because it's used infrequently - the only place it's actually used is in ChangeList.getfilters, which has access to the request. However, it is overridden in the wild without accept args, *kwargs, so it'd not be easy to provide a clean upgrade path.


ImageField unnecessarily adds a postinit signal handler to the model Description While debugging some performance issues in a Django app, I found a codepath where most of the time was being spent on initializing Django models after fetching from the DB. It turns out that 30% of the time was being spent on evaluating postinit signals because we were using ImageField. However, the postinit signal handler is a noop because we don't use the widthfield / heightfield. If widthfield and heightfield are not set, removing the postinit signal should have no effect since the signal handler will return right away. Removing this signal handler gave us a 30-40% speedup on initializing models where ImageField was used.


"random" template filter crashes on an empty list. Description first/last filters applied to a empty list do not raise anything and just returns a empty page. Example: {{ listvar|last }} Rendered with: def index(request): return render(request, '', context={"listvar": []}) Will result in empty page: [26/Apr/2023 09:15:49] "GET / HTTP/1.1" 200 0 As I undestand, 0 means no any characters in response content. The similar situation is with the first tag: {{ listvar|first }} While the random tag raises the IndexError and causes the server to response 500. Code snippet: {{ listvar|random }} In console we have: Internal Server Error: / Traceback (most recent call last): File "", line 47, in inner response = getresponse(request) File "", line 181, in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "/home/alex/Документы/django-", line 8, in index return render(request, '', context={"listvar": []}) File "", line 19, in render content = loader.rendertostring(templatename, context, request, using=using) File "", line 62, in rendertostring return (context, request) File "", line 61, in render return (context) File "", line 176, in render return (context) File "", line 168, in render return (context) File "", line 977, in render return SafeString(''.join([ File "", line 978, in <listcompnode.renderannotated(context) for node in self File "", line 938, in renderannotated return (context) File "", line 1038, in render output = self.filterexpression.resolve(context) File "", line 728, in resolve newobj = func(obj, argvals) File "", line 616, in random return (value) File "", line 347, in choice return seq[self._randbelow(len(seq))] IndexError: list index out of range [26/Apr/2023 09:30:03] "GET / HTTP/1.1" 500 Django version 4.0.1, python version Python 3.9.2 Expected behavior: Since all first, last and random tags returns an element from the list, their conduct in the case when the list is empty must be the same: either all 3 of them should raise an error or all 3 should return empty content.


Translatable URL patterns raise 404 for non-English default language when prefixdefaultlanguage=False is used. Description A simple django project with instruction to replicate the bug can be found here: github repo In brief: prefixdefaultlanguage = False raises HTTP 404 for the default unprefixed pages if LANGUAGECODE is not "en". I think the problem is that the function getlanguagefrompath in returns None in case of failure instead of LANGUAGECODE: diff in 4.2 Consequently, other mechanisms are used to get the language (cookies or headers) that do not work neither. Related issue with my last comment adding some extra context: It is the first time I contribute to django, I hope the bug report is OK. I am also willing to write the patch and test if required.


Error E108 does not cover some cases Description (last modified by Baha Sdtbekov) I have two models, Question and Choice. And if I write listdisplay = ["choice"] in QuestionAdmin, I get no errors. But when I visit /admin/polls/question/, the following trace is returned: Internal Server Error: /admin/polls/question/ Traceback (most recent call last): File "", line 334, in labelforfield field = getnongfkfield(, name) File "", line 310, in getnongfkfield raise FieldDoesNotExist() During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 55, in inner response = getresponse(request) File "", line 220, in getresponse response = () File "", line 111, in render self.content = self.renderedcontent File "", line 89, in renderedcontent return (context, self.request) File "", line 61, in render return (context) File "", line 175, in render return (context) File "", line 167, in render return (context) File "", line 1005, in render return SafeString("".join([node.renderannotated(context) for node in self])) File "", line 1005, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) File "", line 966, in renderannotated return (context) File "", line 157, in render return (context) File "", line 167, in render return (context) File "", line 1005, in render return SafeString("".join([node.renderannotated(context) for node in self])) File "", line 1005, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) File "", line 966, in renderannotated return (context) File "", line 157, in render return (context) File "", line 167, in render return (context) File "", line 1005, in render return SafeString("".join([node.renderannotated(context) for node in self])) File "", line 1005, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) File "", line 966, in renderannotated return (context) File "", line 63, in render result = (context) File "", line 1005, in render return SafeString("".join([node.renderannotated(context) for node in self])) File "", line 1005, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) File "", line 966, in renderannotated return (context) File "", line 63, in render result = (context) File "", line 1005, in render return SafeString("".join([node.renderannotated(context) for node in self])) File "", line 1005, in <listcompreturn SafeString("".join([node.renderannotated(context) for node in self])) File "", line 966, in renderannotated return (context) File "", line 45, in render return super().render(context) File "", line 258, in render dict = (resolvedargs, *resolvedkwargs) File "", line 326, in resultlist headers = list(resultheaders(cl)) File "", line 90, in resultheaders text, attr = labelforfield( File "", line 362, in labelfor_field raise AttributeError(message) AttributeError: Unable to lookup 'choice' on Question or QuestionAdmin [24/Apr/2023 15:43:32] "GET /admin/polls/question/ HTTP/1.1" 500 I suggest that error E108 be updated to cover this case as well For reproduce see github


Reduce Add/RemoveIndex migration operations. Description We should reduce AddIndex/RemoveIndex operations when optimizing migration operations.


Squashing migrations with Meta.indextogether -indexes transition should remove deprecation warnings. Description Squashing migrations with Meta.indextogether -Meta.indexes transition should remove deprecation warnings. As far as I'm aware, it's a 4.2 release blocker because you cannot get rid of the index_together deprecation warnings without rewriting migrations, see comment.


dbshell management command can crash if DATABASES["name"] is a PosixPath Description With a sqlite database configured in like this: DATABASES = { "default": { "ENGINE": "", "NAME": BASEDIR / "db.sqlite3", } } exiting dbshell using Control - d sometimes results in this exception: Traceback (most recent call last): File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 30, in handle (options["parameters"]) File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 28, in runshell (args, env=env, check=True) File "", line 571, in run raise CalledProcessError(retcode, , subprocess.CalledProcessError: Command '['sqlite3', PosixPath('/home/ubuntu/planning-poker/db.sqlite3')]' returned non-zero exit status 1. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/ubuntu/planning-", line 30, in <modulemain() File "/home/ubuntu/planning-", line 26, in main executefromcommandline() File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 442, in executefromcommandline utility.execute() File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 436, in execute self.fetchcommand(subcommand).runfromargv() File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 412, in runfromargv self.execute(args, cmd_options) File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 458, in execute output = (args, options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/virtualenv/lib/python3.11/site-", line 44, in handle " ".join(e.cmd), ^^^^^^^^^^^^^^^ TypeError: sequence item 1: expected str instance, PosixPath found coercing each item in e.cmd to string should fix this.


forceinsert flag is not passed when saving parents on inherited models. Description We're using non-abstract model inheritance (table per model class) and issuing our own primary keys. When saving we pass forceinsert=True to prevent the extra UPDATE statement that precedes the INSERT. The forceinsert flag is respected on the child table but not on the parent. So given: class ParentModel(models.Model): id = models.BigIntegerField(primarykey=True) class ChildModel(ParentModel): pass ChildModel(id=1).save(forceinsert=True) We'll see queries: UPDATE appparentmodel (no rows affected) INSERT appparentmodel INSERT appchildmodel This is because Model.savebase doesn't pass forceinsert along to Model.saveparents, and onto Model.savetable. Doing so would prevent the extra UPDATE and respect the spirit of the forceinsert feature. This is a change I've made locally and seems to work / is pretty straightforward. I'm curious though if there's intent behind not carrying the forceinsert functionality through for parents. I couldn't find any discussion on the topic. For context about why this is particularly problematic in our case -- we're using MySQL w/ Innodb and Innodb will take out different exclusive locks for the UPDATE and INSERT in question -- so if you're creating new ChildModel instances in parallel you can get deadlocks when multiple threads issue query and then need a lock with insert intention in order to get .


no longer called for JSONField Description (last modified by Julie Rymer) Hello, I just upgraded from django 4.1 to 4.2 and I have a custom JSONField with a getprepvalue() override that stopped working. After searching a bit, I saw that was because JSONField.getprepvalue() is no longer called in 4.2 (). I think this issue need a resolution either: JSONField should call getprepvalue() like all other fields type, because this is the method that the documentation tell us to override in custom fields. Otherwise we need to override getdbprepvalue() which is heavier and does not have the same purpose. I think simply replacing (value, self.encoder) with (self.getprepvalue(value), self.encoder) in JSONField.getdbprepvalue() would fix this If there is a good reason to no longer call getprepvalue(), this exception for JSONField should be clearly documented in custom getprepvalue() doc. It should also be to Backwards incompatible changes in 4.2 release note because I got stuck with this issue with no warning when migrating. PS: seems to be related but in fact is about Django 3.2 so it is not the current issue


makemigrations --update should respect the --name option. Description (last modified by David Sanders) This may be intentional behaviour but the docs don't mention this so creating a ticket to update docs or correct behaviour: If you create a migration with a custom name: $ makemigrations --name foo Migrations for 'updaterename': Create model Foo then running --update will change the name "foo" to the autogenerated one based on the operations: $ makemigrations --update Migrations for 'updaterename': Create model Foo Deleted My opinion is that it shouldn't as it violates the principle of least astonishment even though the --name argument wasn't supplied. EDIT: This is my first time using --update and here are a few other observations which could indicate that it requires broader discussion: It doesn't utilise the --name argument so it's impossible to customise the name during --update It'd be nice to provide --no-optimize option to --update, here's my use-case: 3-step non-null field addition. After doing nullable step 1, elidable data migration step 2, I want to merge the step 3 non-null update into the migration but --update optimizes this into a single step. Perhaps --update requires a rethink?


Template filter should not escape the joining string if is Description Consider the following template code snippet: {% autoescape off %} {{ somelist|join:somevar }} {% endautoescape %} in this case, the items inside somelist will not be escaped (matching the expected behavior) but somevar will forcibly be escaped. From the docs for autoescape or join I don't think this is expected behavior. The following testcase illustrates what I think is a bug in the join filter (run inside the templatetests/filtertests folder): from django.template.defaultfilters import escape from import SimpleTestCase from ..utils import setup class RegressionTests(SimpleTestCase): '{{ somelist|join:somevar }}'}) def testjoin01(self): somelist = ["<pHello World!</p", "beta & me", "beta & me


New template filter Description Following , and after some conversations within the security team, it seems appropriate to provide a new template filter escapeseq which would be to escape what safeseq is to safe. An example of usage would be: {{ somelistjoin:"," }} where each item of somelist is escaped before applying the join operation. This usage makes sense in a context where autoescape is off.


Case-When aggregation over aggregated fields doesn't work since 4.2 Description (last modified by Denis Roldán) This same QuerySet was working on 4.1.X and prior versions and stopped working on 4.2.X: annotatedusers = usersqs.annotate( totalthings=Subquery( (userid=OuterRef("pk")).annotate( totalobjs=F("total") ).values("totalobjs") ) ) annotatedusers.aggregate( sumtotalobjs=Sum("totalthings"), avgconversionrate=Case( When( sumtotalobjs=0, then=0, ), default=Round( (Sum("sumtotalconfirmedobjs") / Sum("sumtotalobjs")) 100, 2 ), outputfield=FloatField(), ) ) As you can see sumtotalobjs is an aggregated field that is also used on a second field to calculate the conversion rate. To avoid a zero division problem, we were using a Case-When clause over that field. It works well on any 4.1 and prior versions but stopped working since 4.2, raising a FieldError like: Cannot resolve keyword 'sumtotalobjs' into field The bug is reproducible with an extra test on the django aggregation test suite: def testreferencedgroupbyaggregationoverannotation(self): totalbooksqs = ( (authorspk=OuterRef("pk")) .orderby() .values("pk") .annotate(total=Count("pk")) .values("total") ) annotatedauthors = Author.objects.annotate( totalbooks=Subquery(totalbooksqs.annotate( totalbooks=F("total") ).values("totalbooks")), totalbooksa=Subquery(( nameistartswith="a" ).annotate( totalbooksa=F("total") ).values("totalbooksa")), ).values( "pk", "totalbooks", "totalbooksa", ).orderby("-totalbooks") totals = annotatedauthors.aggregate( sumtotalbooks=Sum("totalbooks"), sumtotalbooksa=Sum("totalbooksa"), aovertotalrate=Case( When( sumtotalbooks=0, then=0, ), default=Round( (Sum("totalbooksa") / Sum("totalbooks")) 100, 2 ), outputfield=FloatField(), ), ) self.assertEqual(totals['sumtotalbooks'], 3) self.assertEqual(totals['sumtotalbooksa'], 0) self.assertEqual(totals['aovertotalrate'], 0) Thanks for the support!


Allow to override table2model conversion for inspectdb command Description When implementing a custom inspectdb command by inheriting from inspectdb.Command users should be able to override the table name to model conversion logic. Public method for column names already exists:  This would allow overriding the default behaviour of command in a custom inspectdb, for example when table names are already PascalCase (i.e. schema.FooBarBuzz), etc. With default implementation, it will be converted into class SchemaFoobarbuzz(models.Model). Proposed PR:


Querying for decimals larger than maxdigits crashes on SQLite Description Introduced in: Refs -- Improved adapting DecimalField values to decimal. Description: I have DecimalField with maxdigits=4. Previously, querying for something larger (e.g. ) would execute the sql and return ObjectNotFound. Now, in 4.2, it throws a decimal.InvalidOperation error, as it tries to quantize the value to have 4 digits. I understand that it doesn't make sense to query for a larger number, but the error that occurs was pretty confusing to me. Also, it is not as easy to check in my application, because I don't have easy access to the maxdigits parameter of the field. In my opinion, the backend should either accept larger values and always return "not found", or the error should be more descriptive, so that it can be caught specifically. Testcase: placed in tests/backends folder and used for git bisect import decimal from import models from import TestCase class DecimalModel(models.Model): decfield = models.DecimalField(decimalplaces=0, maxdigits=4) class InvalidDecimalQuery(TestCase): def testinvaliddecimalquery(self): try: (decfield='') except decimal.InvalidOperation: ("Too large decimal query caused exception.") except DecimalModel.DoesNotExist: pass Stacktrace: Traceback (most recent call last): File "lib/python3.10/site-", line 87, in managermethod return getattr(self.getqueryset(), name)(args, *kwargs) File "lib/python3.10/site-", line 633, in get num = len(clone) File "lib/python3.10/site-", line 380, in len self.fetchall() File "lib/python3.10/site-", line 1881, in fetchall self.resultcache = list(self.iterableclass(self)) File "lib/python3.10/site-", line 91, in iter results = compiler.executesql( File "lib/python3.10/site-", line 1547, in executesql sql, params = () File "lib/python3.10/site-", line 762, in assql self.compile() if is not None else ("", []) File "lib/python3.10/site-", line 544, in compile sql, params = (self, self.connection) File "lib/python3.10/site-", line 145, in assql sql, params = compiler.compile(child) File "lib/python3.10/site-", line 544, in compile sql, params = (self, self.connection) File "lib/python3.10/site-", line 357, in assql return super().assql(compiler, connection) File "lib/python3.10/site-", line 225, in assql rhssql, rhsparams = self.processrhs(compiler, connection) File "lib/python3.10/site-", line 126, in processrhs return self.getdbpreplookup(value, connection) File "lib/python3.10/site-", line 254, in getdbpreplookup else [getdbprepvalue(value, connection, prepared=True)], File "lib/python3.10/site-", line 1761, in getdbprepvalue return ( File "lib/python3.10/site-", line 574, in adaptdecimalfieldvalue return utils.formatnumber(value, maxdigits, decimalplaces) File "lib/python3.10/site-", line 304, in formatnumber value = value.quantize( decimal.InvalidOperation: [<class 'decimal.InvalidOperation']


ModelAdmin: Error message for readonlyfields's check does not include the field name Description When subclassing a ModelAdmin, the current error message for the readonlyfields would indicate the index of the value at fault but it will not include the field's name (from the test suite): The value of 'readonlyfields[0]' is not a callable, an attribute of 'CityInline', or an attribute of 'adminchecks.City'. Other fields like listeditable, rawidfields, listdisplay, etc. would also include this value: The value of 'listeditable[0]' refers to 'originalrelease', which is not contained in 'listdisplay'. It would be good if we can unify this and include the field name in the readonlyfields checks, it also eases the understanding of the error when using the framework.


On databases lacking XOR, Q(…) ^ Q(…) ^ Q(…) wrongly interpreted as exactly-one rather than parity Description On databases that don’t natively support XOR, such as PostgreSQL, Django generates incorrect fallback SQL for Q(…) ^ Q(…) ^ Q(…) with more than 2 arguments. The correct interpretation, and the interpretation of databases natively supporting XOR (e.g. MySQL), is that a ^ b ^ c is true when an odd number of the arguments are true. But Django’s fallback interpretation is that a ^ b ^ c is true when exactly one argument is true: from import Q from import Client (Q(id=37)).count() 1 (Q(id=37) ^ Q(id=37)).count() 0 (Q(id=37) ^ Q(id=37) ^ Q(id=37)).count() 0 (Q(id=37) ^ Q(id=37) ^ Q(id=37) ^ Q(id=37)).count() 0 (Q(id=37) ^ Q(id=37) ^ Q(id=37) ^ Q(id=37) ^ Q(id=37)).count() 0 (Expected: 1, 0, 1, 0, 1.) This was introduced in .


Right() function on Oracle and SQLite returns improper value when the length is zero. Description Hi I have found that the RIGHT database function on Oracle returns the whole string instead of empty string when the given length is 0. You can't explicitly give 0 to the RIGHT function, but it may be computed by the database. Basic example (you can use any model with a CharField): from import Right, Length MyModel.objects.annotate(suffix=Right("foo", Length("foo") - Length("foo"))) On PostgreSQL this will return an empty string under the field suffix, but on Oracle this will return the whole contents of the field foo. This is because Django uses the SUBSTR function on Oracle by multiplying the given length value by -1 and giving it as a position argument. I think it is not intended behavior and it should return the empty string as PostgreSQL does. Or at least be documented as a Note in the Right function documentation.


Diamond inheritance causes duplicated PK error when creating an object, if the primary key field has a default. Description Hi, I'm not sure if this is a bug or an unsupported feature. But I looked into the source code and now have a pretty good understanding of what is happening. My business code uses a diamond shape inheritance to model different types of user posts: UserPost, ImagePost, VideoPost, and MixedContentPost. The inheritance goes like this: both ImagePost and VideoPost extend from UserPost, and the MixedContentPost inherits from ImagePost and VideoPost. All of them are concrete models I read the doc and expected it to work, similar to the example class Piece(models.Model): pass class Article(Piece): articlepiece = models.OneToOneField(Piece, ondelete=models.CASCADE, parentlink=True) ... class Book(Piece): bookpiece = models.OneToOneField(Piece, ondelete=models.CASCADE, parentlink=True) ... class BookReview(Book, Article): pass However, I found out that the doc's example only works when these models use a primary key field that does not have a default. In my case, we are using a UUIDField as the primary key with a default of uuid4. Trying to create a BookReview in our case, causes a UNIQUE constraint failed error, because django tries to create the Piece object twice, with the same uuid. The same behavior is found if I used a AutoField on Piece, with a custom default function, such as id = 99 def increment(): global id id += 1 return id This default function makes no sense in practice, but I just use it here to illustrate the root cause of the problem: The savetable method in has a like this: # Skip an UPDATE when adding an instance and primary key has a default. if ( not raw and not forceinsert and and and is not NOTPROVIDED ): forceinsert = True When a default is not present, which is the case of the documentation example, Django will first insert the first instance of the Piece object, and then for the second one, since forceinsert is False, savetable tries an update and goes through. Therefore there is not duplicate. However, if a default is present, then the second Piece becomes an insertion as well (because and is not NOTPROVIDED, forceinsert is True). This causes a duplication on the primary key. On the other hand, insertparent does an in-order traversal calling savetable on each node, so even in the no-default pk case, it is calling a redundant update on the root node after the insertion.. So which function is at fault? The source code savetable assumes that if you are calling it with a default pk then you can skip an update. This assumption looks weird to me: why only when there IS a default pk you can skip update? Why not just skip update as long as we know we are inserting? (via ) Is it just to make it special so that AutoField works? If savetable's responsibility is to try updating before inserting, except when the params force it to do an update or insert, then it shouldn't override that behavior by this self-assumeption within it. I think the solution is to simply move the check to savebase. And don't do this check in saveparents. Like this: def savebase( self, raw=False, forceinsert=False, forceupdate=False, using=None, updatefields=None, ): """ Handle the parts of saving which should be done only once per save, yet need to be done in raw saves, too. This includes some sanity checks and signal sending. The 'raw' argument is telling savebase not to save any parent models and not to do any changes to the values before save. This is used by fixture loading. """ using = using or router.dbforwrite(, instance=self) assert not (forceinsert and (forceupdate or updatefields)) assert updatefields is None or updatefields cls = origin = # Skip proxies, but keep the origin as the proxy model. if cls = meta = if not meta.autocreated: ( sender=origin, instance=self, raw=raw, using=using, updatefields=updatefields, ) # A transaction isn't needed if one query is issued. if meta.parents: contextmanager = (using=using, savepoint=False) else: contextmanager = transaction.markforrollbackonerror(using=using) with contextmanager: parentinserted = False if not raw: parentinserted = self.saveparents(cls, using, updatefields) # Skip an UPDATE when adding an instance and primary key has a default. if ( not raw and not forceinsert and and and is not NOTPROVIDED ): forceinsert = True updated = self.savetable( raw, cls, forceinsert or parentinserted, forceupdate, using, updatefields, ) # Store the database on which the object was saved = using # Once saved, this is no longer a to-be- instance. = False # Signal that the save is complete if not meta.autocreated: ( sender=origin, instance=self, created=(not updated), updatefields=updatefields, raw=raw, using=using, ) savebase.alters_data = True I have never contributed to Django before. If you think I'm right on this one I'll look into creating a PR.


() doesn't work with selectrelated() on a reverse OneToOneField relation. Description On Django 4.2 calling only() with selectrelated() on a query using the reverse lookup for a OneToOne relation does not generate the correct query. All the fields from the related model are still included in the generated SQL. Sample models: class Main(models.Model): mainfield1 = models.CharField(blank=True, maxlength=45) mainfield2 = models.CharField(blank=True, maxlength=45) mainfield3 = models.CharField(blank=True, maxlength=45) class Secondary(models.Model): main = models.OneToOneField(Main, primarykey=True, relatedname='secondary', ondelete=models.CASCADE) secondaryfield1 = models.CharField(blank=True, maxlength=45) secondaryfield2 = models.CharField(blank=True, maxlength=45) secondaryfield3 = models.CharField(blank=True, maxlength=45) Sample code: Main.objects.selectrelated('secondary').only('mainfield1', 'secondary_secondaryfield1') Generated query on Django 4.2.1: SELECT "bugtestmain"."id", "bugtestmain"."mainfield1", "bugtestsecondary"."mainid", "bugtestsecondary"."secondaryfield1", "bugtestsecondary"."secondaryfield2", "bugtestsecondary"."secondaryfield3" FROM "bugtestmain" LEFT OUTER JOIN "bugtestsecondary" ON ("bugtestmain"."id" = "bugtestsecondary"."mainid") Generated query on Django 4.1.9: SELECT "bugtestmain"."id", "bugtestmain"."mainfield1", "bugtestsecondary"."mainid", "bugtestsecondary"."secondaryfield1" FROM "bugtestmain" LEFT OUTER JOIN "bugtestsecondary" ON ("bugtestmain"."id" = "bugtestsecondary"."main_id")


Fields’ help text and errors should be associated with input Description (last modified by Thibaud Colas) With Django’s default field rendering, all field errors are rendered as a list above the field’s label, and help text is rendered after the field’s form element. Example with asp: <ul class="errorlist"<liThis field is required.</li</ul<p<label for="iddurationrequired"Duration required:</label<input type="text" name="durationrequired" required="" id="iddurationrequired"<span class="helptext"Help</span</pOne problem for screen reader users is that the association between the errors and the field, and between the help text and the field, is only communicated visually. This is a failure of either WCAG 2.1 level A SC 1.3.1: Info and Relationships, or SC 3.3.2: Labels or Instructions. More importantly, it just makes it harder than necessary for screen reader users to make use of help text, and to identify error messages. The fix is relatively straightforward – using aria-describedby, as documented in the (non-normative) ARIA1 Using the aria-describedby property to provide a descriptive label for user interface controls technique. Here is another well-known accessibility-oriented UI library that implements this technique: GOV.UK design system – text input with error message. Here is what implementing aria-describedby would look like in the same example as above: <div class="errorlist" id="iddurationrequirederrorlist"<pThis field is required.</p</div<p<label for="iddurationrequired"Duration required:</label<input type="text" name="durationrequired" required="" id="iddurationrequired" aria-describedby="iddurationrequirederrorlist iddurationrequiredhelptext"<span class="helptext" id="iddurationrequired_helptext"Help</span</pWe have additional id attributes, aria-describedby, and errorlist is no longer a <ul. Result in VoiceOver: Unfortunately I tried to have this with the errorlist kept as a ul, but it wasn’t announced by VoiceOver. I haven’t heard of this limitation before so am not sure why that might be the case – I’d appreciate others taking a look if possible.


Serialization of m2m relation fails with custom manager using selectrelated Description Serialization of many to many relation with custom manager using selectrelated cause FieldError: Field cannot be both deferred and traversed using selectrelated at the same time. Exception is raised because performance optimalization . Workaround is to set simple default manager. However I not sure if this is bug or expected behaviour. class TestTagManager(Manager): def getqueryset(self): qs = super().getqueryset() qs = qs.selectrelated("master") # follow master when retrieving object by default return qs class TestTagMaster(models.Model): name = models.CharField(maxlength=120) class TestTag(models.Model): # default = Manager() # solution is to define custom default manager, which is used by RelatedManager objects = TestTagManager() name = models.CharField(maxlength=120) master = models.ForeignKey(TestTagMaster, ondelete=models.SETNULL, null=True) class Test(models.Model): name = models.CharField(maxlength=120) tags = models.ManyToManyField(TestTag, blank=True) Now when serializing object from import serializers from import TestTag, Test, TestTagMaster tagmaster = (name="master") tag = (name="tag", master=tagmaster) test = (name="test") (tag) () serializers.serialize("json", [test]) Serialize raise exception because is not possible to combine selectrelated and only. File "/opt/venv/lib/python3.11/site-", line 134, in serialize s.serialize(queryset, **options) File "/opt/venv/lib/python3.11/site-", line 167, in serialize self.handlem2mfield(obj, field) File "/opt/venv/lib/python3.11/site-", line 88, in handlem2mfield self.current[] = [m2mvalue(related) for related in m2miter] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 88, in <listcompself.current[] = [m2mvalue(related) for related in m2miter] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 516, in iterator yield from iterable File "/opt/venv/lib/python3.11/site-", line 91, in iter results = compiler.executesql( ^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 1547, in executesql sql, params = () ^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 734, in assql extraselect, orderby, groupby = self.presqlsetup( ^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 84, in presqlsetup self.setupquery(withcolaliases=withcolaliases) File "/opt/venv/lib/python3.11/site-", line 73, in setupquery , self.klassinfo, self.annotationcolmap = self.getselect( ^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 279, in getselect relatedklassinfos = self.getrelatedselections(select, selectmask) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 1209, in getrelatedselections if not selectrelateddescend(f, restricted, requested, selectmask): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-", line 347, in selectrelateddescend raise FieldError( Field cannot be both deferred and traversed using select_related at the same time.


Deprecate formathtml calls without args or kwargs Description (last modified by Michael Howitz) In my experience, a common misuse of formathtml is to format the HTML before calling it: formathtml(f"<i{name}</i") This makes it act like marksafe, allowing data through without escaping. It provides a false sense of security since formathtml is meant to be the "safe way". I propose we deprecate calls to formathtml that don’t pass args or kwargs, and eventually raise a TypeError for such cases. (Following improvement to format_html docs in .)


Django Admin with Inlines not using UUIDField default value Description (last modified by Joseph Metzinger) Hello, I am a long time django user, first time bug reporter, so please let me know if I need to do anything else to help get this bug fixed :) I am using Django 3.1.3 and python 3.8.5 and have cerated a toy project to illustrate the bug. I have the following models: class UUIDModel(models.Model): pkid = models.BigAutoField(primarykey=True, editable=False) id = models.UUIDField(default=uuid.uuid4, editable=False, unique=True) class Meta: abstract = True class Thing(UUIDModel): name = models.CharField(maxlength=191) class SubThing(models.Model): name = models.CharField(maxlength=191) thing = models.ForeignKey( 'bugapp.Thing', tofield='id', ondelete = models.CASCADE, relatedname='subthings', ) And the following file: class SubThingInline(admin.StackedInline): model = SubThing class ThingAdmin(admin.ModelAdmin): list_display = ('name',) ordering = ('pkid',) inlines = (SubThingInline,) When logging into the admin, if you delete all of the entries for "subthings", add a name, and save the model, it will work. As soon as you try to add a subthing alongside the main Thing, it fails with the following exception:  It shows that the value of "id" in the Thing model is being set to null. I believe this is a bug in django. Thanks!


Creating objects with nested MTI crashes. Description (last modified by Mariusz Felisiak) Checking PR I noticed that creating objects with more complicated multi-table inheritance crashes. For example: diff --git index .. a b class ItalianRestaurant(Restaurant): servesgnocchi = models.BooleanField(default=False) 109class ItalianRestaurantManyParents(ItalianRestaurant, Place): 110 pass 111 112 109113class Supplier(Place): customers = models.ManyToManyField(Restaurant, relatedname="provider") diff --git index .. a b from .models import ( 1414 GrandChild, 1515 GrandParent, 1616 ItalianRestaurant, 17 ItalianRestaurantManyParents, 1718 MixinModel, 1819 Parent, 1920 ParkingLot, … … class ModelInheritanceTests(TestCase): # accidentally found). self.assertSequenceEqual((), []) 153 def testcreatediamondmticommonparents(self): 154 with self.assertNumQueries(4): 155 ( 156 name="Ristorante Miron", 157 address="1234 W. Ash", 158 ) 159 def testupdateparentfiltering(self): """ Updating a field of a model subclass doesn't issue an UPDATE crashes with: File "", line 155, in testcreatediamondmticommonparents ( File "", line 87, in managermethod return getattr(self.getqueryset(), name)(args, *kwargs) File "", line 650, in create (forceinsert=True, ) File "", line 814, in save self.savebase( File "", line 876, in savebase parentinserted = self.saveparents(cls, using, updatefields) File "", line 928, in saveparents setattr(self, field.attname, self.getpkval(parent._meta)) AttributeError: 'OneToOneField' object has no attribute 'attname'


Add system check for filterhorizontal/filtervertical on ManyToManyFields with intermediary models. Description Hi team, I'm a huge fan of Django and have been using it since 0.95 but I stumbled over this one. Neither of  and  call out the requirement to not use ManyToManyField(through="") In the same way:  doesn't call out the consequence that filterhorizontal and filtervertical will stop working if one goes down the pathway of: ManyToManyField(through="") I just wasted half a day chasing this down.


Apps.clearcache() does not clear getswappablesettingsname cache. Description We use apps.clearcache() in django-stubs to be able to reset the previous state on consequential mypy runs. Code:  But, looks like we don't clear all the object's cache this way, because getswappablesettingsname (which is a functools.lrucachewrapper) is not cleared. I think that this is not correct. .clearcache doc states: Clear all internal caches, for methods that alter the app registry. Looks like that is not the case. I propose to add: self.getswappablesettingsname.cacheclear() line to def clearcache. If others agree, I will make a PR. Original discussion:


Add a system check for unmatched URL paths' < / Description Currently, unmatched angle brackets are silently ignored: path('<int:question_id/vote/', , name='vote'), This leads to a frustrating debugging experience, where a 404 or NoReverseMatch could occur and only the eagle-eyed could spot why. Similar to the ^/$ system check in , I propose a new check in () for unmatched < or . It's rather unlikely those characters are legitimately used in URLs. (Inspired by a new user making this mistake and asking for help on the forum.)


Deleting objects after searching related many to many field crashes the admin page Description Minimal reproduction: class Post(models.Model): title = models.String(...) authors = models.ManyToMany("User", ...) class User(models.Model): email = models.String(...) class PostAdmin(admin.ModelAdmin): searchfields = ("title", "authorsemail") then opening the admin site, opening the post page that contains only one post (any title and author assigned) and entering a search term (e.g the first 2 characters of the title), selecting the post and then using the delete action results in an Internal Sever Error 500 with an error/stack-trace: Internal Server Error: /admin/post/post/ Traceback (most recent call last): File "", line 47, in inner response = getresponse(request) File "", line 181, in getresponse response = wrappedcallback(request, callbackargs, callbackkwargs) File "", line 616, in wrapper return self.adminsite.adminview(view)(args, kwargs) File "", line 130, in wrappedview response = viewfunc(request, args, kwargs) File "", line 44, in wrappedviewfunc response = viewfunc(request, args, kwargs) File "", line 241, in inner return view(request, args, kwargs) File "", line 43, in wrapper return boundmethod(args, kwargs) File "", line 130, in wrappedview response = viewfunc(request, args, *kwargs) File "", line 1737, in changelistview response = self.responseaction(request, queryset=cl.getqueryset(request)) File "", line 1406, in responseaction response = func(self, request, queryset) File "", line 45, in deleteselected modeladmin.deletequeryset(request, queryset) File "", line 1107, in deletequeryset () File "", line 728, in delete raise TypeError('Cannot call delete() after .distinct().') TypeError: Cannot call delete() after .distinct(). "POST /admin/post/post/?q=my HTTP/1.1" 500 I can confirm that pip install django==3.1.8 fixes the error, and after having a look at the diff between stable/3.2.x and 3.1.8, I suspect the "regression" comes about from the work done on preserving the filters on delete or something along those lines - I haven't done a thorough investigation yet. Presumably .distinct() is being called because of the search involving the many to many field. I am using a Postgres database.


Allow returning IDs in QuerySet.bulkcreate() when updating conflicts. Description Currently, when using bulkcreate with a conflict handling flag turned on (e.g. ignoreconflicts or updateconflicts), the primary keys are not set in the returned queryset, as documented in bulkcreate. While I understand using ignoreconflicts can lead to PostgreSQL not returning the IDs when a row is ignored (see this SO thread), I don't understand why we don't return the IDs in the case of updateconflicts. For instance: MyModel.objects.bulkcreate([MyModel(...)], updateconflicts=True, updatefields=[...], uniquefields=[...]) generates a query without a RETURNING part: INSERT INTO "mymodel" (...) VALUES (...) ON CONFLICT(...) DO UPDATE ... If I append the RETURNING clause, the query is indeed valid and the ID is returned (checked with PostgreSQL). I investigated a bit and this in Django source is where the returningfields gets removed. I believe we could discriminate the cases differently so as to keep those returningfields in the case of updateconflicts. This would be highly helpful when using bulkcreate as a bulk upsert feature.


BoundField.aswidget() ignores aria-describedby in attrs argument Description (last modified by Sage Abdullah) BoundField.aswidget() ignores aria-describedby that is passed in the attrs argument. Use case: In Wagtail, we have our own mechanism for rendering form fields (called "Panels") that is built on top of Django's Forms API. We use BoundField.aswidget() in our rendering process to render only the form field's widget, while its help text element is rendered separately via our Panels API with a custom markup (including HTML id) that conforms to our design system. We've been passing additional attributes to BoundField.aswidget() to improve accessibility, including aria-describedby, to establish the relationship between the field widget rendered by Django and our help text element. As of , Django automatically adds aria-describedby to form fields with a helptext. The logic in buildwidgetattrs() (used by aswidget()) checks for an existing aria-describedby in the field's widget's attrs before automatically generating one. However, it does not take into account the possibility of an existing aria-describedby in the attrs argument. A workaround on Wagtail's side could be to modify the widget's attrs before using aswidget(), but this is not ideal as the widget is customisable by the developer and we would have to copy the widget instance to avoid modifying the existing widget instance (which might be shared across form fields). I believe Django should check for aria-describedby in attrs first, before falling back to and the auto-generated value. Test case: class BoundFieldTests(SimpleTestCase): def testaswidgetwithcustomariadescribedby(self): class TestForm(Form): data = CharField(helptext="Some help text") form = TestForm({"data": "some value"}) self.assertHTMLEqual( form["data"].aswidget(attrs={"aria-describedby": "customhelptextid"}), """ <input type="text" name="data" value="some value" aria-describedby="customhelptextid" required id="iddata"""", ) Patch: diff --git index .. --- +++ -290,10 +290,11 class BoundField(RenderableFieldMixin): # If a custom aria-describedby attribute is given and helptext is # used, the custom aria-described by is preserved so user can set the # desired order. if customariadescribedbyid := ("aria-describedby"): attrs["aria-describedby"] = customariadescribedbyid elif and self.idforlabel: attrs["aria-describedby"] = f"{self.idforlabel}helptext" if not ("aria-describedby"): if customariadescribedbyid := ("aria-describedby"): attrs["aria-describedby"] = customariadescribedbyid elif and self.idforlabel: attrs["aria-describedby"] = f"{self.idforlabel}_helptext" return attrs Happy to submit a PR if this is accepted.


Migration serializer for sets results in non-deterministic order. Description We are using quite a lot of complex indextogether / uniquetogether constraints on our models, and the output in the generated migrations is flip-flopping all the time like follows, causing spurious diffs in our checkouts: migrations.AlterUniqueTogether( uniquetogether={("tenant", "dealer"), ("tenant", "order")}, uniquetogether={("tenant", "order"), ("tenant", "dealer")}, This is happening because these constraints are normalized to sets internally in the ModelState, which kind of makes sense, but unfortunately set iteration order (unlike dicts!) is unstable in Python 3 due to hash randomization. However, migrations serializer doesn't have any special facilities for ensuring stable output for sets and this is what causes annoying diffs for us all the time. I suggest to add a trivial serializer specifically for unordered sequences which ensures stable output no matter the iteration order. Stability can be achieved by sorting elements in the set by their string representation. This only affects the writer output, and doesn't interfere with the rest of Django in any way, so this change only improves developer experience, but has no effect on the performance and/or reliability. I hope that even though it's apparently not a major problem for most users you would still accept the fix to ensure stable migration writer output for the rest of us.


Cannot use aggregate over window functions since 4.2 Description (last modified by younes-chaoui) After upgrading to Django 4.2, I encountered an exception when executing ORM queries that involve aggregates over Window functions. The specific error was aggregate function calls cannot contain window function calls Dependencies : psycopg2 version: 2.9.3 django version: 4.2.3 PostgreSQL version: 13.4 Example Code: queryset = queryset.annotate( cumulDJR=Coalesce(Window(Sum("DJR"), orderby=F("date").asc()), 0.0) ) aggregate = queryset.aggregate( DJRtotal=Sum("DJR"), cumulDJRtotal=Sum("cumulDJR") )


Class methods from nested classes cannot be used as Field.default. Description (last modified by Mariusz Felisiak) Given the following model: class Profile(models.Model): class Capability(models.TextChoices): BASIC = ("BASIC", "Basic") PROFESSIONAL = ("PROFESSIONAL", "Professional") def default(cls) -list[str]: return [cls.BASIC] capabilities = ArrayField( models.CharField(choices=Capability.choices, maxlength=30, blank=True), null=True, default=Capability.default ) The resulting migration contained the following: # ... migrations.AddField( modelname='profile', name='capabilities', (basefield=models.CharField(blank=True, choices=[('BASIC', 'Basic'), ('PROFESSIONAL', 'Professional')], maxlength=30), , null=True, size=None), ), # ... As you can see, migrations.AddField is passed as argument "default" a wrong value "", which leads to an error when trying to migrate. The right value should be "".


Unknown django-admin command raises "settings not configured" when no project exists Description When you try to execute a command that does not exist outside a Django project (e.g. createproject instead of startproject...), Django raise a ImproperlyConfigured that does not fit with the real error: (testdjango):~/testdjango$ django- not-a-real-command test Traceback (most recent call last): File "/root/testdjango/bin/django-", line 5, in <modulemanagement.executefromcommandline() File "/root/testdjango/local/lib/python2.7/site-", line 338, in executefromcommandline utility.execute() File "/root/testdjango/local/lib/python2.7/site-", line 330, in execute self.fetchcommand(subcommand).runfromargv() File "/root/testdjango/local/lib/python2.7/site-", line 182, in fetchcommand settings.INSTALLEDAPPS File "/root/testdjango/local/lib/python2.7/site-", line 48, in getattr (name) File "/root/testdjango/local/lib/python2.7/site-", line 42, in setup % (desc, ENVIRONMENTVARIABLE)) Requested setting INSTALLEDAPPS, but settings are not configured. You must either define the environment variable DJANGOSETTINGS_MODULE or call settings.configure() before accessing settings. The expected output is: (testdjango):~/testdjango$ django- not-a-real-command test Unknown command: 'not-a-real-command' Type 'django- help' for usage.


Set script prefix in () to allow its usage outside of requests Description The script prefix for doesn't get set to anything when being called through , because of course it doesn't know what that value should be. This is a problem if you're rendering views (or otherwise reversing urls) from a command (as one of my sites does to send emails). This is solvable by calling setscriptprefix from , but that feels kind of dirty since it's then about to be rewritten in the WSGI handler. I don't know what a good solution to this would be. Perhaps it would be nice to be able to set a global default script path somewhere that would then get incorporated into the default values of things like LOGIN_URL. Maybe just a note in the documentation would be good. It took me a while to figure out, because I haven't been able to find anything else about this online. (I guess that non-/ script paths are uncommon and reversing urls from is also uncommon, so both together are very uncommon.)


Allow auth machinery to be used without installing auth app Description Django projects should be able to use the auth machinery (like and ) without having in INSTALLEDAPPS See !topic/django-developers/2DzLBbk8w-w This ticket is for resolving the current issue in 1.9. I also want to write a test or two to prevent this issue from happening in the future.


diffsettings should allow to compare to another settings module (not only to default Django settings) Description Rationale: I use a hierarchy of settings modules, where imports imports I want to see what I customized, e.g. diffsettings --


makemigrations router.allowmigrate() calls for consistency checks use incorrect (applabel, model) pairs Description As reported in ticket:#comment:14, I makemigrations incorrectly calls allow_migrate() for each app with all the models in the project rather than for each app with the app's models. It broke the router I use because it was passing invalid combinations for shards since not all shards have the same models. [ PR]


Allow expressions in .filter() calls Description Expressions in filter calls will allow 3rd party apps to create query syntax extensions (for example .filter(F('some_field').lower() == 'anssi')) style). In addition, having the ability to use expressions everywhere unifies the ORM experience.


Add "unified diff" output mode to diffsettings management command Description Most people are used to unified diffs vs. the somewhat strange "###" formatting of diffsettings, so this adds an optional —output=unified.


Add nextpage to LoginView Description LogoutView has a nextpage attribute used to override settings.LOGOUTREDIRECTURL. It would be nice if LoginView had the same mechanism.


Improve program name for Django's command-line utility. Description By default, according to the documentation there are three ways to execute Django's command-line utility: $ django-admin <command[options] $ <command[options] $ python -m django <command[options] When executing the help command, however, the way these are rendered may be incorrect: $ django-admin help Type 'django-admin help <subcommand' for help on a specific subcommand. ... $ help Type ' help <subcommand' for help on a specific subcommand. ... $ python -m django help Type ' help <subcommand' for help on a specific subcommand. ... It should also be noted that will typically not work and is required: $ help bash: command not found On Windows it will work, but only if the file type is associated with the Python interpreter and not an editor. So, the issues here are: should be replaced by python -m django (or possibly use (sys.executable)) Should be replaced with on non-Windows platforms? What about Windows? What should happen with the option in the documentation?


Non-deterministic crash in () Description Hi, There's a logical error somewhere in Query.combine or friends. I have a pretty complicated query with many-to-many self-joins and ORs, and after the upgrade to Django 1.9 and Python 3 combine() sometimes, but not always, crashes with the assertion error assert set(()).intersection(set(())) == set() Inspecting the changemap, it does indeed contain a circular reference (BTW shouldn't you use "not" to test for an empty set rather than comparing with set()?). At first, I didn't understand how it could crash sometimes and sometimes not - we're talking about the same query being executed through a view. But when I examine changemap, its content does indeed change from reload to reload - I suppose this may have something to do with the order of dicts/sets the combine() algorithm is using. Here comes some code, I cut out a bunch of unused stuff, but otherwise it's the same: class Invoice(models.Model): customer = models.ForeignKey(Customer) datecreated = models.DateField(, dbindex=True) datesent = models.DateField(null=True, blank=True) datedue = models.DateField(null=True, blank=True) datepaid = models.DateField(null=True, blank=True) datecredited = models.DateField(null=True, blank=True) datecollect = models.DateField(null=True, blank=True) invoicetype = models.CharField(default="invoice", maxlength=32) reminders = models.ManyToManyField("Invoice", relatedname="remindedset", blank=True) remindercounter = models.IntegerField(null=True, blank=True) That's the model, and in the view: import datetime from import Q date = () invoices = ( Q(datecreatedlte=date), Q(datepaidgt=date) | Q(datepaid=None), Q(datecreditedgt=date) | Q(datecredited=None), customer=1, ) filteredinvoices = () notdue = Q(dateduegte=date) | Q(datedue=None) notremindedyet = ~Q(remindersdatecreatedlte=date) notcollected = Q(datecollectgt=date) | Q(datecollect=None) filteredinvoices |= (notdue, notcollected, datesentlte=date, invoicetype="invoice") filteredinvoices |= (notcollected, notremindedyet, datesentlte=date, dateduelt=date, invoicetype="invoice") for r in [1, 2, 3]: qs = (notcollected, remindersdatecreatedlte=date, remindersremindercounter=r, invoicetype="invoice") for i in range(r + 1, 3 + 1): qs = (~Q(remindersremindercounter=i) | Q(remindersremindercounter=i, remindersdatecreated_gt=date)) filteredinvoices |= qs I realize it's pretty complicated but I'm not sure what's essential and what's not. I hope someone knowledgeable of how combine() works can figure out why ordering in some cases matters.


Paginator just implement the iter function Description (last modified by Alex Gaynor) Right now, when you want to iter into all the pages of a Paginator object you to use the pagerange function. It would be more logical and naturel to use the normal python of doing that by implementing the iter function like that: def iter(self): for pagenum in self.pagerange: yield (pagenum)


Allow management command invocation to suggest commands for mistyped commands Description Too often I can't remember the full mgmt command name, but can remember a part of it. A little enhancement would save me time. Attaching a screenshot and a patch. Thanks.


Reorder management command arguments in --help output to prioritize command-specific arguments Description Currently if you run a custom management command with --help, you will get output that looks like: I have highlighted in yellow the useful information specific to the command that is not boilerplate. Notice that most of this yellow text is at the end of the output, with the boilerplate dominating what the user reads first. I propose reordering the options in the output so that the useful information is at the beginning rather than the end, so that it looks like the following: Discussion on django-developers: !topic/django-developers/PByZfN_IccE


Zero-width figure crashes libpng Bug summary Zero-width figure crashes libpng. This happens when using ` or saving to png. Code for reproduction Actual outcome Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: ubuntu / conda Matplotlib version: 3.0.2, conda 3.0.2-py37h54297110 same at 3.0.3-py37h54297110 libpng 1.6.35-hbc830470, same with 1.6.36-hbc83047 Apparently I broke "conda list" on my machine so getting all the versions seems a bit tricky.


Minor ticklabels are missing at positions of major ticks. <!--To help us understand and resolve your issue, please fill out the form to the best of your ability.--<!--You can feel free to delete the sections that do not apply.--Bug summary Minor ticklabels are missing at positions of major ticks. Code for reproduction Actual outcome The above code run with current master produces ! The minor ticklabels showing the hours are missing. Expected outcome The expected outcome would be the same as when running the code with matplotlib 3.0.2 or below: ! I would expect to see the hours throughout. Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: Win8 Matplotlib version: master Matplotlib backend (): any Python version: 3.6


returning erroneous value on strings of integers <!--To help us understand and resolve your issue, please fill out the form to the best of your ability.--<!--You can feel free to delete the sections that do not apply.--Bug summary returns on strings containing integers, even though these are invalid colours when passed to or equivalent. Code for reproduction <!--A minimum code snippet required to reproduce the bug. Please make sure to minimize the number of dependencies required, and provide any necessary plotted data. Avoid using threads, as Matplotlib is (explicitly) not thread-safe.--Actual outcome <!--The output produced by the above code, which may be a screenshot, console output, etc.--Expected outcome <!--A description of the expected outcome from the code snippet--<!--If this used to work in an earlier version of Matplotlib, please note the version it used to work on--I would expect either the call to successfully recognise these as colours, or the calls to return . The latter is what I would expect of these two. Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: Arch Linux Matplotlib version: 3.0.2 Matplotlib backend (): Qt5Agg Python version: Python 3.7.1 Jupyter version (if applicable): N/A Other libraries: 1.15.0 installed from . The example code runs without any errors on Windows, Python 3.6, 2.2.2. <!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda--<!--If you installed from conda, please specify which channel you used if not the default--


Non-sensical negative radial scale minimum autoset in polar plot When plotting a set of data on a polar plot, the default bottom ylimit might not be zero unexpectedly from the perspective of the user, resulting in confusion about the meaning of the plot, especially for a person (like me) unfamiliar with the concept of a polar plot where r=0 is not at the very center point of the plot. In a Jupyter Notebook (-0., 1.) ! I ran across this when plotting data and wondering if I had a bug in my analysis somewhere that was giving me a hole around the origin. It took me some time to figure out that the problem was simply in the axis scaling as I expected the version on the left (which seems sensible to me as the default) and not the version on the right which has a hole in the middle. Matplotlib version Operating system: Windows 10, also Ubuntu Linux Matplotlib version: 3.0.2 from pip Matplotlib backend (): inline Python version: 3.7, 3.6 Jupyter version (if applicable): JupyterLab 0.35.4


Remove()ing a shared axes prevents the remaining axes from using unit-provided formatters Consider One gets ! i.e. the call to prevented the axs[0] from acquiring the correct tick formatter and locator. Interestingly, using doesn't exhibit the same bug. Looks like the problem comes from where the call to (etc.), which basically call (to update the axis seen by the formatter) also make Matplotlib believe that we had a user-provided formatter (isDefaultmajloc = False, etc.) which should not be overridden by the unit framework. mpl master (ca. 3.0.2)


Tick mark color cannot be set on Axes3D As , the method does not change the color of tick marks on , only the color of tick labels. Several workarounds were proposed, and according to one comment, this used to work as expected in version 1.3.1. Here is code that tries to change the colors of all the axes but fails to get the tick marks:


hist() no longer respects range=... when density=True <!--To help us understand and resolve your issue, please fill out the form to the best of your ability.--<!--You can feel free to delete the sections that do not apply.--Bug summary <!--A short 1-2 sentences that succinctly describes the bug--Code for reproduction <!--A minimum code snippet required to reproduce the bug. Please make sure to minimize the number of dependencies required, and provide any necessary plotted data. Avoid using threads, as Matplotlib is (explicitly) not thread-safe.--Actual outcome <!--The output produced by the above code, which may be a screenshot, console output, etc.--Expected outcome Some array where the first value is 0 and the last one is 1. Note that this bug doesn't happen if density=False. Bisects to (). Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: linux Matplotlib version: master Matplotlib backend (): any Python version: 37 Jupyter version (if applicable): no Other libraries: numpy 1.16.2 <!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda--<!--If you installed from conda, please specify which channel you used if not the default--


bar plot yerr lines/caps should respect zorder Bug summary Bar plot error bars break when zorder is greater than 1. Actual outcome ! Matplotlib version Operating system: Arch Linux Matplotlib version: 2.2.3 Matplotlib backend (): Python version: 3.6 Jupyter version (if applicable): 5.7.0 Conda default channel Possible related issue:


Existing FigureCanvasQT objects destroyed by call to Bug summary For a number of years, I have been maintaining an interactive application that embeds subclassed FigureCanvasQT objects within a PyQt application. Up until Matplotlib v3.0.3., it was possible to create standard Matplotlib PyQt figures, i.e., using within an embedded IPython shell, that coexist with the subclassed canvas objects. Now in Matplotlib v3.1.0, a call to destroys all the other canvas objects. Unfortunately, I cannot debug this within Visual Studio since I am currently unable to install Matplotlib from the source. By going through the code line by line, I can confirm that the windows are destroyed when calling , but not when calling , but I can't work out what triggers the destruction of the other windows. I presume the subclassed canvasses are not being registered somewhere, and assumes they are no longer active, but I am hoping someone will show me where to look. This may not be a Matplotlib bug, but it's certainly an unwelcome side effect in my application. Code for reproduction If you have installed () and launch it, you can reproduce this from within the embedded IPython shell with the following, which creates a new subclassed window and then attempts to open a regular window.: There are two figure numbers, because NeXpy automatically creates a window with a figure number of 1 when it is launched. Actual outcome A new window with an updated figure number is created but all other windows not created by are destroyed. Expected outcome In Matplotlib v3.0.3, a new window is created by the PyQt5 backend without destroying anything else. Matplotlib version Operating system: Mac OS v10.14.5 Matplotlib version: 3.1.0 Matplotlib backend: Qt5Agg Python version: 3.7.2 Jupyter version (if applicable): 1.0.0 Other libraries: <!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda--<!--If you installed from conda, please specify which channel you used if not the default--


Inverting an axis using its limits does not work for log scale Bug summary Starting in matplotlib 3.1.0 it is no longer possible to invert a log axis using its limits. Code for reproduction Actual outcome The yaxis is only inverted for the scale. ! ! Expected outcome I would expect the yaxis to be inverted for both the scale. Matplotlib version Operating system: Linux and MacOS Matplotlib version: 3.1.0 Python version: 3.7.3 Python and matplotlib have been installed using conda.


and fig.savefig interfere with each other and raise StopIteration The following code snippet raises StopIteration: The same also happens if the figure is saved before the animation. The interesting thing is that the two parts of the script work fine when run alone, so there must be some interference between the two. The output produced by the above code: Matplotlib version <!--Please specify your platform and versions of the relevant libraries you are using:--Operating system: Ubunut 18.04 Matplotlib version: 3.2.2 Matplotlib backend (): TkAgg Python version: 3.7.7 Jupyter version (if applicable): No Jupyter Other libraries: Matplotlib installed using pip inside a conda environment