ManagementUtility instantiates CommandParser without passing already-computed prog argument Description ManagementUtility goes to the trouble to parse the program name from the argv it's passed rather than from def init(self, argv=None): = argv or [:] self.progname = ([0]) if self.progname == '': self.progname = 'python -m django' But then when it needs to parse --pythonpath and --settings, it uses the program name from parser = CommandParser(usage='%(prog)s subcommand [options] [args]', addhelp=False, allowabbrev=False) Above "%(prog)s" refers to [0]. Instead, it should refer to self.progname. This can fixed as follows: parser = CommandParser( prog=self.progname, usage='%(prog)s subcommand [options] [args]', addhelp=False, allowabbrev=False) I'm aware that executefromcommandline is a private API, but it'd be really convenient for me if it worked properly in my weird embedded environment where [0] is incorrectly None. If passing my own argv to executefromcommand_line avoided all the ensuing exceptions, I wouldn't have to modify [0] globally as I'm doing in the meantime.


shell command crashes when passing (with -c) the python code with functions. Description The examples below use Python 3.7 and Django 2.2.16, but I checked that the code is the same on master and works the same in Python 3.8. Here's how python -c works: $ python -c <<EOF " import django def f(): print(django.version) f()" EOF 16 Here's how python -m django shell -c works (paths shortened for clarify): $ python -m django shell -c <<EOF " import django def f(): print(django.version) f()" EOF Traceback (most recent call last): File "{sys.baseprefix}", line 193, in runmoduleasmain "main", modspec) File "{sys.baseprefix}", line 85, in runcode exec(code, runglobals) File "{}/lib/python3.7/site-", line 9, in <modulemanagement.executefromcommandline() File "{}/lib/python3.7/site-", line 381, in executefromcommandline utility.execute() File "{}/lib/python3.7/site-", line 375, in execute self.fetchcommand(subcommand).runfromargv() File "{}/lib/python3.7/site-", line 323, in runfromargv self.execute(args, cmdoptions) File "{}/lib/python3.7/site-", line 364, in execute output = (args, options) File "{}/lib/python3.7/site-", line 86, in handle exec(options['command']) File "<string", line 5, in <moduleFile "<string", line 4, in f NameError: name 'django' is not defined The problem is in the usage of exec: def handle(self, **options): # Execute the command and exit. if options['command']: exec(options['command']) return # Execute stdin if it has anything to read and exit. # Not supported on Windows due to () limitations. if sys.platform != 'win32' and not () and ([], [], [], 0)[0]: exec(()) return exec should be passed a dictionary containing a minimal set of globals. This can be done by just passing a new, empty dictionary as the second argument of exec.


Dabase creation backend should use basemanager to serialize database Description class Service(Model): objects = CustomManagerThatFillterOutSomeRecords() class CustomManagerThatFillterOutSomeRecords(Manager): def getqueryset(self): return super().getqueryset().exclude(pk=1) class TestService(TransactionTestCase): serializedrollback = True def testsomething(self): pass Assume we have a migration that creates few records of Service. from import callcommand from import migrations def loadfixtures(, *_): callcommand('loaddata', '') class Migration(migrations.Migration): dependencies = [] operations = [ migrations.RunPython( loadfixtures, , ) ] Then TestService will fail as serializedbtostring by default use defaultmanager that is CustomManagerThatFillterOutSomeRecords. Here is proposed fix:


Augment () optimizations to .union().exists(). Description (last modified by Simon Charette) The method performs optimization by clearing the select clause, dropping ordering, and limiting the number of results to 1 if possible. A similar optimization can be applied for combined queries when using () as some query planers (e.g. MySQL) are not smart enough to prune changes down into combined queries. For example, given (otherauthors).exists() the currently generated is SELECT 1 FROM (SELECT FROM authors WHERE ... ORDER BY ... UNION SELECT FROM authors WHERE ... ORDER BY ...) LIMIT 1; But some planers won't be smart enough to realize that both * and ORDER BY are not necessary and fetch all matching rows. In order to help them we should generate the following SQL SELECT 1 FROM (SELECT 1 FROM authors WHERE ... LIMIT 1 UNION SELECT 1 FROM authors WHERE ... LIMIT 1) LIMIT 1; This can already be done manually through filteredauthors.orderby().values(Value(1))[:1].union(otherauthors.orderby().values(Value(1))[:1]).exists() but that involves a lot of boilerplate. Note that the optimization is only possible in this form for union and not for intersection and difference since they require both sets to be unaltered.


dateformat.y() doesn't support years < 1000. Description (last modified by Sam) When using the the dateformat of django with a date before 999 (or 99 and 9 for similar matters) and the format character "y" no leading zero will be printed. This is not consistent with the way the python datetime module and PHP handle that character "y" in format strings: django (version 3.1): import datetime from import dateformat (datetime.datetime(123, 4, 5, 6, 7), "y") '3' python (version 3.8): import datetime datetime.datetime(123, 4, 5, 6, 7).strftime("%y") '23' php (version 7.4): echo date("y", strtotime("0123-04-05 06:07:00")) 23 I have a pull-request ready for this:


Allow cache.getorset() to cache a None result Description (last modified by Phill Tornroth) getorset docstring says "If the key does not exist, add the key and set it to the default value." -- but that's not quite what it does. It will perform a set if the key doesn't exist, or if the cached value is None. I think in order to be doing what it says on the tin it'd need to be: if self.has_key(key, version=version): return (key, version=version) else: if callable(default): default = default() (key, default, timeout=timeout, version=version) # Fetch the value again to avoid a race condition if another # caller a value between the first get() and the add() # above. return (key, default, version=version) I'd find this useful in cases where None was an expensive result to arrive at. If there's spiritual alignment with the suggestion, I'm happy to prepare and submit a change with tests.


Improve error messages from forgetting to call .asview() on a CBV Description (last modified by Angus Holder) We can detect early-on that the user has forgotten to call .asview() on their CBV when passing it into path(). For: urlpatterns = [ path('home', HomeView) ] The error currently happens only when you first load the route (rather than when constructing the routes), and looks like Internal Server Error: / Traceback (most recent call last): File "C:\Users\Angus.virtualenvs\django-WBTbdxDv\lib\site-packages\django\core\handlers\", line 47, in inner response = getresponse(request) File "C:\Users\Angus.virtualenvs\django-WBTbdxDv\lib\site-packages\django\core\handlers\", line 179, in getresponse response = wrappedcallback(request, callbackargs, *callbackkwargs) TypeError: init() takes 1 positional argument but 2 were given Which is especially hard to work out given that the traceback doesn't even include any of the user's own code, and this is an error that's easy for beginners to run into when first using CBVs. My PR changes it to fail early, inside the call to (), with a clear error: URL route 'foo' should pass in 'EmptyCBView.as_view()' instead of 'EmptyCBView' Pull request:


Misleading error when loading a database backend fails with ImportError Description For example, in the following case, djangocockroachdb is installed but has an import error itself. Traceback (most recent call last): File "", line 110, in loadbackend return importmodule('' % backendname) File "", line 127, in importmodule return bootstrap.gcdimport(name[level:], package, level) File "<frozen importlib.bootstrap", line 1030, in gcdimport File "<frozen importlib.bootstrap", line 1007, in findandload File "<frozen importlib.bootstrap", line 972, in findandloadunlocked File "<frozen importlib.bootstrap", line 228, in callwithframesremoved File "<frozen importlib.bootstrap", line 1030, in gcdimport File "<frozen importlib.bootstrap", line 1007, in findandload File "<frozen importlib.bootstrap", line 986, in findandloadunlocked File "<frozen importlib.bootstrap", line 680, in loadunlocked File "<frozen importlib.bootstrapexternal", line 790, in execmodule File "<frozen importlib.bootstrap", line 228, in callwithframesremoved File "/home/tim/code/cockroach-", line 2, in <modulefrom .lookups import patchlookups File "/home/tim/code/cockroach-", line 1, in <modulefrom import HasKeyLookup, KeyTransform ModuleNotFoundError: No module named '' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 10, in <moduleexecutefromcommandline() File "", line 381, in executefromcommandline utility.execute() File "", line 357, in execute () File "", line 24, in setup apps.populate(settings.INSTALLEDAPPS) File "", line 114, in populate appconfig.importmodels() File "", line 211, in importmodels self.modelsmodule = importmodule(modelsmodulename) File "", line 127, in importmodule return bootstrap.gcdimport(name[level:], package, level) File "<frozen importlib.bootstrap", line 1030, in gcdimport File "<frozen importlib.bootstrap", line 1007, in findandload File "<frozen importlib.bootstrap", line 986, in findandloadunlocked File "<frozen importlib.bootstrap", line 680, in loadunlocked File "<frozen importlib.bootstrapexternal", line 790, in execmodule File "<frozen importlib.bootstrap", line 228, in callwithframesremoved File "", line 12, in <moduleclass Question(models.Model): File "", line 117, in new newclass.addtoclass('meta', Options(meta, applabel)) File "", line 321, in addtoclass value.contributetoclass(cls, name) File "", line 204, in contributetoclass self.dbtable = truncatename(self.dbtable, ()) File "", line 28, in getattr return getattr(connections[DEFAULTDBALIAS], item) File "", line 201, in getitem backend = loadbackend(db['ENGINE']) File "", line 121, in loadbackend raise ImproperlyConfigured( 'djangocockroachdb' isn't an available database backend. Try using '', where XXX is one of: 'mysql', 'oracle', 'postgresql', 'sqlite3' I'm not sure there's anything that could be done besides rephrasing the error message to say something like this: 'djangocockroachdb' isn't an available database backend or couldn't be imported. Check the above exception. In this case, "Try using..." isn't so helpful either. Perhaps we could only include that if backend_name begins with django..


Aggregating when grouping on an ExpressionWrapper omits the expression from the group by Description I ran into this with Postgres on Django 3.1.3, I'm not sure what other versions it exists on. print( Fred.objects.annotate( bobidisnull=ExpressionWrapper( Q(bobid=None), outputfield=BooleanField() ) ).values( "bobidisnull" ).annotate( idcount=Count("id", distinct=True) ).values( "bobidisnull", "idcount" ).query ) SELECT "mainfred"."bobid" IS NULL AS "bobidisnull", COUNT(DISTINCT "mainfred"."id") AS "idcount" FROM "mainfred" GROUP BY "mainfred"."bobid" On the last line there the group by has dropped the "IS NULL"


Error with values with JSON path lookup in Sqlite when value is numeric. Description (last modified by Gordon Wrigley) This Django 3.1.3, I only see this with Sqlite, it works fine with MySQL and Postgres. When I do a path lookup with values like ("myjsonfieldposition") if there is an integer, float or bool in "position" then I get a JSON decode error. Strings, nones, dicts and lists all work, fetching the top level dict works and filtering on the path lookup works. TypeError: the JSON object must be str, bytes or bytearray, not float .....venv\databrowser\lib\site-packages\django\db\models\ in iter self.fetchall() .....venv\databrowser\lib\site-packages\django\db\models\ in fetchall self.resultcache = list(self.iterableclass(self)) .....venv\databrowser\lib\site-packages\django\db\models\ in iter for row in compiler.resultsiter(chunkedfetch=self.chunkedfetch, chunksize=self.chunksize): .....venv\databrowser\lib\site-packages\django\db\models\sql\ in applyconverters value = converter(value, expression, connection) .....venv\databrowser\lib\site-packages\django\db\models\fields\ in fromdb_value return (value, cls=self.decoder)


django-admin runserver and getchildarguments() crashes on Windows and Python < 3.8. Description "django-admin runserver" fails with the following error: Traceback (most recent call last): File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 193, in runmoduleasmain "main", modspec) File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 85, in runcode exec(code, runglobals) File "C:\Users\someone\AppData\Local\Programs\Python\Python37\Scripts\django-\", line 7, in <moduleFile "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\", line 401, in executefromcommandline utility.execute() File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\", line 395, in execute self.fetchcommand(subcommand).runfromargv() File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\", line 330, in runfromargv self.execute(args, cmdoptions) File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\commands\", line 61, in execute super().execute(args, options) File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\", line 371, in execute output = (args, options) File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\commands\", line 96, in handle (options) File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\core\management\commands\", line 103, in run autoreload.runwithreloader(self.innerrun, options) File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\utils\", line 616, in runwithreloader exitcode = restartwithreloader() File "c:\users\someone\appdata\local\programs\python\python37\lib\site-packages\django\utils\", line 244, in restartwithreloader p = (args, env=newenviron, closefds=False) File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 488, in run with Popen(popenargs, kwargs) as process: File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 800, in init restoresignals, startnewsession) File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 1148, in executechild args = list2cmdline(args) File "c:\users\someone\appdata\local\programs\python\python37\lib\", line 555, in list2cmdline needquote = (" " in arg) or ("\t" in arg) or not arg TypeError: argument of type 'WindowsPath' is not iterable Environment: Windows 10 Python: 3.7.6 Django: 3.1.3 Steps to reproduce: django-admin startproject mysite set PYTHONPATH=<thelocaldirdjango-admin runserver --settings=mysite.settings Apparently returns WindowsPath("C:\Users\someone\AppData\Local\Programs\Python\Python37\Scripts\django-") as the first argument, and subprocess.Popen expects a string and is not able to use Path. The following monkey-patch fixes the error: def getchildargumentsoverride(): args = autoreload.getchildarguments() for i, arg in enumerate(args): if isinstance(arg, Path): args[i] = str(arg) return args autoreload.getchildarguments = autoreload.getchildarguments autoreload.getchildarguments = getchildargumentsoverride


Add a 'durable' flag to () Description As discussed on Twitter following David Seddon's blog post. () guarantees the wrapped database operations is atomic - at the end of its wrapped block, operations within that block will all be applied, or all rolled back. In some situations it's also useful to guarantee that the wrapped operations are durable - at the end of the wrapped block, all operations have definitely been committed. atomic() doesn't guarantee this at the moment since its use may be wrapped by another atomic() higher in the stack, delaying the commit. David Seddon's technique to guarantee durability is to wrap atomic() to first check if an atomic() is already active for the current connection (in other words, connection.get_autocommit() returns False), and raise an error if so. On Twitter, Ian Foote suggested adding a durable flag to atomic() to add this behaviour to Django, to which Aymeric Augustin said it sounds like a good idea.


Use Admin Inline verbosename as default for Inline verbosenameplural Description Django allows specification of a verbosename and a verbosenameplural for Inline classes in admin views. However, verbosenameplural for an Inline is not currently based on a specified verbosename. Instead, it continues to be based on the model name, or an a verbosename specified in the model's Meta class. This was confusing to me initially (I didn't understand why I had to specify both name forms for an Inline if I wanted to overrule the default name), and seems inconsistent with the approach for a model's Meta class (which does automatically base the plural form on a specified verbosename). I propose that verbosenameplural for an Inline class should by default be based on the verbosename for an Inline if that is specified. I have written a patch to implement this, including tests. Would be happy to submit that.


SQLite3 bad filepath raising as JSON extension error. Description Applying migrations on an installation using spatialite backend (probably sqlite too) where the path to the database file does not exist causes a confusing JSON field error. Take this following DATABASES configuration. Note that the dev.sqlite3 file does not exist. DATABASES = { 'default': { 'ENGINE': '', 'NAME': '/home/user/app/dev.sqlite3', } } When running migrate the following exception is raised SystemCheckError: System check identified some issues: ERRORS: rumble_common.RumbleEvent: (fields.E180) SQLite does not support JSONFields. We assert the JSON extension is present with the following code block, as per import sqlite3 conn = sqlite3.connect(':memory:') cursor = () cursor.execute('SELECT JSON(\'{"a": "b"}\')') The issue is resolved by changing DATABASES['default']['NAME'] to point to the right path. The issue is that the error is misleading, and leads developers down the wrong rabbit hole Hope thats descriptive enough, thanks all! Extra: Django 3.1.3 Debian Buster Docker environment


Add a hook to customize the admin's formsets parameters Description New feature that adds a method on InlineModelAdmin for providing initial data for the inline formset. By default there is no implementation, although one could be implemented to use GET parameters like getchangeforminitial_data, but it wouldn't be trivial due to the list nature of formset initial data.


Set disabled prop on ReadOnlyPasswordHashField Description Currently the defines a cleanpassword method that returns the initial password value to prevent (accidental) changes to the password value. It is also documented that custom forms for the User model need to define this method:  A while ago the forms.Field base class gained the disabled argument to: [disable] a form field using the disabled HTML attribute so that it won’t be editable by users. Even if a user tampers with the field’s value submitted to the server, it will be ignored in favor of the value from the form’s initial data. It seems to me that this property could be set to True be default on the ReadOnlyPasswordHashField used to display the password hash. This way the cleanpassword is no longer necessary and the potential pitfall when using the ReadOnlyPasswordHashField without implementing clean_password is removed.


ConnectionAbortedError should be treated as a broken pipe error in the development server Description People using Windows have been complaining about very long stack traces in the development server for a long time:   These happen under normal interaction with the development server using Chrome. I have fixed one issue in CPython in wsgiref, but I belive that we should also fix isbrokenpipe_error in


Deprecate MemcachedCache. Description python-memcached is not maintained anymore (see python-memcached) and it makes difficulties in fixing some issues (e.g. ). Moreover we a cache backend for pymemcache () so we have a good builtin alternative. I think it's time to deprecate the backend in Django 3.2 and remove it in Django 4.1.


Using isnull=True on a KeyTransform should not match JSON null on SQLite and Oracle Description The KeyTransformIsNull lookup borrows the logic from HasKey for isnull=False, which is correct. If isnull=True, the query should only match objects that do not have the key. The query is correct for MariaDB, MySQL, and PostgreSQL. However, on SQLite and Oracle, the query also matches objects that have the key with the value null, which is incorrect. To confirm, edit tests.modelfields.testjsonfield.TestQuerying.testisnullkey. For the first assertion, change self.assertSequenceEqual( (valueaisnull=True), [:3] + [5:], ) to self.assertSequenceEqual( (valuejisnull=True), [:4] + [5:], ) The test previously only checks with valuea which could not catch this behavior because the value is not JSON null.


Log exceptions handled in Signal.sendrobust() Description As pointed out by Haki Benita on Twitter, by default Signal.sendrobust() doesn't have any log messages for exceptions raised in receivers. Since Django logs exceptions in other similar situations, such as missing template variables, I think it would be worth adding a logger.exception() call in the except clause of send_robust() . Users would then see such exceptions in their error handling tools, e.g. Sentry, and be able to figure out what action to take from there. Ultimately any expected exception should be caught with a try in the receiver function.


squashmigrations optimizer crashes when fields' names are swapped using a temporary name Description If you rename fields using a pattern like a-c; b-a; c-b (such as if previously DateTimeFields using autonow and autonowadd had been mixed-up) and then attempt to squashmigrations with an optimization barrier between the CreateModel and RenameFields, the migration optimizer will attempt to create a CreateModel operation object with two fields using the same name and fail. I'll attach a migration file that triggers the failure. I believe the root cause of this issue is that allows itself to optimize through (i.e be moved to the right of, I may have gotten this terminology wrong) other RenameField operations that reference oldname.


prefetchrelatedobjects() does not work for reused model instances. Description (last modified by Dennis Kliban) Our project processes instances in a stream. In some cases the instances are repeated. In these cases, we discovered that prefetchrelatedobjects() does not set the toattr on all of the instances if the first instance in the list already has it set. When Django determines that the very first instance in the list isfetched[0], it does not call into the the prefetchonelevel()[1]. The attributed specified in the toattr parameter is only set in the prefetchonelevel() method[2]. isfetched is set by looking for the to_attr attribute on the instance[3]. [0]  [1]  [2]  [3]


squashmigrations does not unset model options when optimizing CreateModel and AlterModelOptions Description When an operation resembling AlterModelOptions(name="testmodel", options={}) is squashed into the corresponding CreateModel operation, model options are not cleared on the resulting new CreateModel operation object. () sets the new options as options={self.options, operation.options} in this case ( line 144 on commit ), with no logic to remove options not found in operation.options as is found in AlterModelOptions.stateforwards(). I believe this issue still exists on the master branch based on my reading of the code, but I've only tested against 2.2.


parseduration() ISO string sign is ignored when the timedelta only has days Description I'm pretty sure that this is a bug even though I'm not an expert on the ISO 8601 standard. The sign of a timedelta string will be ignored by if the input string only contains days. Compare the following (notice the minus signs): In [4]: timedelta(days=-1) Out[4]: datetime.timedelta(days=-1) In [5]: td = timedelta(days=-1) In [6]: durationisostring(td) Out[6]: '-P1DT00H00M00S' In [7]: parseduration(durationisostring(td)) Out[7]: datetime.timedelta(days=1) # <-- Why is this 1 and not -1? In [8]: td = timedelta(days=-1, microseconds=1) In [9]: durationisostring(td) Out[9]: '-P0DT23H59M59.999999S' In [10]: parseduration(durationisostring(td)) Out[10]: datetime.timedelta(days=-1, microseconds=1) I guess the problem is in line 147 that reads return days + sign * datetime.timedelta(kw). However, if datetime.timedelta(kw) ends up being zero (timedelta(0)) then the sign multiplication ends up in zero, not -0. This is just a preliminary quick look though and maybe the problem is something else.


add filter is unable to concatenate strings with lazy string Description If you try to concatenate a string with a lazy string with the add template filter, the result is always the empty string because the add filter generates an exception (TypeError: can only concatenate str (not "proxy") to str).


Add support for fixtures compression in dumpdata Description Since loaddata command support compressed fixtures [1] loading, I would propose to add fixtures compression support in dumpdata command. [1]


Not RFC 6265 compliant cookies in contrib.messages. Description (last modified by Nico Giefing) Hi A Customer of mine is using a WAF which is handling Cookies as it is described tin the RFC:  The issue now is that Django is trying to use an escape-character in cookie-Values which is not supported in the RFC an example of such a cookie: messages=\"123\\"NOTRECEIVED\"" Please consider to get this fixed so there can be a protection of this system. Regards, Nico


loaddata crashes on SQLite when table names are SQL keywords. Description Steps to reproduce: Create a Model called Order. (order is a SQL reserved word) Create fixtures for the model Use loaddata to load the fixture. Notice that it fails with the following error. This is because the table name order is not quoted properly (0.000) PRAGMA foreignkeycheck(order); args=None Traceback (most recent call last): File "python3.7/site-", line 82, in execute return (sql) File "python3.7/site-", line 411, in execute return Database.Cursor.execute(self, query) sqlite3.OperationalError: near "order": syntax error Root Cause File: python3.7/site- line 327 Function: checkconstraints Details: due to missing back ticks around %s in the SQL statement PRAGMA foreignkeycheck(%s) Here in checkconstraints line 327 in context if tablenames is None: violations = cursor.execute('PRAGMA foreignkeycheck').fetchall() else: violations = chain.fromiterable( cursor.execute('PRAGMA foreignkeycheck(%s)' % tablename).fetchall() for tablename in tablenames ) And here line 333 for tablename, rowid, referencedtablename, foreignkeyindex in violations: foreignkey = cursor.execute( 'PRAGMA foreignkeylist(%s)' % tablename ).fetchall()[foreignkey_index] Issue confirmed in 0 2


Allow postgresql database connections to use postgres services Description (last modified by levihb) Postgres offers a way to make database connections through the use of services, which are basically equivalent to MySQL's options files. Server, database, username, etc information is stored by default in and takes a very similar format to MySQL cnf files: [myalias] host=10.0.19.10 user=postgres dbname=postgres port=5432 And password can be stored in under a different format. I think being able to just add them to the DATABASES config would be useful, similar to how you can add MySQL cnf files. psycopg2 supports it just fine through the service argument/string connect(service='myalias') connect('service=myalias'). At the moment it can be like this: DATABASES = { 'default': { 'ENGINE': '', 'NAME': 'postgres', 'OPTIONS': {'service': 'myalias'} } } Which works, however it involves repeating the database name. I don't think the database name should be repeated twice because it couples the config and the service file together, and makes it harder to just move it between different environments. I think ideally you would just specify the service, either like this: DATABASES = { 'default': { 'ENGINE': '', 'OPTIONS': {'service': 'myalias'} } } Or maybe a better way would be?: DATABASES = { 'default': { 'ENGINE': '', 'SERVICE': 'my_alias } } It seems like something that would be super easy to add. I don't mind creating a pull request for it, but would like to know why it hasn't been , and how it would be recommended to add it.


Add --skip-checks option to the runserver command. Description Rationale: It would be consistent with other management commands performing system checks It would help people like me who would rather have checks enabled exclusively in CI/CD than wait 15-20 seconds for each project reload during development Related StackOverflow question:


MiddlewareNotUsed leaves undesired side effects when loading middleware in ASGI context Description I experienced strange issues when working with ASGI , django-debug-toolbar and my own small middleware. It was hard problem to debug, I uploaded an example project here:  (the name is misleading - I initially thought it's a bug with django-debug-toolbar). The SESSIONFILEPATH setting is intentionally broken to cause a 500 error. When starting the application and accessing /admin (any location really, but I wanted to leave it at a minimum and didn't add any views) it gives TypeError: object HttpResponse can't be used in 'await' expression. Commenting out asgidjangotoolbarbug.middleware.DummyMiddleware fixes the issue (in that I receive a 500 ImproperlyConfigured exception). I'm not sure about the overall role of django-debug-toolbar here - removing it causes Daphne to return a 500 error page but without debug information and there's no traceback in console either. I decided to leave it since it helped me approximate the causes of issue. I notice that in  while MiddlewareNotUsed causes the loop to skip futher processing and go to next middleware, it does leave handler variable overwritten with output of self.adaptmethodmode(). On next pass, this handler is passed to next middleware instance, disregarding all the previous checks for (lack of) async support. This likely causes the middleware chain to be "poisoned" from this point onwards, resulting in last middleware in response cycle to return an HttpResponse as a synchronous middleware would, instead of coroutine that is expected. This is probably avoided by adding async support to my middleware, but unless I'm missing something docs indicate it should work as it is. It is my intention that it's applied only on synchronous requests, so I didn't make it async compatible on purpose. If it's intentional in Django that every middleware needs to support async if the application is run as ASGI app, the documentation should probably state that clearly. Though it kinda defeats the purpose of having async_capable = False flag in the first place.


Raising InvalidTemplateLibrary completely masks out real exception in getpackagelibraries Description Summary In , function getpackagelibraries on line 119 completely masks out the ImportError and raises InvalidTemplateLibrary. This makes it incredibly difficult to debug application issues. Probably better not to handle the exception in the first place since it only raises another type and inner exception looses the stack trace. To reproduce Create two apps e.g. formutils and reports. Write a template tag in reports e.g. reports.templatetags.reporttags. ( Add a simple module in formutils e.g. In , import a none-existent module e.g. from import flatatt (was removed in django 1.4) import in reporttags e.g. from formutils.widgets import CalendarWidget A quick way to reproduce the error would be to register some models with admin and navigate to /admin The following error will be raised in getpackagelibraries: InvalidTemplateLibrary at /admin/login/ Invalid template library specified. ImportError raised when trying to load 'reports.templatetags.reporttags': No module named util Request Method: GET Request URL: http://localhost:2017/admin/login/?next=/admin/ Django Version: 1.10.1 Exception Type: InvalidTemplateLibrary Exception Value: Invalid template library specified. ImportError raised when trying to load 'reports.templatetags.reporttags': No module named util Exception Location: D:\repo\django110\lib\site-packages\django\template\backends\ in getpackagelibraries, line 130 However, if the exception was not caught and "wrongly" re-raised as an InvalidTemplateLibrary, the following errors would be printed: ImportError at /admin/login/ No module named util Request Method: GET Request URL: http://localhost:2017/admin/login/?next=/admin/ Django Version: 1.10.1 Exception Type: ImportError Exception Value: No module named util Exception Location: D:\repo\projects\evincehr\apps\formutils\ in <module, line 3 The second behavior is more appropriate to debugging the error and the error would be quickly found.


Permit migrations in non-namespace packages that don't have file Description Summary This feature request, for which I will post a PR shortly, aims to improve the specificity of the migration loader's check for and rejection of PEP-420 namespace packages. I am NOT asking to allow namespace packages for apps' migrations. I merely want to make the existing check more compliant with Python's documented import API. This would remove one impediment to using Django in so-called frozen Python environments (such as those mentioned in ) that do not set file on regular packages by default. This narrow proposal does not change Django's behavior at all for normal Python environments. The only change for frozen environments is that Django will learn how to find existing migrations. In particular, at this time I am not proposing to enable any other Django feature that does not already work in frozen environments. I would love for this feature to land in Django 3.2. Details I initially broached this idea on the django-developers mailing list. This is my second ticket related to frozen Python environments, the first being . The current implementation of the migration loader's no-namespace-package check in skips searching for migrations in a module m if getattr(m, 'file', None) is false. The trouble with this implementation is that namespace packages are not the only modules with no file. Indeed, the Python documentation states that file is optional. If set, this attribute's value must be a string. The import system may opt to leave file unset if it has no semantic meaning (e.g. a module loaded from a database). However, Python's documentation also states Namespace packages do not use an ordinary list for their path attribute. They instead use a custom iterable type.... The class of namespace packages' path in CPython is NamespacePath, but that is a CPython implementation detail. Instead, I propose to augment getattr(m, 'file', None) with and isinstance(m.path, list).


Drop support for SQLite < 3.9.0 Description (last modified by Tim Graham) Indexes on expressions (see ) and the SQLITEENABLEJSON1 compile-time option are supported on SQLite 3.9.0+. Ubuntu Xenial ships with SQLite 3.11.0 (which will still by supported by Django) and will EOL in April 2021. Debian Jessie ships with 3.8.7 and was EOL June 30, 2020. SQLite 3.9.0 was released in October 2015. SQLite version support seems like a similar situation as GEOS libraries which we generally support about 5 years after released.


fields.E305 is raised on ManyToManyFields with relatedname='+' in models in different apps but with the same name. Description (last modified by Aleksey Ruban) Django raises an error during creation a db migration if two models with the same name refer to the same model in m2m field. relatedname='+' or 'foo+' don't impact anything. In some my project there are 50 apps and almost each one has a model with the same name. So I have to come up with a related name and write it in for each m2m field. Just try to make a migration for my test project


Django adds spurious "/" prefix to settings.STATICURL="http://server/" Description (last modified by Adam Hooper) Here's a piece of settings from a totally reasonable, sensible, okay Docker integration-test environment STATICURL = "http://minio/static/" Django 3.1 will implicitly add "/" to the URL, so my URLs look like /http://minio/static/images/app- The features and bugs that interact here: commit , adding feature , prepends SCRIPTNAME to STATICURL when STATICURL isn't a URL. bug and : according to Django, "http://minio/static/" isn't a valid URL. (It is.) Top me, the easiest fix is to address / . Or to make STATICURL use some logic that is different from URLValidator.


Allow autoreloading of Description (last modified by William Schwartz)  detects if Python was launched as python -m django. Currently it detects only when -m was passed specifically django (and only in Python environments in which file is set on modules, which is not true of all Python environments). Like , this ticket aims to remove one impediment to creating Django-based command-line utilities that have their own main sub-module while overriding Django's built-in management commands—in this case, runserver. The fix, which I have submitted in the attached PR, is to use Python's documented way of determining if -m was used in getchild_arguments: The top-level main module is always the entry point of a complete Python program. is not None if and only if Python was launched with -m or the name of a "directory, zipfile or other entry." In the latter cases, the documentation says If the script name refers to a directory or zipfile, the script name is to the start of and the file in that location is executed as the main module. Hence (which is usually but not always main.package) exists and is the empty string when Python is started with the name of a directory or zip file. Therefore Python was started with -m pkg if and only if == "pkg". Following this algorithm is guaranteed to work as long as Python obeys its own documentation, and has the side benefit of avoiding use of file.


Access file lazily rather than at module level Description (last modified by William Schwartz) So-called frozen Python environments (such as those mentioned in ) that do not set all modules' file variable, which need not be defined, cannot even import Django (without some workarounds) because a small number of Django modules use file at the module level, in a class defined at the module level, or in a function that is called automatically upon import. Five modules that use file like this are likely to be imported when using Django and thereby cause a frozen Python to crash with a NameError or similar exception. Importing can be avoided only by avoiding both forms and the ORM altogether as it's imported from Importing might be avoidable if DEBUG=False or by avoiding all of the views and URLs APIs. 's getgitchangeset is called when django is imported in pre-alpha development versions. Importing is only avoidable by not using the Auth app. uses file to find Django's localization files upon activation; this avoidable only by setting USEI18N=False. Dealing with transreal is sufficiently thorny (and, being an English speaker with English-speaking clients, I can avoid it for now) that I will not address it further here except to say that it might need to be part of the larger discussion at . What this ticket is not I am not proposing removing use of file at this time. That would require a longer discussion of intended semantics such as . This ticket is only about removing use of file at the module (or class definition) level in Django application code (not test code). Further I am not proposing banning use of file at the module level at this time, hence minimal new tests and no update to the Django coding style documentation. That too would require a longer conversation. Proposed fixes I have pushed PR GH- to address the four of those modules other than transreal. I dealt with each module's use of file in separate commits to make them easier to discuss and separate/cherry-pick if needed. Below I link to the individual commits as I discuss each of the four modules. These first two are fairly easy, but the second two may require further consideration. () Remove the undocumented module constant ROOT and replace its single use. () Treat the lack of module-global file the same as a failure of git log by returning None from getgitchangeset. () The module-level constant CURRENTDIR is used only in the module itself and is undocumented, so I'm assuming it's an obscure private symbol that no one will miss. I've replaced it with a module-level private function builtintemplatepath that refactors and centralizes finding built-in templates for the entire module. The one tricky part is that the htmltemplatepath and texttemplatepath attributes I didn't want to disturb 's goal of making the template paths easily override-able, so I avoided calling builtintemplatepath in the class definition by making detecting the presence of the attributes in init and setting defaults there. Alternatives include making the attributes properties with setters or cached properties without setters. () The CommonPasswordValidator-class constant DEFAULTPASSWORDLISTPATH is used only in one place, the class's instance constructor. While the nature of DEFAULTPASSWORDLISTPATH is not documented, its existence is inside the docs for the constructor's signature. I've changed DEFAULTPASSWORDLISTPATH from a class constant into an instance attribute. Another possibility is making DEFAULTPASSWORDLIST_PATH be a


i18n.setlanguage unquotes nexturl and produces wrong url for url params containing "&" character Description (last modified by Johannes Maron) When changing the language and the current URL parameter include a parameter value with an encoded "&" like ?paramter=some%20%26%20thing the redirect response from setlangauge is ?paramter=some%20&%20thing where I would still expect the same URL from as in the beginning. I've written a Django test that shows this bug: def testsetlanguageurlparams(): from import RequestFactory from import setlanguage rf = RequestFactory() request = ("", next="") request.META['HTTPREFERER'] = '/someurl/?paramter=some%20%26%20thing' response = setlanguage(request) assert == '/someurl/?paramter=some%20%26%20thing' i18n.setlanguage unquotes nexturl and produces wrong url for url params containing "&" character Description (last modified by Johannes Maron) When changing the language and the current URL parameter include a parameter value with an encoded "&" like ?paramter=some%20%26%20thing the redirect response from setlangauge is ?paramter=some%20&%20thing where I would still expect the same URL from as in the beginning. I've written a Django test that shows this bug: def testsetlanguageurlparams(): from import RequestFactory from import setlanguage rf = RequestFactory() request = ("", next="") request.META['HTTPREFERER'] = '/someurl/?paramter=some%20%26%20thing' response = setlanguage(request) assert == '/someurl/?paramter=some%20%26%20thing'


Add system checks for invalid model field names for functional indexes in Meta.indexes. Description (last modified by Mariusz Felisiak) The current system checks don't cover invalid models fields in functional indexes. This is not straightforward because resolving expressions with non-existent throws FieldError, so we cannot just collect columns from Query.gencols(). Follow up to . I attached tests.


Migrations are marked applied even if deferred SQL fails to execute Description The changes introduced in to address fail to account for the possibility of the schema editor accumulation of deferred SQL which is run at time.


models.W042 is raised on inherited manually specified primary key. Description I have models which inherit from other models, and they should inherit the primary key. This works fine with Django 3.1. However, if I install Django 3.2 alpha, when I run makemigrations I get the following error messages: System check identified some issues: WARNINGS: accounts.ReservedUsername: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the SpeedyCoreAccountsConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. accounts.User: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the SpeedyCoreAccountsConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. blocks.Block: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. contactbyform.Feedback: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the SpeedyCoreContactByFormConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. coremessages.ReadMark: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the SpeedyCoreMessagesConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. friendship.Block: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. friendship.Follow: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. friendship.Friend: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. friendship.FriendshipRequest: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. likes.UserLike: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. uploads.Image: (models.W042) Auto-created primary key used when not defining a primary key type, by default ''. HINT: Configure the DEFAULTAUTOFIELD setting or the AppConfig.defaultautofield attribute to point to a subclass of AutoField, e.g. ''. These models should not use auto-created primary keys! I already defined the primary key in the ancestor of the model. For example class Entity which class User inherits from. It looks to me like a bug in Django 3.2 alpha.


ModelChoiceField does not provide value of invalid choice when raising ValidationError Description (last modified by Aaron Wiegel) Compared with ChoiceField and others, ModelChoiceField does not show the value of the invalid choice when raising a validation error. Passing in parameters with the invalid value and modifying the default error message for the code invalidchoice should fix this. From source code: class ModelMultipleChoiceField(ModelChoiceField): """A MultipleChoiceField whose choices are a model QuerySet.""" widget = SelectMultiple hiddenwidget = MultipleHiddenInput defaulterrormessages = { 'invalidlist': ('Enter a list of values.'), 'invalidchoice': ('Select a valid choice. %(value)s is not one of the' ' available choices.'), 'invalidpkvalue': ('“%(pk)s” is not a valid value.') } ... class ModelChoiceField(ChoiceField): """A ChoiceField whose choices are a model QuerySet.""" # This class is a subclass of ChoiceField for purity, but it doesn't # actually use any of ChoiceField's implementation. defaulterrormessages = { 'invalidchoice': _('Select a valid choice. That choice is not one of' ' the available choices.'), } ...


Migrate signals verbose stdout emissions are not capturable Description The migrate command takes a --verbosity flag that is passed down to emitpremigratesignal and emitpostmigratesignal functions but these are not provided which stdout the output should be directed to. This makes testing migrate -v2 through call_command pollute when it should be directed to the provided stdout as discovered in


Saving parent object after setting on child leads to data loss for parents with non-numeric primary key. Description (last modified by Charlie DeTar) Given a model with a foreign key relation to another model that has a non-auto CharField as its primary key: class Product(models.Model): sku = models.CharField(primarykey=True, maxlength=50) class Order(models.Model): product = models.ForeignKey(Product, ondelete=models.CASCADE) If the relation is initialized on the parent with an empty instance that does not yet specify its primary key, and the primary key is subsequently defined, the parent does not "see" the primary key's change: with (): order = Order() order.product = Product() = "foo" () () assert (productid="").exists() # Succeeds, but shouldn't assert (product=order.product).exists() # Fails Instead of productid being populated with , it is set to emptystring. The foreign key constraint which would enforce the existence of a product with sku="" is deferred until the transaction commits. The transaction does correctly fail on commit with a ForeignKeyViolation due to the non-existence of a product with emptystring as its primary key. On the other hand, if the related unsaved instance is initialized with its primary key before assignment to the parent, it is persisted correctly: with (): order = Order() order.product = Product(sku="foo") () () assert (product=order.product).exists() # succeeds Committing the transaction also succeeds. This may have something to do with how the Order.productid field is handled at assignment, together with something about handling fetching of auto vs non-auto primary keys from the related instance.


Delete distinct produces an unexpected and potentially harmful SQL Description (last modified by egism) I was looking for a way to delete the first Comment of each Post (a sample domain). Since I know that every new Post starts with a system generated comment I decided to go with: Comment.objects.orderby('postid', 'createdat').distinct('postid').delete() Before proceeding I tested it with: Comment.objects.orderby('postid', 'createdat').distinct('postid').count() Made sure the result actually contains what I needed and proceeded with the delete(). The result was rather surprising. I was notified that the whole table was wiped clean. I then checked the actual SQL that was executed and it was a simple DELETE FROM comments;. As an ORM user, I would say it is the worst outcome possible and I would at least expect an error in such a case or ideally a SQL of what I was trying to achieve. At the same time, count and delete produces an inconsistent result which is even more mistaking. Potential solutions: raise an error with a decent explanation produce a desired SQL according to the query Since I have never submitted a change to Django, I have a very limited knowledge of the ORM and its intricacies. I could give it a try and issue a patch for this with some guidance.


Add in admindocs Description (last modified by Nat S Dunn) From the documentation (): "The models section of the admindocs page describes each model in the system along with all the fields, properties, and methods available on it." Currently, properties decorated with are not included. Please include them. And possibly include other (or all) descriptors/attributes.


Database converters (fromdbvalue) not called for returningfields on insert Description Making a subclass of BigAutoField, I've found that, unlike all other query pathways, on insert the returned integer is not passed through any database converters defined for the field - including the fromdbvalue hook. This means that a field which would normally use a wrapper class has instead a plain integer. Take this field: class MyAutoField(models.BigAutoField): def fromdbvalue(self, value, expression, connection): if value is None: return None return MyIntWrapper(value) def getprepvalue(self, value): if value is None: return None return int(value) And a model that uses it: class AutoModel(models.Model): id = MyAutoField(primarykey=True) Queried instances have the wrapper class for id: am = () <MyIntWrapper: 1But on creation, the returned integer is directly set as an attribute on the class: am2 = () 2 This also affects bulkcreate on backends that support fetching the primary key value: ams = [AutoModel()] AutoModel.objects.bulkcreate(ams) [<AutoModel: AutoModel object (2)] ams[0].id 2


LiveServerTestCase's ThreadedWSGIServer doesn't close database connections after each thread Description In Django 2.2.17, I'm seeing the reappearance of after it was fixed in 1.11. is the issue where the following error will occur at the conclusion of a test run when destroytestdb() is called: OperationalError: database "testmyapp" is being accessed by other users This error happens when not all of the database connections are closed. In my case today, I'm seeing this when running a single test that is a LiveServerTestCase. I see it in approximately half of my test runs, so it's not wholly deterministic (it's a race condition). There weren't a whole lot of changes in the LiveServerTestCase-related code between 1.11 and 2.2, so I looked at them individually. Issue threading support to LiveServerTestCase. One of the changes it made was changing LiveServerThread to use ThreadedWSGIServer instead of WSGIServer. LiveServerThread is used by LiveServerTestCase. When I tried modifying LiveServerThread to use the old WSGIServer, I could no longer reproduce the above error. My changes were as follows: class NonThreadedLiveServerThread(LiveServerThread): def createserver(self): return WSGIServer((, ), QuietWSGIRequestHandler, allowreuseaddress=False) class MyTest(LiveServerTestCase): serverthread_class = NonThreadedLiveServerThread The CPython docs describe ThreadingMixIn as defining an attribute "which indicates whether or not the server should wait for thread termination." Consistent with what I described above, Aymeric said the following on ticket , seeming to foreshadow issues like this one: more threading will certainly create more race conditions on shutdown, especially when it comes to the database connections — it's taken months to eliminate most from LiveServerTestCase, and I'm sure there are still some left,


Introspection doesn't return column order for unique constraints on SQLite. Description Introspection doesn't return column order for unique constraints on SQLite.


"TypeError: cannot pickle" when applying | operator to a Q object Description (last modified by Daniel Izquierdo) Using a reference to a non-pickleable type of object such as dictkeys in a Q object makes the | operator fail: from import Q Q(xin={}.keys()) <Q: (AND: ('xin', dictkeys([])))Q() | Q(xin={}.keys()) Traceback (most recent call last): ... TypeError: cannot pickle 'dict_keys' object Even though this particular example could be solved by doing Q() | Q(xin={}) it still feels like using .keys() should work. I can work on a patch if there's agreement that this should not crash.


Q(...) & Exists(...) raises a TypeError Description Exists(...) & Q(...) works, but Q(...) & Exists(...) raise a TypeError Here's a minimal example: In [3]: Exists(()) & Q() Out[3]: <Q: (AND: object at 0x7fc18dd0ed90, (AND: ))In [4]: Q() & Exists(()) TypeError Traceback (most recent call last) <ipython-input-4-21d3dea0fcb9in <module----1 Q() & Exists(()) ~/Code/venv/ecom/lib/python3.8/site- in and(self, other) 90 91 def and(self, other): ---92 return self.combine(other, self.AND) 93 94 def invert(self): ~/Code/venv/ecom/lib/python3.8/site- in combine(self, other, conn) 71 def _combine(self, other, conn): 72 if not isinstance(other, Q): ---73 raise TypeError(other) 74 75 # If the other Q() is empty, ignore it and just use . TypeError: object at 0x7fc18dd21400The & (and |) operators should be commutative on Q-Exists pairs, but it's not I think there's a missing definition of rand somewhere.


Make Index and Constraint repr consistent. Description (last modified by Hannes Ljungberg) Index, UniqueConstraint, ExclusionConstraint and CheckConstraint currently have slightly different formatting on their repr methods. We should keep them consistent.


Support a default value for Sum (and possibly other aggregation functions) Description By default, annotate(sumfield = Sum(...)) results in sumfield being NULL if there were no values to sum. In most cases, 0 would be a better option here, for proper sorting in a later order_by, and for comparisons using lt/gt/gte/lte. A monkeypatch to implement default values for NULL using COALESCE is available here:


Admin nevercache decorators needs methoddecorator Description The login and logout views in Django's admin site are using the nevercache decorator, but they are both instance methods, so they need to use the methoddecorator. This is not a bug because the nevercache decorator is only operating on the response. The first argument to the decorator is supposed to be the request, but it is in-fact the adminsite instance (self). All the arguments are then passed to the view function and the decorator operate on the response. If you try to use a different decorator that uses the request (such as requiredhttpmethods) you will fail. AttributeError: 'CustomAdminSite' object has no attribute 'method' Related issue from long time ago:


Ticket breaks autocomplete for inherited models Description Unfortunately, Closed ticket breaks autocomplete for the case of inherited models. For example, we have the following code: from import models class Tag(models.Model): pass class Foo(models.Model): tags = models.ManyToManyField(Tag) class Bar(Foo): pass from django.contrib import admin from . import models class Foo(admin.ModelAdmin): autocompletefields = ('tags',) class Bar(admin.ModelAdmin): autocompletefields = ('tags',) Now, autocomplete for admin.Foo will work but not for admin.Bar because () calculates a wrong value of a variable tofieldname, namely fooptr instead of id, whereupon following look up at ().filter(**{'%sin' % tofieldname: selectedchoices}) raises an exception because models.Tag does not have foo_ptr.


MultiValueField ignores a required value of a sub field Description (last modified by Takayuki Hirai) A field and a form definition: from import ( Form, CharField, MultiValueField, MultiWidget, ) class MF(MultiValueField): widget = MultiWidget def init(self): fields = [ CharField(required=False), CharField(required=True), ] widget = (widgets=[ f.widget for f in fields ], attrs={}) super(MF, self).init( fields=fields, widget=widget, requireallfields=False, required=False, ) def compress(self, value): return [] class F(Form): mf = MF() When the form is passed empty values for both sub fields, form.isvalid() == True. But I expected isvalid() returns False, because one of the sub fields is set as required. f = F({ 'mf0': '', 'mf1': '', }) assert f.isvalid() == True # I expect this should return False On the other hand, When one of its sub field is passed a non-empty value, form.isvalid() == False f = F({ 'mf0': 'xxx', 'mf1': '', }) assert f.is_valid() == Flase If above behavior is not expected, please fix this problem.


Add support for PostgreSQL passfile to dbshell. Description (last modified by Mariusz Felisiak) The dbshell management commands all carry the risk of leaking passwords through process information (as noted in a comment in ). As of Django 3.2, there is the settingstocmdargsenv class method, which provides an API to generate the environment needed to call the utility. Using the environment is somewhat more secure, but the environment of processes can potentially still be read. Both MySQL and PostgreSQL advise against using the respective environment variables. Specifying a password file works for connections but dbshell doesn't support it, see comment. I want to propose a way to solve this. I already did this in django-dbbackup, which also happened to construct a command line before:  The mechanism is that in addition to the environment and args, a temporary file is generated. For PostgreSQL, this is a file in .pgpass format; for MySQL, it could be an options file. I wrapped that handling in a neat context manager. For Django itself, I did a quick shot at PostgreSQL as well, as attached in the patch. The patch is not complete, and is only intended as a base for discussion. If we find consensus about the mechanism, I will happily complete it and extend to the other backends.


HashedFilesMixin's postprocess() yields multiple times for the same file Description As part of fixing , the implementation of HashedFilesMixin (used by both ManifestStaticFilesStorage and CachedStaticFilesStorage) was changed such that it performs several passes against the found files, therefore ensuring that nested references between the files are correctly handled. Performing these several passes is both necessary and not a problem in itself, however at present postprocess() returns (via yield) the same original filename multiple times back to collectstatic's collect(). For example using Django 1.11.5 with the app enabled: $ collectstatic --noinput | grep '' Copying '/home/vagrant/python/lib/python2.7/site-' Post-processed '' as '' Post-processed '' as '' Post-processed '' as '' ...whereas I would have only expected: $ collectstatic --noinput | grep '' Copying '/home/vagrant/python/lib/python2.7/site-' Post-processed '' as '' The problem with this is that: 1) collectstatic's collect() assumes that the number of yields is the number of files that were post-processed. As such, by yielding multiple times for the same original file, the stats shown at the end (eg "X files copied, ..., Y post-processed") are wrong, since there can be more files post processed than were copied 2) For anyone subclassing ManifestStaticFilesStorage who handles the yielded files as they come in, duplicate work is performed. For example WhiteNoise ends up compressing the same file multiple times, increasing deploy times due to expensive Brotli compression. And I'm guessing S3 backends similarly might upload multiple times. 3) Even if it were argued that all files should be yielded, this isn't what is actually happening since only some of the intermittent files are yielded (compare the "Post-processed ..." output to the file list in -- the instance is missing). Note that this issue whilst related to is actually different for two reasons: 1) Even if intermediate files need to be left around for now, IMO they still shouldn't be passed back to collectstatic and/or subclasses (they are a lower-level implementation detail) 2) The duplicate yields occur even for assets that don't need adjusting during the second pass. For example: $ collectstatic --noinput | grep '' Post-processed '' as '' Post-processed '' as '' Post-processed '' as '' This issue was actually mentioned in the PR that the feature:


Make warn (rather than blow up) on missing directories Description At present if the STATICFILES_DIRS setting contains references to directories which do not exist then the whole command will die with an OSError. A situation I've seen bite a few newcomers to Django is that they will have an empty static directory which (being empty) won't get tracked by git. This means that collectstatic works when they run it locally, but blows up with a (to them) cryptic error when they try to deploy. If we made collectstatic simply log a warning in the case of non-existent directories and continue processing then we could avoid this problem and remove one possible source of frustration for newcomers. If this approach seems acceptable to others I am happy to submit an appropriate patch.


Can create model instance with conflicting args and kwargs value for the same field Description (last modified by Anton Samarchyan) Discovered while working on To reproduce it, add the following test (it's using ) from import six def testcannotspecifysamefieldwithargsandkwargstoo(self): six.assertRaisesRegex( self, TypeError, "init() got multiple values for argument 'headline'", Article, None, # id 'args based headline', headline='kwargs based headline', pubdate=datetime(2005, 7, 31), )


Admin's rawidfield check admin.E002 doesn't catch .attname mis-references Description Since admin.E002 relies on models.Options.getfield which allows retrieval of fields by both name and attname referring to fields by attname while only name is taken into consideration allows the check to pass while rawidfields is not honoured. e.g. class BookAdmin(ModelAdmin): rawidfields = ['authorid'] passes admin.E002 but the author field won't use the rawid feature. The checkrawidfieldsitem method should also make sure to check == fieldname on field retrieval success and return refertomissingfield(field=field_name, option=label, obj=obj, id='admin.E002') when it's not the case.


()/valueslist() with JSONField returns integers instead of booleans on SQLite. Description (last modified by Matthew Cornell) I have a model with a JSONField: class PredictionData(models.Model): data = models.JSONField() One of the rows contains this dict: {'value': True}. I'm querying the model's JSON using 'datavalue': PredictionData.objects.valueslist('data', 'datavalue') I get correct results for postgres (a boolean) but incorrect for sqlite3 (an int). For this query, sqlite3 wrongly returns: ({'value': True}, 1) whereas postgres correctly returns ({'value': True}, True) Same behavior with False/0. versions: Python 3.9.1 sqlite3.sqlite_version # '3.33.0' django.version # '3.1.7'


Allow calling reversed() on an OrderedSet Description Currently, OrderedSet isn't reversible (i.e. allowed to be passed as an argument to Python's reversed()). This would be natural to support given that OrderedSet is ordered. This should be straightforward to add by adding a reversed() method to OrderedSet.


Change automatic migration naming from date-based to operation-based Description Following and a discussion on django-developers with broad consensus, change the way migrations are automatically named from date-based to operation-based. That is never name migrations based upon the current date (auto_YYYYMMDD) and instead always based on the operations they contain, rather than the current behaviour which uses either style.


Meta.ordering fields must not be included in GROUP BY clause Description This continues (closed) [1] ticket. I beleave it was not properly fixed in commit []. While commit [] removes ORDER BY when Meta.ordering is used it still does populates GROUP BY with Meta.ordering fields thus leads to wrong aggregation. PR with test case was at [2]. [1] [2]


handler500 as a Class-based view raises SystemCheckError Description (last modified by Daniyal Abbasi) Setting handler500 as a Class-Based view raises the following error which running checks. $ python check SystemCheckError: System check identified some issues: ERRORS: ?: (urls.E007) The custom handler500 view '' does not take the correct number of arguments (request). In my root , I have the following configuration, handler404 = () handler500 = () I believe this is due to the function checkcustomerrorhandlers in The signature variable in this function is expected to match (request, exception) for all handlers except for handler500 which is expected to have only (request). A positional argument, templatename is also present. While using class based views, we get two positional arguments (self, request) and then it recieves args and kwargs. The check is permitting other handlers as the number of arguments coincidentally match. I suggest a fix in the checkcustomerror_handlers which first checks if the handler are function based or class based, and then it preceed the check with the appropriate number of arguments.


Combining Q() objects with boolean expressions crashes. Description (last modified by jonathan-golorry) Currently Q objects with 1 child are treated differently during deconstruct. from import Q Q(x=1).deconstruct() ('', (), {'x': 1}) Q(x=1, y=2).deconstruct() ('', (('x', 1), ('y', 2)), {}) This causes issues when deconstructing Q objects with a non-subscriptable child. from import getusermodel from import Exists Q(Exists(getusermodel()(username='jim'))).deconstruct() Traceback (most recent call last): File "<console", line 1, in <moduleFile "...", line 90, in deconstruct kwargs = {child[0]: child[1]} TypeError: 'Exists' object is not subscriptable Patch  removes the special case, meaning single-child Q objects deconstruct into args instead of kwargs. A more backward-compatible approach would be to keep the special case and explicitly check that the child is a length-2 tuple, but it's unlikely that anyone is relying on this undocumented behavior.


Validate the type of ALLOWEDHOSTS Description Python has soft behavior for conducting iteration process over lists and over strings making them look the same: for charoritem in strorlist: -- can be character or list item It would be better if it would have more strict behavior, for example, for char in (): -- now can be only of string type and class would not have method and for list for item in somelist.listitems(): -- class would not have method This soft behavior usually leads to many nasty bugs to appear. Our two software engineers from our team wasted about 1 hour debugging the issue with ALLOWEDHOSTS being initialized with string in which is included at the end of Django was matching each separate character of ALLOWEDHOSTS string against the "Host:" header from an incoming HTTP request. An obvious self-suggesting solution is to add a new system check that will check the type of ALLOWED_HOSTS if it is string or not and notify the developer about possible improper configuration. I think blacklist checking (string or not) is more appropiate here, but I can be wrong.


CsrfViewMiddleware assumes referer header can be parsed Description Django's CsrfViewMiddleware assumes that the HTTP referer header is valid when checking it. Specifically, it doesn't handle the case of urlparse() raising a ValueError in this line (e.g. for urls like 'https://['):


() doesn't handle functools.partial() nicely. Description (last modified by Nick Pope) When a partial function is passed as the view, the repr shows the func argument as functools.partial which isn't very helpful, especially as it doesn't reveal the underlying function or arguments provided. Because a partial function also has arguments provided up front, we need to handle those specially so that they are accessible in repr. ISTM that we can simply unwrap functools.partial objects in ().


tolocale should be idempotent Description In summary: from import tolocale tolocale('en-us') 'enUS' # <- Fine tolocale(tolocale('en-us')) 'enus' # <- Bad Typically, this breaks using e.g. with a locale code like ptBR or zh_Hans. of course, we could document that override takes a language code, not a locale code, but if we can support both without much effort, this would be a lot better.


Extra dot in cloned test database file names on SQLite when using --parallel. Description When asking Django to create on-disk (instead of in-memory) sqlite cloned test databases by using the --parallel flag, the cloned file names are created as, e.g., "testdb1..sqlite3", "testdb2..sqlite3", etc. (with two dots instead of one). If the specified test database name lacks any extension, e.g. just 'testdb', then the cloned file names have a trailing dot: "testdb1.", "testdb2.", "testdb3.", etc. This is due to this line:  gettestdbclonesettings() which constructs the file name using the string: '{}{}.{}'.format(root, suffix, ext) However, () already includes the dot in the returned extension ('ext'). Removing the dot from the format string seems the only change needed to fix it: '{}{}{}'.format(root, suffix, ext) From the github file history it looks like this quirk has been there since the --parallel flag was first introduced (commit in Sep 2015). To reproduce the issue: In , force on-disk instead of in-memory test databases by specifying any test db name: DATABASES = { 'default': { ENGINE: '', NAME: 'db.sqlite3', TEST: { 'NAME': testdb.sqlite3', } } } Create any model with migrations, and more than one TestCase class (to ensure --parallel creates clones of the test database). And on the command line, run tests with --parallel and --keepdb to see the db files generated. This should be run on a multi-core processor. test --parallel --keepdb


Query optimization in YearLookup breaks filtering by "isoyear" Description (last modified by Florian Demmer) The optimization to use BETWEEN instead of the EXTRACT operation in YearLookup is also registered for the "isoyear" lookup, which breaks the functionality provided by ExtractIsoYear when used via the lookup. This has unfortunately been broken ever since ExtractIsoYear was introduced in Django 2.2 via and wasn't easy to track down since ExtractIsoYear when used by itself eg. in an annotation works perfectly fine. Just when using the lookup in a filter, the optimization is used (even when explicitly using an annotation): qs = DTModel.objects.annotate(extracted=ExtractIsoYear('startdate')).only('id') print() SELECT "dbfunctionsdtmodel"."id", EXTRACT('isoyear' FROM "dbfunctionsdtmodel"."startdate") AS "extracted" FROM "dbfunctionsdtmodel" print((extracted=2020).query) SELECT "dbfunctionsdtmodel"."id", EXTRACT('isoyear' FROM "dbfunctionsdtmodel"."startdate") AS "extracted" FROM "dbfunctionsdtmodel" WHERE "dbfunctionsdtmodel"."startdate" BETWEEN 2020-01-01 AND 2020-12-31 print((startdateisoyear=2020).only('id').query) SELECT "dbfunctionsdtmodel"."id" FROM "dbfunctionsdtmodel" WHERE "dbfunctionsdtmodel"."startdate" BETWEEN 2020-01-01 AND 2020-12-31 This results in the wrong data being returned by filters using isoyear. This PR fixes the behaviour, reverts the invalid changes to the tests and extends one test to catch this problem:


Handle request.gethost() raising DisallowedHost in CsrfViewMiddleware.originverified() Description Currently, on this line, CsrfViewMiddleware.originverified() doesn't handle request.gethost() raising DisallowedHost:  Since Django was previously fixed to handle request.gethost() raising DisallowedHost elsewhere in CsrfViewMiddleware.processview() (see ticket ), it seems like it should be handled here, too.


Add support for precision argument to Round Description Django's Round function currently only supports rounding to the nearest integer. If you need to round to a more specific precision you need to roll out your own solution. But as far as I can tell, all backends natively supported by Django support a second argument to Round: Postgres:  Sqlite:  MySQL:  Oracle:  In my project (postgres only) my work around was to declare my own custom function: class Round(Func): function = 'ROUND' arity = 2 But having this built-in would be quite useful.


Value() with DecimalField crashes on SQLite. Description If you try to annotate anything to a decimal field Django will pass a string to createdecimalfromfloat function. These are the offending lines example code: ().annotate(shearlength=Value(1, output_field=DecimalField()))


DEFAULTAUTOFIELD subclass check fails for subclasses of BigAutoField and SmallAutoField. Description Set DEFAULTAUTOFIELD = "" , with contents of from import models class MyBigAutoField(models.BigAutoField): pass class MyModel(models.Model): pass Django then crashes with: Traceback (most recent call last): File "", line 21, in <modulemain() File "", line 17, in main executefromcommandline() File "/.../venv/lib/python3.9/site-", line 419, in executefromcommandline utility.execute() File "/.../venv/lib/python3.9/site-", line 395, in execute () File "/.../venv/lib/python3.9/site-", line 24, in setup apps.populate(settings.INSTALLEDAPPS) File "/.../venv/lib/python3.9/site-", line 114, in populate appconfig.importmodels() File "/.../venv/lib/python3.9/site-", line 301, in importmodels self.modelsmodule = importmodule(modelsmodulename) File "", line 127, in importmodule return bootstrap.gcdimport(name[level:], package, level) File "<frozen importlib.bootstrap", line 1030, in gcdimport File "<frozen importlib.bootstrap", line 1007, in findandload File "<frozen importlib.bootstrap", line 986, in findandloadunlocked File "<frozen importlib.bootstrap", line 680, in loadunlocked File "<frozen importlib.bootstrapexternal", line 790, in execmodule File "<frozen importlib.bootstrap", line 228, in callwithframesremoved File "", line 8, in <moduleclass MyModel(models.Model): File "/.../venv/lib/python3.9/site-", line 320, in new newclass.prepare() File "/.../venv/lib/python3.9/site-", line 333, in prepare opts.prepare(cls) File "/.../venv/lib/python3.9/site-", line 285, in prepare pkclass = self.getdefaultpkclass() File "/.../venv/lib/python3.9/site-", line 246, in getdefaultpkclass raise ValueError( ValueError: Primary key '' referred by DEFAULTAUTOFIELD must subclass AutoField. This can be fixed in AutoFieldMeta.subclasscheck by allowing subclasses of those classes in the subclasses property.


()/valueslist() crash on combined querysets ordered by unannotated columns. Description (last modified by Iuri de Silvio) Django 3.2 fails with this query: qs1 = () qs2 = () (qs2).valueslist('name').first() It worked until Django 3.1.8. This commit[1] to be exactly.  This is the broken generated query. In the second query, it fetches from the first table. SQL SELECT "queriescelebrity"."name", "queriescelebrity"."id" AS "orderbycol2" FROM "queriescelebrity" UNION SELECT "queriesreservedname"."name", "queriescelebrity"."id" AS "orderbycol2" -- HERE IS THE PROBLEM FROM "queriesreservedname" ORDER BY (2) ASC LIMIT 1 Before, it was: SQL SELECT "queriescelebrity"."name", "queriescelebrity"."id" FROM "queriescelebrity" UNION SELECT "queriesreservedname"."name", "queriesreservedname"."id" FROM "queriesreservedname" ORDER BY (2) ASC LIMIT 1


CookieStorage for contrib.messages crashes after upgrade to django 3.2 Description (last modified by Jan Pieter Waagmeester) After upgrading to django 3.2, a previously stored cookie for contrib.messages crashes in  Django Version: 3.2 Python Version: 3.8.2 Traceback (most recent call last): File "/home/obs/virtualenv/lib/python3.8/site-", line 47, in inner response = getresponse(request) File "/home/obs/virtualenv/lib/python3.8/site-", line 119, in call response = self.processresponse(request, response) File "/home/obs/virtualenv/lib/python3.8/site-", line 23, in processresponse unstoredmessages = (response) File "/home/obs/virtualenv/lib/python3.8/site-", line 127, in update messages = self.loadedmessages + self.queuedmessages File "/home/obs/virtualenv/lib/python3.8/site-", line 79, in loadedmessages messages, allretrieved = () File "/home/obs/virtualenv/lib/python3.8/site-", line 25, in get messages, allretrieved = () File "/home/obs/virtualenv/lib/python3.8/site-", line 86, in get messages = (data) File "/home/obs/virtualenv/lib/python3.8/site-", line 175, in decode return (data, serializer=MessageSerializer) File "/home/obs/virtualenv/lib/python3.8/site-", line 195, in unsignobject data = b64decode(base64d) File "/home/obs/virtualenv/lib/python3.8/site-", line 68, in b64decode return base64.urlsafeb64decode(s + pad) File "", line 133, in urlsafeb64decode return b64decode(s) File "", line 87, in b64decode return binascii.a2bbase64(s) Exception Type: Error at /user/login/ Exception Value: Invalid base64-encoded string: number of data characters (369) cannot be 1 more than a multiple of 4 (redacted) contents of the 'messages' cookie: '[["jsonmessage",0,25,"Successfully signed in as ' '"],["jsonmessage",0,25,"Successfully ' 'signed in as jieter."],["jsonmessage",0,25,"Ingelogd als ' '"],["jsonmessage",0,25,"Ingelogd ' 'als ' '"],["jsonmessage",0,20,"Bevestigingsmail ' 'verzonden naar "],["jsonmessage",0,25,"Ingelogd ' 'als ' '"]]:1lTkj1:j3PlpYSKiqPTMAB6p2Q00eE8j6k7n0Sg-IpXG7Y')


Cannot combine two queryset in a subquery Description (last modified by Raffaele Salmaso) [Sample project  and run query] Django 3.2 fails this query (a combined queryset in a subquery): import datetime as dt from decimal import Decimal from import settings from import models from import Case, OuterRef, Q, Subquery, Value, When from import timezone class UserQuerySet(models.QuerySet): def annotateactivesubscriptionid(self): return self.annotate( activesubscriptioniddb=Subquery( () .annotate( planorder=Case( When(plancode="BASE", then=Value(1)), default=Value(0), outputfield=models.PositiveSmallIntegerField(), ) ) .filter(user=OuterRef("id")) .orderby("planorder", "-id") .values("id")[:1] ) ) class User(models.Model): objects = models.Manager.fromqueryset(UserQuerySet)() class Plan(models.Model): code = models.CharField(verbosename="Codice", maxlength=255) class SubscriptionQuerySet(models.QuerySet): def willberenewedtoday(self): today = () return (startdatelte=today).exclude(usersubscriptionsstartdate=today).distinct() def active(self): return (enabled=True).distinct() | self.willberenewedtoday() class Subscription(models.Model): user = models.ForeignKey(User, verbosename="Utente", ondelete=models.CASCADE, relatedname="subscriptions") plan = models.ForeignKey(Plan, ondelete=models.CASCADE, verbosename="Piano di abbonamento") startdate = models.DateField(verbosename="Data di inizio", ) enabled = models.BooleanField(verbosename="Abilitato", default=True) objects = models.Manager.fromqueryset(SubscriptionQuerySet)() print(User.objects.annotateactivesubscriptionid().count()) with django 3.1.8 SELECT "subqueryuser"."id", ( SELECT "subquery"."id" FROM ( SELECT DISTINCT U0."id", CASE WHEN (U2."code" = BASE) THEN 1 ELSE 0 END FROM "subquerysubscription" U0 INNER JOIN "subqueryplan" U2 ON (U0."planid" = U2."id") WHERE ( ( U0."enabled" OR ( U0."startdate" <= 2021 - 04 - 13 AND NOT ( U0."userid" IN ( SELECT U2."userid" FROM "subquerysubscription" U2 WHERE U2."startdate" = 2021 - 04 - 13 ) ) ) ) AND U0."userid" = "subqueryuser"."id" ) ORDER BY CASE WHEN (U2."code" = BASE) THEN 1 ELSE 0 END ASC, U0."id" DESC LIMIT 1 ) subquery ) AS "activesubscriptioniddb" FROM "subqueryuser" with django 3.2 ( SELECT "subqueryuser"."id", ( SELECT "subquery"."id" FROM ( SELECT DISTINCT U0."id", CASE WHEN (U2."code" = BASE) THEN 1 ELSE 0 END FROM "subquerysubscription" U0 INNER JOIN "subqueryplan" U2 ON (U0."planid" = U2."id") WHERE ( ( U0."enabled" OR ( U0."startdate" <= 2021 - 04 - 13 AND NOT ( EXISTS( SELECT (1) AS "a" FROM "subquerysubscription" V2 WHERE ( V2."startdate" = 2021 - 04 - 13 AND V2."userid" = V0."userid" ) LIMIT 1 ) ) ) ) AND U0."userid" = "subqueryuser"."id" ) ORDER BY CASE WHEN (U2."code" = BASE) THEN 1 ELSE 0 END ASC, U0."id" DESC LIMIT 1 ) subquery ) AS "activesubscriptioniddb" FROM "subqueryuser" Traceback (most recent call last): File ".venvs/django32/lib/python3.8/site-", line 84, in execute return (sql, params) File ".venvs/django32/lib/python3.8/site-", line 423, in execute return Database.Cursor.execute(self, query, params) sqlite3.OperationalError: no such column: The above exception was the direct cause of the following exception: Traceback (most recent call last): File "", line 22, in <modulemain() File "", line 18, in main executefromcommandline() File ".venvs/django32/lib/python3.8/site-", line 419, in executefromcommandline utility.execute() File ".venvs/django32/lib/python3.8/site-", line 413, in execute self.fetchcommand(subcommand).runfromargv() File ".venvs/django32/lib/python3.8/site-", line 354, in runfromargv self.execute(args, cmdoptions) File ".venvs/django32/lib/python3.8/site-", line 398, in execute output = (args, options) File "/home/raf/src/fiscozen/django-", line 11, in handle print(()) File ".venvs/django32/lib/python3.8/site-", line 412, in count return () File ".venvs/django32/lib/python3.8/site-", line 526, in getcount number = obj.getaggregation(using, ['count'])['count'] File ".venvs/django32/lib/python3.8/site-", line 511, in getaggregation result = compiler.executesql(SINGLE) File ".venvs/django32/lib/python3.8/site-", line 1175, in executesql cursor.execute(sql, params) File ".venvs/django32/lib/python3.8/site-", line 98, in execute return super().execute(sql, params) File ".venvs/django32/lib/python3.8/site-", line 66, in execute return self.executewithwrappers(sql, params, many=False, executor=self.execute) File ".venvs/django32/lib/python3.8/site-", line 75, in executewithwrappers return executor(sql, params, many, context) File ".venvs/django32/lib/python3.8/site-", line 84, in execute return (sql, params) File ".venvs/django32/lib/python3.8/site-", line 90, in exit raise djexcvalue.withtraceback(traceback) from excvalue File ".venvs/django32/lib/python3.8/site-", line 84, in execute return (sql, params) File ".venvs/django32/lib/python3.8/site-", line 423, in execute return Database.Cursor.execute(self, query, params) no such column: V0.user_id Tested with 3.2,  (next 3.2.1) and  (main)


Cannot run makemigrations management command without a SECRETKEY Description I believe intended to fix this issue. Steps to reproduce: (mktemp -d) $ python -m venv venv $ source venv/bin/activate $ pip install 'Django=3.2' $ python -m django startproject foo $ sed -ri '/SECRETKEY/d' # Remove SECRETKEY from settings $ PYTHONPATH=foo DJANGOSETTINGS_MODULE="foo.settings" python -m django makemigrations --check The output is attached.


caches.W002 check does not support tuples in STATICFILESDIRS Description The caches.W002 check defined here:  inspects the values of each entry in STATICFILESDIRS here:  and passes them to pathlib.Path(staticfilesdir) which expects a string, however according to the documentation each entry in STATICFILESDIRS may be either a string or a tuple:  If a STATICFILESDIRS entry is provided as a tuple, this check fails with: TypeError: expected str, bytes or os.PathLike object, not tuple in in parse_args at line 680


Allow autoreloading of Description (last modified by Mariusz Felisiak) The original fix [1] only attempted to deal with -m where bar is a package and exists under foo/bar. When a dotted name for a module (for example, where resides under foo/bar) is specified like -m , the resulting arguments end up being -m , which is uncalled for. [1]  Fixed detection when started non-django modules with "python -m" in autoreloader.


database client runshell doesn't respect os.environ values in some cases Description (last modified by Konstantin Alekseev) postgresql client returns empty dict instead of None for env as a result os.environ is not used and empty env passed to subprocess. Bug introduced in  PR


Migration autodetector changes relatedname for self-referential ManyToManyField. Description Migration autodetector no longer adds a model name to the relatedname attribute for self-referential ManyToManyField, e.g. for a field class MyModel2(models.Model): field3 = models.ManyToManyField('self') it creates a migration with relatedname='field3rel+' instead of relatedname='mymodel2field3+'. Regression in (see ).


LocaleMiddleware not recognising properly zh-Hant-HK from the accept-language header Description If both zh-hans and zh-hant are in settings.LANGUAGES (and in this order) the LocaleMiddleware is choosing the first one instead of the second as would be expected. The actual faulty code seems to be in


case sensitive issue on subquery aliases generated Description When generating aliases for subqueries, django uses 2 different cases: 'Col%d'  and 'col%d'  Here is a concrete example, using --v 2 --debug-sql SELECT COUNT(*) FROM (SELECT "subquery"."col1", "subquery"."col2", "subquery"."col3", "subquery"."col4" FROM (SELECT DISTINCT "queriestag"."id" AS Col1, "queriestag"."name" AS Col2, "queriestag"."parentid" AS Col3, "queriestag"."categoryid" AS Col4, "queriesnamedcategory"."name" AS Col5 FROM "queriestag" LEFT OUTER JOIN "queriesnamedcategory" ON ("queriestag"."categoryid" = "queriesnamedcategory"."dumbcategoryptrid") ORDER BY "queriesnamedcategory"."name" ASC LIMIT 3) subquery) subquery; This would fail on a database with case sensitive column names. It will be trivial to fix if deemed necessary.


Database uses cursor after it is closed Description (last modified by ecogels) The return bool(cursor.rowcount) is outside of the with block, so the cursor will have been closed at that point. From the DB API 2.0 spec: "The cursor will be unusable from this point forward"  As the main backend drivers don't mind it I suppose that is is open to interpretation. PR


URLValidator tests failing on Python versions patched for bpo- Description On Python versions with a fix for bpo- (i.e. 3.10.0b1 and the 3.9 git branch, not released yet) the following tests fail: FAIL: testvalidators () [URLValidator] (value='\n') Traceback (most recent call last): File "", line 59, in testPartExecutor yield File "", line 546, in subTest yield File "/tmp/portage/dev-python/django-3.2.1/work/Django-", line 328, in testvalidators validator(value) File "", line 203, in exit self.raiseFailure("{} not raised".format(excname)) File "", line 135, in raiseFailure raise self.testcase.failureException(msg) AssertionError: ValidationError not raised FAIL: testvalidators () [URLValidator] (value='http://[::ffff:192.9.5.5]\n') Traceback (most recent call last): File "", line 59, in testPartExecutor yield File "", line 546, in subTest yield File "/tmp/portage/dev-python/django-3.2.1/work/Django-", line 328, in testvalidators validator(value) File "", line 203, in exit self.raiseFailure("{} not raised".format(excname)) File "", line 135, in raiseFailure raise self.testcase.failureException(msg) AssertionError: ValidationError not raised FWICS, the project is that django rejects URLs based on the split URL components. However, the bpo- fix changes URL splitting behavior to strip all instances of LF, CR and tab characters before splitting, so they never reach the validator. I'm not sure what the best fix is. One option is to reject URLs containing the forbidden characters early. Another is to go with the new recommendation and assume that LF, CR and tabs are to stripped silently.


Q object or appears to get all dunder related's default columns and queryset raises ProgrammingError. Description There appears to be a difference in how Q object aliases are setup, when OR'd. The getdefaultcolumns for this agentpropertygroupsidin only uses 1, where as getdefaultcolumns for this agentpropertygroupsin gets all fields, which later results in a " subquery must return only one column" error. ( Q(agentpropertygroupsidin=propertygroups.valueslist("id", flat=True)) | Q(agentpropertygroupscount=0) ).distinct() ( Q(agentpropertygroupsin=propertygroups) | Q(agentpropertygroupscount=0) ).distinct() Here is the error: return (sql, params) File "/venv/lib/python3.6/site-", line 90, in exit raise djexcvalue.withtraceback(traceback) from excvalue File "/venv/lib/python3.6/site-", line 84, in execute return (sql, params) subquery must return only one column LINE 1: ...ativemovingaverage"."id", T5."propertygroupid", (SELECT U0... ^ For example, I was able to force it to work by checking the cols[0].alias checking that it was 'U0' so that the cols, selectfields, and klassinfo would only return the field needed within the Q object or # if cols: selectlist = [] # these two lines, just to hack a debug fix if cols[0].alias == 'U0': cols = [cols[0]] Was working ( 2.2.5 ), now not working ( 3.2 ): propertygroups = PropertyGroup.objects.agentgroups(managementagent) queryset = self.annotate(Count("agentpropertygroups")) return ( Q(agentpropertygroupsin=propertygroups) | Q(agentpropertygroupscount=0) ).distinct() now working: qs = blah propertygroups = PropertyGroup.objects.agentgroups(managementagent) queryset = qs.annotate(Count("agentpropertygroups")) ( Q(agentpropertygroupsidin=propertygroups.valueslist("id", flat=True)) | Q(agentpropertygroupscount=0) ).distinct() the generated sql SELECT COUNT(*) FROM ( SELECT DISTINCT "thingmanagerticketratingcumulativemovingaverage"."id" AS Col1, "thingmanagerticketratingcumulativemovingaverage"."created" AS Col2, "thingmanagerticketratingcumulativemovingaverage"."updated" AS Col3, "thingmanagerticketratingcumulativemovingaverage"."createby" AS Col4, "thingmanagerticketratingcumulativemovingaverage"."updateby" AS Col5, "thingmanagerticketratingcumulativemovingaverage"."tenantobjs" AS Col6, "thingmanagerticketratingcumulativemovingaverage"."date" AS Col7, "thingmanagerticketratingcumulativemovingaverage"."average" AS Col8, "thingmanagerticketratingcumulativemovingaverage"."datapoints" AS Col9, "thingmanagerticketratingcumulativemovingaverage"."agentid" AS Col10, COUNT("managermanagementagentpropertygroup"."propertygroupid") AS "agentpropertygroupscount" FROM "thingmanagerticketratingcumulativemovingaverage" INNER JOIN "managermanagementagent" ON ("thingmanagerticketratingcumulativemovingaverage"."agentid" = "managermanagementagent"."id") LEFT OUTER JOIN "managermanagementagentpropertygroup" ON ("managermanagementagent"."id" = "managermanagementagentpropertygroup"."managementagentid") LEFT OUTER JOIN "managermanagementagentpropertygroup" T5 ON ("managermanagementagent"."id" = T5."managementagentid") GROUP BY "thingmanagerticketratingcumulativemovingaverage"."id", T5."propertygroupid", ( -- the issue is right here SELECT U0."id", U0."created", U0."updated", U0."createby", U0."updateby", U0."tenantobjs", U0."name" -- the issue is the line above FROM "propertypropertygroup" U0 INNER JOIN "managermanagementagentpropertygroup" U1 ON (U0."id" = U1."propertygroupid") WHERE U1."managementagentid" = %s) HAVING ( T5."propertygroupid" IN ( SELECT U0."id" FROM "propertypropertygroup" U0 INNER JOIN "managermanagementagentpropertygroup" U1 ON (U0."id" = U1."propertygroupid") WHERE U1."managementagentid" = %s) OR COUNT("managermanagementagentpropertygroup"."propertygroupid") = %s) ); The sub select which causes the error: SELECT U0."id", U0."created", U0."updated", U0."createby", U0."updateby", U0."tenantobjs", U0."name" Looking into how th Q object looks and how the generated columns look: <Q: (OR: ('agentpropertygroupsin', <PropertyGroupQuerySet []), ('agentpropertygroupscount', 0)),) {} -try: (Pdb) c uhoh {'model': <class '', 'selectfields': [0, 1, 2, 3, 4, 5, 6]} [(Col(U0, ), ('U0."id"', []), None), (Col(U0, property.PropertyGroup.created), ('U0."created"', []), None), (Col(U0, property.PropertyGroup.updated), ('U0."updated"', []), None), (Col(U0, property.PropertyGroup.createby), ('U0."createby"', []), None), (Col(U0, property.PropertyGroup.updateby), ('U0."updateby"', []), None), (Col(U0, property.PropertyGroup.tenantobjs), ('U0."tenantobjs"', []), None), (Col(U0, ), ('U0."name"', []), None)] {'model': <class '', 'selectfields': [0, 1, 2, 3, 4, 5, 6]} {} <Q: (OR: ('agentpropertygroupsidin', <PropertyGroupQuerySet []), ('agentpropertygroupscount', 0)),) {} -try: (Pdb) c uhoh {'model': <class '', 'selectfields': [0]} [(Col(U0, ), ('U0."id"', []), None)] {'model': <class '', 'selectfields': [0]} {} extraselect [] The sub select query: (Pdb) print(self) SELECT U0."id", U0."created", U0."updated", U0."createby", U0."updateby", U0."tenantobjs", U0."name" FROM "propertypropertygroup" U0 INNER JOIN "managermanagementagentpropertygroup" U1 ON (U0."id" = U1."propertygroupid") WHERE U1."managementagentid" = 342 (Pdb) pprint() {'annotationselectcache': None, 'constructorargs': ((<class '',), {}), 'db': None, 'extraselectcache': None, 'filteredrelations': {}, 'lookupjoins': ['propertypropertygroup', 'managermanagementagentpropertygroup', 'managermanagementagent'], 'aliascols': True, 'aliasmap': {'U0': object at 0x7fc1efd77208, 'U1': object at 0x7fc1efd77828, 'U2': object at 0x7fc1efd777f0}, 'aliasprefix': 'U', 'aliasrefcount': {'U0': 1, 'U1': 1, 'U2': 0}, 'annotationselectmask': None, 'annotations': {}, 'basetable': 'U0', 'combinator': None, 'combinatorall': False, 'combinedqueries': (), 'containsaggregate': False, 'defaultcols': True, 'defaultordering': False, 'deferredloading': (frozenset(), True), 'distinct': False, 'distinctfields': (), 'explainformat': None, 'explainoptions': {}, 'explainquery': False, 'externalaliases': {'managermanagementagent': False, 'managermanagementagentpropertygroup': False, 'thingmanagerticketratingcumulativemovingaverage': False, 'propertypropertygroup': False}, 'extra': {}, 'extraorderby': (), 'extraselectmask': None, 'extratables': (), 'filterissticky': False, 'groupby': None, 'highmark': None, 'lowmark': 0, 'maxdepth': 5, 'model': <class '', 'orderby': (), 'select': (), 'selectfornokeyupdate': False, 'selectforupdate': False, 'selectforupdatenowait': False, 'selectforupdateof': (), 'selectforupdateskiplocked': False, 'selectrelated': False, 'standardordering': True, 'subqaliases': frozenset({'T', 'U'}), 'subquery': True, 'tablemap': {'managermanagementagent': ['U2'], 'managermanagementagentpropertygroup': ['U1'], 'propertypropertygroup': ['U0']}, 'usedaliases': {'managermanagementagentpropertygroup', 'propertypropertygroup'}, 'valuesselect': (), 'where': <WhereNode: (AND: object at 0x7fc1efd77860), 'whereclass': <class ''}


Support spaces separaters in ISO-8601 datetimes between times and timezone strings Description (last modified by Ben Wilber) GitHub Pull Request:  According to ISO-8601, there can be any number of whitespace characters between the time strings and timezone strings. Unfortunately the spec isn't public, but here's the link anyway  Examples: This is a valid ISO-8601 datetime string: 2012-04-23T10:20:30.400-02 parses this correctly. This is also a valid ISO-8601 datetime string: 2012-04-23T10:20:30.400 -02 does not parse this correctly and returns None, However, python-dateutil parses it correctly. The difference is that Django uses a (brittle) regex to parse ISO-8601 datetime strings, and python-dateutil does not.   I recommend that Django: 1) Depend on python-dateutil for datetime string parsing OR 2) Inline python-dateutils' parsing functions As far as I know there is no regex that can parse the full spec of ISO-8601 datetime strings. In the meantime, this is a patch to support (valid) whitespace characters between the seconds/millseconds part and the timezone string.