We should not proceed and send client reload events until we know that
all of the server processes have updated to the latest version, or
they may reload into the old server version if they hit a Django
worker which has not yet restarted.
Because the logic controlling the number of workers is mildly complex,
and lives in Puppet, use the `uwsgi` Python bindings to know when the
process being reloaded is the last one, and use that to write out a
file signifying the success of the chain reload. `restart-server`
awaits the creation of this file before proceeding.
Django lazy-loads much of its modules, including the application's.
This defers the time to during the first request it serves. When
doing rolling-restarts, this means that the worker is marked "ready"
despite having multiple seconds more work to do. With small numbers
of workers, this causes a significant capacity drop, since effectively
more than one worker can be still reloading at a time. It also
results in poor user experience for the requests which are unlucky
enough to be served first.
Use the technique detailed in the uwsgi documentation[^1] to fake a
request during initial application load, which forces the application
to be loaded.
[^1]: https://uwsgi-docs.readthedocs.io/en/latest/articles/TheArtOfGracefulReloading.html#dealing-with-ultra-lazy-apps-like-django
The zerver.models hack does not appear to be necessary now.
Meanwhile, get_wsgi_application has its own django.setup call, which
would overwrite the parts of our logging configuration pulled in by
zerver.models.
This fixes part of #15391; specifically, fixes it in production, but
not in development, where ‘manage.py runserver’ calls its own
django.setup and then imports various bits of our code before finding
zproject.wsgi.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Fixes#2665.
Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.
Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start. I expect this change will increase pressure for us to split
those files, which isn't a bad thing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
isort 5 knows not to reorder imports across function calls, so this
will stop isort from breaking our code.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
It's common for a broken settings.py file to result in Django not
starting, thus never writing to `/var/log/zulip/errors.log`. Such
behavior can be discouraging when the server 500s without a traceback
to accompany it. To fix this, we simply catch any exceptions in
django.setup(), if raised, and log the exception appropriately.
Comments tweaked by tabbott.
Closes: #7032.
This reverts commit ec9f6702d8.
Now that pika 0.13.0 has merged our PR to not import twisted unless it
is needed, we don't need to use this performance hack in order to
avoid wasting time importing twisted and all its dependencies.
This is a performance optimization; see the comment. This fixes part
of #9953.
Eventually, we should do the same thing for importing Tornado as well,
but it's less important because Tornado is a much smaller library.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.
Commit message and error text tweaked by tabbott.
Fixes#6543.
The manage.py change effectively switches the Zulip production server
to use the virtualenv, since all of our supervisord commands for the
various Python services go through manage.py.
Additionally, this migrates the production scripts and Nagios plugins
to use the virtualenv as well.
Because import_module does not correctly handle safe circular imports we
need to import zerver.models first before the middleware tries to import it.
(imported from commit 1afebd8c950c44c8d136b0b63a09319ccef02555)
This includes a hack to preserve humbug/backends.py as a symlink, so
that we don't need to regenerate all our old sessions.
(imported from commit b7918988b31c71ec01bbdc270db7017d4069221d)
Alex Vandiver
Anders Kaseorg
Anders Kaseorg
Wyatt Hoodes
Anders Kaseorg
Tim Abbott
julianasobreira
Aditya Bansal
Umair Khan
Eklavya Sharma
Luke Faraone
Tim Abbott
Tim Abbott