Pass the HttpRequest explicitly through the two webhooks that log to
the webhook loggers.
get_current_request is now unused, so remove it (in the same commit
for test coverage reasons).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Combine nginx and Django middlware to stop putting misleading warnings
about `CSRF_TRUSTED_ORIGINS` when the issue is untrusted proxies.
This attempts to, in the error logs, diagnose and suggest next steps
to fix common proxy misconfigurations.
See also #24599 and zulip/docker-zulip#403.
Having exactly 17 or 18 middlewares, on Python 3.11.0 and above,
causes python to segfault when running tests with coverage; see
https://github.com/python/cpython/issues/106092
Work around this by adding one or two no-op middlewares if we would
hit those unlucky numbers. We only add them in testing, since
coverage is a requirement to trigger it, and there is no reason to
burden production with additional wrapping.
The `django-sendfile2` module unfortunately only supports a single
`SENDFILE` root path -- an invariant which subsequent commits need to
break. Especially as Zulip only runs with a single webserver, and
thus sendfile backend, the functionality is simple to inline.
It is worth noting that the following headers from the initial Django
response are _preserved_, if present, and sent unmodified to the
client; all other headers are overridden by those supplied by the
internal redirect[^1]:
- Content-Type
- Content-Disposition
- Accept-Ranges
- Set-Cookie
- Cache-Control
- Expires
As such, we explicitly unset the Content-type header to allow nginx to
set it from the static file, but set Content-Disposition and
Cache-Control as we want them to be.
[^1]: https://www.nginx.com/resources/wiki/start/topics/examples/xsendfile/
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit brings AzureAD config in line with other backends:
- SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET gets fetched in computed_settings.py
instead of default_settings, consistent with github/gitlab/etc.
- SOCIAL_AUTH_AZUREAD_OAUTH2_KEY gets fetched in default_settings via
get_secret(..., development_only=True) like other social backends, to
allow easier set up in dev environment, in the dev-secrets.conf file.
- The secret gets renamed from azure_oauth2_secret to
social_auth_azuread_oauth2_secret to have a consistent naming scheme with
other social backends and with the SOCIAL_AUTH_AZUREAD_OAUTH2_KEY
name. This is backwards-incompatible.
The instructions for setting it up are updated to fit how this is
currently done in AzureAD.
Django has always expected this, but Django 4.0 added a system check
that spews warnings in production.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Technically Django already makes SECRET_KEY mandatory by raising an
ImproperlyConfigured exception when it is not set. We use the
get_mandatory_secret helper here so that we have a narrower type.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This also allows us to remove some assertions as we now know that
AVATAR_SALT will never be None.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This implements get_mandatory_secret that ensures SHARED_SECRET is
set when we hit zerver.decorator.authenticate_notify. To avoid getting
ZulipSettingsError when setting up the secrets, we set an environment
variable DISABLE_MANDATORY_SECRET_CHECK to skip the check and default
its value to an empty string.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This ensures that CAMO_KEY is always defined, so that mypy_django_plugin
will be able to identify its type.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.
This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.
There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.
We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.
TODO:
This is a temporary fix that should be refactored when the bug is fixed.
The encountered mypy issue: https://github.com/python/mypy/issues/13234
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
django-stubs dynamically collects the type annotation for us from the
settings, acknowledging mypy that `HOME_NOT_LOGGED_IN` is an
`Optional[str]`. Type narrowing with assertions does not play well with
the default value of the decorator, so we define the same setting
variable with a different name as `CUSTOM_HOME_NOT_LOGGED_IN` to bypass
this restriction.
Filed python/mypy#13087 to track this issue.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
Similar to the previous commit, we should access request.user only
after it has been initialized, rather than having awkward hasattr
checks.
With updates to the settings comments about LogRequests by tabbott.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
`request.user` gets set in Django's `AuthenticationMiddleware`, which
runs after our `HostDomainMiddleware`.
This makes `hasattr` checks necessary in any code path that uses the
`request.user` attribute. In this case, there are functions in
`context_processors` that get called in the middleware.
Since neither `CsrfMiddleware` nor `HostDomainMiddleware` are required
to run before `AuthenticationMiddleware`, moving it two slots up in
`computed_settings` is sufficient to avoid the `hasattr` checks.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
In zliencer.management.commands.populate_db, we assign the value of
settings.CACHES["default"] to `default_cache`.
django-stubs infers `settings.CACHES` to be `Dict[str, object]`. We make
the type specific enough so that we can access `default_cache` as a
dict.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
This was added in 1fded25025, and is not
necessary for standard Zulip installs. While both Host: and
X-Forwarded-Host: are nominally untrusted, there is no reason to
complicate the deployment by defaulting it on.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).
A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
This cache was added in da33b72848 to serve as a replacement for the
durable database cache, in development; the previous commit has
switched that to be the non-durable memcached backend.
The special-case for "in-memory" in development is mostly-unnecessary
in contrast to memcached -- `./tools/run-dev.py` flushes memcached on
every startup. This differs in behaviour slightly, in that if the
codepath is changed and `run-dev` restarts Django, the cache is not
cleared. This seems an unlikely occurrence, however, and the code
cleanup from its removal is worth it.
Failure to pull the default "zulip" value here can lead to
accidentally applying a `postgres_password` value which is unnecessary
and may never work.
For consistency, always skip password auth attempts for the "zulip"
user on localhost, even if the password is set. This mirrors the
behavior of `process_fts_updates`.
It’s built in to Jinja2 as of 2.9. Fixes “DeprecationWarning: The
'autoescape' extension is deprecated and will be removed in Jinja
3.1. This is built in now.”
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We recently ran into a payload in production that didn't contain
an event type at all. A payload where we can't figure out the event
type is quite rare. Instead of letting these payloads run amok, we
should raise a more informative exception for such unusual payloads.
If we encounter too many of these, then we can choose to conduct a
deeper investigation on a case-by-case basis.
With some changes by Tim Abbott.
Having wantMessagesSigned=True globally means that it's also applied by
python3-saml to regular authentication SAMLResponses - making it require
the response to be signed, which is an issue because a feasible
alternative way that some IdPs (e.g. AzureAD) take by default is to sign
specifically the assertions in the SAMLResponse. This is also secure,
and thus we generally want to accept it.
Without this, the setting of wantMessagesSigned=True globally
in 4105ccdb17 causes a
regression for deployments that have already set up SAML with providers
such as AzureAD, making Zulip stop accepting the SAMLResponses.
Testing that this new logic works is handled by
test_saml_idp_initiated_logout_invalid_signature, which verifies that a
LogoutRequest without signature will be rejected.
Anders Kaseorg
Alex Vandiver
Mateusz Mandera
Zixuan James Li
Tim Abbott
Sahil Batra
Aman Agrawal
Eeshan Garg