Commit Graph

2311 Commits

Author SHA1 Message Date
Mateusz Mandera 34a0139c2b rate_limit: Add interface to override rate limiting rules. 2022-11-08 08:56:42 -08:00
Anders Kaseorg 55342efd33 scim: Upgrade django-scim2; remove request.user monkey patching.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-05 15:13:50 -07:00
Anders Kaseorg 9d2d6c8eb7 ruff: Fix M001 Unused `noqa` directive.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Anders Kaseorg adffad384c lint: Replace pycodestyle and pyflakes with ruff.
https://github.com/charliermarsh/ruff

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-11-03 12:10:15 -07:00
Mateusz Mandera 05913f5e3a saml: Extract logic for enabling wantMessagesSigned locally.
As explained in 158287f998,
wantMessagesSigned can't be enabled globally (as it'll break setups with
IdPs that sign SAMLResponse assertions) - but is needed for
LogoutRequests, and will be for LogoutResponses in the SP-initiated SLO
flow in future commits.

We extract a function with the necessary hacky logic for re-use in the
SP-initiated SLO implementation.
2022-11-02 21:58:46 -07:00
Lauryn Menard 1eda42f509 images: Create `/authentication_backends` directory for icons.
Creates `static/images/authentication_backends` directory for icons
of backend authentication methods, which are used on the log-in page.

And updates the example documentation in the API `/server_settings`
endpoint.
2022-10-31 14:36:54 -07:00
Anders Kaseorg b45484573e python: Use format string for logging str(obj).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
Anders Kaseorg fcd81a8473 python: Replace avoidable uses of __special__ attributes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-10 08:32:29 -07:00
PIG208 df18bbbd48 requirements: Add django-stubs and configure plugin.
Note that django_stubs_ext is required to be placed within common.in
because we need the monkeypatched types in runtime; django-stubs
itself is for type checking only.

In the future, we would like to pin to a release instead of a git
revision, but several patches we've contributed upstream have not
appeared in a release yet.

We also remove the type annotation for RealmAuditLog.event_last_message_id
here instead of earlier because type checking fails otherwise.

Fixes #11560.
2022-10-05 16:15:56 -07:00
Anders Kaseorg 676d40d66b sentry: Initialize sentry in AppConfig ready hook.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg a5cf3b3975 config: Remove Django base class from ZulipSettingsError.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 7908e2c3a5 computed_settings: Move logging callbacks from zerver.lib.logging_util.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 9eefc290a9 template_loaders: Extract TwoFactorLoader to new module.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 02be415122 settings_type: Extract new module for types used in settings.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Mateusz Mandera 16829bd84b auth: Fix up AzureAD Oauth2 docs and config.
This commit brings AzureAD config in line with other backends:
- SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET gets fetched in computed_settings.py
  instead of default_settings, consistent with github/gitlab/etc.
- SOCIAL_AUTH_AZUREAD_OAUTH2_KEY gets fetched in default_settings via
  get_secret(..., development_only=True) like other social backends, to
  allow easier set up in dev environment, in the dev-secrets.conf file.
- The secret gets renamed from azure_oauth2_secret to
  social_auth_azuread_oauth2_secret to have a consistent naming scheme with
  other social backends and with the SOCIAL_AUTH_AZUREAD_OAUTH2_KEY
  name. This is backwards-incompatible.

The instructions for setting it up are updated to fit how this is
currently done in AzureAD.
2022-09-06 11:58:08 -07:00
Lauryn Menard 878b46e758 urls: Simplify process to add URL redirects for documentation pages.
Creates `zerver.lib.url_redirects.py` to record old and new URLs
for documentation pages that have been renamed/moved and need URL
redirects.

This file is then used by `zproject.urls.py` to redirect links and
by `zerver.test.test_urls.py` to test that all of the old URLs
return a success response with a common page header/text depending
on the type of redirect (help center, policy, or API).

Adds a section to contributor docs on writing documentation for
how to use this redirect system when renaming a help center or api
documentation page.

Fixes #21946. Fixes #17897.
2022-08-31 15:01:29 -07:00
Anders Kaseorg 81d0f5bdd9 computed_settings: Set STATICFILES_DIRS to an existing absolute path.
Django has always expected this, but Django 4.0 added a system check
that spews warnings in production.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-08-26 15:55:59 -07:00
Zixuan James Li c21ae3646a settings: Make SECRET_KEY mandatory.
Technically Django already makes SECRET_KEY mandatory by raising an
ImproperlyConfigured exception when it is not set. We use the
get_mandatory_secret helper here so that we have a narrower type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Zixuan James Li c8d5959689 settings: Make AVATAR_SALT mandatory.
This also allows us to remove some assertions as we now know that
AVATAR_SALT will never be None.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Zixuan James Li 059d0e7be8 settings: Make SHARED_SECRET mandatory.
This implements get_mandatory_secret that ensures SHARED_SECRET is
set when we hit zerver.decorator.authenticate_notify. To avoid getting
ZulipSettingsError when setting up the secrets, we set an environment
variable DISABLE_MANDATORY_SECRET_CHECK to skip the check and default
its value to an empty string.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-25 12:13:03 -07:00
Aman Agrawal e54ded49c4 urls: Move URLs that don't belong on self-hosted servers to corporate URLs.
Use absolute URLs for these links in files which will be served
to self-hosted servers.
2022-08-22 15:53:43 -07:00
Aman Agrawal f1a1868de8 urls: Remove duplicate developer-community redirect.
Exactly same call exists below this with a comment.
2022-08-22 15:53:43 -07:00
Zixuan James Li eb88fee745 rest: Remove kwargs from rest_path.
The only caller that passes the kwargs argument is the avatar rest_path.
The application of kwargs can be rewritten with a wrapper.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-15 19:51:08 -07:00
Zixuan James Li 929311642b legacy_urls: Add type annotation to legacy_urls.
Mypy previously infers this to be `List[URLPattern]` which is
incompatible with other urls lists that we concatenate this with.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li c9f54766c3 rate_limiter: Extract rate limit related functions.
This refactors rate limit related functions from `zerver.decorator` to
zerver.lib.rate_limiter.

We conditionally import `RemoteZulipServer`, `RequestNotes`, and
`RateLimitedRemoteZulipServer` to avoid circular dependency.

Most instances of importing these functions from `zerver.decorator` got
updated, with a few exceptions in `zerver.tests.test_decorators`, where
we do want to mock the rate limiting functions imported in
`zerver.decorator`. The same goes with the mocking example in the
"testing-with-django" documentation.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Dinesh 48d2783559 read_receipts: Add support for displaying read receipts.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.

Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.

Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.

Fixes #3618.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-08-12 13:16:35 -07:00
Zixuan James Li 1d7abb3f5d settings: Extract unnecessary test specific settings.
APPLE_ID_TOKEN_GENERATOR_KEY and EXAMPLE_JWK are exclusively used in
test_auth_backends.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li b0ffe45835 settings: Move TEST_WORKER_DIR to computed_settings.
mypy_django_plugin can only access settings from zproject.settings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li f994ba30b8 settings: Remove settings.BACKEND_DATABASE_TEMPLATE.
BACKEND_DATABASE_TEMPLATE was introduced in a507a47778.
This setting is only available for the test cases and it is not that
necessary to have it configurable.

We define it as a global variable in zerver.lib.test_fixtures.

This avoids requiring mypy_django_plugin to know the type of
settings.BACKEND_DATABASE_TEMPLATE for type checking purposes, given the fact
that settings.test_extra_settings is not available in production/development
setup.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li 448253c009 settings: Make CAMO_KEY Optional.
This ensures that CAMO_KEY is always defined, so that mypy_django_plugin
will be able to identify its type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li 77eef5b1ad types: Add OIDCIdPConfigDict.
The presence of `auto_signup` in idp_settings_dict in the test case
test_social_auth_registration_auto_signup is incompatible with the
previous type annotation of SOCIAL_AUTH_OIDC_ENABLED_IDPS, where `bool`
is not allowed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Anders Kaseorg b945aa3443 python: Use a real parser for email addresses.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:47:33 -07:00
Zixuan James Li a2bf76c610 email_backends: Fix type annotation of EmailLogBackEnd.
The base class has the methods to accept `Sequence` of `EmailMessage`.
Because our implementation in fact only supports `EmailMultiAlternatives`,
isinstance checks with assertions need to be added along with the
signature change.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:58:11 -07:00
Zixuan James Li 678c7fdd5f email_backends: Inline the construction of html_alternative.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:58:11 -07:00
Zixuan James Li a3efc777fa urls: Remove a legacy assertion.
This check was added for a legacy implementation of the GitHub integration in
bb6d189fa8,
which later got removed in
a73e8109b7.

No other webhook integration can now have a Falsy `url_object` attribute.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li b5f03b9637 settings: Type JITSI_SERVER_URL as being Optional.
We fixed the case when handling `JITSI_SERVER_URL` being `None`, but the
type annotation didn't get updated along with the fix
2f9d4f5a96

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 036a90f375 settings: Add isinstance check before filtering.
This is a follow-up to https://github.com/typeddjango/django-stubs/pull/1038.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li fe9ed2e69d settings: Make INVITATION_LINK_VALIDITY_MINUTES optional.
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.

This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.

There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.

We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.

TODO:

This is a temporary fix that should be refactored when the bug is fixed.

The encountered mypy issue: https://github.com/python/mypy/issues/13234

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 17:07:48 -07:00
Zixuan James Li 2e248cdbec settings: Add CUSTOM_HOME_NOT_LOGGED_IN for type narrowing.
django-stubs dynamically collects the type annotation for us from the
settings, acknowledging mypy that `HOME_NOT_LOGGED_IN` is an
`Optional[str]`. Type narrowing with assertions does not play well with
the default value of the decorator, so we define the same setting
variable with a different name as `CUSTOM_HOME_NOT_LOGGED_IN` to bypass
this restriction.

Filed python/mypy#13087 to track this issue.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-25 14:11:32 -07:00
Zixuan James Li 52be020d0c custom_profile: Apply ProfileDataElementUpdateDict.
We explicitly annotate variables or parameters with
`ProfileDataElementUpdateDict` as necessary.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 0952b024c1 scim: Add SCIMConfigDict.
This adds a `TypedDict` to provide better type annotation for the
configuration object for SCIM.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 75925fe059 middleware: Reorder middleware to clean up LogRequests hasattr checks.
Similar to the previous commit, we should access request.user only
after it has been initialized, rather than having awkward hasattr
checks.

With updates to the settings comments about LogRequests by tabbott.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-14 17:28:50 -07:00
Zixuan James Li 37a7d1fe7b middleware: Reorder middleware to avoid hasattr checks.
`request.user` gets set in Django's `AuthenticationMiddleware`, which
runs after our `HostDomainMiddleware`.

This makes `hasattr` checks necessary in any code path that uses the
`request.user` attribute. In this case, there are functions in
`context_processors` that get called in the middleware.

Since neither `CsrfMiddleware` nor `HostDomainMiddleware` are required
to run before `AuthenticationMiddleware`, moving it two slots up in
`computed_settings` is sufficient to avoid the `hasattr` checks.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-14 17:24:24 -07:00
Ganesh Pawar 135854db97 emoji: Use path converter for emoji name in URL.
If the emoji name contains forward slashes, the `str` converter
would treat it as a URL delimiter. Instead use the path converter, so
that forward slashes are included in the emoji name variable.

Fixes #22377
2022-07-14 14:29:40 -07:00
Anders Kaseorg 6b4474d4b5 computed_settings: Remove deprecated USE_L10N setting.
This is deprecated and defaults to True in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:07:17 -07:00
Zixuan James Li 3a27a60b2c ldap: Remove unused isinstance check.
`realm_access_control` is always a `dict`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-11 11:46:25 -07:00
Anders Kaseorg 2439914a50 settings: Add two_factor.plugins.phonenumber to INSTALLED_APPS.
I missed this in commit feff1d0411
(#22383) for upgrading to django-two-factor-auth 1.14.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-06 17:23:53 -07:00
Anders Kaseorg feff1d0411 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Anders Kaseorg b4cf9ad777 db: Use cursor_factory psycopg2 option.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Zixuan James Li c968b299db typing: Tighten type annotation for settings.CACHES.
In zliencer.management.commands.populate_db, we assign the value of
settings.CACHES["default"] to `default_cache`.

django-stubs infers `settings.CACHES` to be `Dict[str, object]`. We make
the type specific enough so that we can access `default_cache` as a
dict.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00