Commit Graph

2288 Commits

Author SHA1 Message Date
Zixuan James Li 929311642b legacy_urls: Add type annotation to legacy_urls.
Mypy previously infers this to be `List[URLPattern]` which is
incompatible with other urls lists that we concatenate this with.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li c9f54766c3 rate_limiter: Extract rate limit related functions.
This refactors rate limit related functions from `zerver.decorator` to
zerver.lib.rate_limiter.

We conditionally import `RemoteZulipServer`, `RequestNotes`, and
`RateLimitedRemoteZulipServer` to avoid circular dependency.

Most instances of importing these functions from `zerver.decorator` got
updated, with a few exceptions in `zerver.tests.test_decorators`, where
we do want to mock the rate limiting functions imported in
`zerver.decorator`. The same goes with the mocking example in the
"testing-with-django" documentation.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 16:51:55 -07:00
Dinesh 48d2783559 read_receipts: Add support for displaying read receipts.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.

Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.

Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.

Fixes #3618.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-08-12 13:16:35 -07:00
Zixuan James Li 1d7abb3f5d settings: Extract unnecessary test specific settings.
APPLE_ID_TOKEN_GENERATOR_KEY and EXAMPLE_JWK are exclusively used in
test_auth_backends.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li b0ffe45835 settings: Move TEST_WORKER_DIR to computed_settings.
mypy_django_plugin can only access settings from zproject.settings.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li f994ba30b8 settings: Remove settings.BACKEND_DATABASE_TEMPLATE.
BACKEND_DATABASE_TEMPLATE was introduced in a507a47778.
This setting is only available for the test cases and it is not that
necessary to have it configurable.

We define it as a global variable in zerver.lib.test_fixtures.

This avoids requiring mypy_django_plugin to know the type of
settings.BACKEND_DATABASE_TEMPLATE for type checking purposes, given the fact
that settings.test_extra_settings is not available in production/development
setup.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li 448253c009 settings: Make CAMO_KEY Optional.
This ensures that CAMO_KEY is always defined, so that mypy_django_plugin
will be able to identify its type.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-11 13:45:28 -07:00
Zixuan James Li 77eef5b1ad types: Add OIDCIdPConfigDict.
The presence of `auto_signup` in idp_settings_dict in the test case
test_social_auth_registration_auto_signup is incompatible with the
previous type annotation of SOCIAL_AUTH_OIDC_ENABLED_IDPS, where `bool`
is not allowed.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-03 15:03:15 -07:00
Anders Kaseorg b945aa3443 python: Use a real parser for email addresses.
Now that we can assume Python 3.6+, we can use the
email.headerregistry module to replace hacky manual email address
parsing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-29 15:47:33 -07:00
Zixuan James Li a2bf76c610 email_backends: Fix type annotation of EmailLogBackEnd.
The base class has the methods to accept `Sequence` of `EmailMessage`.
Because our implementation in fact only supports `EmailMultiAlternatives`,
isinstance checks with assertions need to be added along with the
signature change.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:58:11 -07:00
Zixuan James Li 678c7fdd5f email_backends: Inline the construction of html_alternative.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-29 14:58:11 -07:00
Zixuan James Li a3efc777fa urls: Remove a legacy assertion.
This check was added for a legacy implementation of the GitHub integration in
bb6d189fa8,
which later got removed in
a73e8109b7.

No other webhook integration can now have a Falsy `url_object` attribute.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li b5f03b9637 settings: Type JITSI_SERVER_URL as being Optional.
We fixed the case when handling `JITSI_SERVER_URL` being `None`, but the
type annotation didn't get updated along with the fix
2f9d4f5a96

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li 036a90f375 settings: Add isinstance check before filtering.
This is a follow-up to https://github.com/typeddjango/django-stubs/pull/1038.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 18:00:24 -07:00
Zixuan James Li fe9ed2e69d settings: Make INVITATION_LINK_VALIDITY_MINUTES optional.
Type inference does not work when the default value of `REQ` is
non-optional while `ResultT` is optional. Mypy tries to unify
`json_validator` with `Validator[int]` in `invite_users_backend` instead
of the desired `Validator[Optional[int]]` because of the presence of the
default value `settings.INVITATION_LINK_VALIDITY_MINUTES`, which is
inferred to be an `int`. Mypy does not resort to a less specific type but
instead gives up early.

This issue applies to invite_users_backend and generate_multiuse_invite_backend
in zerver.views.invite.

There might be a way that we can add an overload to get around this, but
it's probably not worth the complexity until it comes up again more frequently.

We do in fact allow `invite_expires_in_minutes` to be `None` in places
like `do_invite_users`, `invite_users_backend`, etc, and we have
`settings.INVITATION_LINK_VALIDITY_MINUTES` as the default for them. So
it makes sense to allow having an optional value for this setting. And
since there isn't a way to independently set the value of this constant,
we move it to a different place.

TODO:

This is a temporary fix that should be refactored when the bug is fixed.

The encountered mypy issue: https://github.com/python/mypy/issues/13234

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-26 17:07:48 -07:00
Zixuan James Li 2e248cdbec settings: Add CUSTOM_HOME_NOT_LOGGED_IN for type narrowing.
django-stubs dynamically collects the type annotation for us from the
settings, acknowledging mypy that `HOME_NOT_LOGGED_IN` is an
`Optional[str]`. Type narrowing with assertions does not play well with
the default value of the decorator, so we define the same setting
variable with a different name as `CUSTOM_HOME_NOT_LOGGED_IN` to bypass
this restriction.

Filed python/mypy#13087 to track this issue.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-25 14:11:32 -07:00
Zixuan James Li 52be020d0c custom_profile: Apply ProfileDataElementUpdateDict.
We explicitly annotate variables or parameters with
`ProfileDataElementUpdateDict` as necessary.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:55:03 -07:00
Zixuan James Li 0952b024c1 scim: Add SCIMConfigDict.
This adds a `TypedDict` to provide better type annotation for the
configuration object for SCIM.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-15 14:00:56 -07:00
Zixuan James Li 75925fe059 middleware: Reorder middleware to clean up LogRequests hasattr checks.
Similar to the previous commit, we should access request.user only
after it has been initialized, rather than having awkward hasattr
checks.

With updates to the settings comments about LogRequests by tabbott.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-14 17:28:50 -07:00
Zixuan James Li 37a7d1fe7b middleware: Reorder middleware to avoid hasattr checks.
`request.user` gets set in Django's `AuthenticationMiddleware`, which
runs after our `HostDomainMiddleware`.

This makes `hasattr` checks necessary in any code path that uses the
`request.user` attribute. In this case, there are functions in
`context_processors` that get called in the middleware.

Since neither `CsrfMiddleware` nor `HostDomainMiddleware` are required
to run before `AuthenticationMiddleware`, moving it two slots up in
`computed_settings` is sufficient to avoid the `hasattr` checks.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-14 17:24:24 -07:00
Ganesh Pawar 135854db97 emoji: Use path converter for emoji name in URL.
If the emoji name contains forward slashes, the `str` converter
would treat it as a URL delimiter. Instead use the path converter, so
that forward slashes are included in the emoji name variable.

Fixes #22377
2022-07-14 14:29:40 -07:00
Anders Kaseorg 6b4474d4b5 computed_settings: Remove deprecated USE_L10N setting.
This is deprecated and defaults to True in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-13 16:07:17 -07:00
Zixuan James Li 3a27a60b2c ldap: Remove unused isinstance check.
`realm_access_control` is always a `dict`.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-07-11 11:46:25 -07:00
Anders Kaseorg 2439914a50 settings: Add two_factor.plugins.phonenumber to INSTALLED_APPS.
I missed this in commit feff1d0411
(#22383) for upgrading to django-two-factor-auth 1.14.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-06 17:23:53 -07:00
Anders Kaseorg feff1d0411 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Anders Kaseorg b4cf9ad777 db: Use cursor_factory psycopg2 option.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-07-05 17:54:17 -07:00
Zixuan James Li c968b299db typing: Tighten type annotation for settings.CACHES.
In zliencer.management.commands.populate_db, we assign the value of
settings.CACHES["default"] to `default_cache`.

django-stubs infers `settings.CACHES` to be `Dict[str, object]`. We make
the type specific enough so that we can access `default_cache` as a
dict.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-06-23 22:09:05 -07:00
Anders Kaseorg 9ee636e920 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-06-01 16:01:54 -07:00
Tim Abbott 0255cbda03 settings: Allow customizing HOME_NOT_LOGGED_IN.
This can be useful if one wants a third-party authentication system to
apply first.
2022-06-01 15:37:31 -07:00
Zixuan James Li 4c6f2ae7be typing: Add assertions for authentication.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Zixuan James Li 4a5043dd6e typing: Add none-checks for miscellaneous cases.
Signed-off-by: Zixuan James Li <359101898@qq.com>
2022-05-31 09:43:55 -07:00
Sahil Batra ce34b585a5 streams: Add endpoint "GET /streams/{stream_id}" to get stream by id.
Fixes #22082.
2022-05-23 15:14:04 -07:00
Alex Vandiver 513fcb7bd5 Revert "settings: Add web-public streams beta subdomain list."
This reverts commit 20368a936c.  It is
no longer in beta, and this configuration is no longer needed.
2022-05-16 14:49:14 -07:00
Alya Abbott 86b61d46eb portico: Add Recurse Center case study. 2022-05-05 16:43:00 -07:00
Tim Abbott 2e86ea6540 events: Add support for spectator access to /register.
This is necessary for the mobile/terminal clients to build spectator
support down the line. We'll also be using it for the web application,
in an upcoming commit.
2022-05-05 15:20:44 -07:00
Alex Vandiver 68bc975066 settings: Stop enabling USE_X_FORWARDED_HOST by default.
This was added in 1fded25025, and is not
necessary for standard Zulip installs.  While both Host: and
X-Forwarded-Host: are nominally untrusted, there is no reason to
complicate the deployment by defaulting it on.
2022-05-04 14:41:18 -07:00
Mateusz Mandera 71f6f2310c auth: Don't log email when user tried authing to the wrong subdomain.
We want to avoid logging this kind of potentially sensitive information.
Instead, it's more useful to log ids of the matching accounts on
different subdomains.
2022-05-03 17:06:51 -07:00
Anders Kaseorg f29553d809 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-03 10:10:06 -07:00
Adam Sah dbd03b5054 email_backends: Fix concurrent backend testing for generate_emails.
Previously, this command would reliably fail:

```
tools/test-backend --skip-provision-check --parallel=3
  zerver.tests.test_email_log.EmailLogTest.test_forward_address_details
  zerver.tests.test_email_log.EmailLogTest.test_generate_and_clear_email_log
  zerver.tests.test_example.TestDevelopmentEmailsLog
```

and now it reliably succeeds. :-)

After hours of fiddling/googling/hair-tearing, I found that
mocking-away Django Connection.send_messages() was the best:

- We're testing Zulip and not Django.
- Mocking at this lower level exercises more of our code.
- EmailLogBackEnd._do_send_messages() helper method added to simplify mocking.

Fixes #21925.
2022-05-03 08:45:51 -07:00
Anders Kaseorg c9faefd50e cache: Instantiate only one BMemcached cache backend.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Anders Kaseorg bded7180f7 tornado: Unfork tornado.autoreload.
We previously forked tornado.autoreload to work around a problem where
it would crash if you introduce a syntax error and not recover if you
fix it (https://github.com/tornadoweb/tornado/issues/2398).

A much more maintainable workaround for that issue, at least in
current Tornado, is to use tornado.autoreload as the main module.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-05-02 17:41:49 -07:00
Alex Vandiver 280938691f prod_settings_template: Switch to double quotes in commented lines. 2022-04-28 12:40:28 -07:00
Alex Vandiver 540ce2c765 prod_settings_template: Add some missing quotes in commented lines. 2022-04-28 12:40:28 -07:00
Tim Abbott bd2dc7358b help: Add redirect to new public access page. 2022-04-28 12:27:13 -07:00
Anders Kaseorg a2825e5984 python: Use Python 3.8 typing.{Protocol,TypedDict}.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-04-27 12:57:49 -07:00
Sahil Batra 37793434f7 user_groups: Add API endpoint to get subgroups of a user group. 2022-04-25 10:24:03 -07:00
Sahil Batra 1b3c972d9b user_groups: Add API endpoint to get members of a user group.
This commit adds 'GET /user_groups/{user_group_id}/members'
endpoint to get members of a user group. "direct_member_only"
parameter can be passed as True to the endpoint to get only
direct members of the user group and not the members of
subgroup.
2022-04-25 10:24:03 -07:00
Sahil Batra 374d2a66df user_groups: Add endpoint to check whether a user is member of a group.
This commit adds 'GET /user_groups/{id}/members/{id}' endpoint to check
whether a user is member of a group.

This commit also adds for_read parameter to access_user_group_by_id,
which if passed as True will provide access to read user group even
if it a system group or if non-admin acting user is not part of the
group.
2022-04-25 10:24:03 -07:00
Sahil Batra 6f0a7656ac user_groups: Add API endpoint for updating subgroups of a user group. 2022-04-25 10:24:03 -07:00
Sahil Batra 61365fbe21 invites: Use expiration time in minutes instead of days.
This commit changes the invite API to accept invitation
expiration time in minutes since we are going to add a
custom option in further commits which would allow a user
to set expiration time in minutes, hours and weeks as well.
2022-04-20 13:31:37 -07:00