Commit Graph

15880 Commits

Author SHA1 Message Date
Anders Kaseorg 7dcffca50e tornado: Construct Django BaseHandler once, not per-request.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:20:45 -07:00
Anders Kaseorg 4c808026b2 tests: Disable lazy loading of moto.
This works around some regression in moto 1.3.15 that I bisected to
b8820009e8
where ‘tools/test-backend test_transfer’ fails when run by itself.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-30 11:19:39 -07:00
Zixuan James Li e16de8d9e7 scim: Further slim down SCIMClient removing unused attributes.
This removes everything from SCIMClient except the "is_authenticated`
method. Previously, "realm" and "name" were only needed for logging
purposes. It is the best to keep SCIMClient as minimal as possible, as
it is only intended to be used for authenticating requests to SCIM
views.

This change also gurantees that the "LogRequests" middleware will not
rely on the type unsafe access of the format_requestor_for_logs method
on SCIMClient.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-30 15:34:50 +02:00
Zixuan James Li a9273c9aac scim: Remove unused test case attribute.
The scim_client attribute on SCIMTestCase is currently unused since
9198fe4fac.

The creation of the SCIMClient instance was previously needed because
zerver.middleware.validate_scim_bearer_token returns SCIMClient from the db.
The attribute itself on the was never really used in the test case.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-30 15:34:50 +02:00
Anders Kaseorg e00f883263 test_runner: Remove empty ParallelTestSuite.__init__ override.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-29 18:43:24 -07:00
khanhdq ee2141b0f1 email_notifications: Change missed message for wildcard mention.
Add #stream_name to wildcard mention because it is important
information for interpreting the wildcard mention (larger streams may
mean something very different to you than small ones).

Fixes #22885.
2022-09-29 15:55:14 -07:00
khanhdq bb8365ab36 email_notifications: Add ``realm_name`` to email notifications.
Add {{ realm_name }} to the "Reply to this email directly ..." line.

This ensures the realm name is always present in the email
notification footer area, in a consistent location.
2022-09-29 15:54:35 -07:00
khanhdq 59d0f9b301 email_notifications: Highlight personal mentions in explanations.
Change the explanations for why notifications were received, with
different text for personal and wildcard mentions.
2022-09-29 15:54:21 -07:00
Sahil Batra 04693b6ac1 message_edit: Send only changed settings in event data and api response.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.

Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
2022-09-28 11:47:40 -07:00
Mateusz Mandera 522c159441 test_push_notifs: Change type_id arg of get_message to no default.
This isn't used anywhere, nor would type_id 100 make much sense.
2022-09-28 16:45:34 +02:00
Mateusz Mandera b35ad99035 test_push_notifications: Use proper user.id for Recipient type_id.
Recipient with type PERSONAL type_id 1 is a Recipient for a system bot,
since those get created first. Even if it doesn't break tests, it's
still bad, because it is not the intention of those tests to simulate a
cross-realm private message to a system bot.
2022-09-28 16:45:34 +02:00
Mateusz Mandera 5850c38f4e test_digest: Use proper stream.id in test_get_hot_topics.
Just using values 1 and 2 as stream ids is not good, because there's no
idea in which realm these streams are (or hypothetically if they exist).
This can create weird Messages with sender being a user of "zulip" realm
and the stream being in another realm - which would be a corrupted
state.
2022-09-28 16:45:25 +02:00
Kartik Srivastava 409ef040bb stream_topic: Refactor user_ids_muting_topic.
This refactors and renames user_ids_muting_topic to accept a parameter
'visibility_policy' and fetch user IDs that have a specific
visibility_policy(provided as the parameter) set for a topic.
2022-09-27 17:18:48 -07:00
Anders Kaseorg 8eec4bf171 message_fetch: Move narrowing query builder to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-27 17:02:10 -07:00
Aman Agrawal b84722de01 events: Clean up logic for spectator events_register parameters.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.

Follow-up on #21995.
2022-09-27 16:12:59 -07:00
Mateusz Mandera 2811a1228f import_util: Make build_message only take kwargs.
build_message has a lot of arguments, so it's hard to verify correctness
of callers that just try to get the order right. It's much clearer to be
explicit via kwargs. mattermost.py and rocketchat.py already do this, so
let's bring slack.py and gitter.py up to par.
2022-09-27 15:04:48 -07:00
Mateusz Mandera e360b5e29b export: Remove unnecessary if in export with consent code.
This might be a bit cleaner.
2022-09-27 11:56:27 -07:00
Mateusz Mandera 318d7fd4cd export: Only export messages that a consenting user can access.
As mentioned in the TODO this commit deletes, the export with member
consent system was failing to account for the fact that if consenting
users only have access to a subset of messages of a stream with
protected history, only that subset should be exported - rather than all
the stream's messages.
2022-09-27 11:56:27 -07:00
Lauryn Menard 569863ffa6 templates: Improve footer content on documentation pages.
Makes the footer content on doc pages more contextually appropriate
for self-hosted organizations, when `settings.CORPORATE_ENABLED` is
false.

When `settings.CORPORATE_ENABLED` is true, there is specific footer
content for the policy documentation pages, and for the help center
and API documentation pages.

Fixes #23068.
2022-09-27 11:13:55 -07:00
Mateusz Mandera 065b59213b models: Rename get_huddle to get_or_create_huddle.
Small follow-up to d86e4ac34d.
get_ makes it sound like it doesn't have side-effects, when these are
actually much like the django ORM .get_or_create function.
2022-09-27 10:42:03 -07:00
Zixuan James Li a20c9ea28d scim: Use setattr to set request.user as scim_client.
This is a type-unsafe workaround before we can fix the problem that
django_scim2 relies on request.user being present to authenticate
requests.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-27 10:19:32 -07:00
Anders Kaseorg 676d40d66b sentry: Initialize sentry in AppConfig ready hook.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 7908e2c3a5 computed_settings: Move logging callbacks from zerver.lib.logging_util.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 02be415122 settings_type: Extract new module for types used in settings.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 9198fe4fac scim: Downgrade SCIMClient from a model to an ephemeral dataclass.
SCIMClient is a type-unsafe workaround for django-scim2’s conflation
of SCIM users with Django users.  Given that a SCIMClient is not a
UserProfile, it might as well not be a model at all, since it’s only
used to satisfy django-scim2’s request.user.is_authenticated queries.

This doesn’t solve the type safety issue with assigning a SCIMClient
to request.user, nor the performance issue with running the SCIM
middleware on non-SCIM requests.  But it reduces the risk of potential
consequences worse than crashing, since there’s no longer a
request.user.id for Django to confuse with the ID of an actual
UserProfile.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 11:36:48 -07:00
Anders Kaseorg ce9ceb7f9f tornado: Fix Tornado CSRF check with X-Forwarded-Proto.
Since Django factors request.is_secure() into its CSRF check, we need
this to tell it to consider requests forwarded from nginx to Tornado
as secure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-23 16:01:12 -07:00
Lauryn Menard b2e0b5187d api-docs: Document `away` as deprecated way to access presence_enabled.
Updates API documentation and changelog for user status `away`
now being a deprecated way to access a user's `presence_enabled`
setting for clients supporting older servers.

Final step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 4793f017f9 user-status: Delete status field from UserStatus model.
We are no longer writing to or reading the UserStatus.status field,
so we delete that from the model.

Fifth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 37aca4ac67 user-status: Stop updating the UserStatus model for `away` updates.
Fourth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers, and
checkpoint commit prior to deleting the `status` field from the
UserStatus model.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard e36cfdb0a7 user-status: Send `away=True` if `!presence_enabled` for user status.
We stop sending the `away=True` based on the user's `UserStatus`
object having `status=AWAY`, and instead send that value if
`!presence_enabled` for the user.

Third step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 7b128d6b1b user-settings: Migrate users with away status to `!presence_enabled`.
Now that user status updates with `away=True|False` also update the
user's presence_enabled setting, we do a migration so that users with
`UserStatus.status=AWAY` also have the presence_enabled setting as
False (`away=!presence_enabled`).

Second step in making user status away a deprecated way to access
presence_enabled for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 843eb4e4fc user-status: Update `presence_enabled` with changes to user status `away`.
When a user toggles a status update for `away=True|False`, we now update
their `presence_enabled` setting to match (`away!=presence_enabled`).

First step of making user status `away` updates a deprecated way to
access presence_enabled for clients supporting older servers, and
checkpoint commit before migrating users with a current UserStatus
of `status=AWAY` to have their `presence_enabled` set to `False`.

Note that when user status `away` is updated, we now send 4 events:
user_status, user_settings, presence, and update_global_notifications.

Also, this means that these updates change the UserPresence.status
value, which impacts the test for importing and exporting user
information.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 3428fe86d6 user-status: Move `do_update_user_status` to separate actions file.
We need to move this function to a separate actions file specifically
for `user_status` because otherwise we will have a circular import
between `actions/user_settings.py` and `actions/presence.py` in an
upcoming commit.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 47c1dbaa7d user-status: Refactor function names with "user_info" for clarity.
Rename functions that refer to "user_info" without a reference to
"status" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 32381f8678 user-presence: Refactor function names with "status" for clarity.
Rename functions that refer to "status" without a reference to
"presence" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Matt Keller fd996c286e slack: Filter out non-.json files for processing. 2022-09-23 09:59:34 -07:00
David Rosa c6abb7bedf help: Restructure "Mastering the compose box" article.
This help center article should include more features rather than just
focusing on the "go to conversation" button. We should broaden and
restructure this page to cover other advanced features.

Refactors the "Go to conversation" section as step-by-step instructions,
and adds a `keyboard_tip`.

Adds new section "Toggle between Ctrl+Enter and Enter".
Deletes the "Enable Enter to send" help center article, and adds its
content as a new subheading in this section.
Updates existing links accordingly and adds a URL redirect.
Documents "Enable Control + Enter to send".
Tweaks intro paragraph of "Mastering the compose box".

Fixes: #22817.
2022-09-22 15:20:37 -07:00
Mateusz Mandera 940830055b delete_in_topic: Split up the deletion into batches.
Fixes #22821.

As explained in the comment in the code:

Topics can be large enough that this request will inevitably time out.
In such a case, it's good for some progress to be accomplished, so that
full deletion can be achieved by repeating the request. For that purpose,
we delete messages in atomic batches, committing after each batch.

The additional perk is that the ordering of messages should prevent some
hypothetical deadlocks - ref #19054
2022-09-22 15:01:43 -07:00
Mateusz Mandera cf2f14f04c delete_in_topic: Name unused variable as ignored.
sub isn't used, so let's just call it ignored_sub to be explicit about
that intent.
2022-09-22 15:01:43 -07:00
yogesh sirsat 543f36b7da custom_profile_fields: Add "display_in_profile_summary" field in model.
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.

In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.

Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.

Fixes part of: #21215
2022-09-20 17:03:57 -07:00
Anders Kaseorg 2e9cd20380 timezone: Improve tzdata parser’s compatibility with zic(8).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 16:58:31 -07:00
Anders Kaseorg 7222f3fe2b tornado: Raise the same error for nonexistent and unauthorized queues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:20:36 -07:00
Anders Kaseorg f929050230 external_accounts: Make ExternalAccount dataclass to fix typing.
Commit 1a426fa6be (#22977) changed name
to be a StrPromise rather than a str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:14:33 -07:00
Yogesh Sirsat 1a426fa6be i18n: Add translation tags to default external accounts name. 2022-09-20 10:42:33 -07:00
Zixuan James Li 3193afffe8 test_runner: Remove monkey-patched SubSuiteList.
This monkey-patching approach is not meaningful when what we really need
is just the names of the test, that can already be done in
get_test_names.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-20 10:18:22 -07:00
Lauryn Menard 34f5218cc8 templates: Replace default HTML title and meta-description elements.
Removes the default title element of "Zulip" from `base.html` and
the default meta-description sentence from `meta_tags.html`. Also
removes default open graph metadata.

For portico templates that would would fail tests, set both
`PAGE_TITLE` and `PAGE_DESCRIPTION` variables with appropriate
content.

Co-authored-by: Alya Abbott <alya@zulip.com>
2022-09-19 20:17:17 -07:00
Anders Kaseorg 92f9a9ba41 forms: Fix another 500 error on realm creation with invalid email.
Followup to commit c0287473b9 (#22723).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 14:12:32 -07:00
Zixuan James Li 3400cb425b models: Tighten type annotation for query_for_ids.
This makes use of the type parameters of ValuesQuerySet (a.k.a
_QuerySet) to provide a more accurate type annotation for query_for_ids.

Note that QuerySet[ModelT] is equivalent to _QuerySet[ModelT, ModelT].

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-19 14:10:15 -07:00
Anders Kaseorg 23d9dd3163 streams: Fix can_remove_subscribers_from_stream type.
Added by commit c3fe8420fd (#22589).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 13:53:44 -07:00
Mateusz Mandera 29589a69eb bots: Replace some redundant logic with is_cross_realm_bot_email.
is_cross_realm_bot_email is just
`email.lower() in settings.CROSS_REALM_BOT_EMAILS` which is the same,
aside of looking at .lower() - which is actually more correct.
2022-09-17 17:52:01 -07:00