Commit Graph

15872 Commits

Author SHA1 Message Date
Sahil Batra 04693b6ac1 message_edit: Send only changed settings in event data and api response.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.

Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
2022-09-28 11:47:40 -07:00
Mateusz Mandera 522c159441 test_push_notifs: Change type_id arg of get_message to no default.
This isn't used anywhere, nor would type_id 100 make much sense.
2022-09-28 16:45:34 +02:00
Mateusz Mandera b35ad99035 test_push_notifications: Use proper user.id for Recipient type_id.
Recipient with type PERSONAL type_id 1 is a Recipient for a system bot,
since those get created first. Even if it doesn't break tests, it's
still bad, because it is not the intention of those tests to simulate a
cross-realm private message to a system bot.
2022-09-28 16:45:34 +02:00
Mateusz Mandera 5850c38f4e test_digest: Use proper stream.id in test_get_hot_topics.
Just using values 1 and 2 as stream ids is not good, because there's no
idea in which realm these streams are (or hypothetically if they exist).
This can create weird Messages with sender being a user of "zulip" realm
and the stream being in another realm - which would be a corrupted
state.
2022-09-28 16:45:25 +02:00
Kartik Srivastava 409ef040bb stream_topic: Refactor user_ids_muting_topic.
This refactors and renames user_ids_muting_topic to accept a parameter
'visibility_policy' and fetch user IDs that have a specific
visibility_policy(provided as the parameter) set for a topic.
2022-09-27 17:18:48 -07:00
Anders Kaseorg 8eec4bf171 message_fetch: Move narrowing query builder to zerver.lib.narrow.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-27 17:02:10 -07:00
Aman Agrawal b84722de01 events: Clean up logic for spectator events_register parameters.
Unfortunately, doing so requires forking common API documentation
text, since we're not making any changes to other endpoints that don't
allow unauthenticated requests at all.

Follow-up on #21995.
2022-09-27 16:12:59 -07:00
Mateusz Mandera 2811a1228f import_util: Make build_message only take kwargs.
build_message has a lot of arguments, so it's hard to verify correctness
of callers that just try to get the order right. It's much clearer to be
explicit via kwargs. mattermost.py and rocketchat.py already do this, so
let's bring slack.py and gitter.py up to par.
2022-09-27 15:04:48 -07:00
Mateusz Mandera e360b5e29b export: Remove unnecessary if in export with consent code.
This might be a bit cleaner.
2022-09-27 11:56:27 -07:00
Mateusz Mandera 318d7fd4cd export: Only export messages that a consenting user can access.
As mentioned in the TODO this commit deletes, the export with member
consent system was failing to account for the fact that if consenting
users only have access to a subset of messages of a stream with
protected history, only that subset should be exported - rather than all
the stream's messages.
2022-09-27 11:56:27 -07:00
Lauryn Menard 569863ffa6 templates: Improve footer content on documentation pages.
Makes the footer content on doc pages more contextually appropriate
for self-hosted organizations, when `settings.CORPORATE_ENABLED` is
false.

When `settings.CORPORATE_ENABLED` is true, there is specific footer
content for the policy documentation pages, and for the help center
and API documentation pages.

Fixes #23068.
2022-09-27 11:13:55 -07:00
Mateusz Mandera 065b59213b models: Rename get_huddle to get_or_create_huddle.
Small follow-up to d86e4ac34d.
get_ makes it sound like it doesn't have side-effects, when these are
actually much like the django ORM .get_or_create function.
2022-09-27 10:42:03 -07:00
Zixuan James Li a20c9ea28d scim: Use setattr to set request.user as scim_client.
This is a type-unsafe workaround before we can fix the problem that
django_scim2 relies on request.user being present to authenticate
requests.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-27 10:19:32 -07:00
Anders Kaseorg 676d40d66b sentry: Initialize sentry in AppConfig ready hook.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 7908e2c3a5 computed_settings: Move logging callbacks from zerver.lib.logging_util.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 02be415122 settings_type: Extract new module for types used in settings.
This breaks an import cycle that prevented django-stubs from inferring
types for django.conf.settings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 12:42:36 -07:00
Anders Kaseorg 9198fe4fac scim: Downgrade SCIMClient from a model to an ephemeral dataclass.
SCIMClient is a type-unsafe workaround for django-scim2’s conflation
of SCIM users with Django users.  Given that a SCIMClient is not a
UserProfile, it might as well not be a model at all, since it’s only
used to satisfy django-scim2’s request.user.is_authenticated queries.

This doesn’t solve the type safety issue with assigning a SCIMClient
to request.user, nor the performance issue with running the SCIM
middleware on non-SCIM requests.  But it reduces the risk of potential
consequences worse than crashing, since there’s no longer a
request.user.id for Django to confuse with the ID of an actual
UserProfile.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-26 11:36:48 -07:00
Anders Kaseorg ce9ceb7f9f tornado: Fix Tornado CSRF check with X-Forwarded-Proto.
Since Django factors request.is_secure() into its CSRF check, we need
this to tell it to consider requests forwarded from nginx to Tornado
as secure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-23 16:01:12 -07:00
Lauryn Menard b2e0b5187d api-docs: Document `away` as deprecated way to access presence_enabled.
Updates API documentation and changelog for user status `away`
now being a deprecated way to access a user's `presence_enabled`
setting for clients supporting older servers.

Final step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 4793f017f9 user-status: Delete status field from UserStatus model.
We are no longer writing to or reading the UserStatus.status field,
so we delete that from the model.

Fifth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 37aca4ac67 user-status: Stop updating the UserStatus model for `away` updates.
Fourth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers, and
checkpoint commit prior to deleting the `status` field from the
UserStatus model.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard e36cfdb0a7 user-status: Send `away=True` if `!presence_enabled` for user status.
We stop sending the `away=True` based on the user's `UserStatus`
object having `status=AWAY`, and instead send that value if
`!presence_enabled` for the user.

Third step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 7b128d6b1b user-settings: Migrate users with away status to `!presence_enabled`.
Now that user status updates with `away=True|False` also update the
user's presence_enabled setting, we do a migration so that users with
`UserStatus.status=AWAY` also have the presence_enabled setting as
False (`away=!presence_enabled`).

Second step in making user status away a deprecated way to access
presence_enabled for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 843eb4e4fc user-status: Update `presence_enabled` with changes to user status `away`.
When a user toggles a status update for `away=True|False`, we now update
their `presence_enabled` setting to match (`away!=presence_enabled`).

First step of making user status `away` updates a deprecated way to
access presence_enabled for clients supporting older servers, and
checkpoint commit before migrating users with a current UserStatus
of `status=AWAY` to have their `presence_enabled` set to `False`.

Note that when user status `away` is updated, we now send 4 events:
user_status, user_settings, presence, and update_global_notifications.

Also, this means that these updates change the UserPresence.status
value, which impacts the test for importing and exporting user
information.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 3428fe86d6 user-status: Move `do_update_user_status` to separate actions file.
We need to move this function to a separate actions file specifically
for `user_status` because otherwise we will have a circular import
between `actions/user_settings.py` and `actions/presence.py` in an
upcoming commit.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 47c1dbaa7d user-status: Refactor function names with "user_info" for clarity.
Rename functions that refer to "user_info" without a reference to
"status" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 32381f8678 user-presence: Refactor function names with "status" for clarity.
Rename functions that refer to "status" without a reference to
"presence" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Matt Keller fd996c286e slack: Filter out non-.json files for processing. 2022-09-23 09:59:34 -07:00
David Rosa c6abb7bedf help: Restructure "Mastering the compose box" article.
This help center article should include more features rather than just
focusing on the "go to conversation" button. We should broaden and
restructure this page to cover other advanced features.

Refactors the "Go to conversation" section as step-by-step instructions,
and adds a `keyboard_tip`.

Adds new section "Toggle between Ctrl+Enter and Enter".
Deletes the "Enable Enter to send" help center article, and adds its
content as a new subheading in this section.
Updates existing links accordingly and adds a URL redirect.
Documents "Enable Control + Enter to send".
Tweaks intro paragraph of "Mastering the compose box".

Fixes: #22817.
2022-09-22 15:20:37 -07:00
Mateusz Mandera 940830055b delete_in_topic: Split up the deletion into batches.
Fixes #22821.

As explained in the comment in the code:

Topics can be large enough that this request will inevitably time out.
In such a case, it's good for some progress to be accomplished, so that
full deletion can be achieved by repeating the request. For that purpose,
we delete messages in atomic batches, committing after each batch.

The additional perk is that the ordering of messages should prevent some
hypothetical deadlocks - ref #19054
2022-09-22 15:01:43 -07:00
Mateusz Mandera cf2f14f04c delete_in_topic: Name unused variable as ignored.
sub isn't used, so let's just call it ignored_sub to be explicit about
that intent.
2022-09-22 15:01:43 -07:00
yogesh sirsat 543f36b7da custom_profile_fields: Add "display_in_profile_summary" field in model.
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.

In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.

Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.

Fixes part of: #21215
2022-09-20 17:03:57 -07:00
Anders Kaseorg 2e9cd20380 timezone: Improve tzdata parser’s compatibility with zic(8).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 16:58:31 -07:00
Anders Kaseorg 7222f3fe2b tornado: Raise the same error for nonexistent and unauthorized queues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:20:36 -07:00
Anders Kaseorg f929050230 external_accounts: Make ExternalAccount dataclass to fix typing.
Commit 1a426fa6be (#22977) changed name
to be a StrPromise rather than a str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:14:33 -07:00
Yogesh Sirsat 1a426fa6be i18n: Add translation tags to default external accounts name. 2022-09-20 10:42:33 -07:00
Zixuan James Li 3193afffe8 test_runner: Remove monkey-patched SubSuiteList.
This monkey-patching approach is not meaningful when what we really need
is just the names of the test, that can already be done in
get_test_names.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-20 10:18:22 -07:00
Lauryn Menard 34f5218cc8 templates: Replace default HTML title and meta-description elements.
Removes the default title element of "Zulip" from `base.html` and
the default meta-description sentence from `meta_tags.html`. Also
removes default open graph metadata.

For portico templates that would would fail tests, set both
`PAGE_TITLE` and `PAGE_DESCRIPTION` variables with appropriate
content.

Co-authored-by: Alya Abbott <alya@zulip.com>
2022-09-19 20:17:17 -07:00
Anders Kaseorg 92f9a9ba41 forms: Fix another 500 error on realm creation with invalid email.
Followup to commit c0287473b9 (#22723).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 14:12:32 -07:00
Zixuan James Li 3400cb425b models: Tighten type annotation for query_for_ids.
This makes use of the type parameters of ValuesQuerySet (a.k.a
_QuerySet) to provide a more accurate type annotation for query_for_ids.

Note that QuerySet[ModelT] is equivalent to _QuerySet[ModelT, ModelT].

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-19 14:10:15 -07:00
Anders Kaseorg 23d9dd3163 streams: Fix can_remove_subscribers_from_stream type.
Added by commit c3fe8420fd (#22589).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 13:53:44 -07:00
Mateusz Mandera 29589a69eb bots: Replace some redundant logic with is_cross_realm_bot_email.
is_cross_realm_bot_email is just
`email.lower() in settings.CROSS_REALM_BOT_EMAILS` which is the same,
aside of looking at .lower() - which is actually more correct.
2022-09-17 17:52:01 -07:00
rht a7cff0f091 Slack import: Translate to emoji name to codepoint using iamcal data.
Because Slack emoji naming is different from Zulip's.
According to https://emojipedia.org/slack/, Slack's emoji shortcodes are
derived from https://github.com/iamcal/emoji-data.
There are probably some deviations from that dataset, but this PR should
at least catch the ones that are identical to iamcal's.
2022-09-17 12:04:07 -07:00
Anders Kaseorg 403837e52d python: Use ‘not in’ for negated membership tests
Fixes “E713 Test for membership should be `not in`” found by
ruff (https://github.com/charliermarsh/ruff).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-17 11:48:33 -07:00
Jason Hildebrand 7efd59b6d7 email_mirror: Remove limits (expiry, max uses) to improve usability.
These limits don't appear to provide useful security benefits, and
they do impact usability because they prevented email-based users from
replying more than once, or from replying to message more than 5 days
old.

Fixes #2755.
Fixes #19994.
2022-09-16 18:07:28 -07:00
yogesh sirsat 180a9cbdcb stream_bots: Allow bot owners to unsubscribe their bots from streams.
Users who owns bots can unsubscribe their bots from streams.

Fixes part of: #21402
2022-09-16 17:51:34 -07:00
Lauryn Menard eb377a8872 read_receipts: Exclude muted users from read receipts.
Removes IDs of users who have muted or been muted by the current
user from the list of user IDs returned by the read receipts
endpoint.

Fixes #22909.
2022-09-16 16:19:54 -07:00
Lauryn Menard 5e58f86aa7 models: Add backward relation to MutedUser foreign keys.
Updates the two UserProfile foreign key fields to have a backward
relation in the MutedUser model by changing the `related_name`
property.

This is a prep commit for removing users with a muted relationship
to the current user from read receipts.
2022-09-16 16:19:54 -07:00
Sahil Batra b9e0575829 realm: Allow only owners to change waiting_period_threshold setting.
We allow only owners to change the waiting period setting to become
full member. This commit contains only backend changes, frontend
changes will be done separately.
2022-09-16 15:27:52 -07:00
Sahil Batra f23bfbd462 realm_domains: Allow only owners to add, edit or delete domains.
We allow only owners to add, edit or delete the allowed domains.
This commit only contains backend changes, frontend changes will
be done in a separate commit.
2022-09-16 15:27:52 -07:00