Commit Graph

15855 Commits

Author SHA1 Message Date
Anders Kaseorg ce9ceb7f9f tornado: Fix Tornado CSRF check with X-Forwarded-Proto.
Since Django factors request.is_secure() into its CSRF check, we need
this to tell it to consider requests forwarded from nginx to Tornado
as secure.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-23 16:01:12 -07:00
Lauryn Menard b2e0b5187d api-docs: Document `away` as deprecated way to access presence_enabled.
Updates API documentation and changelog for user status `away`
now being a deprecated way to access a user's `presence_enabled`
setting for clients supporting older servers.

Final step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 4793f017f9 user-status: Delete status field from UserStatus model.
We are no longer writing to or reading the UserStatus.status field,
so we delete that from the model.

Fifth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 37aca4ac67 user-status: Stop updating the UserStatus model for `away` updates.
Fourth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers, and
checkpoint commit prior to deleting the `status` field from the
UserStatus model.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard e36cfdb0a7 user-status: Send `away=True` if `!presence_enabled` for user status.
We stop sending the `away=True` based on the user's `UserStatus`
object having `status=AWAY`, and instead send that value if
`!presence_enabled` for the user.

Third step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 7b128d6b1b user-settings: Migrate users with away status to `!presence_enabled`.
Now that user status updates with `away=True|False` also update the
user's presence_enabled setting, we do a migration so that users with
`UserStatus.status=AWAY` also have the presence_enabled setting as
False (`away=!presence_enabled`).

Second step in making user status away a deprecated way to access
presence_enabled for clients supporting older servers.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 843eb4e4fc user-status: Update `presence_enabled` with changes to user status `away`.
When a user toggles a status update for `away=True|False`, we now update
their `presence_enabled` setting to match (`away!=presence_enabled`).

First step of making user status `away` updates a deprecated way to
access presence_enabled for clients supporting older servers, and
checkpoint commit before migrating users with a current UserStatus
of `status=AWAY` to have their `presence_enabled` set to `False`.

Note that when user status `away` is updated, we now send 4 events:
user_status, user_settings, presence, and update_global_notifications.

Also, this means that these updates change the UserPresence.status
value, which impacts the test for importing and exporting user
information.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 3428fe86d6 user-status: Move `do_update_user_status` to separate actions file.
We need to move this function to a separate actions file specifically
for `user_status` because otherwise we will have a circular import
between `actions/user_settings.py` and `actions/presence.py` in an
upcoming commit.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 47c1dbaa7d user-status: Refactor function names with "user_info" for clarity.
Rename functions that refer to "user_info" without a reference to
"status" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 32381f8678 user-presence: Refactor function names with "status" for clarity.
Rename functions that refer to "status" without a reference to
"presence" to help clarify in the backend between UserPresence
and UserStatus models.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
Matt Keller fd996c286e slack: Filter out non-.json files for processing. 2022-09-23 09:59:34 -07:00
David Rosa c6abb7bedf help: Restructure "Mastering the compose box" article.
This help center article should include more features rather than just
focusing on the "go to conversation" button. We should broaden and
restructure this page to cover other advanced features.

Refactors the "Go to conversation" section as step-by-step instructions,
and adds a `keyboard_tip`.

Adds new section "Toggle between Ctrl+Enter and Enter".
Deletes the "Enable Enter to send" help center article, and adds its
content as a new subheading in this section.
Updates existing links accordingly and adds a URL redirect.
Documents "Enable Control + Enter to send".
Tweaks intro paragraph of "Mastering the compose box".

Fixes: #22817.
2022-09-22 15:20:37 -07:00
Mateusz Mandera 940830055b delete_in_topic: Split up the deletion into batches.
Fixes #22821.

As explained in the comment in the code:

Topics can be large enough that this request will inevitably time out.
In such a case, it's good for some progress to be accomplished, so that
full deletion can be achieved by repeating the request. For that purpose,
we delete messages in atomic batches, committing after each batch.

The additional perk is that the ordering of messages should prevent some
hypothetical deadlocks - ref #19054
2022-09-22 15:01:43 -07:00
Mateusz Mandera cf2f14f04c delete_in_topic: Name unused variable as ignored.
sub isn't used, so let's just call it ignored_sub to be explicit about
that intent.
2022-09-22 15:01:43 -07:00
yogesh sirsat 543f36b7da custom_profile_fields: Add "display_in_profile_summary" field in model.
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.

In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.

Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.

Fixes part of: #21215
2022-09-20 17:03:57 -07:00
Anders Kaseorg 2e9cd20380 timezone: Improve tzdata parser’s compatibility with zic(8).
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 16:58:31 -07:00
Anders Kaseorg 7222f3fe2b tornado: Raise the same error for nonexistent and unauthorized queues.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:20:36 -07:00
Anders Kaseorg f929050230 external_accounts: Make ExternalAccount dataclass to fix typing.
Commit 1a426fa6be (#22977) changed name
to be a StrPromise rather than a str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:14:33 -07:00
Yogesh Sirsat 1a426fa6be i18n: Add translation tags to default external accounts name. 2022-09-20 10:42:33 -07:00
Zixuan James Li 3193afffe8 test_runner: Remove monkey-patched SubSuiteList.
This monkey-patching approach is not meaningful when what we really need
is just the names of the test, that can already be done in
get_test_names.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-20 10:18:22 -07:00
Lauryn Menard 34f5218cc8 templates: Replace default HTML title and meta-description elements.
Removes the default title element of "Zulip" from `base.html` and
the default meta-description sentence from `meta_tags.html`. Also
removes default open graph metadata.

For portico templates that would would fail tests, set both
`PAGE_TITLE` and `PAGE_DESCRIPTION` variables with appropriate
content.

Co-authored-by: Alya Abbott <alya@zulip.com>
2022-09-19 20:17:17 -07:00
Anders Kaseorg 92f9a9ba41 forms: Fix another 500 error on realm creation with invalid email.
Followup to commit c0287473b9 (#22723).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 14:12:32 -07:00
Zixuan James Li 3400cb425b models: Tighten type annotation for query_for_ids.
This makes use of the type parameters of ValuesQuerySet (a.k.a
_QuerySet) to provide a more accurate type annotation for query_for_ids.

Note that QuerySet[ModelT] is equivalent to _QuerySet[ModelT, ModelT].

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-19 14:10:15 -07:00
Anders Kaseorg 23d9dd3163 streams: Fix can_remove_subscribers_from_stream type.
Added by commit c3fe8420fd (#22589).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-19 13:53:44 -07:00
Mateusz Mandera 29589a69eb bots: Replace some redundant logic with is_cross_realm_bot_email.
is_cross_realm_bot_email is just
`email.lower() in settings.CROSS_REALM_BOT_EMAILS` which is the same,
aside of looking at .lower() - which is actually more correct.
2022-09-17 17:52:01 -07:00
rht a7cff0f091 Slack import: Translate to emoji name to codepoint using iamcal data.
Because Slack emoji naming is different from Zulip's.
According to https://emojipedia.org/slack/, Slack's emoji shortcodes are
derived from https://github.com/iamcal/emoji-data.
There are probably some deviations from that dataset, but this PR should
at least catch the ones that are identical to iamcal's.
2022-09-17 12:04:07 -07:00
Anders Kaseorg 403837e52d python: Use ‘not in’ for negated membership tests
Fixes “E713 Test for membership should be `not in`” found by
ruff (https://github.com/charliermarsh/ruff).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-17 11:48:33 -07:00
Jason Hildebrand 7efd59b6d7 email_mirror: Remove limits (expiry, max uses) to improve usability.
These limits don't appear to provide useful security benefits, and
they do impact usability because they prevented email-based users from
replying more than once, or from replying to message more than 5 days
old.

Fixes #2755.
Fixes #19994.
2022-09-16 18:07:28 -07:00
yogesh sirsat 180a9cbdcb stream_bots: Allow bot owners to unsubscribe their bots from streams.
Users who owns bots can unsubscribe their bots from streams.

Fixes part of: #21402
2022-09-16 17:51:34 -07:00
Lauryn Menard eb377a8872 read_receipts: Exclude muted users from read receipts.
Removes IDs of users who have muted or been muted by the current
user from the list of user IDs returned by the read receipts
endpoint.

Fixes #22909.
2022-09-16 16:19:54 -07:00
Lauryn Menard 5e58f86aa7 models: Add backward relation to MutedUser foreign keys.
Updates the two UserProfile foreign key fields to have a backward
relation in the MutedUser model by changing the `related_name`
property.

This is a prep commit for removing users with a muted relationship
to the current user from read receipts.
2022-09-16 16:19:54 -07:00
Sahil Batra b9e0575829 realm: Allow only owners to change waiting_period_threshold setting.
We allow only owners to change the waiting period setting to become
full member. This commit contains only backend changes, frontend
changes will be done separately.
2022-09-16 15:27:52 -07:00
Sahil Batra f23bfbd462 realm_domains: Allow only owners to add, edit or delete domains.
We allow only owners to add, edit or delete the allowed domains.
This commit only contains backend changes, frontend changes will
be done in a separate commit.
2022-09-16 15:27:52 -07:00
Sahil Batra 1396fbab6e realm: Allow only owners to change restricted-login related settings.
We allow only owners to change disallow_disposable_email_addresses
and emails_restricted_to_domains settings. This commit only contains
change in backend part, frontend changes will be done separately.
2022-09-16 15:27:52 -07:00
Sahil Batra 63fa2a9338 realm: Allow only owners to change invite_required in backend.
We allow only owners to change the invite_required setting.
This commit only adds the restriction in backend, frontend
changes will be done separately.

We also add a helper function in test_realm.py to avoid
writing same code repeatedly and this helper will also
be used in tests for other settings to be added in
further commits.
2022-09-16 15:04:54 -07:00
Tim Abbott 76bcb96414 realm_export: Improve estimate of data export size.
As suggested by the new comments, the cost for a Zulip data export
scales with messages actually included in the export, so an
organizations with 1M private messages but only 50K public stream
messages should not be modeled the same as one with 1M public stream
messages for the purpose of the limits here.

Also improve the comments and variable names more generally.
2022-09-16 14:54:45 -07:00
Anders Kaseorg 31b4be3b9b event_queue: Replace sequential queue IDs with UUIDs.
This prevents ID conflicts when resharding between multiple Tornado
shards.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-16 10:06:07 -07:00
Tim Abbott 183e7be31c models: Fix subscribed typo. 2022-09-15 16:21:09 -07:00
Mateusz Mandera d86e4ac34d models: Improve comment documentation around Recipients and Huddles. 2022-09-15 16:12:17 -07:00
Anders Kaseorg 987ab741f9 sharding: Support Tornado sharding by regexes.
One should now be able to configure a regex by appending _regex to the
port number:

[tornado_sharding]
9802_regex = ^[l-p].*\.zulipchat\.com$

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-15 16:07:50 -07:00
Sahil Batra c3fe8420fd streams: Allow unsubscribing others based on can_remove_subscribers_group.
Current value of can_remove_subscribers_group field is admins system group
only so behavior is not changed. We would provide support to change this
setting from API and UI in further commits.
2022-09-14 16:14:15 -07:00
Sahil Batra b9248c75f4 stream: Add do_change_can_remove_subscribers_group and field to objects.
This commit adds do_change_can_remove_subscriber_group function for
changing can_remove_subscribers_group field of a stream. We also add
can_remove_subscribers_group_id field to stream and subscription
objects.

This function will be helpful for writing tests in next commit.
We would add API and UI support to change this setting in further
commits.
2022-09-14 16:03:11 -07:00
Sahil Batra 86c2f6881e models: Update can_remove_subscribers_group field to not be null.
This commit udpates can_remove_subscribers_group to be not null.
We already added a migration to set the value of this field for
existing streams and also added a commit to set this field to
admins system group for now while creating streams.
2022-09-14 16:03:11 -07:00
Sahil Batra c9c230c2da migrations: Add migration to set default of can_remove_subscribers_group.
This migration sets can_remove_subscribers_group value to admins system
group for all the existing streams. In further commit we would change
can_remove_subscribers_group to be not null and thus we add this migration
to ensure all existing streams have this setting value set.
2022-09-14 16:03:11 -07:00
Sahil Batra b0de5c0f36 streams: Set can_remove_subscribers_group while creating streams.
This commit sets can_remove_subscribers_group to admins system
group while creating streams as it will be the default value
of this setting. In further we would provide an option to set
value of this setting to any user group while creating streams
using API or UI.
2022-09-14 16:03:11 -07:00
Sahil Batra 941bb5a372 import: Fix import related code for user groups.
We change the import order to import UserGroup objects before
Stream such that we can set can_remove_subscribers_group correctly.

We do not import UserGroupMembership objects here along with
UserGroup since UserProfile objects are not imported and
GroupGroupMembership are also imported later as these are not
required before.
2022-09-14 16:03:11 -07:00
Sahil Batra 49d61d19f4 models: Add can_remove_subscribers_group field to Stream.
This new setting will be used to check which user group
is allowed to unsubscribe others from the stream.
2022-09-14 16:03:11 -07:00
Zixuan James Li e2bf8c49af test_decorators: Remove cachify test cases.
cachify has been removed in 9d448e73d2.
We don't need to keep its tests.

TODO: functools.lru_cache can be replaced by functools.cache when we
drop Python 3.8.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-09-14 09:40:46 -07:00
Anders Kaseorg 61e149dbb5 subdomain: Remove impossible None case from user_matches_subdomain.
The only two callers pass get_subdomain(request) which can’t be None.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-14 09:35:36 -07:00
Lauryn Menard f0dac7beef docs: Update base titles for documentation pages.
Updates the base titles so that they begin with the page content,
and end with "| Zulip" +  information about the type of doc: "help
center", "API documentation", "terms and policies", or "integrations".
2022-09-13 11:36:52 -07:00