Commit Graph

144 Commits

Author SHA1 Message Date
Tim Abbott b057d8ea17 invites: Delete ScheduledEmails in revoke invites transactions.
This prevents likely rare race conditions where a reminder about a
revoked invitation might get sent.
2022-10-31 15:06:42 -07:00
Mateusz Mandera 3dd0545118 invites: Add atomic() around some invite-revoking blocks.
These should only ever happen together, so should be wrapped in
atomic().
2022-10-31 14:37:46 -07:00
Mateusz Mandera 78a325ac58 internal_send_private_message: Fix edge case of message to system bot.
I don't think this is used anywhere outside of tests, but we should have
this logic correct. If this function is used to send a message from a
user to a cross-realm bot, the message.realm should be the realm of the
user.

In the normal case, where a user send a message to a cross-realm bot
through the API is already handled correctly, this bug is unrelated.
2022-10-31 10:23:57 -07:00
Mateusz Mandera a410f6b241 do_mark_all_as_read: Split up the work into batches.
Fixes #15403.
2022-10-27 16:59:54 -07:00
Julia Bichler 4bb381fc80 message_edit: Support sending notifications with topic changes.
Previously we did not send notification for topic-only edits.
Now, we add backend support for sending notification to topic-only
edits as well.

We would add support for this in webapp in further commits since
message edit UI will be updated as well. We just make sure that no
notifications are sent when editing topic using pencil icon in
message header.

We also change the API default for moving a topic to only notify the
new location, not the old one; this matches the current defaults in
the web UI.

Includes many tests.

We also update the puppeteer tests to test only content edit as
we are going to change the UI to not allow topic editing from
message edit UI. Also fixing the existing tests to pass while
doing topic edits is somewhat complex as notification message
is also sent to new topic by default.

Fixes #21712.

Co-authored-by: Aman Agrawal <amanagr@zulip.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-10-11 11:35:41 -07:00
Mateusz Mandera 470c0458e6 do_deactivate_user: Use .on_commit around send_event calls.
The previous commit did this for revoking sessions. send_events should
be handled similarly too, to correctly handle calling do_deactivate_user
inside a transaction.
2022-10-11 10:49:07 -07:00
Mateusz Mandera a94b2572be do_deactivate_user: Revoke sessions in transaction.on_commit().
Fixes #21709.
2022-10-11 10:49:07 -07:00
Mateusz Mandera 00b3546c9f models: Add denormalized .realm column to Message.
This commit adds the OPTIONAL .realm attribute to Message
(and ArchivedMessage), with the server changes for making new Messages
have this set. Old Messages still have to be migrated to backfill this,
before it can be non-nullable.

Appropriate test changes to correctly set .realm for Messages the tests
manually create are included here as well.
2022-10-07 10:09:38 -07:00
Anders Kaseorg 92ad4455ed requirements: Upgrade Django to 4.1.
zerver/migrations/0240_usermessage_migrate_bigint_id_into_id.py needs
to be updated to account for Django 4.1 creating AutoField as an
identity column rather than a serial column.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-10-06 15:59:07 -07:00
Zixuan James Li 7fd8d77ce0 typing: Import ValuesQuerySet alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Zixuan James Li a4eaa770f0 typing: Import StrPromise alias from django_stubs_ext.
This saves us from using a conditional import.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-10-05 16:15:56 -07:00
Sahil Batra 04693b6ac1 message_edit: Send only changed settings in event data and api response.
Previously, we included all three message edit related settings
("allow_message_editing", "message_content_edit_limit_seconds" and
"edit_topic_policy") in the event data and api response irrespective
of which of these settings were changed. Now, we only include changed
settings and separate events are sent for each setting if more than
one of them is changed.

Note that the previous typed in event_schema.py for
`message_content_edit_limit_seconds` incorrectly did not allow `None`
as a value, which is used to encode no limit.
2022-09-28 11:47:40 -07:00
Kartik Srivastava 409ef040bb stream_topic: Refactor user_ids_muting_topic.
This refactors and renames user_ids_muting_topic to accept a parameter
'visibility_policy' and fetch user IDs that have a specific
visibility_policy(provided as the parameter) set for a topic.
2022-09-27 17:18:48 -07:00
Lauryn Menard 37aca4ac67 user-status: Stop updating the UserStatus model for `away` updates.
Fourth step in making user status `away` a deprecated way to access
`presence_enabled` for clients supporting older servers, and
checkpoint commit prior to deleting the `status` field from the
UserStatus model.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 843eb4e4fc user-status: Update `presence_enabled` with changes to user status `away`.
When a user toggles a status update for `away=True|False`, we now update
their `presence_enabled` setting to match (`away!=presence_enabled`).

First step of making user status `away` updates a deprecated way to
access presence_enabled for clients supporting older servers, and
checkpoint commit before migrating users with a current UserStatus
of `status=AWAY` to have their `presence_enabled` set to `False`.

Note that when user status `away` is updated, we now send 4 events:
user_status, user_settings, presence, and update_global_notifications.

Also, this means that these updates change the UserPresence.status
value, which impacts the test for importing and exporting user
information.

Part of transitioning from 'unavailable' user status feature to
'invisible mode' user presence feature.
2022-09-23 12:27:54 -07:00
Lauryn Menard 3428fe86d6 user-status: Move `do_update_user_status` to separate actions file.
We need to move this function to a separate actions file specifically
for `user_status` because otherwise we will have a circular import
between `actions/user_settings.py` and `actions/presence.py` in an
upcoming commit.

Prep commit for migrating "unavailable" user status feature to
"invisible" user presence feature.
2022-09-23 12:27:54 -07:00
yogesh sirsat 543f36b7da custom_profile_fields: Add "display_in_profile_summary" field in model.
To allow `custom_profile_field` to display in user profile popover,
added new boolean field "display_in_profile_summary" in its model class.

In `custom_profile_fields.py`, functions are edited as per conditions,
like currently we can display max 2 `custom_profile_fields` except
`LONG_TEXT` and `USER` type fields.

Default external account custom profile fields made updatable for only
this new field, as previous they were not updatable.

Fixes part of: #21215
2022-09-20 17:03:57 -07:00
Anders Kaseorg f929050230 external_accounts: Make ExternalAccount dataclass to fix typing.
Commit 1a426fa6be (#22977) changed name
to be a StrPromise rather than a str.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-09-20 15:14:33 -07:00
Sahil Batra b9248c75f4 stream: Add do_change_can_remove_subscribers_group and field to objects.
This commit adds do_change_can_remove_subscriber_group function for
changing can_remove_subscribers_group field of a stream. We also add
can_remove_subscribers_group_id field to stream and subscription
objects.

This function will be helpful for writing tests in next commit.
We would add API and UI support to change this setting in further
commits.
2022-09-14 16:03:11 -07:00
Christopher Chong 28173cafc8 message_flags: Fix deadlocks when updating message flags.
Previously, an active production Zulip server would experience a class
of deadlocks caused by two or more concurrent bulk update operations
on the UserMessage table.

This is because UPDATE ... SET ... WHERE statements that execute in
parallel take row-level UPDATE locks as they get results; since the
query plans may result in getting rows in different orders between two
queries, this can result in deadlocks.

Some databases allow ORDER BY on their UPDATE ... WHERE statements;
PostgreSQL does not. In PostgreSQL, the answer is to do a sub-select
with an ORDER BY ... FOR UPDATE to ensure consistent ordering on row
locks.

We do this all code paths using bitand or bitor as part of bulk
editing message flags, which should ensure that these concurrent
operations obtain row level locks on the table in the same order.

Fixes #19054.
2022-09-06 16:06:58 -07:00
Mateusz Mandera 01d659b672 internal_prep_private_message: Remove redundant realm arg.
This is redundant and is taken based on recipient_user anyway.
2022-09-06 15:03:17 -07:00
madrix01 4303ba1efc actions: Create a separate message_delete.py file.
This is preparatory commit for #18941.
Importing `do_delete_message` from `message_edit.py` was causing a
circular import error. In order to avoid that, we create a separate
message_delete.py file which has all the functions related to deleting
messages.
The tests for deleting messages are present in
`zerver/tests/test_message_edit.py`.

Fixes a part of #18941
2022-09-01 14:18:38 -07:00
Zixuan James Li 3df8dd4b38 typing: Broaden parameter type for bulk_regenerate_api_keys.
In zerver.management.commands.logout_all_users,
we pass a values queryset containing the ids into
this function, which is not actually a list. This
broadens the type annotation so that the ValuesQuerySet
is accepted.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 11:06:41 -07:00
Zixuan James Li bb9e80d7a2 i18n: Deal with lazy strings more carefully.
This uses a more specific type `_StrPromise` to replace `Promise`
providing typing information for lazy translation strings.

In places where the callee evaluates the `_StrPromise` object in all
cases we simply force the evaluation with `str()`. This includes
`JsonableError` that ends up handled by the error handler middleware,
and `internal_send_stream_message` that depends on `check_stream_topic`,
requiring the `topic` to be evaluated anyway. In other siuations, the
callee is expected to be able to handle `StrPromise` explicitly.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-29 10:52:45 -07:00
Adam Sah 637867dad1 settings: Send presence update event toggling presence_enabled.
Fixes #21180.
2022-08-25 12:30:12 -07:00
Julia Bichler 33c790e2fc message_edit: Return a bool in maybe_send_resolve_topic_notifications.
Whether we sent a resolve topic notification or not may be useful in
the caller. It was originally intended to be used in #21712, but may
only be relevant for future logging.

Part of #21712.
2022-08-18 15:39:23 -07:00
Julia Bichler 83254a3d79 message_edit: Check resolve topic notifications earlier.
This will have no real effect in most situations. However, a user
moves a topic to another stream while also adding/removing the
resolved-topic checkmark from the topic name, then the "This topic was
resolved" notificaiton will now appear just before the "This topic was
moved" notification rather than just after.

This is likely slightly less confusing to users, since the topic
having been moved from somewhere else is likely the most salient fact
to a reader.

We expect to change things to not send both notifications in an
upcoming commit.

This refactoring helps with #21712.
2022-08-18 15:39:17 -07:00
Mateusz Mandera 2c693f3bd9 billing: Fix licenses amount check during user signup/invitation.
Our seat count calculation is different for guest user than normal users
(a number of initial guests are free, and additional marginal guests are
worth 1/5 of a seat) - so these checks we apply when a user is being
invited or signing up need to know whether it's a guest or non-guest
being added.
2022-08-18 11:56:54 -07:00
Lauryn Menard 885c3d6593 subscriptions: Send update events for `is_muted` property.
In Zulip 2.1.0, the `is_muted` stream subscription property was
added and replaced the `in_home_view` property. But the server has
still only been sending subscription update events with the
`in_home_view` property.

Updates `do_change_subscription_property` to send a subscription
update event for both `is_muted` and `in_home_view`, so that
clients can fully migrate away from using `in_home_view` allowing
us to eventually remove it completely.
2022-08-16 16:50:51 -07:00
Mateusz Mandera d48075de58 logout_all_users: Add --rotate-api-keys option to the command.
Fixes #19397.
2022-08-15 17:58:05 -07:00
Sahil Batra d3468e3f78 streams: Extract code for sending events in bulk_remove_subscriptions.
This is a prep commit such that we can avoid duplicate code when we
unsubscribe bots for inaccessible private streams when changing owner
or reactivating them.
2022-08-13 14:28:30 -07:00
Sahil Batra a1f40ccda5 message_edit: Make zero invalid value for message_content_edit_time_limit_seconds.
This commit changes the code to consider zero as an invalid value for
message_content_edit_time_limit_seconds. Now to represent the setting that
user can edit the message anytime, the setting value will be "None" in
database and "unlimited" will be passed to API from clients.
2022-08-12 18:09:53 -07:00
Zixuan James Li c1f5609e59 streams: Add None check for stream.recipient_id.
Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Zixuan James Li 7acf43f475 bots: Allow changing bot owner without an acting_user.
This applies to the usecase in
zerver.tests.test_bots.BotTest.test_patch_bot_role.

Signed-off-by: Zixuan James Li <p359101898@gmail.com>
2022-08-12 17:08:04 -07:00
Dinesh 48d2783559 read_receipts: Add support for displaying read receipts.
Adds an API endpoint for accessing read receipts for other users, as
well as a modal UI for displaying that information.

Enables the previously merged privacy settings UI for managing whether
a user makes read receipts data available to other users.

Documentation is pending, and we'll likely want to link to the
documentation with help_settings_link once it is complete.

Fixes #3618.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-08-12 13:16:35 -07:00
Sahil Batra 8eed801380 user_groups: Add MEMBERS_GROUP_NAME constant.
We now use MEMBERS_GROUP_NAME instead of writing
the actual group name at multiple places, so that we
can have all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Sahil Batra 31d639160f user_groups: Add FULL_MEMBERS_GROUP_NAME constant.
We now use FULL_MEMBERS_GROUP_NAME instead of
writing the actual full members system group
name at multiple places, so that we can have
all the group names coded at one place only.
2022-08-11 04:38:36 -07:00
Tim Abbott 66b29fbb8c streams: Centralize logic for computing stream permissions.
I found the previous model for computing what settings to use for
streams increasingly difficult to understand, which is generally a
recipe for future bugs.

Refactor to have a clear computation of what complete permissions
state the client is requesting, validate that state, and then pass
that state to the do_change_stream_permission.
2022-08-09 17:05:38 -07:00
Sahil Batra 3e6463804e streams: Allow changing history access without is_private parameter.
We now allow changing access to history of the stream by only passing
"history_public_to_subscribers" parameter. Previously, "is_private"
parameter was also required to change history_public_to_subscribers
otherwise the request was silently ignored.

We also raise error when only history_public_to_subscribers parameter
is passed with value False without "is_private: True" for a public
or web-public stream since we do not allow public streams with
protected history.
2022-08-09 17:05:38 -07:00
Sahil Batra 6a860c74d6 streams: Remove redundant asserts in do_change_stream_permission.
This commit removes the unnecessary assertion statements in
do_change_stream_permission for case when "is_web_public" is
True, since we already check those cases in the view function
update_stream_backend and this is the only place from where
do_change_stream_permission is called.

We aim to remove other assertions also from there as mentioned
in the comment and instead check the values in caller itself.
2022-08-09 17:05:38 -07:00
Tim Abbott bd04733c0f muted_topics: Clean up unmute topic event code duplication.
The previous construction was pretty fragile, and had a long comment
to reflect that fact; rework to achieve the same effect in a much
cleaner way.
2022-08-04 17:44:00 -07:00
Kartik Srivastava 1291e7000b user_topic: Add user_topic event.
We now send a new user_topic event while muting and unmuting topics.
fetch_initial_state_data now returns an additional user_topics array to
the client that will maintain the user-topic relationship data.
This will support any future addition of new features to modify the
relationship between a user-topic pair.

This commit adds the relevent backend code and schema for the new
event.
2022-08-04 17:44:00 -07:00
Sahil Batra 2713dd4e78 message_edit: Separate permission checks for topic edits.
We separate the permission checks for content and topic edits
by changing the can_edit_topic_or_content to can_edit_topic
and use it only for checking topic edits and check content
edits separately in check_update_message itself. There is no
change in behavior as of this PR, there will be more changes
as per #21739.

This is a prep commit for #21739. The permission checks for
them are essentially separate except the one that message
sender is allowed to edit content and topic irresepctive of
edit_topic_policy setting, and this will too be changed in
future commit and so it will be better to have these checks
separate for readability.

We can also probably create a new function for checking content
edits but currently we only check the sender is same as the use
who is editing and it does not make sense to have a separate
function for just one check. We can do so in future in case we
do some more refactoring for #21739.
2022-08-03 15:59:51 -07:00
Sahil Batra f2a3236b42 actions: Use transaction.atomic for do_remove_realm_emoji. 2022-07-31 18:32:28 -07:00
Sahil Batra 9d9252ef08 realm: Create RealmAuditLog entries when removing custom emoji.
We also add acting_user parameter to do_remove_realm_emoji in
this commit, so that we can use it while creating RealmAuditLog
objects.
2022-07-31 18:32:28 -07:00
Sahil Batra 90898cc9c4 realm: Create RealmAuditLog entries when adding custom emoji. 2022-07-31 18:32:28 -07:00
Sahil Batra 1307976d34 actions: Pass realm_emoji dict to notify_realm_emoji.
We pass the realm_emoji dict to notify_realm_emoji instead
of computing it in notify_realm_emoji itself, because its
caller functions will require the dict in further commits
for creating RealmAuditLog entries.
2022-07-31 18:32:28 -07:00
Sahil Batra e64879e68a actions: Use transaction.atomic for do_remove_linkifier. 2022-07-31 18:32:28 -07:00
Sahil Batra fd6dc754d9 actions: Use transaction.atomic for do_update_linkifier. 2022-07-31 18:32:28 -07:00
Sahil Batra 34b65b9942 actions: Use transaction.atomic for do_add_linkifier.
This commit also renames the legacy event variable to
legacy_event from event because same event is sent
twice if we keep only single variable and change its
value. This was happening because the event value was
changed when the first send_event was actually executed.
2022-07-31 18:32:28 -07:00