Commit Graph

19049 Commits

Author SHA1 Message Date
Lauryn Menard 517b15dc8d help: Create "Configure send message keys" article.
Document setting the keys that will send a message or create a new
line when composting a message via the compose box and via the
personal settings overlay.

Updates links and redirect from former help center article about
this feature.

Fixes #31620.
2024-10-10 14:33:38 -07:00
Tim Abbott 3001f59d00 api docs: Clarify text about deactivated users and groups.
This should make it easier for client developers to understand
how to correctly implement the updated protocol.
2024-10-10 11:37:44 -07:00
Sahil Batra 4784c71bf9 user_groups: Do not allow updating memberships of deactivated users.
This commit updates backend code to not allow adding deactivated
users to groups including when creating groups and also to not
allow removing deactivated users from groups.
2024-10-10 11:37:44 -07:00
Sahil Batra 320081ccd6 mention: Do not include deactivated users in group mention data.
There is no behavioral changes to deactivated users as we do
not create UserMessage rows or call the notification code path
for deactivated users in a user group mention. But it is better
to not include the deactivated users in fields like
"mention_user_ids", so this commit updates the code to not
include deactivated users in the computed mention data.
2024-10-10 11:37:44 -07:00
Sahil Batra 0b58820294 user_groups: Do not include deactivated users in anonymous group settings.
This commit updates code to not include deactivated users in the
anonymous group settings data sent to clients, where the setting
value is sent as a dict containing members and subgroups of the
anonymous group.
2024-10-10 11:37:44 -07:00
Sahil Batra 9292ad8186 user_groups: Do not include deactivated users in members list.
This commit updates code to not include deactivated users in
members list in the user groups object sent in "/register"
and "GET /user_groups" response and also in the response
returned by endpoint like "GET /user_groups/{group_id}/members".

The events code is also update to handle this -
- We expect clients to update the members list on receiving
"realm_user/update" event on deactivation. But for guests
who cannot access the user, "user_group/remove_members"
event is sent to update the group members list on deactivation.
- "user_group/add_members" event is sent to all the users on
reactivating the user.
2024-10-10 11:37:44 -07:00
Sahil Batra b5732b90d6 create_user: Do not send reactivation event for inaccessible users. 2024-10-10 11:37:44 -07:00
Sahil Batra a7c534b1b8 events: Fix updating subscribers list when deactivating user.
We previously did not update the subscribers list for unsubscribed
and never subscribed streams when a user is deactivated or a
guest user loses access to some user.
2024-10-10 11:37:44 -07:00
Lauryn Menard 229212106f api-docs: Centralize history of realm message edit/move settings.
For the six realm settings mentioned in the main description of
the /api/update-message endpoint, link back to that page in the
/api/register-queue and /api/get-events endpoints. This way we
can maintain a centralized point of documentation for how these
settings work for message content edits and moving messages.

The descriptions in the events and register pages focuses on the
specifics for each realm setting, e.g., when a value is added or
changed for a particular realm setting.
2024-10-10 11:28:58 -07:00
Kislay Udbhav Verma a787c7ff80 topic: Add a first-time explanation for "Resolve topic".
We show a confirmation dialog explaining the "resolve topics"
feature when the user marks a topic resolved for the first time.
If the user confirms the action, we mark the
topic resolved, else we don't.

We don't show anything the first time a topic is marked
unresolved.

Fixes #31242
2024-10-09 18:12:55 -07:00
Mateusz Mandera 6c069f4365 api: Improve handling of delivery_email in the GET /users/{email} API.
Limiting lookups by delivery_email to users with "everyone" email
visibility is overly simplistic. We can successfully do these lookups
whenever the requester has the permission to view the real email address
of the user they're looking up.
2024-10-08 18:01:49 -07:00
Mateusz Mandera 8e51442043 users: Add ROLE_TO_ACCESSIBLE_EMAIL_ADDRESS_VISIBILITY_IDS dict.
This is helpful for taking an "acting user" and getting the list of
email_address_visibility values such that the UserProfiles with those
values of the setting permit the acting user to view their
deliver_email.

This can be used for a query "all users whose delivery_email is viewable
by <requester>" in an upcoming commit.

The added code is ugly, but at least it lets us simplify some similarly
ugly logic in can_access_delivery_email.
2024-10-08 18:01:49 -07:00
Tim Abbott ba8801b07f import: Fix incorrect RealmEmoji query.
The name field is not present in exports from the S3 backend.
2024-10-08 17:43:18 -07:00
Tim Abbott 4caa396f49 import_realm: Do not check acting_user for Zulip updates.
The question is whether this was previously exported, not whether it
was previously exported without a traceable acting user.
2024-10-08 17:43:18 -07:00
Tim Abbott 4451db08c3 api docs: Clean up some details around group settings. 2024-10-08 12:18:13 -07:00
Sahil Batra 6d0d1a0700 user_groups: Check can_join_group setting when user tries to join.
Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Sahil Batra 1033230b52 user_groups: Include "can_join_group" field in user group objects.
Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Sahil Batra e1d7f57da7 user_groups: Add can_join_group setting for user group.
This field will be used to control permission for who can
join a user group.

Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Tim Abbott c39e86504a groups: Rework documentation of permitted values.
Ths hardcoded documentation of which values are possible was destined
to end up inaccurate and out-of-date; and meanwhile, we do have a part
of the API that already has these data in machine-readable format.
2024-10-07 17:20:04 -07:00
Prakhar Pratyush 5c23a3f186 reaction: Use the generic 'event_recipient_ids_for_action_on_messages'.
This commit updates the 'notify_reaction_update' function to use
the generic 'event_recipient_ids_for_action_on_messages' function.

It helps to add hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, it has minimal user impact.
2024-10-07 11:35:06 -07:00
Prakhar Pratyush c3f2615e20 submessage: Fix recipients of "submessage" event.
Earlier, submessage was not live-updated for users who joined
the stream after the message was sent.

This commit fixes that bug.

Also, now we use 'event_recipient_ids_for_action_on_messages'.
It helps to add hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, it has minimal user impact.
2024-10-07 11:35:06 -07:00
Prakhar Pratyush d6c48b7185 delete_message: Fix recipients of "delete_message" event.
Earlier, we were sending 'delete_message' event to all active
subscribers of the stream.

We shouldn't send event to those users who don't have access
to the deleted message in a private stream with protected history.

This commit fixes that bug.

Also, now we use 'event_recipient_ids_for_action_on_messages'.
It helps to add hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, it has minimal user impact.
2024-10-07 11:35:06 -07:00
Prakhar Pratyush 388464fcf4 submessage: Add 'durable=True' to the outermost transaction. 2024-10-07 11:35:06 -07:00
Aman Agrawal 3f726e25e4 message_fetch: Add message_ids parameter to /messages request.
This allows us to fetch messages for a list of message ids in a
single request.
2024-10-07 11:00:40 -07:00
Aman Agrawal c16459ca3c test_openapi: Remove use_first_unread_anchor from curl examples. 2024-10-07 11:00:40 -07:00
Aman Agrawal 57589bcffa message_fetch: Rename message_ids variable for clarity.
Added `result_` prefix to differentiate it from upcoming `message_ids`
parameter to the API request. Also, this is final `message_ids` that
we will fetch the messages for. So, a `result` prefix makes sense here.
2024-10-07 11:00:40 -07:00
Alex Vandiver 4b4b6c5ebe scheduled_messages: Remove separate logfile.
This is all captured in `events_deliver_scheduled_messages.log` (note
the leading `events_`) via supervisor.
2024-10-04 14:22:37 -07:00
Prakhar Pratyush cf879a5f48 realm_export: Add a RealmAuditLog entry for delete operation.
This commit adds a RealmAuditLog entry for when someone deletes
an export.

This helps to track the acting_user.
2024-10-04 14:17:20 -07:00
Prakhar Pratyush 07dcee36b2 export_realm: Add RealmExport model.
Earlier, we used to store the key data related to realm exports
in RealmAuditLog. This commit adds a separate table to store
those data.

It includes the code to migrate the concerned existing data in
RealmAuditLog to RealmExport.

Fixes part of #31201.
2024-10-04 12:06:35 -07:00
Prakhar Pratyush 5d9eb4e358 realm_export: Save stats in '.json' format instead of '.txt'.
This commit updates code to store the realm export stats in
json format instead of plain text.

This will help in storing the stats as JsonField in RealmExport table.
2024-10-04 12:06:35 -07:00
Shubham Padia 0634f75582 settings: Rearrange group settings in alphabetical order. 2024-10-04 11:15:01 -07:00
Anders Kaseorg 1b4e02c5d0 thumbnail: Remove type: ignore.
(An alternate solution is message_classes: list[type[Message |
ArchivedMessage]].)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-04 13:54:14 -04:00
Alex Vandiver 912c1b5984 thumbnail: Tighten and clarify the "type: ignore" limitation. 2024-10-04 09:10:14 -07:00
Alex Vandiver 3cbbf2307b thumbnail: Only lock the message row, not the Attachment row.
This prevents a deadlock between the thumbnailing worker and message
sending, as follows:

1. A user uploads an image, making Attachment and ImageAttachment
   rows, as well as enqueuing a job in the thumbnailing queue.

2. Message sending starts a transaction, creates the Message row,
   and calls `do_claim_attachments`, which edits the Attachment row
   of the upload (implicitly locking it).

3. The thumbnailing worker starts a transaction, locks the
   ImageAttachment row for its image, thumbnails it, and then
   attempts to `select_for_update()` the message objects (joined to
   the Attachments table) to find the ones which link to the
   attachment in question. This query blocks, since "a locking
   clause without a table list affects all tables used in the
   statement"[^1] and the message-send request already has a write
   lock on the Attachments row in question.

4. The message-send request attempts to re-fetch the ImageAttachment
   row inside the transaction, which tries to pull a lock on it.

5. Deadlock, because the message-send request has the Attachment
   lock, and waits for the ImageAttachment lock; the thumbnailing
   worker has the ImageAttachment lock, and waits for the Attachment
   lock.

We break this deadlock by limiting the
`update_message_rendered_content` `select_for_update` to only take
the lock on the Message table, and not also the Attachments table --
no changes will be made to the Attachments, so no lock is necessary
there. This allows the thumbnailing worker to successfully pull the
empty list of messages (since the message-send request has not
commits its transaction, and thus the Message row is not visible
yet), and release its ImageAttachment lock so that the message-send
request can proceed.

[^1]: https://www.postgresql.org/docs/current/sql-select.html#SQL-FOR-UPDATE-SHARE
2024-10-04 09:10:14 -07:00
Pierre Carru 5cbe3203f5
slack_incoming: add ok=true to json in case of success.
This better simulates the Slack API, which is important, since some
integrations check this response and decide whether the Slack endpoint
is working based on what they receive.
2024-10-04 08:42:27 -07:00
Alex Vandiver ed8058d060 message: Do not differentiate topics by case when aggregating. 2024-10-03 16:35:45 -07:00
Alex Vandiver b4b1551b81 tusd: Do not delete the .info files.
These files are necessary for the protocol to verify that the file
upload was completed successfully.  Rather than delete them, we update
their StorageClass if it is non-STANDARD.
2024-10-02 13:21:04 -07:00
Tim Abbott 1ff14fd0f1 analytics: Pass subgroup=None to improve indexing.
Because the main indexes on end_time either don't include realm_id or
do include subgroup, passing an explicit subgroup=None for
single-realm queries to read CountStats that don't use the subgroups
feature greatly improves the query plans.
2024-10-02 14:11:44 -04:00
Shubham Padia b4d377a2fa settings: Test named and anonymous groups for org level group settings. 2024-10-01 17:35:14 -07:00
Shubham Padia 12ebd97f1f settings: Add group_creator as default for can_manage_group.
We create an unnamed user group with just the group creator as it's
member when trying to set the default. The pattern I've followed across
most of the acting_user additions is to just put the user declared
somewhere before the check_add_user_group and see if the test passes.
If it does not, then I'll look at what kind of user it needs to be set
to `acting_user`.
2024-10-01 17:35:14 -07:00
Shubham Padia 91edf59873 settings: Rename can_edit_all_user_groups to use `manage`.
Having both `manage` and `edit` terminologies was confusing, so
this commit ensures that we use `manage` wherever applicable.
2024-10-01 17:35:14 -07:00
Shubham Padia 4d06626b6f realm: Remove user_group_edit_policy from the backend.
Fixed #25929.
2024-10-01 17:35:14 -07:00
Shubham Padia 6e9d56eaf4 user_groups: Check permission to manage groups based on group setting.
We also add exception for the group creator to manage groups. See
https://chat.zulip.org/#narrow/stream/3-backend/topic/Group.20creation.20-.20who.20can.20change.20the.20setting.2E/near/1943861
for more details. For the tests, wherever possible, we've just added an
acting_user when creating a group to test.
We've also added an acting_user argument to create_user_group_for_test.
We will not remove `user_group_edit_policy` yet. That will be removed
once we have introduced this setting to the frontend.
2024-10-01 17:35:14 -07:00
Shubham Padia 91953eca28 realm: Only owners should be able to configure `can_manage_all_groups`. 2024-10-01 17:35:14 -07:00
Shubham Padia 2b6414acfb settings: Add can_manage_all_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.
Only owners can modify this setting, but we will add that logic in a
later commit in order to keep changes in this commit minimal.
Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
Although, most lists where we insert this setting follow an unofficial
alphabetical order, `can_manage_all_groups` has been bunched together
with `can_create_groups` since keeping those similar settings together
would be nicer when checking any code related to creating/managing a
user group.
2024-10-01 17:35:14 -07:00
Shubham Padia 16389a2ab5 user_groups: Check permission to create groups based on group setting.
We will not remove `user_group_edit_policy` yet. That will be removed
once we have introduced a user group setting to manage edit permissions
to groups.
2024-10-01 17:35:14 -07:00
Shubham Padia 4a1b67804f realm: Default can_create_groups to moderators for education realms. 2024-10-01 17:35:14 -07:00
Shubham Padia d7d03628d0 realm: Only owners should be able to configure `can_create_groups`.
We might introduce a generic testing function similar to
do_test_changing_settings_by_owners_only later, but not right now, since
there is only 1 setting at the moment needing that test.
2024-10-01 17:35:14 -07:00
Shubham Padia fc46673f23 settings: Add can_create_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.

Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
2024-10-01 17:35:14 -07:00
Lauryn Menard 4283207f6f help-relative: Add option for "Not subscribed" tab of channels overlay.
Updates the relative help link for "Browse and subscribe to channels"
in help center "Introduction to channels" article.

Fixes #31605.
2024-10-01 10:28:30 -07:00