Commit Graph

8262 Commits

Author SHA1 Message Date
Sanchit Sharma 795b2ba14e do_deactivate_stream: Remove unnecessary mutations.
Streams should not be marked as private, and subscribers
of the deactivated stream should not be removed.

Update the confirmation message when archiving a stream.
2024-10-25 16:06:42 -07:00
sanchi-t af7ebde9e4 subscription: Include archived channels in streams list.
`is_archived` field is added to the stream and types.

Include a new `archived_channeels` client capability, to allow clients
to access data on archived channels, without breaking
backwards-compatibility for existing clients that don't know how to
handle these.

Also, included `exclude_archived` parameter to `/get-streams`,
which defaults to `true` as basic clients may not be interested
in archived streams.
2024-10-25 16:06:40 -07:00
sanchi-t c6fc25e5df do_deactivate_stream: Do not rename streams during archiving.
Functions related to archived streams are also updated.
2024-10-25 15:56:04 -07:00
Shubham Padia fe9952eab5 groups: Rename test to state that it checks allow_system_group.
We also remove the default group check which is already being checked in
do_set_realm_permission_group_setting_to_anonymous_groups_test.
2024-10-24 14:42:21 -07:00
Shubham Padia 1f0906aef7 groups: Remove list specifying group settings with new API format.
All the groups are now following the new API format, making the list
redundant.
2024-10-24 14:42:21 -07:00
Shubham Padia 9f9d5b2f98 groups: Accept anonymous groups for can_access_all_users_group.
On the frontend, the selection is still a dropdown of system groups but
on the API level, we have started accepting anonymous groups similar to
other settings
We've kept require system groups true for now until we switch to group
picker on the frontend.
2024-10-24 14:42:19 -07:00
Shubham Padia ceb0197c1b groups: Accept anonymous groups for create_multiuse_invite_group.
On the frontend, the selection is still a dropdown of system groups but
on the API level, we have started accepting anonymous groups similar to
other settings.
We've kept require system groups true for now until we switch to group
picker on the frontend.
2024-10-24 14:41:09 -07:00
Mateusz Mandera 481ef24b53 auth: Tweak invalid credentials error message.
We decided this is the better string to use.
2024-10-24 14:35:31 -07:00
Mateusz Mandera da4443f392 thumbnail: Make thumbnailing work with data import.
We didn't have thumbnailing for images coming from data import and this
commit adds the functionality.

There are a few fundamental issues that the implementation needs to
solve.

1. The images come from an untrusted source and therefore we don't want
   to just pass them through to thumbnailing without checking. For that
   reason, we cannot just import ImageAttachment rows from the export
   data, even for zulip=>zulip imports.
   The right way to process images is to pass them to maybe_thumbail(),
   which runs libvips_check_image() on them to verify we're okay with
   thumbnailing, creates ImageAttachment rows for them and sends them
   to the thumbnailing queue worker. This approach lets us handle both
   zulip=>zulip and 3rd party=>zulip imports in the same way,

2. There is a somewhat circular dependency between the Message,
   Attachment and ImageAttachment import process:

- ImageAttachments would ideally be created after importing
  Attachments, but they need to already exist at the time of Message
  import. Otherwise, the markdown processor doesn't know it has to add
  HTML for image previews to messages that reference images. This would
  mean that messages imported from 3rd party tools don't get image
  previews.
- Attachments only get created after Message import however, due to the
  many-to-many relationship between Message and Attachment.

This is solved by fixing up some data of Attachments pre-emptively, such
as the path_ids. This gives us the necessary information for creating
ImageAttachments before importing Messages.

While we generate ImageAttachment rows synchronously, the actual
thumbnailing job is sent to the queue worker. Theoretically, the worker
could be very backlogged and not process the thumbnails anytime soon.
This is fine - if the app is loaded and tries to display a message with
such a not-yet-generated thumbnail, the code in `serve_file` will
generate the thumbnails synchronously on the fly and the user will see
the image preview displayed normally. See:

1b47134d0d/zerver/views/upload.py (L333-L342)
2024-10-24 10:32:51 -07:00
Mateusz Mandera a6b0385229 tests: Extract upload_image helpers from test_markdown_thumbnail.
These are pretty general and can be useful utils for other tests.
2024-10-24 10:32:51 -07:00
Mateusz Mandera af9b44ed02 auth: Fix invalid credentials message in login form.
Email is not case-sensitive. And password is obviously case-sensitive,
so no point mentioning that.
2024-10-24 08:23:16 -07:00
PieterCK 068ab6e11e avatar: Add checks to make sure system bot avatar exists.
This commit introduces an assertion to verify that the avatar file for
system bots exists and findable.

In development, it'll assert that the avatar file exists in the static
directory. This isn't done in production environment to avoid
unnecessary overhead. It helps verify that the protocol to fetch system
bot avatars still works when making changes during development.

In production it'll check if the avatar file exists in the STATIC_ROOT
and return a default avatar png if it doesn't.
2024-10-23 10:35:42 -07:00
PieterCK 516d1ab82b avatar: Ensure system bots' avatar URLs follow convention.
Previously, requesting system bots URLs did not return any -medium.png
variants and SVG file was also used for notification bots' avatar, which
was problematic.

In this commit, the -medium.png variants is added for the avatars of
system bots and zulip-icon-square.svg is also converted into
notification-bot.png for the notification bot. The get_avatar_url method
has been updated to return the "medium" file variants for the system
bots.

Additionally, the system bots' avatar files is moved to a dedicated
directory to simplify the hashing logic for these files. Now, all files
in the "images/static_avatars/" directory will be hashed.
2024-10-23 10:35:42 -07:00
Mateusz Mandera 77e7a2d30f users: Add API endpoint to update_user_backend by real email.
The old endpoint for updating a user worked only via user id. Now we add
a different entry to this functionality, fetching the user by
.delivery_email.

update_user_backend becomes the main function handling all the logic,
invoked by the two endpoints.
2024-10-22 16:36:38 -07:00
Mateusz Mandera 389b851f81 update_user_backend: Allow authorized org owners to change user emails.
This adds a new special UserProfile flag can_change_user_emails(disabled
by default) and the ability for changing the email address of users in
the realm via update_user_backend. This is useful for allowing
organizations to update user emails without needing to set up a SCIM
integration, but since it gives the ability to hijack user accounts, it
needs to be behind this additional permission and can't be just given to
organization owners by default. Analogical to how the
create_user_backend endpoint works.
2024-10-22 16:36:38 -07:00
Sahil Batra 0a77f06163 test_user_groups: Fix comments in group creation tests.
One comment did not mention the correct thing that was
being tested and one comment had a typo.
2024-10-22 10:23:36 -07:00
Anders Kaseorg 3ec58fd3d5 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-20 18:16:27 -07:00
Prakhar Pratyush eaee5763d6 realm_export: Add realm_export_consent feature to API.
Fixes part of #31201.
2024-10-18 14:08:20 -07:00
Tim Abbott 46db52dc96 avatar: Use fixed avatars for system bots.
This makes a Zulip server more isolated than relying on gravatar, and
avoids complex logistics if in the future we move system bots to live
inside individual realms.

Co-authored-by: PieterCK <pieterceka123@gmail.com>
2024-10-17 15:47:17 -07:00
Sahil Batra e5043b991a user_groups: Add API support to add subgroups during group creation.
This commit adds support to add subgroups to a group while
creating it.

User can add the subgroups to group irrespective of permissions
like user can add members during creating it.
2024-10-17 14:27:21 -07:00
Sahil Batra 1e818c4708 user_groups: Allow updating subgroups and members using same endpoint.
`POST /user_groups/{user_group_id}/members` now allows updating
subgroups as well.
2024-10-17 14:27:21 -07:00
Sahil Batra 47a611f989 user_groups: Check permission when adding subgroups.
This commit updates code to allow users with permission
to add members to add subgroups as well. And only users
with permission to manage the group can remove subgroups.

Also updated tests to check permissions in separate tests
and removed them from the existing test.
2024-10-17 14:27:21 -07:00
Sahil Batra 30e0c72919 test_user_groups: Remove incorrect comment.
The comment about non-admins and non-moderators who are not
member of the group cannot update subgroups of that group
is not correct since there is no such restriction now after
c9d527603. The test passes because the member user is not
part of can_manage_group or can_manage_all_groups.
2024-10-17 14:27:21 -07:00
Tim Abbott 010410c849 rocketchat: Validate custom emoji before larger data sets.
This is a data set that's relatively likely to have weird failures,
and also likely to be fairly small.
2024-10-17 12:25:18 -07:00
Vector73 8d341d1af9 settings: Remove `move_messages_between_streams_policy` setting.
Removed `move_messages_between_streams_policy` property, as the permission
to move messages between channels is now controlled by
`can_move_messages_between_channels_group` setting.
2024-10-16 14:26:20 -07:00
Vector73 1be0cb1b75 settings: Add `can_move_messages_between_channels_group` realm setting.
Added `can_move_messages_between_channels_group` realm setting to replace
`move_messages_between_streams_policy`.
2024-10-16 14:26:18 -07:00
Sahil Batra 3d65a8f78a user_groups: Fix permissions for joining the group.
We want to allow the user, who can add others to group, to
join the group as well irrespective of can_join_group setting.

Previously, the permission to add others (or say anyone) was
controlled by can_manage_group setting, but now it is controlled
by can_add_members_group setting. This commit fixes the code to
use can_add_members_group setting to check permission for joining
the group.

This commit also improves the tests for checking permission to
join the group such that different settings are tested in isolation.
2024-10-16 09:40:07 -07:00
PieterCK 6289a551aa data_import: Add email validation to third-party data converters.
This commit makes the third-party data converters check for invalid user
emails. If it finds any, it’ll raise an Exception and show an error
message with all the bad emails listed out.

Fixes: #31783
2024-10-15 16:04:43 -07:00
Tim Abbott 7e7113ad84 groups: Enable group-settings value on groups in production.
The main change is redefining ALLOW_GROUP_VALUED_SETTINGS to not
control code, but instead to instead control the configuration for
whether settings that have not been converted to use our modern UI
patterns should require system groups.

Fundamentally, it's the same for the realm/stream group-valued
settings, which don't have the new UI patterns yet.

We remove the visual hiding of the "can manage group" setting, which
was hidden only due to transitions being incomplete.
2024-10-15 15:58:54 -07:00
Tim Abbott b4ae76be65 upload: Improve error messages when uploads limited by plan.
To make the tests work, we ensure that MAX_FILE_UPLOAD_SIZE is
enforced even in the plans case.
2024-10-15 13:30:29 -07:00
Shubham Padia bf46747735 user_groups: Check can_leave_group when removing members. 2024-10-14 11:44:27 -07:00
Vector73 6bc8651d22 settings: Remove `add_custom_emoji_policy` setting.
Removed `add_custom_emoji_policy` setting as the permission to
add custom emoji is now controlled by `can_add_custom_emoji_group`
setting.
2024-10-13 16:09:04 -07:00
Vector73 f733ab112c settings: Add `can_add_custom_emoji_group` realm setting.
Added `can_add_custom_emoji_group` setting to replace `add_custom_emoji_policy`.
2024-10-13 16:09:04 -07:00
Prakhar Pratyush 808acc9e47 events: Migrate plan_type & upload_quota to update_dict event format.
'realm_upload_quota_mib` is updated when `plan_type` changes.

Earlier, we were including 'upload_quota' to update
`realm_upload_quota_mib` in extra_data field of 'realm op: update'
event format when property='plan_type'.

This commit migrate those two parameters to `realm op: update_dict`
event format.

* None of the clients processes these fields, so no compatibility
  code required.
* Renamed `upload_quota` to `upload_quota_mib` as it better aligns
  with our goal to encode units in the client-facing API names.
  Also, it helps to avoid extra code to update 'realm_upload_quota_mib`
  in web client, web client simply aligns with
  'realm["realm_" + key] = value'.
2024-10-11 17:16:48 -07:00
Lauryn Menard 70ab893d34 urls: Generate narrow links in backend with "channel" operator. 2024-10-11 17:00:23 -07:00
Shubham Padia 2b6df48ebb user_groups: Split update members test into add/remove tests. 2024-10-11 16:31:18 -07:00
Shubham Padia c9d5276031 user_groups: Set can_manage_all_groups to administrator group.
Earlier we use to restrict admins, moderators or members of a group to
manage that group if they were part of the realm wide
`can_manage_all_groups`. We will not do that anymore and even
non-members of a group regardless of role can manage a group if they are
part of `can_manage_all_groups`.

See
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952902
to check more about the migration plan for which this is the last step.
2024-10-11 16:31:18 -07:00
Shubham Padia 9bbd6a7316 user_groups: Check can_add_members_group when adding members.
Fixes #25942.
Users with permission to manage the group (either on the group level or
realm level) should be able to add members to the group without being
present in can_add_members_group.
2024-10-11 16:31:18 -07:00
Shubham Padia f134662312 user_groups: Check can_add_members_group before adding members.
Removing members will be controlled by `can_manage_group` until we add
`can_remove_members_group` in the future.

Users with permission to manage a group can add members to that group by
default without being present in `can_add_members_group`.
2024-10-11 16:31:18 -07:00
Shubham Padia b305ca14dd user_groups: Add add_can_members_group to user group.
The default value for this field that we wanted to have was that group
itlself. But we are deferring that to later in order to reach the point
of switching over to the groups system sooner. Till then, we will use
`group_creator` as the default. See
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952904
for more details.

For migration plan details, see
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952902

The increase in query count from 7 to 9 in the query count test for
creating a user group is because of group_creator being the default for
the new field.
2024-10-11 16:31:18 -07:00
Sahil Batra 4784c71bf9 user_groups: Do not allow updating memberships of deactivated users.
This commit updates backend code to not allow adding deactivated
users to groups including when creating groups and also to not
allow removing deactivated users from groups.
2024-10-10 11:37:44 -07:00
Sahil Batra 320081ccd6 mention: Do not include deactivated users in group mention data.
There is no behavioral changes to deactivated users as we do
not create UserMessage rows or call the notification code path
for deactivated users in a user group mention. But it is better
to not include the deactivated users in fields like
"mention_user_ids", so this commit updates the code to not
include deactivated users in the computed mention data.
2024-10-10 11:37:44 -07:00
Sahil Batra 0b58820294 user_groups: Do not include deactivated users in anonymous group settings.
This commit updates code to not include deactivated users in the
anonymous group settings data sent to clients, where the setting
value is sent as a dict containing members and subgroups of the
anonymous group.
2024-10-10 11:37:44 -07:00
Sahil Batra 9292ad8186 user_groups: Do not include deactivated users in members list.
This commit updates code to not include deactivated users in
members list in the user groups object sent in "/register"
and "GET /user_groups" response and also in the response
returned by endpoint like "GET /user_groups/{group_id}/members".

The events code is also update to handle this -
- We expect clients to update the members list on receiving
"realm_user/update" event on deactivation. But for guests
who cannot access the user, "user_group/remove_members"
event is sent to update the group members list on deactivation.
- "user_group/add_members" event is sent to all the users on
reactivating the user.
2024-10-10 11:37:44 -07:00
Sahil Batra b5732b90d6 create_user: Do not send reactivation event for inaccessible users. 2024-10-10 11:37:44 -07:00
Kislay Udbhav Verma a787c7ff80 topic: Add a first-time explanation for "Resolve topic".
We show a confirmation dialog explaining the "resolve topics"
feature when the user marks a topic resolved for the first time.
If the user confirms the action, we mark the
topic resolved, else we don't.

We don't show anything the first time a topic is marked
unresolved.

Fixes #31242
2024-10-09 18:12:55 -07:00
Mateusz Mandera 6c069f4365 api: Improve handling of delivery_email in the GET /users/{email} API.
Limiting lookups by delivery_email to users with "everyone" email
visibility is overly simplistic. We can successfully do these lookups
whenever the requester has the permission to view the real email address
of the user they're looking up.
2024-10-08 18:01:49 -07:00
Sahil Batra 6d0d1a0700 user_groups: Check can_join_group setting when user tries to join.
Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Sahil Batra 1033230b52 user_groups: Include "can_join_group" field in user group objects.
Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Prakhar Pratyush d6c48b7185 delete_message: Fix recipients of "delete_message" event.
Earlier, we were sending 'delete_message' event to all active
subscribers of the stream.

We shouldn't send event to those users who don't have access
to the deleted message in a private stream with protected history.

This commit fixes that bug.

Also, now we use 'event_recipient_ids_for_action_on_messages'.
It helps to add hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, it has minimal user impact.
2024-10-07 11:35:06 -07:00