Commit Graph

8733 Commits

Author SHA1 Message Date
Anders Kaseorg 08db41660a python: Avoid deprecated cgi module, removed in Python 3.13.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-22 10:05:01 -07:00
Anders Kaseorg 3ec58fd3d5 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-20 18:16:27 -07:00
Anders Kaseorg 10271fb850 ruff: Fix SIM910 Use `.get(key)` instead of `.get(key, None)`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-10-20 18:16:27 -07:00
Prakhar Pratyush eaee5763d6 realm_export: Add realm_export_consent feature to API.
Fixes part of #31201.
2024-10-18 14:08:20 -07:00
PieterCK c9375fb5ee storage: Hash system bots avatar files.
This commit makes sure system bots avatar files are hashed when served
as static files. This way, requests for these avatar files will be
served with long-lived caching headers by our nginx (see #22275).

We don't need to worry about stale caches for these files because they
will only be used by system bots.

Fixes #31458.
2024-10-17 15:47:40 -07:00
Tim Abbott 46db52dc96 avatar: Use fixed avatars for system bots.
This makes a Zulip server more isolated than relying on gravatar, and
avoids complex logistics if in the future we move system bots to live
inside individual realms.

Co-authored-by: PieterCK <pieterceka123@gmail.com>
2024-10-17 15:47:17 -07:00
Sahil Batra e5043b991a user_groups: Add API support to add subgroups during group creation.
This commit adds support to add subgroups to a group while
creating it.

User can add the subgroups to group irrespective of permissions
like user can add members during creating it.
2024-10-17 14:27:21 -07:00
Sahil Batra 47a611f989 user_groups: Check permission when adding subgroups.
This commit updates code to allow users with permission
to add members to add subgroups as well. And only users
with permission to manage the group can remove subgroups.

Also updated tests to check permissions in separate tests
and removed them from the existing test.
2024-10-17 14:27:21 -07:00
Vector73 1be0cb1b75 settings: Add `can_move_messages_between_channels_group` realm setting.
Added `can_move_messages_between_channels_group` realm setting to replace
`move_messages_between_streams_policy`.
2024-10-16 14:26:18 -07:00
Sahil Batra f24f1bfd14 user_groups: Refactor code to check permission for updating groups.
Users with permission to manage the group have all the permissions
including joining/leaving the group, adding others group which also
have a separate setting to control them.

So, it makes sense to just check managing permissions first in
access_user_group_for_update and then check the specific permission.
There is no behavioral change in this commit, it only changes the
order of checking permissions.
2024-10-16 09:40:07 -07:00
Tim Abbott 7e7113ad84 groups: Enable group-settings value on groups in production.
The main change is redefining ALLOW_GROUP_VALUED_SETTINGS to not
control code, but instead to instead control the configuration for
whether settings that have not been converted to use our modern UI
patterns should require system groups.

Fundamentally, it's the same for the realm/stream group-valued
settings, which don't have the new UI patterns yet.

We remove the visual hiding of the "can manage group" setting, which
was hidden only due to transitions being incomplete.
2024-10-15 15:58:54 -07:00
Shubham Padia 060156fca4 user_groups: Add can_leave_group setting for user group.
This field will be used to control permission for who can
leave a user group.
2024-10-14 11:44:27 -07:00
Vector73 f733ab112c settings: Add `can_add_custom_emoji_group` realm setting.
Added `can_add_custom_emoji_group` setting to replace `add_custom_emoji_policy`.
2024-10-13 16:09:04 -07:00
Prakhar Pratyush 3314c89288 realm: Add maximum file size upload restriction.
This commit adds a restriction to the maximum file size
that can be uploaded to a realm based on its plan_type.
2024-10-11 17:16:48 -07:00
Prakhar Pratyush 808acc9e47 events: Migrate plan_type & upload_quota to update_dict event format.
'realm_upload_quota_mib` is updated when `plan_type` changes.

Earlier, we were including 'upload_quota' to update
`realm_upload_quota_mib` in extra_data field of 'realm op: update'
event format when property='plan_type'.

This commit migrate those two parameters to `realm op: update_dict`
event format.

* None of the clients processes these fields, so no compatibility
  code required.
* Renamed `upload_quota` to `upload_quota_mib` as it better aligns
  with our goal to encode units in the client-facing API names.
  Also, it helps to avoid extra code to update 'realm_upload_quota_mib`
  in web client, web client simply aligns with
  'realm["realm_" + key] = value'.
2024-10-11 17:16:48 -07:00
Lauryn Menard 70ab893d34 urls: Generate narrow links in backend with "channel" operator. 2024-10-11 17:00:23 -07:00
Shubham Padia c9d5276031 user_groups: Set can_manage_all_groups to administrator group.
Earlier we use to restrict admins, moderators or members of a group to
manage that group if they were part of the realm wide
`can_manage_all_groups`. We will not do that anymore and even
non-members of a group regardless of role can manage a group if they are
part of `can_manage_all_groups`.

See
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952902
to check more about the migration plan for which this is the last step.
2024-10-11 16:31:18 -07:00
Shubham Padia f134662312 user_groups: Check can_add_members_group before adding members.
Removing members will be controlled by `can_manage_group` until we add
`can_remove_members_group` in the future.

Users with permission to manage a group can add members to that group by
default without being present in `can_add_members_group`.
2024-10-11 16:31:18 -07:00
Shubham Padia b305ca14dd user_groups: Add add_can_members_group to user group.
The default value for this field that we wanted to have was that group
itlself. But we are deferring that to later in order to reach the point
of switching over to the groups system sooner. Till then, we will use
`group_creator` as the default. See
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952904
for more details.

For migration plan details, see
https://chat.zulip.org/#narrow/stream/101-design/topic/Group.20add.20members.20dropdown/near/1952902

The increase in query count from 7 to 9 in the query count test for
creating a user group is because of group_creator being the default for
the new field.
2024-10-11 16:31:18 -07:00
Prakhar Pratyush 55f97cd06f realm_export: Add support to create full data export via /export/realm.
Earlier, only public data export was possible via `POST /export/realm`
endpoint. This commit adds support to create full data export with
member consent via that endpoint.

Also, this adds a 'export_type' parameter to the dictionaries
in `realm_export` event type and `GET /export/realm` response.

Fixes part of #31201.
2024-10-11 13:20:42 -07:00
Lauryn Menard 517b15dc8d help: Create "Configure send message keys" article.
Document setting the keys that will send a message or create a new
line when composting a message via the compose box and via the
personal settings overlay.

Updates links and redirect from former help center article about
this feature.

Fixes #31620.
2024-10-10 14:33:38 -07:00
Sahil Batra 4784c71bf9 user_groups: Do not allow updating memberships of deactivated users.
This commit updates backend code to not allow adding deactivated
users to groups including when creating groups and also to not
allow removing deactivated users from groups.
2024-10-10 11:37:44 -07:00
Sahil Batra 320081ccd6 mention: Do not include deactivated users in group mention data.
There is no behavioral changes to deactivated users as we do
not create UserMessage rows or call the notification code path
for deactivated users in a user group mention. But it is better
to not include the deactivated users in fields like
"mention_user_ids", so this commit updates the code to not
include deactivated users in the computed mention data.
2024-10-10 11:37:44 -07:00
Sahil Batra 0b58820294 user_groups: Do not include deactivated users in anonymous group settings.
This commit updates code to not include deactivated users in the
anonymous group settings data sent to clients, where the setting
value is sent as a dict containing members and subgroups of the
anonymous group.
2024-10-10 11:37:44 -07:00
Sahil Batra 9292ad8186 user_groups: Do not include deactivated users in members list.
This commit updates code to not include deactivated users in
members list in the user groups object sent in "/register"
and "GET /user_groups" response and also in the response
returned by endpoint like "GET /user_groups/{group_id}/members".

The events code is also update to handle this -
- We expect clients to update the members list on receiving
"realm_user/update" event on deactivation. But for guests
who cannot access the user, "user_group/remove_members"
event is sent to update the group members list on deactivation.
- "user_group/add_members" event is sent to all the users on
reactivating the user.
2024-10-10 11:37:44 -07:00
Sahil Batra a7c534b1b8 events: Fix updating subscribers list when deactivating user.
We previously did not update the subscribers list for unsubscribed
and never subscribed streams when a user is deactivated or a
guest user loses access to some user.
2024-10-10 11:37:44 -07:00
Kislay Udbhav Verma a787c7ff80 topic: Add a first-time explanation for "Resolve topic".
We show a confirmation dialog explaining the "resolve topics"
feature when the user marks a topic resolved for the first time.
If the user confirms the action, we mark the
topic resolved, else we don't.

We don't show anything the first time a topic is marked
unresolved.

Fixes #31242
2024-10-09 18:12:55 -07:00
Mateusz Mandera 6c069f4365 api: Improve handling of delivery_email in the GET /users/{email} API.
Limiting lookups by delivery_email to users with "everyone" email
visibility is overly simplistic. We can successfully do these lookups
whenever the requester has the permission to view the real email address
of the user they're looking up.
2024-10-08 18:01:49 -07:00
Mateusz Mandera 8e51442043 users: Add ROLE_TO_ACCESSIBLE_EMAIL_ADDRESS_VISIBILITY_IDS dict.
This is helpful for taking an "acting user" and getting the list of
email_address_visibility values such that the UserProfiles with those
values of the setting permit the acting user to view their
deliver_email.

This can be used for a query "all users whose delivery_email is viewable
by <requester>" in an upcoming commit.

The added code is ugly, but at least it lets us simplify some similarly
ugly logic in can_access_delivery_email.
2024-10-08 18:01:49 -07:00
Tim Abbott ba8801b07f import: Fix incorrect RealmEmoji query.
The name field is not present in exports from the S3 backend.
2024-10-08 17:43:18 -07:00
Tim Abbott 4caa396f49 import_realm: Do not check acting_user for Zulip updates.
The question is whether this was previously exported, not whether it
was previously exported without a traceable acting user.
2024-10-08 17:43:18 -07:00
Sahil Batra 1033230b52 user_groups: Include "can_join_group" field in user group objects.
Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Sahil Batra e1d7f57da7 user_groups: Add can_join_group setting for user group.
This field will be used to control permission for who can
join a user group.

Fixes part of #25938.
2024-10-08 12:18:13 -07:00
Prakhar Pratyush d6c48b7185 delete_message: Fix recipients of "delete_message" event.
Earlier, we were sending 'delete_message' event to all active
subscribers of the stream.

We shouldn't send event to those users who don't have access
to the deleted message in a private stream with protected history.

This commit fixes that bug.

Also, now we use 'event_recipient_ids_for_action_on_messages'.
It helps to add hardening such that if the invariant "no usermessage
row corresponding to a message exists if the user loses access to the
message" is violated due to some bug, it has minimal user impact.
2024-10-07 11:35:06 -07:00
Aman Agrawal 3f726e25e4 message_fetch: Add message_ids parameter to /messages request.
This allows us to fetch messages for a list of message ids in a
single request.
2024-10-07 11:00:40 -07:00
Prakhar Pratyush 07dcee36b2 export_realm: Add RealmExport model.
Earlier, we used to store the key data related to realm exports
in RealmAuditLog. This commit adds a separate table to store
those data.

It includes the code to migrate the concerned existing data in
RealmAuditLog to RealmExport.

Fixes part of #31201.
2024-10-04 12:06:35 -07:00
Prakhar Pratyush 5d9eb4e358 realm_export: Save stats in '.json' format instead of '.txt'.
This commit updates code to store the realm export stats in
json format instead of plain text.

This will help in storing the stats as JsonField in RealmExport table.
2024-10-04 12:06:35 -07:00
Shubham Padia 0634f75582 settings: Rearrange group settings in alphabetical order. 2024-10-04 11:15:01 -07:00
Alex Vandiver ed8058d060 message: Do not differentiate topics by case when aggregating. 2024-10-03 16:35:45 -07:00
Tim Abbott 1ff14fd0f1 analytics: Pass subgroup=None to improve indexing.
Because the main indexes on end_time either don't include realm_id or
do include subgroup, passing an explicit subgroup=None for
single-realm queries to read CountStats that don't use the subgroups
feature greatly improves the query plans.
2024-10-02 14:11:44 -04:00
Shubham Padia 12ebd97f1f settings: Add group_creator as default for can_manage_group.
We create an unnamed user group with just the group creator as it's
member when trying to set the default. The pattern I've followed across
most of the acting_user additions is to just put the user declared
somewhere before the check_add_user_group and see if the test passes.
If it does not, then I'll look at what kind of user it needs to be set
to `acting_user`.
2024-10-01 17:35:14 -07:00
Shubham Padia 91edf59873 settings: Rename can_edit_all_user_groups to use `manage`.
Having both `manage` and `edit` terminologies was confusing, so
this commit ensures that we use `manage` wherever applicable.
2024-10-01 17:35:14 -07:00
Shubham Padia 6e9d56eaf4 user_groups: Check permission to manage groups based on group setting.
We also add exception for the group creator to manage groups. See
https://chat.zulip.org/#narrow/stream/3-backend/topic/Group.20creation.20-.20who.20can.20change.20the.20setting.2E/near/1943861
for more details. For the tests, wherever possible, we've just added an
acting_user when creating a group to test.
We've also added an acting_user argument to create_user_group_for_test.
We will not remove `user_group_edit_policy` yet. That will be removed
once we have introduced this setting to the frontend.
2024-10-01 17:35:14 -07:00
Shubham Padia 2b6414acfb settings: Add can_manage_all_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.
Only owners can modify this setting, but we will add that logic in a
later commit in order to keep changes in this commit minimal.
Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
Although, most lists where we insert this setting follow an unofficial
alphabetical order, `can_manage_all_groups` has been bunched together
with `can_create_groups` since keeping those similar settings together
would be nicer when checking any code related to creating/managing a
user group.
2024-10-01 17:35:14 -07:00
Shubham Padia fc46673f23 settings: Add can_create_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.

Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
2024-10-01 17:35:14 -07:00
Lauryn Menard 4283207f6f help-relative: Add option for "Not subscribed" tab of channels overlay.
Updates the relative help link for "Browse and subscribe to channels"
in help center "Introduction to channels" article.

Fixes #31605.
2024-10-01 10:28:30 -07:00
Sahil Batra b8a039ee99 user_groups: Include settings and supergroups in error response.
The error response when a user group cannot be deactivated due
to it being used as a subgroup or for a setting includes details
about the supergroups, streams, user groups as well the settings
for which it is used.
2024-10-01 09:45:33 -07:00
Sahil Batra c1973d2263 user_groups: Add access_user_group_to_read_membership.
This commit adds access_user_group_to_read_membership function
so that we can avoid calling get_user_group_by_id_in_realm with
"for_read=True" from views functions, which is better for security
since that function does not do any access checks.
2024-10-01 09:36:01 -07:00
Tim Abbott 096fea48a0 user_groups: Add docstrings for several functions. 2024-09-30 18:42:14 -07:00
Sahil Batra b554106e6d user_groups: Allow members of subgroups to manage group.
Previously, if the user_group_edit_policy was set to allow
members or full members to manage the group, the user had
to be the direct member of the group being managed.

This commit updates the code to allow members of the subgroups
as well to manage the group as technically members of the
subgroups are member of the group.

This also improves the code to not fetch all the group members
to check this, and instead directly call is_user_in_group
which uses "exists" to check it.
2024-09-30 18:42:14 -07:00