Commit Graph

8697 Commits

Author SHA1 Message Date
Prakhar Pratyush 5d9eb4e358 realm_export: Save stats in '.json' format instead of '.txt'.
This commit updates code to store the realm export stats in
json format instead of plain text.

This will help in storing the stats as JsonField in RealmExport table.
2024-10-04 12:06:35 -07:00
Shubham Padia 0634f75582 settings: Rearrange group settings in alphabetical order. 2024-10-04 11:15:01 -07:00
Alex Vandiver ed8058d060 message: Do not differentiate topics by case when aggregating. 2024-10-03 16:35:45 -07:00
Tim Abbott 1ff14fd0f1 analytics: Pass subgroup=None to improve indexing.
Because the main indexes on end_time either don't include realm_id or
do include subgroup, passing an explicit subgroup=None for
single-realm queries to read CountStats that don't use the subgroups
feature greatly improves the query plans.
2024-10-02 14:11:44 -04:00
Shubham Padia 12ebd97f1f settings: Add group_creator as default for can_manage_group.
We create an unnamed user group with just the group creator as it's
member when trying to set the default. The pattern I've followed across
most of the acting_user additions is to just put the user declared
somewhere before the check_add_user_group and see if the test passes.
If it does not, then I'll look at what kind of user it needs to be set
to `acting_user`.
2024-10-01 17:35:14 -07:00
Shubham Padia 91edf59873 settings: Rename can_edit_all_user_groups to use `manage`.
Having both `manage` and `edit` terminologies was confusing, so
this commit ensures that we use `manage` wherever applicable.
2024-10-01 17:35:14 -07:00
Shubham Padia 6e9d56eaf4 user_groups: Check permission to manage groups based on group setting.
We also add exception for the group creator to manage groups. See
https://chat.zulip.org/#narrow/stream/3-backend/topic/Group.20creation.20-.20who.20can.20change.20the.20setting.2E/near/1943861
for more details. For the tests, wherever possible, we've just added an
acting_user when creating a group to test.
We've also added an acting_user argument to create_user_group_for_test.
We will not remove `user_group_edit_policy` yet. That will be removed
once we have introduced this setting to the frontend.
2024-10-01 17:35:14 -07:00
Shubham Padia 2b6414acfb settings: Add can_manage_all_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.
Only owners can modify this setting, but we will add that logic in a
later commit in order to keep changes in this commit minimal.
Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
Although, most lists where we insert this setting follow an unofficial
alphabetical order, `can_manage_all_groups` has been bunched together
with `can_create_groups` since keeping those similar settings together
would be nicer when checking any code related to creating/managing a
user group.
2024-10-01 17:35:14 -07:00
Shubham Padia fc46673f23 settings: Add can_create_groups setting to realm.
This commit does not add the logic of using this setting to actually
check the permission on the backend. That will be done in a later
commit.

Adding the setting breaks the frontend, since the frontend tries to find
a dropdown widget for the setting automatically. To avoid this, we've
added a small temporary if statement to `settings_org.js`.
2024-10-01 17:35:14 -07:00
Lauryn Menard 4283207f6f help-relative: Add option for "Not subscribed" tab of channels overlay.
Updates the relative help link for "Browse and subscribe to channels"
in help center "Introduction to channels" article.

Fixes #31605.
2024-10-01 10:28:30 -07:00
Sahil Batra b8a039ee99 user_groups: Include settings and supergroups in error response.
The error response when a user group cannot be deactivated due
to it being used as a subgroup or for a setting includes details
about the supergroups, streams, user groups as well the settings
for which it is used.
2024-10-01 09:45:33 -07:00
Sahil Batra c1973d2263 user_groups: Add access_user_group_to_read_membership.
This commit adds access_user_group_to_read_membership function
so that we can avoid calling get_user_group_by_id_in_realm with
"for_read=True" from views functions, which is better for security
since that function does not do any access checks.
2024-10-01 09:36:01 -07:00
Tim Abbott 096fea48a0 user_groups: Add docstrings for several functions. 2024-09-30 18:42:14 -07:00
Sahil Batra b554106e6d user_groups: Allow members of subgroups to manage group.
Previously, if the user_group_edit_policy was set to allow
members or full members to manage the group, the user had
to be the direct member of the group being managed.

This commit updates the code to allow members of the subgroups
as well to manage the group as technically members of the
subgroups are member of the group.

This also improves the code to not fetch all the group members
to check this, and instead directly call is_user_in_group
which uses "exists" to check it.
2024-09-30 18:42:14 -07:00
Sahil Batra 233775d257 user_groups: Rename has_user_group_access.
This commit renames has_user_group_access function to
has_user_group_access_for_subgroup, since the function
is only used to check access for using a group as subgroup.
2024-09-30 18:42:14 -07:00
Sahil Batra 0c89621fa6 user_groups: Refactor code to check permissions.
This commit refactors the code to check permission for
accessing user group in such a way that we can avoid
duplicate code in future when we will have different
settings controlling the permissions for editing group
details and settings, joining the group, adding others
to group, etc.
2024-09-30 18:42:14 -07:00
Mateusz Mandera 06fa99e87c do_change_user_delivery_email: Add acting_user kwarg.
This is standard for our do_change_... functions.
2024-09-30 12:00:14 -07:00
Lauryn Menard ef1275e6fe help: Remove 'Getting your organization started with Zulip' guide.
Creates a URLRedirect for this help center article to go to the
new "Moving to Zulip" guide.

Updates the astro.config.mjs file for the changes to the help
center sidebar that have been made as part of the replacement
of this help center guide.

Fixes #31499.
2024-09-30 11:58:31 -07:00
Lauryn Menard f9de3f9a45 onboarding: Use "Moving to Zulip" guide in emails & Welcome bot message.
Replaces links to "Getting your organization started with Zulip"
in onboarding emails and Welcome bot direct message for owners of
new organizations.

Revises text in those emails and messages to reflect the new
"Moving to Zulip" help center guide that is now used.
2024-09-30 11:58:31 -07:00
Lauryn Menard 7d21e20cb4 help-links: Limit billing related relative gear menu links.
In order to only generate relative links for Zulip Cloud billing
specific gear menu options in relevant help center articles, we
pass down settings.CORPORATE_ENABLED to be set as a global variable
for zerver/lib/markdown/help_relative_links.py so that self-hosted
servers' help center documentation will not have these links.
2024-09-30 11:35:45 -07:00
Tim Abbott e6e3b12124 import_realm: Fix crash importing data with topic moves. 2024-09-28 11:53:30 -07:00
Alex Vandiver a20673a267 upload: Allow filtering to just a prefix (e.g. a realm id). 2024-09-26 12:01:11 -07:00
Alex Vandiver 579cf4ada7 upload: Make local-file save_attachment_contents chunk-at-a-time.
This means it does not attempt to hold large files entirely in memory
when writing them to a new location on disk.
2024-09-26 12:01:11 -07:00
Alex Vandiver 2dc737335e upload: Switch from BinaryIO to IO[bytes].
This is slightly more generally-compatible.
2024-09-26 12:01:11 -07:00
Alex Vandiver 638c579c56 tusd: Set metadata correctly in S3.
The Content-Type, Content-Disposition, StorageClass, and general
metadata are not set according to our patterns by tusd; copy the file
to itself to update those properties.
2024-09-26 12:00:43 -07:00
Alex Vandiver 287850d08d tusd: Remove non-ASCII characters from path-ids. 2024-09-26 12:00:43 -07:00
Alex Vandiver 84280ed7c2 upload: When serving s3 download URLs, send real filename.
Setting `ResponseContentDisposition=attachment` means that we override
the stored `ContentDisposition`, which includes a filename.  This
means that using the "Download" link on servers with S3 storage
produced a file named the sanitized version we stored.

Explicitly build a `ContentDisposition` to tell S3 to return, which
includes both `attachment` as well as the filename (if we have it
locally).
2024-09-26 12:00:43 -07:00
Tim Abbott 9b67164270 email_notifications: Fix emoji being giant in Outlook.
Apparently, Outlook ignores height/width CSS rules, but does support
the attribute on the image element itself, so specify that instead.

I don't think there are likely to be image tag implementations that
don't support the attribute, given that's the only thing that works in
Outlook.
2024-09-26 11:58:12 -07:00
Anders Kaseorg 88782f2917 integrations: Lazily load webhook integrations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-24 18:17:52 -07:00
Anders Kaseorg 6c442273ee test_urls: Remove dead URLResolutionTest.
This test was written back when Django accepted view function names as
strings that might be wrong; that’s not possible in Django ≥ 1.10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-24 18:17:52 -07:00
Anders Kaseorg 184c0203f3 upload: Lazily import boto3.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-24 16:38:37 -07:00
Vector73 9e4e85e140 saved_snippets: Add backend for saved snippets.
Part of #31227.
2024-09-24 15:27:58 -07:00
Tim Abbott 90a4b4934a text_fixtures: Fix buggy skip-checks placement. 2024-09-24 15:00:46 -07:00
Prakhar Pratyush 65f465562f export_realm: Remove the 'react on consent message' approach.
For exporting full with consent:

* Earlier, a message advertising users to react with thumbs up
  was sent and later used to determine the users who consented.

* Now, we no longer need to send such a message. This commit
  updates the logic to use `allow_private_data_export` user-setting
  to determine users who consented.

Fixes part of #31201.
2024-09-24 14:32:42 -07:00
Tim Abbott 9d74abee0b tools: Skip checks in recursive management calls. 2024-09-24 14:26:34 -07:00
tnmkr ddecba4e1c custom_profile_fields: Add "editable_by_user" setting.
This new property allows organization administrators to specify whether
users can modify the custom profile field value on their own account.
This property is configurable for individual fields.

By default, existing and newly created fields have this property set to
true, that is, they allow users to edit the value of the fields.

Fixes part of #22883.

Co-Authored-By: Ujjawal Modi <umodi2003@gmail.com>
2024-09-23 18:09:38 -07:00
Alex Vandiver ce0df00e44 export: Notify all realm admins on realm export. 2024-09-23 10:02:43 -07:00
Alex Vandiver 7afe6800f7 export: Use relative paths, include more data. 2024-09-23 10:02:43 -07:00
Alex Vandiver e125ad823d exports: Add a separate bucket for realm exports.
This allows finer-grained access control and auditing.  The links
generated also expire after one week, and the suggested configuration
is that the underlying data does as well.

Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
2024-09-20 15:43:49 -07:00
Alex Vandiver c1e8ecd08f uploads: Cache boto client in the module and be writable.
The `get_signed_upload_url` code is called for every S3 file serve
request, and is thus in the hot path.  The boto3 client caching
optimization is thus potentially useful as a performance optimization.
2024-09-20 15:43:49 -07:00
Alex Vandiver 1a7b3ef7ed upload: Use get_export_tarball_url in upload_export_tarball. 2024-09-20 15:43:49 -07:00
Alex Vandiver 4cf835d9dd upload: Remove common cache from get_export_tarball_url.
This is not called in the hot path like get_avatar_url is.
2024-09-20 15:43:49 -07:00
Alex Vandiver a5bf452202 upload: Realm is not Optional in upload_export_tarball.
af4eb8c0d5 marked the base class and local backend as non-Optional,
but left the S3 backend as Optional for some reason.

Remove it.
2024-09-20 15:43:49 -07:00
Alex Vandiver 91ac5c3c8b export: Log before the compression step, which can be slow. 2024-09-20 15:43:49 -07:00
Tim Abbott 51d0dfb504 docs: Tweak some documentation around send_event. 2024-09-20 15:28:18 -07:00
Prakhar Pratyush e763d065a3 django_api: Rename 'send_event' to 'send_event_rollback_unsafe'.
This commit renames the 'send_event' function to
'send_event_rollback_unsafe' to reflect the fact that it doesn't
wait for the db transaction (within which it gets called, if any)
to commit and sends event irrespective of commit or rollback.

In most of the cases we don't want to send event in the case of
rollbacks, so the caller should be aware that calling the function
directly is rollback unsafe.
2024-09-20 15:20:18 -07:00
Mateusz Mandera 18fbb5d146 retention: Limit number of ids passed to db in delete messages query.
If do_delete_messages (and friends) are called for a massive number of
messages, the giant list of message ids is passed to Postgres even
though chunk_size makes all but the first chunk_size of message ids
useless.
2024-09-20 09:31:21 -07:00
Mateusz Mandera ed7c330548 retention: Rename run_archiving_in_chunks to run_archiving. 2024-09-20 09:31:21 -07:00
Sahil Batra c53563d0e7 user_groups: Handle deactivated groups in webapp. 2024-09-18 13:41:13 -07:00
Sahil Batra 5c790aac72 user_groups: Rename allow_deactivated in 'GET /user_groups'.
This commit renames "allow_deactivated" parameter in
"GET /user_groups" endpoint to "include_deactivated_groups", so
that we can have consistent naming here and for client capability
used for deciding whether to send deactivated groups in register
response and how to handle the related events.
2024-09-18 13:41:13 -07:00