Commit Graph

18829 Commits

Author SHA1 Message Date
Mateusz Mandera 833dce8a13 saml: Add support for syncing user role.
Replace the SOCIAL_AUTH_SYNC_CUSTOM_ATTRS_DICT with
SOCIAL_AUTH_SYNC_ATTRS_DICT, designed to support also regular user attrs
like role or full name (in the future).

Custom attributes can stay configured as they were and will get merged
into SOCIAL_AUTH_SYNC_ATTRS_DICT in computed_settings, or can be
specified in SOCIAL_AUTH_SYNC_ATTRS_DICT directly with "custom__"
prefix.

The role sync is plumbed through to user creation, so users can
immediately be created with their intended role as provided by the IdP
when they're creating their account, even when doing this flow without
an invitiation.
2024-08-20 11:53:24 -07:00
Mateusz Mandera 8c1a1ea8db scim: Extract ROLE_TYPE_TO_NAME dict to UserProfile.
This allows these mappings to used in other APIs. Specifically, we want
to use this for syncing role during SAML auth.
2024-08-20 11:53:24 -07:00
Mateusz Mandera 76b41e433a maybe_send_to_registration: Enforce kwargs.
Most of these should be kwargs, to make sure we're not accidentally
getting the order wrong when calling it.
2024-08-20 11:53:24 -07:00
Kenneth Rodrigues 86bf0d3ea3 decorators: Migrate to typed_endpoint.
Modify `semgrep-py.yml` to treat arguments after `*args` as
keyword-only arguments.
2024-08-20 10:15:30 -07:00
Kenneth Rodrigues 50712bfa81 scheduled_messages: Migrate to typed_endpoint.
Migrate `scheduled_message.py` to typed_endpoint.

Perform Json parsing in the endpoint itself instead of
in `recipient_parsing.py`.
2024-08-20 10:03:22 -07:00
Ujjawal Modi 5d613ce98d user_groups: Add support to update can_manage_group setting.
This commit adds API support to update can_manage_group setting
of a user group.
2024-08-20 09:28:05 -07:00
Ujjawal Modi b0b36e884c user_groups: Add support to set can_manage_group during creation.
This commit adds API support to set can_manage_group while
creating a user group.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 2505813359 user_groups: Add can_manage_group field to UserGroup objects.
This commit make changes in code to include can_manage_group
field to UserGroup objects passed with response of various endpoints
including "/register" endpoint and also in the group object
send with user group creation event.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 423d5c42f6 user_groups: Update code to check whether user can edit a user group.
Earlier there was only a realm level setting for configuring
who can edit user groups. A new group level setting is also added
for configuring who can manage that particular group.
Now, a user group can be edited by a user if it is allowed from
realm level setting or group level setting.

This commit make changes to also use group level setting
in determining whether a group can be edited by user or not.

Also, updated tests to use api_post and api_delete helpers instead
of using client_post and client_delete helpers with different users
being logged in.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 03220ba456 user_groups: Add can_manage_group field in database.
This commit adds a new group level setting can_manage_group
for configuring who can manage a group. This commit only adds
the field in database and make changes to automatically create
single user groups corresponsing to acting user
which will be the default value for this setting.

Fixes part of #25928.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 9e699dfc85 tests: Separate test of group edit setting for creation and deletion.
Earlier there was a single backend test for testing group edit policy
for creating and deleting user group.This commit make changes in the test
and now there are two separate tests for testing group edit policy for
creating and deleting user groups.

This was done because in future commits we will be adding a
realm level setting for configuring who can create user groups.

Also, updated tests to use api_post and api_delete helpers instead
of using client_post and client_delete helpers with different users
being logged in.
2024-08-20 09:28:05 -07:00
Ujjawal Modi bb0b6900df user_groups: Add a decorator to check group creation permission.
Earlier there was a single decorator function to check whether
user can create and edit user groups. This commit adds a new
decorator function to check whether user has permissions to
create user groups.

This was done because in future commits we will be adding a
realm level setting for configuring who can create user groups.
2024-08-20 09:28:05 -07:00
Sahil Batra 3f800002be user_groups: Allow not passing old_setting_api_value. 2024-08-20 09:28:05 -07:00
evykassirer b1198785be message_edit: Convert module to typescript. 2024-08-19 16:33:04 -07:00
Sahil Batra 3dc82eed57 user_groups: Refactor code to compute user group objects.
This commit refactors code in user_groups_in_realm_serialized
such that we do not prefetch "can_mention_group__direct_members"
and "can_mention_group__direct_subgroups" using prefetch_related
and instead fetch members and subgroups for all groups in separate
queries and then use that data to find the members and subgroups
of the group used for that setting.

This change helps us in avoiding two prefetch queries for each
setting when we add more group settings.
2024-08-19 15:41:55 -07:00
Prakhar Pratyush 2406a3feea streams: Update do_change...retention_days to send event on commit.
Earlier, we were using 'send_event' in
do_change_stream_message_retention_days which can lead to a situation
where we enqueue events but the function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush a9af1c0ce2 streams: Update do_change_stream_description to send event on commit.
Earlier, we were using 'send_event' in do_change_stream_description
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush e474ab7ee0 streams: Add transaction.atomic decorator to 'do_rename_stream'.
This commit updates the 'do_rename_stream' action function
to make the db operations atomic.

Also, send_event is replaced with send_event_on_commit.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 69dec9dbc6 streams: Update 'do_change_stream_post_policy' to send event on commit.
Earlier, we were using 'send_event' in do_change_stream_post_policy
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 41d947b89d streams: Make db operations in do_change_subscription_property atomic.
This commit adds a transaction.atomic decorator to the
'do_change_subscription_property' function to make
the db operations in the action function atomic.

Also, send_event is changed to send_event_on_commit.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 5d0b3d1a6d muted_users: Update 'do_unmute_user' to send event on commit.
Earlier, we were using 'send_event' in 'do_unmute_user'
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 3b9a21443a muted_users: Update 'do_mute_user' to send event on commit.
Earlier, we were using 'send_event' in 'do_mute_user' which
can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 9026e6ecc1 message_flags: Update 'do_update_message_flags' to send event on commit.
Earlier, we were using 'send_event' in do_update_message_flags
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 32a4a112b1 message_flags: Update do_mark_muted_user...read to send event on commit.
Earlier, we were using 'send_event' in
'do_mark_muted_user_messages_as_read' which can lead to a
situation where we enqueue events but the function fails at a
later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 62eb850423 message_flags: Update transactions to be durable in do_mark_all_as_read.
In 'do_mark_all_as_read', the transactions which mark the messages
as read in batches should be marked as durable to avoid addition
of any outer atomic block as we support marking a few batches
(not all messages) as read in the case of a timeout.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush 64beea2765 message_flags: Update 'do_mark_stream...read' to send event on commit.
Earlier, we were using 'send_event' in do_mark_stream_messages_as_read
codepath which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-19 10:14:49 -07:00
Prakhar Pratyush ed512f06bb message_edit: Queue event on commit in do_update_message codepath.
'do_update_message' is within a db transaction, this commit
updates the 'do_clear_mobile_push_notifications_for_ids' function
used in 'do_update_message' to queue event on commit.

Events should not be sent until we know we're not rolling back,
otherwise it can lead to a situation where we enqueue events but
the function fails at a later stage.
2024-08-19 10:14:49 -07:00
Tim Abbott a3806b4165 migrations: Fix quoting bug in tutorial_status migration.
Fixes #31381.
2024-08-17 22:17:21 -07:00
bedo 00c9f36434 create stream: Bulk conversion of principals to UserProfiles.
Previously, this logic did the database queries to look up UserProfile
objects in a loop.

Fixes #21820.

Significantly improves Stream creation time and also unsusbcribing users.
Tested stream creation with 10k stream subscribers:

- before: 127 seconds ~2 mins
- after: 17 seconds ~0.3 min

Add a test case for user unsubscribing themself.
2024-08-16 17:46:19 -07:00
Alex Vandiver 854ec40def home: Add rel=dns-prefetch to Gravatar and any Zulip avatar host.
This hints to the browser that it should start DNS lookups for the
host, since it is likely to be necessary.  It is a softer form than
`rel-preconnect`, which may be unnecessary in these cases, if the
client has the resources cached already.
2024-08-16 14:33:25 -07:00
Lauryn Menard cabe0b2e05 api-docs: Update topics array description in get-stream-topics response. 2024-08-15 18:27:43 -07:00
Shubham Padia 2b3a41be58 user_card_popover: Show popover on all .messagebox user mention.
Show user card popover for scheduled messages overlay, compose box
preview, message edit preview, message edit history.

`.messagebox` was chosen as the selector since that was the nearest
parent class that was common for all of the above.

`@all` does not have a popover and that's why it will have the same
pointer as its parent element. We also introduce a new class called
`.user-mention-all` for managing css rules specific to that mention.
2024-08-15 16:28:08 -07:00
Prakhar Pratyush 52a9846cdf user_profile: Remove 'tutorial_status' field.
The 'tutorial_status' field on 'UserProfile' model is
no longer used to show onboarding tutorial.

This commit removes the 'tutorial_status' field,
'POST users/me/tutorial_status' endpoint, and
'needs_tutorial' parameter in 'page_params'.

Fixes part of zulip#30043.
2024-08-15 13:31:40 -07:00
Prakhar Pratyush ee806c49b9 ui_init: Use OnboardingStep for narrowing on first web app load.
We plan to remove the 'tutorial_status' field from UserProfile
table as it is no longer used to show tutorial.

The field is also used to narrow a new user in DM with
welcome bot on the first load.

This prep commit updates the logic to use a new OnboardingStep
for the narrowing behaviour on the first load. This will help
in removing the 'tutorial_status' field.
2024-08-15 13:31:40 -07:00
PieterCK d974c10efc integration-docs: Update Open Collective for new doc format.
Other than reformatting documentation for Open Collective, this
commit also moves it to the "Financial" category from "Communications".
This is because Open Collective is mainly a fundrising + legal status +
money management platform, as stated in https://opencollective.com/.

Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 00f9892d8d integration-docs: Update Mention for new doc format.
Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 07fda86ff9 integration-docs: Update Netlify for new doc format.
Besides reformatting the Netlify doc, this commit also updates the
instructions to match some UI changes in Netlify. The "Outgoing Webhook"
menu is now called "HTTP Post request".

Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 9494620022 integration: Delete Opbeat integration.
Opbeat was acquired by Elastic in 2017[1] and maintained until Elastic
APM went live[2]. Now that Elastic APM is online, it looks like Opbeat
repositories have been marked as archived/unmaintained[3].

[1]: https://www.elastic.co/blog/welcome-opbeat-to-the-elastic-family
[2]:
https://www.elastic.co/observability/application-performance-monitoring
[3]: https://github.com/opbeat
2024-08-15 13:16:14 -07:00
Vector73 80969a62e6 settings: Add `can_delete_any_message_group` setting.
Fixes #30717.
2024-08-15 12:03:49 -07:00
Prakhar Pratyush 19d56f77b5 rocketchat: Fix "OSError: [Errno 36] File name too long" error.
Earlier, we were replacing too long attachment name with random uuid
when the character count of the file name was greater than 255.

This results in "OSError: [Errno 36] File name too long" error in
few cases when the file name has less than 255 characters but more
than 255 bytes (file name with Non-ASCII characters).

This commit updates the code to check the file name's byte size
instead of characters count.
2024-08-14 18:18:31 -07:00
Mateusz Mandera bf7fc5701f management: Add change_auth_backends command.
A utility command to enable or disable certain authentication backends
for a realm from the command line. Can be helpful e.g. if the
administrator accidentally disables some auth methods in the UI leaving
themselves with none remaining that they could actually use to log back
into the organization.

Example usage:
```
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --show
Current authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --disable GitHub
Disabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Disabled backends:
  GitHub
Done!
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --enable GitHub
Enabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Done!
```
2024-08-14 14:04:46 -07:00
Alex Vandiver 2bfdccd065 narrow: Do not error when given an empty DM recipient list. 2024-08-14 10:39:40 -07:00
Vector73 fa408a969e api-docs: Handle multiple examples in `responses`. 2024-08-14 09:37:15 -07:00
Tim Abbott 07c927ae88 migrations: Fix Nagios bot cleanup crash.
We didn't correctly check for the possibility that these users don't
exist.
2024-08-13 23:30:32 -07:00
Shubham Padia 8b96aa277a css: Use classname instead of `div` selector for .blocks.
Having the `div` selector there affects the performance. See
https://chat.zulip.org/#narrow/stream/6-frontend/topic/CSS.20selector.20performance/near/1845719
for more details.
2024-08-13 22:36:35 -07:00
Alex Vandiver 16dffeb941 migrations: Fix missing Literal() calls in 0564. 2024-08-13 08:04:13 -07:00
Prakhar Pratyush e7f2a0958a custom_profile_fields: Send event on commit in notify_realm...fields.
Earlier, we were using 'send_event' in
'notify_realm_custom_profile_fields' which can lead to a situation,
if any db operation is added after the 'send_event' in the action
functions using it, where we enqueue event but the action function
fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush bae6188d92 user_groups: Add transaction.atomic decorator to add_user_group view.
The database operations in 'access_user_group_for_setting' and
'check_add_user_group' used in 'add_user_group' view should be
collectively atomic.

This commit adds transaction.atomic decorator for that purpose.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 427c59ec68 user_groups: Update delete_user_group codepath to send event on commit.
Earlier, we were using 'send_event' in 'delete_user_grou' codepath
which can lead to a situation, if any db operation is added after
the 'send_event' in future, where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 66ff646e99 user_groups: Update edit_user_group codepath to send event on commit.
Earlier, we were using 'send_event' in 'edit_user_group' codepath
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00