Commit Graph

8624 Commits

Author SHA1 Message Date
Lauryn Menard df1e9093a9 audit-log: Move stream/channel event types to AuditLogEventType enum.
Renamed event types below in the enum class to use channel instead of
stream.

Event types moved: STREAM_CREATED, STREAM_DEACTIVATED, STREAM_NAME_CHANGED
STREAM_REACTIVATED, STREAM_MESSAGE_RETENTION_DAYS_CHANGED
STREAM_PROPERTY_CHANGED, STREAM_GROUP_BASED_SETTING_CHANGED
2024-09-09 11:50:13 -07:00
Lauryn Menard 10d161638e audit-log: Move subscription event types to AuditLogEventType enum.
Event types moved: SUBSCRIPTION_CREATED, SUBSCRIPTION_ACTIVATED,
SUBSCRIPTION_DEACTIVATED, SUBSCRIPTION_PROPERTY_CHANGED.
2024-09-09 11:50:13 -07:00
Lauryn Menard 56c8cbde1e audit-log: Move realm event types to AuditLogEventType enum.
Event types moved: REALM_CREATED, REALM_DEFAULT_USER_SETTINGS_CHANGED
REALM_ORG_TYPE_CHANGED, REALM_DOMAIN_ADDED, REALM_DOMAIN_CHANGED
REALM_DOMAIN_REMOVED, REALM_PLAYGROUND_ADDED, REALM_PLAYGROUND_REMOVED
REALM_LINKIFIER_ADDED, REALM_LINKIFIER_CHANGED, REALM_LINKIFIER_REMOVED
REALM_EMOJI_ADDED, REALM_EMOJI_REMOVED, REALM_LINKIFIERS_REORDERED
REALM_IMPORTED
2024-09-09 11:50:13 -07:00
Lauryn Menard d2c32f23db audit-log: Move realm event types to AuditLogEventType enum.
Event types moved: REALM_DEACTIVATED, REALM_REACTIVATED, REALM_SCRUBBED
REALM_PLAN_TYPE_CHANGED, REALM_LOGO_CHANGED, REALM_EXPORTED
REALM_PROPERTY_CHANGED, REALM_ICON_SOURCE_CHANGED, REALM_DISCOUNT_CHANGED
REALM_SPONSORSHIP_APPROVED, REALM_BILLING_MODALITY_CHANGED
REALM_REACTIVATION_EMAIL_SENT, REALM_SPONSORSHIP_PENDING_STATUS_CHANGED
REALM_SUBDOMAIN_CHANGED
2024-09-09 11:50:13 -07:00
Lauryn Menard caeeaf3c3f audit-log: Move user event types to AuditLogEventType enum.
Event types moved: USER_SOFT_ACTIVATED, USER_SOFT_DEACTIVATED
USER_PASSWORD_CHANGED, USER_AVATAR_SOURCE_CHANGED, USER_FULL_NAME_CHANGED
USER_EMAIL_CHANGED, USER_TERMS_OF_SERVICE_VERSION_CHANGED
USER_API_KEY_CHANGED, USER_BOT_OWNER_CHANGED,
USER_DEFAULT_SENDING_STREAM_CHANGED, USER_DEFAULT_REGISTER_STREAM_CHANGED
USER_DEFAULT_ALL_PUBLIC_STREAMS_CHANGED, USER_SETTING_CHANGED
USER_DIGEST_EMAIL_CREATED
2024-09-09 11:50:13 -07:00
Lauryn Menard e5daa3470f audit-log: Move user event types to AuditLogEventType enum.
Event types moved: USER_CREATED, USER_ACTIVATED, USER_DEACTIVATED
USER_REACTIVATED, USER_ROLE_CHANGED, USER_DELETED
USER_DELETED_PRESERVING_MESSAGES
2024-09-09 11:50:13 -07:00
joseph b0a20d2cae attachments: Return a list of removed attachments while editing.
Currently, we want to ask users if they would like to delete their
attachments after they have removed the attachments while editing. These
changes are preparatory changes on the backend to return a list of removed
attachments after the user has removed attachments while editing.

Fixes part of #25525.
2024-09-06 11:40:51 -07:00
Kenneth Rodrigues 1d580f143e endpoints: Modify comments mentioning has_request_variables and REQ. 2024-09-05 16:02:12 -07:00
Kenneth Rodrigues dc32396180 endpoints: Remove the has_request_variables decorator.
All endpoints have been migrated to the typed_endpoint decorator,
therefore the has_request_variables decorator and the REQ function are
no longer needed and have been removed.
2024-09-05 16:02:12 -07:00
Anders Kaseorg e3abd09e67 thumbnail: Fix corrupted email notifications due to HTML5 entities.
BeautifulSoup with formatter="html5" unnecessarily escapes many
characters with HTML5-specific entities that cannot be correctly
parsed by lxml during generation of email notifications.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-05 16:00:45 -07:00
Alex Vandiver cec0942b4b users: Allow zero-width-joiners in user names.
Partially fixes: #20128.
2024-09-03 16:34:55 -07:00
Anders Kaseorg 91ade25ba3 python: Simplify with str.removeprefix, str.removesuffix.
These are available in Python ≥ 3.9.
https://docs.python.org/3/library/stdtypes.html#str.removeprefix

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-09-03 12:30:16 -07:00
Alya Abbott d7761f7163 help: Add a guide on moving to Zulip. 2024-09-01 16:16:04 -07:00
Alya Abbott 7df0db142b help: Document how to invite users in imported organizations. 2024-09-01 16:16:04 -07:00
bedo 3da91e951c auth: Migrate to @typed_endpoint.
Since this is the last has_request_variables endpoint outside tests,
more test_openapi code needs to be deleted in this transition.
2024-08-30 16:56:14 -07:00
Prakhar Pratyush 656c65c74c edit_history: Remove 'prev_rendered_content_version' field.
This commit removes the 'prev_rendered_content_version'
field from:

* the 'edit_history' object within message objects in the
API response of `GET /messages`, `GET /messages/{message_id}`
and `POST /zulip-outgoing-webhook`.
* the 'update_message' event type

as it is an internal server implementation detail not used
by any client.

Note: The field is still stored in the 'edit_history' column
of the 'Message' table as it will be helpful when making
major changes to the markup rendering process.
2024-08-29 15:37:12 -07:00
Alex Vandiver 0c07c6531c thumbnail: Enqueue thumbnails when we render a spinner.
Thumbnails are usually enqueued in the worker when the image is
uploaded.  However, for images which were uploaded before the
existence of the thumbnailing worker, and whose metadata was
backfilled (see previous commit) this leaves a permanent spinner,
since nothing triggers the thumbnail worker for them.

Enqueue a thumbnail worker for every spinner which we render into
Markdown.  This ensures that _something_ is attempting to resolve the
spinner which the user sees.  In the case of freshly-uploaded images
which are still in the queue, this results in a duplicate entry in the
thumbnailing queue -- this is harmless, since the worker determines
that all of the thumbnails we need have already been generated, and it
does no further work.  However, in the case of historical uploads, it
properly kicks off the thumbnailing process and results in a
subsequent message update to include the freshly-generated thumbnail.

While specifically useful for backfilled uploads, this is also
generally a good safety step for a good user experience, as it also
prevents dropped events in the queue from unknown causes from leaving
perpetual spinners in the message feed.

Because `get_user_upload_previews` is potentially called twice for
every message with spinners (see 6f20c15ae9), we add an additional
flag to `get_user_upload_previews` to suppress a _second_ event from
being enqueued for every spinner generated.
2024-08-29 12:11:51 -07:00
Lauryn Menard 8ace23ffa9 docs: Update help center links for stream to channel rename.
Fixes the URLRedirects for "help/about-streams-and-topics" and
"help/streams-and-topics" to go to "/help/introduction-to-topics"
since "help/channels-and-topics" has been redirected to that URL.
2024-08-27 13:30:28 -07:00
Anders Kaseorg 8c0b2d14aa mypy: Remove use of ValuesQuerySet and QuerySetAny.
This was made unnecessary in django-stubs 5.0.1 and mypy 1.10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-08-24 17:30:41 -07:00
Alex Vandiver 73e5364838 import: Use inspection to determine sequence names. 2024-08-23 17:15:35 -07:00
afeefuddin f013eb1037 message_fetch: Convert module to TypeScript. 2024-08-23 13:34:13 -07:00
Mateusz Mandera 669e0a3e47 import: Fix export->import of emojis from Slack.
Ideally this would besplit up into two commits, but it's hard to split
into self-contained, atomic chunks now that this segment of the
import/export system is generally kind of broken after thumbnailing
system changes.

1. 3rd party export converters don't make .original image files.
Insteadthey provide a single file, which the import should treat as if
it's .original.

2. 3rd party converters create all the records with is_animated=False.
That's an issue, because without setting that correctly on the
RealmEmoji objects, Zulip doesn't know that it should use the "still"
thumbnail when the emoji is being used in a user's status. Which leads
to incorrectly displaying the user status with the distracting
animation.
2024-08-21 16:30:19 -07:00
Mateusz Mandera 5476340b52 import: Export and import .original emoji files correctly.
The export tool was only exporting the already-thumbnailed emoji file,
omitting the original one. Now we make sure to export the .original file
too, like we do for avatars, and make the import tool process it
directly, to thumbnail it directly and generate a still in the case of
animated emojis.

Otherwise, the imported realm wouldn't have the <emoji>.png.original
file that we generally expect to have accessible, and stills for
animated emojis were completely missing.
2024-08-21 16:30:19 -07:00
Tim Abbott 9348b87d73 topic: Remove now-unused REQ_topic. 2024-08-21 11:27:54 -07:00
Kenneth Rodrigues 2483e600a2 message_send: Convert to typed endpoint.
Convert `message_send.py` use `typed endpoint`.

Disable `message_send` endpoint `to` parameter in the `openapi`
`validate_json_schema` check, because it is a special case where the
content type of the parameter is application/json but the
parameter may or may not be JSON encoded since previously we also
accepted a raw string and some ad-hoc bot might still depend on sending
a raw string.

Remove unused validators from `validator.py`.
2024-08-21 11:13:00 -07:00
Mateusz Mandera 8c1a1ea8db scim: Extract ROLE_TYPE_TO_NAME dict to UserProfile.
This allows these mappings to used in other APIs. Specifically, we want
to use this for syncing role during SAML auth.
2024-08-20 11:53:24 -07:00
Kenneth Rodrigues 50712bfa81 scheduled_messages: Migrate to typed_endpoint.
Migrate `scheduled_message.py` to typed_endpoint.

Perform Json parsing in the endpoint itself instead of
in `recipient_parsing.py`.
2024-08-20 10:03:22 -07:00
Ujjawal Modi 2505813359 user_groups: Add can_manage_group field to UserGroup objects.
This commit make changes in code to include can_manage_group
field to UserGroup objects passed with response of various endpoints
including "/register" endpoint and also in the group object
send with user group creation event.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 423d5c42f6 user_groups: Update code to check whether user can edit a user group.
Earlier there was only a realm level setting for configuring
who can edit user groups. A new group level setting is also added
for configuring who can manage that particular group.
Now, a user group can be edited by a user if it is allowed from
realm level setting or group level setting.

This commit make changes to also use group level setting
in determining whether a group can be edited by user or not.

Also, updated tests to use api_post and api_delete helpers instead
of using client_post and client_delete helpers with different users
being logged in.
2024-08-20 09:28:05 -07:00
Ujjawal Modi 03220ba456 user_groups: Add can_manage_group field in database.
This commit adds a new group level setting can_manage_group
for configuring who can manage a group. This commit only adds
the field in database and make changes to automatically create
single user groups corresponsing to acting user
which will be the default value for this setting.

Fixes part of #25928.
2024-08-20 09:28:05 -07:00
Sahil Batra 3dc82eed57 user_groups: Refactor code to compute user group objects.
This commit refactors code in user_groups_in_realm_serialized
such that we do not prefetch "can_mention_group__direct_members"
and "can_mention_group__direct_subgroups" using prefetch_related
and instead fetch members and subgroups for all groups in separate
queries and then use that data to find the members and subgroups
of the group used for that setting.

This change helps us in avoiding two prefetch queries for each
setting when we add more group settings.
2024-08-19 15:41:55 -07:00
bedo 00c9f36434 create stream: Bulk conversion of principals to UserProfiles.
Previously, this logic did the database queries to look up UserProfile
objects in a loop.

Fixes #21820.

Significantly improves Stream creation time and also unsusbcribing users.
Tested stream creation with 10k stream subscribers:

- before: 127 seconds ~2 mins
- after: 17 seconds ~0.3 min

Add a test case for user unsubscribing themself.
2024-08-16 17:46:19 -07:00
Shubham Padia 2b3a41be58 user_card_popover: Show popover on all .messagebox user mention.
Show user card popover for scheduled messages overlay, compose box
preview, message edit preview, message edit history.

`.messagebox` was chosen as the selector since that was the nearest
parent class that was common for all of the above.

`@all` does not have a popover and that's why it will have the same
pointer as its parent element. We also introduce a new class called
`.user-mention-all` for managing css rules specific to that mention.
2024-08-15 16:28:08 -07:00
Prakhar Pratyush 52a9846cdf user_profile: Remove 'tutorial_status' field.
The 'tutorial_status' field on 'UserProfile' model is
no longer used to show onboarding tutorial.

This commit removes the 'tutorial_status' field,
'POST users/me/tutorial_status' endpoint, and
'needs_tutorial' parameter in 'page_params'.

Fixes part of zulip#30043.
2024-08-15 13:31:40 -07:00
Prakhar Pratyush ee806c49b9 ui_init: Use OnboardingStep for narrowing on first web app load.
We plan to remove the 'tutorial_status' field from UserProfile
table as it is no longer used to show tutorial.

The field is also used to narrow a new user in DM with
welcome bot on the first load.

This prep commit updates the logic to use a new OnboardingStep
for the narrowing behaviour on the first load. This will help
in removing the 'tutorial_status' field.
2024-08-15 13:31:40 -07:00
PieterCK d974c10efc integration-docs: Update Open Collective for new doc format.
Other than reformatting documentation for Open Collective, this
commit also moves it to the "Financial" category from "Communications".
This is because Open Collective is mainly a fundrising + legal status +
money management platform, as stated in https://opencollective.com/.

Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 9494620022 integration: Delete Opbeat integration.
Opbeat was acquired by Elastic in 2017[1] and maintained until Elastic
APM went live[2]. Now that Elastic APM is online, it looks like Opbeat
repositories have been marked as archived/unmaintained[3].

[1]: https://www.elastic.co/blog/welcome-opbeat-to-the-elastic-family
[2]:
https://www.elastic.co/observability/application-performance-monitoring
[3]: https://github.com/opbeat
2024-08-15 13:16:14 -07:00
Vector73 80969a62e6 settings: Add `can_delete_any_message_group` setting.
Fixes #30717.
2024-08-15 12:03:49 -07:00
Alex Vandiver 2bfdccd065 narrow: Do not error when given an empty DM recipient list. 2024-08-14 10:39:40 -07:00
Shubham Padia 8b96aa277a css: Use classname instead of `div` selector for .blocks.
Having the `div` selector there affects the performance. See
https://chat.zulip.org/#narrow/stream/6-frontend/topic/CSS.20selector.20performance/near/1845719
for more details.
2024-08-13 22:36:35 -07:00
Prakhar Pratyush 0fddda232d drafts: Update do_delete_draft to send event on commit.
Earlier, we were using 'send_event' in 'do_delete_draft' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 096647fbd5 drafts: Update do_edit_draft to send event on commit.
Earlier, we were using 'send_event' in 'do_edit_draft' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush ed7c399644 drafts: Update do_create_drafts to send event on commit.
Earlier, we were using 'send_event' in 'do_create_drafts' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Alex Vandiver 1f045bc794 migrations: Allow zulipinternal bots to delete their own messages. 2024-08-12 12:02:04 -07:00
adnan-td 83094860b1 integration-docs: Update Pingdom to the new doc format.
Replaced the image to show **Add Integration** with text
instructions.

Part of #29592.
2024-08-12 11:43:16 -07:00
Lauryn Menard 481fb8e518 help: Rename and redirect allow-image-link-previews article.
Note that the link in zulip_update_announcements.py is not updated
so that the content in the source code reflects what users actually
received in the update announcement message.
2024-08-12 11:22:37 -07:00
Sahil Batra 6919448917 settings: Remove create_web_public_stream_policy setting.
This commit removes create_web_public_stream_policy setting
since web-public channel creation permissions are now
handled by group-based setting.

We still pass "realm_create_web_public_stream_policy" in
"/register" response though for older clients with its
value being set depending on the value of group based
setting. If we cannot set its value to an appropriate enum
corresponding to the group setting, then we set it to
"Admins and moderators" considering that server will not
allow the users without permissions to create web-public
channels but the client can make sure that UI is
available to the users who have permission.
2024-08-01 22:49:33 -07:00
Sahil Batra 09102c5780 streams: Check permission to create web-public streams based on group setting. 2024-08-01 22:49:33 -07:00
Sahil Batra a15bc5e264 realm: Add new group setting for web-public stream creation. 2024-08-01 22:49:33 -07:00
Alex Vandiver 6f20c15ae9 thumbnail: Resolve a race condition when rendering messages.
Messages are rendered outside of a transaction, for performance
reasons, and then sent inside of one.  This opens thumbnailing up to a
race where the thumbnails have not yet been written when the message
is rendered, but the message has not been sent when thumbnailing
completes, causing `rewrite_thumbnailed_images` to be a no-op and the
message being left with a spinner which never resolves.

Explicitly lock and use he ImageAttachment data inside the
message-sending transaction, to rewrite the message content with the
latest information about the existing thumbnails.

Despite the thumbnailing worker taking a lock on Message rows to
update them, this does not lead to deadlocks -- the INSERT of the
Message rows happens in a transaction, ensuring that either the
message rending blocks the thumbnailing until the Message row is
created, or that the `rewrite_thumbnailed_images` and Message INSERT
waits until thumbnailing is complete (and updated no Message rows).
2024-08-01 16:48:16 -07:00