Commit Graph

19094 Commits

Author SHA1 Message Date
PieterCK 00f9892d8d integration-docs: Update Mention for new doc format.
Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 07fda86ff9 integration-docs: Update Netlify for new doc format.
Besides reformatting the Netlify doc, this commit also updates the
instructions to match some UI changes in Netlify. The "Outgoing Webhook"
menu is now called "HTTP Post request".

Part of #29592.
2024-08-15 13:16:14 -07:00
PieterCK 9494620022 integration: Delete Opbeat integration.
Opbeat was acquired by Elastic in 2017[1] and maintained until Elastic
APM went live[2]. Now that Elastic APM is online, it looks like Opbeat
repositories have been marked as archived/unmaintained[3].

[1]: https://www.elastic.co/blog/welcome-opbeat-to-the-elastic-family
[2]:
https://www.elastic.co/observability/application-performance-monitoring
[3]: https://github.com/opbeat
2024-08-15 13:16:14 -07:00
Vector73 80969a62e6 settings: Add `can_delete_any_message_group` setting.
Fixes #30717.
2024-08-15 12:03:49 -07:00
Prakhar Pratyush 19d56f77b5 rocketchat: Fix "OSError: [Errno 36] File name too long" error.
Earlier, we were replacing too long attachment name with random uuid
when the character count of the file name was greater than 255.

This results in "OSError: [Errno 36] File name too long" error in
few cases when the file name has less than 255 characters but more
than 255 bytes (file name with Non-ASCII characters).

This commit updates the code to check the file name's byte size
instead of characters count.
2024-08-14 18:18:31 -07:00
Mateusz Mandera bf7fc5701f management: Add change_auth_backends command.
A utility command to enable or disable certain authentication backends
for a realm from the command line. Can be helpful e.g. if the
administrator accidentally disables some auth methods in the UI leaving
themselves with none remaining that they could actually use to log back
into the organization.

Example usage:
```
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --show
Current authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --disable GitHub
Disabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Disabled backends:
  GitHub
Done!
(zulip-py3-venv) vagrant@c32c137f59a0:/srv/zulip$ ./manage.py change_auth_backends -r zulip --enable GitHub
Enabling GitHub backend for realm Zulip Dev
Updated authentication backends for the realm:
Enabled backends:
  Dev
  Email
  GitHub
  GitLab
  Google
  Apple
  SAML
  OpenID Connect
Done!
```
2024-08-14 14:04:46 -07:00
Alex Vandiver 2bfdccd065 narrow: Do not error when given an empty DM recipient list. 2024-08-14 10:39:40 -07:00
Vector73 fa408a969e api-docs: Handle multiple examples in `responses`. 2024-08-14 09:37:15 -07:00
Tim Abbott 07c927ae88 migrations: Fix Nagios bot cleanup crash.
We didn't correctly check for the possibility that these users don't
exist.
2024-08-13 23:30:32 -07:00
Shubham Padia 8b96aa277a css: Use classname instead of `div` selector for .blocks.
Having the `div` selector there affects the performance. See
https://chat.zulip.org/#narrow/stream/6-frontend/topic/CSS.20selector.20performance/near/1845719
for more details.
2024-08-13 22:36:35 -07:00
Alex Vandiver 16dffeb941 migrations: Fix missing Literal() calls in 0564. 2024-08-13 08:04:13 -07:00
Prakhar Pratyush e7f2a0958a custom_profile_fields: Send event on commit in notify_realm...fields.
Earlier, we were using 'send_event' in
'notify_realm_custom_profile_fields' which can lead to a situation,
if any db operation is added after the 'send_event' in the action
functions using it, where we enqueue event but the action function
fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush bae6188d92 user_groups: Add transaction.atomic decorator to add_user_group view.
The database operations in 'access_user_group_for_setting' and
'check_add_user_group' used in 'add_user_group' view should be
collectively atomic.

This commit adds transaction.atomic decorator for that purpose.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 427c59ec68 user_groups: Update delete_user_group codepath to send event on commit.
Earlier, we were using 'send_event' in 'delete_user_grou' codepath
which can lead to a situation, if any db operation is added after
the 'send_event' in future, where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 66ff646e99 user_groups: Update edit_user_group codepath to send event on commit.
Earlier, we were using 'send_event' in 'edit_user_group' codepath
which can lead to a situation where we enqueue events but the
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 69a2775e9b video_calls: Update do_set_zoom_token to send event on commit.
Earlier, we were using 'send_event' in 'do_set_zoom_token' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 0fddda232d drafts: Update do_delete_draft to send event on commit.
Earlier, we were using 'send_event' in 'do_delete_draft' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 096647fbd5 drafts: Update do_edit_draft to send event on commit.
Earlier, we were using 'send_event' in 'do_edit_draft' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush ed7c399644 drafts: Update do_create_drafts to send event on commit.
Earlier, we were using 'send_event' in 'do_create_drafts' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 475a25e739 scheduled_messages: Send event on commit in delete_scheduled_message.
Earlier, we were using 'send_event' in 'delete_scheduled_messages'
which can lead to a situation, if any db operation is added after
the 'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 500fb3d804 scheduled_messages: Send event on commit in edit_scheduled_message.
Earlier, we were using 'send_event' in 'edit_scheduled_messages'
which can lead to a situation, if any db operation is added after
the 'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush b0390ce1ee scheduled_messages: Send event on commit in try_deliver_..._message.
Earlier, we were using 'send_event' in
'try_deliver_one_scheduled_messages' which can lead to a situation
where we enqueue events but the action function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Prakhar Pratyush 567615a484 onboarding_steps: Send event on commit in mark onboarding_step as read.
Earlier, we were using 'send_event' in 'do_mark_onboarding_step_as_read'
which can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-08-12 12:16:14 -07:00
Alex Vandiver b287efce43 migrations: Purge database of old nagios messages.
This deletes them directly, rather than move them into archival,
because that would be slow, and bloat the archival table in a way
which might interfere with other deletions.
2024-08-12 12:02:04 -07:00
Alex Vandiver 1f045bc794 migrations: Allow zulipinternal bots to delete their own messages. 2024-08-12 12:02:04 -07:00
PieterCK 68c87dbdf8 integrations: Update comment related notifications in Jira integration.
Previously, comment related notifications only displayed the issue
title as a plain string. This commit reformats the issue title to
include a link back to the Jira issue.
2024-08-12 11:45:09 -07:00
PieterCK c952e7ae86 integrations : Update Jira doc.
Adjusted the Jira documentation for recent changes in their UI
when setting up webhooks, reformatted the note about compatible
Jira version, and added a link to Jira's official webhook guide.
2024-08-12 11:45:09 -07:00
adnan-td e95b51098e integration-docs: Update Papertrail the new doc format.
Part of #29592.
2024-08-12 11:43:16 -07:00
adnan-td 164217f7a3 integration-docs: Update PagerDuty to the new doc format.
Part of #29592.
2024-08-12 11:43:16 -07:00
adnan-td 6e08798c02 integration-docs: Update Patreon to the new doc format.
Part of #29592.
2024-08-12 11:43:16 -07:00
adnan-td 83094860b1 integration-docs: Update Pingdom to the new doc format.
Replaced the image to show **Add Integration** with text
instructions.

Part of #29592.
2024-08-12 11:43:16 -07:00
Lauryn Menard 481fb8e518 help: Rename and redirect allow-image-link-previews article.
Note that the link in zulip_update_announcements.py is not updated
so that the content in the source code reflects what users actually
received in the update announcement message.
2024-08-12 11:22:37 -07:00
Lauryn Menard d54f747f05 api-docs: Update realm_export event to be clearer about export type.
Clarifies that these events are sent for public data exports of a
realm.
2024-08-12 10:12:08 -07:00
Lauryn Menard 80707d97d9 api-docs: Revise descriptions of RealmExport schema fields. 2024-08-12 10:12:08 -07:00
Vector73 b0bd13207d api_docs: Document "/export/realm" endpoints. 2024-08-12 10:12:08 -07:00
Mateusz Mandera 114f13e0ee auth: Fix re-enabling of SAML/AzureAD in organization settings UI.
This bug was introduced in da9e4e6e54.
validate validate_plan_for_authentication_methods is already called
inside validate_authentication_methods_dict_from_api, conditionally on
settings.BILLING_ENABLED. This additional, redundant call runs
regardless of BILLING_ENABLED, and thus prevents a self-hosted server
from enabling certain backends in the organization settings UI.

The impact of this is limited - in order to encounter this bug, a
self-hosted server would have to first disable the backend in the UI, as
self-hosted realms are created with all backend flags enabled. A backend
doesn't show up in the org settings UI until it is first enabled in
AUTHENTICATION_BACKENDS in settings.py - that's why this is a rare
state. A sequence of steps like this has to be followed to reproduce:
1. Add the backend to AUTHENTICATION_BACKENDS in settings.py.
2. Disable the backend in the org settings UI.
3. Now try to re-enable it, which fails due to the bug.
2024-08-05 21:14:43 -07:00
Mateusz Mandera 3f472ec664 test_auth_backends: Remove copied-and-pasted comments.
These makes no sense and were copied from the block at the top of the
test.
2024-08-05 21:14:43 -07:00
Sahil Batra 6919448917 settings: Remove create_web_public_stream_policy setting.
This commit removes create_web_public_stream_policy setting
since web-public channel creation permissions are now
handled by group-based setting.

We still pass "realm_create_web_public_stream_policy" in
"/register" response though for older clients with its
value being set depending on the value of group based
setting. If we cannot set its value to an appropriate enum
corresponding to the group setting, then we set it to
"Admins and moderators" considering that server will not
allow the users without permissions to create web-public
channels but the client can make sure that UI is
available to the users who have permission.
2024-08-01 22:49:33 -07:00
Sahil Batra 09102c5780 streams: Check permission to create web-public streams based on group setting. 2024-08-01 22:49:33 -07:00
Sahil Batra a15bc5e264 realm: Add new group setting for web-public stream creation. 2024-08-01 22:49:33 -07:00
Anders Kaseorg c03839f42f mypy: Reenable explicit-override for models.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-08-01 17:39:04 -07:00
Alex Vandiver 6f20c15ae9 thumbnail: Resolve a race condition when rendering messages.
Messages are rendered outside of a transaction, for performance
reasons, and then sent inside of one.  This opens thumbnailing up to a
race where the thumbnails have not yet been written when the message
is rendered, but the message has not been sent when thumbnailing
completes, causing `rewrite_thumbnailed_images` to be a no-op and the
message being left with a spinner which never resolves.

Explicitly lock and use he ImageAttachment data inside the
message-sending transaction, to rewrite the message content with the
latest information about the existing thumbnails.

Despite the thumbnailing worker taking a lock on Message rows to
update them, this does not lead to deadlocks -- the INSERT of the
Message rows happens in a transaction, ensuring that either the
message rending blocks the thumbnailing until the Message row is
created, or that the `rewrite_thumbnailed_images` and Message INSERT
waits until thumbnailing is complete (and updated no Message rows).
2024-08-01 16:48:16 -07:00
Anders Kaseorg ea60bc3ff8 ruff: Fix PLR1714 Consider merging multiple comparisons.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-08-01 16:46:58 -07:00
Alex Vandiver ab1e858eb9 invites: Use the realm on the PreregistrationUser, to save a join. 2024-08-01 16:44:18 -07:00
Alex Vandiver 2bc118892a migrations: Add missing dependency.
This migration references the "confirmation" app for the first time,
which means we must have migrated at least part of it by this point.

Set the migration to depend on the latest "confirmation" migration at
the time of this migration.
2024-08-01 16:44:18 -07:00
bedo 1ab4723d59 user_groups: Migrate to @typed_endpoint.
Migrate the following endpoints from @has_request_variables
to @typed_endpoint:

- get_user_group()
- delete_user_group()
- update_user_group_backend()
- update_subgroups_of_user_group()
- get_is_user_group_member()
- get_user_group_members()
- get_subgroups_of_user_group()

With tweaks from tabbott to avoid calling thunks unnecessarily.
2024-08-01 09:56:06 -07:00
Alex Vandiver d70935ae04 clients: Drop "desktop app" substring check.
The last desktop client to use that was 0.5.8, released 2017-04-12.
There are no clients with such old versions still observed in the
wild.
2024-08-01 09:23:16 -07:00
roanster007 7b3e163d55 refactor: Rename `huddle` to `direct_message_group` in non api files.
This commit completes rename of "huddle" to "direct_message_group"
in all the non API files.

Part of #28640
2024-07-31 23:25:56 -07:00
Prakhar Pratyush 04347bb315 schedule_messages: Update do_schedule_messages to send event on commit.
Earlier, we were using 'send_event' in 'do_schedule_messages' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-07-31 22:33:52 -07:00
Prakhar Pratyush 7302d533fe user_status: Update do_update_user_status to send event on commit.
Earlier, we were using 'send_event' in 'do_update_user_status' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-07-31 22:33:52 -07:00
Prakhar Pratyush dc27711399 alert_words: Update remove_alert_word codepath to send event on commit.
Earlier, we were immediately enqueueing event in
'do_remove_alert_words' which can lead to a situation, if any
db operation is added after enqueueing event in future, where the
action function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-07-31 22:33:52 -07:00
Prakhar Pratyush ba80266b7e alert_words: Update add_alert_words codepath to send event on commit.
Earlier, we were using 'send_event' in 'do_add_alert_words' which
can lead to a situation, if any db operation is added after the
'send_event' in future, where we enqueue events but the action
function fails at a later stage.

Events should not be sent until we know we're not rolling back.

Fixes part of #30489.
2024-07-31 22:33:52 -07:00
Kenneth Rodrigues 454f2f5b0c video_calls: Migrate to typed_endpoint. 2024-07-31 17:10:06 -07:00
Kenneth Rodrigues 97f15d8811 events: Migrate to typed_endpoint.
Migrate `event_register.py` and `tornado` to typed_endpoint.
Modified the tests to work with the migrated endpoints.
2024-07-31 17:10:06 -07:00
Kenneth Rodrigues 0f692436ca user_settings: Migrate to typed_endpoint.
Migrate `user_settings.py` to `typed_endpoint`.
Fix the error messages for the tests.
Migrate required validators.
2024-07-31 17:10:06 -07:00
Mateusz Mandera aaca394813 presence: Remove the queue worker. 2024-07-31 16:46:42 -07:00
Anders Kaseorg 8f1a97f168 users: Reduce date_joined precision to minutes.
Fixes #31188, removing a lot of useless high-entropy data from the
/register response.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-31 12:53:09 -07:00
lumpleme b5c63cfb85 integrations: Prevent duplicate GitHub pull request review messages.
GitHub sends two almost identical payloads when a pull
request is reviewed, which results in two duplicative
notification messages.

The payloads have different "action" value, with one
having the "submitted" action, whereas the other is
"edited" and has an empty "changes" dict.

We now ignore the payload with the "edited" action type
and an empty "changes" dict.

Fixes #26145.

Co-authored-by: Pieter CK <pieterceka123@gmail.com>
2024-07-31 10:42:41 -07:00
adnan-td 76df435dab integration-docs: Update Semaphore to the new doc format.
Part of #29592.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-07-31 10:16:54 -07:00
adnan-td bdbe3adf1b integration-docs: Update Sentry to the new doc format.
Part of #29592.
2024-07-31 10:16:54 -07:00
Tim Abbott e7cc9090fd api docs: Clarify event structure when moving channels.
The previous language would most naturally be read as suggesting that
moving topics between channels would have these fields.
2024-07-31 09:22:44 -07:00
Mateusz Mandera a0971934d9 thumbnail: Fix typo in comment. 2024-07-30 00:17:59 +02:00
Anders Kaseorg 3a17946c85 zulip_update_announcements: Fix December 2024 typo.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-26 10:20:27 -07:00
Alya Abbott 7a1f5765b7 updates: Add update message (level=8). 2024-07-25 13:19:25 -07:00
Alex Vandiver c726d2ec01 thumbnail: Do not Camo old thumbor URLs; serve images directly.
Providing a signed Camo URL for arbitrary URLs opened the server up to
being an open redirector.  Return 403 if the URL is not a user upload,
and the backend image if it is.  Since we do not have ImageAttachment
rows for uploads at a time we wrote `/thumbnail?` URLs, return the
full-size content.
2024-07-24 16:04:34 -07:00
Alex Vandiver e3a238fc89 thumbnail: Remove unused thumbnail sizes.
47683144ff switched the web client to prefer the 840x560 size, as the
mobile apps prefer; remove the now-unused 300x200 size.  No client was
using the generated `.jpg` formats, as all clients support `.webp`, so
remove the unused `.jpg` thumbnail as well.
2024-07-24 09:57:20 -07:00
Alex Vandiver e4a8304f57 thumbnail: Store the post-orientation-transformation dimensions.
Modern browsers respect the EXIF orientation information of images,
applying rotation and/or mirroring as specified in those tags.  The
the `width="..."` and `height="..."` tags are to size the image
_after_ applying those orientation transformations.

The `.width` and `.height` properties of libvips' images are _before_
any transformations are applied.  Since we intend to use these to hint
to rendering clients the size that the image should be _rendered at_,
change to storing (and providing to clients) the dimensions of the
rendered image, not the stored bytes.
2024-07-24 09:56:42 -07:00
Alex Vandiver e7ac62aad7 tests: Add a test that the thumb is actually the expected size. 2024-07-24 09:56:42 -07:00
Alex Vandiver d824988bc4 tests: Remove THUMBNAIL_IMAGES overrides, since the setting was dropped.
These were omitted from b42863be4b.
2024-07-24 09:53:14 -07:00
Mateusz Mandera 8f461b3c03 email_mirror: Change default topic name if subject ends up empty.
If the email subject is something like `Fwd:`, it gets stripped to an
empty string, activating the "(no topic)" override. This however leads
to failure if the organization enables the setting forcing every message
to have a topic. Such emails should still go through, so we should just
change the topic value used.
2024-07-22 22:31:45 -07:00
Alex Vandiver 2ea0cc0005 thumbnail: Add a data-original-dimensions attribute.
This allows clients to potentially lay out the thumbnails more
intelligently, or to provide a better "progressive-load" experience
when enlarging the thumbnail.
2024-07-22 22:41:10 -04:00
Alex Vandiver 65828b20e9 thumbnail: Factor out a dataclass for markdown image metadata. 2024-07-22 22:41:10 -04:00
roanster007 66a96bee71 settings: Add setting to control how animated images are played.
Previously animated images were automatically played in the
message feed of the web app.

Now that we have still thumbnails available for them, we can add a new
personal setting, "web_animate_image_previews", which controls how the
animated images would be played in the web app message feed -- always
played, on hover, or only in the image viewer.

Fixes #31016.
2024-07-22 14:53:31 -07:00
Prakhar Pratyush 6838a7302d test_import_export: Add test coverage for OnboardingUserMessage. 2024-07-22 10:26:33 -07:00
Prakhar Pratyush f48e87cd3c fetch_initial_state: Avoid doing one db query per announcement stream.
In 'fetch_initial_state_data' we were doing one database query
per announcement stream.

This commit updates the logic to prefetch those streams using
select_related hence avoiding the extra db queries.

Fixes #28909.
2024-07-22 10:21:22 -07:00
Alex Vandiver 3ac14632d8 thumbnail: Disable libvips cache.
The libvips cache is 100MB, 100 operations, or 100 files, whichever is
less.  A single Django process or worker is extremely unlikely to ever
see the same image twice, much less within those timeframes.

Disable the cache, since it is mostly useless memory usage for our use
case.
2024-07-22 10:19:33 -07:00
Alex Vandiver b42863be4b markdown: Show thumbnails for uploaded images.
Fixes: #16210.
2024-07-21 18:41:59 -07:00
Alex Vandiver 71406ac767 thumbnail: Factor frames into account for IMAGE_BOMB_TOTAL_PIXELS. 2024-07-21 18:41:59 -07:00
Alex Vandiver aacf28f7e3 test_classes: Extract a thumbnailing output format helper. 2024-07-21 18:41:59 -07:00
Alex Vandiver 94ff443c00 test_classes: Merge verbose assertEquals into ZulipTestCase. 2024-07-21 18:41:59 -07:00
Niloth P e1d35b1e72 integration-docs: Update Opsgenie for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P d972d76725 integration-docs: Update Pivotal for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P d571410ba3 integration-docs: Update Rundeck for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P 650a1384ca integration-docs: Update RhodeCode for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P 8fe7f4e409 integration-docs: Update Review Board for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P ec018780b5 integration-docs: Update Raygun for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Niloth P 335f9b8119 integration-docs: Update Radarr for new doc format.
Part of zulip#29592.
2024-07-21 18:09:13 -07:00
Mateusz Mandera f50104d475 import_realm: Update incorrect comment about emoji dir in exported data.
The emoji dir is present in the data from our export tool. This was
added in 468afe4840.

This comment hasn't been updated since
c4b886d8ae, so probably we just forgot to
refresh it when custom emoji export was added.
2024-07-21 13:16:49 -07:00
Kenneth Rodrigues 8b489f4b96 streams: Convert to typed_enpoint. 2024-07-21 12:57:50 -07:00
Kenneth Rodrigues 6815cded83 zerver: Migrate some files to typed_endpoint.
Migrates `invite.py`, `registration.py` and
`email_mirror.py` to use `typed_endpoint`.
2024-07-20 15:46:48 -07:00
Niloth P 16abd82fa5 integrations: Remove Solano integration. 2024-07-20 15:44:17 -07:00
Sahil Batra c6e9173297 exceptions: Add new error class to be used for invalid parameter values.
This would help us in avoiding adding translation everytime we use
this error for a new pair of parameters.
2024-07-19 09:24:54 -07:00
Anders Kaseorg d574200423 tests: Consume streaming responses.
Fixes warnings like “ResourceWarning: unclosed file <_io.FileIO
name='/srv/zulip/var/044e5d44-87aa-4c43-abbb-28a144fa6654/test-backend/run_1238680/worker_0/test_uploads/files/thumbnail/2/1e/jmUuDhQC8WlaSRCuc0zQyx7D/img.tif/100x75.webp'
mode='rb' closefd=True>” with warnings enabled.

deque(…, 0) is an efficient way to consume an iterator documented at
https://docs.python.org/3/library/itertools.html#itertools-recipes
under consume.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-19 09:20:56 -07:00
Anders Kaseorg c7281b406f test_delete_unclaimed_attachments: Close file in make_attachment.
Fixes “ResourceWarning: unclosed file <_io.BufferedReader
name='/srv/zulip/zerver/tests/images/text.txt'>” with warnings enabled.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-19 09:20:56 -07:00
Tim Abbott de35f44624 zulip_updates: Mention default for send navigation.
Apparently I forgot to merge this before -beta1.
2024-07-18 14:06:04 -07:00
Alex Vandiver 4351cc5914 thumbnail: Move get_image_thumbnail_path and split_thumbnail_path. 2024-07-18 13:50:28 -07:00
Alex Vandiver d474600056 tests: Remove a nonsensical no-op vimeo test.
Embeds are disabled in this test, and it tests that it does not embed
anything.
2024-07-18 13:50:28 -07:00
Alex Vandiver 424f3dd503 tests: Break up and reorganize test_markdown.py. 2024-07-18 13:50:28 -07:00
Alex Vandiver ecdf5713c2 tests: Factor out ZulipVerboseEqualTest class, and use more. 2024-07-18 13:50:28 -07:00
Alex Vandiver 08191d3f69 tests: Default ENABLE_FILE_LINKS to False.
Test configuration should mirror default production as much as possible.
2024-07-18 13:50:28 -07:00
Alex Vandiver e02e9c9181 tests: INLINE_URL_EMBED_PREVIEW=False is the default in tests. 2024-07-18 13:50:28 -07:00
Alex Vandiver 060754980c tests: INLINE_IMAGE_PREVIEW=True is already the default. 2024-07-18 13:50:28 -07:00
Alya Abbott d775b6ffff updates: Add update message (level=7). 2024-07-18 12:38:26 -07:00
Tim Abbott f37082c4ca models: Switch font size default to non-compact.
This is the intended longer-term default, and it's polished enough for
testing.
2024-07-18 12:37:06 -07:00
Niloth P cbe2677a2a integration-docs: Include suggestion for HTTPS hint in Stripe.
With some wording tweaks by tabbott to make it more obviously only
shown in certain environments.
2024-07-18 12:28:08 -07:00
Niloth P ed885ec806 integration-docs: Format keyboard shortcuts in Trello doc. 2024-07-18 12:24:45 -07:00
Niloth P 525f1e28d4 integration-docs: Fix typo in Splunk doc hint. 2024-07-18 12:24:36 -07:00
Sahil Batra c1c4c95ce7 settings: Require clients to pass dense_mode value when needed.
Previously, if someone changed the font-size or line height
settings to some value other than the legacy values, we set
dense_mode to False if it was True. This commit changes the
code to require clients to pass dense_mode as False in such
cases and raise an error otherwise.
2024-07-18 11:36:53 -07:00
Sahil Batra 0ea5daece8 user_settings: Flush cache after commiting changes to DB.
We should flush the cache after commiting changes to DB to
make sure that the cache does not contain stale data when
updating multiple settings in a single request.
2024-07-18 11:36:53 -07:00
Anders Kaseorg 97753fe3b0 emoji_names: Rebuild with CLDR 45, emoji-data 15.1.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 22:49:52 -07:00
Mateusz Mandera 4a93149435 settings: Rework how push notifications service is configured.
Instead of the PUSH_NOTIFICATIONS_BOUNCER_URL and
SUBMIT_USAGE_STATISTICS settings, we want servers to configure
individual ZULIP_SERVICE_* settings, while maintaining backward
compatibility with the old settings. Thus, if all the new
ZULIP_SERVICE_* are at their default False value, but the legacy
settings are activated, they need to be translated in computed_settings
to the modern way.
2024-07-17 17:14:06 -07:00
Anders Kaseorg 722842a0aa rocketchat: Remove unnecessary SHA-1 hashing of direct message groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 541699a1c6 mattermost: Remove unnecessary MD5 hashing of direct message groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 90cc88c88e mattermost: Split get_recipient_id_from_receiver_name to 3 functions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 27b0618704 data_import: Fix IdMapper typing.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Anders Kaseorg 1fd3f983a5 data_import: Remove int detection from IdMapper.
This seems to have been used only for HipChat.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-17 15:56:00 -07:00
Alex Vandiver ab6b6639e6 migrations: Derive emoji content-type from the bytes. 2024-07-17 11:39:04 -07:00
Sahil Batra cf14d8f611 users: Change bot owner only if needed.
This is a small optimization to avoid DB queries if the bot owner
is not changed.

This also helps in avoiding showing the error message in UI if
anything other than the owner is updated for a deactivated bot.
Ideally if the bot owner is not changed, the bot_owner_id field
should not be passed in the request, but we would handle that
later given this is anyways a small nice optimization and we
follow this pattern at other places as well in the API.
2024-07-17 10:07:51 -07:00
Lauryn Menard b9c33ce5f1 api-docs: Add changes section in construct a narrow article.
Adds a header to the section of changes notes about new and
deprecated narrow filters, so that it's easier to directly
link to the relevant content in the construct a narrow article
from the main API changelog and endpoint documentation.
2024-07-16 23:40:45 -07:00
Vector73 7a80fcf042 events: Send `delete_message` event to user who deleted the message.
Fixes #29826.

Co-authored-by: Mukul Goyal <goyal.mukul7689@gmail.com>
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
2024-07-16 23:29:02 -07:00
Niloth P 56aa72af90 integration-docs: Update Trello for new doc format.
Part of zulip#29592.
2024-07-16 17:02:55 -07:00
Niloth P c8781b4de7 integration-docs: Update TeamCity for new doc format.
Part of zulip#29592.
2024-07-16 17:02:55 -07:00
Alex Vandiver 9983dc761f migrations: Further fix emoji re-thumbnailing.
90701978d9 was an incomplete fix.
2024-07-16 15:38:22 -07:00
Niloth P f663544353 integration-docs: Update Slack incoming-webhook for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Niloth P 32177a9a1c integration-docs: Update Sonarqube for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Niloth P a546fa719a integration-docs: Update Sonarr for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Niloth P 1ddf92a98c integration-docs: Update Splunk for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Niloth P 8549a6dc8a integration-docs: Update Statuspage for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Niloth P c0ab89ec52 integration-docs: Update Stripe for new doc format.
Part of zulip#29592.
2024-07-16 14:55:56 -07:00
Alex Vandiver e8a049ea57 migrations: Handle missing emoji/images/still/ directory. 2024-07-16 14:55:45 -07:00
Niloth P a17a54b077 integration-docs: Update Zabbix for new doc format.
Part of zulip#29592.
2024-07-16 14:55:27 -07:00
Niloth P c271cdb113 integration-docs: Update WordPress for new doc format.
Part of zulip#29592.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-07-16 14:55:02 -07:00
Lauryn Menard 3e100d872a integrations: Remove HookPress events from WordPress webhook.
The HookPress plugin has been unavailable since July 2019, so it
doesn't make sense to continue supporting these events in the
WordPress webhook integration.
2024-07-16 14:55:02 -07:00
Niloth P 1e327cc054 integration-docs: Update Wekan for new doc format.
Part of zulip#29592.
2024-07-16 14:55:02 -07:00
Anders Kaseorg dd0d482d76 narrow: Migrate legacy SQLAlchemy select syntax.
Fixes “sqlalchemy.exc.RemovedIn20Warning: Deprecated API features
detected! These feature(s) are not compatible with SQLAlchemy 2.0.”
with warnings enabled.

https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#change-5284

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 14:50:30 -07:00
Sahil Batra a0efc0ee76 settings: Refetch realm and user profile objects.
This is a temporary solution to make sure stale objects from
cache are not used when previous request updated multiple settings.
2024-07-16 14:04:19 -07:00
Sahil Batra 83c4c27b18 settings: Do not allow invalid combinations for information density settings. 2024-07-16 14:04:19 -07:00
Sahil Batra 17541ddacb settings: Toggle dense mode if font and line height values are changed. 2024-07-16 14:04:19 -07:00
Anders Kaseorg ad9837911e runtornado: Remove explicit AsyncIOMainLoop installation.
Tornado ≥ 5 uses AsyncIOLoop and AsyncIOMainLoop automatically when
appropriate.  Fixes “DeprecationWarning: make_current is deprecated;
start the event loop first” with warnings enabled.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 14:01:32 -07:00
Prakhar Pratyush df7ed437c2 compose: Show banner to explain interleaved view messages fading.
In an interleaved view when composing a message we fade messages
which the user is not replying to, to reduce the chance they send
a message to a recipient they didn't intend to.
Also, it reduces the visual/cognitive processing required
to figure out where their message is going to go.

But, it's not necessarily clear to users that what the
fading means, so this commit adds a one-time compose banner
to explain what's going on the first time this comes up.

Fixes part of #29076.
2024-07-16 13:52:30 -07:00
Prakhar Pratyush 35380b095f compose: Show banner to explain non interleaved view messages fading.
In a non interleaved view when composing a message to another
conversation we fade messages which the user is not replying to,
to reduce the chance they send a message to a recipient they didn't
intend to. Also, it reduces the visual/cognitive processing required
to figure out where their message is going to go.

But, it's not necessarily clear to users that what the
fading means, so this commit adds a one-time compose banner
to explain what's going on the first time this comes up.

Fixes part of #29076.
2024-07-16 13:52:29 -07:00
Alex Vandiver 556b92810b thumbnail: Advertize the thumbnail formats at client registration. 2024-07-16 13:22:15 -07:00
Alex Vandiver 6c624805ce upload: Return the closest-rendered thumbnail. 2024-07-16 13:22:15 -07:00
Alex Vandiver d121a80b78 upload: Serve thumbnailed images. 2024-07-16 13:22:15 -07:00
Alex Vandiver 2e38f426f4 upload: Generate thumbnails when images are uploaded.
A new table is created to track which path_id attachments are images,
and for those their metadata, and which thumbnails have been created.
Using path_id as the effective primary key lets us ignore if the
attachment is archived or not, saving some foreign key messes.

A new worker is added to observe events when rows are added to this
table, and to generate and store thumbnails for those images in
differing sizes and formats.
2024-07-16 13:22:15 -07:00
Alex Vandiver 7aa5bb233d tests: Clarify tests in test_thumbnail.py. 2024-07-16 13:22:15 -07:00
Alex Vandiver 63dbf67120 tests: Ensure we start with a local storage backend. 2024-07-16 13:22:15 -07:00
Alex Vandiver 229dcd0218 upload: Clean up empty directories in local storage. 2024-07-16 13:22:15 -07:00
Alex Vandiver 5f25fae0fa upload: Batch deleting old attachments.
delete_message_attachments() is faster than calling
delete_message_attachment() one-by-one.
2024-07-16 13:22:15 -07:00
Anders Kaseorg 8843f9f62a tests: Remove deprecated SHA1PasswordHasher.
SHA1PasswordHasher will be removed in Django 5.1.  MD5PasswordHasher
will remain for the purpose of speeding up tests.

Followup to commit ac5161f439 (#29620).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-16 13:06:31 -07:00
Alex Vandiver 90701978d9 migrations: Fix emoji re-thumbnailing for imports.
Imports (including conversions from Slack, Hipchat, etc) do not write
the `.original` of the file (nor the still versions, for animations),
only the thumbnailed (and possibly animated) version.  Looking for
`.original` would thus fail, and overwrite the (working) emoji with
the "?" and disable it.

In the event that we do not find the `.original`, fall back to the
bytes from the non-`.original`, to handle these import cases.
2024-07-16 12:57:49 -07:00
Alex Vandiver 1745d87f2e migrations: Add enough information to logs to reverse bad emoji "fixes." 2024-07-16 12:57:49 -07:00
Niloth P 6afed1432c integration-docs: Update Travis CI for new doc format.
Part of zulip#29592.
2024-07-16 09:40:45 -07:00
Niloth P 5db83f87e2 integration-docs: Update Transifex for new doc format.
Part of zulip#29592.
2024-07-16 09:40:45 -07:00
Niloth P 1f4310f893 integration-docs: Update Thinkst for new doc format.
Part of zulip#29592.
2024-07-16 09:40:45 -07:00
Niloth P 0f7b1ba401 integration-docs: Update Taiga for new doc format.
Part of zulip#29592.
2024-07-16 09:40:45 -07:00
Prakhar Pratyush 0de9f7f5cf import_realm: Add support to import 'OnboardingUserMessage' table. 2024-07-16 09:36:02 -07:00
Prakhar Pratyush 762b1c0f5b import_realm: Improve assert statements verifying table name.
This commit improves the assert statements to verify
that the table name is not "usermessage' instead of
verifying that table name doesn't include a substring
"usermessage".

This prep commit will help to avoid assertion error when
importing "onboardingusermessage" table.
2024-07-16 09:36:02 -07:00
Prakhar Pratyush 7d379e00b0 export: Fix 'OnboardingUserMessage' table not being exported.
Earlier, the export tool was logging a warning:
"??? NO DATA EXPORTED FOR TABLE zerver_onboardingusermessage!!!"

This bug was due to not configuring a Config object for
'OnboardingUserMessage' in 'get_realm_config()'.

This commit fixes the bug to export the table properly.
2024-07-16 09:36:02 -07:00
Kenneth Rodrigues 246df3c884 realm: Migrate smaller files to typed_endpoint.
Migrate `realm_domains.py`, `realm_emoji.py`,
`realm_linkifiers.py`, `realm_logo.py`, `realm_playgrounds.py`
to typed_endpoint.
2024-07-15 16:49:32 -07:00
Kenneth Rodrigues ba79d759f1 realm: Migrate to typed_endpoint. 2024-07-15 16:49:32 -07:00
Kenneth Rodrigues a7da24a36f validators: Use cleaner syntax for AfterValidator.
Created a function that returns an `AfterValidator` for `check_int_in`
and `check_string_in` instead of having to use a
`lambda` wraper everytime.
2024-07-15 16:49:32 -07:00
Niloth P 0ec4b0285e integration-docs: Update Zendesk for new doc format.
Part of zulip #29592.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-07-15 16:42:22 -07:00
Niloth P 2c2f530fe0 integration-docs: Update Zapier for new doc format.
Part of zulip #29592.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-07-15 16:42:22 -07:00
Mateusz Mandera d89ab2694f register_server: Call crudini with --inplace when rotating secret key.
This is needed for the command to work in docker-zulip, where the zulip
user may not have write permissions to the parent directory of the
secrets file. By default, crudini creates a temporary file, which makes
the command fail under those conditions.
With --inplace, the secret file gets written to directly without
creation of temporary files.
2024-07-15 13:24:14 -07:00
Vector73 d21ee6fa23 api: Deprecate uri and add url parameter in "/user_uploads" endpoint. 2024-07-14 22:32:36 -07:00
Vector73 a07ebba860 zerver: Replace uri with url in local variables and comments. 2024-07-14 22:30:28 -07:00
Anders Kaseorg fe71eaa8f0 ruff: Fix FURB148 `enumerate` index or value is unused.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:54:32 -07:00
Anders Kaseorg 80d3e60df8 ruff: Fix FURB180 Use of `metaclass=abc.ABCMeta`.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:53:40 -07:00
Anders Kaseorg e3a191b99b ruff: Fix FURB154 Use of repeated consecutive `global`, `nonlocal`.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:53:18 -07:00
Anders Kaseorg 6412c2d630 ruff: Fix FURB142 Use of set.add() in a for loop.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:52:59 -07:00
Anders Kaseorg 1e9b6445a9 ruff: Fix PLR6104 Use `+=` to perform an augmented assignment directly.
This is a preview rule, not yet enabled by default.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:49:51 -07:00
Anders Kaseorg b96feb34f6 ruff: Fix SIM117 Use a single `with` statement with multiple contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-14 13:48:32 -07:00
Tim Abbott b0f144327d narrow: Update new function for modern Python typing syntax.
I apparently needed to be more careful rebasing the typing syntax
modernization work.
2024-07-13 23:06:47 -07:00
Anders Kaseorg 48202389b8 ruff: Bump target-version from py38 to py310.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 3f29bc42b1 ruff: Fix B905 `zip()` without an explicit `strict=` parameter.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 1464009fae ruff: Fix UP038 Use `X | Y` in `isinstance` call instead of `(X, Y)`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 0fa5e7f629 ruff: Fix UP035 Import from `collections.abc`, `typing` instead.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg 531b34cb4c ruff: Fix UP007 Use `X | Y` for type annotations.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg e08a24e47f ruff: Fix UP006 Use `list` instead of `List` for type annotation.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg c2214b3904 test_typed_endpoint: Inline pointless type variable.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg f04e7fc625 test_openapi: Make get_standardized_argument_type understand X | Y.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Anders Kaseorg a2fad74a28 typed_endpoint: Make is_optional understand T | None.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-07-13 22:28:22 -07:00
Aman Agrawal 70be9e8c51 narrow: Implement rendering of `with` narrow operators.
Adds server and web app support for processing the new `with`
search operator.

Fixes part of #21505.

Co-authored-by: roanster007 <rohan.gudimetla07@gmail.com>
Co-authored-by: Tim Abbott <tabbott@zulip.com>
2024-07-13 07:18:24 -07:00
bedo bfd54e27b1 custom_profile_fields: Bulk fetch of UserProfile.
bulk fetch query of UserPfrofile against which
user_ids are validated, instead of looping
over user_ids and fetchingeach UserPfrofile resulting
in O(n) queries.
2024-07-13 07:05:13 -07:00
Alya Abbott bd04a30bbc updates: Add update message (level=6). 2024-07-12 16:40:41 -07:00
Alex Vandiver 4df9b03451 migrations: Re-thumbnail and rename existing emoji. 2024-07-12 13:26:47 -07:00
Alex Vandiver 0385e5bab9 emoji: Store in S3 with a long public cache-control. 2024-07-12 13:26:47 -07:00
Alex Vandiver 0442e95276 emoji: Use a non-predictable filename.
We use a truncated SHA256 of the id and a server-side secret to make
emoji have non-guessable filenames, while also making collisions
unlikely.

We also adjust the Slack import to use the same SHA-based name,
instead of taking the same name as it had in Slack.
2024-07-12 13:26:47 -07:00
Alex Vandiver 262689da76 thumbnail: Fix MAX_EMOJI_GIF_FILE_SIZE_BYTES check to be post-resize.
This check was intended to check the post-resized image size, not the
pre-resized image.
2024-07-12 13:26:47 -07:00
Alex Vandiver 544d3df057 thumbnail: Stop applying MAX_EMOJI_GIF_FILE_SIZE_BYTES before resizing.
b14a33c659 attempted to make the 128k limit apply _after_ resizing,
but left this check, which examines the pre-resized image size.
2024-07-12 13:26:47 -07:00
Alex Vandiver 54f2fabac0 thumbnail: Still emoji are always pngs. 2024-07-12 13:26:47 -07:00
Alex Vandiver 6110ac7339 emoji: Use a transaction, rather than create and delete-on-fail. 2024-07-12 13:26:47 -07:00
Alex Vandiver 2b3da0e70f fixup! thumbnail: Fix MAX_EMOJI_GIF_FILE_SIZE_BYTES check to be post-resize. 2024-07-12 13:26:47 -07:00
Alex Vandiver f6b99171ce emoji: Derive the file extension from a limited set of content-types.
We thumbnail and serve emoji with the same format as they were
uploaded.  However, we preserved the original extension, which might
mismatch with the provided content-type.

Limit the content-type to a subset which is both (a) an image format
we can thumbnail, and (b) a media format which is widely-enough
supported that we are willing to provide it to all browsers.  This
prevents uploading a `.tiff` emoji, for instance.

Based on this limited content-type, we then reverse to find the
reasonable extension to use when storing it.  This is particularly
important because the local file storage uses the file extension to
choose what content-type to re-serve the emoji as.

This does nothing for existing emoji, which may have odd or missing
file extensions.
2024-07-12 13:26:47 -07:00
Alex Vandiver 62a0611ddb emoji: Pass down content-type, rather than guessing from extension. 2024-07-12 13:26:47 -07:00
Alex Vandiver f3473defe1 transfer: Log when local emoji files are missing. 2024-07-12 13:26:47 -07:00
Alex Vandiver fa28e3aa0f tests: Split up test_upload.EmojiTest into test_thumbnail. 2024-07-12 13:26:47 -07:00
PieterCK f5a834c2d6 integration-docs: Update Jotform for new doc format.
Part of #29592.
2024-07-12 13:21:07 -07:00
PieterCK e034a715cb integration-docs: Update JSON formatter for new doc format.
Part of #29592.
2024-07-12 13:21:07 -07:00
PieterCK ecbbf53c0e integration-docs: Update Greenhouse recruitment for new doc format.
Part of #29592.
2024-07-12 13:21:07 -07:00
PieterCK 0e967e4633 integration-docs: Update Harbor for new doc format.
Aside from updating the doc structure, this commit
also rephrases step 4 to make it read better. The
list of supported events in step 5 is also removed.

Part of #29592.
2024-07-12 13:21:07 -07:00
Sahil Batra ffb7744974 settings: Allow settings to be set to system groups only in production.
We use the already existing server level setting to only allow
settings to be set to system groups, not a named user defined
group as well, in production. But we allow to settings to be set
to any named or anonymous user group in tests and development server.

"can_mention_group" setting can be set to user defined groups
because some of the realms already do that in production.

The existing server level setting is also renamed to make it clear
that both user defined groups and anonymous groups are not allowed
if that setting is set to False.

This commit also changes the error message to be consistent for the
case when a setting cannot be set to user defined groups as per
server level and setting and when a particular setting cannot be set
to user defined groups due to the configuration of that particular
setting. For this we add a new class SystemGroupRequiredError in
exceptions.py so that we need not re-write the error message in
multiple places.
2024-07-12 09:51:49 -07:00
Niklas Fiekas af3a2500df
email_mirror: Also strip "AW:" from subject.
Besides "RE:" and "FWD:", "AW:" (from German "Antwort" for answer) is
another common prefix in email subjects. Let email_mirror automatically
remove it as well.
2024-07-11 11:16:24 -07:00
Alex Vandiver d5a4941691 django: Switch to .alias() instead .annotate() where possible.
When using the sub-expression purely for filtering, and not for
accessing the value in the resultset, .alias() is potentially faster
since it does not pull the value in as well.
2024-07-11 09:26:23 -07:00
Alex Vandiver e1a9473bd6 thumbnail: Log and revert to gravatar on migration failure.
This is preferable to leaving the user with a broken avatar image.
2024-07-11 07:31:39 -07:00
Alex Vandiver 536cf0abc8 thumbnail: Be slightly more specific about our exception catches. 2024-07-11 07:31:39 -07:00
Alex Vandiver b53a33a5bc thumbnail: Properly re-derive Content-Disposition for original uploads. 2024-07-11 07:31:39 -07:00
Alex Vandiver 1f9925b1e3 thumbnail: Skip avatars which have already been migrated. 2024-07-11 07:31:39 -07:00
Alex Vandiver a4981c13f7 thumbnail: Skip avatars whose old versions are missing. 2024-07-11 07:31:39 -07:00
Alex Vandiver 4d3d3efcf1 thumbnail: Newly-thumbnailed avatars are all image/png. 2024-07-11 07:31:39 -07:00
Alex Vandiver a9d1adadb7 thumbnail: Add or move progress earlier in loop.
This makes the counts correct and consistent.
2024-07-11 07:31:39 -07:00
Alex Vandiver 382cb5bb13 thumbnail: Lock down which formats we parse. 2024-07-11 07:31:39 -07:00
Alex Vandiver 4bc563128e thumbnail: Use a consistent set of supported image types. 2024-07-11 07:31:39 -07:00
Alex Vandiver a091b9ef81 thumbnail: Provide a more explicit hint than "Bad". 2024-07-11 07:31:39 -07:00
Alex Vandiver 5ae34dc42b slack: Store the content-type of realm icons. 2024-07-11 07:31:39 -07:00
Alex Vandiver ff90e5355f upload: Pass down content-type of realm icon/logo to backend.
This saves having to try to re-derive it from the file extension,
which may be ".original" in some cases.
2024-07-11 07:31:39 -07:00
Alex Vandiver b050c644e3 data_import: Mark downloaded png avatars as image/png.
This means that, when imported, they have the right content-type set
in S3.
2024-07-11 07:31:39 -07:00
Alex Vandiver 51766271a1 data_import: Remove a reference to removed code.
The only use of the code was removed in c685d36821, and the code
itself was removed in 4206e5f00b.
2024-07-11 07:31:39 -07:00
Vector73 a43d8159a9 exceptions: Update DM permission error string. 2024-07-10 18:43:50 -07:00
Prakhar Pratyush d92eb77c98 add_subscriptions_backend: Add 'transaction.atomic' decorator.
This commit adds an 'transaction.atomic' decorator to the
'add_subscriptions_backend' view thus making the db operations
within the view atomic and helps to avoid race between events sent.

In tests where we make POST requests to this view, we have
wrapped the API calls with a transaction.atomic() context
manager. It helps us with NOT rolling back the entire test
transaction due to error responses.
2024-07-10 10:50:37 -07:00
Prakhar Pratyush 9c614531fb test_subs: Use 'common_subscribe_to_streams' helper function.
In 'test_subs' we were making POST request to add
subscrption using 'self.api_post'.

This commit updates the code to use the test helper
function 'common_subscribe_to_streams' instead.

This prep commit will also help us to avoid adding
'transaction.atomic' context manager to these API calls
individually in the case of error response in the next commit.
2024-07-10 10:50:37 -07:00
Tim Abbott c9af4d571b onboarding: Fix whitespace for code block examples. 2024-07-09 16:42:23 -07:00
Lauryn Menard 53c575d8cb integration-docs: Update Linear for new doc format.
Part of #29592.
2024-07-09 13:41:06 -07:00
Lauryn Menard 0fa9b94480 integration-docs: Update Lidarr for new doc format.
Part of #29592.
2024-07-09 13:41:06 -07:00
Lauryn Menard 5c7ad8be12 integration-docs: Update Librato for new doc format.
Part of #29592.
2024-07-09 13:41:06 -07:00
Lauryn Menard bd9a10cfbb integration-docs: Update Jira for new doc format.
Part of #29592.
2024-07-09 13:41:06 -07:00
Lauryn Menard 95b74a7bfb integration-docs: Update Intercom for new doc format.
Part of #29592.
2024-07-09 13:41:06 -07:00
Alya Abbott b6e2a1b50d onboarding: Update Welcome Bot's responses to commands. 2024-07-09 13:08:02 -07:00
codewithnick b926ae0904 onboarding: Modify message that welcome bot does not understand.
Previously the bot sent bot commands whenever an undefined message
was sent by the user. This commit intends to fix the problem so that
the bot will only respond to the first message it does not understand
and not reply to any future undefined messages.

Fixes part of #30049.
2024-07-09 13:08:02 -07:00
codewithnick 4b79a77427 onboarding: Change default reply message of welcome bot.
The reply given by welcome bot when it does not understand a message,
sent by the user is changed, so that the bot is more clear to the user.

Fixes part of #30049.
2024-07-09 13:08:02 -07:00
codewithnick e41fda78d4 onboarding: Remove welcome bot message.
The previous message that welcome bot sent with
the bot commands has been removed in order to be more
clear and concise to the user. Previously weclome bot
also sent bot commands in intial message, this commit
intends to remove those commands.

Fixes part of #30049.
2024-07-09 13:08:02 -07:00
Vector73 121043b6f2 settings: Remove private_message_policy setting.
Removes `private_message_policy` field from both the API and
the backend.
2024-07-08 19:34:17 -07:00
Vector73 6098c2cebe settings: Add two realm settings to restrict direct messages.
Fixes #24467.
2024-07-08 19:34:17 -07:00
Vector73 318d3e3cca user_groups: Add `is_any_user_in_group` function in `user_groups.py`. 2024-07-08 19:34:17 -07:00
PieterCK 374210fe7b integration-docs: Update Groove for new doc format.
Part of #29592.
2024-07-08 16:29:33 -07:00
PieterCK 3e83de7d5a integration-docs: Update Gosquared for new doc format.
Part of #29592.
2024-07-08 16:29:33 -07:00
PieterCK 58b32a09a8 integration-docs: Update Gogs for new doc format.
Part of #29592.
2024-07-08 16:29:33 -07:00
PieterCK 98f44262df integration-docs: Update Gitea for new doc format.
Part of #29592.
2024-07-08 16:29:33 -07:00
adnan-td c1421f0847 integration-docs: Update IFTTT for new doc format.
Part of #29592.
2024-07-08 16:29:02 -07:00
adnan-td 928c84233f integration-docs: Update HelloSign for new doc format.
Part of #29592.
2024-07-08 16:29:02 -07:00
adnan-td a35c9795fa integration-docs: Update Hello World for new doc format.
Part of #29592.
2024-07-08 16:29:02 -07:00
adnan-td 6846d2a520 integration-docs: Update Heroku for new doc format.
Part of #29592.
2024-07-08 16:29:02 -07:00
adnan-td a34e50bbae integration-docs: Update Insping for new doc format.
Part of #29592.
2024-07-08 16:29:02 -07:00
Niloth P d763131fde integration-docs: Update Updown for new doc format.
Part of zulip#29592.
2024-07-08 16:28:24 -07:00
Niloth P 4759b6aed7 integration-docs: Update UptimeRobot for new doc format.
Part of zulip#29592.
2024-07-08 16:28:24 -07:00
Niloth P 9c520ae23f integration-docs: Update Github Sponsors for new doc format.
Part of zulip#29592.
2024-07-08 16:28:24 -07:00
Niloth P b7f263cd5c integration-docs: Update Github for new doc format.
Part of zulip#29592.
2024-07-08 16:28:24 -07:00
bedo 88a0a3061e corporate-upgrade: Migrate to @typed_endpoint.
migrate the following endpoints from @has_request_variables
to @typed_endpoint :

- upgrade()
- remote_realm_upgrade()
- upgrade_page()
- remote_server_upgrade()
2024-07-08 16:14:57 -07:00
Lauryn Menard fb3ec350c7 help-settings-links: Fix for new invitations section in users tab.
Updates the instructions for `{settings_tab|invitations}` to work
like the deactivated users link, and updates the one article that
currently uses it: `help/invite-new-users.md`.
2024-07-08 13:56:18 -07:00
Aman Agrawal 9fc6b93347 user_settings: Add option to configure channel click behavior.
This commit does not implement the setting's behavior, just the API
change and settings UI.
2024-07-08 13:56:13 -07:00
Lauryn Menard 7e7e6b7ba9 integration-docs: Clarify use of event-filtering-additional-feature.
Using an if block when {!event-filtering-additional-feature.md!}
is used implied that we plan to have this in all webhook doc .md
files.

But, it actually makes sense to only use this macro when the
webhook integration actually supports event filtering.
2024-07-08 13:51:50 -07:00
Prakhar Pratyush 83414db72e settings: Add 'web_navigate_to_sent_message' setting.
In a2ef1d7e93, we made changes so
that when you send a message, your view jumps to the conversation
where you sent it.

For some users it was an improvement, few reported that it
disrupts their workflows.

This prep commit adds a setting which will be used to allow users
to decide whether to automatically go to conversation where they
sent a message.
2024-07-08 13:00:12 -07:00
Prakhar Pratyush 5bb66e6c99 onboarding: Update initial direct message content.
This commit updates the Welcome Bot's initial
direct message content.

We inform about the tracked onboarding messages
via direct message only if it exists.

Fixes #30051.
2024-07-08 10:32:19 -07:00
Alya Abbott 721c44df10 portico: Update quotes on home page. 2024-07-07 22:44:04 -07:00
roanster007 02d0566dc5 refactor: Rename `Huddle` Django model class to `DirectMessageGroup`.
This commit renames the "Huddle" Django model class to
"DirectMessageGroup", while maintaining the same table --
"zerver_huddle".

Fixes part of #28640.
2024-07-07 21:31:30 -07:00
Alex Vandiver 79f858b4b8 upload: Pass bytes to create_attachment.
This will be used to analyze the bytes for image metadata.
2024-07-07 14:40:07 -07:00
Alex Vandiver f97a30f240 upload: Reorder arguments to parallel upload_message_attachment. 2024-07-07 14:40:07 -07:00
Alex Vandiver f52a93bc14 upload: Stop requiring callers pass in the file size.
This can be calculated because we have the contents.
2024-07-07 14:40:07 -07:00
Alex Vandiver 58a9fe9af1 upload: Drop unused parameters to upload_message_attachment. 2024-07-07 14:40:07 -07:00
Alex Vandiver 0a296b2a6e upload: Start storing content-type for new uploads. 2024-07-07 14:40:07 -07:00
Alex Vandiver 45afdc2d35 attachment: Add a nullable content-type.
This will be backfilled in a future commit, to prevent walking the
Attachments more than once.
2024-07-07 14:40:07 -07:00
Alex Vandiver e29a455b2d avatars: Encode version into the filename.
Hash the salt, user-id, and now avatar version into the filename.
This allows the URL contents to be immutable, and thus to be marked as
immutable and cacheable.  Since avatars are served unauthenticated,
hashing with a server-side salt makes the current and past avatars not
enumerable.

This requires plumbing the current (or future) avatar version through
various parts of the upload process.

Since this already requires a full migration of current avatars, also
take the opportunity to fix the missing `.png` on S3 uploads (#12852).

We switch from SHA-1 to SHA-256, but truncate it such that avatar URL
data does not substantially increase in size.

Fixes: #12852.
2024-07-07 14:40:07 -07:00
Alex Vandiver feca9939bb s3: Support setting a cache-control on uploads. 2024-07-07 14:40:07 -07:00
Alex Vandiver 6258817bfd s3: Stop setting empty Content-Disposition header. 2024-07-07 14:40:07 -07:00
Pedro Almeida ddfc2d230f resolve_topic: Prevent incorrect notification during message move.
This commit fixes the bug where the "topic unresolved" notification
is wrongly triggered when moving a message between a resolved and
unresolved topic, except for when the topics have the same name.

To resolve this issue, the commit ensures that resolved/unresolved
notifications are not sent if a message has been moved to a new
topic. This is achieved by comparing the names of the old and new
topics without considering the "resolved prefix".

The commit also accounts for the scenario where `new_topic_name`
has been truncated, indicating that it was resolved and the name
had to change to accommodate the "resolved prefix".

This solution does not try to specially handle the possible case that
a stream has two topics with the same name, even if one is resolved
and another unresolved.

Fixes #29007.
2024-07-07 11:38:14 -07:00
Pedro Almeida f4ca8025da move_stream: Trigger only move notification.
This commit updates the code so that only the moved notification is
triggered when moving a message between a resolved and unresolved
topic in different streams or when moving a topic itself.

This change takes place even when both stream change and resolve
or unresolve a topic takes place in the same API request, as we
now consider it only a move operation.

This fixes a case where a message is moved between topics that
have the same name, but one resolved and another unresolved and
in different streams. Previously a resolved or unresolved
notification would also be sent. Now, this will not happen,
ensuring only the move operation is notified.

Fixes part of #29007.
2024-07-07 11:38:14 -07:00
Tim Abbott 0ea162c5fb migrations: Fix a duplicate migration number. 2024-07-05 17:26:27 -07:00
Shashank Singh 4cce94b667 invites: Add option to receive notification on accepted invitations.
Previously, when a referrer's invitation to Zulip was accepted,
they got a notification from notification-bot indicating
their invitation has been accepted.

This commit adds an option for referrer to decide
whether he wants to receive the direct notification
from the notification-bot.

Fixes: #20398
2024-07-05 17:14:45 -07:00
PieterCK 0587855210 integration-docs: Update Grafana for new doc format.
Reformatted Grafana doc into 3 part sections. One for
creating the Zulip bot and webhook URL. One for setting
up Grafana version 8.3 and above and the last one for
setting up Grafana version 8.2 and below.

Part of #29592.

Co-authored-by: Lauryn Menard <lauryn@zulip.com>
2024-07-05 16:22:19 -07:00
Kenneth Rodrigues fe2097fd26 zerver: Migrate several small files to typed_endpoint.
Migrate "submessage.py", "thumbnail.py", "tutorial.py", "zephyr.py" and
"dev_login.py" to `typed_endpoint`.
2024-07-05 16:18:57 -07:00
Kenneth Rodrigues defd6748d6 integrations: Convert to typed endpoint.
Migrate `integrations.py` and `webhooks` to typed_endpoint.
2024-07-05 16:18:27 -07:00
Prakhar Pratyush ed605328fb onboarding: Mark a few onboarding messages as starred.
To improve onboarding experience following onboarding
messages are marked as starred:
* First message in each onboarding topic.
* Initial DM sent by Welcome bot

Note: The onboarding topic messages needs to be tracked
in 'OnboardingUserMessage' model to get starred.

Fixes #29298.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush 3c5dc73f50 create_user: Mark only tracked onboarding topic messages as unread.
We give the user some messages in their feed, so that they can
learn how to use the home view in a realistic way.

For realms having older onboarding messages, we mark the very
most recent messages as unread.

This commit updates the logic to ONLY mark the tracked onboarding
messages (if present) i.e. messages tracked in 'OnboardingUserMessage'
as unread.

Fixes part of #29298.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush ffc95204be create_user: Send initial realm messages before process_new_human_user.
This prep commit reorders code blocks in 'do_create_user'
to call the function 'send_initial_realm_messages' before
'process_new_human_user'.

This is required because for the first user (who creates realm)
we need to send the onboarding messages before adding message
history (performed by process_new_human_user) otherwise messages
won't be starred as required.

This commit also takes care of NOT marking the onboarding
messages as 'historical' for the realm creator.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush 1fd6b09598 onboarding: Create OnboardingUserMessage rows during realm creation.
This commit updates the 'send_initial_realm_messages' function
to create OnboardingUserMessage rows corresponding to the
initial realm messages we send.

It helps to distinguish new onboarding messages from the
onboarding messages in the old realms.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush fb836a4f0a onboarding: Add 'OnboardingUserMessage' model.
This prep commit adds a new OnboardingUserMessage model
that will be used to mark the new onboarding messages
for new users as unread and the first message of each
onboarding topic as starred.

This table won't include the old onboarding messages.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush 485a8a1421 add_new_user_history: Add missing historical flags.
This commit adds the missing historical flag to recent
messages added in a new user's feed.

Reason:
* User didn't receive when they were sent, so semantically
the 'historical' flag should be present.
* It helps to avoid the "You (un)subscribed to.." bookend
while reading older messages.
The bookend appears whenever the historical flag flips
between adjacent messages. Earlier, the bookend was visible
between 'recent messages' and 'older messages'. This makes
sure that the bookend is visible only at the moment new
message is sent by the user after account creation.
2024-07-05 15:39:32 -07:00
Prakhar Pratyush f5a0755f16 create_user: Rename constants used in 'add_new_user_history'.
This commit renames the constants:
MAX_NUM_ONBOARDING_MESSAGES to MAX_NUM_RECENT_MESSAGES
MAX_NUM_ONBOARDING_UNREAD_MESSAGES to MAX_NUM_RECENT_UNREAD_MESSAGES
ONBOARDING_RECENT_TIMEDELTA to RECENT_MESSAGES_TIMEDELTA

The term 'onboarding' is preffered to be used for the
new messages sent during realm creation or new user creation.
These constants are related to already present recent messages.
2024-07-05 15:39:32 -07:00
Lauryn Menard 6dfe063c91 integration-docs: Add comma to all uses of "e.g." in integration docs. 2024-07-05 15:36:24 -07:00
Lauryn Menard 7ec3f235de integration-docs: Update Prometheus Alertmanager for new doc format.
Also, revises text of `name` and `desc` URL parameters to be clearer
and note the default values that are used for the integration.

Part of #29592.
2024-07-05 15:27:06 -07:00
Lauryn Menard 307681d718 integration-docs: Update CircleCI for new doc format.
Part of #29592.
2024-07-05 15:20:59 -07:00
Lauryn Menard 66b26738ea integration-docs: Update Clubhouse for new doc format.
Part of #29592.
2024-07-05 15:20:59 -07:00
Lauryn Menard 0698a90b36 integration-docs: Update Canarytoken for new doc format.
Part of #29592.
2024-07-05 15:20:59 -07:00
Lauryn Menard 6e8e0a5455 integration-docs: Update Bitbucket Server for new doc format.
Part of #29592.
2024-07-05 15:20:59 -07:00
Lauryn Menard 2e972db84f integration-docs: Update Bitbucket Cloud SAAS for new doc format.
Part of #29592.
2024-07-05 15:20:59 -07:00
roanster007 52692a6448 refactor: Rename `huddle` to `direct_message_group` in non API.
This commit performs a sweep on the first batch of non API
files to rename "huddle" to "direct_message_group`.

It also renames variables and methods of type -
"huddle_message" to "group_direct_message".

This is a part of #28640
2024-07-04 07:56:31 -07:00
Alya Abbott cb1531b061 help: Don't link to the direct message feed.
We should teach users how to get there.
2024-07-03 16:30:52 -07:00
Aman Agrawal 4b091aafc2 makemessages: Allow excluding whitespace before `*inline` custom tag.
This otherwise throws capitalization error that handlebars are included
in the translated string when trying to use {{~#*inline}}.
2024-07-03 10:38:35 -07:00
Lauryn Menard c9c8bcad57 integration-docs: Update Front for new doc format.
Part of #29592.
2024-07-02 16:07:22 -07:00
Lauryn Menard 3ae09fe880 integration-docs: Update Freshstatus for new doc format.
Part of #29592.
2024-07-02 16:07:22 -07:00
Lauryn Menard 5c2b15d68f integration-docs: Update Freshping for new doc format.
Part of #29592.
2024-07-02 16:07:22 -07:00
Lauryn Menard dc28117384 integration-docs: Update Flock for new doc format.
Part of #29592.
2024-07-02 16:07:22 -07:00
Tim Abbott da6c2e0c3c settings: Rename Deactivated users settings subtab.
Includes help center updates to better link to the tab.

Fixes https://chat.zulip.org/#narrow/stream/9-issues/topic/deactivated.20users.20tab/near/1843114
2024-07-02 13:53:31 -07:00
Sahil Batra 5ef14c3a8e users: Fix uploading user avatars.
Due to recent refactoring in 9fb03cb2c7, a user could not
upload avatar if the server uses local upload backend and there
was already an avatar file for that user.

This commit fixes it to just check if there exists a file only
when importing and not when the user is actually trying to
change the avatar.

Fixes #30676.
2024-07-02 13:26:21 -07:00
Lauryn Menard 250f68ad25 integration-docs: Update Beeminder for new doc format.
Part of #29592.
2024-07-02 12:50:25 -07:00
Lauryn Menard f442b8baaf integration-docs: Update Basecamp for new doc format.
Part of #29592.
2024-07-02 12:50:25 -07:00
Lauryn Menard eb2bcd5808 integration-docs: Update AppVeyor for new doc format.
Part of #29592.
2024-07-02 12:50:25 -07:00
Lauryn Menard ecd41d0f6d integration-docs: Update Appfollow for updated doc format.
Part of #29592.
2024-07-02 12:50:25 -07:00
Lauryn Menard 558b3e7cba integration-docs: Update Ansible Tower for new doc format.
Part of #29592.
2024-07-02 12:50:25 -07:00
Lauryn Menard 80b86c7b16 documentation: Fix "you you" typos in docs/comments/logs. 2024-07-02 11:57:44 -07:00
Lauryn Menard 1140e8402b integrations: Remove Google Code-in integration.
The Google Code-in program ended in 2019/2020 so there is no need
to maintain this Webhook integration.
2024-07-02 10:04:34 -07:00
Vector73 ff0cc79fd5 api_docs: Add test to check links of pages in the sidebar. 2024-07-01 16:42:11 -07:00
Lauryn Menard c8fdfc92a1 integration-docs: Update Freshdesk for new doc format.
Revises doc to have three instructions sections. One for creating
the Zulip bot and webhook URL. One for adding notifications for
new Freshdesk tickets. And one for getting notifications for
existing Freshdesk tickets.

Part of #29592.
2024-07-01 15:11:30 -07:00
Lauryn Menard 6f14e0b5f7 integration-docs: Update Errbit for new doc format.
Part of #29592.
2024-07-01 13:27:45 -07:00
Lauryn Menard ddd94c5904 integration-docs: Update Dropbox for new doc format.
Part of #29592.
2024-07-01 13:27:45 -07:00
Lauryn Menard 56d6080dd0 integration-docs: Update Delighted for new doc format.
Part of #29592.
2024-07-01 13:27:45 -07:00
Lauryn Menard dcfec61365 integration-docs: Update Crashlytics for new doc format.
Part of #29592.
2024-07-01 13:27:45 -07:00
Lauryn Menard 45dd3d52ef integration-docs: Update Codeship for new doc format.
Part of #29592.
2024-07-01 13:27:45 -07:00
Sayam Samal 073b116c55 zcommand: Rename night->dark and day->light in the color scheme backend.
As a follow up for f49a11c810, this
commit standardizes the naming of the day and night themes to light
and dark, respectively in the backend. This makes the backend
consistent with the naming used in the frontend and UI.

This also solves a regression introduced in
f49a11c810, where the frontend was sending
"/light" and "/dark" commands to the backend, but the backend was
expecting "/day" and "/night" commands.
2024-06-30 22:45:26 -07:00
Anders Kaseorg 5af8dfda3e ruff: Fix FURB167 Use of regular expression alias.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-30 22:37:15 -07:00
Anders Kaseorg 6a73006723 ruff: Fix FURB105 Unnecessary empty string passed to `print`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-30 22:37:15 -07:00
Prakhar Pratyush fb5b721421 onboarding_step: Don't show 'visibility_policy_banner' to new users.
Earlier, a one-time 'visibility_policy_banner' was displayed to
existing as well as new users to inform them about the new
"follow/unmute topics" feature.

It makes sense to educate only the existing Zulip users about
the new feature using this banner. New users don't need to know
about following topics right away.

This commit makes changes to NOT show the banner to new users.

This will also help to avoid banner overload in the new user
experience.

Fixes #30615.
2024-06-28 10:41:48 -07:00
Vector73 43eb591ed6 api_docs: Rename titles of endpoint docs and add missing links.
Renames titles of endpoints to make them consistent across left sidebar and
main documentation page.

Adds missing links in API documentation.
2024-06-28 10:07:27 -07:00
Lauryn Menard e68da43b67 api-docs: Update /users/me/presence documentation.
Revises descriptive text for this endpoint to be clearer and more
robust.

Fixes issues with the rendering of the modern and legacy formats
for the user presence data.
2024-06-27 19:15:08 -07:00
Lauryn Menard ae4ba8761c api-docs: Update "presences" object description in register response.
Updates the description for `slim_presence` parameter as it relates
to the `presences` object that's returned in the response.

Adds the modern format for the `presences` object in the response
as that is what will be returned when the parameter above is True.
2024-06-27 19:15:08 -07:00
Lauryn Menard d66ee33859 api-docs: Add ModernPresenceFormat schema. 2024-06-27 19:15:08 -07:00
Lauryn Menard 22fcf6cede api-docs: Rename Presence schema to LegacyPresenceFormat. 2024-06-27 19:15:08 -07:00
Anders Kaseorg b115d44b6a requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-27 15:31:43 -07:00
Anders Kaseorg 997dbcd5d9 clubhouse: Fix PLR1704 Redefining argument with the local name `action`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-27 15:31:43 -07:00
Anders Kaseorg 9fdc0eeb8e scim: Fix PLR1704 Redefining argument with the local name `path`.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-27 15:31:43 -07:00
Tim Abbott 47db96d730 migrations: Add migration for RealmAuditLog ordering.
Because this might impact how past RealmAuditLog-reading migrations
are processed, I think it's better to this one as an actual Django
migration, even though it has no actual SQL to run.
2024-06-26 17:18:50 -07:00
Lauryn Menard 673a01ea0c realm-deactivation: Send email to owners as part of deactivation.
Creates a new "realm_deactivated" email that can be sent to realm
owners as part of `do_deactivate_realm`, via a boolean flag,
`email_owners`.

This flag is set to `False` when `do_deactivate_realm` is used for
realm exports or changing a realm's subdomain, so that the active
organization owners are not emailed in those cases.

This flag is optional for the `deactivate_realm` management command,
but as there is no active user passed in that case, then the email
is sent without referencing who deactivated the realm.

It is passed as `True` for the support analytics view, but the email
that is generated does not include information about the support
admin user who completed the request for organization deactivation.

When an active organization owner deactivates the organization, then
the flag is `True` and an email is sent to them as well as any other
active organization owners, with a slight variation in the email text
for those two cases.

Adds specific tests for when `email_owners` is passed as `True`. All
existing tests for other functionality of `do_deactivate_user` pass
the flag as `False`.

Adds `localize` from django.util.formats as a jinja env filter so
that the dates in these emails are internationlized for the owner's
default language setting in the "realm_deactivated" email templates.

Fixes #24685.
2024-06-26 16:48:18 -07:00
Alex Vandiver 2eaf098c5d upload: Content-type is always defined. 2024-06-26 16:43:11 -07:00
Alex Vandiver 17fb23746f upload: Move methods into zerver.lib.upload from .base. 2024-06-26 16:43:11 -07:00
Alex Vandiver c826d80061 upload: Factor out common code into zerver.lib.upload. 2024-06-26 16:43:11 -07:00
Alex Vandiver c0d9a95aa8 email_mirror: Fix a misleading variable name. 2024-06-26 16:43:11 -07:00
Alex Vandiver 5cd10ce51d s3: Allow setting a CloudFront URL prefix for avatar and emoji images. 2024-06-26 16:43:11 -07:00
Alex Vandiver 08b24484d1 upload: Remove redundant acting_user_profile argument.
This argument, effectively added in 9eb47f108c, was never actually
used.
2024-06-26 16:43:11 -07:00
Alex Vandiver 41d1b417e7 avatars: Clean up now-irrelevant assumptions.
This x=x hack was removed in 3bd3173b1f.
2024-06-26 16:43:11 -07:00
Alex Vandiver fb929ca218 thumbnailing: Remove unnecessary third return value from resize_emoji. 2024-06-26 16:43:09 -07:00
Alex Vandiver 0070b5da78 tests: Switch from PIL to pyvips. 2024-06-26 16:42:59 -07:00
Alex Vandiver b14a33c659 thumbnailing: Switch to libvips, from PIL/pillow.
This is done in as much of a drop-in fashion as possible.  Note that
libvips does not support animated PNGs[^1], and as such this
conversion removes support for them as emoji; however, libvips
includes support for webp images, which future commits will take
advantage of.

This removes the MAX_EMOJI_GIF_SIZE limit, since that existed to work
around bugs in Pillow.  MAX_EMOJI_GIF_FILE_SIZE_BYTES is fixed to
actually be 128KiB (not 128MiB, as it actually was), and is counted
_after_ resizing, since the point is to limit the amount of data
transfer to clients.

[^1]: https://github.com/libvips/libvips/discussions/2000
2024-06-26 16:42:57 -07:00
Alex Vandiver 9fb03cb2c7 upload: Factor out common avatar logic. 2024-06-26 16:38:01 -07:00
Alex Vandiver d92993c972 upload: Factor out common emoji logic. 2024-06-26 16:38:01 -07:00
Alex Vandiver 872e2ad2cd models: Enforce an ordering to on audit logs to solve a flaky test.
Multiple events may be created with the same timestamp; we add a
default ordering by "id" such that these have a consistent ordering in
tests (to solve for flaky tests), as well as in the internal API.
2024-06-26 16:38:00 -07:00
Lauryn Menard d356577c29 api-docs: Add example for BigBlueButton meeting_name parameter.
Also adds a link to the BigBlueButton integration documentation in
the endpoint's main description.
2024-06-26 10:04:09 -07:00
Vector73 6aa3a55796 openapi: Reorder group-setting-value oneOf clause. 2024-06-25 15:37:31 -07:00
Vector73 b285e631e9 api_docs: Fix missing content from API documentation. 2024-06-25 15:37:31 -07:00
Kenneth Rodrigues 29cbdd5050 documentation: Convert to typed_endpoint. 2024-06-25 11:07:03 -07:00
Prakhar Pratyush 1b9143b511 onboarding: Update message contents to use triple quotes.
This commit updates the message contents to use triple quotes.

It helps to maintain a consistent pattern for multi-line strings
in 'zerver/lib/onboarding.py' file and it'll be a lot more
convenient to edit in that form in the future.
2024-06-24 10:44:30 -07:00
Lauryn Menard 3f8d5ea0ab api-docs: Update message_ids description in update_message event.
Clarifies that the ID of the edited message will always be included
in the `message_ids` list, even if there are no topic or channel
updates to be made.
2024-06-24 10:36:49 -07:00
Sahil Batra b44d30add7 events: Optimize computing can_create_streams and related fields.
Currently, for computing fields like can_create_public_streams
and can_create_private_steams fields, is_user_in_group is called
to check whether the user is part of the group which has the
permission. This means that there will be one DB query for each
field.

To optimize this, we now first fetch all the groups that the
user is member of, including the anonymous groups which are
used for settings, such that we can then just check whether
the user is part of the group which has the permission meaning
we would need only one query to compute all the fields.

This would be helpful when settings for other similar fields
will also be migrated to groups framework.
2024-06-24 10:00:08 -07:00
Mateusz Mandera 9f24b30e78 test_push_notifs: Adjust .filter() in test_analytics_api.
The test asserts entries about the zephyr realm anyway. The reason the
filter hasn't been limiting the query to zephyr is that we might
simultaneously want to ensure no other realms received changes - but
that doesn't seem quite right, given that the test doesn't dilligently
set up the initial conditions for all realms to have control over what
exactly happens with them. That makes this logic pretty fragile since if
some new (potentially unrelated) changes to Realm/RemoteRealm initial
state make it so some updates to other realms occur during the early
analytics upload calls in the test, the remote_audit_logs asserts here
will break in a very annoying to debug way.

If we want this query to be general, without limiting to the zephyr
realm, we could alter the test a bit to set up initial conditions
precisely.
2024-06-23 07:44:11 -07:00
Mateusz Mandera f8616fa013 analytics: Send ZULIP_MERGE_BASE to the bouncer. 2024-06-23 07:44:11 -07:00
Anders Kaseorg f7eecb0e03 sentry: Untangle from page_params.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2024-06-22 18:53:56 -07:00
tnmkr 9af71c690f custom_profile_fields: Fix "required" field in OpenAPI documentation.
This is not an optional field and should be present in all API responses
of custom profile field object.
2024-06-21 16:32:15 -07:00
tnmkr b6231f5568 zulip.yaml: Fix typo in IncludeCustomProfileFields description. 2024-06-21 16:32:15 -07:00
Vector73 5230aada2b python_examples: Validate error responses.
Adds assert statements to validate error response for "400" error code. The
current validation using `validate_against_openapi_schema` doesn't work for
"400" responses.

Adds separate functions to validate "200" and "400" responses and removes `validate_response_result`.
2024-06-21 09:49:07 -07:00
Alex Vandiver be9369541a create_user: Copy the avatar before announcing it. 2024-06-20 23:06:08 -04:00