Commit Graph

14769 Commits

Author SHA1 Message Date
Mateusz Mandera 4102816240 upload: Pass the target realm to create_attachment.
The target realm was not being passed to create_attachment in
upload_message_file implementations. This was a bug in the edge-case of
cross-realm messages - in particular, causing a bug in the email
gateway:
When an email with an attachment is sent, the message is mirrored to
Zulip with Email Gateway Bot as the message sender and uploader of the
attachment. Due to the realm not being passed to create_attachment, the
Attachment would get created with .realm being the system bot realm,
making the attachment inaccessible under some conditions due to failing
the following condition check (that's expected to pass, provided that
the .realm is set correctly):
```
    if (
        attachment.is_realm_public
        and attachment.realm == user_profile.realm
        and user_profile.can_access_public_streams()
    ):
        # Any user in the realm can access realm-public files
        return True
```
2022-01-27 17:23:44 -08:00
Lauryn Menard a832a8a3af api_docs: Fix enum strings in parameters to have quotes.
Fixes the rendering of enums to show strings with quotation marks,
while integers will continue to be rendered without quotation marks.

This allows for an empty string to be passed as an enum value and be
rendered as such in the documentation. Null will be rendered without
quotation marks, like integer values.
2022-01-27 13:03:20 -08:00
Sahil Batra 5e506a833f actions: Use transaction.atomic for do_change_avatar_fields. 2022-01-27 10:33:55 -08:00
Sahil Batra a2df7470d5 actions: Use transaction.atomic for do_change_realm_plan_type. 2022-01-27 10:33:55 -08:00
Sahil Batra f5ea13eea8 actions: Use transaction.atomic for do_change_user_delivery_email. 2022-01-27 10:33:55 -08:00
Lauryn Menard aaa627229e api: Update `update_message` event required fields.
Makes `edit_timestamp` and `user_id` required fields for all
`update_message` events.

Adds `rendering_only` as another required field to signal if
events are only updating the rendered content of the message,
which is currently the case for adding inline url previews.

Updates `test_event.py` so that `do_update_message` and
`do_update_embedded_data` refer to the same testing schema
for `update_message` events, and therefore reflect the same
required fields for the `update_message` event.

The OpenAPI definition for `update_message` events is also
updated to reflect the required field and descriptions of
various properties are updated for the addition of the
`rendering_only` property.
2022-01-26 13:11:26 -08:00
Lauryn Menard 7077871111 api_documentation: Update rate limit error description.
Moves details about the rate limit error object and handling to
the OpenAPI documentation description for that common error.

Previously, this information was on the general rest error
handling documentation page without clear connection to the
specific rate limit error.

Fixes a typo in the changelog (feature 36) for that same error
and also fixes a misplaced colon in the description of the error
for missing request parameters.
2022-01-26 12:51:02 -08:00
Lauryn Menard fee7c5712f api_docs: Fix small typos in organizational role description.
Fixes two small typos in the description of the `role` value
returned from the `/get-own-user` endpoint.
2022-01-26 10:36:51 -08:00
Sharif Naas d560d124a3 python: Replace string concatenations with f-strings. 2022-01-25 17:32:59 -08:00
Mateusz Mandera 40aa156b28 migrations: Fix RealmAuditLog creation in migration 0374. 2022-01-25 17:20:13 -08:00
Lauryn Menard ce49c4ca9e api: Add `subscription_data` details to `/update-subscription-settings`.
Adds detailed definition of objects in the `subscription_data` parameter
array for the `/update-subscription-settings` endpoint.

Fixes #20825. Follow-up to #20409.
2022-01-25 17:19:58 -08:00
Lauryn Menard 7b1597d4cf api_docs: Format required or optional detail of param object field.
Formats and moves whether a field of an object in a request
parameter is required or optional to be in the same location
and have the same formatting as the general api parameter
documentation.

Also formats any examples within the object detailed
description to be the same as the general api parameter
documentation.

Follow up to #20409.
2022-01-25 11:20:18 -08:00
Lauryn Menard e479acc809 api_docs: Add line break before return value description text.
Adds a line break before the descriptive text for return
values and events in the api documentation in order to
help with readability of descriptions with multiple
paragraphs of descriptive text.

Adjustments made to the CSS of list items in unordered
lists to visually group the first paragraph of text
to any following paragraphs or unordered lists.
2022-01-24 10:02:02 -08:00
Anders Kaseorg 4922632601 mypy: Add types-beautifulsoup4.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 23:39:40 -08:00
Anders Kaseorg 2caeb38e9e python: Replace IOError with OSError.
IOError is an alias for OSError in Python ≥ 3.3.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 3437e38612 string_validation: Use set comprehension syntax.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 97e4e9886c python: Replace universal_newlines with text.
This is supported in Python ≥ 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:16:01 -08:00
Anders Kaseorg b729f00fc2 test_upload: Uncomment subTest contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:43 -08:00
Anders Kaseorg 5fb935bf60 python: Replace deprecated urllib3 BACKOFF_MAX.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg 2612f57d51 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg ee71650908 python: Replace requests.packages.urllib3 alias with urllib3.
requests stopped vendoring urllib3 in 2.16.0 (2017-05-26).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg 18aa4f4df4 tornado: Replace deprecated urllib3 method_whitelist.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Eeshan Garg 93329c2d70 send_custom_email: Only send emails to active remote servers. 2022-01-21 15:03:08 -08:00
Tim Abbott 367fed89f2 register_server: Improve HTTP error reporting. 2022-01-21 15:02:30 -08:00
Eeshan Garg fb09c18462 management/commands: Refactor error handling in register_server. 2022-01-21 14:57:09 -08:00
Eeshan Garg bd197886fe management/commands: Refactor logging code in register_server. 2022-01-21 14:57:09 -08:00
Eeshan Garg 3bc0f8c6f9 zilencer: Add endpoint for deactivating remote server registration. 2022-01-21 14:57:04 -08:00
Eeshan Garg 94d00ca942 zilencer: Stop serving requests from deactivated remote servers. 2022-01-21 14:56:04 -08:00
Tim Abbott afd8d5ef0c openapi: Improve stream description details. 2022-01-21 14:10:48 -08:00
Palash 61e87b0ac8 api_documentation: Update subscriptions parameter in OpenAPI doc.
regarding -
POST https://yourZulipDomain.zulipchat.com/api/v1/users/me/subscriptions

The definition of the "subscription" parameter didn't include full
information about the parameter. It only said that an array of objects
is passed as a parameter, and relied on description of the parameter
to explain what the object contained. I edited the definition to contain
the full information about the object.

Fixes #20824.
2022-01-21 14:04:46 -08:00
Eeshan Garg aa8b3f9729 streams: Add RealmAuditLog entries for permission changes. 2022-01-21 13:59:35 -08:00
Eeshan Garg 0d99809fd3 streams: Add notifications for permission policy changes.
The change to curl_param_value_generators.py warrants a brief
explanation. Stream permission changes now generate a notification
message. Our curl example test for removing a reaction comes after
the two tests for updating the stream permission changes, thus the
hardcoded message ID in that test needs to be incremented by 2 to
account for the two notification messages that now come before it.

This is a part of #20289.
2022-01-21 13:59:34 -08:00
Eeshan Garg fab1b7f5d5 actions: Refactor functions for stream permission changes.
do_make_stream_web_public and do_change_stream_invite_only seem
to contain very similar logic that could just live inside the
do_change_stream_permission function that handles all permission
changes in one place.
2022-01-21 13:59:34 -08:00
Eeshan Garg f0ee065292 streams: Use bulleted format for description change notifications.
We want the format for our description change notifications to be
consistent with the format of our stream posting policy change
notifications.
2022-01-21 13:59:34 -08:00
Anders Kaseorg 1f3e87b2a3 test_queue_error_json: Acknowledge the received message.
Otherwise it stays in the queue.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg 773c909d7f test_queue: Ensure the test_suite queue exists in setUp.
queue_client.queues does not list all the queues that exist on the
server (you can’t do that over AMQP); the condition "test_suite" in
queue_client.queues was always false.  So the test_suite queue could
accumulate extra messages that broke test_queue_error_json.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg db24eca832 test_auth_backends: Verify the avatar contents.
Consuming response.streaming_content fixes this warning from
TestLDAP.test_login_success_when_user_does_not_exist_with_valid_subdomain:
“ResourceWarning: unclosed file <_io.FileIO
name='/srv/zulip/var/…/3cc4b5a15b6f4f06b3f9a6ecb179b08702329716.png'
mode='rb' closefd=True>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Anders Kaseorg ba7ea7cc80 test_classes: Extract assert_streaming_content helper.
This also fixes a warning from
RealmExportTest.test_endpoint_local_uploads: “ResourceWarning:
unclosed file <_io.BufferedReader
name='/srv/zulip/var/…/test-export.tar.gz'>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Lauryn Menard 7993aed530 api_documentation: Render object details in parameter description.
Adds a check for object in parameter type that will render the details
of the object in the parameter description if they are in the object
definition in the OpenAPI documentation.

Fixes #19424.
2022-01-19 13:04:04 -08:00
Alex Vandiver 915c2b2fd9 muting: Fix a race in topic unmuting.
Rather than check if the topic exists and then try to delete it, just
try to delete it, and catch the lack of matching rows.
2022-01-18 14:15:06 -08:00
Mateusz Mandera f8b06ed952 events: Send invites_changed event if user deactivation revokes invites.
revoke_invites_generated_by_user should send invites_changed event if it
actually revokes some invitations. This is called in the user
deactivatoin codepath.
2022-01-18 14:12:55 -08:00
Mateusz Mandera 74d2aea76a apply_events: Update state["subscribers"] upon "remove user" event.
Event of type "realm_user", op "remove", emitted by do_deactivate_user
should remove the user id from subscriptions in the state. We weren't
catching this bug, because test_do_deactivate_bot uses a newly created
bot, so no stream subscriptions are affected. The bug shows up if
deactivating e.g. cordelia - thus we want to have two tests instead,
one for testing bot deactivation and one for user deactivation.
2022-01-18 14:12:55 -08:00
Steve Howell dd1c9c45c7 stream colors: Try harder to avoid collisions.
We now use recipient_id % 24 for new stream colors
when users have already used all 24 of our canned
colors.

This fix doesn't address the scenario that somebody
dislikes one of our current canned colors, so if a
user continually changes canned color N to some other
color for new streams, their new streams will continue
to include color N (and the user will still need to
change them).

This fix doesn't address the fact that it can be expensive
during bulk-add situations to query for all the colors
that users have already used up.

See https://chat.zulip.org/#narrow/stream/3-backend/topic/assigning.20stream.20colors
for more discussion.
2022-01-18 13:56:54 -08:00
Sahil Batra 06cba4ae1f actions: Use transaction.atomic in do_change_bot_owner. 2022-01-18 12:43:04 -08:00
Sahil Batra 7c44151135 actions: Use transaction.atomic in do_change_tos_version. 2022-01-18 12:43:04 -08:00
Sahil Batra 06d715a41d actions: Use transaction.atomic in do_change_icon_source. 2022-01-18 12:43:04 -08:00
Sahil Batra 64d1dc6525 actions: Use transaction.atomic in do_change_logo_source. 2022-01-18 12:43:04 -08:00
Sahil Batra 8945a64024 actions: Use transaction.atomic in do_change_realm_org_type. 2022-01-18 12:43:04 -08:00
Sahil Batra c8f81ded4e actions: Use transaction.atomic in do_change_default_sending_stream. 2022-01-18 12:43:04 -08:00