Commit Graph

14747 Commits

Author SHA1 Message Date
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Eeshan Garg 93329c2d70 send_custom_email: Only send emails to active remote servers. 2022-01-21 15:03:08 -08:00
Tim Abbott 367fed89f2 register_server: Improve HTTP error reporting. 2022-01-21 15:02:30 -08:00
Eeshan Garg fb09c18462 management/commands: Refactor error handling in register_server. 2022-01-21 14:57:09 -08:00
Eeshan Garg bd197886fe management/commands: Refactor logging code in register_server. 2022-01-21 14:57:09 -08:00
Eeshan Garg 3bc0f8c6f9 zilencer: Add endpoint for deactivating remote server registration. 2022-01-21 14:57:04 -08:00
Eeshan Garg 94d00ca942 zilencer: Stop serving requests from deactivated remote servers. 2022-01-21 14:56:04 -08:00
Tim Abbott afd8d5ef0c openapi: Improve stream description details. 2022-01-21 14:10:48 -08:00
Palash 61e87b0ac8 api_documentation: Update subscriptions parameter in OpenAPI doc.
regarding -
POST https://yourZulipDomain.zulipchat.com/api/v1/users/me/subscriptions

The definition of the "subscription" parameter didn't include full
information about the parameter. It only said that an array of objects
is passed as a parameter, and relied on description of the parameter
to explain what the object contained. I edited the definition to contain
the full information about the object.

Fixes #20824.
2022-01-21 14:04:46 -08:00
Eeshan Garg aa8b3f9729 streams: Add RealmAuditLog entries for permission changes. 2022-01-21 13:59:35 -08:00
Eeshan Garg 0d99809fd3 streams: Add notifications for permission policy changes.
The change to curl_param_value_generators.py warrants a brief
explanation. Stream permission changes now generate a notification
message. Our curl example test for removing a reaction comes after
the two tests for updating the stream permission changes, thus the
hardcoded message ID in that test needs to be incremented by 2 to
account for the two notification messages that now come before it.

This is a part of #20289.
2022-01-21 13:59:34 -08:00
Eeshan Garg fab1b7f5d5 actions: Refactor functions for stream permission changes.
do_make_stream_web_public and do_change_stream_invite_only seem
to contain very similar logic that could just live inside the
do_change_stream_permission function that handles all permission
changes in one place.
2022-01-21 13:59:34 -08:00
Eeshan Garg f0ee065292 streams: Use bulleted format for description change notifications.
We want the format for our description change notifications to be
consistent with the format of our stream posting policy change
notifications.
2022-01-21 13:59:34 -08:00
Anders Kaseorg 1f3e87b2a3 test_queue_error_json: Acknowledge the received message.
Otherwise it stays in the queue.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg 773c909d7f test_queue: Ensure the test_suite queue exists in setUp.
queue_client.queues does not list all the queues that exist on the
server (you can’t do that over AMQP); the condition "test_suite" in
queue_client.queues was always false.  So the test_suite queue could
accumulate extra messages that broke test_queue_error_json.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg db24eca832 test_auth_backends: Verify the avatar contents.
Consuming response.streaming_content fixes this warning from
TestLDAP.test_login_success_when_user_does_not_exist_with_valid_subdomain:
“ResourceWarning: unclosed file <_io.FileIO
name='/srv/zulip/var/…/3cc4b5a15b6f4f06b3f9a6ecb179b08702329716.png'
mode='rb' closefd=True>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Anders Kaseorg ba7ea7cc80 test_classes: Extract assert_streaming_content helper.
This also fixes a warning from
RealmExportTest.test_endpoint_local_uploads: “ResourceWarning:
unclosed file <_io.BufferedReader
name='/srv/zulip/var/…/test-export.tar.gz'>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Lauryn Menard 7993aed530 api_documentation: Render object details in parameter description.
Adds a check for object in parameter type that will render the details
of the object in the parameter description if they are in the object
definition in the OpenAPI documentation.

Fixes #19424.
2022-01-19 13:04:04 -08:00
Alex Vandiver 915c2b2fd9 muting: Fix a race in topic unmuting.
Rather than check if the topic exists and then try to delete it, just
try to delete it, and catch the lack of matching rows.
2022-01-18 14:15:06 -08:00
Mateusz Mandera f8b06ed952 events: Send invites_changed event if user deactivation revokes invites.
revoke_invites_generated_by_user should send invites_changed event if it
actually revokes some invitations. This is called in the user
deactivatoin codepath.
2022-01-18 14:12:55 -08:00
Mateusz Mandera 74d2aea76a apply_events: Update state["subscribers"] upon "remove user" event.
Event of type "realm_user", op "remove", emitted by do_deactivate_user
should remove the user id from subscriptions in the state. We weren't
catching this bug, because test_do_deactivate_bot uses a newly created
bot, so no stream subscriptions are affected. The bug shows up if
deactivating e.g. cordelia - thus we want to have two tests instead,
one for testing bot deactivation and one for user deactivation.
2022-01-18 14:12:55 -08:00
Steve Howell dd1c9c45c7 stream colors: Try harder to avoid collisions.
We now use recipient_id % 24 for new stream colors
when users have already used all 24 of our canned
colors.

This fix doesn't address the scenario that somebody
dislikes one of our current canned colors, so if a
user continually changes canned color N to some other
color for new streams, their new streams will continue
to include color N (and the user will still need to
change them).

This fix doesn't address the fact that it can be expensive
during bulk-add situations to query for all the colors
that users have already used up.

See https://chat.zulip.org/#narrow/stream/3-backend/topic/assigning.20stream.20colors
for more discussion.
2022-01-18 13:56:54 -08:00
Sahil Batra 06cba4ae1f actions: Use transaction.atomic in do_change_bot_owner. 2022-01-18 12:43:04 -08:00
Sahil Batra 7c44151135 actions: Use transaction.atomic in do_change_tos_version. 2022-01-18 12:43:04 -08:00
Sahil Batra 06d715a41d actions: Use transaction.atomic in do_change_icon_source. 2022-01-18 12:43:04 -08:00
Sahil Batra 64d1dc6525 actions: Use transaction.atomic in do_change_logo_source. 2022-01-18 12:43:04 -08:00
Sahil Batra 8945a64024 actions: Use transaction.atomic in do_change_realm_org_type. 2022-01-18 12:43:04 -08:00
Sahil Batra c8f81ded4e actions: Use transaction.atomic in do_change_default_sending_stream. 2022-01-18 12:43:04 -08:00
Sahil Batra cb43bdab93 actions: Use transaction.atomic for do_change_default_all_public_streams. 2022-01-18 12:43:04 -08:00
Sahil Batra 4a7461361e actions: Use transaction.atomic for do_change_default_events_register_stream. 2022-01-18 12:43:04 -08:00
Sahil Batra 5c758af3b4 actions: Use transaction.atomic for do_change_user_setting. 2022-01-18 12:43:04 -08:00
Alex Vandiver 19f891968d markdown: Increase the maximum number of image previews per message.
The limit here is purely to prevent breakage in case of a pathological
number of images in a single message; 5 images is entirely possible in
a reasonable message, and causes user confusion when they are not
expended.

Increase the limit to 10 per message.
2022-01-14 11:30:07 -08:00
Anders Kaseorg 6e00d6f97a change_password: Set requires_system_checks to a list.
Django 3.2 expects a list, and Django 4.1 will require one.  Fixes
“RemovedInDjango41Warning: Using a boolean value for
requires_system_checks is deprecated. Use '__all__' instead of True,
and [] (an empty list) instead of False.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 21:02:41 -08:00
Anders Kaseorg 9e70a47f93 test_push_notifications: Close event loops.
Fixes “ResourceWarning: unclosed event loop <_UnixSelectorEventLoop
running=False closed=False debug=False>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 20:40:46 -08:00
Anders Kaseorg 87b4e9259f documentation: Replace deprecated request.is_ajax() method.
This was deprecated in Django 3.1 for being jQuery-specific, and
removed in Django 4.0.  Replicate the jQuery-specific check.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 15:40:22 -08:00
Anders Kaseorg 4147da24dd tests: Use read_test_image_file.
Fixes a ResourceWarning from the unclosed file at test_upload.py:1954.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:59:46 -08:00
Anders Kaseorg 031f4596ab openapi: Use openapi_core ResponseValidator to validate responses.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 4cd5e0e578 openapi: Fix display_brief_error not to rely on naively_merge_allOf.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 86c39e5792 test_openapi: Make testing.yml a full conformant specification.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 465ea4ac51 openapi: Validate Python example responses against the entire schema.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 02c1196e3e openapi: Merge success examples for POST users/me/subscriptions.
oneOf with two identical branches (modulo example) is a bug because
oneOf means exclusive or.  It’s also a totally inappropriate kludge
for encoding multiple examples.  The OpenAPI specification provides a
perfectly good standard way to do that:

https://spec.openapis.org/oas/v3.0.3#example-object

However, we don’t handle that in our OpenAPI documentation generator
yet, so for now just merge the examples.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg e9fd52da01 openapi: Mark subscribe error as a JsonError.
This was a oneOf with two identical branches modulo example, which is
always a bug because oneOf means exclusive or.  But the example for
the first branch did not fit the schema for AddSubscriptionsResponse,
which is a subset of JsonSuccessBase.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg a503d19eae openapi: Declare items: {} for “inherited” array properties.
This should not be needed, but works around an openapi-core bug:
https://github.com/p1c2u/openapi-core/issues/380

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 6c25c628b1 openapi: Mark “inherited” nullable properties as nullable.
Although allOf is often used to indicate inheritance, its semantics
are that of a plain set intersection.  The intersection of a nullable
property with a non-nullable property is a non-nullable property.
Therefore, if we want an inherited property to remain nullable, we
need to mark it as such.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 1cb397c1ed openapi: Remove additionalProperties: false from BasicStreamBase.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 834542d097 openapi: Don’t forbid result, msg with additionalProperties: false.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg bea64f8608 python: Convert deprecated Django ugettext alias to gettext again.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Commit e7ed907cf6 (#18174) fixed this
before, but new instances have been added.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:28:12 -08:00
Anders Kaseorg 19b470c260 openapi: Fix principals array type.
We do not accept heterogeneous arrays containing both user ids and
email addresses.

This also happens to disallow an empty array, which is fine since the
principals parameter should be omitted if the default to the calling
user is desired.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:25:18 -08:00
Anders Kaseorg 78e54a0d7a python: Replace deprecated jinja2.utils.Markup with markupsafe.Markup.
Fixes “DeprecationWarning: 'jinja2.Markup' is deprecated and will be
removed in Jinja 3.1. Import 'markupsafe.Markup' instead.”

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:22:48 -08:00
Lauryn Menard cf9483e6a9 api_docs: Fix typos in UserBase schema descriptions.
Fixes two small typos and adds backticks to a reference to an
object field.
2022-01-13 14:21:05 -08:00