Commit Graph

6250 Commits

Author SHA1 Message Date
Anders Kaseorg 0ba0620000 push_notifications: Fix for aioapns 2.1.
aioapns 2.1 removed the loop parameter from the aioapns.APNs
constructor, because Python 3.10 removed the loop parameter from the
asyncio.Lock constructor.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-08 15:16:31 -08:00
Anders Kaseorg e1f42c1ac5 docs: Add missing space to compound verbs “back up”, “log in”, etc.
Noun: backup, login, logout, lookup, setup.

Verb: back up, log in, log out, look up, set up.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 19:20:54 -08:00
Anders Kaseorg b0ce4f1bce docs: Fix many spelling mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-02-07 18:51:06 -08:00
Alya Abbott 5eda383833 portico: Add iDrift AS case study. 2022-02-07 14:40:31 -08:00
Aman Agrawal ca71e28cd6 accounts_accept_terms: Make elements looks similar to other pages.
Add even vertical space between elements.
2022-02-04 15:48:38 -08:00
Mateusz Mandera a1fa2a8cf5 scim: Upgrade to django-scim2 0.17.1.
The new release adds the commit:
20ac22b96d

Which allows us to get rid of the entire ugly override that was needed
to do this commit's job in our code. What we do here in this commit:
* Use django-scim2 0.17.1
* Revert the relevant parts of f5a65846a8
* Adjust the expected error message in test_exception_details_not_revealed_to_client
  since the message thrown by django-scim2 in this release is slightly
  different.

We do not have to add anything to set EXPOSE_SCIM_EXCEPTIONS, since
django-scim2 uses False as the default, which is what we want - and we
have the aforementioned test verifying that indeed information doesn't
get revealed to the SCIM client.
2022-02-04 15:43:45 -08:00
Tim Abbott 1045737be6 test_subs: Use HostRequestMock for request object.
I incorrectly removed this when simplifying
dbddbee5a115b9352862cb13d4c66820865c30b6; while that commit did not
require the hunk re-added here, the later commit
3be622ffa7 added a call that did require it.
2022-02-04 15:36:57 -08:00
Lauryn Menard 3be622ffa7 backend: Add request as parameter to json_success.
Adds request as a parameter to json_success as a refactor towards
making `ignored_parameters_unsupported` functionality available
for all API endpoints.

Also, removes any data parameters that are an empty dict or
a dict with the generic success response values.
2022-02-04 15:16:56 -08:00
Sahil Batra 9f0d498994 message: Check mandatory_topics setting in backend as well.
Previously, we only checked mandatory_topics setting before
sending message in frontend and there was no restriction in
backend. This commit adds the check in backend also making
sure messages without topic cannot be sent through API as
well if mandatory_topics setting is set to True.
2022-02-02 10:11:35 -08:00
Alex Vandiver d1234ef18c string_validation: Prohibit non-printable characters in stream names.
Fixes part of #20128.

Co-authored-by: Shlok Patel <shlokcpatel2001@gmail.com>
2022-02-01 16:04:34 -08:00
Tim Abbott 1a46df40c3 resolve topic: Limit unread flag on automated notifications.
Previously, users found it annoying that the automated "Resolve topic"
notifications triggered an unread for everyone in the stream; this
discouraged some users from using the feature on older threads for
fear of being annoying. We change this to a better default, of only
users who participated in the topic (via either messages or reactions)
being eligible for the new message being unread.

We will likely want to create global and stream-level notifications
settings to control this behavior as a follow-up -- some users, like
me, might prefer the simpler "Always unread" behavior in some streams.

Note that the automated notifications that a topic was resolved will
still result in the topic being moved to the top of the left sidebar.
This would be somewhat difficult to change, since the left sidebar
algorithm just looks at the highest message ID in the topic.

Fixes #19709.

Tests added by Aman Agrawal (amanagr@zulip.com).
2022-02-01 11:35:50 -08:00
Aman Agrawal f00db27b64 message_feed_errors: Convert to handlebars. 2022-01-31 17:10:05 -08:00
Mateusz Mandera d5a784a1ca oidc: Don't raise AssertionError if no name is provided.
Closes #20821.
Just like we did this for SAML in
cee4da64fa, so should we for oidc, as some
providers like Keycloak may not send the name by default.
2022-01-31 10:15:24 -08:00
Mateusz Mandera c0f7158378 push_notifications: Include stream_id in the notification data.
Closes #18067.
Previous only the stream name was sent, which is an unstable stream
identifier.
2022-01-29 17:37:48 -08:00
Anders Kaseorg 90e202cd38 docs: Consistently hyphenate “web-public”.
In English, compound adjectives should essentially always be
hyphenated.  This makes them easier to parse, especially for users who
might not recognize that the words “web public” go together as a
phrase.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-28 17:45:45 -08:00
jai2201 c14b3a8844 settings: Don't mention username in fetch API key error message.
There isn't a username prompt in the form for it, so the only
possibility for triggering this error is a wrong password.

Fixes #20924.
2022-01-28 14:10:15 -08:00
Mateusz Mandera 120de1db19 do_deactivate_stream: Use transaction.atomic. 2022-01-28 13:03:39 -08:00
Mateusz Mandera 0dd97eeaab do_set_realm_property: Use transaction.atomic. 2022-01-28 13:03:39 -08:00
Mateusz Mandera 4102816240 upload: Pass the target realm to create_attachment.
The target realm was not being passed to create_attachment in
upload_message_file implementations. This was a bug in the edge-case of
cross-realm messages - in particular, causing a bug in the email
gateway:
When an email with an attachment is sent, the message is mirrored to
Zulip with Email Gateway Bot as the message sender and uploader of the
attachment. Due to the realm not being passed to create_attachment, the
Attachment would get created with .realm being the system bot realm,
making the attachment inaccessible under some conditions due to failing
the following condition check (that's expected to pass, provided that
the .realm is set correctly):
```
    if (
        attachment.is_realm_public
        and attachment.realm == user_profile.realm
        and user_profile.can_access_public_streams()
    ):
        # Any user in the realm can access realm-public files
        return True
```
2022-01-27 17:23:44 -08:00
Lauryn Menard aaa627229e api: Update `update_message` event required fields.
Makes `edit_timestamp` and `user_id` required fields for all
`update_message` events.

Adds `rendering_only` as another required field to signal if
events are only updating the rendered content of the message,
which is currently the case for adding inline url previews.

Updates `test_event.py` so that `do_update_message` and
`do_update_embedded_data` refer to the same testing schema
for `update_message` events, and therefore reflect the same
required fields for the `update_message` event.

The OpenAPI definition for `update_message` events is also
updated to reflect the required field and descriptions of
various properties are updated for the addition of the
`rendering_only` property.
2022-01-26 13:11:26 -08:00
Sharif Naas d560d124a3 python: Replace string concatenations with f-strings. 2022-01-25 17:32:59 -08:00
Anders Kaseorg 4922632601 mypy: Add types-beautifulsoup4.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 23:39:40 -08:00
Anders Kaseorg 2caeb38e9e python: Replace IOError with OSError.
IOError is an alias for OSError in Python ≥ 3.3.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:17:02 -08:00
Anders Kaseorg 97e4e9886c python: Replace universal_newlines with text.
This is supported in Python ≥ 3.7.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:16:01 -08:00
Anders Kaseorg b729f00fc2 test_upload: Uncomment subTest contexts.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:43 -08:00
Anders Kaseorg 2612f57d51 requirements: Upgrade Python requirements.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg ee71650908 python: Replace requests.packages.urllib3 alias with urllib3.
requests stopped vendoring urllib3 in 2.16.0 (2017-05-26).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-23 22:14:17 -08:00
Anders Kaseorg a58a71ef43 Remove Ubuntu 18.04 support.
As a consequence:

• Bump minimum supported Python version to 3.7.
• Move Vagrant environment to Debian 10, which has Python 3.7.
• Move CI frontend tests to Debian 10.
• Move production build test to Debian 10.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 17:26:14 -08:00
Eeshan Garg 3bc0f8c6f9 zilencer: Add endpoint for deactivating remote server registration. 2022-01-21 14:57:04 -08:00
Eeshan Garg 94d00ca942 zilencer: Stop serving requests from deactivated remote servers. 2022-01-21 14:56:04 -08:00
Eeshan Garg aa8b3f9729 streams: Add RealmAuditLog entries for permission changes. 2022-01-21 13:59:35 -08:00
Eeshan Garg 0d99809fd3 streams: Add notifications for permission policy changes.
The change to curl_param_value_generators.py warrants a brief
explanation. Stream permission changes now generate a notification
message. Our curl example test for removing a reaction comes after
the two tests for updating the stream permission changes, thus the
hardcoded message ID in that test needs to be incremented by 2 to
account for the two notification messages that now come before it.

This is a part of #20289.
2022-01-21 13:59:34 -08:00
Eeshan Garg fab1b7f5d5 actions: Refactor functions for stream permission changes.
do_make_stream_web_public and do_change_stream_invite_only seem
to contain very similar logic that could just live inside the
do_change_stream_permission function that handles all permission
changes in one place.
2022-01-21 13:59:34 -08:00
Eeshan Garg f0ee065292 streams: Use bulleted format for description change notifications.
We want the format for our description change notifications to be
consistent with the format of our stream posting policy change
notifications.
2022-01-21 13:59:34 -08:00
Anders Kaseorg 1f3e87b2a3 test_queue_error_json: Acknowledge the received message.
Otherwise it stays in the queue.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg 773c909d7f test_queue: Ensure the test_suite queue exists in setUp.
queue_client.queues does not list all the queues that exist on the
server (you can’t do that over AMQP); the condition "test_suite" in
queue_client.queues was always false.  So the test_suite queue could
accumulate extra messages that broke test_queue_error_json.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:38:13 -08:00
Anders Kaseorg db24eca832 test_auth_backends: Verify the avatar contents.
Consuming response.streaming_content fixes this warning from
TestLDAP.test_login_success_when_user_does_not_exist_with_valid_subdomain:
“ResourceWarning: unclosed file <_io.FileIO
name='/srv/zulip/var/…/3cc4b5a15b6f4f06b3f9a6ecb179b08702329716.png'
mode='rb' closefd=True>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Anders Kaseorg ba7ea7cc80 test_classes: Extract assert_streaming_content helper.
This also fixes a warning from
RealmExportTest.test_endpoint_local_uploads: “ResourceWarning:
unclosed file <_io.BufferedReader
name='/srv/zulip/var/…/test-export.tar.gz'>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-21 13:37:26 -08:00
Mateusz Mandera f8b06ed952 events: Send invites_changed event if user deactivation revokes invites.
revoke_invites_generated_by_user should send invites_changed event if it
actually revokes some invitations. This is called in the user
deactivatoin codepath.
2022-01-18 14:12:55 -08:00
Mateusz Mandera 74d2aea76a apply_events: Update state["subscribers"] upon "remove user" event.
Event of type "realm_user", op "remove", emitted by do_deactivate_user
should remove the user id from subscriptions in the state. We weren't
catching this bug, because test_do_deactivate_bot uses a newly created
bot, so no stream subscriptions are affected. The bug shows up if
deactivating e.g. cordelia - thus we want to have two tests instead,
one for testing bot deactivation and one for user deactivation.
2022-01-18 14:12:55 -08:00
Steve Howell dd1c9c45c7 stream colors: Try harder to avoid collisions.
We now use recipient_id % 24 for new stream colors
when users have already used all 24 of our canned
colors.

This fix doesn't address the scenario that somebody
dislikes one of our current canned colors, so if a
user continually changes canned color N to some other
color for new streams, their new streams will continue
to include color N (and the user will still need to
change them).

This fix doesn't address the fact that it can be expensive
during bulk-add situations to query for all the colors
that users have already used up.

See https://chat.zulip.org/#narrow/stream/3-backend/topic/assigning.20stream.20colors
for more discussion.
2022-01-18 13:56:54 -08:00
Anders Kaseorg 9e70a47f93 test_push_notifications: Close event loops.
Fixes “ResourceWarning: unclosed event loop <_UnixSelectorEventLoop
running=False closed=False debug=False>”.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 20:40:46 -08:00
Anders Kaseorg 4147da24dd tests: Use read_test_image_file.
Fixes a ResourceWarning from the unclosed file at test_upload.py:1954.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:59:46 -08:00
Anders Kaseorg 031f4596ab openapi: Use openapi_core ResponseValidator to validate responses.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Anders Kaseorg 86c39e5792 test_openapi: Make testing.yml a full conformant specification.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-13 14:34:11 -08:00
Sahil Batra 9b8713fc1e users: Send peer_add subscription events on reactivating users.
The subscriber list was not updating without a refresh on
reactivating user, because the subscriptions data with the
client was not updated on reactivation.

This commit adds code to send peer_add subscription events
on reactivating the user.

We do not send peer_remove events on deactivating the user,
but the subscriber list is still live-updated because we
have the data of the streams which the deactivated user is
susbcribed to and the clients itself updates the data and UI
on receiving event of deactivation of user, which it is not
possible when reactivating the user.

Fixes #20383.
2022-01-12 14:30:21 -08:00
Mateusz Mandera 93e8740218 do_deactivate_user: Revoke invitations generated by the user.
Leaving old invitations valid, potentially for a very long time, is
clearly unexpected and undesired behavior under normal circumstances. A
user shouldn't be able to e.g. generate a multiuse invite link, get
banned from the organization by being deactivated and then just re-join
using the link they've created for themselves.
2022-01-12 13:53:34 -08:00
Mateusz Mandera ff688c3a8d actions: Give do_get_user_invites a more specific name.
The added docstrings elaborates on why the new name is more appropriate.
2022-01-12 13:53:34 -08:00
Anders Kaseorg 82f2707278 test_openapi: Fix incorrect use of assertRaises msg parameter.
The msg parameter is a string to be displayed when the expected
exception wasn’t raised, not a pattern to match against the raised
exception’s message.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2022-01-11 21:01:45 -08:00
Alex Vandiver df50280c54 string_validation: Loosen to allow some `Cn` unicode characters.
Under the unicodedata distributed with Python 3.6, some Emoji are
classified as `Cn`, and not `So`:

```
$ unicode 1f929 --long
U+1F929 GRINNING FACE WITH STAR EYES
UTF-8: f0 9f a4 a9 UTF-16BE: d83edd29 Decimal: &#129321; Octal: \0374451
🤩
Category: So (Symbol, Other); East Asian width: W (wide)
Unicode block: 1F900..1F9FF; Supplemental Symbols and Pictographs
Bidi: ON (Other Neutrals)

$ python3.6 -c 'import unicodedata; print(unicodedata.category("\U0001f929"))'
Cn

$ python3.7 -c 'import unicodedata; print(unicodedata.category("\U0001f929"))'
So
```

Drop `Cn` from the list of excluded Unicode character classes, and
replace it with an explicit list of the 66 non-characters, which are
invariant.

Co-authored-by: Shlok Patel <shlokcpatel2001@gmail.com>
2022-01-11 15:17:53 -08:00