Commit Graph

5541 Commits

Author SHA1 Message Date
Tim Abbott 6e9683fe6f test_compatibility: Mock desktop minimum versions.
This fixes the tests failing since
454eced466, in a way that won't require
us to change these tests for future version bumps.
2021-05-13 13:03:42 -07:00
Anders Kaseorg 668b5137b0 version: Display Zulip version in About Zulip dialog.
We record Git details about the merge-base with upstream branches in
the zulip-git-version file, if the upstream repository is available.

Note that the first Git upgrade after merging the parent commit will
not include the merge-base details, since the upstream repository will
not have been available.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-13 11:36:12 -07:00
sahil839 213eda1f32 message: Check stream_post_policy when moving messages between streams.
Previously only admins were allowed to move messages between streams
and admins are allowed to post in any stream irresepctive of stream
post policy, so there was no need to check for stream post policy.

But as we now allow other members to also move messages, we need
to check whether the user who is moving the message is allowed
to post to the target stream (i.e. stream to which the messages
are being moved) and thus we allow moving messages only if the
user is allowed to post in target stream.
2021-05-13 08:42:24 -07:00
Mateusz Mandera f1cf9f740d tests: Add test for bulk_access_messages scenarios in message moving. 2021-05-13 08:42:24 -07:00
sahil839 b52ad3e536 message_edit: Allow moving message to stream based on setting value.
We allow the users to move message between streams according to the
value of 'move_messages_between_streams_policy'.
2021-05-13 08:40:57 -07:00
Mateusz Mandera de6bd22ee9 outgoing_webhook: Treat "" json in response as response_not_required.
b7b1ec0aeb made our checks of the response
format stronger, to enforce that the json translates to a valid dict.
However, old client code (zulip_botserver) was using "" as equivalent to
response_not_required - so we need to keep backward-compatibility to not
break things built on it.
2021-05-13 08:07:38 -07:00
Tim Abbott 41d499d44c message_edit: Require access to messages to move between streams.
Currently, moving messages between streams is an action limited to
organization administrators. A big part of the motivation for that
restriction was to prevent users from moving messages from a private
stream without shared history as a way to access messages they should
not have access to.

Organization administrators can already just make the stream have
shared history if they want to access its messages, but allowing
non-administrators to move messages between would have
introduced a security bug without this change.
2021-05-12 16:23:22 -07:00
Tim Abbott 0bfef96543 bulk_access_messages: Bulk fetch Subscription details.
This completes the effort to make it possible to use
bulk_access_message in contexts where there are more than a handful of
messages without creating performance issues.
2021-05-12 16:23:22 -07:00
Tim Abbott c6e1702335 bulk_access_messages: Support passing in a stream for all messages.
This lets us avoid fetching the Stream in a loop.
2021-05-12 16:23:22 -07:00
Tim Abbott 4a8dcf14ef bulk_access_messages: Use a bulk UserMessage query.
This removes one of the three queries that runs in a loop with this
bulk access function.
2021-05-12 16:23:22 -07:00
Tim Abbott ee780e1f08 message: Test queries used by bulk_access_messages. 2021-05-12 16:23:22 -07:00
Tim Abbott c84ea01869 message: Refactor has_message_access parameters. 2021-05-12 16:23:22 -07:00
Gaurav Pandey dada7bb4bb tests: Clean up tests for realm_domain and update_streams endpoint.
Cleaning up test_realm_domains.RealmDomainTest.test_list_realm_domains,
test_subs.StreamAdminTest.test_private_stream_live_updates,
test_subs.StreamAdminTest.test_realm_admin_can_update_unsub_private_stream
and test_subs.StreamAdminTest.test_non_admin_cannot_access_unsub_private_stream.
2021-05-12 12:38:16 -07:00
Vishnu KS bbcf396303 tests: Create assertNotInHomePage helper function. 2021-05-12 10:34:12 -07:00
Vishnu KS 75a3059b57 tests: Create assertInHomePage helper function. 2021-05-12 10:34:12 -07:00
Tim Abbott 06f1ea657e navbar: Adjust label for usage statistics. 2021-05-12 10:34:12 -07:00
Vishnu KS 051dab58ea navbar: Add gear menu advertisement for sponsoring zulip. 2021-05-12 10:21:48 -07:00
Steve Howell b4470ac8e1 performance: Add get_subscriptions_for_send_message.
This new function optimizes how we fetch subscriptions
for streams.  Basically, it excludes most long-term-idle
users from the query.

With 8k users, of which all but 400 are long term idle,
this speeds up get_recipient_info from about 150ms
to 50ms.

Overall this change appears to save a factor of 2-3 in the backend
processing time for sending or editing a message in large, public
streams in chat.zulip.org (at 18K users today).
2021-05-12 08:10:57 -07:00
Tim Abbott b15941610d message: Support avoiding database queries in has_message_access.
If the caller has already fetched the Stream or subscription details
for the user, those can be passed to has_message_access to avoid extra
database queries.
2021-05-11 20:46:49 -07:00
Alex Vandiver 237efdee6c send_email: Show more information about messages which failed to send. 2021-05-11 14:27:45 -10:00
Mateusz Mandera c95cac2dd7 outgoing_webhook: Send response payload to bot owner if it was invalid.
When the format of the response received from the outgoing webhook
server is invalid (unparsable json, or just wrong format that doesn't
translate into a dictionary etc.), a message with the error is sent to
the bot owner. We should include the actual payload to make reasonable
debugging possible.

In notify_bot_owner we have to move the `if response_content` block to
append the payload to the message whenever it was specified as an
argument to the function. It shouldn't be nested inside
`elif status_code` as before.
2021-05-11 14:27:00 -07:00
Alex Vandiver 0f1611286d management: Rename the deliver_email command to deliver_scheduled_email.
This makes it parallel with deliver_scheduled_messages, and clarifies
that it is not used for simply sending outgoing emails (e.g. the
`email_senders` queue).

This also renames the supervisor job to match.
2021-05-11 13:07:29 -07:00
Steve Howell 1e63282cc2 tests: Test email/push more thoroughly for idle users. 2021-05-11 14:33:12 +00:00
Steve Howell 3161b5d892 tests: Simplify test_user_message_filter. 2021-05-11 13:53:55 +00:00
Ganesh Pawar 529f72fa3f markdown: Add support for sms and tel links.
Fixes #18390
2021-05-10 15:15:34 -07:00
Dinesh 7413966255 tests: Remove duplicate verify_backend() calls. 2021-05-10 13:56:06 -07:00
akshatdalton 55f4996f16 markdown: Fix silent wildcard mentions bug.
A message containing wildcard mention when quoted (which
is turned into a silent mention) or message with silent
wildcard mention notifies the users by sending desktop,
sound, and missed message email notifications. This
is clearly a bug which is fixed by this commit.

Fixes: #18354.
2021-05-10 12:19:40 -07:00
Tim Abbott 70244896e5 test_subs: Remove now invalid JSON encoding of parameters.
All of these were missed in the previous commit.
2021-05-10 10:42:11 -07:00
Gaurav Pandey 96e035a2f0 api: Fix encoding of strings in streams endpoint.
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameter.

Bump api feature level and highlight the fix for `emojiset`
parameter of `settings/display` endpoint in zulip.yaml file.

Fixes part of #18035.
2021-05-10 10:29:22 -07:00
Gaurav Pandey 9abe1cafbe api: Fix encoding of strings in display settings endpoint.
* Remove unnecessary json validator for string validator.
* Update frontend to pass right validator.
* Update zulip.yaml to pass right parameter for curl request
  in openapi.
* Update python_examples to pass right paramater.

Fixes part of #18035.
2021-05-10 10:03:32 -07:00
Anders Kaseorg 544bbd5398 docs: Fix capitalization mistakes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-10 09:57:26 -07:00
Tim Abbott 8b75b6f14f message_edit: Use target_message as local variable name.
This name helps emphasize the fact that there's a single targeted
message, even though multiple messages may be affected by the edit.
2021-05-09 21:10:32 -07:00
PIG208 7150fe5dc5 backend: Extract check_update_message from update_message_backend. 2021-05-09 20:44:04 -07:00
Anders Kaseorg d0c6f4f400 python: Strip leading and trailing spaces from docstrings.
This is enforced by Black ≥ 21.4b0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-07 22:42:39 -07:00
Gaurav Pandey 00647b6fa7 api: Fix encoding of strings in video calls endpoint.
This removes unnecessary json_validator for string parameters in the
BigBlueButton video calls endpoints.  Note that this breaks links to
video meetings sent before the upgrade; there's not much we can do
about that.

Since this is the last commit in this series, we update the
ZULIP_FEATURE_LEVEL for this batch of changes.

Fixes part of #18035.
2021-05-07 14:11:48 -07:00
Gaurav Pandey 36ad9b7d0e api: Fix encoding of strings in user settings endpoints.
* Remove unnecessary json_validator for string parameters.
* Remove unnecessary JSON encoding in frontend calls.  Structurally,
  JavaScript does correct encoding without explicit JSON encoding.

Fixes part of #18035.
2021-05-07 14:10:45 -07:00
Gaurav Pandey 232c7107eb api: Fix encoding of strings in tutorial endpoint.
* Remove unnecessary json validator for string parameter.
* Update frontend to pass right parameter.

Fixes part of #18035.
2021-05-07 14:06:54 -07:00
Gaurav Pandey d4219da254 api: Fix encoding of strings in realm domain endpoint.
* Remove unnecessary json_validator for string parameters.
* Update frontend to pass right parameters.

Fixes part of #18035.
2021-05-07 14:06:54 -07:00
Gaurav Pandey b2111f5753 api: Fix encoding of strings in default stream groups.
Remove unnecessary json_validator for string parameters.  This change
does not modify JavaScript because we don't have a frontend for these
API endpoints yet.

Fixes part of #18035.
2021-05-07 14:06:37 -07:00
Gaurav Pandey d6cf1ef27b api: Fix encoding of strings in hotspot endpoint.
* Remove unnecessary json_validator for string parameter.
* Update frontend to pass right parameters.

Fixes part of #18035.
2021-05-07 11:45:25 -07:00
Vishnu KS 772500d1c6 validators: Make to_positive_or_allowed_int an optional argument. 2021-05-07 09:37:41 -07:00
Mateusz Mandera 98ec84eae2 tests: Change password hashers used in tests that set them.
The previous hashers mirrored the ones used in production, but that was
non-ideal because those are slow. Replacing them with quick hashers is a
performance improvement for those tests.
2021-05-07 09:10:45 -07:00
Mateusz Mandera 4006bb5153 auth: Improve display of errors when user needs to reset password.
Raising jsonableError in the authentication form was non-ideal because
it took the user to an ugly page with the returned json.

We also add logging of this rare occurence of the scenario being
handled here.
2021-05-07 09:10:45 -07:00
Dinesh 00e0b63c6c auth: Handle user_profile.check_password raising PasswordTooWeakError.
user_profile.check_password(password) in authenticate of
EmailAuthBackend can raise PasswordTooWeakError; this happens when the
user's password is weaker than the current required policies and needs
to be rehashed (E.g. because, as in Django 3.2, the minimum salt
entropy increased).

This is a very rare case, but still needs a good user-facing error
message. We raise a json error to handle this with a user-facing error
message.

See this comment by Mateusz Mandera for a detailed explanation
about this case along with a traceback it generates.
https://github.com/zulip/zulip/pull/15449#discussion_r448308614
2021-05-07 09:10:45 -07:00
Alex Vandiver 8df82f50e4 outgoing_http: Provide a convenient way to set default headers. 2021-05-07 08:39:36 -07:00
Alex Vandiver 6339e7fd47 outgoing_http: Put the X-Smokescreen-Role in the proxy headers. 2021-05-07 08:39:36 -07:00
Alex Vandiver b88d7a741e outgoing_http: Factor out outgoing HTTP session with timeout. 2021-05-07 08:39:36 -07:00
Alex Vandiver 2dc6df33ae tests: Switch outgoing_webhook_system to use responses. 2021-05-07 08:39:36 -07:00
Alex Vandiver af26849554 outgoing_webhook: Set a default timeout of 10s.
Support for the timeouts, and tests for them, was added in
53a8b2ac87 -- though no code could have set them after 31597cf33e.

Add a 10-second default timeout.  Observationally, p99 is just about
5s, with everything else being previously being destined to meet the
30s worker timeout; 10s provides a sizable buffer between them.

Fixes #17742.
2021-05-07 08:39:36 -07:00
Anders Kaseorg 405bc8dabf requirements: Remove Thumbor.
Thumbor and tc-aws have been dragging their feet on Python 3 support
for years, and even the alphas and unofficial forks we’ve been running
don’t seem to be maintained anymore.  Depending on these projects is
no longer viable for us.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-06 20:07:32 -07:00