Commit Graph

13524 Commits

Author SHA1 Message Date
Tim Abbott 9f2daeee45 upload: Use get_public_upload_url for export tarballs too.
This deduplicates the code so that we now just have one function for
constructing S3 URLs.
2021-05-27 23:26:45 -07:00
ryanreh99 5a4aecfc40 s3 uploads: Refactor to access objects via `get_public_upload_url`.
Our current logic only allows S3 block storage providers whose
upload URL matches with the format used by AWS. This also allows
other styles such as the "virtual host" format used by Oracle cloud.

Fixes #17762.
2021-05-27 23:26:42 -07:00
Abhijeet Prasad Bodas 15f78abd68 message edit: Handle topic edit tries for private messages.
Fixes #18604.
2021-05-27 23:09:33 -07:00
Abhijeet Prasad Bodas 4b30fc01e4 message edit: Extract data validity checks from check_update_message.
These checks are more related to the API than the editability
or permissions logic, so it makes sense to handle them first
before further processing the request.
Also split the main test class to separate out the tests for
this logic.

This also simplifies some tests by reducing the data setup
required to reach failure.

Tweaked by tabbott to avoid losing the topic_name.strip().
2021-05-27 23:07:59 -07:00
Abhijeet Prasad Bodas 10be798a3c message edit: Extract test helpers in their own class.
This is mostly a direct code move and a prep for splitting the
main MessageEdit test class into smaller chunks.
2021-05-27 22:52:55 -07:00
Abhijeet Prasad Bodas 3d99292b4c message edit: Fix `stream_id` description in API docs.
Since caa08d76b5, we no longer have
a common component for stream IDs in zulip.yaml, so we might as
well change the description to be specific and clear.
2021-05-27 22:52:55 -07:00
Alex Vandiver 4ffda1be87 send_email: Fix sleep logic.
This was broken in the refactor in 1e67e0f218.
2021-05-27 22:49:28 -07:00
Alex Vandiver b9948994d7 send_email: Increase loglevel of stuck ScheduledEmail messages to error.
Since these stay in the queue indefinitely, they deserve to be raised
at a higher error than just warning.
2021-05-27 22:49:28 -07:00
Alex Vandiver ff9126ac1e data_import: Protect better against bad Slack tokens.
An invalid token would be treated the same as a token with no scopes;
differentiate these better.
2021-05-27 22:46:58 -07:00
Alex Vandiver 94e4f33b29 data_import: Support importing from Slack conversions in a directory.
Sometimes the Slack import zip file we get isn't quite the canonical
form that Slack produces -- often because the user has unzip'd it,
looked at it, and re-zip'd it, resulting in extra nested directories
and the like.

For such cases, support passing in a path to an unpacked Slack export
tree.
2021-05-27 22:46:58 -07:00
Alex Vandiver 8228ea2a17 import_data: Do some quick verification of Slack import formats. 2021-05-27 22:46:58 -07:00
Suyash Vardhan Mathur 6107679f61 minor: Fix typo in summary.
The summary for add_code_playground wrongly mentions
'Remove' instead of 'Add' in the summary. Fixed the typo.
2021-05-27 16:56:53 -07:00
Abhijeet Prasad Bodas c19b7b4db3 tests: Use tornado_redirected_to_list instead of mocking send_event. 2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas 090f2d6664 test_helpers: Remove capture_event helper.
We now consistently use the `tornado_redirected_to_list`
helper, which asserts the number of events sent too.
2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas d2bc93f6dd tests: Extract helper in test_user_status.
This deduplicates some common code for all these scenarios.
2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas 481a890ec5 tests: Assert num_events in tornado_redirected_to_list. 2021-05-27 13:16:22 -07:00
Anders Kaseorg 0ce015348e actions: Fix incorrect audit logging in bulk_remove_subscriptions.
modified_user=sub_info.user and modified_stream=sub_info.stream, added
by commit 6d1f9de7d3 (#16553), were
always coming from the last entry in the loop above, not from the
enclosing list comprehension.

Found by the Pylint rule undefined-loop-variable.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-26 18:12:03 -07:00
Suyash Vardhan Mathur 52b9c96c5d apidocs: Add support for using OpenAPI title for /api pages.
The current logic of geneerating HTML titles requires the title to be
present as a heading in the first line of .md file. However, this will
shortly be no longer true for /api pages where these are
auto-generated from OpenAPI data.  Modified the code to fetch the
title from OpenAPI data in case of such pages.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 58a8009786 apidocs: Add markdown processor for title.
Currently, the title of each endpoint are hardcoded in .md
files, but these are also added in summary parameter of
openapi. Added a markdown processor to insert the title
of a given endpoint.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 94c4e22898 apidocs: Add summary field to endpoints.
Some pages did not have .md files, and so the summaries for these
was newly written. Commit split for easy review.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur e15af5ae14 apidocs: Add summary field to endpoints.
As a part of effort of removing .md files for /api pages, the
title of each page can be added into the summary OpenAPI
parameter. This also adds a nice summary visible in Swagger
and enhances the documentation. Added the parameter for all endpoints.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur b41be98c42 apidocs: Fix playground URLs and operationIDs.
We standardized on "Code playground" for this feature, so use that
in the URLs and API documentation.
2021-05-26 17:38:00 -07:00
akshatdalton 832c763c38 minor: Remove unnecessary `__init__` method in `InlineInterestingLinkProcessor`.
Subclass `Treeprocessor` takes care of the `__init__` method.
2021-05-26 17:13:03 -07:00
Adam Birds 303bb61f9c integrations: Add Sonarr Integration. 2021-05-26 17:12:28 -07:00
Adam Birds 17fe2e4a08 integrations: Add Radarr inegration. 2021-05-26 17:11:57 -07:00
Adam Birds 11e7f27031 integrations: Add Lidarr integration. 2021-05-26 17:11:24 -07:00
Abhijeet Prasad Bodas f236a0d10d message send: Rename `always_push_notify` -> `online_push_enabled`.
This is a better name, since it clearly denotes a user
configured setting.
2021-05-26 15:19:32 -07:00
Abhijeet Prasad Bodas ddd123f133 message send: Rename `push_notify_user_ids` -> `online_push_user_ids`.
The old name `push_notify_user_ids` was misleading, because
it does not contain user ids which should be notified for
the current message, but rather user ids who have the online
push notifications setting enabled.

When the Tornado server is restarted during an upgrade, if
server has old events with the `push_notify_user_ids` fields,
the server will throw error after this rename. Hence, we need
to explicitly handle such cases while processing the event.
2021-05-26 15:13:08 -07:00
sahil839 8ec9987999 emails: Remove referrer name from 'From' header in invitation emails.
We should only show the referrer name in subject of invitation emails,
and show only 'Zulip' in the 'From' header. This helps in preventing
the email from being marked as suspicious by the detection systems
when they see an employee's name as sender of an email sent from an
unrelated domain.

The behavior is already the same for reminder invitation emails where
we do not show name and only 'Zulip' in the 'From' header.

Fixes #18256.
2021-05-26 11:41:13 -07:00
Gaurav Pandey 64bd461bad register api: Rename realm_upload_quota.
Rename `realm_upload_quota` to `realm_upload_quota_mib`
reflecting file size in mebibytes. Update frontend and bump
API_FEATURE_LEVEL.
2021-05-26 11:38:24 -07:00
Gaurav Pandey 1da818f848 register api: Rename max_logo_file_size.
Renaming `max_logo_file_size` to `max_logo_file_size_mib`
reflecting the file size in mebibyte and update the frontend.
2021-05-26 11:38:24 -07:00
Gaurav Pandey 531cf041f1 register api: Rename max_icon_file_size.
Renaming `max_icon_file_size` to `max_icon_file_size_mib`
reflecting the file size in mebibyte and update the frontend.
2021-05-26 11:38:24 -07:00
Gaurav Pandey 38614bbaec events api: Delete incorrect parameter in response.
max_logo_file_size isn't included in the responses
of GET /events api but is mentioned in the api
documentation. Hence it is deleted.
2021-05-26 11:38:24 -07:00
Mateusz Mandera 496e744053 queue_processors: Log more detailed info when marking messages as read. 2021-05-26 11:17:21 -07:00
Adam Birds c7421cd69c integrations: Add `label_create_activity` to unsupported pivotal events.
Fixes #18580.
2021-05-25 20:56:52 -07:00
Mateusz Mandera 0e42a3f117 tests: Rename helper method to generate_access_token_url_payload.
generate_access_url_payload was not a good name - there's no such thing
as "access_url" in this context - access_token_url is what was meant.
2021-05-25 14:22:46 -07:00
Mateusz Mandera 2e2bcd9619 apple: Rewrite comment in tests in generate_access_url_payload.
The original comment is worded rather unclearly, we should explain these
details better.
2021-05-25 14:22:46 -07:00
Anders Kaseorg bac96cae80 markdown: Fix Dropbox image previews.
?dl=1 causes Dropbox to send Content-Type: application/binary, which
can’t be interpreted by Camo.  Use ?raw=1 instead.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-25 13:42:29 -07:00
Sumanth V Rao 98253f261e models: Fix bug in unique_together condition on RealmPlayground.
We don't need to worry about breaking already configured playgrounds
since this tweak makes the condition less strict.
2021-05-24 23:11:41 -07:00
Anders Kaseorg 276d95b58b actions: Fix “Fix UserActivityInterval overlap bug” bug.
Commit 1a7ddd9ea3 “Fix
UserActivityInterval overlap bug” introduced a mathematically
incorrect assertion about how intervals work.  There’s a third way two
intervals could overlap: both the start and end of the old interval
could be inside the new interval.  This probably can’t happen here
because the old interval should be at least as long as the new
interval.  However, a correct overlap test can be formulated in a
simpler way anyway.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-24 18:06:05 -07:00
Abhijeet Prasad Bodas 8bcdbc720e tests: Move tornado_redirected_list to ZulipTestCase.
This is will make it easier to systematically use Django's
`capturOnCommitCallbacks` in tests outside of the main
`test_events` file which involve assertions on events.
2021-05-24 17:45:13 -07:00
AdamVB a9e73358bf
integrations: Enhance Grafana integration with alert state.
Having the alert state in the message body is useful when alert topics 
are not defined by alert description but encoded in the url.

E.g. in large environments having a topic for each alert [alerting] and [ok] would 
make it harder to properly track if an alert has been resolved.

When each alert is in a single topic, so far, the alert state has been missing.

This change will add the current alert state and a fitting icon in front
of the alert name.(Similar to the prometheus alertmanager integration)

The test cases have been amended to cover all possible alert states, even
though realistically grafana only fires the ok and alerting states via
webhook.
2021-05-24 14:25:33 -07:00
akshatdalton 503247ebfa refactor: Add class `CompiledInlineProcessor` to de-duplicate code. 2021-05-23 14:30:22 -07:00
akshatdalton 78f26b6031 minor: Use `super` to initialize subclass. 2021-05-23 14:30:22 -07:00
sahil839 8424a82dec setting: Add moderator option in email visibility setting in backend. 2021-05-23 14:20:20 -07:00
sahil839 43cdbdbf9d tests: Move a part of email_address_visibility tests to test_users.py.
We should only keep tests for changing email_address_visibility in
test_realm.py. The tests for checking the value of delivery_email
and email in the user dicts returned by 'GET users/{user_id}'
endpoint according to email_address_visibility should be in
test_users.py and not test_realm.py.

The tests of other realm settings are also arranged in the same way.

This commit makes the test more robust by checking for all
possible values of email_address_visibility and checking
emails and delivery emails values received by different
user roles.
2021-05-23 14:20:20 -07:00
Mateusz Mandera cd88a6e6fb tests: Rename APPLE_JWK to a generic name for re-use in other tests.
This can be re-used for the JWK in OIDC implementation in the follow-up
commits.
2021-05-23 13:30:17 -07:00
Mateusz Mandera d17a4f64d6 registration: Return status code 404 if invalid confirmation key passed.
We were returning 200 before - 404 is more appropriate.
2021-05-20 17:29:09 -07:00
Mateusz Mandera 40169353ad auth: Validate multiuse invite link confirmation key in social auth. 2021-05-20 17:29:09 -07:00
Aman Agrawal cba766f66f do_make_stream_web_public: Update all affected fields.
Convert this function that absolutely makes a stream web public.
We already have do_change_stream_invite_only to convert
streams to public and private streams.

We also update all the fields that should be set when a stream
is made web public.
2021-05-20 15:23:19 -07:00