Commit Graph

42633 Commits

Author SHA1 Message Date
Tim Abbott 9f2daeee45 upload: Use get_public_upload_url for export tarballs too.
This deduplicates the code so that we now just have one function for
constructing S3 URLs.
2021-05-27 23:26:45 -07:00
ryanreh99 5a4aecfc40 s3 uploads: Refactor to access objects via `get_public_upload_url`.
Our current logic only allows S3 block storage providers whose
upload URL matches with the format used by AWS. This also allows
other styles such as the "virtual host" format used by Oracle cloud.

Fixes #17762.
2021-05-27 23:26:42 -07:00
Abhijeet Prasad Bodas 15f78abd68 message edit: Handle topic edit tries for private messages.
Fixes #18604.
2021-05-27 23:09:33 -07:00
Abhijeet Prasad Bodas 4b30fc01e4 message edit: Extract data validity checks from check_update_message.
These checks are more related to the API than the editability
or permissions logic, so it makes sense to handle them first
before further processing the request.
Also split the main test class to separate out the tests for
this logic.

This also simplifies some tests by reducing the data setup
required to reach failure.

Tweaked by tabbott to avoid losing the topic_name.strip().
2021-05-27 23:07:59 -07:00
Abhijeet Prasad Bodas 10be798a3c message edit: Extract test helpers in their own class.
This is mostly a direct code move and a prep for splitting the
main MessageEdit test class into smaller chunks.
2021-05-27 22:52:55 -07:00
Abhijeet Prasad Bodas 3d99292b4c message edit: Fix `stream_id` description in API docs.
Since caa08d76b5, we no longer have
a common component for stream IDs in zulip.yaml, so we might as
well change the description to be specific and clear.
2021-05-27 22:52:55 -07:00
sahil839 d37ddf13a4 registration: Show spinner and disable button while processing.
This commit disables the button and shows a loading spinner on
the button when signup request is being processed to avoid race
conditions caused by user clicking on the button multiple times.

The fix is done observing that for the case when form is invalid
the whole page is rerendered and thus we do not need to remove
the spinner and enable the button again and for other errors
we redirect to some other page.

And for the validation taking place in client-side, the button
is disabled and spinner is shown, only is form is valid, by
using "$('#registration').valid()".
2021-05-27 22:51:13 -07:00
Alex Vandiver 4ffda1be87 send_email: Fix sleep logic.
This was broken in the refactor in 1e67e0f218.
2021-05-27 22:49:28 -07:00
Alex Vandiver b9948994d7 send_email: Increase loglevel of stuck ScheduledEmail messages to error.
Since these stay in the queue indefinitely, they deserve to be raised
at a higher error than just warning.
2021-05-27 22:49:28 -07:00
Alex Vandiver e080a05b05 node_cache: Serialize to structured data before hashing.
Appending data back-to-back without serializing it loses the
information about where the breaks between them lie, which can lead to
different inputs having the same hash.
2021-05-27 22:47:56 -07:00
Alex Vandiver ff9126ac1e data_import: Protect better against bad Slack tokens.
An invalid token would be treated the same as a token with no scopes;
differentiate these better.
2021-05-27 22:46:58 -07:00
Alex Vandiver 94e4f33b29 data_import: Support importing from Slack conversions in a directory.
Sometimes the Slack import zip file we get isn't quite the canonical
form that Slack produces -- often because the user has unzip'd it,
looked at it, and re-zip'd it, resulting in extra nested directories
and the like.

For such cases, support passing in a path to an unpacked Slack export
tree.
2021-05-27 22:46:58 -07:00
Alex Vandiver 8228ea2a17 import_data: Do some quick verification of Slack import formats. 2021-05-27 22:46:58 -07:00
Alex Vandiver 915e5a40d7 docs: Clarify that the Slack app and API token should be removed. 2021-05-27 22:46:02 -07:00
Alex Vandiver c9141785fd puppet: Use concat fragments to place port allows next to services.
This means that services will only open their ports if they are
actually run, without having to clutter rules.v4 with a log of `if`
statements.

This does not go as far as using `puppetlabs/firewall`[1] because that
would represent an additional DSL to learn; raw IPtables sections can
easily be inserted into the generated iptables file via
`concat::fragment` (either inline, or as a separate file), but config
can be centralized next to the appropriate service.

[1] https://forge.puppet.com/modules/puppetlabs/firewall
2021-05-27 21:14:48 -07:00
Alex Vandiver 4f79b53825 puppet: Factor out firewall config. 2021-05-27 21:14:48 -07:00
Alex Vandiver 87a109e3e0 puppet: Pull in pinned puppet modules.
Using puppet modules from the puppet forge judiciously will allow us
to simplify the configuration somewhat; this specifically pulls in the
stdlib module, which we were already using parts of.
2021-05-27 21:14:48 -07:00
Suyash Vardhan Mathur 6107679f61 minor: Fix typo in summary.
The summary for add_code_playground wrongly mentions
'Remove' instead of 'Add' in the summary. Fixed the typo.
2021-05-27 16:56:53 -07:00
sahil839 df43d6d99e stream_edit: Fix bug of disabling of "Add" button on subscribing.
We live update the "Add subscribers" UI on subscribing/unsubscribing
the stream and the field sub.can_add_subscribers is used to check
whether user is allowed to add subscribers or not.

The sub object used here is not the complete sub object and thus
sub.can_add_subscribers is undefined. We need to use the updated
sub obejcts with all the setting fields that we can get from
'stream_settings_data.get_sub_for_settings' before rendering UI.

This calculation of additional fields was done previously by
'stream_data.update_calculated_fields', but in d50462568b
call to 'updated_calculated_fields' was removed with the aim to
call 'get_sub_for_settings' before updating UI which was missed
in this part of UI update.

Fixes #18616.
2021-05-27 16:49:10 -07:00
Gilbert Bishop-White e2654b7154 docs: Add instructions for Apple Silicon Macs. 2021-05-27 16:01:42 -07:00
Gilbert Bishop-White b43cab3e3b install-shfmt: Add aarch64 support.
This fixes a problem provisioning on an ARM64 computer. Tested by
running on M1 Mac.

Fixes part of #18614
2021-05-27 16:01:42 -07:00
Gilbert Bishop-White 11257933b3 provision: Add support for aarch64 ubuntu/debian.
Install prerequisite needed on aarch64, as pip package has to build
from scratch.Tested by running on M1 Mac.

Fixes part of #18614
2021-05-27 16:01:42 -07:00
Gilbert Bishop-White d808dca28f install-shellcheck: Add aarch64 support.
This fixes a problem provisioning on an ARM64 computer. Tested by
running on M1 Mac.

Fixes part of #18614
2021-05-27 16:01:42 -07:00
Abhijeet Prasad Bodas c19b7b4db3 tests: Use tornado_redirected_to_list instead of mocking send_event. 2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas 090f2d6664 test_helpers: Remove capture_event helper.
We now consistently use the `tornado_redirected_to_list`
helper, which asserts the number of events sent too.
2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas d2bc93f6dd tests: Extract helper in test_user_status.
This deduplicates some common code for all these scenarios.
2021-05-27 13:16:22 -07:00
Abhijeet Prasad Bodas 481a890ec5 tests: Assert num_events in tornado_redirected_to_list. 2021-05-27 13:16:22 -07:00
Tim Abbott 8960e64443 docs: Provide more detailed instructions for requesting API key.
Some users had trouble getting an API key that was resolved by doing
this.

Fixes #18524.
2021-05-27 12:09:40 -07:00
Tim Abbott 05352f417a i18n: Update translation data from Transifex. 2021-05-27 00:13:49 -07:00
Anders Kaseorg 0ce015348e actions: Fix incorrect audit logging in bulk_remove_subscriptions.
modified_user=sub_info.user and modified_stream=sub_info.stream, added
by commit 6d1f9de7d3 (#16553), were
always coming from the last entry in the loop above, not from the
enclosing list comprehension.

Found by the Pylint rule undefined-loop-variable.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-26 18:12:03 -07:00
Suyash Vardhan Mathur 05b7d4ba13 apidocs: Replace hardcoded titles with generated ones.
This commit replaces the hardcoded titles
of /api pages with the auto-generated ones
from the openapi data using the markdown
processor.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 52b9c96c5d apidocs: Add support for using OpenAPI title for /api pages.
The current logic of geneerating HTML titles requires the title to be
present as a heading in the first line of .md file. However, this will
shortly be no longer true for /api pages where these are
auto-generated from OpenAPI data.  Modified the code to fetch the
title from OpenAPI data in case of such pages.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 58a8009786 apidocs: Add markdown processor for title.
Currently, the title of each endpoint are hardcoded in .md
files, but these are also added in summary parameter of
openapi. Added a markdown processor to insert the title
of a given endpoint.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur 94c4e22898 apidocs: Add summary field to endpoints.
Some pages did not have .md files, and so the summaries for these
was newly written. Commit split for easy review.
2021-05-26 17:38:01 -07:00
Suyash Vardhan Mathur e15af5ae14 apidocs: Add summary field to endpoints.
As a part of effort of removing .md files for /api pages, the
title of each page can be added into the summary OpenAPI
parameter. This also adds a nice summary visible in Swagger
and enhances the documentation. Added the parameter for all endpoints.
2021-05-26 17:38:01 -07:00
Tim Abbott 66052954e0 apidocs: Fix messages match narrow URL.
The operationId was clearly a better label.
2021-05-26 17:38:00 -07:00
Suyash Vardhan Mathur b41be98c42 apidocs: Fix playground URLs and operationIDs.
We standardized on "Code playground" for this feature, so use that
in the URLs and API documentation.
2021-05-26 17:38:00 -07:00
Anders Kaseorg cb8d9a1f8a create-db: Default dbuser and dbname to zulip.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-05-26 17:19:11 -07:00
akshatdalton 832c763c38 minor: Remove unnecessary `__init__` method in `InlineInterestingLinkProcessor`.
Subclass `Treeprocessor` takes care of the `__init__` method.
2021-05-26 17:13:03 -07:00
Adam Birds 303bb61f9c integrations: Add Sonarr Integration. 2021-05-26 17:12:28 -07:00
Adam Birds 17fe2e4a08 integrations: Add Radarr inegration. 2021-05-26 17:11:57 -07:00
Adam Birds 11e7f27031 integrations: Add Lidarr integration. 2021-05-26 17:11:24 -07:00
Tim Abbott 0294ce76b4 docs: Mention TODO/compatibility comments for events migrations. 2021-05-26 15:21:48 -07:00
Abhijeet Prasad Bodas f236a0d10d message send: Rename `always_push_notify` -> `online_push_enabled`.
This is a better name, since it clearly denotes a user
configured setting.
2021-05-26 15:19:32 -07:00
Abhijeet Prasad Bodas ddd123f133 message send: Rename `push_notify_user_ids` -> `online_push_user_ids`.
The old name `push_notify_user_ids` was misleading, because
it does not contain user ids which should be notified for
the current message, but rather user ids who have the online
push notifications setting enabled.

When the Tornado server is restarted during an upgrade, if
server has old events with the `push_notify_user_ids` fields,
the server will throw error after this rename. Hence, we need
to explicitly handle such cases while processing the event.
2021-05-26 15:13:08 -07:00
Alex Vandiver f3eea72c2a setup: Merge multiple setup-apt-repo scripts into one.
This moves the `.asc` files into subdirectories, and writes out the
according `.list` files into them.  It moves from templates to
written-out `.list` files for clarity and ease of
implementation (Debian and Ubuntu need different templates for
`zulip`), and as a way of making explicit which releases are supported
for each list.  For the special-case of the PGroonga signing key, we
source an additional file within the directory.

This simplifies the process for adding another class of `.list` file.
2021-05-26 14:42:29 -07:00
Tim Abbott 7d1db086c3 api docs: Clarify how mention flags can change.
The previous description didn't make clear that you can edit a message
to add a mention.

Also, adjust indentation to be the normal 4-space style we use for HTML.
2021-05-26 11:59:40 -07:00
Tim Abbott 55f30458e1 emails: Use 'has invited you' phrasing for invitations. 2021-05-26 11:41:13 -07:00
sahil839 8ec9987999 emails: Remove referrer name from 'From' header in invitation emails.
We should only show the referrer name in subject of invitation emails,
and show only 'Zulip' in the 'From' header. This helps in preventing
the email from being marked as suspicious by the detection systems
when they see an employee's name as sender of an email sent from an
unrelated domain.

The behavior is already the same for reminder invitation emails where
we do not show name and only 'Zulip' in the 'From' header.

Fixes #18256.
2021-05-26 11:41:13 -07:00
Gaurav Pandey 64bd461bad register api: Rename realm_upload_quota.
Rename `realm_upload_quota` to `realm_upload_quota_mib`
reflecting file size in mebibytes. Update frontend and bump
API_FEATURE_LEVEL.
2021-05-26 11:38:24 -07:00