Commit Graph

37808 Commits

Author SHA1 Message Date
Mohit Gupta 05cce86670 refactor: Change BugdownRenderingException to MarkdownRenderingException.
This commit is part of series of commits aimed at renaming bugdown to
markdown.
2020-06-26 17:08:37 -07:00
Mohit Gupta 3f5fc13491 refactor: Rename zerver.lib.bugdown to zerver.lib.markdown .
This commit is first of few commita which aim to change all the
bugdown references to markdown. This commits rename the files,
file path mentions and change the imports.
Variables and other references to bugdown will be renamed in susequent
commits.
2020-06-26 17:08:37 -07:00
Steve Howell 7c2be24cf8 tests: Refine external_authentication_methods.
This removes our last use of `check_dict` in
`zerver/tests`, except for `test_decorators.py`,
which is testing the function itself.
2020-06-26 17:00:30 -07:00
Steve Howell a46c28eea2 tests: Improve authentication_methods schema.
This also removes the last use of the "loose"
check_dict() validator in the events tests.
2020-06-26 17:00:30 -07:00
Steve Howell 5faea773b1 tests: Improve schema check in test_realm_emoji_events. 2020-06-26 17:00:30 -07:00
Tim Abbott cc8353f8b1 github: Enable new codeql-analysis feature.
This file was generated by GitHub's code analysis tutorial; we were
just approved from their waitlist.

I deleted the part to run compilers as it is not relevant for us.
2020-06-26 16:59:28 -07:00
sahil839 6eb8442a59 invites: Send user_id of the referrer instead of email in invites dict.
We send user_id of the referrer instead of email in the invites dict.
Sending user_ids is more robust, as those are an immutable reference
to a user, rather than something that can change with time.

Updates to the webapp UI to display the inviters for more convenient
inspection will come in a future commit.
2020-06-26 16:57:50 -07:00
Tim Abbott dc05b5c317 docs: Highlight that Zulip now defaults to postgres 12. 2020-06-26 16:26:57 -07:00
Alex Vandiver eb6802057a upgrade: Don't prompt in the second apt-get upgrade. 2020-06-26 16:16:12 -07:00
Alex Vandiver 16d131bb17 docs: Add an explicit section about upgrading PostgreSQL.
Fixes #15415.
2020-06-26 16:16:12 -07:00
Alex Vandiver c94aed905e docs: Remove one indirection in the docs.
Pointing at Xenial to Bionic, which just points to Bionic to Focal, is
merely an extra hop.
2020-06-26 16:16:12 -07:00
Alex Vandiver 2494d4bcae docs: Repeat the post-upgrade check stanza after each section. 2020-06-26 16:16:12 -07:00
Alex Vandiver da03761871 docs: Clarify that earlier distro upgrades cannot be done with 3.0.
As alluded to in the previous commit, only 3.0 can use the new tooling
-- indeed, it requires it, as the zulip.conf entry must be changed.
Clarify that in the upgrade steps for earlier distributions.
2020-06-26 16:16:11 -07:00
Alex Vandiver f642c7c568 docs: Update Bionic -> Focal database upgrade steps for db tool.
Update the upgrade documentation for the tool added in the previous
step.  Only the Bionic -> Focal upgrade step need be updated, because
none of the other upgrade steps can be run starting from a Zulip 3.0
installation.

Fixes #15415.
2020-06-26 16:07:39 -07:00
Alex Vandiver b7a135f037 upgrade: Add a tool to upgrade PostgreSQL.
This is based on the existing steps in the documentation, with
additional changes now that the PostgreSQL version is stored in
`/etc/zulip/zulip.conf`.
2020-06-26 16:07:39 -07:00
orientor 5629dcc8a6 openapi_docs: Display deprecated parameters with a `deprecated` tag.
In zulip.yaml, add `deprecated` tags to all parameters/keys with
`Deprecated` in the description. Then add tests to ensure that deprecated
parameters/keys will always have the `deprecated` key. Also, in
the API docs, sort the parameters according to presence of `deprecated`
key, presenting the `deprecated` keys at the end and add a `deprecated`
tag next to them.
2020-06-26 16:05:41 -07:00
Vishnu KS 0a36f04c20 i18n: Mark notification bot message in queue_processors for translation. 2020-06-26 14:57:18 -07:00
Vishnu KS cc0b3a08c9 i18n: Set the correct language for translation in add_subscriptions_backend. 2020-06-26 14:57:18 -07:00
Vishnu KS 28f5e86c7c i18n: Set the correct language in notify_topic_moved_streams. 2020-06-26 14:57:18 -07:00
Vishnu KS 5178c58209 i18n: Mark notification bot message in do_create_realm for translation. 2020-06-26 14:57:18 -07:00
Vishnu KS d42515df1f i18n: Set the correct language in do_rename_stream. 2020-06-26 14:57:18 -07:00
Vishnu KS e27921dbe3 i18n: Set the correct language for translation in send_pm_if_empty_stream. 2020-06-26 14:57:17 -07:00
Vishnu KS a174d8b755 i18n: Mark notification bot message in process_new_human_user for translation. 2020-06-26 14:57:17 -07:00
Vishnu KS ce6203906f i18n: Mark notification bot message in notify_new_user for translation. 2020-06-26 14:57:16 -07:00
sahil839 544760daa7 retention: Add link to message-retention-policy docs next to the heading.
This commit adds link to the message-retention-policy docs next to the
heading for both realm-level and stream-level settings.
2020-06-26 14:43:47 -07:00
sahil839 c650f3af47 stream_edit: Replace -1 with settings_config.retain_message_forever.
This commit replaces -1 with settings_config.retain_message_forever
for the message retention days setting.
2020-06-26 14:43:47 -07:00
sahil839 ca8f907eaf retention: Remove checks for null value of realm_message_retention_days.
This commit removes "realm_message_retention_days === null" check from
the conditionals, as we had updated the backend to replace NULL value
with -1 in 7a03e2a.
2020-06-26 14:43:47 -07:00
sahil839 cc991f58bd retention: Fix UI of stream message retention days to restrict admins.
This commit fixes the UI for stream message retention days to allow
only owners to set or update the setting.
We hide the setting for non-owners in the stream creation form and
disable it in the stream_privacy_modal.

Fixes a part of #15558.
2020-06-26 14:43:47 -07:00
orientor f188708b20 attachments: Change data type and make variable names more accurate.
Change variable `name` to `date_sent` as `name` actually stores
the date sent. Also change the data types of `name` and `create_time`
to integer. As they actually have empty decimal value.
2020-06-26 14:39:18 -07:00
Anders Kaseorg 4a88e2a732 semgrep: Ban eval.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-26 12:35:40 -07:00
Tim April 8e2a79095d mobile: Add support for alternative mobile URI.
Due to authentication restrictions, a deployment may need to direct
traffic for mobile applications to an alternate uri to take advantage
alternate authentication mechansism. By default the standard realm URI
will be usedm but if overridden in the settings file, an alternate uri
can be substituted.
2020-06-26 12:13:26 -07:00
Aman Agrawal 984c2d2777 push_notification: Remove notification if user no longer mentioned.
We send a remove mobile push notification to the users who were
no longer mentioned after the content of the message was edited.

This also corrects the notification count for the mobile apps
where a user was prior mentioned in a muted stream / topic and the
message was edited and the user is no longer mentioned now.
Hence, fixing the case where user has read all his unreads
but the notification badge on the app is still positive.

Fixes #15428.
2020-06-26 11:45:28 -07:00
Aman Agrawal 4612ee511f clear_push_notification: Upgrade method to accept multiples users.
do_clear_mobile_push_notifications_for_ids can now be used to
clear push_notification for multiple users at once. This method
loops over users, so no performance optimization is gained.
2020-06-26 11:25:59 -07:00
Tim Abbott a5be2a30fa events: Fix buggy realm_user/update events during user creation.
We've been seeing an exception in server_event_dispatch.js in
production where in large organizations, sometimes when a new user
joined, every other browser in the organization would throw an
exception processing some sort of realm_user/update event.

It turns out the cause was that when a user copies their profile from
an existing user account with a user-uploaded avatar, the code path we
reused to set the avatar properly send a realm_user/update event about
the avatar change -- for a user that hadn't been fully created and
certainly hadn't have the realm_user/add event sent for.

We fix this and add tests and comments to prevent it recurring.

(Removed an incorrect docstring while working on this).
2020-06-26 11:21:11 -07:00
Steve Howell bc2ed25d2d pointer tests: Use restart for test_collapse_event.
The restart event was always handled pretty similarly
to pointer, so I use restart events now for this
test (in preparation of eliminating pointer events).
2020-06-26 10:02:37 -07:00
Steve Howell 677f9361fe tests: Simplify test_event_collapsing.
We now use update_message_flags instead of
pointer events.
2020-06-26 10:02:37 -07:00
Steve Howell 93899c1d98 pointer tests: Fix test_one_event. 2020-06-26 10:02:37 -07:00
Tim Abbott 19d48d0667 docs: Update changelog for 2.1.7 release. 2020-06-25 17:17:42 -07:00
Anders Kaseorg 47913fb091 CVE-2020-15070: Replace eval with ast.literal_eval.
This eval function performs the inverse of the implicit
stringification that’s implied by this type-incorrect assignment in
do_update_user_custom_profile_data_if_changed:

field_value.value = field['value']

We believe there’s sufficient validation for the data being passed to
this eval that it could only have been exploited by a PostgreSQL
administrator editing the database manually.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-25 17:17:07 -07:00
Anders Kaseorg 6e98d6d73f isort: Set src_paths, remove unnecessary known_third_party overrides.
Tim thought commit 51acca2672 was needed
because he had an extra social_django directory in his checkout.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-25 16:08:08 -07:00
Tim Abbott ab918c139b docs: Create GitHub SECURITY.md file.
It seems worth participating in this GitHub standard.
2020-06-25 15:23:22 -07:00
Chris Heald 42f2399155 markdown: Escape HTML entities in inline code blocks.
This fixes an issues that causes HTML entities inside of inline code
blocks to be converted rather than being displayed literally.

The upstream python-markdown now handles this correctly, so we just use
their implementation with our changes for removing .strip(). As a result
of this migration, we switch backtick pattern to an inline processor
too.

Fixes #12056.

For the codeblock counterpart of this issue, we should follow the
upstream PR https://github.com/Python-Markdown/markdown/pull/990.

Co-authored-by: Rohitt Vashishtha <aero31aero@gmail.com>
2020-06-25 14:46:33 -07:00
Tim Abbott def6189d53 docs: Document local echo paramters for sending messages. 2020-06-25 14:44:16 -07:00
Tim Abbott 6412ea6413 api docs: Document changes in API topic encoding. 2020-06-25 14:44:04 -07:00
Tim Abbott 0ecdc663b9 api docs: Correct errors in the stream creation documentation.
* Reordered the settings relevant without stream creation to the top.
* Removed useless/misleading defaults for optional parameters.
* Clarified description of the announce and authorization_errors_fatal settings.
* Clarified that `invite_only` only applies for stream creation.
  (It's annoying to do so for its friends because they are including
   common description content and OpenAPI doesn't have a way to have
   extra content in a place you included something)

Fixes #14705.
2020-06-25 14:34:10 -07:00
Steve Howell 2fb67b3f32 refactor: Extract add/remove_subscriptions_schema.
Now we are consistent about validating color/description.

Ideally we wouldn't need to validate the
`streams_raw` parameters multiple times per
request, but the outer function here changes
the error messages to explicitly reference
the "delete" and "add" request variables.

And for the situation where the user-supplied
parameters are correct, the performance penalty
for checking them twice is extremely negligible.

So it's probably fine for now to just make sure
we use the same validators in all the relevant
places.

There's probably some deeper refactor that we
can do to eliminate the whole `compose_views`
scheme.  And it's also not entirely clear to
me that we really need to support the update
endpoint.  But that's all out of the scope of
this commit.
2020-06-25 13:52:59 -07:00
Steve Howell 6b910ff3b4 widgets: Make type checks more explicit.
Note that I don't actually convert the
checker from check_dict to check_dict_only,
because that would be a user-facing change,
but I think we can sweep a lot of things
like this after the next release.
2020-06-25 13:52:59 -07:00
Steve Howell f960df04e8 narrows: Validate negated field. 2020-06-25 13:52:59 -07:00
Steve Howell 0039c858a4 test: Extract basic_stream_fields.
This avoids some code duplication as well
as adding some missing fields.

We also use check_dict_only to prevent
folks from adding new fields to the
relevant events without updating these
tests.  (A bigger sweep comes later.)
2020-06-25 13:52:59 -07:00
Steve Howell e0ebc1307a tests: Extract ad_hoc_config_data_schema.
As the code comment indicates, we just
use a strict check here rather than
pretending that the test exercises a
more complicated schema for the config
data, which is dynamic in nature.

Cleaning up config_data is outside the
scope of this PR; my main goal is to
eliminate check_dict calls (usually in favor
of check_dict_only).
2020-06-25 13:51:24 -07:00