Commit Graph

37270 Commits

Author SHA1 Message Date
Tim Abbott a0eb26951c docs: Update changelog for Zulip Server 2.1.6 release. 2020-06-17 00:42:28 -07:00
Tim Abbott 7872b197c2 docs: Update changelog for 2.1.5 release. 2020-06-16 23:35:39 -07:00
Tim Abbott a9877d6765 migrations: Add merged migration following 2.1.5 release. 2020-06-16 23:35:39 -07:00
Tim Abbott a2767e8c50 CVE-2020-14194: Use noopener/noreferrer for external links.
We fixed the main issue of this form in CVE-2020-9444, but the audit
done at that time only included links found in rendered_markdown; this
change completes our audit for links with target=_blank anywhere in
the codebase.
2020-06-16 23:35:39 -07:00
Tim Abbott 9f4905d7e3 CVE-2020-14215: Add migration to clear INVITED_AS_REALM_ADMIN.
This migration fixes any PreregistrationUser objects that might have
been already corrupted to have the administrator role by the buggy
original version of migration 0198_preregistrationuser_invited_as.

Since invitations that create new users as administrators are rare, it
is cleaner to just remove the role from all PreregistrationUser
objects than to filter for just those older invitation objects that
could have been corrupted by the original migration.
2020-06-16 23:35:39 -07:00
Tim Abbott 3f55bfc53b CVE-2020-14215: Fix migration 0198_preregistrationuser_invited_as.
This migration incorrectly swapped the role associated with invitation
objects between members and organization administrators, resulting in
most invitation objects that existed before the upgrade to Zulip
2.0.0-rc1 or later to be incorrectly administrator invitations.

Fixing the migration is safe and will help those installations
upgrading directly from 1.9.x to 2.1.5 or later.

A migration to fix the corrupted records will appear in an upcoming
commit.
2020-06-16 23:35:39 -07:00
Mateusz Mandera 8d2d64c100 CVE-2020-14215: Fix validation in PreregistrationUser queries.
The most import change here is the one in maybe_send_to_registration
codepath, as the insufficient validation there could lead to fetching
an expired PreregistrationUser that was invited as an administrator
admin even years ago, leading to this registration ending up in the
new user being a realm administrator.

Combined with the buggy migration in
0198_preregistrationuser_invited_as.py, this led to users incorrectly
joining as organizations administrators by accident.  But even without
that bug, this issue could have allowed a user who was invited as an
administrator but then had that invitation expire and then joined via
social authentication incorrectly join as an organization administrator.

The second change is in ConfirmationEmailWorker, where this wasn't a
security problem, but if the server was stopped for long enough, with
some invites to send out email for in the queue, then after starting it
up again, the queue worker would send out emails for invites that
had already expired.
2020-06-16 23:35:39 -07:00
Tim Abbott 4fff858aa2 templates: Fix missing quoting of attributes in HTML templates.
This fixes a bundle of issues where we were missing "" around
attributes coming from variables.  In most cases, the variables were
integers or fixed constants from the Zulip codebase (E.g. the name of
an installed integration), but in at least one case it was
user-provided data that could potentially have security impact.
2020-06-16 23:35:39 -07:00
Anders Kaseorg 87f7874a79 CVE-2020-12759: Fix reflected XSS vulnerability in Dropbox webhook.
Also check the challenge argument’s presence before using it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-16 22:46:16 -07:00
YashRE42 a0a7170f48 navbar: Use `filter._sub` instead of calling `stream_data`.
Previously, in `make_tab_data()` we were using the stream name,
which we got from the filter, to call `stream_data.get_sub_by_name()`.

This commit switches to just using `filter._sub`, which is simpler and
better.
2020-06-16 18:07:50 -07:00
YashRE42 ab2e7e097a navbar: Switch make_tab_bar conditionals to using filter/sub values.
Previously, this function relied on the return value of
`filter.get_icon()` which made it brittle.
Directly using the properties of the filter and sub object makes this
more explicit about the intentions and robust.
2020-06-16 18:07:50 -07:00
YashRE42 7ea60ea1ab filter: Store reference to `_sub` instead of `_stream_params`.
In commit 4f6377d493 we added
`_stream_params` as a way of storing attributes such as stream name
and stream privacy, this involved adding a few calls within functions
that updated these values (in order to maintain consistency).

This commit replaces `_stream_params` with an always consistent `_sub`
object and removes unnecessary `_stream_params` related code. Once the
`_sub` object is available, calls to `stream_data` may be considered
suspicious as they can often be avoided by just picking the desired
attribute off of the `_sub` object.
2020-06-16 18:07:50 -07:00
YashRE42 b0b53c8543 navbar: Use a more semantic selector in colorize_tab_bar().
Previously, this bit of code was looking for specific icons on the
navbar, but it's more semantic to just look for the `.fa` which is a
direct child of `.stream`. It also makes the code cleaner, to have a
single call here.
2020-06-16 18:07:50 -07:00
YashRE42 6766b0ab43 navbar: Just use stored stream color in `colorize_tab_bar()`.
This commit removes a redundant line of code which was converting from
hex to RGB rounding off and then converting from RGB to hex again.

This line was (mistakenly) introduced in
eb4a2b9d4e while removing a hover effect
that had become irrelevant.
2020-06-16 18:07:50 -07:00
Tim Abbott 5c0a52585c typeahead: Fix buggy code block language notice.
This was incorrectly checking for the empty string, not null.
2020-06-16 17:57:02 -07:00
YashRE42 c3d322f1a7 navbar: Improve spacing between stream name and sub_count. 2020-06-16 17:26:55 -07:00
YashRE42 63f69c48a5 navbar: Fix clickable area between sub_count and narrow_description.
Previously, there was a small dead spot in the click area between the
sub_count and narrow_description, such that the mouse cursor would
switch from pointer to the default.

This commit corrects the dead spot by adjusting the margins and styles
on navbar elements.

This should be workable, but there is scope for improvement especially
given that the current margins and paddings are messy and not very
semantic.

The end result is that the entire navbar becomes a smooth, clickable
region.
2020-06-16 17:26:55 -07:00
YashRE42 71e393575b navbar: Improve click area around stream name.
Previously the click area to open the settings modal was limited to
just the stream name (just the text). This, inconveniently, created a
lot of empty, unclickable space around the stream name.

This commit resolves the problem by:
  * Extracting the title and icon into a separate template as
    `navbar_title_and_icon.hbs` and calls this partial in
    `tab_bar.hbs`.
  * Calling the partial within an <a> tag for stream based narrows
    and in a <span> tag for non-stream narrows.
  * Making some CSS changes so that everything still renders correctly
    (visually).

This commit also:
  * Leads us to "piggy back" all stream based narrow elements on the
    `stream_settings_link` conditional. (Previously the only "piggy
    backing" was by `narrow_description` on `sub_count`, which was
    necessary for the rendering of the `(no description)` string.)

The end goal here is that the entire navbar is clickable. This is a
step towards that goal, but some of the margins on the sub count and
its ::before and ::after pseudo-elements still need to be fixed.
2020-06-16 17:26:55 -07:00
YashRE42 846383f4a0 navbar: Set user count region as click target to open settings modal.
Previously the click area to open the settings modal was limited to
just the stream name (just the text).

A nice goal to strive for here is to make the entire navbar a
continuous clickable region.

This adds the same click action as `stream_name` to the `sub_count`.

There's still scope for improvement after this change because of the
margins on `sub_count::before` and `sub_count::after` as well as
because only the text in `stream_name` is clickable.
2020-06-16 17:26:55 -07:00
YashRE42 94746977ad refactor: Extract pseudoclass based tags out of span in tab_bar.
Currently the styles for the navbar are in a confusing and ugly state.

One of the problems is that we have several styles within the `span`
including some nested pseudotag selectors within the `span`.
This is bad because it gives semantic meaning to the `span` element
which we do not intend. We should remove as many styles which intend
to target "direct children" instead of "direct children that are
spans" and (iff there are styles for the later) then substitute the
"span" for a semantically meaningful class name.

Another problem here is that these pseudotag based selectors aren't
very clear and readable, which is something we can look into
correcting now that they are separate from the `span` tag.

This is a prep commit that aims to set us on the path for further
improvements. It also enables us to switch some tags around and allows
us to use the styles in the `span` block with other selectors via `,`.
This should make no visual or behavioral changes.
2020-06-16 17:26:55 -07:00
Alex Vandiver 49a7a66004 install: Pin new apt-based installs to the latest postgresql.
Since we now support Postgres versions from 10 to 12, we might as well
have new installations start on Postgres 12 to avoid unnecessary
migration/upgrade work.
2020-06-16 17:08:16 -07:00
Alex Vandiver 6979ed9d97 install: Use the apt postgres server packages from postgres.
This allows Debian and Ubuntu administrators to reasonably seamlessly
swap over to more recent version of postgres than ships with their
distribution.
2020-06-16 17:05:46 -07:00
Alex Vandiver 03bffd3938 upgrade-zulip: Pin the postgres version to the OS default.
We would prefer to use the postgres packages from Postgres themselves,
if available.  However, this requires ensures that, for existing
installs, we preserve the same version of postgres as their base
distribution installed.

Move the version-determination logic from being computed at puppet
interpolation time, to being computed at install time and pinned into
zulip.conf.
2020-06-16 17:05:46 -07:00
Alex Vandiver e788ea52d2 upgrade-zulip: Use existing config helper functions. 2020-06-16 17:05:46 -07:00
Clara Dantas ddbde66af5 realm: Remove Google Hangouts integration.
Google  has removed the Google Hangouts brand, thus we are removing
them as video chat provider option.
This commit removes Google Hangouts integration and make a migration
that sets all realms that are using Hangouts as their video chat
provider to the default, jitsi.

With changes by tabbott to improve the overall video call documentation.

Fixes: #15298.
2020-06-16 17:02:27 -07:00
Tim Abbott f4ac4be851 sentry webhook: Check for platform support early.
Otherwise, we don't know whether the event format might not have the
`filename` or other parameters.
2020-06-16 16:49:31 -07:00
Tim Abbott b0d1386fbb api changelog: Document recent major markdown features. 2020-06-16 16:44:39 -07:00
Rohitt Vashishtha bda2e1c5a7 markdown-timestamp: Document the syntax in /help pages.
We add a separate page to advertise this feature for communities
with people in multiple timezones.
2020-06-16 16:44:39 -07:00
Mateusz Mandera 2ac6a8f829 auth: Change the "continue in browser" link in desktop flow end page.
Fixes #14828.
Giving the /subdomain/<token>/ url there could feel buggy if the user
ended up using the token in the desktop app, and then tried clicking the
"continue in browser" link - which had the same token that would now be
expired. It's sufficient to simply link to /login/ instead.
2020-06-16 16:27:53 -07:00
Sara Gulotta 1cb040647b markdown: Add support for spoilers.
This adds support for a "spoiler" syntax in Zulip's markdown, which
can be used to hide content that one doesn't want to be immediately
visible without a click.

We use our own spoiler block syntax inspired by Zulip's existing quote
and math block markdown extensions, rather than requiring a token on
every line, as is present in some other markdown spoiler
implementations.

Fixes #5802.

Co-authored-by: Dylan Nugent <dylnuge@gmail.com>
2020-06-16 16:14:10 -07:00
Tim Abbott 54604257e0 sentry: Provide more clarity around unsupported platforms. 2020-06-16 14:04:16 -07:00
jagansivam28 4576742b2f user avatar: Remove `user_avatar_file_input_error` id.
Now we can remove `user_avatar_file_input_error` id and added new class
`image_file_input_error`.we can access this class using
`#user-avatar-upload-widget .image_file_input` so that we can
have only one id at top-level and 'image_upload_widget.hbs`
can be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 c141daa624 user avatar: Remove `user_avatar_file_input` id.
Now we can remove `user-avatar-block` id and added new class
'image_file_input'.we can access this class using
`#user-avatar-upload-widget .image_file_input` so that we can have
only one id at top-level and 'image_upload_widget.hbs`
can be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 4fe066c437 user avatar: Remove `image-block` id.
Now we can remove `user-avatar-block` id and add common class `image_block`.
we can access this class using `#user-avatar-upload-widget .image_block`
so that we can have only one id at top-level and 'image_upload_widget.hbs`
can be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 9fde085536 user avatar: Remove `avatar-spinner-background` id.
Now we can remove the id `avatar-spinner-background` and access spinner
element from `#user-avatar-upload-widget .image_upload_spinner` so
that we can have only one id at top-level and 'image_upload_widget.hbs` can
be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 f609f675ae user avatar: Remove `avatar-spinner-background` id.
Now we can remove the id `avatar-spinner-background` and access spinner
element from `#user-avatar-upload-widget .settings-page-upload-text` so
that we can have only one id at top-level and 'image_upload_widget.hbs` can
be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 0e5c6fa578 user avatar: Remove `user_avatar_upload_button` id.
The upload text element is wrongly named as id=user_avatar_upload_button.
now  we can remove that id and access upload text element from
`#user-avatar-upload-widget .settings-page-upload-text` so that we
can have only one id at top-level  and 'image_upload_widget.hbs` can
be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 df4c8ab1a2 user avatar: Remove `user_avatar_delete` id.
We can remove id="user_avatar_delete" and access delete-text from
`#user-avatar-upload-widget .settings-page-delete-text` so that
we can have only one id at top-level  and 'image_upload_widget.hbs`
can be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 b3fca96254 user avatar: Remove user_avatar_delete_button id.
we can remove `user_avatar_delete_button` id and access delete button
from `#user-avatar-upload-widget .settings-page-delete-button` so that
we can have only one id at top level and 'image_upload_widget.hbs`
can be more dynamic so we can use for other similar widgets also.
2020-06-16 12:12:21 -07:00
jagansivam28 95de217326 user avatar: Rename and convert "#user-settings-avatar".
Renaming "user-settings-avatar" to "image_upload_button" since the
`user-settings-avatar` name is irrelevant/confusing for the upload
button, and converting the id into a class so that we could just have
only one outer id.
2020-06-16 12:12:21 -07:00
jagansivam28 14a77c8b9a settings: Refactor `image_upload_widget.hbs`.
We can check for the `is_editable_by_current_user` condition once in
the upper level instead of checking twice in middle for the same
conditions and to match the implementation of style realm icon and
realm logo since similar implementation between avatar, logo, the icon
will help us to use `image_upload_widget.hbs` for logo and icon
widgets also.

This likely fixes a bug with the delete text being shown incorrectly
for non-administrator users.
2020-06-16 12:12:21 -07:00
jagansivam28 20740de700 settings: Extract image_upload_widget.hbs from avatar upload widget.
We extract image_upload_widget.hbs from user avatar upload widget.
The plan is to the same HTML template for all 4 widgets (user avatar,
icon, day logo, night logo) across the two settings UIs and different
image upload widgets as possible in future.

This breaks i18n; we'll fix it in follow-up work.
2020-06-16 12:12:04 -07:00
jagansivam28 19490fe8b0 settings: Change user avatar image display HTML.
This changes the user avatar image display implementation to more
closely match how the realm icon and realm logo image features are
structured.  This is early preparatory work towards sharing this code
between the various widgets.
2020-06-16 12:03:50 -07:00
Aman Agrawal d7ca5bad62 do_delete_messages: Create a TypedDict declaration for the event.
This will help us ensure that other functions defining delete_message
event use the same formatting.
2020-06-16 11:40:45 -07:00
Priyank Patel cb7237c3aa puppeteer: Update the structure of common.get_rendered_messages.
This structure will allows us to verify correct interleaved view.
Design of the API was by Tim.
2020-06-16 11:40:30 -07:00
Hashir Sarwar ecd35b9565 push_notifications: Add support for setting counts in iOS.
This adds a new function `get_apns_badge_count()` to
fetch count value for a user push notification and
then sends that value with the APNs payload.

Once a message is read from the web app, the count is
decremented accordingly and a push notification with
`event: remove` is sent to the iOS clients.

Fixes #10271.
2020-06-16 11:26:36 -07:00
Hashir Sarwar 2bc34bb3ff test_push_notifications: Remove mocking of `get_base_payload()`.
Mocking `get_base_payload()` verifies the wrong output
when the code is actually correct. So, its better that
we call the real function here, especially when we are
adding the Apple case.
2020-06-16 11:26:36 -07:00
Ryan Rehman 62aab0d9ee message view: Fetch again when "newest" is discarded.
The previous commit introduced a bug where it was not intuitive
for the user to scroll again.
For the current narrow, new messages were fetched again only when
scrolled to the bottom as usually there are many messages displayed.

However when the edge case mentioned in the previous commit
occured, it was not very obvious that a scroll should be done
or we could already be at the bottom and could not scroll again
to trigger a fetch.
`message_viewport.at_bottom` has a relevant comment explaining
this behaviour.

The previous commit handled the rare race condition. However,
there is a possibility that the rare race condition might occur
again while we are handling the previous condition.

This commit resolves these 2 problems by performing a re-fetch
while also resetting the `expected_max_message_id` and this
approach has two benefits:

1. The reset prevents an infinite loop, if somehow the expected
   max message's id gets corrupted resulting in a situation
   where the server can never send an id greater than that even
   after fetching.

2. Even though we stop after just one re-fetch the race condition
   might recursively occur while we handle the previous race
   condition. And even though the reset prevents multiple re-fetches,
   we don't have the missing message problem.

   This is because we treat the next race condition as a new race
   condition instead of it being a continuation of the previous.

   The `expected_max_message_id` gets updated again, on receiving
   a new message. Thus it can again enter the `fetch_status` block
   as the reset value is updated again.
2020-06-16 11:11:16 -07:00
Ryan Rehman 6637f2dbb7 message list: Render new messages only after "newest" is found.
If a user sends a message while the latest batch of
messages are being fetched, the new message recieved
from `server_events` gets displayed temporarily out of
order (just after the the current batch of messages)
for the current narrow.

We could just discard the new message events if we havent
recieved the last message i.e. when `found_newest` = False,
since we would recieve them on furthur fetching of that
narrow.
But this would create another bug where the new messages
sent while fetching the last batch of messages would not
get rendered. Because, `found_newest` = True and we would
no longer fetch messages for that narrow, thus the new
messages would not get fetched and are also discarded from
the events codepath.

Thus to resolve both these bugs we use the following approach:

* We do not add the new batch of messages for the current narrow
  while `has_found_newest` = False.
* We store the latest message id which should be displayed at the
  bottom of the narrow in `fetch_status`.
* Ideally `expected_max_message_id`'s value should be equal to the
  last item's id in `MessageListData`.
* So the messages received while `has_found_newest` = False,
  will be fetched later and also the `expected_max_message_id`
  value gets updated.
* And after fetching the last batch where `has_found_newest` = True,
  we would again fetch messages if the `expected_max_message_id` is
  greater than the last message's id found on fetching by refusing to
  update the server provided `has_found_newest` = True in `fetch_status`.

Another benefit of not discarding the events is that the
message gets processed not rendered i.e. we still get desktop
notifications and unread count updates.

Fixes #14017
2020-06-16 10:47:52 -07:00
Tim Abbott 59b68aaa98 events: Remove incorrect line from new-stream handler.
This line was effectively hardcoding a specific stream_post_policy,
overriding the value already present in the event, to no purpose.

(I believe it got here via cargo-culting induced by #13787.)
2020-06-16 10:41:50 -07:00