Commit Graph

41398 Commits

Author SHA1 Message Date
Alex Vandiver 0023d561dd ci: Switch to hosting the CI images under Zulip on Dockerhub. 2021-03-31 16:54:34 -07:00
Alex Vandiver 21bafe1e1e tornado: Drop unused command-line flags.
These flags were put in place in the first commit that introduced
Tornado (9afd63692f) with unclear
utility.

Remove them, since they have never been documented, and do not have a
clear need.
2021-03-31 14:19:38 -07:00
Alex Vandiver de46edf966 docs: Remove unused X-Client-IP header from haproxy configuration.
This is unrelated to the X-Real-IP configuration that nginx will add,
and is unused by Zulip.  Remove it, to reduce confusion.
2021-03-31 14:19:38 -07:00
Alex Vandiver 07779ea879 middleware: Do not trust X-Forwarded-For; use X-Real-Ip, set from nginx.
The `X-Forwarded-For` header is a list of proxies' IP addresses; each
proxy appends the remote address of the host it received its request
from to the list, as it passes the request down.  A naïve parsing, as
SetRemoteAddrFromForwardedFor did, would thus interpret the first
address in the list as the client's IP.

However, clients can pass in arbitrary `X-Forwarded-For` headers,
which would allow them to spoof their IP address.  `nginx`'s behavior
is to treat the addresses as untrusted unless they match an allowlist
of known proxies.  By setting `real_ip_recursive on`, it also allows
this behavior to be applied repeatedly, moving from right to left down
the `X-Forwarded-For` list, stopping at the right-most that is
untrusted.

Rather than re-implement this logic in Django, pass the first
untrusted value that `nginx` computer down into Django via `X-Real-Ip`
header.  This allows consistent IP addresses in logs between `nginx`
and Django.

Proxied calls into Tornado (which don't use UWSGI) already passed this
header, as Tornado logging respects it.
2021-03-31 14:19:38 -07:00
Sundar Guntnur 5aefb5e656 webhook: Catch potential JsonableError when parsing widget_content.
The `widget_content` key is expected to contain a string which parses
as JSON; in the event that it does not, log the error and notify the
bot owner, instead of failing silently.

Fixes #16850.
2021-03-31 13:31:42 -07:00
Arun Sankar 954aced781 populate_db: Add non ascii and non bmp characters to Stream names.
Added non ascii and non bmp characters to stream names.

A Stream Name will now consist of a random stream name +
a number (to avoid name duplicates) + a 15% to contain
a emoji.
2021-03-31 13:00:56 -07:00
Arun Sankar b26b647b1d populate_db: Add non ASCII and non BMP characters to usernames.
Added non ASCII and non bmp characters to full name.

Created a new list for non_ascii_names and emojis
to store them explicitly.

A full name will now consist of first name +
(a non ASCII name or a plain middle name) + (a emoji
or a plain last name).

First name will not have any non ASCII or non bmp text
as it is also being used as email.
2021-03-31 13:00:56 -07:00
Riken Shah bfc1e45a91 password_change: Avoid unnecessary redirect to the login page.
This commits adds on to 9884226f, which was added to
handle a rare race condition that occurs when the
session hash is not updated by the backend during the
password change process.

It handles a variant race situation where the request was initiated
before/during the password change event and completed after it was
completed. Hence, forcing the page to redirect to the login page.
2021-03-31 11:03:09 -07:00
Tim Abbott 85b3157b47 css: Fix webapp alert styling incorrectly applying to portico.
In a00f5dd90e, we needed to move the
`alert-box` styles from alerts.css to be visible in portico pages.
However, when doing so, we incorrectly moved all of alerts.css, which
also has styles for `alert` and` alert-error` designed to make it
convenient to include hidden elements for potential errors in the
webapp settings UIs directly in the HTML template (and then use
show/hide to manage them).

We fix this by moving just the alert-box scope to the common
components.css module, which is designed as the place for styles
shared between the webapp and portico pages.

This fixes an issue where the error messages for wrong password and
the like were invisible :(.
2021-03-31 10:06:40 -07:00
m-e-l-u-h-a-n aea31eb31f api: Add REALM_DEACTIVATED error code.
In `validate_account_and_subdomain` we check
if user's realm is not deactivated. In case
of failure of this check, we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of users with deactivated realms for non-browser
clients.

So we register a new REALM_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises RealmDeactivatedError if user's realm
is deactivated.

This error is also documented in
`/api/rest-error-handling`.

Testing: I have mostly relied on automated
backend tests to test this.

Fixes #17763.
2021-03-31 08:46:13 -07:00
m-e-l-u-h-a-n 2eeb82edba api: Add USER_DEACTIVATED error code.
In validate_account_and_subdomain we check if
user's account is not deactivated. In case of
failure of this check we raise our standard
JsonableError. While this works well in most
cases but it creates difficulties in handling
of deactivated accounts for non-browser clients.

So we register a new USER_DEACTIVATED error
code so that clients can distinguish if error
is because of deactivated account. Following
these changes `validate_account_and_subdomain`
raises UserDeactivatedError if user's account
is deactivated.

This error is also documented in
`/api/rest-error-handling`.

Testing: I have mostly relied on automated
backend tests to test this.

Partially addresses issue #17763.
2021-03-31 08:46:13 -07:00
Signior-X 66d7e711b2 refractor: Replace manual autosize of textarea with compose_ui autosize.
This commit make compose_ui.autosize_textarea handle most of the autosize
logic of the textarea. It audits for any logic that is trying to do
autosize manually and replace it with compose_ui.autosize_textarea.
This allows to have better check for when the textarea is autosized.

Mainly done this so that we can have a check on #compose-textarea when
to autosize and when not to, thus helping to have all the logic in only
one function.
2021-03-31 07:49:47 -07:00
Riken Shah a527d196d7 puppeteer_tests: Use more stable selectors.
This commit changes some fragile selectors (like
`a[href=#link]`) to more stable selectors because they
are more prone to break from doing something normal
like adding another link in the app.

It also solves an inconsistency in `07-navigation.ts`,
where the subscription overlay was opened by clicking
on the header stream instead of the menu list.

It also fixes a rare flake (in `07-navigation.ts`), where
the close button of subscription overlay was not clicked
due to a delay in the opening. The delay was caused by
clicking the header stream to open subscription overlay
which caused unnecessary loading of the stream
setting(Verona).
2021-03-31 09:00:51 -04:00
Riken Shah c5f0806308 puppeteer_tests: Rename navigate_to func to navigate_using_left_sidebar.
As we are using the 'navigate_to' function to navigate
the links on the left sidebar, It'd be more clear to rename
the function to 'navigate_using_left_sidebar'.

Also adding '#left-sidebar' when selecting the element,
to be sure it will select the element from the left sidebar.
2021-03-31 09:00:51 -04:00
Riken Shah e61f73c239 refactor: Lift out log-out call from individual tests to `common.ts`.
We recently added the commit to add the log-out call
after each test (52706908b).

This commit cleans that approach by using
just one log-out call after the test is executed at
`common.ts`
2021-03-31 08:57:18 -04:00
sahil839 f73d101854 stream: Use 'hidden.bs.modal' event for enabling mouse background events.
We now unconditionally enable backgroung events when 'hidden.bs.modal'
event is triggered on closing of modal. We do not need to handle them
separateley for closing modal by close_modal, data-dismiss or escape.
We handle this by single handler for modals in settings and subscription
overlay.

Fixes #16688.
2021-03-30 17:02:46 -07:00
sahil839 46db1da9a9 puppeteer: Wait till background mouse events are enabled in settings test.
This commit adds waitForFunction to wait till the background mouse events
are enabled after closing the modal in the settings test.

This change is needed to avoid the failure that will be caused after we
change the code to handle re-enabling of mouse events only at one place
using 'hidden.bs.modal' event of bootstrap, as this event is fired only
after the modal is completely hidden, and we would want the mouse events
to be enabled before using clicks in further tests.
2021-03-30 17:02:46 -07:00
sahil839 7489213768 modals: Do not remove modal element from DOM after closing the modal.
This commit changes the code to not remove the modal element from DOM
after closing the modal for the deactivation stream modal and stream
privacy modal.

This is a prep commit for enabling background mouse events
unconditionally using 'hidden.bs.modal', because removing element
using '.remove' remove all events attached to it and this will also
remove the 'hidden.bs.modal' event which we do not want.

This remove behavior is inconsistent as we remove some of the modals
but do not remove some, so for now we are not removing the modal
after closing but it is anyway removed before opening a modal to handle
case of having two elements of same id and avoids any bugs.
This behavior of when to remove the element from DOM and when to not
remove needs to be discussed and may be modified in future.
2021-03-30 17:02:46 -07:00
sahil839 677ba4bc25 streams: Show warning when unsubscribing a private stream.
We show a modal as a warning when unsubscribing a private stream
because it is a irreversible action and one cannot re-subscribe
tovit until added by other member of stream.

Fixes #9254.
2021-03-30 17:02:44 -07:00
sahil839 60f486d941 confirm_dialog: Add close-modal-btn class to cancel buttons.
This commit adds "close-modal-btn" class to cancel button in
the modal and cross icon.

We do this change because we would add a modal for unsubscribing
from private stream in further commit using confirm_dialog module.
We would need to avoid the unexpected closing of stream settings
on closing the modal which can be done by calling 'e.stopPropagation'
to prevent propagating of events in other elements.
Thus, adding this class will mean that the handler used for stream
privacy modal for this same task will be used for unsubscribe modal
also.
2021-03-30 16:51:42 -07:00
sahil839 208f3d7f1b confirm_dialog: Focus on the yes_button after opening the modal.
This commit adds the code to focus the confirmation/yes button
after opening the modal such that the user can just confirm by
pressing Enter.
2021-03-30 16:51:42 -07:00
sahil839 2e6723f337 stream: Add close-modal-btn class to cancel btn in deactivate stream modal.
This commit adds 'close-modal-btn' class to cancel button and cross icon
in the deactivation stream modal.

This change is a prep commit for enabling background events on
'hidden.bs.modal' event. As we would enable background events in furhter
commit using the 'hidden.bs.modal' event, we would need to remove the
'hide.bs.modal' event of deactivation_stream_modal which removes the
element from DOM.
When we remove this event we would need a e.stopPropagation call to avoid
unexpected closing of stream settings which was not a problem previously
because the element was being removed from DOM before actual closing of
modal.
So instead of adding a different handler, we can use the handler used
for stream privacy modal here by adding this new class.
2021-03-30 16:51:42 -07:00
sahil839 4d3a2c10fb stream_edit: Rename class of cancel btn in stream-privacy modal.
This commit renames the class of both cancel button and the cross
icon to close-modal-btn.

This change is a prep commit for enabling background events on
'hidden.bs.modal' event. As we would enable background events in
furhter commit using the 'hidden.bs.modal' event, we would need to
remove the 'hide.bs.modal' event of deactivation_stream_modal which
removes the modal element from DOM.
When we remove this we would need a e.stopPropagation call to avoid
unexpected closing of subscription settings, which was not a problem
before as the element was removed from DOM before the actual closing
of modal.
So instead of adding a separate `e.stopPropagation' call, we can use
the same handler that is being used for stream privacy modal and this
is the reason the class name of cancel button of privacy modal is
being changed.
2021-03-30 16:51:42 -07:00
sahil839 0fb4045807 subs: Do not remove stream row when unsubscribing using hotkey.
We do not remove the stream row instantly from the subscribed list in
subscription overlay when unsubscribing from public streams in most
of the cases but we do so when unsubscribing using hotkey.
This commit makes it consistent by not removing the stream row on
unsubscribing using hotkey.

We also not remove the 'active' class as streams settings is still
open in the right seciton and this behavior is also consistent with
the other ways of unsubscribing.

Note that this behavior is only for public streams, we remove the
stream row in case the stream is private, as user cannot
resubscribe himself and this behavior is consistent across all ways
of unsubscribing.
2021-03-30 16:51:42 -07:00
Tim Abbott 36e320cf18 api docs: Fix link to old Help Center linkifiers URL. 2021-03-30 16:51:42 -07:00
aryanshridhar ce148ce770 frontend: Fix broken label tag and removed label expression.
- Rectifies broken label tag having a misleading 'for' attribute.
- Removed 'name' attribute from unlabelled span tag.
- Removed label expression from DropdownListWidget to built an,
  abstraction for control group only.

Fixes #17311.
2021-03-30 16:14:42 -07:00
aryanshridhar 134a6f8bba edit_bot: Fix dropdown username capitalization.
Restructured dropdown_list_widget template to unwrap label tag
from the control group, hence defaulting the edit_bot
dropdown items to their original text size.
2021-03-30 16:14:42 -07:00
Sumanth V Rao e12f682e2e markdown: Include text & url in `topic_links` parameter of our API.
The linkifier code now includes both the shortened text and the expanded
URL, sorted by the order of the occurrence in a topic. This list is passed
back in the `topic_links` parameter of the /messages and the /events APIs.

topic_links earlier vs now:

earlier: ['https://www.google.com', 'https://github.com/zulip/zulip/32']

now: [{'url': 'https://www.google.com', 'text': 'https://www.google/com},
      {'url': 'https://github.com/zulip/zulip/32', 'text': '#32'}]

Similarly, the topic_links local echo logic in the frontend now returns
back an object.

Fixes: #17109.
2021-03-30 15:53:07 -07:00
Tim Abbott de1660e407 message_flags: Add missing set_global import. 2021-03-30 15:53:03 -07:00
Sumanth V Rao 7f6fe128f6 hotspots: Add TUTORIAL_ENABLED setting to toggle INTRO_HOTSPOTS.
We add a TUTORIAL_ENABLED setting for self-hosters who want to
disable the tutorial entirely on their system. For this, the
default value (True) is placed in default_settings.py, which
can be overwritten by adding an entry in /etc/zulip/settings.py.
2021-03-30 14:46:42 -07:00
Anders Kaseorg fdefc4275a computed_settings: Remove unused TUTORIAL_ENABLED setting.
It’s unused as of commit 88bec16452
(#6621).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-03-30 14:44:09 -07:00
YashRE42 b56ad453b3 message_flags: Add coverage for unstar_all_messages call.
This commit also marks message_flags.js to have 100% coverage.
2021-03-30 14:40:13 -07:00
YashRE42 feabaebcad message_flags: Add coverage for get_starred_message_ids_in_topic call. 2021-03-30 14:40:13 -07:00
YashRE42 f461139d26 message_flags: Verify early return when starring local echo. 2021-03-30 14:40:13 -07:00
YashRE42 e0cedb15fc message_flags: Verify early return from update_starred_flag. 2021-03-30 14:40:13 -07:00
YashRE42 53473decc7 message_flags: Add coverage to setTimeout call from send_read. 2021-03-30 14:40:13 -07:00
YashRE42 bb51fe075a message_flags: Verify early return from send_read on_sucess. 2021-03-30 14:40:13 -07:00
YashRE42 ba8a6affd4 message_flags: Add collapse and uncollapse coverage. 2021-03-30 14:40:13 -07:00
YashRE42 6394a59a5f message_flags: Use override to mock ui.update_starred_view. 2021-03-30 14:40:13 -07:00
lukem1 105a5a95ee topic_mutes: Filter deactivated streams from get_topic_mutes.
Updated database query to filter out deactivated streams from the
return of the get_topic_mutes method. Added optional
include_deactivated parameter to the method to make the behavior
default but overrideable. Added test case in test_muting for these
changes. Fixes blueslip warnings thrown by muting.js set_muted_topics
when passed deactivated streams via page_params.
2021-03-30 12:11:35 -07:00
Steve Howell ec80d2d5db sidebar toggles: Lift code to display PM counts.
We now consistently set the PM counts for the right
sidebar toggle in unread_ui, similar to what we
do for the overall counts in the left sidebar toggle.
(Use a thin window to see the code in action.)

This breaks a dependency cycle.

In passing I improve the test coverage for the
actual job that pm_list still does (updating its
own total count in the "Private Messages" section).
2021-03-30 12:07:51 -07:00
Vishnu KS 92316ef4d1 statuspage: Properly detect the update is for component or incident.
The value of "status_indicator" can be "none" for both the component
and incident updates[1]. Also, it is not at all necessary that the value of
"status_indicator" is always "none" for incident updates[2][3]. So our previous
logic of using the value of "status_indicator" to determine whether
the update is that of a component or incident was incorrect. Instead, we
should use "incident" or "component" keys to determine the type of update.

This commit fixes issues [1] and [2] in sentry.

1. https://sentry.io/organizations/zulip/issues/2303217561
2. https://sentry.io/organizations/zulip/issues/2303197407
3. https://support.atlassian.com/statuspage/docs/enable-webhook-notifications/
2021-03-30 12:06:09 -07:00
Mateusz Mandera 353e1a2016 migrations: Subscription.is_user_active denormalization - final step.
With the previous two commits deployed, we're ready to use the
denormalization to optimize the query.

With dev environment db prepared using
./manage.py populate_db --extra-users=2000 --extra-streams=400
this takes the execution time of the query in
bulk_get_subscriber_user_ids from 1.5-1.6s to 0.4-0.5s on my machine.
2021-03-30 09:29:36 -07:00
Mateusz Mandera 1e37fde59f migrations: Subscription.is_user_active denormalization - step two.
This adds a migration to do the backfill and add the index that'll be
used to optimize the bulk_get_subscriber_user_ids query in the next
commit.
2021-03-30 09:29:36 -07:00
Tim Abbott 53ed759fc1 users: Fix ordering issue with deactivating bots.
The new comment explains the issue in some detail, but basically if we
deactivate the bots first, then an error partway through is corrected
by a retry; if we deactivate the user first, then we may leak
undeactivated bots if a failure occurs.
2021-03-30 09:21:41 -07:00
Mateusz Mandera f329878376 migrations: Subscription.is_user_active denormalization - step one.
This adds the is_user_active with the appropriate code for setting the
value correctly in the future. In the following commit a migration to
backfill the value for existing Subscriptions will be added.

To ensure correct user_profile.is_active handling also in tests, we
replace all direct .is_active mutation with calls to appropriate
functions.
2021-03-30 09:19:03 -07:00
Mateusz Mandera d236d3f738 users: Improve db transaction structure in user (de)activation process.
These procedures should be done atomically overall, with the exception
of the code that sends events to avoid block if there's a delay
communicating with Tornado.
We add the savepoint=False on underlying function that already
executes inside an atomic context - to avoid the overhead of creating
savepoints where they aren't needed.
2021-03-30 09:15:24 -07:00
Mateusz Mandera 0e6d230804 users: Fix do_deactivate_user to save is_mirror_user.
This was a bug - is_mirror_user was not listed in update_fields despite
being changed.
2021-03-30 09:15:24 -07:00
sahil839 54be0dd1a4 streams: Add moderators option in stream_post_policy.
This commit adds a new option of STREAM_POST_POLICY_MODERATORS
in stream_post_policy which will allow only realm admins and
moderators to post in that stream.
2021-03-30 09:06:20 -07:00
sahil839 a061240251 streams: Extract helper for checking access to stream based on post-policy.
We extract a helper which checks whether to allow the sender to send the
message to a stream according to the stream_post_policy. The purpose
of extracting it out is to avoid additional code for checking the access
for bot owners in case of bot sending the messages and instead calling
the handler two times - one time for sender and one time for bot owner if
sender is a bot.
2021-03-30 09:06:20 -07:00