Commit Graph

37080 Commits

Author SHA1 Message Date
Anders Kaseorg 1ed2d9b4a0 logging: Use logging.exception and exc_info for unexpected exceptions.
logging.exception() and logging.debug(exc_info=True),
etc. automatically include a traceback.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 4b6d2cf25f logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 1a3441dbf5 confirmation: Pass realm rather than host to confirmation_url.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Hemanth V. Alluri 545f40b2c2 webhooks/circleci: Add support for super minimal payloads.
After merging PR #15355 we found that CircleCI may send
super minimal payloads. This does not seem to depend on
.circleci/config.yml since we received two different
types of payloads off of the same configuration.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-14 22:35:41 -07:00
Aman Agrawal cda7b2f539 deletion: Add support for bulk message deletion events.
This is designed to have no user-facing change unless the client
declares bulk_message_deletion in its client_capabilities.

Clients that do so will receive a single bulk event for bulk deletions
of messages within a single conversation (topic or PM thread).

Backend implementation of #15285.
2020-06-14 22:34:00 -07:00
Aman Agrawal 21ac1fb32c register: Refactor to pass client_capabilities directly.
We extract values from client_capabilities directly in
do_events_register where we decide how to process the
extracted variables.
2020-06-14 21:42:50 -07:00
Tim Abbott d97c891afe realm owners: Remove unnecessary duplicate strings. 2020-06-14 21:32:10 -07:00
sahil839 18429cfd29 realm: Allow only owners to set message retention policy for a realm.
This commits adds restriction on admins to set message retention policy.
We now only allow only organization owners to set message retention
policy.

Dropdown for changing retention policy is disabled in UI for admins also.
2020-06-14 21:23:53 -07:00
sahil839 87e72ac8e2 realm: Allow only owners to configure auth methods for a realm.
This commit adds the restriction on configuring auth methods for
admins. We now allow only owners to configure the auth methods
for realm.
2020-06-14 21:23:51 -07:00
sahil839 a6f31c3668 settings: Disable organization deactivation button for admins.
This commit adds the code to disable deactivate organization button
for admins. We now allow only owners to deactivate the organization.

The backend implementation for allowing only owners to deactivate
is already added in 81c28c1.
2020-06-14 21:20:30 -07:00
sahil839 dbd64c4362 settings_users: Only owner can deactivate other owners.
This commit adds the restriction of deactivating owners for admins
by disabling the deactivating button in the UI. Only owners are
allowed to deactivate other owners. The backend part of this is
already implemented in 86b52ef.
2020-06-14 21:20:30 -07:00
sahil839 c7a68d60f3 realm owner: Add frontend implementation of owner role.
This commit adds the option of owner role in user role dropdown
and also takes care of the restrictions while adding/removing
owner status of the user.
This commit also handles the places where we dispaly role of
the user in UI.
2020-06-14 21:20:30 -07:00
arpit551 aa70baba71 analytics: Add backend for messages read over time graph.
This commit includes changes in the api /json/analytics/chart_data to
send data for the newly added graph, as well as tests.
2020-06-14 21:19:24 -07:00
arpit551 a4b857b635 analytics: Populate data for messages_read::hour in development.
Generate data for messages_read::hour so that we can test messages_
read_over_time graph.
2020-06-14 21:19:14 -07:00
arpit551 c4b5d09283 analytics: Add LoggingCount for messages read stats.
Whenever we use API queries to mark messages as read we now increment
two new LoggingCount stats, messages_read::hour and
messages_read_interactions::hour.

We add an early return in do_increment_logging_stat function if there
are no changes (increment is 0), as an optimization to avoid
unnecessary database queries.

We also log messages_read_interactions::hour Logging stat
as the number of API queries to mark messages as read.

We don't include tests for the case where do_update_pointer is called
because do_update_pointer will most likely be removed from the
codebase in the near future.
2020-06-14 21:15:27 -07:00
arpit551 27daf38587 analytics: Derive AnalyticsTestCase class from ZulipTestCase.
The ZulipTestCase class contains various useful utility functions that
we could be using in test_counts.
2020-06-14 21:08:24 -07:00
SiddharthVarshney 177ec5b2d5 sidebars: Replace chevron with ellipsis-v icon from the sidebars.
The chevron sometime can be confused as an icon for expanding the
stream topics especially for the new users.

This commit replaces the confusing chevron icon from the stream-sidebar,
topic-list, user-presence-row, all-messages and starred-messages with
ellipsis-v icon(vertical three dots).

Fixes: #7115
2020-06-14 20:57:43 -07:00
Hashir Sarwar 47ece353e6 test_unread: Change flag condition in get_mobile_push_notification_ids.
This uses `where_active_push_notification()` instead of
the flag condition to keep it consistent with the app
code as we use the same function to filter active push
notifications everywhere else.
2020-06-14 20:43:04 -07:00
Anders Kaseorg a803e68528 email-mirror-postfix: Handle 8-bit messages correctly.
Since JSON can’t represent bytes, we encode them with base64.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 20:24:06 -07:00
Anders Kaseorg bff3dcadc8 email: Migrate to new Python ≥ 3.3 email API.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 20:24:06 -07:00
Anders Kaseorg 62f1a9da26 docs: Use Mozilla recommended SSL settings for Apache.
• Specify disabled rather than enabled protocols, so as not to disable
  TLS 1.3.
• Provide an explicit cipher suite list (Mozilla intermediate config
  version 5.4).
• Respect the browser’s preferred cipher suite ordering over the
  server’s.
• Use FFDHE2048 Diffie-Hellman parameters.
• Disable SSL session tickets.

(SSL stapling is also recommended but SSLStaplingCache cannot be
configured inside a <VirtualHost> block.)

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 20:17:12 -07:00
Dinesh d308c12ae2 auth: Add native flow support for Apple authentication.
Overrides some of internal functions of python-social-auth
to handle native flow.

Credits to Mateusz Mandera for the overridden functions.

Co-authored-by: Mateusz Mandera <mateusz.mandera@zulip.com>
2020-06-14 16:20:12 -07:00
Siddharth Varshney 5b940fc1b8 ui: Fix stream filter top margin in left-sidebar.
Because of the negative margin in `#stream_filters` the
`STREAMS` header was slighty overlapping the first
stream filter resulting sharp border.
2020-06-14 16:09:33 -07:00
Siddharth Varshney b7100d8293 ui: Fix simplebar position in the left-sidebar.
The left simplebar sometime interferes with the
chevrons on the stream filters.

This commit reduces the width of active stream filter
by adding margin-right and hence fixes the overlap of
the simplebar over the active filters
2020-06-14 16:09:33 -07:00
Tim Abbott 80589099d8 puppet: Fix typo in logic for whether to install certbot.
Fixes #15372.
2020-06-14 16:04:39 -07:00
Tim Abbott 365fed531a docs: Remove nginx reverse proxy websockets documentation.
Zulip no longer uses websockets in production, so this code was
unnecessary.
2020-06-14 15:17:32 -07:00
qnxor 6399bccc07 docs: Add Apache2 reverse proxy instructions and example.
Tweaked by tabbott to disable older SSL and remove websockets logic,
which isn't relevant in master.
2020-06-14 15:15:23 -07:00
orientor 61c64b3df9 tests: Use validate_against_openapi_schema in testing.
This adds a powerful end-to-end test for Zulip's API documentation:
For every documented API endpoint (with a few declared exceptions that
we hope to remove), we verify that every API response received by our
extensive backend test suite matches the declared schema.

This is a critical step towards being able to have complete, high
quality API documentation.

Fixes #15340.
2020-06-14 15:05:52 -07:00
Dinesh 1201ecf332 puppeteer: Add 02-message-basic tests.
We plan to combine 02-site.js, 03-narrow.js of casper test into one
03-messages-basics.js tests in puppeteer.
2020-06-14 13:19:35 -07:00
Priyank Patel 0c239c9ede puppeteer: Add get_renderered_messages and check_messages_sent.
The get_rendered_messages now returns a Map, that is formatted as:
{
  'Verona > topic 1': ['message 1', 'message 2'],
  'You and Cordelia Lear': ['private message 1']
}

We also add check_messages_sent (which was expected_messages in
casper). It now takes in a object, formatted similar to
get_rendered_messages above, instead of an array with topic and
messages.
2020-06-14 13:19:35 -07:00
Dinesh 535fce8c45 puppeteer: Add common methods needed for 02-messages-basics.js.
We include all method needed from casper except
get_rendered_messages, and expected_messages. We will want to tweak
their API when we migrate them.

We also rename bunch of method here while migrating them.
2020-06-14 13:19:35 -07:00
Hemanth V. Alluri 5bbfaa225b webhooks/circleci: Overhaul the CircleCI integration.
This change should add a lot more information to
our CircleCI integration's output.

Signed-off-by: Hemanth V. Alluri <hdrive1999@gmail.com>
2020-06-14 13:14:55 -07:00
Tim Abbott a361646221 docs: Fix references to removed puppet rules. 2020-06-14 12:47:22 -07:00
Ryan Rehman 542a3dea77 minor: Refactor the top of narrow notices.
The changes made here are as follows:
* We rename `show_history_limit_message` and `hide_history_limit_message`
  to `show_history_limit_notice` and `hide_history_limit_notice`
  respectively.
* We rename `hide_or_show_history_limit_message` to
  `update_top_of_narrow_notices` as now this function is responsible
  for updating the history limit notice as well the end of results
  notice.
* We extract 2 functions responsible for hiding and showing the end
  of results notice, similar to that of the history limit notice.
  All instances of `$(".all-messages-search-caution").hide();` are
  replaced with the call of `hide_end_of_results_notice` function.
2020-06-14 11:06:54 -07:00
Ryan Rehman e0b1096253 narrow: Show streams:all notice only after "oldest" is found.
The streams:all advertisement notice in search should only appear
after all results have been fetched to indicate we've gotten to the
beginning of the target feed.

The notice gets hidden at the start of `narrow.activate` and is
shown just after we've fetched an older batch of messages if the
"oldest" message has been found.
Previously it would get displayed after the first fetch which
takes place from `narrow.activate`. Thus we move this logic to
`notifications.hide_or_show_history_limit_message` which gets
called after a successful message fetch.

Since the home message view contains all the messages we are not
required to display this notice. However if it is already shown
we hide it as a part of `handle_post_narrow_deactivate_processes`.

To accomplish this we need to add `has_found_oldest` key to the
`fetch_status` API.
We also removed the `pre_scroll_cont` parameter as this was it's
only use case and is now redundant.
2020-06-14 11:06:54 -07:00
Aman Agrawal bcf4bf1222 recent_topics: Optimize filters_should_hide_topic.
We now get info from other libraries when it is required
in the current set of filters.
2020-06-14 11:05:57 -07:00
Aman Agrawal 57aee34845 recent_topics: Rename `muted` filter to `include_muted`.
This adds clarity to what this filter really does.
Revert muted variable rename via
e769323d0c.
2020-06-14 11:05:57 -07:00
Aman Agrawal 8a163840f4 message_events: Change narrow after updating local data.
On update_message events, we were changing narrow before we
locally updated the data, this resulted in a weird mismatch
between locally available data and that fetched from the server.
Ideally, we should not be requesting any data from the server
in most scenarios since the messages for new narrow is
locally available.

As a result, the new narrow didn't have any messages other than
a breadcrumb message. To fix this, we change narrow post
locally updating the data.

The original bug was not exactly reproduced, but a similar version
of it was simulated and was found to be fixed.

Tweaked by tabbott to preserve an optimization.
2020-06-14 11:02:50 -07:00
Tim Abbott e60771020a message_edit: Move selected ID fetching logic earlier.
This ensures that we do this fetch, which is intended to get data on
the pre-event state, before we start perturbing the message list data
structures via rerendering.
2020-06-14 10:49:22 -07:00
Tim Abbott ff7f69e9d2 emoji_picker: Add test that all emoji are in categories.
This should prevent the category of bug seen in the last commit.
2020-06-14 10:41:07 -07:00
Gittenburg 12671fbc66 emoji_picker: Add missing Flags category.
Apparently iamcal/emoji-data has a dedicated category for flag emojis.

And get_all_emoji_categories() in emoji_picker.js doesn't return the
Flags category, because we haven't declared that category in our emoji
data logic.

Note that the category looks quite sparse because it lacks country
flags, since we don't yet support emojis combined with a ‍Zero Width
Joiner (ZWJ) (see #992 & #11767).

Fixes #15303.
2020-06-14 10:41:00 -07:00
Tim Abbott 67053ff479 message_fetch: Revert home view bottom loading indicators.
In the very common event that one ends up looking at not the home view
while the browser is catching the home view up, this ended up
resulting in loading indicators being displayed at the bottom of
whatever narrowed view one was looking at incorrectly.

A proper fix for this will involve making these loading indicators
conditional on what view one is looking at.  Since one can change
views rapidly from a narrowed message list to the home view (and in
the future, between narrows), probably the best approach would be to
move the state in `message_scroll.js` the state for whether a loading
indicator is expected to be shown into the `fetch_status` data
structures, and then make all decisions about whether to show/hide a
loading indicator be calls to a function with a name like:

current_msg_list.data.fetch_status.update_newer_loading_indicator()

At least, that's probably what we should call in places like
`narrow.deactivate()`.
2020-06-14 10:05:35 -07:00
Tim Abbott a1259c2521 narrow: Hide loading indicators unconditionally before setting state.
Before 77a26d41ae, there was only one
loading indicator (at the top of the page), so the if/else logic for
hiding loading indicators was correct, if confusing.  Since we've now
added a new bottom-of-page loading indicator, it's important to have
the logic correctly reset the state to hide all existing loading
indicators on narrowing, and then just render the ones needed/desired
by the current view.

Combined with similar code in `narrow.deactivate`, this achieves the
goal that we correctly update loading indicator state when switcing
views.
2020-06-14 09:58:23 -07:00
Anders Kaseorg 674158b817 migrations: Escape more pedantically in pgroonga.0003_v2_api_upgrade.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 21:50:37 -07:00
Anders Kaseorg 0cc897d08d migrations: Escape more pedantically in pgroonga.0001_enable.
The psycopg2.SQL API unfortunately doesn’t work with
django.db.migrations.RunSQL, so we need to take a detour into
PL/pgSQL for EXECUTE and format.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 21:50:37 -07:00
rht 89af2f381d puppet: Link postgres dict symlinks to hunspell files on CentOS.
This is a temporary measure until we can find the directory of postgresql dicts
on CentOS.
2020-06-13 17:53:38 -07:00
rht 36a5ca5015 puppet: Add cyrus-sasl to memcached_packages on RedHat.
This is to mirror the sasl2-bin package on Debian.
2020-06-13 17:49:51 -07:00
rht e776d2d159 puppet: Abstract out owner:group of memcached-sasldb2. 2020-06-13 17:49:51 -07:00
Anders Kaseorg 5050fb19f6 nagios: Don’t crash on missing cron file.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00
Anders Kaseorg 69a2ad02cd circleci: Send stderr through ts too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 16:49:32 -07:00