Commit Graph

2421 Commits

Author SHA1 Message Date
Steve Howell b3a69154a6 refactor: Export compare_for_relevance.
This future-proofs us a bit more for test coverage.
2020-01-03 14:58:05 -08:00
Steve Howell 0985842c62 Fix sorting for broadcast mentions.
We had a potentially nasty bug where we
weren't guaranteeing that all/stream/everyone
collated in consistent ways inside of
`compare_people_for_relevance`, which can
send certain types of sort algorithms into
an infinite loop. I doubt this ever happened
in practice, but it's obviously worth fixing.

Now we also have a clear tiebreaker between
any two all/everyone/stream mentions, which
is the idx field.

Finally, this should be a bit more efficient.
2020-01-03 14:58:05 -08:00
Steve Howell 758786ab87 refactor: Extract broadcast_mentions.
This will be helpful for testing.
2020-01-03 14:58:05 -08:00
Steve Howell 773161cbb7 tests: Test "all" mentions more realistically.
We don't have people named "all".  Instead, we
create pseudo person objects with email/full_name
of "all" (along with some other fields).  The tests
now reflect this.
2020-01-03 14:58:05 -08:00
Steve Howell d227988519 tests: Split up sort_recipient tests. 2020-01-03 14:58:05 -08:00
Steve Howell cde01aeeb0 tests: Avoid list mutation.
To test dups we can just create a new list.
2020-01-03 14:58:05 -08:00
Steve Howell 5c43180a70 tests: Use names for test objects. 2020-01-03 14:58:05 -08:00
Steve Howell b5d0eab0c6 dict: Add filter_values() method.
This method can help us avoid some memory
allocations.
2020-01-02 12:03:45 -08:00
Steve Howell 7229a943f0 tests: Use add_in_realm for "me" in people tests.
This is more realistic for testing.
2020-01-02 12:03:04 -08:00
Steve Howell 54cb857fee refactor: Rename people.get_rest_of_realm().
We want to mostly deprecate this function (see
the comment I added), so I gave it a more specific
name.

Ideally I'd just fix `stream_create`, but it does
use this function in a couple places, and it's helpful
to reuse the same sort here.  In one place stream_create
actually unshifts the "me" user back to the top of the
list, which makes sense for its use case.
2020-01-02 12:03:04 -08:00
Steve Howell 6e93f330c6 bug fix: Fix huddles in "Private Messages".
If two user_ids in a recent huddle have ids
that sort lexically differently than numerically,
such as 7 and 66, then we were creating two
different buckets in pm_conversations.

This regression was introduced in
263ac0eb45 on
November 21, 2019.
2020-01-02 11:59:58 -08:00
Steve Howell 0e68387975 refactor: Have pm_conversations take user_ids.
Instead of having our callers pass in a possibly
non-canonical version of a user_ids_string, just
have them pass in a list.

The next commit will canonicalize the sort.
2020-01-02 11:59:58 -08:00
Steve Howell ab6f4af33a tests: Use tricky server data in unit tests.
The server may send us ids in the order
[11, 2], instead of [2, 11].  We don't want
to rely on server behavior, regardless, for
the sort.

Our tests now show we process that data.

The current code is is still buggy and causes
us to show the same huddle two different times
for situations where the lexical sort doesn't
match the numerical sort.

This happens on czo often, where Tim is user
7, and his id sorts lexically after ids like
58, 622, 4444, etc.
2020-01-02 11:59:58 -08:00
Steve Howell 0711c7ea49 performance: Avoid dup calls to subscribed_streams().
In stream_sort.sort_groups, we now have the caller
pass us in the list of streams, since they are getting
them anyway.
2019-12-30 09:50:22 -08:00
Steve Howell 631811e686 streams: Add BinaryDict for stream_data.
This should make any operation on subscribed
streams faster (we won't need to filter out
unsubscribed streams every time).

I started writing this before I realized we
had a bug where we call `subscribed_streams`
in a nested loop.

After fixing the bugs, this is not as much of
a bottleneck, but it's still a speedup in many
important places:

    * build left sidebar
    * every keystroke in search bar
    * first keystroke in making #stream_links
    * every keystroke in compose stream box

The streams settings code is kinda complicated.
It does a non-deterministic sort of the "others"
bucket when you add elements to the left panel.
They get hidden, anyway.  Our values() call now
puts subscribed streams first.  It never guaranteed
order, but putting subscribed streams first is
probably a good behavior for most situations.
2019-12-30 09:50:20 -08:00
Steve Howell a3512553a8 streams: Add LazySet for subscribers.
This defers O(N*S) operations, where

    N = number of streams
    S = number of subscribers per stream

In many cases we never do an O(N) operation on
a stream.  Exceptions include:

    - checking stream links from the compose box
    - editing a stream
    - adding members to a newly added stream

An operation that used to be O(N)--computing
the number of subscribers--is now O(1), and we
don't even pay O(N) on a one-time basis to
compute it (not counting the cost to build the
array from JSON, but we have to do that).
2019-12-30 09:47:55 -08:00
Steve Howell e804f39f0e performance: Avoid expensive call in stream_data.is_active.
Calling `set_filter_out_inactives` is expensive, since we
count up the number of subscribed streams, which iterates
through all your streams, creates a new list of subscribed
streams, then counts them.

In my dev setup, I created 700 streams, and this shaved
about 700ms off of the initial call to `build_stream_list`.
2019-12-30 09:45:46 -08:00
Steve Howell 70470dea1c settings: Use correct email when searching users.
If we aren't showing users emails, then we don't
want to use emails in the search.

And if we are showing users emails, we want to
search on the email that's displayed to them.
For admins this will be delivery_email.

For regular users we arguably shouldn't search
on emails either, since it mostly causes confusion,
but this commit just preserves the current
behavior for those users (unless `show_email` is
false).
2019-12-30 09:43:24 -08:00
Steve Howell 3e4326afda refactor: Extract email_for_user_settings.
We want to be able to unit test this value,
since it's conditional on several factors:

    - am I an admin?
    - can non-admins view emails?
    - do we have delivery_email for the user?

I'm mocking show_email in the tests, since the
show_email code is in `settings_org` and
kind of hard to unit test.  It's not impossible,
but it's too much for this commit.  (Either
we need to extract it out to a nice file or
deal with mocking jQuery.  That module is
mostly data-oriented, so it would be nice
to have something like `settings_config` that
is actually pure data.)
2019-12-28 11:22:24 -08:00
Steve Howell 3a95be2f2f refactor: Extract matches_user_settings_search.
This was duplicate code.  I'm moving it to people
for pragmatic reasons--it's hard to unit test stuff
in settings_users.js due to all the jQuery.

It's also nice to have all people-related search
code in one place, just for auditing purposes.
2019-12-28 11:22:24 -08:00
Steve Howell ab34ee0800 search performance: Stop at max_items.
Once we have max_items results, stop trying
to get more items.

This should really help large realms when
you do a search on streams that turns up
more than N streams (where N is about 12).
We won't even bother to find people.
2019-12-28 11:09:28 -08:00
Steve Howell 4141abc171 search: Slightly speed up stream highlighting.
This isn't a huge speedup, but it's an easy
code change.

We remove the two-liner highlight_with_escaping,
which was only called in one place, and when
we inline it into the caller, we can pull the
first line, which builds the regex, out of the
loop.
2019-12-28 11:09:23 -08:00
Steve Howell 49cd719273 tests: Verify subscriptions more thoroughly.
When you subscribe/unsubscribe yourself, you want
to make sure these calls work too:

    subscribed_subs
    unsubscribed_subs
2019-12-28 07:29:51 -05:00
Steve Howell 32a1ef20d1 minor: Extract helper for search tests. 2019-12-28 05:43:04 -05:00
Steve Howell f3ebb5fbee test cleanup: Extract a sorter() helper. 2019-12-28 05:43:04 -05:00
Steve Howell 9ac2fe2826 test cleanup: Extract a matcher() helper. 2019-12-28 05:43:04 -05:00
Tim Abbott e72da08f09 narrow: Fix streams:all notice appearing too early.
The streams:all adveritsement notice in search should only appear
after we've already received the response from the server, to avoid a
mix of problems ranging from misplaced loading indicator to scrolling
issues to the notice just being distracting while you're waiting for
the server to return results.

We need to add a pre_scroll_cont parameter to the message_fetch API,
since adding this notice would otherwise potentially throw off the
scroll positioning logic for which message to select.

Fixes #13441.
2019-12-10 18:10:39 -08:00
Mohit Gupta a0c11b6c78 narrow: Use search reading behavior in all searches.
In 452e226ea2 and
648a60baf6, we changed how `search:`
narrows work to:

(1) Never mark messages as read inside searches (search:)
(2) Take you to the bottom, not the first unread, if a `near:` or
    similar wasn't specified.

This is far better behavior for these use cases, because in these
narrows, you can't actually see all the context around the target
messages, so marking them as read is counterproductive.  This is
especially important in `has:mention` where you goal is likely
specifically to keep track of which threads mentioning you haven't
been read.  But in many other narrows, the current behavior is
effectively (1) setting the read bit on random messages and (2) if the
search term matches many messages in a muted stream with 1000s of
unreads, making it hard or impossible to find recent search matches.

The new behavior is that any narrow that is structurally a search of
history (including everything that that isn't a stream, topic,
pm-with, "all messages" or "private messages") gets that new behavior
of being unable to mark messages as read and narrows taking you to the
latest matching messages.

A few corner cases of interest:
* `is:private` is keeping the old behavior, because users on
  chat.zulip.org found it confusing for `is:private` to not mark
  messages as read when one could see them all.  Possibly a more
  complex answer is required here.

* `near:` narrows are getting the new behavior, even if it's a stream:
  + topic: narrow.  This is debatable, but is probably better than
  what was happening before.

Modified significantly by tabbott for cleanliness of implementation,
this commit message, and unit tests.

Fixes #9893.  Follow-up to #12556.
2019-12-10 16:26:06 -08:00
Tim Abbott 2eae0b3e57 notifications: Support wildcard_mentions_notify for desktop.
In 1fe4f795af, we added the
wildcard_mentions_notify setting, which controls whether wildcard
mentions should be treated as mentions for the purposes of
notifications.  The original implementation focused on the more
important area of email/push notifications, and neglected to address
desktop notifications for wildcard mentions.

This change makes the wildcard_mentions_notify flag behave correctly
for desktop/sound notifications, including unit tests.

Fixes #13073.
2019-12-10 13:12:36 -08:00
Tim Abbott 22cefeede8 notifications: Extract should_send_*_notification for testing. 2019-12-10 12:54:36 -08:00
Tim Abbott 016487163f node tests: Refactor notifications tests for better reuse. 2019-12-10 12:47:21 -08:00
Tim Abbott fa7ae6fa7f node tests: Fix missing coverage on stream_data.js.
This fixes a testing coverage regression in
d5f005fd61.
2019-12-09 18:11:12 -08:00
Nat1405 d5f005fd61 wildcard_mentions_notify: Add per-stream override of global setting.
Adds required API and front-end changes to modify and read the
wildcard_mentions_notify field in the Subscription model.

It includes front-end code to add the setting to the user's "manage
streams" page. This setting will be greyed out when a stream is muted.
The PR also includes back-end code to add the setting the initial state of
a subscription.

New automated tests were added for the API, events system and front-end.
In manual testing, we checked that modifying the setting in the front end
persisted the change in the Subscription model. We noticed the notifications
were not behaving exactly as expected in manual testing; see
https://github.com/zulip/zulip/issues/13073#issuecomment-560263081 .

Tweaked by tabbott to fix real-time synchronization issues.

Fixes: #13429.
2019-12-09 16:09:38 -08:00
Rohitt Vashishtha 85c669e366 markdown: Remove redundant checks from /me.
If a message begins with /me, we do not have any cases where the
rendered content would not begin with `<p>/me`. Thus, we can safely
remove the redundant checks both on the backend and frontend.
2019-12-03 17:17:10 -08:00
Tim Abbott 8b55a310f1 typing: Fix invalid typing notifications for stream messages.
In e42c3f7418, we made the assumption
that compose_pm_pill.get_recipient() would return no users for stream
messages.  It turns out, due to the confusing name of
compose_state.recipient (which we just renamed to
compose_state.private_message_recipient), this assumption was wrong.

As a result, when composing a stream message using the reply hotkeys,
we'd end up sending typing notiifcations to the person who sent the
message we're replying to as though a PM was being composed.

We fix this by avoiding passing an (expected to be unused) value for
private_message_recipient to compose_state.start.
2019-12-02 09:31:16 -08:00
Tim Abbott ea7c6d395f compose_state: Rename compost_state.recipient to be about PMs only.
The compose_state.recipient field was only actually the recipient for
the message if it was a private_message_recipient (in the sense of
other code); we store the stream in compose_state.stream instead.

As a result, the name was quite confusing, resulting in the
possibility of problematic correctness bugs where code assumes this
field has a valid value for stream messages.  Fix this by changing it
to compose_state.private_message_recipient for clarity.
2019-12-02 08:53:55 -08:00
Mohit Gupta 452e226ea2 narrow: Fix to show last message in narrow when narrow allows.
Fixes commit id 648a60baf6. When
allow_use_first_unread_when_narrowing() is false last message of
narrow is shown in view.

Comments rewritten by tabbott to explain in detail what's happening.
2019-11-22 12:31:43 -08:00
Tim Abbott 263ac0eb45 pm_conversations: Initialize using server data.
This simple change switches us to take advantage of the
server-maintained data for the pm_conversations system we implemented
originally for mobile use.

This should make it a lot more convenient to find historical private
message conversations, since one can effectively scroll infinitely
into the history.

We'll need to do some profiling of the backend after this is deployed
in production; it's possible we'll need to add some database indexes,
denormalization, or other optimizations to avoid making loading the
Zulip app significantly slower.

Fixes #12502.
2019-11-21 17:01:41 -08:00
Tim Abbott 93b83b28a7 pm_conversations: Refactor to sort by message ID.
message_id, rather than timestamps, is our standard way to sort by
time.  And this refactor is important because we're about to start
using data from the server to populate this data structure.
2019-11-21 17:01:41 -08:00
Jack Tiggleman 1682d75ea8 message_edit: Add message edit local echo.
Updates the message editing process to do a local 'echo'.

On slow connections, now there is visual confirmation of the edit,
similar to when sending messages.  The contains_backend_only_syntax
logic and check are the same as there.

We showing "(SAVING)" until the edit is completed, and on successful
edit, the word "(EDITED)" appears.  There's likely useful future work
to do on making the animation experience nicer.

Substantially rewritten by tabbott to better handle corner cases and
communicate more clearly about what's happening.

Fixes: #3530.
2019-11-20 17:40:19 -08:00
Vinit Singh 19234f8705 sidebar: Move the buddy list tooltip content logic to JS.
Moved the logic from static/templates/buddy_list_tooltip_content.hbs to
the get_title_data function to simplify the template.

Fixes #13426.
2019-11-20 17:04:31 -08:00
Tim Abbott 1fe4f795af settings: Add notification settings checkboxes for wildcard mentions.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
2019-11-20 16:58:46 -08:00
Tim Abbott 67efed0b64 node tests: Reformat some notification settings lists.
This just cleans up line-wrapping.
2019-11-20 15:44:44 -08:00
Anders Kaseorg 0a75fdff6d buddy_data: Fix node tests.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 15:16:08 -08:00
Anders Kaseorg f9f104a4f8 js: Automatically convert var to let and const in more files.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, after an `.eslintrc.json` change.

A half dozen files were removed from the changes by tabbott pending
further work to ensure we avoid breaking valuable PRs with merge
conflicts.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 14:10:47 -08:00
Vinit Singh 329d0126bd user status: Add JS tooltips for Buddy List and PM List.
Hovering over user names (and user circles for PM List) now displays
Name, Status Message and Last online time in a js tooltip.
Hovering over group names displays the names of all group members.
Unavailable users are shown as "Last active: Today".

Hovering on a user circle in the Buddy List results in a js tooltip
with Active/Idle/Offline/Unavailable for
green/orange/white/white-with-line.

Resolves #11607.
2019-11-20 12:49:37 -08:00
Dinesh c2e0c492f8 i18n: Fix translation of multi-line strings.
When strings are tagged for translation using `tr this`, the strings
were passed into the frontend i18n as-is (including new line and tab
characters that are not functional in the text, existing just to
format the HTML files reasonably).

This did not match the algorithm used in `manage.py makemessages` for
extracting strings for translation, which (correctly) removed that
whitespace to provide a good experience for translators.  The fix is
for the `tr this` implementation to use that same whitespace-stripping
algorithm.

Tested manually by checking if those strings that were not translated
earlier were translated, and also fixed an automated test that had the
wrong result, which should help prevent regressions.

Fixes #13389.
2019-11-20 10:58:15 -08:00
Anders Kaseorg 428956c086 zjsunit: Remove set_global side effect from zrequire.
ES6 and TS modules don’t insert themselves into `window`, so our tests
shouldn’t insert them either.  Since the test `window` behaves like
`global` now, we can rely on legacy modules that do insert themselves
to do it themselves.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 14:29:17 -08:00
Anders Kaseorg 99563eb150 zjsunit: Make window a Proxy for global.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 14:27:13 -08:00
Anders Kaseorg fffef412bc dependencies: Upgrade to-markdown 3.1.1 to turndown 5.0.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00