Commit Graph

2741 Commits

Author SHA1 Message Date
Steve Howell 7d0082f5c2 topic history: Avoid duplicate server fetches.
We now only go the server if both of these
conditions are true:

    - our message data seems incomplete for
      the stream
    - we haven't already fetched history
2020-01-23 13:28:52 -08:00
Steve Howell e812dd60d6 Extract topic_data.is_complete_for_stream_id().
This function will make more sense when we start
tracking api calls that retrieve topic history.

The unit tests here are kinda duplicating what we
have in the stream_data tests.  If we move the
function out of stream_data, we can kill off the
tests there, but for now I think a bit of duplicate
testing is fine here.
2020-01-23 13:28:52 -08:00
Steve Howell f24de074d4 node tests: Get 100% coverage on topic_list_data. 2020-01-22 14:31:33 -08:00
Steve Howell fa82d12525 topic list: Extract topic_list_data.js.
This is mostly for tactical reasons.  It's hard to
get 100% test coverage on topic_list.js, but it
should be easy to get 100% test coverage on this
very important function.

I considered just moving this code into topic_data.js,
but it just didn't feel quite right.  I feel like
this is a pretty core piece of code that's nice
to be by itself and not be near other complicated
code that does stuff like build widgets or talk
to servers.  (And, again, it's not just the actual
code here, which is pretty small, it's the unit
tests, which are inherently verbose to exercise
all the edge cases.)
2020-01-22 14:31:33 -08:00
Steve Howell 588f34e02f node tests: Remove complicated topic_list test.
This test mostly tests how we glue everything
together, but I want to change that in an upcoming
commit.

Also, the data stuff that it tests is now better
covered by the test recent tests I added.
2020-01-22 14:31:33 -08:00
Steve Howell a7a47fc730 node tests: Test muting-related changes.
This test exercises some code from Tim's recent
patch to exclude muted topics from the topic
list.
2020-01-22 14:31:33 -08:00
Steve Howell 6d4c16c8d7 tests: Add coverage for topic_list.get_list_info(). 2020-01-22 14:31:33 -08:00
Steve Howell 613b8ade20 topic list: Add is_active_topic to template.
The only place we ever set active-sub-filter is
right after we build the template, so there is
no reason to have it be a separate step.

(I made a similar fix to pm_list recently, and
this helps set the stage for doing vdom-like
stuff.)
2020-01-22 14:28:50 -08:00
Anders Kaseorg 4341b7b252 user_groups: Convert members from Dict to Set.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-16 13:23:47 -08:00
Tim Abbott ef1f6b1c33 filter: Allow marking is:mentioned messages as read.
We may revisit this in the future, but similar to is:private, the
current Zulip user experience makes users expect that in the
is:mentioned view, they should really be able to mark messages as
read.

Further, the practice use case for not marking them as read is very
low, since it's rare for someone to have so many mentions that
revisiting the mentions view isn't sufficient to see everything that
needs their attention.
2020-01-16 11:10:36 -08:00
Tim Abbott 409e320d9d filter: Add streams:public to sorted_term_types.
This is for consistency with in:, has:, and similar values where
there's a fixed set of RHS entries.
2020-01-16 11:05:07 -08:00
Rohitt Vashishtha d4ef50634c filter: Remove is_exactly().
Previously, is_exactly() had already been repalced with can_bucket_by().
This commit removes is_exactly() and replaces its usage in our tests
with can_bucket_by().
2020-01-16 10:54:59 -08:00
Rohitt Vashishtha 8a1299d076 filter: Add 'in:*' to sorted_term_types.
This simplifies our handling of in-home and in-all cases in
can_mark_messages_read().
2020-01-16 10:54:59 -08:00
Steve Howell 316eda071d refactor: Clean up can_mark_messages_read.
We now explicitly enumerate various cases, which
should make it easier to change this code.
2020-01-16 10:54:59 -08:00
Rohitt Vashishtha bb579f8823 unreads: Remove is_reading_mode().
This was a part of an experiment we ran on chat.zulip.org in Jul 2018
and surrounding code that used it never got merged to master.

See: https://chat.zulip.org/#narrow/stream/2-general/topic/un-narrow.20view/near/609506
and c407ba5175.
2020-01-16 10:54:59 -08:00
Steve Howell 741da61407 node: Flatten pm_list testing code a bit.
The `messages` part of the data is going away in
the next commits.  This pre-factoring just makes
the next diff a bit more clean.
2020-01-16 10:32:36 -08:00
Tim Abbott 5b30138a31 casper tests: Add an additional waitWhileVisible check.
This fix at least one race and help prevent more race issues in the
casper tests for the organization administration pages.
2020-01-15 17:51:46 -08:00
Tim Abbott d816a12db9 casper tests: Fix buggy submit_notifications_stream_settings.
This new function Casper testing function improperly used
`casper.then` in a nested fashion rather than in series, which doesn't
work how one expects.  This likely caused the test flakes we've
started seeing with this code path since adding
submit_notifications_stream_settings (though it's hard to prove).
2020-01-15 14:11:29 -08:00
Steve Howell aea369f878 Refine user-related typeahead results for large realms.
We now incorporate people.get_message_people() in our
logic for compose/PM typeaheads.  This not only gives
users better results in some cases, but it will also
improve performance for large realms in some cases.
2020-01-15 12:22:23 -08:00
Steve Howell 9ed5545abb Add test coverage for filter_and_sort_mentions.
A recent commit removed test coverage for the
actual filtering/sorting of mention typeaheads
when you did a non-silent method.  This commit
now tests that important step again.

Note that we also had (and still have) tests
that make sure the is_silent flag is set
correctly by get_candidates.

We don't have a true full-stack test, but those
can be quite tricky to set up and maintain.
2020-01-15 12:22:23 -08:00
Steve Howell e1213ca30a minor: Add person/group to composebox_typeahead tests. 2020-01-15 12:22:23 -08:00
Steve Howell 0aa9decd86 blueslip: Add feature to time common operations.
This is relatively unobtrusive, and we don't send
anything to the server.

But any user can now enter blueslip.timings in the
console to see a map of how long things take in
milliseconds.  We only record one timing per
event label (i.e. the most recent).

It's pretty easy to test this by just clicking
around.  For 300 users/streams most things are
fast except for:

    - initialize_everything
    - manage streams (render_subscriptions)

Both do lots of nontrivial work, although
"manage streams" is a bit surprising, since
we're only measuring how long to build the
HTML from the templates (whereas the real
time is probably browser rendering costs).
2020-01-15 12:01:16 -08:00
Steve Howell 890a4b1247 refactor: Add filterer for user settings.
This change sets us up to optimize how we
filter users in the admin user settings.
See #13554 for more context on the user
facing issues.

This fix is basically three related things:

    - Add filterer options to list_render.
    - Add helper method to people.js.
    - Use filterer in settings_users.js.
2020-01-14 22:43:08 -08:00
Steve Howell 110c15737f Rename filter.callback to filter.predicate.
The filter "callback" was only a "callback" in the
most general sense of the word.

It's just a filter predicate that returns a bool.

This is to prepare for another filtering option,
where the caller can filter the whole list
themselves.  I haven't figured out what I will name
the new option yet, but I know I want to make the
two options have specific names.
2020-01-14 22:43:08 -08:00
Steve Howell 3f3b9c3b70 list_render: Make callbacks required.
We are already providing callbacks everywhere, so
it would be nice to eliminate some dead code.

This also speeds things up ever so slightly (no
longer type-checking the option every time through
the loop).

We also split out exports.filter to make unit testing
easier.  The function seems kinda silly now, being so
small, but I hope to add another filtering option soon.
2020-01-14 22:43:08 -08:00
Anders Kaseorg ea6934c26d dependencies: Remove WebSockets system for sending messages.
Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013.  We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not.  Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.

While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.

This WebSockets code path had a lot of downsides/complexity,
including:

* The messy hack involving constructing an emulated request object to
  hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
  needs and must be provisioned independently from the rest of the
  server).
* A duplicate check_send_receive_time Nagios test specific to
  WebSockets.
* The requirement for users to have their firewalls/NATs allow
  WebSocket connections, and a setting to disable them for networks
  where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
  been poorly maintained, and periodically throws random JavaScript
  exceptions in our production environments without a deep enough
  traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
  server restart, and especially for large installations like
  zulipchat.com, resulting in extra delay before messages can be sent
  again.

As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients.  We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.

If we later want WebSockets, we’ll likely want to just use Django
Channels.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-14 22:34:00 -08:00
Pragati Agrawal 6fc2a317e9 org settings: Use save/discard widget for notification stream settings.
Currently, if we change stream we see the immediate saving of stream, but
it is more convenient to have "Save" and "Discard" buttons as we use
everywhere else in the organization setting subsystem.
2020-01-14 17:16:23 -08:00
Pragati Agrawal 48b6734b73 org settings: Minor refactor to add notification stream id data in widget.
As the part of making notification stream settings to change using
"save/discard" widget instead of immediate saving, we need to access the
stream id which is being selected at the moment.
2020-01-14 17:16:23 -08:00
Pragati Agrawal bde8838d7e org settings: Use id instead of class for a specific element.
This is a preliminary commit to do some deduplication with notification
stream dropdown widget.
2020-01-14 17:16:23 -08:00
Pragati Agrawal e7c40f69de org settings: Remove old method of success status for notification streams.
For "New stream notifications" and "New user notifications" it is more
intuitive to just use the new system for showing success/saving status
feedback.
2020-01-14 17:16:23 -08:00
Steve Howell 29e63c0417 Fix type errors in LazySet.
I think the only place that was broken is where
we copy users from streams.
2020-01-14 15:40:40 -08:00
Steve Howell c2af2c1fd1 refactor: Extract is_subscriber_subset().
Extracting the function makes it a bit easier to
test and use in a generic way.

Also, I wanted this to live in stream_data, so that
it's easier to find if we change how we model
subscriber data.

Finally, I use _.every to do the subset check
instead of `_.difference`, since _.difference
is actually N-squared:

  _.difference = restArguments(function(array, rest) {
    rest = flatten(rest, true, true);
    return _.filter(array, function(value){
      return !_.contains(rest, value);
    });
  });

And we don't actually want to build a list only
to check that it's zero vs. nonzero length.

We now do this, which short circuits as soon
as it finds any key that is only in sub1:

    return _.every(sub1.subscribers.keys(), (key) => {
        return sub2_set.has(key);
    });
2020-01-14 13:19:49 -08:00
Steve Howell 34b21bc0ee refactor: Use is_broadcast flag for mention check.
I also clean up the noop tests here, which were
actually redundant (all three cases were short
circuiting on the "everyone" mention).
2020-01-14 13:19:49 -08:00
Steve Howell e638361728 minor: Move unit test to module scope.
This test is really no longer an "event" test.
2020-01-14 13:19:49 -08:00
Steve Howell 593049d551 compose: Extract warn_if_mentioning_unsubscribed_user.
First, there are no more convoluted signals.

We also simplify the parameter to just the "mentioned"
object corresponding to either a user or a broadcast
mention.

For the user group scenario, this has always been dead
code, which you only realized when you got to the comment
at the bottom.  Now we actually do nothing.
And I moved the relevant commment to the
the typeahead code (with new wording).

I also moved the is_silent check to the caller.  I don't
feel too strongly about that either way. It's kind of silly
to call a function only to give that function an additional
responsibility to worry about.  On the other hand, I see
the logic of that function enforcing everything.  I went
with the former for now.

Arguably we should have a warning for silent mentions,
since doing a silent mention of somebody not on a stream
is a good indication of a typo.  I do understand the use
case, but the user can always ignore the warning.  Anyway,
we have decent test coverage on this.
2020-01-14 13:19:45 -08:00
Steve Howell b91a19df43 refactor: Extract warn_if_private_stream_is_linked.
This isn't really an extraction; it's more giving
a name to an anonymous function and moving it to
higher module scope.

We convert this to an ordinary function call, which
allows us to move it out of intialize().

Since there's just one simple parameter now (linked_stream),
we can avoid some error checking.

We also avoid the comment that describes the function,
since it now has a name.

And then one minor tweak is to do the inexpensive
`invite_only` higher in the function.  This will be
a nice speedup when you link to really large public
streams.

The unit tests are also a bit easier to read now--less
setup and more explicit names.
2020-01-14 13:13:48 -08:00
Steve Howell 9f590889b7 refactor: Use a Set for away_user_ids. 2020-01-14 17:52:25 +00:00
Tlazypanda 30ee0c2a49 invitations: Improve experience around reactivating users.
Previously, if you tried to invite a user whose account had been
deactivated, we didn't provide a clear path forward for reactivating
the users, which was confusing.

We fix this by plumbing through to the frontend the information that
there is an existing user account with that email address in this
organization, but that it's deactivated.  For administrators, we
provide a link for how to reactivate the user.

Fixes #8144.
2020-01-13 18:30:51 -08:00
Steve Howell 15e7f5828b performance: Improve sort_recipients.
The sort_recipients helper is used for many different
typeaheads, such as compose PMs, compose mentions,
and some settings-related code.

We now avoid unnecessary sorting steps in cases
where we have plenty of results in the top buckets
(such as users who match on prefix).

This change should not have any user-facing
implications.
2020-01-12 20:53:03 -08:00
Steve Howell 9830d0a4c7 typeahead: Remove get_person_or_user_group_matcher.
We no longer need this, because we now filter
persons and groups separately.
2020-01-12 20:52:59 -08:00
Steve Howell bbb8abf4c5 typeahead: Extract composebox_typeahead.get_person_suggestions.
This method is a bit complex, but I think it's
worthwhile to force PM autocompletes and mention
autocompletes through the same code path.

We also kill off this method:

    typeahead_helper.sort_people_and_user_groups
2020-01-12 20:52:57 -08:00
Steve Howell 8fb9820b40 typeahead: Extract filter_and_sort_mentions.
We want a bit more control over the logic here.

The end game here is to converge a bit with
the code for autocompleting PM recipients.
2020-01-12 20:52:53 -08:00
Steve Howell 95daa50954 pm_list: Simplify is_all_privates logic.
For historical reasons pm_list was handling just
one possible edge case of where is:private was
combined with other search terms, namely the
pm-with operator.

The code was correct in realizing the is:private
was redundant there, but now we handle that
upstream in Filter.fix_operators (see previous
commit).

Now we just look for any is:private term.
2020-01-09 17:46:03 -08:00
Steve Howell 1af41dd954 filter: Remove redundant is:private operators.
If we have a pm-with, then is:private is redundant
and just forces us to write confusing/verbose code
in various places.
2020-01-09 17:46:03 -08:00
Steve Howell 4a03f91daa search: Retrofit recent changes to pills code.
This change makes these two functions more alike:

    - get_search_result
    - get_search_result_legacy

To test the UI modify zerver/views/home.py by
replacing `settings.SEARCH_PILLS_ENABLED` with
`True`.  I only did a quick sanity check, since
any bugs with the new system are more likely due
to bitrot than any changes I have made here.

The history is this:

    Tim cloned the code (before the smaller
    helpers were extracted):

        db4f6e278f

    In 8b153f6452
    Shubham removed get_operator_subset_suggestions but
    accidentally left a `concat` statement in that got
    misapplied to the previous suggestions:

    -    suggestions = get_operator_subset_suggestions(operators);
         result = result.concat(suggestions);

    The error there was carried over in some recent changes,
    but this commit fixes that strangeness.

    In 73e4f3b3fa
    Shubham made this change, which makes sense only for
    pills, and this code remains intact.

        -    if (operators.length > 0) {
        -        last = operators.slice(-1)[0];
        +    if (query_operators.length > 0) {
        +        last = query_operators.slice(-1)[0];
        +    } else {
        +        // If query_operators = [] then last will remain
        +        // {operator: '', operand: '', negated: false}; from above.
        +        // `last` has not yet been added to operators/query_operators.
        +        // The code below adds last to operators/query_operators
        +        operators.push(last);
        +        query_operators.push(last);
             }

    Mohit made a couple changes to both old and new.

    Anders made a couple non-substantive changes related to
    the ES6 migration.

    Steve (me) made several structural changes to the code.  For
    some of them I only changed the legacy code, not the pills
    code.  I didn't fix Shubham's mistake until this change.

Now the two functions should look similar except in the places
where they are intentionally different.  I also added a comment
explaining the get_operator_subset_suggestions difference.

Fixes #13609
2020-01-08 14:02:34 -08:00
Steve Howell f47f27d110 refactor: Use filter_taken_users in compose typeahead.
The composebox_typeahead code now gets people directly,
which will allow us to do some optimizations upcoming.
2020-01-08 12:55:42 -08:00
Steve Howell 94761b806c node tests: Restore 100% coverage to pm_list. 2020-01-06 10:21:23 -08:00
Steve Howell c22c796f1d refactor: Extract is_all_privates().
I want to be able to easily test this without
having to simulate all the jQuery side effects.

This simply preserves the old logic, which seems
to handle one edge case without handling every
possible edge case.  The edge cases aren't super
important here, though, since the only thing it affects
is bolding "Private Messages", and when to do that
is somewhat up to personal tastes.

Having said that, we could definitely improve
this code and possibly should move some of this
logic to either narrow_state.js or filter.js.
2020-01-06 10:21:23 -08:00
Steve Howell 5b168d0530 pm_list: Set active-sub-filter in template.
Instead of doing various ad-hoc calculations of
which PM is "active" and plumbing it through various
functions and then updating it via jQuery instead of
just the template, we now just calculate `is_active`
in `_build_private_messages_list` with a little
helper function.
2020-01-06 10:21:23 -08:00
Steve Howell da1392efd2 node test: Remove complicated pm_list test.
This test mostly tests logic that I'm about
to remove in subsequent commits, and it's a bit
messy.

This commit removes 100% line coverage, but I
will restore that a few commits later.
2020-01-06 10:21:23 -08:00
Steve Howell 066a02a987 pm_list: Remove obsolete active_conversation parameter.
In 3cfc3ca24b I removed
the feature that limited PM conversations to five or
less (including the active conversation), but I
didn't clean up this parameter.  I think lint was
confused by the fact that we did mutate it.

I am wondering if this started out as an experiment
and was never fully polished before the push?  Or
maybe I was just careless.  Anyway, I don't
think were any symptoms here--it was just dead code
that we didn't need.
2020-01-06 10:21:23 -08:00
Tim Abbott 9f72e5fc87 int_dict: Move filter_values helper to dict.
This fixes a rebase issue between the int_dict introduction and use
for people.js with the introduce of filter_values on dict.js and use
inside people.js.
2020-01-05 13:18:34 -08:00
Steve Howell 493afcb9f0 zjsquery: Add data support.
Before this we just noop'ed it, since at one time
we were trying to deprecate this is in favor
of attr calls.
2020-01-05 12:28:37 -08:00
Steve Howell 9ba1829243 streams: Use IntDict for stream/topic unread counts.
Note that we haven't fully swept this for Dict,
since some dicts are keyed by strings.  For
example PM counts can have a huddle like
"101,102,103" as a key.
2020-01-05 12:28:34 -08:00
Steve Howell 9f7be51ce8 streams: Replace Dict with IntDict in stream_data.
There's another Dict that we'll convert to a Set
in a subsequent commit.
2020-01-05 12:28:28 -08:00
Steve Howell 73d0350a24 people: Use ints in is_my_user_id().
This should be slightly more performant, and we
often call this function N times, such as when
rendering the buddy list.

There's a minor change to pm_list to avoid
an unnecessary computation on huddles that would
otherwise trigger a blueslip warning for the
huddles case.
2020-01-05 12:28:23 -08:00
Steve Howell bc5589c2a7 people: Clean up recip.id code.
When we are pulling data from message.display_recipient
for private messages, the user_id field is always
called 'id', not 'user_id', so we can simplify
some defensive code.
2020-01-05 12:27:30 -08:00
Steve Howell 7630b859c3 js: Use IntDict in people.js.
This required lots of manual testing:

    - search/navigate user presence
    - send PM and mention user
    - pay attention to compose fade
    - send stream msg and mention user
    - open Private Messages in top-left and click
    - test unread counts
    - invite user who already has account
    - search for users in search bar
    - check user settings
        - User Groups
        - Users
        - Deactivated Users
        - Bots
    - create a bot
    - mention user groups
    - send group PM then click on lower right
    - view/edit/create streams

If there are still pieces of code that don't convert
ids to ints, the code should still work but report
blueslip errors.

I try to mostly convert user_ids to ints in the callers,
since often the callers are dealing with small amounts
of data, like user ids from huddles.
2020-01-05 12:27:28 -08:00
Steve Howell 4e59937632 js: Add IntDict class.
We don't use this yet, but we will soon.

We report errors if users pass in strings instead of
ints, but we try to still use the key.
2020-01-05 12:27:26 -08:00
Steve Howell 26168eaa98 search: Optimize search bar suggestions for large realms.
We only ever show 3 or 4 people in search suggestions
(possibly w/a couple variations, like pm-with/sender/etc.),
so we can try to search a smaller subset of people
before going through the entire realm.

We use message_store.user_ids() for this, since you
typically want to search messages for people that
have sent messages recently, and we already sort
based on PM conversations.
2020-01-04 12:58:00 -08:00
Steve Howell 7016292558 search: Track user_ids in message_store.
We'll use this for search.
2020-01-04 12:57:58 -08:00
Steve Howell d87c5d7b1f search: Use people.filter_all_persons() in search.
This should avoid some memory allocations.

We also use build_person_matcher to avoid
repeating the same logic over and over
again to process the query into termlets.

We also remove people.get_all_persons() and
people.person_matches_query().
2020-01-04 12:53:32 -08:00
Steve Howell 7d7028b7d0 performance: Speed up PM lookaheads.
This looks like simple code cleanup, but it's more
than that.

The code cleanup here is that we don't have three
callbacks to get a list of typeaheads for bootstrap.
Instead, we just have one function that does all the
main work.

And then the speedup comes from the fact we no longer
need to remove diacritics from the query for every
time through our loop of seeing if a person matches
the query.

It's a bit subtle to see in the diff, but these are
the relevant lines:

    const matcher = exports.get_person_or_user_group_matcher(query);
    const filtered_results = _.filter(people_and_groups, matcher);

Before this, bootstrap was doing $.grep, and we'd have
to reinitialize the matcher for every person.

If you profile this before and after, you'll see that
remove_diacritics gets called fewer times.

To profile this, you want to loads lots of users into
your DB and try to autocomplete "Extra", as in "Extra1 User".

If you try to autocomplete something else, then my patch
won't really help, and `remove_diacritics` will still
show up as expensive.  Because it is that expensive a function.
2020-01-03 17:42:29 -08:00
Steve Howell ccf6640660 refactor: Have compose_content_matcher return a function.
This may seem silly now, since we are returning a function
that still dispatches over all flavors of search for
every item, but subsequent commits will make it obvious
why I'm doing this.
2020-01-03 17:39:50 -08:00
Steve Howell b65da7cbe9 compose typeahead: Do matching/sorting without callbacks.
We want to do our own matching of items, rather than
just giving a callback to bootstrap, which does $.grep
on all the items.

Doing our own matching gives us flexibility for future
improvements like custom data structures for searching
through big amounts of data.  Even in the short term
we can speed up searches by pulling expensive operations
outside the grep/filter call.

This architecture has been in place for our search
bar since ~2014.
2020-01-03 17:39:48 -08:00
Steve Howell 9afad9e054 node tests: Add commented-out benchmarks for Dict.
The benchmark is commented out.  It takes only a few
milliseconds to run, so there may be no reason not
to always run it.  It doesn't test correctness, so
it would arguably inflate line coverage, but set/get
are obviously covered elsewhere.
2020-01-03 17:19:59 -08:00
Steve Howell 30ad1b6f16 zjsunit: Remove Dict dependency.
We now require the actual tests to explicitly
to zrequire Dict, rather than magically adding this.

In one case, the use of Dict was clearly just for
the test (not the app), so I converted that an ordinary
JS object (see timerender.js).
2020-01-03 17:19:59 -08:00
Steve Howell d41f714eff comments: Update comment for zjsunit/i18n.js. 2020-01-03 17:19:59 -08:00
Steve Howell 897320b2c4 zjquery: Use Map instead of Dict.
This seems to speed up the whole test suite
by about 20%, although measurements are a bit
noisy.
2020-01-03 17:19:59 -08:00
Steve Howell ee3e488e02 js: Extract FoldDict class.
We have ~5 years of proof that we'll probably never
extend Dict with more options.

Breaking the classes into makes both a little faster
(no options to check), and we remove some options
in FoldDict that are never used (from/from_array).

A possible next step is to fine-tune the Dict to use
Map internally.

Note that the TypeScript types for FoldDict are now
more specific (requiring string keys).  Of course,
this isn't really enforced until we convert other
modules to TS.
2020-01-03 17:19:50 -08:00
Steve Howell b3a69154a6 refactor: Export compare_for_relevance.
This future-proofs us a bit more for test coverage.
2020-01-03 14:58:05 -08:00
Steve Howell 0985842c62 Fix sorting for broadcast mentions.
We had a potentially nasty bug where we
weren't guaranteeing that all/stream/everyone
collated in consistent ways inside of
`compare_people_for_relevance`, which can
send certain types of sort algorithms into
an infinite loop. I doubt this ever happened
in practice, but it's obviously worth fixing.

Now we also have a clear tiebreaker between
any two all/everyone/stream mentions, which
is the idx field.

Finally, this should be a bit more efficient.
2020-01-03 14:58:05 -08:00
Steve Howell 758786ab87 refactor: Extract broadcast_mentions.
This will be helpful for testing.
2020-01-03 14:58:05 -08:00
Steve Howell 773161cbb7 tests: Test "all" mentions more realistically.
We don't have people named "all".  Instead, we
create pseudo person objects with email/full_name
of "all" (along with some other fields).  The tests
now reflect this.
2020-01-03 14:58:05 -08:00
Steve Howell d227988519 tests: Split up sort_recipient tests. 2020-01-03 14:58:05 -08:00
Steve Howell cde01aeeb0 tests: Avoid list mutation.
To test dups we can just create a new list.
2020-01-03 14:58:05 -08:00
Steve Howell 5c43180a70 tests: Use names for test objects. 2020-01-03 14:58:05 -08:00
Steve Howell b5d0eab0c6 dict: Add filter_values() method.
This method can help us avoid some memory
allocations.
2020-01-02 12:03:45 -08:00
Steve Howell 7229a943f0 tests: Use add_in_realm for "me" in people tests.
This is more realistic for testing.
2020-01-02 12:03:04 -08:00
Steve Howell 54cb857fee refactor: Rename people.get_rest_of_realm().
We want to mostly deprecate this function (see
the comment I added), so I gave it a more specific
name.

Ideally I'd just fix `stream_create`, but it does
use this function in a couple places, and it's helpful
to reuse the same sort here.  In one place stream_create
actually unshifts the "me" user back to the top of the
list, which makes sense for its use case.
2020-01-02 12:03:04 -08:00
Steve Howell 6e93f330c6 bug fix: Fix huddles in "Private Messages".
If two user_ids in a recent huddle have ids
that sort lexically differently than numerically,
such as 7 and 66, then we were creating two
different buckets in pm_conversations.

This regression was introduced in
263ac0eb45 on
November 21, 2019.
2020-01-02 11:59:58 -08:00
Steve Howell 0e68387975 refactor: Have pm_conversations take user_ids.
Instead of having our callers pass in a possibly
non-canonical version of a user_ids_string, just
have them pass in a list.

The next commit will canonicalize the sort.
2020-01-02 11:59:58 -08:00
Steve Howell ab6f4af33a tests: Use tricky server data in unit tests.
The server may send us ids in the order
[11, 2], instead of [2, 11].  We don't want
to rely on server behavior, regardless, for
the sort.

Our tests now show we process that data.

The current code is is still buggy and causes
us to show the same huddle two different times
for situations where the lexical sort doesn't
match the numerical sort.

This happens on czo often, where Tim is user
7, and his id sorts lexically after ids like
58, 622, 4444, etc.
2020-01-02 11:59:58 -08:00
Steve Howell 0711c7ea49 performance: Avoid dup calls to subscribed_streams().
In stream_sort.sort_groups, we now have the caller
pass us in the list of streams, since they are getting
them anyway.
2019-12-30 09:50:22 -08:00
Steve Howell 631811e686 streams: Add BinaryDict for stream_data.
This should make any operation on subscribed
streams faster (we won't need to filter out
unsubscribed streams every time).

I started writing this before I realized we
had a bug where we call `subscribed_streams`
in a nested loop.

After fixing the bugs, this is not as much of
a bottleneck, but it's still a speedup in many
important places:

    * build left sidebar
    * every keystroke in search bar
    * first keystroke in making #stream_links
    * every keystroke in compose stream box

The streams settings code is kinda complicated.
It does a non-deterministic sort of the "others"
bucket when you add elements to the left panel.
They get hidden, anyway.  Our values() call now
puts subscribed streams first.  It never guaranteed
order, but putting subscribed streams first is
probably a good behavior for most situations.
2019-12-30 09:50:20 -08:00
Steve Howell a3512553a8 streams: Add LazySet for subscribers.
This defers O(N*S) operations, where

    N = number of streams
    S = number of subscribers per stream

In many cases we never do an O(N) operation on
a stream.  Exceptions include:

    - checking stream links from the compose box
    - editing a stream
    - adding members to a newly added stream

An operation that used to be O(N)--computing
the number of subscribers--is now O(1), and we
don't even pay O(N) on a one-time basis to
compute it (not counting the cost to build the
array from JSON, but we have to do that).
2019-12-30 09:47:55 -08:00
Steve Howell e804f39f0e performance: Avoid expensive call in stream_data.is_active.
Calling `set_filter_out_inactives` is expensive, since we
count up the number of subscribed streams, which iterates
through all your streams, creates a new list of subscribed
streams, then counts them.

In my dev setup, I created 700 streams, and this shaved
about 700ms off of the initial call to `build_stream_list`.
2019-12-30 09:45:46 -08:00
Steve Howell 70470dea1c settings: Use correct email when searching users.
If we aren't showing users emails, then we don't
want to use emails in the search.

And if we are showing users emails, we want to
search on the email that's displayed to them.
For admins this will be delivery_email.

For regular users we arguably shouldn't search
on emails either, since it mostly causes confusion,
but this commit just preserves the current
behavior for those users (unless `show_email` is
false).
2019-12-30 09:43:24 -08:00
Steve Howell 3e4326afda refactor: Extract email_for_user_settings.
We want to be able to unit test this value,
since it's conditional on several factors:

    - am I an admin?
    - can non-admins view emails?
    - do we have delivery_email for the user?

I'm mocking show_email in the tests, since the
show_email code is in `settings_org` and
kind of hard to unit test.  It's not impossible,
but it's too much for this commit.  (Either
we need to extract it out to a nice file or
deal with mocking jQuery.  That module is
mostly data-oriented, so it would be nice
to have something like `settings_config` that
is actually pure data.)
2019-12-28 11:22:24 -08:00
Steve Howell 3a95be2f2f refactor: Extract matches_user_settings_search.
This was duplicate code.  I'm moving it to people
for pragmatic reasons--it's hard to unit test stuff
in settings_users.js due to all the jQuery.

It's also nice to have all people-related search
code in one place, just for auditing purposes.
2019-12-28 11:22:24 -08:00
Steve Howell ab34ee0800 search performance: Stop at max_items.
Once we have max_items results, stop trying
to get more items.

This should really help large realms when
you do a search on streams that turns up
more than N streams (where N is about 12).
We won't even bother to find people.
2019-12-28 11:09:28 -08:00
Steve Howell 4141abc171 search: Slightly speed up stream highlighting.
This isn't a huge speedup, but it's an easy
code change.

We remove the two-liner highlight_with_escaping,
which was only called in one place, and when
we inline it into the caller, we can pull the
first line, which builds the regex, out of the
loop.
2019-12-28 11:09:23 -08:00
Steve Howell 49cd719273 tests: Verify subscriptions more thoroughly.
When you subscribe/unsubscribe yourself, you want
to make sure these calls work too:

    subscribed_subs
    unsubscribed_subs
2019-12-28 07:29:51 -05:00
Steve Howell 32a1ef20d1 minor: Extract helper for search tests. 2019-12-28 05:43:04 -05:00
Steve Howell f3ebb5fbee test cleanup: Extract a sorter() helper. 2019-12-28 05:43:04 -05:00
Steve Howell 9ac2fe2826 test cleanup: Extract a matcher() helper. 2019-12-28 05:43:04 -05:00
Tim Abbott e72da08f09 narrow: Fix streams:all notice appearing too early.
The streams:all adveritsement notice in search should only appear
after we've already received the response from the server, to avoid a
mix of problems ranging from misplaced loading indicator to scrolling
issues to the notice just being distracting while you're waiting for
the server to return results.

We need to add a pre_scroll_cont parameter to the message_fetch API,
since adding this notice would otherwise potentially throw off the
scroll positioning logic for which message to select.

Fixes #13441.
2019-12-10 18:10:39 -08:00
Mohit Gupta a0c11b6c78 narrow: Use search reading behavior in all searches.
In 452e226ea2 and
648a60baf6, we changed how `search:`
narrows work to:

(1) Never mark messages as read inside searches (search:)
(2) Take you to the bottom, not the first unread, if a `near:` or
    similar wasn't specified.

This is far better behavior for these use cases, because in these
narrows, you can't actually see all the context around the target
messages, so marking them as read is counterproductive.  This is
especially important in `has:mention` where you goal is likely
specifically to keep track of which threads mentioning you haven't
been read.  But in many other narrows, the current behavior is
effectively (1) setting the read bit on random messages and (2) if the
search term matches many messages in a muted stream with 1000s of
unreads, making it hard or impossible to find recent search matches.

The new behavior is that any narrow that is structurally a search of
history (including everything that that isn't a stream, topic,
pm-with, "all messages" or "private messages") gets that new behavior
of being unable to mark messages as read and narrows taking you to the
latest matching messages.

A few corner cases of interest:
* `is:private` is keeping the old behavior, because users on
  chat.zulip.org found it confusing for `is:private` to not mark
  messages as read when one could see them all.  Possibly a more
  complex answer is required here.

* `near:` narrows are getting the new behavior, even if it's a stream:
  + topic: narrow.  This is debatable, but is probably better than
  what was happening before.

Modified significantly by tabbott for cleanliness of implementation,
this commit message, and unit tests.

Fixes #9893.  Follow-up to #12556.
2019-12-10 16:26:06 -08:00
Tim Abbott 2eae0b3e57 notifications: Support wildcard_mentions_notify for desktop.
In 1fe4f795af, we added the
wildcard_mentions_notify setting, which controls whether wildcard
mentions should be treated as mentions for the purposes of
notifications.  The original implementation focused on the more
important area of email/push notifications, and neglected to address
desktop notifications for wildcard mentions.

This change makes the wildcard_mentions_notify flag behave correctly
for desktop/sound notifications, including unit tests.

Fixes #13073.
2019-12-10 13:12:36 -08:00
Tim Abbott 22cefeede8 notifications: Extract should_send_*_notification for testing. 2019-12-10 12:54:36 -08:00
Tim Abbott 016487163f node tests: Refactor notifications tests for better reuse. 2019-12-10 12:47:21 -08:00
Tim Abbott fa7ae6fa7f node tests: Fix missing coverage on stream_data.js.
This fixes a testing coverage regression in
d5f005fd61.
2019-12-09 18:11:12 -08:00
Nat1405 d5f005fd61 wildcard_mentions_notify: Add per-stream override of global setting.
Adds required API and front-end changes to modify and read the
wildcard_mentions_notify field in the Subscription model.

It includes front-end code to add the setting to the user's "manage
streams" page. This setting will be greyed out when a stream is muted.
The PR also includes back-end code to add the setting the initial state of
a subscription.

New automated tests were added for the API, events system and front-end.
In manual testing, we checked that modifying the setting in the front end
persisted the change in the Subscription model. We noticed the notifications
were not behaving exactly as expected in manual testing; see
https://github.com/zulip/zulip/issues/13073#issuecomment-560263081 .

Tweaked by tabbott to fix real-time synchronization issues.

Fixes: #13429.
2019-12-09 16:09:38 -08:00
Rohitt Vashishtha 85c669e366 markdown: Remove redundant checks from /me.
If a message begins with /me, we do not have any cases where the
rendered content would not begin with `<p>/me`. Thus, we can safely
remove the redundant checks both on the backend and frontend.
2019-12-03 17:17:10 -08:00
Tim Abbott 8b55a310f1 typing: Fix invalid typing notifications for stream messages.
In e42c3f7418, we made the assumption
that compose_pm_pill.get_recipient() would return no users for stream
messages.  It turns out, due to the confusing name of
compose_state.recipient (which we just renamed to
compose_state.private_message_recipient), this assumption was wrong.

As a result, when composing a stream message using the reply hotkeys,
we'd end up sending typing notiifcations to the person who sent the
message we're replying to as though a PM was being composed.

We fix this by avoiding passing an (expected to be unused) value for
private_message_recipient to compose_state.start.
2019-12-02 09:31:16 -08:00
Tim Abbott ea7c6d395f compose_state: Rename compost_state.recipient to be about PMs only.
The compose_state.recipient field was only actually the recipient for
the message if it was a private_message_recipient (in the sense of
other code); we store the stream in compose_state.stream instead.

As a result, the name was quite confusing, resulting in the
possibility of problematic correctness bugs where code assumes this
field has a valid value for stream messages.  Fix this by changing it
to compose_state.private_message_recipient for clarity.
2019-12-02 08:53:55 -08:00
Mohit Gupta 452e226ea2 narrow: Fix to show last message in narrow when narrow allows.
Fixes commit id 648a60baf6. When
allow_use_first_unread_when_narrowing() is false last message of
narrow is shown in view.

Comments rewritten by tabbott to explain in detail what's happening.
2019-11-22 12:31:43 -08:00
Tim Abbott 263ac0eb45 pm_conversations: Initialize using server data.
This simple change switches us to take advantage of the
server-maintained data for the pm_conversations system we implemented
originally for mobile use.

This should make it a lot more convenient to find historical private
message conversations, since one can effectively scroll infinitely
into the history.

We'll need to do some profiling of the backend after this is deployed
in production; it's possible we'll need to add some database indexes,
denormalization, or other optimizations to avoid making loading the
Zulip app significantly slower.

Fixes #12502.
2019-11-21 17:01:41 -08:00
Tim Abbott 93b83b28a7 pm_conversations: Refactor to sort by message ID.
message_id, rather than timestamps, is our standard way to sort by
time.  And this refactor is important because we're about to start
using data from the server to populate this data structure.
2019-11-21 17:01:41 -08:00
Jack Tiggleman 1682d75ea8 message_edit: Add message edit local echo.
Updates the message editing process to do a local 'echo'.

On slow connections, now there is visual confirmation of the edit,
similar to when sending messages.  The contains_backend_only_syntax
logic and check are the same as there.

We showing "(SAVING)" until the edit is completed, and on successful
edit, the word "(EDITED)" appears.  There's likely useful future work
to do on making the animation experience nicer.

Substantially rewritten by tabbott to better handle corner cases and
communicate more clearly about what's happening.

Fixes: #3530.
2019-11-20 17:40:19 -08:00
Vinit Singh 19234f8705 sidebar: Move the buddy list tooltip content logic to JS.
Moved the logic from static/templates/buddy_list_tooltip_content.hbs to
the get_title_data function to simplify the template.

Fixes #13426.
2019-11-20 17:04:31 -08:00
Tim Abbott 1fe4f795af settings: Add notification settings checkboxes for wildcard mentions.
This change makes it possible for users to control the notification
settings for wildcard mentions as a separate control from PMs and
direct @-mentions.
2019-11-20 16:58:46 -08:00
Tim Abbott 67efed0b64 node tests: Reformat some notification settings lists.
This just cleans up line-wrapping.
2019-11-20 15:44:44 -08:00
Anders Kaseorg 0a75fdff6d buddy_data: Fix node tests.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 15:16:08 -08:00
Anders Kaseorg f9f104a4f8 js: Automatically convert var to let and const in more files.
This commit was automatically generated by `tools/lint --only=eslint
--fix`, after an `.eslintrc.json` change.

A half dozen files were removed from the changes by tabbott pending
further work to ensure we avoid breaking valuable PRs with merge
conflicts.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-20 14:10:47 -08:00
Vinit Singh 329d0126bd user status: Add JS tooltips for Buddy List and PM List.
Hovering over user names (and user circles for PM List) now displays
Name, Status Message and Last online time in a js tooltip.
Hovering over group names displays the names of all group members.
Unavailable users are shown as "Last active: Today".

Hovering on a user circle in the Buddy List results in a js tooltip
with Active/Idle/Offline/Unavailable for
green/orange/white/white-with-line.

Resolves #11607.
2019-11-20 12:49:37 -08:00
Dinesh c2e0c492f8 i18n: Fix translation of multi-line strings.
When strings are tagged for translation using `tr this`, the strings
were passed into the frontend i18n as-is (including new line and tab
characters that are not functional in the text, existing just to
format the HTML files reasonably).

This did not match the algorithm used in `manage.py makemessages` for
extracting strings for translation, which (correctly) removed that
whitespace to provide a good experience for translators.  The fix is
for the `tr this` implementation to use that same whitespace-stripping
algorithm.

Tested manually by checking if those strings that were not translated
earlier were translated, and also fixed an automated test that had the
wrong result, which should help prevent regressions.

Fixes #13389.
2019-11-20 10:58:15 -08:00
Anders Kaseorg 428956c086 zjsunit: Remove set_global side effect from zrequire.
ES6 and TS modules don’t insert themselves into `window`, so our tests
shouldn’t insert them either.  Since the test `window` behaves like
`global` now, we can rely on legacy modules that do insert themselves
to do it themselves.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 14:29:17 -08:00
Anders Kaseorg 99563eb150 zjsunit: Make window a Proxy for global.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 14:27:13 -08:00
Anders Kaseorg fffef412bc dependencies: Upgrade to-markdown 3.1.1 to turndown 5.0.3.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Sophie 2eba3e7827 org_settings: Change new user 24-hour setting to dropdown.
These should work consistently with how the individual user setting
works; see the last commit.

With changes from tabbott to fix real-time sync.

Fixes #12553.
2019-11-08 17:39:59 -08:00
Matheus Melo c96762b7a9 settings: Add setting for who can edit user groups.
Fixes #12380.
2019-11-03 16:45:13 -08:00
Anders Kaseorg 28f3dfa284 js: Automatically convert var to let and const in most files.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`.  It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-03 12:42:39 -08:00
Thomas Ip c93522d847 blueslip: Make stack trace more readable.
The stack trace popup is now sourcemapped and each stackframe have a
expandable code context window.

[anders@zulipchat.com: Rebased and simplified.]
2019-10-31 13:47:54 -07:00
Anders Kaseorg f077508c6d test-backend, run-casper: Remove proxy vars instead of setting to "".
Setting http_proxy to the empty string caused a spurious error from
xmlsec:

>>> import os
>>> os.environ["http_proxy"] = ""
>>> import xmlsec
error : Unknown IO error

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:47:45 -07:00
Anders Kaseorg 02004c9b0f js: Convert self-referential vars to const.
ESLint won’t convert these automatically because it can’t rule out a
behavior difference arising from an access to a self-referential var
before it’s initialized:

> var x = (f => f())(() => x);
undefined
> let y = (f => f())(() => y);
Thrown:
ReferenceError: Cannot access 'y' before initialization
    at repl:1:26
    at repl:1:15

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:02:43 -07:00
Anders Kaseorg 4d37dfcf85 js: Convert vars declared separately and assigned once to const.
Because of the separate declarations, ESLint would convert them to
`let` and then trigger the `prefer-const` error.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:02:43 -07:00
Anders Kaseorg c3b4c0a229 frontend_tests: Use let with explicit null in components test.
Even though this variable was only assigned once, it was accessed
before its initialization, so it couldn’t be converted directly to
`let` or `const`.  Use `let` with an explicit `null` to make it
clearer what’s going on and satisfy ESLint.  (Why not `undefined`?
There’s an ESLint rule against that too.)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-28 15:02:43 -07:00
Anders Kaseorg d17b577d0c js: Purge useless IIFEs.
With webpack, variables declared in each file are already file-local
(Global variables need to be explicitly exported), so these IIFEs are
no longer needed.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-10-25 13:51:21 -07:00
Greg Price 71596648c2 typing_status: Switch sentinel "recipient" value to `null`.
This feels a bit more semantically appropriate: it more clearly says
"here's some information: there is no (relevant) recipient", rather
than "no information available".  (Both `null` and `undefined` in JS
can have either meaning, but `undefined` especially commonly means
the latter.)

Concretely, it ensures a bit more explicitness where the value
originates: a bare `return;` becomes `return null;`, reflecting the
fact that it is returning a quite informative value.

Also make the implementation more explicit about what's expected here,
replacing truthiness tests with `!== null`.  (A bit more idiomatic
would be `!= null`, which is equivalent when the value is well-typed
and a bit more robust to ill-typing bugs.  But lint complains about
that version.)
2019-10-24 14:56:56 -07:00
Greg Price a191890213 typing_status: Fold `stop` into main method `update`.
It'd already been the case for some while that calling `stop` had the
same effect as calling `update` (previously `handle_text_input`) with
a falsy recipient.  With the API changes in the previous few commits,
this becomes quite natural to make explicit in the API.
2019-10-24 14:56:56 -07:00
Greg Price e639b0a6f8 typing_status: Write jsdoc for main entry point, and rename.
This was named after when it gets called from the UI, rather than
after what it can be expected to do.

Naming it after what it's meant to do -- and giving a summary line to
expand on that -- provides a more helpful semantic idea for reasoning
about the function.  Doubly so for using the function in a different
client with its own UI, like the mobile app.
2019-10-24 14:56:56 -07:00
Greg Price dcb5bb7914 typing_status: Combine two parameters into one, with a maybe-type.
The main motivation for this change is to simplify this interface
and make it easier to reason about.

The case where it affects the behavior is when
is_valid_conversation() returns false, while current_recipient
and get_recipient() agree on some truthy value.

This means the message-content textarea is empty -- in fact the
user just cleared it, because we got here from an input event on
it -- but the compose box is still open to some PM thread that we
have a typing notification still outstanding for.

The old behavior is that in this situation we would ignore the
fact that the content was empty, and go ahead and prolong the
typing notification, by updating our timer and possibly sending a
"still typing" notice.

This contrasts with the behavior (both old and new) in the case
where the content is empty and we *don't* already have an
outstanding typing notification, or we have one to some other
thread.  In that case, we cancel any existing notification and
don't start a new one, exactly as if `stop` were called
(e.g. because the user closed the compose box.)

The new behavior is that we always treat clearing the input as
"stopped typing": not only in those cases where we already did,
but also in the case where we still have the same recipients.
(Which seems like probably the common case.)

That seems like the preferable behavior; indeed it's hard to see
the point of the "compose_empty" logic if restricted to the other
cases.  It also makes the interface simpler.

Those two properties don't seem like a coincidence, either: the
complicated interface made it difficult to unpack exactly what
logic we actually had, which made it easy for surprising wrinkles
to hang out indefinitely.
2019-10-24 14:56:56 -07:00
Greg Price dcccef9b3a typing_status: Make some test cases slightly less artificial.
All these cases are meant to simulate having a user actually typing a
message to some actual recipients, so the `conversation_is_valid`
parameter would be true.

We make this change so that in an upcoming change that eliminates this
parameter, the adjustments to the test cases can be highly regular and
we don't have to introduce a new wrinkle to correspond to these values
being false.
2019-10-24 14:56:56 -07:00
Greg Price 5c220ed11a typing_status: Use parameters for data rather than callbacks.
The real purpose these two callbacks serve is exactly what an ordinary
parameter is perfect for:
 * Each has just one call site, at the top of the function.
 * They're not done for side effects; the point is what they return.
 * The function doesn't pass them any arguments of its own, or
   otherwise express any internal knowledge that doesn't just as
   properly belong to its caller.

So, push the calls to these callbacks up into the function's caller,
and pass in the data they return instead.

This greatly simplifies the interface of `handle_text_input` and of
`typing_status` in general.
2019-10-24 14:56:56 -07:00
Rohitt Vashishtha 8d172d8bf6 tests/message_store: Assert both flag modifying code paths are consistent. 2019-10-21 22:25:25 -07:00
Rohitt Vashishtha 4cfb209dc5 unread: Don't count wildcard mentions in muted streams/topics.
Users generally don't expect wildcard mentions in muted streams and
topics to be treated as a mention, either for the purposes of desktop
notifications or the unread mention counts.

This fixes the unread mention counts part of the issue.

Fixes part of #13073.
2019-10-21 22:23:29 -07:00
Vishnu KS 62a8e378a6 support: Refactor realm details into a seperate template. 2019-10-21 16:52:46 -07:00
Greg Price a63786ac0d shared: Set up a way to share some frontend code with the mobile app.
This adds the general machinery required, and sets it up for the file
`typing_status.js` as a first use case.

Co-authored-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Anders Kaseorg a3475b422d typing_status: Convert to ES6 module.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Anders Kaseorg 173c9cee42 frontend_tests: Switch from ts-node to Babel; add rewire-ts plugin.
This will let tests rewrite TypeScript/ES6 module bindings that would
otherwise be read-only.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-17 16:48:23 -07:00
Vinit Singh 01b19291e7 search: Advertise the ability to search shared history.
When a user performs a search that might contain historical public
streams messages that the user has access to (but doesn't because
we're searching the user's own personal history), we add a notice
above the first search result to let the user know that not all
messages may have been searched.

Fixes #12036.
2019-10-09 15:12:52 -07:00
Tim Abbott d6c9de6036 filter: Extract filter.contains_only_private_messages.
This will be a useful reusable function for determining whether to
display other alerts as well.
2019-10-09 14:47:38 -07:00
Rohitt Vashishtha c298163a67 typeahead: Prioritize language names subset of another for sorting.
This ensures that typing '```java' and pressing enter would result in
getting dropped into a java codeblock instead of javascript codeblock.

We implement this by pushing the exact match of a query to be pushed to
the top of the returned matches in `sort_languages`.

With some comments added by tabbott in the tests explaining the
current reasoning.

Fixes #13109.
2019-09-26 13:00:21 -07:00
clarammdantas cf5d3a3ef3 left sidebar: Fix bot availability status in "private messages".
This changes the availability icon for bot users to user_circle_green;
previously it was accidentally defaulting to user_circle_empty, making
it appear that bots were never available.

Fixes #13149.
2019-09-18 17:40:25 -07:00
Tim Abbott edee1251c8 message_list: Replace buggy rerender_the_whole_thing.
As it turns out, our rerender_the_whole_thing function (used whenever
we were adding messages and discovered that the resulting message list
would be out-of-order) was just broken and scrolled the browser to a
random location.

This caused two user-facing bugs:

* On very fast networks, if two users sent messages at very close to
  the same time, we could end up with out-of-order message deliveries,
  triggering this code path, which was intended to silently correct
  the situation, but failed.

* In some narrows to streams with muted topics in the history but some
  recent traffic, the user's browser-cached history might have some
  gaps that mean the server fetch we do after narrowing discovers the
  history is out-of-order, again triggering the
  rerender_the_whole_thing code path.

The fix is to just remove that function, adding a new option to the
well-tested rerender_preserving_scrolltop (which has explicit logic to
preserve the scroll position) instead.

Fixes #12067.  Likely also fixes #12498.
2019-09-18 11:43:21 -07:00
Thomas Ip 574c35c0b8 markdown: Render ordered lists using <ol> markup.
This brings us in line, and also allows us to style these more like
unordered lists, which is visually more appealing.

On the backend, we now use the default list blockprocessor + sane list
extension of python-markdown to get proper list markup; on the
frontend, we mostly return to upstream's code as they have followed
CommonMark on this issue.

Using <ol> here necessarily removes the behaviour of not renumbering
on lists written like 3, 4, 7; hopefully users will be OK with the
change.

Fixes #12822.
2019-09-08 16:42:20 -07:00
Mateusz Mandera bf7f4f3f1b stream settings: Replace email address hint popup with link to docs.
Fixes #13134 as the last commit in the series for this issue.
Solves the "The (?) should just be a target=_blank link to
/help/message-a-stream-by-email." part of the issue.
As a result, a bunch code managing the email hint popup can be deleted,
together with a node test for that.
2019-09-05 11:48:32 -07:00
Mateusz Mandera d70e1bcdb7 settings: Add FAKE_EMAIL_DOMAIN setting.
Fixes #9401.

This adds a FAKE_EMAIL_DOMAIN setting, which should be used if
EXTERNAL_HOST is not a valid domain, and something else is needed to
form bot and dummy user emails (if email visibility is turned off).
It defaults to EXTERNAL_HOST.

get_fake_email_domain() should be used to get this value. It validates
that it's correctly set - that it can be used to form valid emails.

If it's not set correctly, an exception is raised. This is the right
approach, because it's undesirable to have the server seemingly
peacefully operating with that setting misconfigured, as that could
mask some hidden sneaky bugs due to UserProfiles with invalid emails,
which would blow up the moment some code that does validate the emails
is called.
2019-08-30 14:59:00 -07:00
Wyatt Hoodes e64b5a2b88 data export: Fix success banner not clearing.
There was a bug where the success banner stuck
around even after the export completed.  We now
nicely fade and remove the banner upon a successful
population of the export in the table.

Fixes: #13045
2019-08-28 15:23:39 -07:00
Anders Kaseorg 43c8c720ef node_cache: Run yarn with --ignore-scripts.
As predicted in https://www.kb.cert.org/vuls/id/319816/, a malicious
worm is beginning to spread across the npm ecosystem through package
postinstall scripts.  Only instead of direct self-replicating code,
the replication vector is the temptation to monetize postinstall
scripts by polluting the console logs with paid advertisements.  The
effect will be the same unless we all put a stop to this while we
still can.

Apply the recommended VU#319816 workaround, which is to disable
lifecycle scripts when installing npm packages.  The only fallout is:

* node-sass can’t run because it uses compiled native code; we replace
  it with Dart Sass.
* phantomjs-prebuilt doesn’t download the binary at install time; we
  tell it to download it in run-casper.
* ttf2woff2 transparently falls back from native code to an Emscripten
  build.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-26 14:36:03 -07:00
Kanishk Kakar e4f0d3d79b notifications: Add 'none' to unread count options. 2019-08-25 21:29:10 -07:00
Mateusz Mandera 16479fa606 populate_db: Extract internal realm creation into a function. 2019-08-23 12:57:24 -07:00
Mohit Gupta e5482adec0 search: Add streams:public to search entire history of public streams.
Add ability to search entire message history of all public streams at
once. It includes all subscibed, non subscribed public streams messages
and even historical public stream messages sent before user had joined
an organization or stream.

Fixes #8859.
2019-08-22 13:40:49 -07:00
Thomas Ip d41d965eed refactor: Group header and body under table for .progressive-table-wrapper. 2019-08-22 13:13:24 -07:00
Thomas Ip cbae51db63 settings: Move API key form into its own modal.
The modal is rendered dynamically to avoid password managers
inserting passwords into the input field too aggressively.

Fixes #12523.
2019-08-14 10:50:45 -07:00
Wyatt Hoodes f623540409 data export: Add UI to trigger data export.
This commit serves as the frontend piece for the "public export"
webapp feature.

Fixes: #11930
2019-08-12 18:21:38 -07:00
vinitS101 3d01921e1a user status: Changes to Last active field of Full User Profile.
If a user was active within the last 90 days,
show number of days (23 Days ago).
If the user was active more than 90 days ago and in the same year,
then show MMM DD (Mar 15).
In any other case show MMM DD YYYY (Nov 10 2018),
Change timerender.js test to accomodate changes.
2019-08-07 16:20:19 -07:00
vinitS101 232f588d4e user status: Change Online now to Active now in full user profile.
Change "Online now" to "Active now" in Last seen field of
full user profile.
2019-08-07 16:20:19 -07:00
Anders Kaseorg 0962393933 cleanup: Delete trailing newlines.
Delete trailing newlines from all files, except
tools/ci/success-http-headers.txt and tools/setup/dev-motd, where they
are significant, and static/third, where we want to stay close to
upstream.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Pragati Agrawal ac2f1cea9c settings_org: Enhance `show_emails` for admins only case too.
This adds on the `is_admin` clause to show_emails.
2019-08-02 15:20:55 -07:00
Tim Abbott 22a85b0e1e zjsunit: Add support for using i18n without escaping.
This adds support for the `- __foo__` syntax supported by i18n.t.
2019-08-01 12:58:11 -07:00
Mateusz Mandera d3d62f99e4 tests: Adjust to new changes in populate_db. 2019-07-31 16:01:37 -07:00
YashRE42 7a6f4630dc compose_box: Prepopulate stream if possible.
When users are only subbed to a single stream, this autofills the stream
field of the compose box.
Fixes #12507.
2019-07-31 10:20:24 -07:00
Tim Abbott fac886ce05 Revert "compose: Fix cursor placement timing bug when selecting a typeahead."
This reverts commit 76e50af78e.

Empirically, this caused weird issues with the cursor jumping around,
so more investigation is required into the right way to fix it.
2019-07-29 18:05:46 -07:00
Vinit Singh 03180752db compose: Update placeholder text depending on the narrow.
Change the `compose-textarea` placeholder text depending on the
stream/topic or PM recipients that the message will be sent to.

Resolves #12834.
2019-07-29 15:51:50 -07:00
Anders Kaseorg f0e0fe1c15 ui.get_scroll_element: Set up SimpleBar if it’s expected but missing.
Although SimpleBar automatically sets itself up on elements with a
`data-simplebar` attribute, sometimes we try to set event listeners
before that happens.  Create the SimpleBar early in that case.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-25 16:18:54 -07:00
Rohitt Vashishtha 76e50af78e compose: Fix cursor placement timing bug when selecting a typeahead.
When you press enter on a typeahead and start typing, your cursor is
placed at the end of the textbox, whereas we want it to be placed at
the end of the typeahead immediately. This causes some characters to
appear at the end of the message before you again get to typing from
where you left off.

To fix, we use the change event triggered on typeahead completion to
reposition the cursor instead of using a setTimeout().

Fixes #12621.
2019-07-25 15:01:24 -07:00
Rohitt Vashishtha 4f03d82ff0 compose: Do not trigger topic mention if already completed. 2019-07-25 14:53:43 -07:00
Priyank Patel e69c2f1aa2 notifications: Send message received from desktop app notification reply.
Adds a electron_bridge event that takes in message id and reply recived from
the notification reply and sends a message. We do this in webapp so desktop
doesn't have to depend on narrow and channel modules.

We also modify zjunit to reset window.electron_bridge after every run
to avoid leaking it.
2019-07-22 17:20:44 -07:00
Rohitt Vashishtha 5fc37c5f9b compose: Add compose typeahead for stream+topic mentions.
We implement 3 changes:

1. Partial Stream Typeahead

   In addition to regular stream completion, we do partial completion
   of stream typeahead on pressing '>'. We use our custom addition to
   typeahead.js: this.trigger_selection to start topic_list typeahead.

   Implements: `#stream na|` (press >) => `#**stream name>|`.

2. Topic Jump Typeahead

   'topic_jump' typeahead moves the cursor from just ahead of a
   completed stream-mention to just after the end of the mention
   text and is triggered by typing '>' after the stream mention.
   This typeahead merely uses the regex matching and event hooks of
   the typeahead library instead of displaying any text completions.

   Implements: `#**stream name** >|` => `#**stream name>|`.

3. Topic List Typeahead

   'topic_list' typeahead shows the list of recent topics of a stream
   and if your current text doesn't match one of them, also shows you
   the current query text, allowing you to create mentions for topics
   that do not exist yet.

   Implements: `#**stream name>someth|` => `#**stream name>something** |`.

At the end of this commit, we support the following mechanisms to
complete the stream-topic mention:

1. Type "#denmar|".
2. Press Enter to get "#**Denmark** |".
3. Press > to get "#**Denmark>|".
4. Type topic name and press enter.

OR

1. Type "#denmar|".
2. Type > to get "#**Denmark>|".
3. Type topic name and press enter.

Both result in the final inserted syntax: "#**Denmark>topic name**".

Documentation is still pending.

Fixes #4836.
2019-07-21 20:38:17 -07:00
Rohitt Vashishtha 5d20c4b8fb typeahead: Clear rendered stream html on stream rename.
Previously, after a stream name, you could search for it using its
new name but the typeahead would still display the old name.
2019-07-21 20:18:29 -07:00
YashRE42 9f5fca5579 notifications: Refactor and test notifiable unreads logic.
In this refactor, we extract two functions in unread.js.  Which one to
use depends on whether res has already been fetched or not.

This also adds node tests to maintain coverage of unread.js.

Tweaked by tabbott for cleaner variable names and tests.
2019-07-21 14:56:42 -07:00
YashRE42 8cf15fe01d node_tests: Mark unread PM as read after testing.
This fixes a spot in unread.js where we weren't marking our unread PM as
read at the end of the test.
2019-07-21 14:50:00 -07:00
Anders Kaseorg 06aa87296c dependencies: Upgrade eslint to 6.0.1.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-20 16:12:51 -07:00
Rohitt Vashishtha 9d6727d18c echo: Update topic_links when we get messages back from server. (#12832) 2019-07-20 14:38:52 -07:00
Wyatt Hoodes 9b69c1f7fe Merge pull request #12833 from whoodes/clean-up-new-user-bot
bots: Replace remnants of new-user-bot with default-bot@zulip.com.
2019-07-20 14:38:05 -07:00
vinitS101 fe2ec995b6 message_view: Add js tooltip hovers for emoji reactions.
This removes HTML title hovers for emoji reaction buttons below messages
and replaces them with js tooltips.

Fixes #8679.
2019-07-19 12:45:44 -07:00
Tim Abbott 500b161aab unread: Enable the load_server_counts setting for everyone.
This change is long overdue.  After implementing this much more robust
system and deploying it on chat.zulip.org, we hesitated to make
load_server_counts the default behavior in master, because of data
anomalies present for many existing users (basically messages far back
in their history that they had never read, on streams they believed
themselves caught up on), which would have been confusing for many
users.

However, because the mobile apps have been using this data set for a
long time, we've likely cleared out the anomalies from active users'
data set.  And for older users, they're going to come back to
approximately infinite unread messages anyway, so the data anomalies
are unlikely to be important.

Fixes #7096.
2019-07-18 13:34:55 -07:00
Mohit Gupta 648a60baf6 narrow: Add condition whether to show unread message first in narrow.
All narrows that have is: query or can mark unread message as read
will show unread message first.
2019-07-17 17:58:20 -07:00
Mohit Gupta 6ec40cf9a0 search: Don't mark messages as read in search narrow.
Don't mark unread messages as read while searching.
This behavior will be extended to other narrows later.

Fixes: #12556.
2019-07-17 17:58:20 -07:00
Anders Kaseorg b1aa304c4a templates: Suppress Handlebars automatic partial indentation.
Fixes #12795.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-17 16:07:17 -07:00
Cynthia Lin 50cab76c59 user settings: Hide disabled name change info icon for admins.
This icon should only show when the user is not an admin and either the
realm or server settings have disabled name changes. Previously the icon
always showed for admin users.
2019-07-16 11:43:57 -07:00
Cynthia Lin a73155c78f node tests: Fix checking for invalid :checked property in templates. 2019-07-16 11:43:57 -07:00
Cynthia Lin a65007dde4 right sidebar: Ensure .user-with-count gets added to correct li element.
The count_span element is parented by a .selectable_sidebar_block element
which is parented by the li element that the class is supposed to be added
to. Thus, use the parents() jQuery method for locating the li parent so
that the class gets added to the correct element.
2019-07-16 11:33:02 -07:00
David Wood 9bace3f2cd notifications: Allow only notifiable in unread count.
This commit adds a new setting to the user's notification settings that
will change the behaviour of the unread count in the title bar and
desktop application.

When enabled, the title bar will show the count of unread private messages
and mentions. When disabled, the title bar will act as before, showing
the total number of unread messages.

Fixes #1736.
2019-07-13 15:49:04 -07:00
Priyank Patel 73b19672c3 message_fetch: Use user IDs for supported operators.
The approach taken here is basically use user IDs in operator that
support it when sending the request for fetching the messages
(see comments in code for more details).
2019-07-13 11:35:37 -07:00
vsvipul e830853aee desktop-presence: Use system presence data from electron-bridge.
Combined with work in the desktop app, this makes it possible for the
desktop app to clearly indicate to other users whether the current
user is active on the system and thus would see a desktop
notification, not just whether they are active in the current Zulip
window.

Essentially rewritten by tabbott to add unit tests and consider the
desktop app data authoritative.
2019-07-13 11:21:22 -07:00
Anders Kaseorg 0bd5c46e40 zjsunit/handlebars: Support auto-registering partials.
webpack with handlebars-loader already does this, so we only need to
fix up the tests.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg db0b33842c templates: Replace templates.render with require calls.
This removes an unnecessary layer of indirection and allows webpack to
catch filename mistakes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg 706f13b671 zjsunit: Wrap Handlebars with mocking support.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg fb3fac1d96 zjsunit: Add make_handlebars abstraction.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg a0122abf9a zjsunit: Add stub_templates abstraction.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg 8761e09eed zjsunit: Remove render.js.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:14 -07:00
Anders Kaseorg 3c3471b720 templates: Rename *.handlebars ↦ *.hbs and - ↦ _.
Tweaked by tabbott to avoid accidentally disabling the linter for
handlebars templates.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-12 21:11:03 -07:00
Tim Abbott e8e420bbd9 markdown: Fix marked generation of unnecessarily absolute URLs.
The new versions should exactly match the HTML we generate in the
backend unit test suite.
2019-07-11 15:09:38 -07:00
Rohitt Vashishtha 3698cdcc58 topic-mention: Add Marked implementation as HandleStreamTopic. 2019-07-11 14:53:10 -07:00
Yashashvi Dave 27ead227c0 custom fields: Add separate alert-save widget for create field.
Add separate alert-notification widget for create-custom-field
in admin view.

Fixes part of #12748
2019-07-11 12:52:21 -07:00
Tim Abbott 5c32e2ff49 popovers: Fix migration to popovers.hide_all_except_sidebars().
I'm not really sure how I missed this in the original commit.
2019-07-09 23:03:41 -07:00
Tim Abbott 78bb3f7ea1 tests: Fix mock for hide_streamlist_popover.
Apparently, I did my grep wrong to check for references in
f5b1081c93.
2019-07-09 17:54:55 -07:00
Yashashvi Dave d7ee2aced1 models: Add `external_account` in custom profile field types.
Add new custom profile field type, External account.
External account field links user's social media
profile with account. e.g. GitHub, Twitter, etc.

Fixes part of #12302
2019-07-09 17:21:54 -07:00
Anders Kaseorg e0a18d3394 blueslip: Replace jQuery wrappers with error event listener.
Not all our errors actually happen in the contexts we were
wrapping (e.g. `setTimeout` and `_.throttle`).  Also this fixes the
neat Firefox inspector feature that shows you where your event
handlers for a given DOM element actually live.

Using this "semi-modern" browser event means that Safari 9 and older
and IE10 and older may not have our browser error reporting active;
that seems fine giving the vanishing market share of those browsers.

https://blog.sentry.io/2016/01/04/client-javascript-reporting-window-onerror

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-09 11:38:20 -07:00
Anders Kaseorg 0c80fc6aba hashchange: Use hashchange event listener.
The comment that jQuery “doesn’t have” this was nonsense: jQuery
supports every event the browser does.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-09 11:31:35 -07:00
Anders Kaseorg 9368c33098 unread.js: Add setter for suppress_unread_counts.
After migration to an ES6 module, `suppress_unread_counts` would no
longer be mutable from outside the module.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-07-08 21:22:54 -07:00
Anders Kaseorg f9bf414b58 pointer.js: Add setter for server_furthest_read.
After migration to an ES6 module, `server_furthest_read` would no
longer be mutable from outside the module.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-07-08 21:22:54 -07:00
Anders Kaseorg 055ebe76aa pointer.js: Add setter for furthest_read.
After migration to an ES6 module, `furthest_read` would no longer be
mutable from outside the module.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-07-08 21:22:54 -07:00
Anders Kaseorg 15192d4417 message_list.js: Add setter for narrowed.
After migration to an ES6 module, `narrowed` would no longer be
mutable from outside the module.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-07-08 21:22:54 -07:00
Yashashvi Dave d3e83483c8 static/js/common: Add `Ctrl` key mapping to `Cmd` for MacOS. 2019-07-08 20:07:49 -07:00
Vishnu Ks 1f250a3fd2 casper: Add support for exporting results as XUnit XML file. 2019-07-07 22:30:01 -07:00
Anders Kaseorg bde31a54c7 bot_data: Rename delete to del.
`delete` is a reserved keyword that would interfere with the migration
to an ES6 module.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-07-04 16:48:33 -07:00
Wyatt Hoodes a238795823 run_casper: Add error output regarding Casper artifacts in CircleCI. 2019-07-03 14:58:48 -07:00
Anders Kaseorg b0be0d5285 settings_account: Use webpack asynchronous require to load zxcvbn.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:50:03 -07:00
Anders Kaseorg f346d0e511 dependencies: Use core-js for String.prototype polyfills.
It seems like the de facto standard ES polyfill library these days,
and we already depend on it through simplebar.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:50:03 -07:00
Anders Kaseorg 1b94733953 webpack: Remove resolve.modules override.
The minimal syntactic sugar it might provide isn’t worth the
unexpected side effects (including side effects on third party
modules).

For now, we allow zrequire to emulate the previous syntax in the Node
test suite, even though stealing part of the NPM namespace is
confusing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:38:26 -07:00
Thomas Ip f6aaf43029 refactor: Use explicit path when referencing handlebars templates. 2019-07-02 16:23:30 -07:00
Anders Kaseorg 11a305e320 zjsunit: Simplify typeRoots path (it’s relative to tsconfig.json).
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-02 16:09:29 -07:00
Tim Abbott 746206f071 activity: Rename has_focus to client_is_active.
This makes it a lot more clear what it actually means, which is not
directly related to whether the browser window is focused.
2019-06-28 18:06:30 -07:00
Puneeth Chaganti 7da6d5a168 narrow: Fix broken narrows for invalid email IDs.
When pm-with or group-pm-with searches had invalid email IDs, the code
to set the narrow title would break. This commit handles that
correctly.
2019-06-28 17:06:20 -07:00
Puneeth Chaganti 91b2ac27ae narrow: Extract test for non existing user narrow. 2019-06-28 16:56:58 -07:00
Mohit Gupta 487861554f search: Fix searching and search suggestion in Group PM.
Fix the .get_suggestions and .get_suggestions_legacy
to correctly handle search terms in group PM and treat
it as search term by not concatenating it at end of pm-with
email list operand.
2019-06-28 12:45:37 -07:00
Rohitt Vashishtha 4489df7cd6 echo: Add tests for echo.process_from_server. 2019-06-28 06:13:41 +00:00
Rohitt Vashishtha 047086b81c markdown: Make raw urls in topic names navigable.
We reuse the link regexes we use elsewhere inn markdown
for parsing links in topic names and add a button to open
them in new tabs similar to our behavior with linkifiers
in topic names.

Fixes #12391.
2019-06-27 15:18:42 -07:00
Rohitt Vashishtha 96d7c1f3b0 markdown: Test escaping of topic_links and document. 2019-06-27 15:18:30 -07:00
Anders Kaseorg 23cd064c86 webpack: Elide node_modules when importing JS modules.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-26 16:49:32 -07:00
Puneeth Chaganti 3cab98169b casper: Fix link to casper Zulip docs. 2019-06-26 16:15:08 -07:00
Puneeth Chaganti 9c33261728 casper: Add a verbose flag to run casper tests in verbose mode. 2019-06-26 16:15:08 -07:00
Puneeth Chaganti 0409db3c9d narrow: Improve window titles of is:<type> narrows.
We capitalize the type name, and add a " messages" suffix.
2019-06-26 16:06:22 -07:00
SatyendraBanjare 066c168448 narrow: Window title shows user's name, in private message narrows.
Modified heavily by punchagan to correctly handle narrowing to huddles, and
for `group-pm-with` narrows. Also, fixed broken tests in the original PR.

Closes #5876
2019-06-26 16:06:22 -07:00
Rishi Gupta de2ba4743e compose: Rename New conversation button to New private message.
Right now we have buttons for "New conversation" and "New private message"
in different views, but both buttons do the same thing.

The current state is confusing for new users, since there is already a lot
of terminology one needs to learn in order to understand the Zulip
conversation model. It's very plausible a user would think a "conversation"
is something different from a "private message" or a "topic".
2019-06-26 14:12:24 -07:00
Pragati Agrawal e42abc2396 org_settings: Optimize data collected by `populate_data_for_request`.
With the help of `check_property_changed` function now we collect the data
whose values are changed from the current one. Currently this optimizes
only for those elements whose values are collected by
`populate_data_for_request` function i.e. it doesn't optimize data
collected by `get_complete_data_for_subsection`.
2019-06-25 16:57:59 -07:00
Pragati Agrawal 81492362d7 org_settings: Refactor `populate_data_for_request`.
This is a preliminary commit which refactors `populate_data_for_request`
function, now this function traverse on all "property elements" of a given
subsection, but get the data only of those properties which have
`setting-widget-type` data attribute. Therefore, it doesn't change the
functionality of this function and overall changes don't make any
difference. In upcoming commits, we're going to use `input_elem` as an
argument to `check_property_changed` function, so that only those elements
whose values are changed are sent to the backend.
2019-06-25 16:57:59 -07:00
Pragati Agrawal 747e797092 stream settings: Disable Mobile notifications if push bouncer not set up.
This commit disables "Mobile notifications" if `push_notifications` are not
enabled. It also adds a tooltip explaining why this is disabled.

Fixes #12208
2019-06-24 14:46:45 -07:00
Yashashvi Dave 02e82ef10c static/js/stream_events: Deduplicate notification settings updates. 2019-06-24 14:46:45 -07:00
Yashashvi Dave e0a78af494 static/js/stream_data: Extract function `receives_notifications`. 2019-06-24 14:46:45 -07:00
Yashashvi Dave 1b9c1a7898 stream settings: Deduplicate template rendering. 2019-06-24 14:46:45 -07:00
Yashashvi Dave 1f5720d1b8 stream settings: Add `name` attr to settings elements.
Add `name` attribute to stream settings `input` elements
and change `id` attribute corresponding to name.
`name` attribute stores value of settings name which is
stored in frontend data sets.
2019-06-24 14:46:45 -07:00
Yashashvi Dave 47f9cbf824 display settings: Deduplicate template rendering. 2019-06-24 14:41:53 -07:00
Yashashvi Dave fed64cc59d static/js/common: Extract function `adjust_mac_shortcuts`. 2019-06-24 14:04:42 -07:00
Yashashvi Dave 251ed94bfc static/js/common: Use `platform` navigator to detect MacOS.
Replace `userAgent` navigator with `platform` navigator
to detect user's OS.
2019-06-24 14:04:42 -07:00
Yashashvi Dave 371343709b static/js/common: Extract function `has_mac_keyboard`. 2019-06-24 14:04:42 -07:00
YashRE42 50d43902fb narrows: Show invalid banner for invalid narrows.
Some search queries always return empty because of how we handle search,
this adds text that ensures users trying bad searches realize that they
are doing so.
2019-06-24 13:14:10 -07:00
YashRE42 02413f9a1b search: Show stopwords in multi-operator search.
Fixes #10592.
2019-06-24 13:08:08 -07:00
Aman Agrawal b2b49089fd tools: Extract get_provisioning_status check logic.
Move get_provisioning_status check logic into
assert_provisioning_status_ok and use it instead of duplicating the
check code.
2019-06-23 21:55:02 -07:00
Aman Agrawal 904422d8bc css: Convert colors from rgba to hsla format. 2019-06-20 11:34:59 -07:00
Aman Agrawal 3e589cf65e css: Convert colors from rgb to hsl format.
Note that we are not converting the stream colors.
2019-06-20 11:34:34 -07:00
Yashashvi Dave 25dbda1e53 stream settings: Fix bug in deactivate stream modal appearance.
Stream deactivation modal should be append to main
unique id, stream overlay element.
This commit also add `new-style` class to modal.

Fixes #12394
2019-06-20 11:09:52 -07:00
Tim Abbott e6cb83b6e9 docs: Add a new heading section on Casper debugging.
Also enhance the error output from the casper tool.
2019-06-16 22:20:41 -07:00
Thomas Ip 0253ef12c6 popovers: Encode brackets in URI to avoid conflict with markdown links.
This fixes an issue where one could end up with a `(` in the markdown
syntax for a link after copy-pasting this, which doesn't work in
markdown.

Fixes #12579.
2019-06-16 15:18:33 -07:00
Yashashvi Dave 981433d13c notification settings: Deduplicate template rendering. 2019-06-14 14:14:03 -07:00
Vishnu Ks 14e582fb59 support: Add functionality to copy admin emails.
Also renamed a bunch of functions in test_views for better
readability.
2019-06-14 10:19:50 -07:00
Yashashvi Dave 8e269b4651 models: Rename notification to `enable_stream_audible_notifications`.
Rename notification property `enable_stream_sounds` to
`enable_stream_audible_notifications` to match with other
notification property patterns.

Fixes part of #12304
2019-06-12 16:24:51 -07:00
Seth Nickell 93696729c6 notifications: Disable default permission pop up.
Prior to this commit, we'd put up the green "Enable desktop
notifications" bar on page load AND the first time a desktop
notification worthy message was received, it would attempt to notify,
automatically triggering a browser permission popup (the same one as
clicking the green bar results in).

Now, desktop notifications are not attempted at all until the green
bar is clicked. Additionally Firefox and Webkit browser-specific
checks are made more uniform and done at the same point.

Tested written by YashRE42.

Fixes #11504.
2019-06-12 16:12:13 -07:00
Priyank Patel b70bd6be30 tests: Add tests for the logic of typing_status.handle_text_input.
This tests was added to make sure we catch subtle bug related to
comparing new_recipient and current_recipient. When we changed the
recipient to use arrays instead of string to use new user IDs based
api we encoured this bug and out testing suite couldn't detect this.
2019-06-06 19:56:24 -07:00
Thomas Ip 3b90eed007 jQuery: Replace positional selectors on the first of multiple selectors.
A selector like `$('.elem1:first .elem2')` selects all `.elem2` under
the first `.elem1`. We can instead use `$('.elem1').first().find('.elem2')`.
2019-06-06 15:21:26 -07:00
Thomas Ip b72f30bd06 jQuery: Replace positional selectors at the end of multiple selectors.
A selector like `$('.elem1 .elem2:first')` selects the first descendant
with class name `.elem2` under `.elem1`. This is the same as saying
`$('.elem1 .elem2').first()`. See example here:
https://jsbin.com/bohehesari/edit?html,js,output
2019-06-06 15:21:26 -07:00
Thomas Ip 6cad1988a9 jQuery: Use positional methods on single selectors.
For selectors like `$('.element:first')`, we can simply write
`$('.element').first()`.
2019-06-06 15:21:26 -07:00
Thomas Ip c6cdcd082e dependencies: Upgrade node to 12.3.1.
API changes:
* The behaviour of Date.toLocaleTimeString() reverts to pre 8.0.0,
  this only affects automated tests. Lots of other API changes but
  we didn't use any of those.
* The internal sorting algorithm changed which causes one of our own
  compare function to miss coverage.
2019-06-05 17:15:52 -07:00
sahil839 5a130097bf settings: Add display setting for demoting inactive streams.
This adds a setting to control Zulip's default behavior of sorting to
bottom and graying out inactive streams.  The previous logic is still
the default "automatic", but this gives users more control.  See the
models.py comment for details.

Fixes #11524.
2019-06-03 23:07:56 -07:00
YashRE42 1ab4eaf819 stream_list: Pinned streams are never grayed out as inactive.
We have had a longtime bug where the state of pinned streams would not
update properly from the greyed out/inactive state to the active state
when a first message arrived to them.

After some discussion, we determined that likely the right fix for
this is to simply configure pinned streams to never be marked as
inactive; that's more in line with the intended user experience.

Fixes #8201.
2019-06-03 17:23:01 -07:00
Yashashvi Dave 2a943d3b40 static/js/stream_muting: Rename `update_in_home_view` function. 2019-05-30 21:39:06 -07:00
Yashashvi Dave 784d02bf60 static/js/stream_data: Rename `in_home_view` functions. 2019-05-30 21:39:06 -07:00
Yashashvi Dave 40f550038d subs: Replace all `in_home_view` uses with `is_muted` property.
Replace all uses of `in_home_view` subscription property
with `is_muted` property in frontend.

Fixes #12322
2019-05-30 21:39:06 -07:00
Pragati Agrawal 69ec96b63e search suggestion: Hide email under hidden email-address-visibility cases.
This commit hides the email address from the search bar for the email
hidden cases.
2019-05-30 16:22:54 -07:00
Vaibhav 3462db3ecc node: Add test for updating bot_owner_id and enable user_events test suite.
Reverts the commit 4442509ec3
2019-05-30 15:43:18 -07:00
Vishnu Ks 8c249644b1 billing: Seperate out tests for invoice and autopay form fields. 2019-05-30 10:34:56 -07:00
Vishnu Ks a46b2386b6 billing: Add node tests for helpers.js. 2019-05-30 10:34:56 -07:00
Puneeth Chaganti 75635844db lightbox: Set URL in payload instead of computing from video IDs.
Lightbox previews for youtube playlists use the "current" video in the playlist
for the preview. The open link for such previews is incorrectly set to the first
video alone, and not the playlist. This commit fixes the bug by linking to the
original URL for lightbox preview is being shown, instead of computing the URL.
2019-05-29 23:01:54 -07:00
Puneeth Chaganti 85b8be0ace zjquery: Add length attribute to wrapped elements. 2019-05-29 23:01:54 -07:00
Puneeth Chaganti be527905ca zjquery: Add closest() method. 2019-05-29 23:01:54 -07:00
Priyank Patel 3f32ffc4eb compose: Use new ID-based api for sending messages.
This only happens if the realm is not a zephyr realm.

Finishes part of #9474.
2019-05-27 22:58:42 -07:00
David Wood 6b9a9b1e31 org settings: Extract setting for new user waiting period.
This commit separates the `waiting_period_threshold` setting from
the `create_stream_policy` setting, adding a new setting that the user
can use to select a waiting period threshold.

Both the invite to stream policy and create stream policy now have
three options: admins only, members and admins, or members after
waiting period/admins.
2019-05-21 17:36:48 -07:00
Aditya Bansal 43591d6c71 archives: Display globe icon for web public streams.
In this commit we start reserving the globe icon for web public
streams and replace instances of mixed usage of globe for public
streams with a '#'.
2019-05-20 18:13:31 -07:00
Pragati Agrawal 8eac7394f8 typeahead helper: Hide email under hidden email-address-visibility cases.
In email hidden case (that is when `email_address_visibilty` is set to
everyone), for "non admins", this commit hides emails from:
- compose box user typeahead.
- PM user typeahead
In email hidden case, for admins, email is shown in user typeaheads.
2019-05-20 15:56:23 -07:00
Pragati Agrawal cdc50090b6 popovers: Hide email under hidden email-address-visibility cases.
In email hidden case (that is when `email_address_visibilty` is set to
everyone), for "non admins", this commit hides emails from:
- user popover
- custom profile popover
In email hidden case, for admins, email is shown in both user popovers and
custom profile popovers.
2019-05-20 15:56:21 -07:00
Pragati Agrawal 4df971c3c2 org settings: Add save/discard widget for realm authentication methods.
Along with this, we refactored settings_org.populate_auth_methods to use
HTML function after rendering all auth methods rows rather than appending
each row individually, which actually is a good practice.

Also in this commit, to compare `current_val` and `changed_val` in
`check_property_changed` function of the property
`realm_authentication_methods`, which are objects, and we found here
https://stackoverflow.com/a/1144249 that there is no easy way to do so. So
I followed this approach,

```js
 JSON.stringify(obj1) === JSON.stringify(obj2)
```

but before converting them to string we want the same order of keys, so we
used `sort_object_by_key` to sort `current_val` by keys and
`get_auth_method_table_data` always return `changed_val` having keys
sorted.

Since these refactor were closely related we kept them as a single commit
here.

Fixes: #11954.
2019-05-20 15:42:15 -07:00
Steve Howell 6b39d6004e zjquery: Use Proxy to detect undefined stubs.
We now use a Proxy to wrap zjquery elements, so
that we can detect callers trying to invoke methods
(or access attributes) that do not exist.  We try
to give useful error messages in those cases.

The main impact here is that we force lots of tests
to explicitly stub `length`.

Also, we can't do equality checks on zjquery
objects any more due to the proxy object, but the
easy workaround is to compare selectors.  (This
is generally an unnecessary technique, anyway.)

The proxy wrapper is fairly straightforward, and
we just have a few special cases for things like
"inspect" that happen when you try to print out
objects.
2019-05-20 11:28:32 -07:00
Anders Kaseorg 01613e71fb ui: Replace set_up_scrollbar with data-simplebar attribute.
With perfectScrollbar, we needed to call a function from JavaScript to
enable a scrollbar on a new element, but simplebar has a much simpler
default API one can do by using data-simplebar attributes in the HTML.

So we can delete all the scrollbar creation/deletion code.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-19 18:06:38 -07:00
Anders Kaseorg 3daa4fa64d pm_list: Set up #private-container once and replace its contents.
Instead of deleting and rebuilding #private-container every time its
contents need to be updated, just replace its contents.  This
eliminates some scrollbar flashing.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-17 12:06:51 -07:00
Anders Kaseorg 141088586b Completely replace perfect-scrollbar with SimpleBar.
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms.  SimpleBar customizes the appearance while preserving the
native behavior.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-17 12:06:51 -07:00
Mohit Gupta 1902f5210c message_flags: Batch read flag requests to at most 1k message IDs at a time.
Send at most 1k message ids in a single read flag request to avoid locking
large number of rows in server database in a single request and avoid long
processing time.

Fixes #11956.
2019-05-15 17:56:48 -07:00
Vaibhav d3b201337e copy_and_paste: Bind default copy handler to hotkeys.
This removes the 'copy' event listener and binds the copy handler
to `Ctrl+C` or `Cmd+C` (OSX) keys;
2019-05-13 17:58:12 -07:00
Yashashvi Dave 2598db4a77 streams: Hide unsubscribed streams in settings to guest users. 2019-05-13 17:27:32 -07:00
Yashashvi Dave 80ceb1a46d components: Add `disable_tab` attribute. 2019-05-13 17:27:32 -07:00
Yashashvi Dave cb85ca8601 models: Alter video_chat_provider field type to integer.
Migration rewritten by tabbott because it did not work.
2019-05-13 12:02:28 -07:00
Pragati Agrawal a838de63d8 settings: Add option for fixed/fluid width.
This adds a new option for "fluid width" under `Display settings` section
of SETTINGS/DISPLAY SETTINGS tab.

Fixes: #11844.
2019-05-09 13:57:57 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
vinitS101 18a424be79 uploads: Remove unusable UI elements if file uploading is disabled.
If MAX_FILE_UPLOAD_SIZE is set to 0, then UI elements like the upload
icon in the compose and message edit UI and "Attachments" menu in
"/#settings" are not displayed.
A different error message is also displayed if a user tries to drag and
drop or paste a file into the compose message box.

Fixes #12152.
2019-05-08 17:10:07 -07:00
Puneeth Chaganti b016ab257e digest: Toggle display of digest weekday setting dynamically.
Hide or show the digest weekday setting, based on whether
`digest_emails_enabled` flag is turned on or off.
2019-05-08 14:39:12 -07:00
Yashashvi Dave 02a1b2e26b server events: Add realm-time-sync event to update stream privacy.
Add event to update stream settings whenever stream privacy is changed
accordingly.

Fixes #9470
2019-05-07 17:33:59 -07:00
Yashashvi Dave a68a06d94f static/js/stream_events: Modify function `update_property`.
This commits modifies function `update_property` to
update more one stream-sub-property in one event.
2019-05-07 17:33:59 -07:00
Vaibhav 1f76374e32 message_list_view: Move edited position conditionals to JS.
This adds three bools to message_container object which calculate bools
where the "(EDITED)" label should appear:

* `edited_in_left_col` -- when label appears in left column.
* `edited_alongside_sender` -- when label appears alongside sender info.
* `edited_status_msg` -- when label appears for a "/me" message.

We use the new bools and remove the complicated if else statements
from the templates for the "(EDITED)" label.

This also allows us to add a unit test to verify the logic.
2019-05-06 21:37:54 -07:00
Vishnu Ks 6c58603eaf support: Add support for scrubbing realm. 2019-05-06 20:12:54 -07:00
David Wood f53a8f8bb6 settings: Rename `create_stream_permission` in templates.
This commit renames the `create_stream_permission` field in the
templates to `create_stream_policy`, matching the field used in the
database model. This matches what `invite_to_stream_policy` does and
will be clearer when the `waiting_period_threshold` is split into its
own field.
2019-05-06 16:30:01 -07:00
David Wood 34d810aac3 settings: Migrate to create_stream_policy structure.
This commit replaces the `create_stream_by_admins_only` setting with a
new `create_stream_policy` setting, which mirroring the structure of
the existing `invite_to_stream_policy`.

This is important preparation for migrating the waiting period feature
to be its own independent setting.

Fixes #12236.
2019-05-06 16:27:55 -07:00
vinitS101 81b5a72252 admin_settings: Change maxfilesize to max_file_upload_size.
Renamed maxfilesize to max_file_upload_size for consistency.
Related to #12152.
2019-05-03 17:36:09 -07:00
Shubham Padia 884c19c3bc settings: NAME_CHANGES_DISABLED disables the name change button.
Fixes #12251.

Previously when disabling name changes in server settings, instead
of realm settings, the name edit button did not get disabled.
Changing name resulted in a message stating `no changes made`.
2019-05-03 12:58:51 -07:00
Shubham Padia 8c01f396db settings: Add AVATAR_CHANGES_DISABLED server setting.
Fixes #12132.
Realm setting to disable avatar changes is already present.
The `AVATAR_CHANGES_DISABLED` setting now follows the same
2-setting model as `NAME_CHANGES_DISABLED`.
2019-05-03 12:58:51 -07:00
Shubham Padia 4994a6c2c9 realm: Add setting to disable avatar changes.
This is useful when syncing avatars from an integrated LDAP/active
directory.

The upload avatar and delete avatar buttons are hidden if avatar
changes are disabled and the user is a non-admin.
If the user has a gravatar set, then the user will not be able to
upload an image as their avatar if avatar changes are disabled.

Part of #12132.
2019-05-03 12:52:43 -07:00
David Wood 272ed90685 settings: Create an explicit invite_to_stream_policy setting.
This commit creates a new organization setting that determines whether
a user can invite other users to streams. Previously this was linked
to the waiting period threshold, but this was both not documented and
overly limiting.

With significant tweaks by tabbott to change the database model to not
involve two threshhold fields, edit the tests, etc.

This requires follow-up work to make the create stream policy setting
work how this code implies it should.

Fixes #12042.
2019-04-29 17:11:28 -07:00
Puneeth Chaganti 0987ef8809 Merge pull request #12221 from punchagan/safari-upload-fix
Fixes couple of problems with image pastes on Safari
2019-04-28 22:51:17 -07:00