Commit Graph

187 Commits

Author SHA1 Message Date
Anders Kaseorg 5ba593f124 int_dict: Replace with Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 15:37:37 -08:00
Anders Kaseorg 2868b7c3e3 dict: Replace with Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 15:37:37 -08:00
Anders Kaseorg 7cc16757aa int_dict: Remove filter_values method.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 15:35:42 -08:00
Steve Howell e8de4abb0e markdown: Clean up userMentionHandler().
This mostly moves logic into people.js.
The people functions added here are glorified
two-liners.

One thing that changes here is that we
are a bit more rigorous about duplicate
names.

The code is slightly awkward, because this
commit preserves the strange behavior
that if 'alice|42' doesn't match on
the user with the name "alice" and user_id
"42", we instead look for a user whose
name is "alice|42".  That seems like a
misfeature to me, but there's a test for
it, so I want to check with Tim that it's not
intentional behavior before I simplify
the code.
2020-02-18 16:04:12 -08:00
Steve Howell 9ab07d1038 util.js: Remove util from window.
We now treat util like a leaf module and
use "require" to import it everywhere it's used.

An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that.  Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.

It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile.  There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.

The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.

I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util.  The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart.  Also, we
can continue to nibble away at the
cruftier parts of the module.
2020-02-15 12:20:20 -08:00
Tim Abbott 906160f1a3 presence: Re-introduce data filtering when offline.
This should return us to a situation where we won't get blueslip
browser error reporting for users created while a device was offline
just before it reloads.
2020-02-13 15:45:39 -08:00
Anders Kaseorg 62ad436c64 js: Convert _.compact(a) to a.filter(Boolean).
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg cdd774b790 js: Convert _.all(a, …), _.every(a, …) to a.every(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg 70ff164f89 js: Convert _.any(a, …), _.some(a, …) to a.some(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg ef50346a29 js: Convert _.reject(a, … => …) to a.filter(… => !…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg 4948240619 js: Convert _.filter(a, …) to a.filter(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg ac7b09d57e js: Convert _.map(a, …) to a.map(…).
And convert the corresponding function expressions to arrow style
while we’re here.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import K from "ast-types/gen/kinds";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;

  recast.visit(ast, {
    visitCallExpression(path) {
      const { callee, arguments: args } = path.node;
      if (
        n.MemberExpression.check(callee) &&
        !callee.computed &&
        n.Identifier.check(callee.object) &&
        callee.object.name === "_" &&
        n.Identifier.check(callee.property) &&
        callee.property.name === "map" &&
        args.length === 2 &&
        checkExpression(args[0]) &&
        checkExpression(args[1])
      ) {
        const [arr, fn] = args;
        path.replace(
          b.callExpression(b.memberExpression(arr, b.identifier("map")), [
            n.FunctionExpression.check(fn) ||
            n.ArrowFunctionExpression.check(fn)
              ? b.arrowFunctionExpression(
                  fn.params,
                  n.BlockStatement.check(fn.body) &&
                    fn.body.body.length === 1 &&
                    n.ReturnStatement.check(fn.body.body[0])
                    ? fn.body.body[0].argument || b.identifier("undefined")
                    : fn.body
                )
              : fn,
          ])
        );
        changed = true;
      }
      this.traverse(path);
    },
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg 719546641f js: Convert a.indexOf(…) !== -1 to a.includes(…).
Babel polyfills this for us for Internet Explorer.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import K from "ast-types/gen/kinds";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;

  recast.visit(ast, {
    visitBinaryExpression(path) {
      const { operator, left, right } = path.node;
      if (
        n.CallExpression.check(left) &&
        n.MemberExpression.check(left.callee) &&
        !left.callee.computed &&
        n.Identifier.check(left.callee.property) &&
        left.callee.property.name === "indexOf" &&
        left.arguments.length === 1 &&
        checkExpression(left.arguments[0]) &&
        ((["===", "!==", "==", "!=", ">", "<="].includes(operator) &&
          n.UnaryExpression.check(right) &&
          right.operator == "-" &&
          n.Literal.check(right.argument) &&
          right.argument.value === 1) ||
          ([">=", "<"].includes(operator) &&
            n.Literal.check(right) &&
            right.value === 0))
      ) {
        const test = b.callExpression(
          b.memberExpression(left.callee.object, b.identifier("includes")),
          [left.arguments[0]]
        );
        path.replace(
          ["!==", "!=", ">", ">="].includes(operator)
            ? test
            : b.unaryExpression("!", test)
        );
        changed = true;
      }
      this.traverse(path);
    },
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Steve Howell e9c6653852 node tests: Always enforce blueslip warn/error/fatal.
We now require all of our unit tests to handle
blueslip errors for warn/error/fatal.  This
simplifies the zblueslip code to not have any
options passed in.

Most of the places changed here fell into two
categories:

    - We were just missing a random piece of
      setup data in a happy path test.

    - We were testing error handling in just
      a lazy way to ensure 100% coverage.  Often
      these error codepaths were fairly
      contrived.

The one place where we especially lazy was
the stream_data tests, and those are now
more thorough.
2020-02-07 14:15:44 -08:00
Anders Kaseorg 02511bff1c js: Automatically convert _.each to for…of.
This commit was automatically generated by the following script,
followed by lint --fix and a few small manual lint-related cleanups.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import { Context } from "ast-types/lib/path-visitor";
import K from "ast-types/gen/kinds";
import { NodePath } from "ast-types/lib/node-path";
import assert from "assert";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);
const checkStatement = (node: n.Node): node is K.StatementKind =>
  n.Statement.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;
  let inLoop = false;
  let replaceReturn = false;

  const visitLoop = (...args: string[]) =>
    function(this: Context, path: NodePath) {
      for (const arg of args) {
        this.visit(path.get(arg));
      }
      const old = { inLoop };
      inLoop = true;
      this.visit(path.get("body"));
      inLoop = old.inLoop;
      return false;
    };

  recast.visit(ast, {
    visitDoWhileStatement: visitLoop("test"),

    visitExpressionStatement(path) {
      const { expression, comments } = path.node;
      let valueOnly;
      if (
        n.CallExpression.check(expression) &&
        n.MemberExpression.check(expression.callee) &&
        !expression.callee.computed &&
        n.Identifier.check(expression.callee.object) &&
        expression.callee.object.name === "_" &&
        n.Identifier.check(expression.callee.property) &&
        ["each", "forEach"].includes(expression.callee.property.name) &&
        [2, 3].includes(expression.arguments.length) &&
        checkExpression(expression.arguments[0]) &&
        (n.FunctionExpression.check(expression.arguments[1]) ||
          n.ArrowFunctionExpression.check(expression.arguments[1])) &&
        [1, 2].includes(expression.arguments[1].params.length) &&
        n.Identifier.check(expression.arguments[1].params[0]) &&
        ((valueOnly = expression.arguments[1].params[1] === undefined) ||
          n.Identifier.check(expression.arguments[1].params[1])) &&
        (expression.arguments[2] === undefined ||
          n.ThisExpression.check(expression.arguments[2]))
      ) {
        const old = { inLoop, replaceReturn };
        inLoop = false;
        replaceReturn = true;
        this.visit(
          path
            .get("expression")
            .get("arguments")
            .get(1)
            .get("body")
        );
        inLoop = old.inLoop;
        replaceReturn = old.replaceReturn;

        const [right, { body, params }] = expression.arguments;
        const loop = b.forOfStatement(
          b.variableDeclaration("let", [
            b.variableDeclarator(
              valueOnly ? params[0] : b.arrayPattern([params[1], params[0]])
            ),
          ]),
          valueOnly
            ? right
            : b.callExpression(
                b.memberExpression(right, b.identifier("entries")),
                []
              ),
          checkStatement(body) ? body : b.expressionStatement(body)
        );
        loop.comments = comments;
        path.replace(loop);
        changed = true;
      }
      this.traverse(path);
    },

    visitForStatement: visitLoop("init", "test", "update"),

    visitForInStatement: visitLoop("left", "right"),

    visitForOfStatement: visitLoop("left", "right"),

    visitFunction(path) {
      this.visit(path.get("params"));
      const old = { replaceReturn };
      replaceReturn = false;
      this.visit(path.get("body"));
      replaceReturn = old.replaceReturn;
      return false;
    },

    visitReturnStatement(path) {
      if (replaceReturn) {
        assert(!inLoop); // could use labeled continue if this ever fires
        const { argument, comments } = path.node;
        if (argument === null) {
          const s = b.continueStatement();
          s.comments = comments;
          path.replace(s);
        } else {
          const s = b.expressionStatement(argument);
          s.comments = comments;
          path.replace(s, b.continueStatement());
        }
        return false;
      }
      this.traverse(path);
    },

    visitWhileStatement: visitLoop("test"),
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg c48eb3d827 extract_people_from_message: Add missing default case.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Steve Howell b8f01f9cda people: Rename method to get_by_user_id().
This name is consistent with:

    get_by_email()
    get_by_name()
2020-02-05 12:04:56 -08:00
Anders Kaseorg de3146c137 js: Replace [...x] with Array.from(x).
Babel strict generates more code for [...x] than you’d like, while
Babel loose mode assumes x is an array.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 11:52:52 -08:00
Anders Kaseorg 9e1343ff8a dict, lazy_set: Return an iterator from values method.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg 45d3be5449 dict, lazy_set: Return an iterator from keys method.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg 60fac80c8e dict, lazy_set: Replace num_items method with size property.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg fe54e73c77 dict, lazy_set: Rename del method to delete, for consistency with Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Steve Howell d0453dc8f4 performance: Use startsWith in many places.
Using startsWith is faster than indexOf, especially for long strings
and short prefixes.  It's also a lot more readable.  The only reason
we weren't using it was when a lot of the code was originally written,
it wasn't available.
2020-01-28 12:47:37 -08:00
Steve Howell 4125eb28fd mobile sharing: Extract shared/js/typeahead.js.
This extracts get_emoji_matcher and all the
functions it depended on, most of which were
in composebox_typeahead.js.

We also move remove_diacritics out of the people
module.

This is the first major step for #13728.
2020-01-27 16:32:11 -08:00
Anders Kaseorg 1a07f7b158 js: Clean up user_id type confusion.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-01-16 13:23:47 -08:00
Steve Howell 8f35700da8 refactor: Extract get_message_people.
We'll use this in two places coming up, so it's
worth extracting, plus I wanted to add the
fairly lengthy comment here.  (Tim, feel free
to edit down the comment as you see fit).
2020-01-15 12:22:23 -08:00
Steve Howell 890a4b1247 refactor: Add filterer for user settings.
This change sets us up to optimize how we
filter users in the admin user settings.
See #13554 for more context on the user
facing issues.

This fix is basically three related things:

    - Add filterer options to list_render.
    - Add helper method to people.js.
    - Use filterer in settings_users.js.
2020-01-14 22:43:08 -08:00
Steve Howell 73d0350a24 people: Use ints in is_my_user_id().
This should be slightly more performant, and we
often call this function N times, such as when
rendering the buddy list.

There's a minor change to pm_list to avoid
an unnecessary computation on huddles that would
otherwise trigger a blueslip warning for the
huddles case.
2020-01-05 12:28:23 -08:00
Steve Howell 552f07428d people: Simplify people.get_recipient_count.
Once we get past the special check for fake
person objects already having `pm_recipient_count`,
we can rely on the object being a `person`
object with `user_id` set.
2020-01-05 12:27:45 -08:00
Steve Howell bc5589c2a7 people: Clean up recip.id code.
When we are pulling data from message.display_recipient
for private messages, the user_id field is always
called 'id', not 'user_id', so we can simplify
some defensive code.
2020-01-05 12:27:30 -08:00
Steve Howell 7630b859c3 js: Use IntDict in people.js.
This required lots of manual testing:

    - search/navigate user presence
    - send PM and mention user
    - pay attention to compose fade
    - send stream msg and mention user
    - open Private Messages in top-left and click
    - test unread counts
    - invite user who already has account
    - search for users in search bar
    - check user settings
        - User Groups
        - Users
        - Deactivated Users
        - Bots
    - create a bot
    - mention user groups
    - send group PM then click on lower right
    - view/edit/create streams

If there are still pieces of code that don't convert
ids to ints, the code should still work but report
blueslip errors.

I try to mostly convert user_ids to ints in the callers,
since often the callers are dealing with small amounts
of data, like user ids from huddles.
2020-01-05 12:27:28 -08:00
Steve Howell 26168eaa98 search: Optimize search bar suggestions for large realms.
We only ever show 3 or 4 people in search suggestions
(possibly w/a couple variations, like pm-with/sender/etc.),
so we can try to search a smaller subset of people
before going through the entire realm.

We use message_store.user_ids() for this, since you
typically want to search messages for people that
have sent messages recently, and we already sort
based on PM conversations.
2020-01-04 12:58:00 -08:00
Steve Howell d87c5d7b1f search: Use people.filter_all_persons() in search.
This should avoid some memory allocations.

We also use build_person_matcher to avoid
repeating the same logic over and over
again to process the query into termlets.

We also remove people.get_all_persons() and
people.person_matches_query().
2020-01-04 12:53:32 -08:00
Steve Howell d91a0ab9c7 typeahead: Remove diacritics on full names, not pieces.
This may actually be a slowdown for the worst case
scenario, but it sets us up to be able to easily
short circuit the removal of diacritic characters
for users that have pure ascii names.

For example, czo has lots of names like this:

    - Tim Abbott
    - Steve Howell

Since they're pure ascii, we can do a one-time
check.  A subsequent commit will show how we use
this.
2020-01-03 17:46:59 -08:00
Steve Howell ee3e488e02 js: Extract FoldDict class.
We have ~5 years of proof that we'll probably never
extend Dict with more options.

Breaking the classes into makes both a little faster
(no options to check), and we remove some options
in FoldDict that are never used (from/from_array).

A possible next step is to fine-tune the Dict to use
Map internally.

Note that the TypeScript types for FoldDict are now
more specific (requiring string keys).  Of course,
this isn't really enforced until we convert other
modules to TS.
2020-01-03 17:19:50 -08:00
Steve Howell 9cd075ffb1 people: Use Set() in track_duplicate_full_name().
This is more idiomatic and probably
faster for most browsers.  (This function
gets called for each name in page load,
so any slowness is magnified.)
2020-01-03 17:19:38 -08:00
Steve Howell 8b04cf1288 people: Use is_my_user_id in get_people_for_stream_create.
We want to get away from email-based checks.
2020-01-02 12:03:43 -08:00
Steve Howell 54cb857fee refactor: Rename people.get_rest_of_realm().
We want to mostly deprecate this function (see
the comment I added), so I gave it a more specific
name.

Ideally I'd just fix `stream_create`, but it does
use this function in a couple places, and it's helpful
to reuse the same sort here.  In one place stream_create
actually unshifts the "me" user back to the top of the
list, which makes sense for its use case.
2020-01-02 12:03:04 -08:00
Steve Howell 70470dea1c settings: Use correct email when searching users.
If we aren't showing users emails, then we don't
want to use emails in the search.

And if we are showing users emails, we want to
search on the email that's displayed to them.
For admins this will be delivery_email.

For regular users we arguably shouldn't search
on emails either, since it mostly causes confusion,
but this commit just preserves the current
behavior for those users (unless `show_email` is
false).
2019-12-30 09:43:24 -08:00
Steve Howell 3e4326afda refactor: Extract email_for_user_settings.
We want to be able to unit test this value,
since it's conditional on several factors:

    - am I an admin?
    - can non-admins view emails?
    - do we have delivery_email for the user?

I'm mocking show_email in the tests, since the
show_email code is in `settings_org` and
kind of hard to unit test.  It's not impossible,
but it's too much for this commit.  (Either
we need to extract it out to a nice file or
deal with mocking jQuery.  That module is
mostly data-oriented, so it would be nice
to have something like `settings_config` that
is actually pure data.)
2019-12-28 11:22:24 -08:00
Steve Howell 3a95be2f2f refactor: Extract matches_user_settings_search.
This was duplicate code.  I'm moving it to people
for pragmatic reasons--it's hard to unit test stuff
in settings_users.js due to all the jQuery.

It's also nice to have all people-related search
code in one place, just for auditing purposes.
2019-12-28 11:22:24 -08:00
Steve Howell 1df7a7280a Avoid unnecessary is_ascii checks on search termlets.
We now can call is_ascii only once per search termlet
when we are filtering multiple persons on the same
query.  (This requires the caller to use
`build_person_matcher` outside a loop or before
a `_.filter` call.)
2019-12-28 11:14:21 -08:00
Steve Howell 399e83aa70 minor: Tweak build_person_matcher.
This is not a major speedup, but we do a couple
simple things here:

    - trim the query outside the function we
      build (that might be called multiple times)

    - don't split names before we possibly
      early-exit with an email match
2019-12-28 11:14:21 -08:00
Steve Howell a718b47095 refactor: Speed up filter_people_by_search_terms.
We now call build_person_matcher outside the loop.
2019-12-28 11:14:21 -08:00
Steve Howell 9c525f8ecb refactor: Extract build_person_matcher().
This will allow use to change some O(N) behavior
to O(1) where we are performing the same query
on a bunch of people.  (Subsequent commits will
actually take advantage of this prefactoring.)
2019-12-28 11:14:21 -08:00
Anders Kaseorg b932525368 people: Use Unicode normalization for diacritic removal.
Fixes #13481.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-12-09 13:02:54 -08:00
Anders Kaseorg 28f3dfa284 js: Automatically convert var to let and const in most files.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`.  It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-03 12:42:39 -08:00
Anders Kaseorg d17b577d0c js: Purge useless IIFEs.
With webpack, variables declared in each file are already file-local
(Global variables need to be explicitly exported), so these IIFEs are
no longer needed.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-10-25 13:51:21 -07:00
Rohitt Vashishtha 6a08efc065 typeahead: Precompile regexes for removing diacritics.
Precompiling regexes gives a performance increase of around 10-15%
based on tests. See https://jsperf.com/typeahead-regex. This stacks
up when we have a lot of users in an organisation.
2019-09-19 12:27:01 -07:00
Priyank Patel 73b19672c3 message_fetch: Use user IDs for supported operators.
The approach taken here is basically use user IDs in operator that
support it when sending the request for fetching the messages
(see comments in code for more details).
2019-07-13 11:35:37 -07:00