Commit Graph

34675 Commits

Author SHA1 Message Date
Anders Kaseorg 56ab5b4cf4 schema: Iterate over fields with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Hashir Sarwar dcbd3e486f stream_subscription: Remove unused TypedDict `SubInfo`. 2020-02-10 14:04:22 -08:00
Chris Heald 86faebc2e5 lightbox: Don't blow up for messages not in the message store.
This should somewhat reduce the gravity of the failure mode for cases
where the message the user clicked cannot be found (which would be a
significant bug on its own merit in any case).
2020-02-10 14:00:40 -08:00
Chris Heald fb09bd5df6 node: Fix lightbox tests to use message_store.get().
While there are still a number of problems with these tests, they now
more effectively exercise the real code.
2020-02-10 14:00:10 -08:00
Chris Heald c802a1393f lightbox: Cast zid to an int before looking it up in message_store.
The keys for message_store are since the recent Map migration intended
to be integer message IDs, not strings (and likely were always
intended to be integers; the failure mode may simply have shifted).

This may just be a new bug, but this max also fix #9549; certainly
we'll want to redo any investigation with this fix in place.

Fixes #9549.
2020-02-10 13:56:49 -08:00
Steve Howell fa1059aa2e stream_data: Remove stream_name param from add_sub().
We just get the stream_name from the sub struct now.

This mostly affects node tests.

The only place in real code where we called add_sub()
was when we initialized data from the server.
2020-02-09 22:08:50 -08:00
Steve Howell 3af8dc6c7d node tests: Add some color to output. 2020-02-09 22:07:51 -08:00
Steve Howell e1977f4680 node tests: Use EXEMPT_FILES for line coverage.
We now have 100% line coverage on 71 JS files.
This is thanks to about 150 people who have
contributed code to frontend/node_tests.

And then 126 files are still short of 100% line
coverage.

We now enforce line coverage with a set called
EXEMPT_FILES, which are the files for which
we do NOT expect to have 100% coverage.

Using an exemption list makes it so that adding
a new JS file to the project without 100% line
coverage will cause the build to fail.  This will
encourage folks to be intentional about their
lack of test coverage.
2020-02-09 22:07:51 -08:00
Steve Howell fe48ede9cf node tests: Fix bug with zero-coverage case.
If a file that had 100% coverage somehow regressed
to 0% coverage, we would report an error to the
console, but we weren't treating it as an actual
failure.

We've probably always had this bug, but it probably
rarely was an issue, since devs might have seen
the error locally, or hopefully whatever crazy
thing you did to totally remove coverage would
have had other symptoms.

If this was intentional, I suspect it might have
had something to do with wanting to get coverage
reports when you just run individual tests.  But
a while back we changed it so that when you run
individual tests, we don't do the line coverage
enforcement.
2020-02-09 22:07:51 -08:00
Steve Howell 2ff41bf9e5 /json/users: Use field.realm for realm lookup.
This avoids an unnecessary join to UserProfile.

To verify this, you can do `print(queries)` in the
`test_get_custom_profile_fields_from_api` test.  It's
kinda noisy, so I excerpted them below...

Before:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_userprofile" ON ("zerver_customprofilefieldvalue"."user_profile_id" = "zerver_userprofile"."id")
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_userprofile"."realm_id" = 2

After:

    SELECT ...
    FROM "zerver_customprofilefieldvalue"
    INNER JOIN "zerver_customprofilefield" ON ("zerver_customprofilefieldvalue"."field_id" = "zerver_customprofilefield"."id")
    WHERE "zerver_customprofilefield"."realm_id" = 2'

I don't have any way to measure the two queries with
realistic data, but I would assume the second
query is significantly faster on most of our instances,
since CustomProfileField should be tiny.
2020-02-09 22:04:02 -08:00
Steve Howell 9303c386b8 tests: Count queries for /json/users.
I am trying to optimize a query in this endpoint.
I don't think I'll actually reduce the number of
queries, but I wanted to capture the query and
this was the easiest way to do it, so might as
well check in the code! :)
2020-02-09 22:04:02 -08:00
Steve Howell 01f180d042 minor: Remove unused line of code in get_raw_user_data().
The line removed here is a noop, as both sides of the
immediately following conditional reassign the
same variable.

This harmless cruft was the result of the recent commit
1ae5964ab8, which added
support for single-user GETs.
2020-02-09 22:04:02 -08:00
Anders Kaseorg 591856713e package.json: Set "private": true to prevent NPM publication.
https://docs.npmjs.com/files/package.json#private

Although our “package” is named zulip, it is not
https://www.npmjs.com/package/zulip and should not be published there.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-09 22:02:49 -08:00
Tim Abbott 986706c7e5 tornado: Use common code for copying headers.
This fixes a bug where our asynchronous requests were only copying the
Content-Type header (i.e. the one case where we're noticed) from the
Django HttpResponse.  I'm not sure what the impact of this would be;
the rate-limiting headers rarely come up when breaking a long-polled
request.  But it seems clearly an improvement to do this in a
consistent fashion.

Only the headers piece is a change; in Tornado

    self.finish(x)

is equivalent to:

    self.write(x)
    self.finish()
2020-02-07 16:14:19 -08:00
Tim Abbott 224a73a3ec tornado: Extract a function for writing Tornado responses.
This increases the readability of what's happening in our core Tornado
handlers code, as well as making this logic reusable.
2020-02-07 16:13:49 -08:00
Tim Abbott 5305e8af85 tornado: Extract convert_tornado_request_to_django_request. 2020-02-07 16:03:58 -08:00
Tim Abbott fc58ae117a handlers: Rename confusingly named response to result_dict.
This should somewhat increase the readability of zulip_finish.
2020-02-07 16:03:58 -08:00
Vishnu KS 4572be8c27 api: Rename subject_links to topic_links.
Fixes #13588
2020-02-07 14:35:22 -08:00
Steve Howell e9c6653852 node tests: Always enforce blueslip warn/error/fatal.
We now require all of our unit tests to handle
blueslip errors for warn/error/fatal.  This
simplifies the zblueslip code to not have any
options passed in.

Most of the places changed here fell into two
categories:

    - We were just missing a random piece of
      setup data in a happy path test.

    - We were testing error handling in just
      a lazy way to ensure 100% coverage.  Often
      these error codepaths were fairly
      contrived.

The one place where we especially lazy was
the stream_data tests, and those are now
more thorough.
2020-02-07 14:15:44 -08:00
Steve Howell 996d054fe9 messages: Send stream_id for stream messages.
This saves a tiny bit of bandwidth, but more
importantly, it protects us against races for
stream name changes.  There's some argument that
if the user is thinking they're sending to
old_stream_name, and unbeknownst to them, the
stream has changed to new_stream_name, then we
should fail.  But I think 99% of the time the
user just wants the message to go that stream
despite any renames.

In order to verify the blueslip error, we
had to turn on error checking, which required
a tiny fix to a place where we left out
a stream_id for add_sub.
2020-02-07 14:15:44 -08:00
Anders Kaseorg 02511bff1c js: Automatically convert _.each to for…of.
This commit was automatically generated by the following script,
followed by lint --fix and a few small manual lint-related cleanups.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import { Context } from "ast-types/lib/path-visitor";
import K from "ast-types/gen/kinds";
import { NodePath } from "ast-types/lib/node-path";
import assert from "assert";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);
const checkStatement = (node: n.Node): node is K.StatementKind =>
  n.Statement.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;
  let inLoop = false;
  let replaceReturn = false;

  const visitLoop = (...args: string[]) =>
    function(this: Context, path: NodePath) {
      for (const arg of args) {
        this.visit(path.get(arg));
      }
      const old = { inLoop };
      inLoop = true;
      this.visit(path.get("body"));
      inLoop = old.inLoop;
      return false;
    };

  recast.visit(ast, {
    visitDoWhileStatement: visitLoop("test"),

    visitExpressionStatement(path) {
      const { expression, comments } = path.node;
      let valueOnly;
      if (
        n.CallExpression.check(expression) &&
        n.MemberExpression.check(expression.callee) &&
        !expression.callee.computed &&
        n.Identifier.check(expression.callee.object) &&
        expression.callee.object.name === "_" &&
        n.Identifier.check(expression.callee.property) &&
        ["each", "forEach"].includes(expression.callee.property.name) &&
        [2, 3].includes(expression.arguments.length) &&
        checkExpression(expression.arguments[0]) &&
        (n.FunctionExpression.check(expression.arguments[1]) ||
          n.ArrowFunctionExpression.check(expression.arguments[1])) &&
        [1, 2].includes(expression.arguments[1].params.length) &&
        n.Identifier.check(expression.arguments[1].params[0]) &&
        ((valueOnly = expression.arguments[1].params[1] === undefined) ||
          n.Identifier.check(expression.arguments[1].params[1])) &&
        (expression.arguments[2] === undefined ||
          n.ThisExpression.check(expression.arguments[2]))
      ) {
        const old = { inLoop, replaceReturn };
        inLoop = false;
        replaceReturn = true;
        this.visit(
          path
            .get("expression")
            .get("arguments")
            .get(1)
            .get("body")
        );
        inLoop = old.inLoop;
        replaceReturn = old.replaceReturn;

        const [right, { body, params }] = expression.arguments;
        const loop = b.forOfStatement(
          b.variableDeclaration("let", [
            b.variableDeclarator(
              valueOnly ? params[0] : b.arrayPattern([params[1], params[0]])
            ),
          ]),
          valueOnly
            ? right
            : b.callExpression(
                b.memberExpression(right, b.identifier("entries")),
                []
              ),
          checkStatement(body) ? body : b.expressionStatement(body)
        );
        loop.comments = comments;
        path.replace(loop);
        changed = true;
      }
      this.traverse(path);
    },

    visitForStatement: visitLoop("init", "test", "update"),

    visitForInStatement: visitLoop("left", "right"),

    visitForOfStatement: visitLoop("left", "right"),

    visitFunction(path) {
      this.visit(path.get("params"));
      const old = { replaceReturn };
      replaceReturn = false;
      this.visit(path.get("body"));
      replaceReturn = old.replaceReturn;
      return false;
    },

    visitReturnStatement(path) {
      if (replaceReturn) {
        assert(!inLoop); // could use labeled continue if this ever fires
        const { argument, comments } = path.node;
        if (argument === null) {
          const s = b.continueStatement();
          s.comments = comments;
          path.replace(s);
        } else {
          const s = b.expressionStatement(argument);
          s.comments = comments;
          path.replace(s, b.continueStatement());
        }
        return false;
      }
      this.traverse(path);
    },

    visitWhileStatement: visitLoop("test"),
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 811e128787 check-openapi: Fix lint errors and remove lint exclusion.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg b3f63262af lint: Delegate console.log check to ESLint.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 0c8d199a3d tests: Mock empty lists correctly in page_params.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-02-07 14:09:47 -08:00
Anders Kaseorg ece4d9344a tests: Add missing options argument to poll_data_holder.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg c48eb3d827 extract_people_from_message: Add missing default case.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 19f7c6f012 zjsunit: Replace add_extensions with Object.assign.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 940ff9e95f zjsunit: Use modern spread arguments syntax.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg dc742a5629 tests: Convert sub_row_data from object to array.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg f912572887 settings_profile_fields: Iterate over field_data with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg dd10108c24 user_status: Iterate over page_params.user_status with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 5564b39ea4 tests: Iterate over page_param_checkbox_options with Object.keys.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 91d3d5d9df settings_org: Iterate over auth_methods with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg a9f5730270 drafts: Iterate over drafts with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 0d05decbe4 presence: Iterate over presence info with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 10edd2d0a1 server_events_dispatch: Iterate over event data with Object.entries.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 3b4dc2d8d8 tests: Iterate over tabs with Object.values.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 8bb515dbd9 emoji: Iterate over emoji_codes data with Object.{entries,values}.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Anders Kaseorg 5de013b11e emoji: Iterate over realm_emoji with Object.values.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Tim Abbott 84edb5c516 test_fixtures: Fix buggy reuse of status_dir between databases.
Apparently, the arguments passed to template_database_status were
incorrect for the manual testing development database, in that we
didn't pass a status_dir when calling into that code from provision.

The result was that provisioning before running `test-backend` would
ignore changes to the list of check_files (etc.) made after rebasing,
and vice versa.

The cleanest fix is to compute status_dir from other values passed in;
I'm also going to open a follow-up issue for creating a better overall
interface here.
2020-02-07 13:33:08 -08:00
akashaviator 1ae5964ab8 api: Add an api endpoint for GET /users/{id}
This adds a new API endpoint for querying basic data on a single other
user in the organization, reusing the existing infrastructure (and
view function!) for getting data on all users in an organization.

Fixes #12277.
2020-02-07 10:36:31 -08:00
Tim Abbott e39840c705 users: Add read-only mode for access_user_by_id.
We've be using this in the upcoming GET /users/{id} method.
2020-02-07 10:36:31 -08:00
Tim Abbott aa9286a1f9 users: Move query into caller of get_custom_profile_field_values.
This will be useful for supporting a smaller query for a single user.
2020-02-07 10:36:31 -08:00
Tim Abbott 79e5dd1374 users: Rename get_raw_user_data user parameter to acting_user.
This is for improved clarity as we extend this function to take
multiple user objects.
2020-02-07 10:36:31 -08:00
Anders Kaseorg 07602c4aac message_edit: Convert currently_echoing_messages from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00
Anders Kaseorg b8c8ba544d message_edit: Convert currently_editing_messages from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00
Anders Kaseorg 52a8449a0e widgetize: Convert widget_contents from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00
Anders Kaseorg 56436188a4 widgetize: Convert widgets from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00
Anders Kaseorg 88c73602e0 vdom: Convert new_dict, old_dict from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00
Anders Kaseorg 737efd1fac presence: Convert presence_info from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-06 17:24:43 -08:00