Commit Graph

129 Commits

Author SHA1 Message Date
Ryan Rehman cfc87e3925 message list: Move the `FetchStatus` object to MessageListData class.
The reason for this change is that, this is where `Filter` and
actual tracking of what messages are contiguous lives. This
will be beneficial when we will to move to a model where we
cache `MessageListData` objects for a large number of views.
2020-06-02 15:45:39 -07:00
clarammdantas aae7c79c00 people.js: Rename add() to add_active_user(). 2020-05-26 21:41:54 -07:00
Rohitt Vashishtha 8bf407878d stream_data: Create realm_has_notifications_stream().
We use this as part of our effort to wrap the use of -1 for null/None
from the rest of the code.
2020-04-22 17:57:16 -07:00
Rohitt Vashishtha e79935dbf7 stream_data: Remove page_params.notifications_stream.
We shouldn't add redundant data to page_params. Since we already have
page_params.realm_notifications_stream_id, we can use that value instead
of creating page_params.notifications_stream.

We, however, still need the name of the notifications stream to render
it in templates. Thus we create stream_data.get_notifications_stream().
2020-04-22 17:57:16 -07:00
Rohitt Vashishtha c8b0627ffe zblueslip: Run blueslip.reset after each test.
This simplifies the blueslip interface to just declaring expected
errors and calling the code to test.
2020-04-20 08:17:20 -04:00
Rohitt Vashishtha 9fd645f9ee zblueslip: Remove redundant get_test_logs calls. 2020-04-20 08:17:20 -04:00
Rohitt Vashishtha 0def4a97ae zblueslip: Implement tracking extra/lesser blueslip calls.
We change the user facing interface to allow specifying expected
number of error messages (default=1). Now an average test can look
like:

```
    // We expect 3 error messages;
    blueslip.expect('error', 'an error message', 3);
    throwError();
    throwError();
    throwError();
    blueslip.reset();
```
2020-04-20 08:17:20 -04:00
Hashir Sarwar ee0d4541b4 topic_data: Rename `topic_data` module to `stream_topic_history`.
`stream_topic_history` is a more appropriate name as this
module will contain information about last message of a
stream in upcoming commits. Function and variable names
are changed accordingly like:

* topic_history() -> per_stream_history()
* get_recent_names() -> get_recent_topic_names()
* name -> topic_name
2020-04-16 20:11:04 -07:00
Steve Howell e64059de79 node tests: Remove most test_log length assertions.
For all the places where we just make zero or one
blueslip call, asserting for length is either
unnecessary or overkill.
2020-04-08 11:37:27 -04:00
Steve Howell 9943a07e8c node tests: Improve handling of blueslip.fatal().
We now use `assert.throws()` to test that we're
properly calling `blueslip.fatal`.

In order to not break line coverage here, we have
to remove an unreachable `return` in `stream_data.js`.

Usually we test `fatal` for line coverage reasons.
Most places where we use `blueslip.fatal` fall in
these categories:

    * the code is theoretically unreachable, but
      we have `blueslip.fatal` for defensive reasons

    * we have some upstream bug that we should just
      fix

    * the code should recover gracefully and just
      use blueslip.errors()

It's possible that we should eliminate `blueslip.fatal`
from our API and just throw errors when really important
invariants get broken.  This will make it more obvious
to somebody reading the code that we're not going to
continue after the call, and `blueslip` already knows
how to catch exceptions and report them.
2020-04-08 11:37:27 -04:00
Steve Howell f7b432afec node tests: Auto-include zblueslip for node tests.
We already use blueslip stubs in ~45 tests, so we
may as well just auto-include it.
2020-04-03 12:56:49 -04:00
Steve Howell df84c52a7f zblueslip: Change API to expect/reset.
The `set_test_data` never made complete sense to
me, since it wasn't really data that we were
setting.
2020-04-03 12:56:49 -04:00
Ryan Rehman c759083cb8 notification settings: Add helper function for stream notifications.
This function returns a list of objects to create a
list_render object, and each item contains the streams
whose atleast one notification setting differs from the
default set by the user.

This is done by comparing the global settings in the
`#settings/notifications` page with those settings
present in the subscribed streams.

Work towards #9228.
2020-03-31 15:36:59 -07:00
Steve Howell 2cd9c77979 refactor: Stop using page_params.realm_default_streams.
We now only use `page_params.realm_default_streams` during
initialization, and then after that we use `stream_data`
APIs to get default stream ids and related info.  (And
for the event that replace the data, we just update our
internal data structures as well.)

Long term we should have the server just send us ids here,
since we are now hydrating info from stream data in all places.
2020-03-25 17:11:25 -07:00
Steve Howell 6313917143 stream_data: Remove invite_streams().
This function was made obsolete in
b4e2313cbc.

It was replaced by get_invite_stream_data(),
which has plenty of coverage.
2020-03-25 17:11:25 -07:00
Steve Howell 4610ef3169 refactor: Remove stream_data.get_default_status().
There was no reason to have this when it's more
robust to look up streams by id than name.
2020-03-25 17:11:25 -07:00
Steve Howell 8d8226117c refactor: Extract get_default_stream_ids().
We only used get_default_stream_names() in a
test, so now it's being replaced with a function
that just gets ids.

We'll have use for get_default_streams_ids()
in an upcoming commit.
2020-03-25 17:11:25 -07:00
Steve Howell c86ccd8c6e tests: Remove obsolete page_params test setup.
This test code has basically been dead since
we started passing in `params` to
`stream_data.initialize()`, and now it's
more confusing than helpful.
2020-03-25 17:11:25 -07:00
Steve Howell f0c99b42ec Rename people.add_in_realm to people.add().
We had this API:

    people.add_in_realm = full-fledged user
    people.add = not necessarily in realm

Now the API is this:

    people.add = full-fledged user
    people._add_user = internal API for cross-realm bots
        and deactivated users

I think in most of our tests the distinction between
people.add() and people.add_in_realm() was just an
accident of history and didn't reflect any real intention.

And if I had to guess the intention in 99% of the cases,
folks probably thought they were just creating ordinary,
active users in the current realm.

In places where the distinction was obviously important
(because a test failed), I deactivated the user via
`people.deactivate`.

For the 'basics' test in the people test suite, I clean
up the test setup for Isaac.  Before this commit I was
adding him first as a non-realm user then as a full-fledged
user, but this was contrived and confusing, and we
didn't really need it for test coverage purposes.
2020-03-22 10:55:11 -07:00
Steve Howell 156ff33d22 stream edit: Extract stream_data.potential_subscribers().
We want to move more logic to stream_data to facilitate
testing.

Both before and after this commit, we essentially build a
new list of users for typeahead, but now the new list
excludes subscribed users.  We can do even better than
this in a follow-up commit.
2020-03-22 10:55:11 -07:00
Steve Howell b994889315 node tests: Just set i18n every time.
Explicitly stubbing i18n in 48 different files
is mostly busy work at this point, and it doesn't
provide much signal, since often it's invoked
only to satisfy transitive dependencies.
2020-02-28 17:11:24 -08:00
Steve Howell da79fd206a ui_init: Handle page_params more cleanly.
This cleans up the handoff of page_params
data between ui_init and modules that
take over ownership of page_params-derived
data.

Read the long comment in ui_init for a bit
more context.

Most of this diff is actually test cleanup.
And a lot of the diff to "real" code is
just glorified `s/page_params/params/`
in the `initialize` functions.

One little oddity is that we don't actually
surrender ownership of `page_params.user_id`
to `people.js`.  We could plausibly sweep
the rest of the codebase to just use
`people.my_user_id()` consistently, but it's
not a super high priority thing to fix,
since the value never changes.

The stream_data situation is a bit messy,
since we consume `page_params` data in the
initialize() function in addition to the
`params` data we "own".  I added a comment
there and intend to follow up.  I tried
to mostly avoid the "word soup" by extracting
three locals at the top.

Finally, I don't touch `alert_words` yet,
despite it also doing the delete-page-params-data
dance.  The problem is that `alert_words`
doesn't have a proper `initialize()`.  We
should clean that up and have it use a
`Map` internally, too.
2020-02-26 13:14:09 -08:00
Steve Howell 5e8279c2fb refactor: Extract settings_config.
This moves some code from settings_display.js
into the new module settings_config.js.

Extracting this module breaks some dependencies
on settings_display.js (which has some annoying
transitive dependencies, including jQuery).

In particular this isolates stream_data from
from settings_display.js.

Two of the three structures that we moved here
weren't even directly used by settings_display.js,
since we do a lot of rendering in the modules
admin.js and setting.js.

We make get_all_display_settings() a function
to avoid a require-time dependency on page_params.

Breaking the dependencies simplifies a few
node tests.

Most of the node test complexity came from the
following commit in March 2019:

5a130097bf

The commit itself seems harmless enough, but
dependencies can have a somewhat "viral" nature,
where making stream_data depend on settings_display
caused us to modify four different node tests.
2020-02-21 12:06:31 -08:00
Steve Howell 9ab07d1038 util.js: Remove util from window.
We now treat util like a leaf module and
use "require" to import it everywhere it's used.

An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that.  Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.

It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile.  There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.

The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.

I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util.  The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart.  Also, we
can continue to nibble away at the
cruftier parts of the module.
2020-02-15 12:20:20 -08:00
Anders Kaseorg 70ff164f89 js: Convert _.any(a, …), _.some(a, …) to a.some(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg 719546641f js: Convert a.indexOf(…) !== -1 to a.includes(…).
Babel polyfills this for us for Internet Explorer.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import K from "ast-types/gen/kinds";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;

  recast.visit(ast, {
    visitBinaryExpression(path) {
      const { operator, left, right } = path.node;
      if (
        n.CallExpression.check(left) &&
        n.MemberExpression.check(left.callee) &&
        !left.callee.computed &&
        n.Identifier.check(left.callee.property) &&
        left.callee.property.name === "indexOf" &&
        left.arguments.length === 1 &&
        checkExpression(left.arguments[0]) &&
        ((["===", "!==", "==", "!=", ">", "<="].includes(operator) &&
          n.UnaryExpression.check(right) &&
          right.operator == "-" &&
          n.Literal.check(right.argument) &&
          right.argument.value === 1) ||
          ([">=", "<"].includes(operator) &&
            n.Literal.check(right) &&
            right.value === 0))
      ) {
        const test = b.callExpression(
          b.memberExpression(left.callee.object, b.identifier("includes")),
          [left.arguments[0]]
        );
        path.replace(
          ["!==", "!=", ">", ">="].includes(operator)
            ? test
            : b.unaryExpression("!", test)
        );
        changed = true;
      }
      this.traverse(path);
    },
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Steve Howell fa1059aa2e stream_data: Remove stream_name param from add_sub().
We just get the stream_name from the sub struct now.

This mostly affects node tests.

The only place in real code where we called add_sub()
was when we initialized data from the server.
2020-02-09 22:08:50 -08:00
Steve Howell e9c6653852 node tests: Always enforce blueslip warn/error/fatal.
We now require all of our unit tests to handle
blueslip errors for warn/error/fatal.  This
simplifies the zblueslip code to not have any
options passed in.

Most of the places changed here fell into two
categories:

    - We were just missing a random piece of
      setup data in a happy path test.

    - We were testing error handling in just
      a lazy way to ensure 100% coverage.  Often
      these error codepaths were fairly
      contrived.

The one place where we especially lazy was
the stream_data tests, and those are now
more thorough.
2020-02-07 14:15:44 -08:00
Anders Kaseorg 02511bff1c js: Automatically convert _.each to for…of.
This commit was automatically generated by the following script,
followed by lint --fix and a few small manual lint-related cleanups.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import { Context } from "ast-types/lib/path-visitor";
import K from "ast-types/gen/kinds";
import { NodePath } from "ast-types/lib/node-path";
import assert from "assert";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);
const checkStatement = (node: n.Node): node is K.StatementKind =>
  n.Statement.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;
  let inLoop = false;
  let replaceReturn = false;

  const visitLoop = (...args: string[]) =>
    function(this: Context, path: NodePath) {
      for (const arg of args) {
        this.visit(path.get(arg));
      }
      const old = { inLoop };
      inLoop = true;
      this.visit(path.get("body"));
      inLoop = old.inLoop;
      return false;
    };

  recast.visit(ast, {
    visitDoWhileStatement: visitLoop("test"),

    visitExpressionStatement(path) {
      const { expression, comments } = path.node;
      let valueOnly;
      if (
        n.CallExpression.check(expression) &&
        n.MemberExpression.check(expression.callee) &&
        !expression.callee.computed &&
        n.Identifier.check(expression.callee.object) &&
        expression.callee.object.name === "_" &&
        n.Identifier.check(expression.callee.property) &&
        ["each", "forEach"].includes(expression.callee.property.name) &&
        [2, 3].includes(expression.arguments.length) &&
        checkExpression(expression.arguments[0]) &&
        (n.FunctionExpression.check(expression.arguments[1]) ||
          n.ArrowFunctionExpression.check(expression.arguments[1])) &&
        [1, 2].includes(expression.arguments[1].params.length) &&
        n.Identifier.check(expression.arguments[1].params[0]) &&
        ((valueOnly = expression.arguments[1].params[1] === undefined) ||
          n.Identifier.check(expression.arguments[1].params[1])) &&
        (expression.arguments[2] === undefined ||
          n.ThisExpression.check(expression.arguments[2]))
      ) {
        const old = { inLoop, replaceReturn };
        inLoop = false;
        replaceReturn = true;
        this.visit(
          path
            .get("expression")
            .get("arguments")
            .get(1)
            .get("body")
        );
        inLoop = old.inLoop;
        replaceReturn = old.replaceReturn;

        const [right, { body, params }] = expression.arguments;
        const loop = b.forOfStatement(
          b.variableDeclaration("let", [
            b.variableDeclarator(
              valueOnly ? params[0] : b.arrayPattern([params[1], params[0]])
            ),
          ]),
          valueOnly
            ? right
            : b.callExpression(
                b.memberExpression(right, b.identifier("entries")),
                []
              ),
          checkStatement(body) ? body : b.expressionStatement(body)
        );
        loop.comments = comments;
        path.replace(loop);
        changed = true;
      }
      this.traverse(path);
    },

    visitForStatement: visitLoop("init", "test", "update"),

    visitForInStatement: visitLoop("left", "right"),

    visitForOfStatement: visitLoop("left", "right"),

    visitFunction(path) {
      this.visit(path.get("params"));
      const old = { replaceReturn };
      replaceReturn = false;
      this.visit(path.get("body"));
      replaceReturn = old.replaceReturn;
      return false;
    },

    visitReturnStatement(path) {
      if (replaceReturn) {
        assert(!inLoop); // could use labeled continue if this ever fires
        const { argument, comments } = path.node;
        if (argument === null) {
          const s = b.continueStatement();
          s.comments = comments;
          path.replace(s);
        } else {
          const s = b.expressionStatement(argument);
          s.comments = comments;
          path.replace(s, b.continueStatement());
        }
        return false;
      }
      this.traverse(path);
    },

    visitWhileStatement: visitLoop("test"),
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Steve Howell b8f01f9cda people: Rename method to get_by_user_id().
This name is consistent with:

    get_by_email()
    get_by_name()
2020-02-05 12:04:56 -08:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Steve Howell c2af2c1fd1 refactor: Extract is_subscriber_subset().
Extracting the function makes it a bit easier to
test and use in a generic way.

Also, I wanted this to live in stream_data, so that
it's easier to find if we change how we model
subscriber data.

Finally, I use _.every to do the subset check
instead of `_.difference`, since _.difference
is actually N-squared:

  _.difference = restArguments(function(array, rest) {
    rest = flatten(rest, true, true);
    return _.filter(array, function(value){
      return !_.contains(rest, value);
    });
  });

And we don't actually want to build a list only
to check that it's zero vs. nonzero length.

We now do this, which short circuits as soon
as it finds any key that is only in sub1:

    return _.every(sub1.subscribers.keys(), (key) => {
        return sub2_set.has(key);
    });
2020-01-14 13:19:49 -08:00
Steve Howell 9f7be51ce8 streams: Replace Dict with IntDict in stream_data.
There's another Dict that we'll convert to a Set
in a subsequent commit.
2020-01-05 12:28:28 -08:00
Steve Howell e804f39f0e performance: Avoid expensive call in stream_data.is_active.
Calling `set_filter_out_inactives` is expensive, since we
count up the number of subscribed streams, which iterates
through all your streams, creates a new list of subscribed
streams, then counts them.

In my dev setup, I created 700 streams, and this shaved
about 700ms off of the initial call to `build_stream_list`.
2019-12-30 09:45:46 -08:00
Steve Howell 49cd719273 tests: Verify subscriptions more thoroughly.
When you subscribe/unsubscribe yourself, you want
to make sure these calls work too:

    subscribed_subs
    unsubscribed_subs
2019-12-28 07:29:51 -05:00
Tim Abbott fa7ae6fa7f node tests: Fix missing coverage on stream_data.js.
This fixes a testing coverage regression in
d5f005fd61.
2019-12-09 18:11:12 -08:00
Anders Kaseorg 28f3dfa284 js: Automatically convert var to let and const in most files.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`.  It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-03 12:42:39 -08:00
Yashashvi Dave e0a78af494 static/js/stream_data: Extract function `receives_notifications`. 2019-06-24 14:46:45 -07:00
Yashashvi Dave 8e269b4651 models: Rename notification to `enable_stream_audible_notifications`.
Rename notification property `enable_stream_sounds` to
`enable_stream_audible_notifications` to match with other
notification property patterns.

Fixes part of #12304
2019-06-12 16:24:51 -07:00
sahil839 5a130097bf settings: Add display setting for demoting inactive streams.
This adds a setting to control Zulip's default behavior of sorting to
bottom and graying out inactive streams.  The previous logic is still
the default "automatic", but this gives users more control.  See the
models.py comment for details.

Fixes #11524.
2019-06-03 23:07:56 -07:00
Yashashvi Dave 784d02bf60 static/js/stream_data: Rename `in_home_view` functions. 2019-05-30 21:39:06 -07:00
Yashashvi Dave 40f550038d subs: Replace all `in_home_view` uses with `is_muted` property.
Replace all uses of `in_home_view` subscription property
with `is_muted` property in frontend.

Fixes #12322
2019-05-30 21:39:06 -07:00
Yashashvi Dave 2598db4a77 streams: Hide unsubscribed streams in settings to guest users. 2019-05-13 17:27:32 -07:00
Harshit Bansal b553507412 subscriptions: Migrate notification setting defaults model.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.

Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.

The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side.  We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.

This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
2019-05-08 17:45:10 -07:00
Yashashvi Dave 02a1b2e26b server events: Add realm-time-sync event to update stream privacy.
Add event to update stream settings whenever stream privacy is changed
accordingly.

Fixes #9470
2019-05-07 17:33:59 -07:00
Rishi Gupta 52fb22277b onboarding: Remove get_newbie_stream from stream_data.js.
A relic of a past system removed in
9fbd990d1a. It is not being called
anywhere in the code.
2019-03-21 12:34:16 -07:00
Challa Venkata Raghava Reddy 8623a02d98 streams: Avoid showing more topics option when it's useless.
This makes the "more topics" option which appears below the list of
known topics in the left sidebar appear only when it's possible there
are actually more topics to be displayed.  Two specific cases it
resolves completely include:

* Newly created realms; this widget was a common source of confusion
  for new organization administrators.
* Newly created streams.

There are still some corner cases this doesn't handle, e.g. if you
just joined a private stream with protected history, but there isn't
as easy a fix for those.

Essentially rewritten by tabbott to fix code duplication and comment
extensively.

Fixes #10265.
2019-03-14 21:51:24 -07:00
Steve Howell 6fc45fd941 left sidebar: Disable "gray-out" feature for new users.
If a user has 30 subscribed streams or less, don't gray
them out if they haven't had recent activity.
2019-02-12 17:43:48 +00:00
Hemanth V. Alluri 683ec852fd stream_data: Use stream rendered_description provided by the backend.
Use the results of commit #73d26c8 to remove the method
`render_stream_description` in static/js/stream_data.js and instead
use the rendered_description attribute now being sent by the backend.

This will be a valuable optimization and a step towards removing the
need for the marked.js markdown parser and speeding up the client end.
2019-02-11 12:24:27 -08:00
Shubham Dhama fbd73ba637 stream_data: Add get_invite_stream_data helper function.
This function unlike `invite_streams()` returns an array of objects having
various info (name, stream_id, invite_only, default_stream) related to
streams rather than an array of names of streams.
2019-02-01 15:35:42 -08:00