Commit Graph

203 Commits

Author SHA1 Message Date
Anders Kaseorg e014ea966a eslint: Enable comma-dangle for functions.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 16:55:51 -07:00
Anders Kaseorg a79322bc94 eslint: Enable prefer-arrow-callback.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 16:55:50 -07:00
Anders Kaseorg b0253c5a2e eslint: Enable arrow-parens.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-03 16:53:39 -07:00
Steve Howell 2574efde3d urls: Support simple integer stream slugs.
Fixes #15402
2020-07-03 16:50:18 -07:00
sahil839 28b6af5c4b stream_data: Modify create_sub_from_server_data to use stream id.
This commit changes stream_data.create_sub_from_server_data to use
stream id, instead of stream name, for checking whether subscription
already exists or not. We are using stream ids so that we can avoid
bugs related to live update after stream rename.
2020-06-22 13:39:47 -07:00
sahil839 235601d3b6 stream_data: Modify remove_subscriber to use stream id.
This commit changes stream_data.remove_subscriber to use stream id
instead of stream name. We are using stream ids so that we can
avoid bugs related to live update after stream rename.
2020-06-22 13:39:47 -07:00
sahil839 ddbfbdc386 stream: Modify stream_data.add_subscriber to use stream_id.
Thsi commit changes stream_data.add_subscriber to use stream_id
instead of stream name. We are using stream ids so that we can
avoid bugs related to live update after stream rename.
2020-06-22 13:39:47 -07:00
sahil839 f3604fbb1e stream_edit: Refactor receives_notifications to use stream ids.
This commit changes receives_notifications function to use
stream_ids instead of stream names. We are using stream ids so
that we can avoid bugs related to live update after stream rename.
2020-06-22 13:39:47 -07:00
sahil839 d9b7228444 streams: Add frontend to set/update message_retention_days of a stream.
This commit adds frontend support for setting and updating message
retention days of a stream from stream settings.

Message retention days can be changed from stream privacy modal of the
stream and can be set from stream_creation_form while creating streams.

Only admins can create streams with message_retention_days value other
than realm_default.

This commit also contains relevant changes to docs.
2020-06-18 17:00:34 -07:00
sahil839 791e5de5de api: Remove is_old_stream property from the stream objects.
This commit removes is_old_stream property from the stream objects
returned by the API. This property was unnecessary and is essentially
equivalent to 'stream_weekly_traffic != null'.

We compute sub.is_old_stream in stream_data.update_calculated_fields
in frontend code and it is used to check whether we have a non-null
stream_weekly_traffic or not.

Fixes #15181.
2020-06-16 10:26:33 -07:00
Ryan Rehman 3dff6c0de7 subscriptions: Update wildcard mentions notify on stream create.
The `wildcard_mentions_notify` key was missing from the initial
sub data when a new stream was created. Thus `wildcard_mentions_notify`
was undefined and `wildcard_mentions_notify_display` was false.
(This key is used to render the data in the templates)

This caused a bug where the wildcard notifications was unchecked
in the stream personal settings and the newly created stream was
displayed in the stream specific notifications table.
2020-06-07 11:43:42 -07:00
Ryan Rehman cfc87e3925 message list: Move the `FetchStatus` object to MessageListData class.
The reason for this change is that, this is where `Filter` and
actual tracking of what messages are contiguous lives. This
will be beneficial when we will to move to a model where we
cache `MessageListData` objects for a large number of views.
2020-06-02 15:45:39 -07:00
Anders Kaseorg e5e1a05e74 stream_data: Convert orders from object to Map.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-26 23:33:40 -07:00
Rohitt Vashishtha 8bf407878d stream_data: Create realm_has_notifications_stream().
We use this as part of our effort to wrap the use of -1 for null/None
from the rest of the code.
2020-04-22 17:57:16 -07:00
Rohitt Vashishtha e79935dbf7 stream_data: Remove page_params.notifications_stream.
We shouldn't add redundant data to page_params. Since we already have
page_params.realm_notifications_stream_id, we can use that value instead
of creating page_params.notifications_stream.

We, however, still need the name of the notifications stream to render
it in templates. Thus we create stream_data.get_notifications_stream().
2020-04-22 17:57:16 -07:00
Roland Crosby ac7ec426b0 Add stream sorting widget to subscriptions page
This change adds a toggle widget to the "add streams" page that
lets the user change the sort order of the streams list. So far,
this supports sorting by stream name, by number of subscribers,
or by estimated weekly traffic.
2020-04-19 15:07:37 -04:00
Hashir Sarwar ee0d4541b4 topic_data: Rename `topic_data` module to `stream_topic_history`.
`stream_topic_history` is a more appropriate name as this
module will contain information about last message of a
stream in upcoming commits. Function and variable names
are changed accordingly like:

* topic_history() -> per_stream_history()
* get_recent_names() -> get_recent_topic_names()
* name -> topic_name
2020-04-16 20:11:04 -07:00
Steve Howell 9943a07e8c node tests: Improve handling of blueslip.fatal().
We now use `assert.throws()` to test that we're
properly calling `blueslip.fatal`.

In order to not break line coverage here, we have
to remove an unreachable `return` in `stream_data.js`.

Usually we test `fatal` for line coverage reasons.
Most places where we use `blueslip.fatal` fall in
these categories:

    * the code is theoretically unreachable, but
      we have `blueslip.fatal` for defensive reasons

    * we have some upstream bug that we should just
      fix

    * the code should recover gracefully and just
      use blueslip.errors()

It's possible that we should eliminate `blueslip.fatal`
from our API and just throw errors when really important
invariants get broken.  This will make it more obvious
to somebody reading the code that we're not going to
continue after the call, and `blueslip` already knows
how to catch exceptions and report them.
2020-04-08 11:37:27 -04:00
Ryan Rehman 58d06f3911 settings_config: Move Realm level notification settings.
We make `all_notifications` a function to avoid a
require-time dependency on page_params.
2020-04-01 14:40:41 -07:00
Ryan Rehman c759083cb8 notification settings: Add helper function for stream notifications.
This function returns a list of objects to create a
list_render object, and each item contains the streams
whose atleast one notification setting differs from the
default set by the user.

This is done by comparing the global settings in the
`#settings/notifications` page with those settings
present in the subscribed streams.

Work towards #9228.
2020-03-31 15:36:59 -07:00
Steve Howell 2cd9c77979 refactor: Stop using page_params.realm_default_streams.
We now only use `page_params.realm_default_streams` during
initialization, and then after that we use `stream_data`
APIs to get default stream ids and related info.  (And
for the event that replace the data, we just update our
internal data structures as well.)

Long term we should have the server just send us ids here,
since we are now hydrating info from stream data in all places.
2020-03-25 17:11:25 -07:00
Steve Howell ba495e57eb refactor: Simplify get_invite_stream_data().
This code is a bit simpler.

The previous code was concatenating two lists
and then removing duplicates by calling filter().

Now we just have two loops that append to a single
list, and the second loop detects duplicates
before inserting into the list.

We also now use `default_stream_ids` instead of
`page_params` data, which is convenient for two
reasons:

    - working with sets of ids is convenient
    - we don't need to maintain `page_params`
      data any more
2020-03-25 17:11:25 -07:00
Steve Howell 6313917143 stream_data: Remove invite_streams().
This function was made obsolete in
b4e2313cbc.

It was replaced by get_invite_stream_data(),
which has plenty of coverage.
2020-03-25 17:11:25 -07:00
Steve Howell 4610ef3169 refactor: Remove stream_data.get_default_status().
There was no reason to have this when it's more
robust to look up streams by id than name.
2020-03-25 17:11:25 -07:00
Steve Howell 8d8226117c refactor: Extract get_default_stream_ids().
We only used get_default_stream_names() in a
test, so now it's being replaced with a function
that just gets ids.

We'll have use for get_default_streams_ids()
in an upcoming commit.
2020-03-25 17:11:25 -07:00
Steve Howell 156ff33d22 stream edit: Extract stream_data.potential_subscribers().
We want to move more logic to stream_data to facilitate
testing.

Both before and after this commit, we essentially build a
new list of users for typeahead, but now the new list
excludes subscribed users.  We can do even better than
this in a follow-up commit.
2020-03-22 10:55:11 -07:00
Anders Kaseorg 71dd495214 js: Convert _.pluck(a, "key") to item => item.key.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-27 17:20:34 -08:00
Steve Howell da79fd206a ui_init: Handle page_params more cleanly.
This cleans up the handoff of page_params
data between ui_init and modules that
take over ownership of page_params-derived
data.

Read the long comment in ui_init for a bit
more context.

Most of this diff is actually test cleanup.
And a lot of the diff to "real" code is
just glorified `s/page_params/params/`
in the `initialize` functions.

One little oddity is that we don't actually
surrender ownership of `page_params.user_id`
to `people.js`.  We could plausibly sweep
the rest of the codebase to just use
`people.my_user_id()` consistently, but it's
not a super high priority thing to fix,
since the value never changes.

The stream_data situation is a bit messy,
since we consume `page_params` data in the
initialize() function in addition to the
`params` data we "own".  I added a comment
there and intend to follow up.  I tried
to mostly avoid the "word soup" by extracting
three locals at the top.

Finally, I don't touch `alert_words` yet,
despite it also doing the delete-page-params-data
dance.  The problem is that `alert_words`
doesn't have a proper `initialize()`.  We
should clean that up and have it use a
`Map` internally, too.
2020-02-26 13:14:09 -08:00
Anders Kaseorg 5ba593f124 int_dict: Replace with Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 15:37:37 -08:00
Anders Kaseorg c9dbd13189 js: Convert _.has to Object.prototype.hasOwnProperty.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:09:39 -08:00
Anders Kaseorg fe082248cc js: Convert _.defaults to spread syntax.
This is not always a behavior-preserving translation: _.defaults
mutates its first argument.  However, the code does not always appear
to have been written to expect that.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:09:39 -08:00
Steve Howell 5e8279c2fb refactor: Extract settings_config.
This moves some code from settings_display.js
into the new module settings_config.js.

Extracting this module breaks some dependencies
on settings_display.js (which has some annoying
transitive dependencies, including jQuery).

In particular this isolates stream_data from
from settings_display.js.

Two of the three structures that we moved here
weren't even directly used by settings_display.js,
since we do a lot of rendering in the modules
admin.js and setting.js.

We make get_all_display_settings() a function
to avoid a require-time dependency on page_params.

Breaking the dependencies simplifies a few
node tests.

Most of the node test complexity came from the
following commit in March 2019:

5a130097bf

The commit itself seems harmless enough, but
dependencies can have a somewhat "viral" nature,
where making stream_data depend on settings_display
caused us to modify four different node tests.
2020-02-21 12:06:31 -08:00
Steve Howell 9ab07d1038 util.js: Remove util from window.
We now treat util like a leaf module and
use "require" to import it everywhere it's used.

An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that.  Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.

It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile.  There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.

The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.

I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util.  The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart.  Also, we
can continue to nibble away at the
cruftier parts of the module.
2020-02-15 12:20:20 -08:00
Anders Kaseorg fd89f3a695 stream_data: Convert is_included from object to Set.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-12 10:39:01 -08:00
Anders Kaseorg cdd774b790 js: Convert _.all(a, …), _.every(a, …) to a.every(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg ef50346a29 js: Convert _.reject(a, … => …) to a.filter(… => !…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg 4948240619 js: Convert _.filter(a, …) to a.filter(…).
And convert the corresponding function expressions to arrow style
while we’re here.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Anders Kaseorg ac7b09d57e js: Convert _.map(a, …) to a.map(…).
And convert the corresponding function expressions to arrow style
while we’re here.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import K from "ast-types/gen/kinds";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;

  recast.visit(ast, {
    visitCallExpression(path) {
      const { callee, arguments: args } = path.node;
      if (
        n.MemberExpression.check(callee) &&
        !callee.computed &&
        n.Identifier.check(callee.object) &&
        callee.object.name === "_" &&
        n.Identifier.check(callee.property) &&
        callee.property.name === "map" &&
        args.length === 2 &&
        checkExpression(args[0]) &&
        checkExpression(args[1])
      ) {
        const [arr, fn] = args;
        path.replace(
          b.callExpression(b.memberExpression(arr, b.identifier("map")), [
            n.FunctionExpression.check(fn) ||
            n.ArrowFunctionExpression.check(fn)
              ? b.arrowFunctionExpression(
                  fn.params,
                  n.BlockStatement.check(fn.body) &&
                    fn.body.body.length === 1 &&
                    n.ReturnStatement.check(fn.body.body[0])
                    ? fn.body.body[0].argument || b.identifier("undefined")
                    : fn.body
                )
              : fn,
          ])
        );
        changed = true;
      }
      this.traverse(path);
    },
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-10 14:08:12 -08:00
Steve Howell fa1059aa2e stream_data: Remove stream_name param from add_sub().
We just get the stream_name from the sub struct now.

This mostly affects node tests.

The only place in real code where we called add_sub()
was when we initialized data from the server.
2020-02-09 22:08:50 -08:00
Anders Kaseorg 02511bff1c js: Automatically convert _.each to for…of.
This commit was automatically generated by the following script,
followed by lint --fix and a few small manual lint-related cleanups.

import * as babelParser from "recast/parsers/babel";
import * as recast from "recast";
import * as tsParser from "recast/parsers/typescript";
import { builders as b, namedTypes as n } from "ast-types";
import { Context } from "ast-types/lib/path-visitor";
import K from "ast-types/gen/kinds";
import { NodePath } from "ast-types/lib/node-path";
import assert from "assert";
import fs from "fs";
import path from "path";
import process from "process";

const checkExpression = (node: n.Node): node is K.ExpressionKind =>
  n.Expression.check(node);
const checkStatement = (node: n.Node): node is K.StatementKind =>
  n.Statement.check(node);

for (const file of process.argv.slice(2)) {
  console.log("Parsing", file);
  const ast = recast.parse(fs.readFileSync(file, { encoding: "utf8" }), {
    parser: path.extname(file) === ".ts" ? tsParser : babelParser,
  });
  let changed = false;
  let inLoop = false;
  let replaceReturn = false;

  const visitLoop = (...args: string[]) =>
    function(this: Context, path: NodePath) {
      for (const arg of args) {
        this.visit(path.get(arg));
      }
      const old = { inLoop };
      inLoop = true;
      this.visit(path.get("body"));
      inLoop = old.inLoop;
      return false;
    };

  recast.visit(ast, {
    visitDoWhileStatement: visitLoop("test"),

    visitExpressionStatement(path) {
      const { expression, comments } = path.node;
      let valueOnly;
      if (
        n.CallExpression.check(expression) &&
        n.MemberExpression.check(expression.callee) &&
        !expression.callee.computed &&
        n.Identifier.check(expression.callee.object) &&
        expression.callee.object.name === "_" &&
        n.Identifier.check(expression.callee.property) &&
        ["each", "forEach"].includes(expression.callee.property.name) &&
        [2, 3].includes(expression.arguments.length) &&
        checkExpression(expression.arguments[0]) &&
        (n.FunctionExpression.check(expression.arguments[1]) ||
          n.ArrowFunctionExpression.check(expression.arguments[1])) &&
        [1, 2].includes(expression.arguments[1].params.length) &&
        n.Identifier.check(expression.arguments[1].params[0]) &&
        ((valueOnly = expression.arguments[1].params[1] === undefined) ||
          n.Identifier.check(expression.arguments[1].params[1])) &&
        (expression.arguments[2] === undefined ||
          n.ThisExpression.check(expression.arguments[2]))
      ) {
        const old = { inLoop, replaceReturn };
        inLoop = false;
        replaceReturn = true;
        this.visit(
          path
            .get("expression")
            .get("arguments")
            .get(1)
            .get("body")
        );
        inLoop = old.inLoop;
        replaceReturn = old.replaceReturn;

        const [right, { body, params }] = expression.arguments;
        const loop = b.forOfStatement(
          b.variableDeclaration("let", [
            b.variableDeclarator(
              valueOnly ? params[0] : b.arrayPattern([params[1], params[0]])
            ),
          ]),
          valueOnly
            ? right
            : b.callExpression(
                b.memberExpression(right, b.identifier("entries")),
                []
              ),
          checkStatement(body) ? body : b.expressionStatement(body)
        );
        loop.comments = comments;
        path.replace(loop);
        changed = true;
      }
      this.traverse(path);
    },

    visitForStatement: visitLoop("init", "test", "update"),

    visitForInStatement: visitLoop("left", "right"),

    visitForOfStatement: visitLoop("left", "right"),

    visitFunction(path) {
      this.visit(path.get("params"));
      const old = { replaceReturn };
      replaceReturn = false;
      this.visit(path.get("body"));
      replaceReturn = old.replaceReturn;
      return false;
    },

    visitReturnStatement(path) {
      if (replaceReturn) {
        assert(!inLoop); // could use labeled continue if this ever fires
        const { argument, comments } = path.node;
        if (argument === null) {
          const s = b.continueStatement();
          s.comments = comments;
          path.replace(s);
        } else {
          const s = b.expressionStatement(argument);
          s.comments = comments;
          path.replace(s, b.continueStatement());
        }
        return false;
      }
      this.traverse(path);
    },

    visitWhileStatement: visitLoop("test"),
  });

  if (changed) {
    console.log("Writing", file);
    fs.writeFileSync(file, recast.print(ast).code, { encoding: "utf8" });
  }
}

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-07 14:09:47 -08:00
Steve Howell b8f01f9cda people: Rename method to get_by_user_id().
This name is consistent with:

    get_by_email()
    get_by_name()
2020-02-05 12:04:56 -08:00
Anders Kaseorg de3146c137 js: Replace [...x] with Array.from(x).
Babel strict generates more code for [...x] than you’d like, while
Babel loose mode assumes x is an array.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 11:52:52 -08:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Anders Kaseorg 9e1343ff8a dict, lazy_set: Return an iterator from values method.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg 45d3be5449 dict, lazy_set: Return an iterator from keys method.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg 60fac80c8e dict, lazy_set: Replace num_items method with size property.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg da633e953e lazy_set: Convert LazySet to a real class.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Anders Kaseorg fe54e73c77 dict, lazy_set: Rename del method to delete, for consistency with Map.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-04 12:22:03 -08:00
Steve Howell c2af2c1fd1 refactor: Extract is_subscriber_subset().
Extracting the function makes it a bit easier to
test and use in a generic way.

Also, I wanted this to live in stream_data, so that
it's easier to find if we change how we model
subscriber data.

Finally, I use _.every to do the subset check
instead of `_.difference`, since _.difference
is actually N-squared:

  _.difference = restArguments(function(array, rest) {
    rest = flatten(rest, true, true);
    return _.filter(array, function(value){
      return !_.contains(rest, value);
    });
  });

And we don't actually want to build a list only
to check that it's zero vs. nonzero length.

We now do this, which short circuits as soon
as it finds any key that is only in sub1:

    return _.every(sub1.subscribers.keys(), (key) => {
        return sub2_set.has(key);
    });
2020-01-14 13:19:49 -08:00
Steve Howell 579bad4829 refactor: Use Set for default_stream_ids. 2020-01-05 12:28:33 -08:00