We now treat util like a leaf module and
use "require" to import it everywhere it's used.
An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that. Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.
It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile. There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.
The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.
I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util. The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart. Also, we
can continue to nibble away at the
cruftier parts of the module.
We just get the stream_name from the sub struct now.
This mostly affects node tests.
The only place in real code where we called add_sub()
was when we initialized data from the server.
We now require all of our unit tests to handle
blueslip errors for warn/error/fatal. This
simplifies the zblueslip code to not have any
options passed in.
Most of the places changed here fell into two
categories:
- We were just missing a random piece of
setup data in a happy path test.
- We were testing error handling in just
a lazy way to ensure 100% coverage. Often
these error codepaths were fairly
contrived.
The one place where we especially lazy was
the stream_data tests, and those are now
more thorough.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.
It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.
This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.
With many tweaks by tabbott to documentation under /help, etc.
Fixes#13616.
Extracting the function makes it a bit easier to
test and use in a generic way.
Also, I wanted this to live in stream_data, so that
it's easier to find if we change how we model
subscriber data.
Finally, I use _.every to do the subset check
instead of `_.difference`, since _.difference
is actually N-squared:
_.difference = restArguments(function(array, rest) {
rest = flatten(rest, true, true);
return _.filter(array, function(value){
return !_.contains(rest, value);
});
});
And we don't actually want to build a list only
to check that it's zero vs. nonzero length.
We now do this, which short circuits as soon
as it finds any key that is only in sub1:
return _.every(sub1.subscribers.keys(), (key) => {
return sub2_set.has(key);
});
Calling `set_filter_out_inactives` is expensive, since we
count up the number of subscribed streams, which iterates
through all your streams, creates a new list of subscribed
streams, then counts them.
In my dev setup, I created 700 streams, and this shaved
about 700ms off of the initial call to `build_stream_list`.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`. It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
Rename notification property `enable_stream_sounds` to
`enable_stream_audible_notifications` to match with other
notification property patterns.
Fixes part of #12304
This adds a setting to control Zulip's default behavior of sorting to
bottom and graying out inactive streams. The previous logic is still
the default "automatic", but this gives users more control. See the
models.py comment for details.
Fixes#11524.
This commit migrates the Subscription's notification fields from a
BooleanField to a NullBooleanField where a value of None means to
inherit the value from user's profile.
Also includes a migrations to set the corresponding settings to None
if they match the user profile's values. This migration helps us in
getting rid of the weird "Apply to all" widget that we offered on
subscription settings page.
The mobile apps can't handle None appearing as the stream-level
notification settings, so for backwards-compatibility we arrange to
only send True/False to the mobile apps by applying those defaults
server-side. We introduce a notification_settings_null value within a
client_capabilities structure that newer versions of the mobile apps
can use to request the new model.
This mobile compatibility code is pretty effectively tested by the
existing test_events tests for the subscriptions subsystem.
This makes the "more topics" option which appears below the list of
known topics in the left sidebar appear only when it's possible there
are actually more topics to be displayed. Two specific cases it
resolves completely include:
* Newly created realms; this widget was a common source of confusion
for new organization administrators.
* Newly created streams.
There are still some corner cases this doesn't handle, e.g. if you
just joined a private stream with protected history, but there isn't
as easy a fix for those.
Essentially rewritten by tabbott to fix code duplication and comment
extensively.
Fixes#10265.
Use the results of commit #73d26c8 to remove the method
`render_stream_description` in static/js/stream_data.js and instead
use the rendered_description attribute now being sent by the backend.
This will be a valuable optimization and a step towards removing the
need for the marked.js markdown parser and speeding up the client end.
This function unlike `invite_streams()` returns an array of objects having
various info (name, stream_id, invite_only, default_stream) related to
streams rather than an array of names of streams.
This function used to be called initialize_from_page_params(),
and we called it indirectly through `subs.js`.
Now we call it directly from `ui_init.js`, which gives us a
bit more control over how things are initialized. In fact,
this sets us up for the next commit, where I fix a recent
regression I introduced.
We now let color_data keep its own state for
unused_colors, so that we longer have to pass in
a large list of unused_colors every time we want
to assign a new stream color.
This mostly matters at startup, where we might
be cycling through 5000 streams. We claim all
the unused colors up front.
Each operation now has an upper bound of expensiveness,
where the worst case scenario is basically popping
off the first element of a list of <= 24 colors.
The algorithm is now deterministic, too, to make
it easier to test. It's unclear whether random color
assignment ever had much benefit, and it made unit
testing the algorithm difficult. Now we have 100%
line coverage.
Fixes part of #10902.
We move remove_deactivated_user_from_all_streams
into stream_events.js. There were some minor changes
to rename variables and also to not rely on using
`stream_info`.
This allows several modules to no longer need
to import `narrow` (or, in our current pre-import
world, to not have to use that global).
The broken dependencies are reflected in the node
tests, which should now run slightly faster.
We move some data code from subs.js to stream_data.js.
It's not clear we have been using the optimal sort for
dealing with locales, but this change preserves the
current behavior. The only subtle change here is that
we look up subs using a Dict now instead of a plain
JS object.
These were resulting in inadvertant calls to blueslip.fatal but
the code continued beyond that point, since in our testing environment
we do not stop executing when fatal is called.
This commit also adds checks to ensure that the subs were created.
Now that we've moved it into a bulleted set of options inside a modal,
there's no good reason to have separate variables for the corner cases
around who can manage a stream.
Partially fixes#4708.
Implements a first version (v1) for the feature. The next step would be
to allow admins to toggle `is_announcement_only` in the UI.
This run_test helper sets up a convention that allows
us to give really short tracebacks for errors, and
eventually we can have more control over running
individual tests. (The latter goal has some
complications, since we often intentionally leak
setup in tests.)
If notifications_stream is private and the current user has never been
subscribed, then we would throw an exception when trying to look up
notifications_stream. In this situation, we should just treat it like
the stream doesn't exist for the purposes of this user.
Currently, stream subscriptions aren't getting updated without
hard reload when user is deactivated in realm.
Fix this issue by updating stream subscription widgets on user
deactivation event.
Fixes#5623
In stream settings, if user add subscriber to unsubscribed public
stream from `Add` input widget it gives lots of blueslip warnings,
cause user isn't subscribed to public stream.
Fix this by changing condition to `sub.can_access_subscriber` from
`sub.subscribed` in blueslip warning, cause user can access
subscribers in such cases even if not subscribed to stream.
Tweaked by tabbott to make the node tests pass.
Private streams were not included in stream suggestions for default streams
in org settings.
Remove function, which exclude private streams from stream suggestions
for default streams.
This mostly moves code, and it also removes some unnecessary
coupling to stream_data.js. The topic_data code purely
works in the stream_id space, so there's no need to set up
actual stream data for it.
We now call topic_data.add_message() and
topic_data.remove_message() when we get info about
incoming messages. The old way of passing in a boolean
made the calling code hard to read and added unncessary
conditional logic to the codepath.
We also have vague plans to change how we handle
removing topics, since increment/decrement logic is now
kind of fragile, so making the "remove" path more explicit
prepares us to something smarter in the future, like just
figure out when the last topic has been removed by calling
a filter function or something outside of topic_data.js.
Another thing to note here is that the code changed here
in echo.js is dead code, since we've disabled
message editing for locally edited messages. I considered
removing this code in a preparatory commit, but there's
other PR activity related to local echo that I don't want
to conflict with.
One nice aspect of removing process_message() is that
the new topic_data.js module does not refer to the legacy
field "subject" any more, nor do its node tests.
This commit introduces a per-stream topic_history class
inside of topic_data.js to better encapsulate how we store topic
history.
To the callers, nothing changes here. (Some of our non-black-box
node tests change their way of setting up data, though, since the
internal data structures are different.)
The new class has the following improvements:
* We use message_id instead of timestamp as our sorting key.
(We could have done this in a prep commit, but it wouldn't
have made the diff much cleaner here.)
* We use a dictionary instead of a sorted list to store the
data, so that writes are O(1) instead of O(NlogN). Reads
now do sorts, so they're O(NlogN) instead of O(N), but reads
are fairly infrequent. (The main goal here isn't actually
performance, but instead it just simplifies the
implementation.)
* We isolate `topic_history` from the format of the messages.
This prepares us for upcoming changes where updates to the
data structure may come from topic history queries as well
as messages.
* We split out the message-add path from the message-remove
path. This prepares us to eventually get rid of the "count"
mechanism that is kind of fragile and which has to be
bypassed for historical topics.
This new module tracks the recent topic names for any given
stream.
The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.
We introduced a minor one-line function called stream_has_topics.
We now have all of our callers into recent_topics code just
receive a list of topic names from get_recent_topic_names().
This is more encapsulated than handing off tiny little
structures to the three callers, two of whom immediately
mapped the objects to names, and one of whom needlessly
used the now defunct name canon_subject field.
The consolidation here removes some "subject" references, and
now all lookup are by stream id, not stream name.
The diff here is a bit daunting, but it's mostly simplification
of tests and calling code. Two of the callers now need to look
up stream ids, but they are otherwise streamlined.
The main change here is to stream_data.js, and we replace the
`canon_subject` and `subject` fields with `name`.
The node tests have purged modules from cache that were
included via things like set_global(), but calling require
directly would leak modules into the next test, which made
a couple tests only work when you ran the whole suite. I
fixed those tests to work standalone. And then I now make
dependencies explicitly clear the require cache before we
require them in namespace.js.
This commit changes the key for recent_topics to be a
stream id. For streams that have been renamed, we will now
get accurate data on recent topics and active streams as
long as stream_data.get_stream_id(stream_name) returns a
valid value.
This commit changes stream_data.in_home_view() to
take a stream_id parameter, which will make it more
robust to stream name changes.
This fixes a bug. Now when an admin renames a stream
you are looking at, it will correctly show itself to
be un-muted. (Even with this fix, though, the stream
appears to be inactive.)
Some callers still do lookups by name, and they will
call name_in_home_view() for now, which we can
hopefully deprecate over time.
This is not a user-facing change, but it starts us down the
path to having the JS client be able to look up old stream
names for situations like people clicking old external links
or for live-update scenarios.
We used to have code scattered in multiple places to
calculate things like admin options, preview urls,
subscriber counts, and rendered descriptions for
streams before we rendered templates in the "Manage
Stream" code.
These are all consolidated into a new function
called stream_data.update_calculated_fields().
This is mostly code cleanup, but it also fixes a bug where
the "View Stream" button would not work for a newly created
stream.
When you subscribe to a stream, we now set a newly_subscribed
flag on the object, and we return true during the is_active()
call.
This solves the problem that immediately after you subscribe, you
don't have any messages in the stream, so it would appear active
by our old criteria.
This is still something of a workaround, as once you reload, the
stream will become inactive again, unless other messages come in.
A more permanent solution here would be to have the backend
indicate newly subscribed streams to us (apart from the initial
event), but we may not really need that in practice.
If a url is present in stream description, it will be
rendered as a clickable link under /streams page.
Tweaked by tabbott to use the separate rendered_description element to
avoid duplicate rendering and to live-update.
Fixes#1435.
This adds a button to #subsciption page called "View Stream"
that narrows the user to that particular stream.
This fix involves typical changes to JS/CSS to add new features,
and we also add a "preview_url" field to the sub object in
stream_data.js.
Fixes#3878
If we get invalid events related to stream subscribers, we now
exit earlier to prevent ugly tracebacks. We may eventually
want to upgrade some of these warnings to errors, once we fix some
of our live-update bugs. In particular, we don't yet live-update
users when streams go from private to public, so if you add/remove
subscribers to a newly-public stream that a user still thinks is
private, they will not be able to handle the event through no
fault of the codepath that happens during the add/remove.
(There was a method with the same name before, but it wasn't
being used. The new version will accept stream_id instead
of name, and we will use it as part of deactivating streams.)
Earlier commits removed all uses of page_params.email outside
of people.js, and it turns out we have page_params.user_id, so
we don't even need page_params.email for seeding the data.
This is a major change to the /#subscriptions page, converting it to
by a side-by-side list of streams and their settings in an overlay.
There are no new features added/removed, but it's a huge changeset,
because it replaces the old navigation logic and moves the stream
creation modal to appear in the right side of this overlay.
* In most cases, eslint --fix with the right comma-dangle settings was
able to update the code correctly.
* The exceptions were cases where the parser incorrectly treated the
arguments to functions like `assert_equal` as arguments; we fixed
these manually. Since this is test code, we can be reasonably
confident that just fixing the failures suffices to correct any bugs
introduced by making changes automatically.
We now use stream_id as our key to rename streams, which
should prevent a few race conditions long term. (We are
still possibly contending with other events that use
stream_name as a key, so this is not perfect.)
Steve Howell
Anders Kaseorg
Ryan Rehman
Tim Abbott
Yashashvi Dave
sahil839
Harshit Bansal
Rishi Gupta
Challa Venkata Raghava Reddy
Hemanth V. Alluri
Shubham Dhama
Rohitt Vashishtha
Shubham Padia
cPhost
Jerry Zhang
Joshua Pan
Steve Howell
Aditya Bansal
fionabunny
Raghav Jajodia
Kanak Garg
Brock Whittaker