This adds support for a "spoiler" syntax in Zulip's markdown, which
can be used to hide content that one doesn't want to be immediately
visible without a click.
We use our own spoiler block syntax inspired by Zulip's existing quote
and math block markdown extensions, rather than requiring a token on
every line, as is present in some other markdown spoiler
implementations.
Fixes#5802.
Co-authored-by: Dylan Nugent <dylnuge@gmail.com>
Add methods to extract recent topics from received messages.
Process new messages as they are received.
Use new messages received from the server to extract recent_topics.
Node tests added.
`stream_topic_history` is a more appropriate name as this
module will contain information about last message of a
stream in upcoming commits. Function and variable names
are changed accordingly like:
* topic_history() -> per_stream_history()
* get_recent_names() -> get_recent_topic_names()
* name -> topic_name
We now treat util like a leaf module and
use "require" to import it everywhere it's used.
An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that. Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.
It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile. There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.
The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.
I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util. The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart. Also, we
can continue to nibble away at the
cruftier parts of the module.
webpack optimizes JSON modules using JSON.parse("{…}"), which is
faster than the normal JavaScript parser.
Update the backend to use emoji_codes.json too instead of the three
separate JSON files.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
We now use vdom-ish techniques to track the
list items for the pm list. When we go to update
the list, we only re-render nodes whose data
has changed, with two exceptions:
- Obviously, the first time we do a full render.
- If the keys for the items have changed (i.e.
a new node has come in or the order has changed),
we just re-render the whole list.
If the keys are the same since the last re-render, we
only re-render individual items if their data has
changed.
Most of the new code is in these two modules:
- pm_list_dom.js
- vdom.js
We remove all of the code in pm_list.js that is
related to updating DOM with unread counts.
For presence updates, we are now *never*
re-rendering the whole list, since presence
updates only change individual line items and
don't affect the keys. Instead, we just update
any changed elements in place.
The main thing that makes this all work is the
`update` method in `vdom`, which is totally generic
and essentially does a few simple jobs:
- detect if keys are different
- just render the whole ul as needed
- for items that change, do the appropriate
jQuery to update the item in place
Note that this code seems to play nice with simplebar.
Also, this code continues to use templates to render
the individual list items.
FWIW this code isn't radically different than list_render,
but it's got some key differences:
- There are fewer bells and whistles in this code.
Some of the stuff that list_render does is overkill
for the PM list.
- This code detects data changes.
Note that the vdom scheme is agnostic about templates;
it simply requires the child nodes to provide a render
method. (This is similar to list_render, which is also
technically agnostic about rendering, but which also
does use templates in most cases.)
These fixes are somewhat related to #13605, but we
haven't gotten a solid repro on that issue, and
the scrolling issues there may be orthogonal to the
redraws. But having fewer moving parts here should
help, and we won't get the rug pulled out from under
us on every presence update.
There are two possible extensions to this that are
somewhat overlapping in nature, but can be done
one a time.
* We can do a deeper vdom approach here that
gets us away from templates, and just have
nodes write to an AST. I have this on another
branch, but it might be overkill.
* We can avoid some redraws by detecting where
keys are moving up and down. I'm not completely
sure we need it for the PM list.
If this gets merged, we may want to try similar
things for the stream list, which also does a fairly
complicated mixture of big-hammer re-renders and
surgical updates-in-place (with custom code).
BTW we have 100% line coverage for vdom.js.
Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013. We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not. Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.
While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.
This WebSockets code path had a lot of downsides/complexity,
including:
* The messy hack involving constructing an emulated request object to
hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
needs and must be provisioned independently from the rest of the
server).
* A duplicate check_send_receive_time Nagios test specific to
WebSockets.
* The requirement for users to have their firewalls/NATs allow
WebSocket connections, and a setting to disable them for networks
where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
been poorly maintained, and periodically throws random JavaScript
exceptions in our production environments without a deep enough
traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
server restart, and especially for large installations like
zulipchat.com, resulting in extra delay before messages can be sent
again.
As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients. We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.
If we later want WebSockets, we’ll likely want to just use Django
Channels.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This commit was automatically generated by `tools/lint --only=eslint
--fix`, except for the `.eslintrc.json` change itself.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
With webpack, variables declared in each file are already file-local
(Global variables need to be explicitly exported), so these IIFEs are
no longer needed.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
perfect-scrollbar replaces both the appearance and the behavior of the
scrollbar, and its emulated behavior will never feel native on most
platforms. SimpleBar customizes the appearance while preserving the
native behavior.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
This commit adds `stream_ui_updates.js` module. This module
will includes functions which will update different ui elements
(i.e. subscription button, subscriber count).
ESLint currently forbids using extends in the override block, ie
```
{
"extends": ["plugin:@typescript-eslint/recommended"]
}
```
so we just have to add them manually for now.
See https://github.com/eslint/eslint/issues/8813 .
This adds a new realm_logo field, which is a horizontal-format logo to
be displayed in the top-left corner of the webapp, and any other
places where we might want a wide-format branding of the organization.
Tweaked significantly by tabbott to rebase, fix styling, etc.
Fixing the styling of this feature's loading indicator caused me to
notice the loading indicator for the realm_icon feature was also ugly,
so I fixed that too.
Fixes#7995.
Apparently, we didn't have one of these, and thus had a moderate
number of generally very old violations in the codebase. Fix this and
clear the ones that exist..
We move all of its logic into settings_sections.
Note that this is slightly more than a refactor.
We are slightly more aggressive about resetting
sections. For example, if you go into Settings,
then exit the overlay, then go into Manage
Organization, we will now reset sections for both
groups.
This small modules nicely breaks down the
responsibilities of topic_list and stream_list
when it comes to zooming in and out of topics
(also known as hitting "more topics" or "All
Streams).
Before this, neither module was clearly in
charge, and there were kind of complicated
callback mechanisms. The stream_list code
was asking topic_list to create click handlers
that called back into stream_list.
Now we just topic_zoom set up its own click
handlers and delegate out to the other two
modules.
This module makes it really easy to create are-you-sure
dialogs for dangerous operations.
Basically it's one function with five parameters. You
give three chunks of HTML, a callback function, and
a parent container.
The first use of this will be in settings_user_groups,
coming up in a couple commits.
This is part of work to break some of our
nastier circular dependencies in preparation
for our es6 migration.
This commit should facilitate loading leaf-like
modules such as people.js before all of the things
that reload.js depends on.
We add a padded div to our container for the buddy
list to give scrolling the illusion that we've
rendered every list item, while still letting
the browser do the heavy lifting instead of trying
to fake it out too much.
We probably should have done this a while ago, even
though these functions are pretty tiny. The goal here
is to make it easier to have more consistent search
semantics.
Our first use case is subs.js. In this case we
are able to decouple a bit of generic string
matching from the subs-specific code.
Also adds the file to the static asset pipeline.
search_pill_widget.js will be used to access the pills object for
the search query box. It will act in a similar way to
compose_pm_pill.js. Why is this needed: Consider you've initiated
a pills object in search.js for the search query box. Now you want to
also access that pills object to pre-populate pills after a reload in
hashchange.js. search_pill_widget.js makes this easy without the use
of events.
This implements right-to-left message automatic detection support in
the compose box as well as the message feed. Full unit tests and
support in the message-editing UI are for future work (as are
potentially more fancy things like supporting things like
right-to-left multi-word names for users/streams/etc.).
Fixes#3123.
This commit fixes some modules that were erroneously left out while
transitioning app.js to webpack. This commit exposes them using
expose-loader or setting them directly to window.
Adds search_pill.js to the static asset pipeline. The items
for search pill contain 2 keys, display_value and search_string.
Adding all the operator information i.e the operator, operand and
negated fields along with the search_string and description was tried out.
It was dropped because it didn't provide any advantage as one had to
always calculate the search_string and the description from the operator.
This adds a /ping command that will be useful for users
to see what the round trip to the Zulip server is (including
only a tiny bit of actual server time to basically give a
200).
It also introduce the "/zcommand" endpoint and zcommand.js
module.
A "zform" knows how to render data that follows our
schema for widget messages with form elements like
buttons and choices.
This code won't be triggered until a subsequent
server-side commit takes widget_content from
API callers such as the trivial chat bot and
creates submessages for us.
This starts the concept of a schema checker, similar to
zerver/lib/validator.py on the server. We can use this
to validate incoming data. Our server should filter most
of our incoming data, but it's useful to have client-side
checking to defend against things like upgrade
regressions (i.e. what if we change the name of the field
on the server side without updating all client uses).
This commit lays the foundation to handle submessages for
plugin widgets. Right now it just logs events, but subsequent
commits will add widget functionality.
We now initialize most modules in ui_init.js, which
isn't the perfect place to do it, but at least now
we have a mostly consolidated entry point.
All the new foo.initialize() methods introduced in
this module run the same order relative to each
other as before this commit. (I did some console
logging with a hacked version of the program to
get the order right.) They happen a bit later than
before, though.
A couple modules still have the `$(function() {`
idiom for miscellaneous reasons:
archive - is a different bundle
common - used elsewhere
list_render - non-standard code style
scroll_bar - no exports
setup - probably special?
socket - $(function () is nested!
transmit - coupled to socket
translations - i18n is a bigger problem
ui_init - this bootstraps everything
This run_test helper sets up a convention that allows
us to give really short tracebacks for errors, and
eventually we can have more control over running
individual tests. (The latter goal has some
complications, since we often intentionally leak
setup in tests.)
We don't reference this anymore (it was only ever used by the Dropbox
integration, which was hardcoded-off for years before being removed in
e6833b6427)
This should help prevent problems with folks introducing new code that
doesn't match our whitespace style.
There's a couple things I don't like about this configuration:
* How it handles multi-line JS lists (i.e. the [] syntax)
* That we ended up having to turn off indentation on a half-dozen
files that apparently don't use our standard IIFE indentation style.
* That we have it completely turned off for the node tests; ideally,
we'd just have slightly different rules around the IIFE identation story.
But otherwise, this works pretty well, and should catch a pretty wide
range of indentation regressions.
Most of this was straightforward.
Most functions that were grabbed verbatim and whole from
the original class still have one-line wrappers.
Many functions are just-the-data versions of functions that
remain in MessageList: see add, append, prepend, remove as
examples. In a typical pattern the MessageList code becomes
super simple:
prepend: function MessageList_prepend(messages) {
var viewable_messages = this.data.prepend(messages);
this.view.prepend(viewable_messages);
},
Two large functions had some minor surgery:
triage_messages =
top half of add_messages +
API to pass three lists back
change_message_id =
original version +
two simple callbacks to list
For the function update_muting_and_rerender(), we continue
to early-exit if this.muting_enabled is false, and we copied
that same defensive check to the new function
named update_items_for_muting(), even though it's technically
hidden from that codepath by the caller.
This introduces a generic class called list_cursor to handle the
main details of navigating the buddy list and wires it into
activity.js. It replaces some fairly complicated code that
was coupled to stream_list and used lots of jQuery.
The new code interacts with the buddy_list API instead of jQuery
directly. It also persists the key across redraws, so we don't
lose our place when a focus ping happens or we type more characters.
Note that we no longer cycle to the top when we hit the bottom, or
vice versa. Cycling can be kind of an anti-feature when you want to
just lay on the arrow keys until they hit the end.
The changes to stream_list.js here do not affect the left sidebar;
they only remove code that was used for the right sidebar.
This was a bit more than moving code. I extracted the
following things:
$widget (and three helper methods)
$input
text()
empty()
expand_column
close_widget
activity.clear_highlight
There was a minor bug before this commit, where we were inconsistent
about trimming spaces. The introduction of text() and empty() should
prevent bugs where users type the space bar into search.
This is a pretty thin abstraction to prevent having to put
magic numbers in code, doing the which/keyCode hack, and remembering
to all preventDefault.
Hopefully we'll expand it to handle things like shift/alt keys
for components that want their own keyboard handlers (vs. going
through hotkey.js).
There are several ways we open help for keyboard shortcuts,
markdown help, and search operators.
- from the gear menu
- from the compose box
- from the search box
- hitting ? for keyboard help
- arrowing/clicking through the tabs
This just moves the relevant code into a module and changes a
bunch of one-line calls in various places.
Now that we have support for displaying custom profile fields, this
adds administrator-level support for creating them.
Tweaked by tabbott to fix a few small bugs and clean up the commit message.
Fixes#1760.
Update perfect-scrollbar to fix stutter space-scrolling in #8544. Also
reworked deprecated `element.perfectScrollbar` to `new
PerfectScrollbar(element)`. Lastly, updated provision version and
changed node module path to new path.
This also refactors perfect-scrollbar in help.js to work with updated
version of perfect-scrollbar. Because the update also changed
perfect-scrollbar's css selectors for all scrollbars in zulip, we
update those too.
Fixes#8544.
Also adds a custom rule to eslint. Since the recommended way of extending
eslint is to create plugins as standalone npm packages, the separate rule
is published as 'eslint-plugins-empty-returns'.
Fixes#8669.
This mostly moves code from ui.js.
We change the arguments to `message_fetch.load_more_messages()`
to be `opts` with callbacks for `show_loading` and `hide_loading`.
We also defer starting the scroll handler until `message_fetch.js`
has been initialized.
@brockwhittaker wrote the original prototype for having
pills in the recipient box when users compose PMs (either
1:1 or huddle). The prototype was test deloyed on our
main realm for several weeks.
This commit includes all the original CSS and HTML from
the prototype.
After some things changed with the codebase after the initial
test deployment, I made the following changes:
* In prior commits I refactored out a module called
`user_pill.js` that implemented some common functions
against a more streamlined version of `input_pill.js`,
and this commit largely integrates with that.
* I made changes in a prior commit to handle Zephyr
semantics (emails don't get validated) and tested
this commit with zephyr.
* I fixed a reload bug by extracting code out to
`compose_pm_pill.js` and re-ordering some
calls to `initialize`.
There are still two flaws related to un-pill-ified text in the
input:
* We could be more aggressive about trying to pill-ify
emails when you blur or tab away.
* We only look at the pills when you send the message,
instead of complaining about the un-pill-ified text.
(Some folks may consider that a feature, but it's
probably surprising to others.)
The main point of this change is to streamline the core
code for input pills, and we use also modify user groups.
The main change to input_pill.js is that you now
configure a function called `create_item_from_text`, and
that can return an arbitrary object, and it just needs
a field called `display_value`.
Other changes:
* You now call `input.create(opts)` to create the
widget.
* There is no longer a cache, because we can
write smarter code in typeahead `source` functions
that exclude ids up front.
* There is no value/optinalKey complexity, because
the calling code can supply arbitrary objects and
do their own external data management on the pill
items.
* We eliminate `prependPill`.
* We eliminate `data`, `keys`, and `values`, and just
have `items`.
We now isolate the code to transmit messages into transmit.js.
It is stable code that most folks doing UI work in compose.js don't
care about the details of, so it's just clutter there. Also, we may
soon have other widgets than the compose box that send messages.
This change mostly preserves test coverage, although in some cases
we stub at a higher level for the compose path (this is a good thing).
Extracting out transmit.js allows us to lock down 100% coverage on that
file.
The original code made a 3/4-hearted effort to generically accommodate
more banners/"panels" later, but named itself after the first one made.
[greg: expanded commit message.]
This adds custom CSS through JavaScript for things that do not
scope well and will override other inherited styles.
This should ONLY be used for problematic CSS that has no obvious
or easy CSS-only solution.
(Specifically, we need this for the "default link" styling, which is
hard to override because we don't want to start winning ties due to
specificity that we would not have won in the light theme).
Tweaked by tabbott to move changes from the next commit that are
required for this to pass tests into this commit.
Note that this exports a few items that were not previously exported.
This uses the to-markdown.js library to do all the hard work of
parsing HTML and turning it into markdown and not e.g. uploaded files.
Tweaked by tabbott to better scope when it activates to just include
pastes of HTML content.
Fixes#5853.
This change prepares us to have the server send avatar_url
of None when somebody wants a gravatar avatar (as opposed
to a user-uploaded one).
Subsequent commits will change behavior on both the server
and client to have this happen. So this commit has no-op
code for now, but it will soon use the fallback-to-gravatar
logic.
Lets administrators view a list of open(unconfirmed) invitations and
resend or revoke a chosen invitation.
There are a few changes that we can expect for the future:
* It is currently possible to invite an email that you have already
invited, it might make sense to change this behavior.
* Resend currently sends an invite reminder instead of resending the
original invite, this is because 'custom_body' was not stored when
the first invite was sent.
Tweaked in various minor ways, primarily in the backend, by tabbott,
mostly for style consistency with the rest of the codebase.
Fixes: #1180.
This is a two-step notifications process that will ask a user
to enable notifications and if they click exit give them three
options:
1. Enable notifications.
2. Ask later.
3. Never ask on this computer again.
The first two are self-explanatory (ask later = next session it
asks again). The third is captured and stored in localStorage and
a check is done on page load to see whether or not notifications
should be displayed.
Commit modified heavily by Brock Whittaker <brock@zulipchat.com>.
Fixes#1189.
Here are the functions in top_left_corner:
get_global_filter_li: pure code move
update_count_in_dom: simplifed copy of similar function in stream_list.js
update_dom_with_unread_counts: pure code move, split out from function
of same name in stream_list.js
delselect_top_left_corner_items: pure code move
handle_narrow_activated: pure code move + rename
handle_narrow_deactivated: pure code move, split out from from function
of smae name in stream_list.js
This commit extract send_messages.js to clean up code related
to the following things:
* sending data to /json/report_send_time
* restarting the event loop if events don't arrive on time
The code related to /json/report changes the following ways:
* We track the state almost completely in the new
send_messages.js module, with other modules just
making one-line calls.
* We no longer send "displayed" times to the servers, since
we were kind of lying about them anyway.
* We now explicitly track the state of each single sent
message in its own object.
* We now look up data related to the messages by local_id,
instead of message_id. The problem with message_id was
that is was mutable. Now we use local_id, and we extend
the local_id concept to messages that don't get rendered
client side. We no longer need to react to the
'message_id_changed' event to change our hash key.
* The code used to live in many places:
* various big chunks were scattered among compose.js,
and those were all moved or reduced to one-line
calls into the new module
* echo.js continues to make basically one-line calls,
but it no longer calls compose.report_as_received(),
nor does it set the "start" time.
* message_util.js used to report received events, but
only when they finally got drawn in the home view;
this code is gone now
The code related to restarting the event loop if events don't arrive
changes as follows:
* The timer now gets set up from within
send_messages.message_state.report_server_ack,
where we can easily inspect the current state of the
possibly-still-in-flight message.
* The code to confirm that an event was received happens now
in server_events.js, rather than later, so that we don't
falsely blame the event loop for a downstream bug. (Plus
it's easier to just do it one place.)
This change removes a fair amount of code from our node tests. Some
of the removal is good stuff related to us completing killing off
unnecessary code. Other removals are more expediency-driven, and
we should make another sweep at ramping up our coverage on compose.js,
with possibly a little more mocking of the new `send_messages` code
layer, since it's now abstracted better.
There is also some minor cleanup to echo.resend_message() in this
commit.
See #5968 for a detailed breakdown of the changes.
This new module tracks the recent topic names for any given
stream.
The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.
We introduced a minor one-line function called stream_has_topics.
This removes the `no-new` rule which is relatively detrimental to
code cleanliness in our codebase because third-party libraries may
utilize data structures that don't fly well with our linting rules.
This also fixes abstractions that were created due to the limitations
and impositions of this lint rule.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).
When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779. We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
This is mostly straightforward moving of code out of compose.js.
The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.
The following function names in this commit are new, but their
code was basically pulled over verbatim:
process_success (was process_send_time)
set_timer_for_restarting_event_loop
clear
initialize
All the code in the new module is covered by previous tests that
had been written for compose.js. This commit only modifies
a few things to keep those tests.
The new module has 100% node coverage, so we updated `enforce_fully_covered`.
This system hasn't been in active use for several years, and had some
problems with it's design. So it makes sense to just remove it to declutter
the codebase.
Fixes#5655.
In pm_conversations.js, added function to make a user a PM partner and
another function to check if a user is a PM partner. A PM partner is
someone with whom the user has been in a PM with.
In recent_senders module, added a data structure to hold timestamps of
users' latest message in a topic. Also added a function to compare 2
users based on above timestamp. Added a function to process messages for
the data structure and a call in add_message_metadata. Also added node
tests for insertion of data into recent_senders.senders.
Anders Kaseorg
Sara Gulotta
Sharif Naas
Aman Agrawal
Rohitt Vashishtha
Hashir Sarwar
Anders Kaseorg
Steve Howell
Thomas Ip
Wyatt Hoodes
Priyank Patel
Vishnu Ks
Anders Kaseorg
Yashashvi Dave
sahil839
Tim Abbott
Joshua Pan
Shubham Padia
Rhea Parekh
Shayan Toqraee
Armaan Ahluwalia
Shubham Dhama
Aditya Bansal
Umair Khan
Gooca
Brock Whittaker
Cynthia Lin
Garvit
Henrik Pettersson
Aastha Gupta
Steve Howell
Vaida
Harshit Bansal
Mehanig
Yago González
Cory Lynch
Akhil
Mahim Goyal