When we were preparing the conversion to ES modules in 2019, the
primary obstacle was that the Node tests extensively relied on the
ability to reach into modules and mutate their CommonJS exports in
order to mock things. ES module bindings are not mutable, so in
commit 173c9cee42 we added
babel-plugin-rewire-ts as a kludgy transpilation-based workaround for
this to unblock the conversion.
However, babel-plugin-rewire-ts is slow, buggy, nonstandard,
confusing, and unmaintained. It’s incompatible with running our ES
modules as native ES modules, and prevents us from taking advantage of
modern tools for ES modules. So we want to excise all use of
__Rewire__ (and the disallow_rewire, override_rewire helper functions
that rely on it) from the tests and remove babel-plugin-rewire-ts.
Commits 64abdc199e and
e17ba5260a (#20730) prepared for this by
letting us see where __Rewire__ is being used. Now we go through and
remove most of the uses that are easy to remove without modifying the
production code at all.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Use fully resolvable request paths because we need to be able to refer
to third party modules, and to increase uniformity and explicitness.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
For most functions that we were using __Rewire__ for,
it's better to just use the override helper, which
use __Rewire__ under the hood, but also resets
the reference at the end of run_tests.
Another nice thing about override() is that it reports
when you never actually needed the mock, and this
commit fixes the instances found here.
I didn't replace every call to __Rewire__. The
remaining ones fall under these categories:
* I looked for ") =>" in my code sweep,
so I missed stuff like "noop" helpers.
* Sometimes we directly update something
in a module that's not a function. This
is generally evil, and we should use setters.
* Some tests have setup() helpers or similar
that complicated this code sweep, so I
simply punted.
* Somes modules rely on intra-test leaks. We
should fix those, but I just punted for the
main code sweep.
This is a deceptively ugly diff. It makes
the actual code way more tidy.
I basically inlined some calls to mock_module
and put some statements in lexical order.
We now just use a module._load hook to inject
stubs into our code.
For conversion purposes I temporarily maintain
the API of rewiremock, apart from the enable/disable
pieces, but I will make a better wrapper in an
upcoming commit.
We can detect when rewiremock is called after
zrequire now, and I fix all the violations in
this commit, mostly by using override.
We can also detect when a mock is needlessly
created, and I fix all the violations in this
commit.
The one minor nuisance that this commit introduces
is that you can only stub out modules in the Zulip
source tree, which is now static/js. This should
not really be a problem--there are usually better
techniques to deal with third party depenencies.
In the prior commit I show a typical workaround,
which is to create a one-line wrapper in your
test code. It's often the case that you can simply
use override(), as well.
In passing I kill off `reset_modules`, and I
eliminated the second argument to zrequire,
which dates back to pre-es6 days.
We no longer export make_zjquery().
We now instead have a singleton zjquery instance
that we attach to global.$ in index.js.
We call $.clear_all_elements() before each module.
(We will soon get even more aggressive about doing
it in run_test.)
Test functions can still override $ with set_global.
A good example of this is copy_and_paste using the
real jquery module.
We no longer exempt $ as a global variable, so
test modules that use the zjquery $ need to do:
const $ = require("../zjsunit/zjquery");
We still need to write to these globals with set_global because the
code being tested reads from them, but the tests themselves should
never need to read from them.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
ES and TypeScript modules are strict by default and don’t need this
directive. ESLint will remind us to add it to new CommonJS files and
remove it from ES and TypeScript modules.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
Prettier would do this anyway, but it’s separated out for a more
reviewable diff. Generated by ESLint.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
We now treat util like a leaf module and
use "require" to import it everywhere it's used.
An earlier version of this commit moved
util into our "shared" library, but we
decided to wait on that. Once we're ready
to do that, we should only need to do a
simple search/replace on various
require/zrequire statements plus a small
tweak to one of the custom linter checks.
It turns out we don't really need util.js
for our most immediate code-sharing goal,
which is to reuse our markdown code on
mobile. There's a little bit of cleanup
still remaining to break the dependency,
but it's minor.
The util module still calls the global
blueslip module in one place, but that
code is about to be removed in the next
few commits.
I am pretty confident that once we start
sharing things like the typeahead code
more aggressively, we'll start having
dependencies on util. The module is barely
more than 300 lines long, so we'll probably
just move the whole thing into shared
rather than break it apart. Also, we
can continue to nibble away at the
cruftier parts of the module.
Zulip has had a small use of WebSockets (specifically, for the code
path of sending messages, via the webapp only) since ~2013. We
originally added this use of WebSockets in the hope that the latency
benefits of doing so would allow us to avoid implementing a markdown
local echo; they were not. Further, HTTP/2 may have eliminated the
latency difference we hoped to exploit by using WebSockets in any
case.
While we’d originally imagined using WebSockets for other endpoints,
there was never a good justification for moving more components to the
WebSockets system.
This WebSockets code path had a lot of downsides/complexity,
including:
* The messy hack involving constructing an emulated request object to
hook into doing Django requests.
* The `message_senders` queue processor system, which increases RAM
needs and must be provisioned independently from the rest of the
server).
* A duplicate check_send_receive_time Nagios test specific to
WebSockets.
* The requirement for users to have their firewalls/NATs allow
WebSocket connections, and a setting to disable them for networks
where WebSockets don’t work.
* Dependencies on the SockJS family of libraries, which has at times
been poorly maintained, and periodically throws random JavaScript
exceptions in our production environments without a deep enough
traceback to effectively investigate.
* A total of about 1600 lines of our code related to the feature.
* Increased load on the Tornado system, especially around a Zulip
server restart, and especially for large installations like
zulipchat.com, resulting in extra delay before messages can be sent
again.
As detailed in
https://github.com/zulip/zulip/pull/12862#issuecomment-536152397, it
appears that removing WebSockets moderately increases the time it
takes for the `send_message` API query to return from the server, but
does not significantly change the time between when a message is sent
and when it is received by clients. We don’t understand the reason
for that change (suggesting the possibility of a measurement error),
and even if it is a real change, we consider that potential small
latency regression to be acceptable.
If we later want WebSockets, we’ll likely want to just use Django
Channels.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
When we are pulling data from message.display_recipient
for private messages, the user_id field is always
called 'id', not 'user_id', so we can simplify
some defensive code.
This commit was originally automatically generated using `tools/lint
--only=eslint --fix`. It was then modified by tabbott to contain only
changes to a set of files that are unlikely to result in significant
merge conflicts with any open pull request, excluding about 20 files.
His plan is to merge the remaining changes with more precise care,
potentially involving merging parts of conflicting pull requests
before running the `eslint --fix` operation.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
This seems like a small change (apart from all the
test changes), but it fundamentally changes how
the app finds "topic" on message objects. Now
all code that used to set "subject" now sets "topic"
on message-like objects. We convert incoming messages
to have topic, and we write to "topic" all the way up
to hitting the server (which now accepts "topic" on
incoming endpoints).
We fall back to subject as needed, but the code will
emit a warning that should be heeded--the "subject"
field is prone to becoming stale for things like
topic changes.
This is part of work to break some of our
nastier circular dependencies in preparation
for our es6 migration.
This commit should facilitate loading leaf-like
modules such as people.js before all of the things
that reload.js depends on.
This makes sure that CSRF token is available while initializing
Socket, irrespective of the order of execution of deferred callbacks
after document becomes ready.
This is part of #9416.
This run_test helper sets up a convention that allows
us to give really short tracebacks for errors, and
eventually we can have more control over running
individual tests. (The latter goal has some
complications, since we often intentionally leak
setup in tests.)
Anders Kaseorg
Steve Howell
Riken Shah
Priyank Patel
clarammdantas
Rohitt Vashishtha
Steve Howell
Anders Kaseorg
Shubham Dhama
Tim Abbott