This makes parse() more re-entrant.
This also drives out a change to the linkifiers
test, where I no longer couple the linkifiers
logic to markdown concerns. I probably should have
done this in an earlier commit, but better late
than never. I didn't bother to split out a commit
for the test stuff, since it's just tests and
the commit is still fairly atomic in nature.
It has always been pretty arbitrary what we did inside
of setup() vs. parse(), and we want to avoid unpredictable
results from other platforms neglecting to call setup().
On my machine you can parse a simple message in about
25 microseconds, based on a trial of a million messages
with the content of "**bold**". Whatever portion of
that time is related to setup-related things like
compiling regexes should be negligible from the user's
perspective, since we never run parse() in a loop.
These are the low-hanging-fruit places where we
can avoid using the helpers global.
The long term goal here is to make the markdown
code truly re-entrant, but some challenges still
remain.
Before this change, we would use **some** options relating
to parsing messages, but not all of them. The reason for
this was completely unintentional.
It's mostly a moot point, since the server sends back pretty
generic messages when you do something like invoke the
"/dark" command, and the message would parse the same way
whether or not the parser was looking for things like user
mentions or stream links.
In order to make this code predictable, I had to decide
whether we do a completely vanilla parse or a full message
parse. My decision now is mostly tactical. It's a trivial
one-line change to just use all the options for message
parsing, whereas it requires a major overhaul to allow a
vanilla parse.
I also predict that we will eventually want to parse these
server responses as if they were messages. I doubt the
zcommand responses would ever take advantage of it, but I
could imagine things like nag messages wanting to use user
mentions.
Even if my predictions are wrong, my decisions here are
pretty easy to reverse once we learn more.
For the particular case of zcommands, it is puzzling to me
why the server doesn't just send back HTML, but I don't want
to open that can of worms yet, as that would technically be
an API change.
For now I am happy with the one-line fix.
The zcommand code was calling directly into the "marked"
library, which was extremely misleading, since you don't
get a vanilla parse of the markdown due to the fact
that markdown.js calls setOptions at initialize time.
This commit shifts the responsibility to markdown.js
as well as adding a bit of test coverage, but it is
otherwise just a pure code-move refactoring.
The next commit will tweak things further.
The mobile app was never able to use the shared
version of emoji.js, because, among other problems
with our code organization, the emoji.js module
is strongly based on a mutate-the-data paradigm
that doesn't play nice with React. The way
that we mutate data and violate encapsuation
here is something that we would mostly want to fix
without even trying to shared code with mobile, so
subsequent commits will try to extract some pure
functions into a shared module.
This is definitely better than having linkifiers
reach directly into marked.js, but there is
probably further improvement we can do here
to clean up how these regexes get set.
This introduces a circular dependency between
markdown.js and linkifiers.js, but we will
soon break it in the other direction.
Note we now avoid linkifier checks for the case that a message
contains more obvious backend-only syntax such as attachments.
The next commit will eliminate the ugly early-return.
This gets us closer to having an API that can
be used my mobile.
The parse() function becomes a subset of
apply_markdown() that is no longer coupled
to the shape of a webapp object, and it can
be supplied with a new helper_config for each
invocation. Mobile will likely call this directly.
The setup() function becomes a subset of
initialize() that allows you to set up the
parser **before** having to build any kind of
message-specific helpers. Mobile will likely
call this directly.
The webapp continues to call these functions,
which are now thin wrappers:
* apply_markdown (wrapping parse)
* initialize (wrapping setup)
Note we still have several other problems to
solve before mobile can use this code, but we
introduce this now so that we can get a head
start on prototyping and unit testing.
Also, this commit does not address the fact
that contains_backend_only_syntax() is still
bound to the webapp config.
This eliminates an annoying bundle of complexity that caused the
frontend markdown processor's interface with the rest of Zulip's new
message processing code paths being more similar to that of a new
message from the server.
It also cuts down on code duplication.
Prefer a regexp match over using String#replace to strip expected
prefixes and suffixes because (a) it implicitly verifies that the
input has the expected format and (b) it won’t unexpectedly strip from
the middle of the string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
This reverts commit 1965584eec.
This syntax has a bad interaction with table syntax and needs to be
rethought.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
The regex could have match a bunch of characters including `>`
and hence end up matching across multiple adjacent spans in
a weird way. This commit fixes such an issue.
The extracted logic is in linkifier.js.
We have decided to name it linkifier.js instead of realm_linkifier.js
because in future when we will add stream-level linkifiers, we'll
likely want them to be managed by this same file.
* This introduces a new event type `realm_linkifiers` and
a new key for the initial data fetch of the same name.
Newer clients will be expected to use these.
* Backwards compatibility is ensured by changing neither
the current event nor the /register key. The data which
these hold is the same as before, but internally, it is
generated by processing the `realm_linkifiers` data.
We send both the old and the new event types to clients
whenever the linkifiers are changed.
Older clients will simply ignore the new event type, and
vice versa.
* The `realm/filters:GET` endpoint (which returns tuples)
is currently used by none of the official Zulip clients.
This commit replaces it with `realm/linkifiers:GET` which
returns data in the new dictionary format.
TODO: Update the `get_realm_filters` method in the API
bindings, to hit this new URL instead of the old one.
* This also updates the webapp frontend to use the newer
events and keys.
Extend our markdown system to support mentioning of users
by id also. Following these changes, it would be possible
to mention users with @**|user_id** and silently mention
using @_**|user_id**.
Main intention for extending the mention syntax is to make
it convenient for bots to mention a users using their ids. It
is to be noted that previous syntax are also supported.
Documentation tweaked by tabbott for better readability.
The changes were tested manually in development server, and also
by adding some new backend and frontend tests.
Fixes: #17487.
Replaced methods/functions of moment.js with date-fns library.
The motive was to replace it with a smaller frontend timezone library.
Date-fns ~ 11.51 kb
moment.js ~ 217.87 kb
Some of the format strings change because date-fns encodes them
differently from how moment did.
Fixes#16373.
Steve Howell
Tim Abbott
Anders Kaseorg
Ganesh Pawar
akshatdalton
Abhijeet Prasad Bodas
Sumanth V Rao
m-e-l-u-h-a-n
aryanshridhar