Commit Graph

44 Commits

Author SHA1 Message Date
Rohitt Vashishtha 912e372c4e markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:

`!avatar(alice@example.com) Alice: hey!`

---

Reimplementation

If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.

Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:

`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`

---

History

We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:

> We'll use this internally for the commit bot.  We might eventually
> disable it for external users.

We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.

When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!

Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.

We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.

This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.

---

Relevant commits:

- Oct 2012, Initial commit        c31462c278
- Nov 2013, Update commit bot     968c393826
- Nov 2013, Add avatar syntax     761c0a0266
- Sep 2017, Avoid email use       c3032a7fe8
- Apr 2019, Remove from webhook   674fcfcce1
2020-07-07 10:39:44 -07:00
Rohitt Vashishtha 732ec3c0e6 timestamp: Change syntax to `<time:timestammp>`.
We had been using !time() syntax for timestamps so far. Since its
an unreleased feature, we can make changes without affecting many
people.

Fixes #15442.
2020-07-06 15:53:56 -07:00
Rohitt Vashishtha b062e8332f markdown: Add timestamp syntax to markdown processors.
This adds support for syntax like: !time(Jun 7 2017, 6:30 PM) so that
everyone sees the time in their own local timezone. This can be used
when scheduling online meetings, etc.

This adds some hardcoded values for timezones, because of there
being no sureshot way of determining the timezone easily. However,
since the main way of using the feature should be a typeahead for
entering the time, this shouldn't be cause of much concern.

Fixes #5176.
2020-05-20 14:23:55 -07:00
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Anders Kaseorg 8e356368f7 markdown: Fix HTML escaping of &.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-13 17:50:59 -08:00
Steve Howell c2e6423b41 marked: Use startsWith instead of indexOf.
We are sweeping the codebase to use startsWith
when possible.  I kept this on a separate
commit due to us vendoring the library, just
to reduce some noise.
2020-01-28 12:48:02 -08:00
Rohitt Vashishtha 9bfef83efd minor: Fix accidental global variable leak in marked. 2019-12-09 16:13:02 -08:00
Thomas Ip 574c35c0b8 markdown: Render ordered lists using <ol> markup.
This brings us in line, and also allows us to style these more like
unordered lists, which is visually more appealing.

On the backend, we now use the default list blockprocessor + sane list
extension of python-markdown to get proper list markup; on the
frontend, we mostly return to upstream's code as they have followed
CommonMark on this issue.

Using <ol> here necessarily removes the behaviour of not renumbering
on lists written like 3, 4, 7; hopefully users will be OK with the
change.

Fixes #12822.
2019-09-08 16:42:20 -07:00
Rohitt Vashishtha a7f2bedb15 markdown: Enable hashheadings syntax.
Our implementation requires at least 1 space after the
'#' not not break existing linkifiers like '#123', etc.
that generally follow the convention we show in linkifier
examples.

- [valid]  : # Hello
- [valid]  : #  Hello
- [invalid]: #Hello

For the frontend, we have taken the code from v0.7.0 of
upstream marked and made minor changes to avoid having
to refactor a significant part of our marked code.

For the backend, we merely have to change the regex to
force require spaces after #, and add hashheader to our
list of blockparsers.

Fixes #11418.
2019-08-02 15:15:34 -07:00
Rohitt Vashishtha 3698cdcc58 topic-mention: Add Marked implementation as HandleStreamTopic. 2019-07-11 14:53:10 -07:00
Rohitt Vashishtha 57b9991396 markdown: Change syntax of silent mentions ( _@person -> @_person). 2019-02-20 10:41:42 -08:00
Rohitt Vashishtha 19672241e6 markdown: Disable definition/reference links in marked.
We had disabled reference style links in bugdown, however,
we hadn't disabled them in marked. This commit rectifies
that and adds test cases for the same.

Fixes #11350.
2019-02-04 11:16:37 -08:00
Rohitt Vashishtha 96aa1d4b37 markdown: Reduce mentions inside blockquotes to silent-mentions.
On the backend, we extend the BlockQuoteProcessor's clean function that
just removes '>' from the start of each line to convert each mention to
have the silent mention syntax, before UserMentionPattern is invoked.

The frontend, however, has an edge case where if you are mentioned in
some message and you quote it while having mentioned yourself above
the quoted message, you wouldn't see the red highlight till we get the
final rendered message from the backend.

This is such a subtle glitch that it's likely not worth worrying about.

Fixes #8025.
2019-01-16 16:08:37 -08:00
Rohitt Vashishtha f993fdd480 markdown: Add _@**Name** syntax for silent mentions.
These mentions look like regular mentions except they do not
trigger any notification for the person mentioned. These are
primarily to be used when you make a bot take an action and
the bot mentions you, or when you quote a message that mentions
you.

Fixes #11221.
2019-01-16 16:01:06 -08:00
Rohitt Vashishtha 38c7969555 marked: Warn against placing capture blocks below 'text' block. 2018-08-09 17:20:18 -07:00
Rohitt Vashishtha 3bdc8bbaa5 CVE-2018-9986: Fix XSS issues with frontend markdown processor.
This fixes a set of XSS issues with Zulip's frontend markdown
processor, which is used in a limited set of contexts, such as local
echo of messages and the drafts feature.

The implementation of several syntax elements, including the <em>
syntax, user and stream mentions, and some others failed to properly
escape the content inside the syntax.

Fix this, and add tests for each corrected code path.

Thanks to w2w for reporting this issue.
2018-04-12 09:46:37 -07:00
Shubham Dhama a32e1eb913 markdown: Require double-asterisk around all mentions.
This enforces `**` around all the mentions including "at-all" and
"at-everyone" mentions. Hence this makes `@all` and `@everyone`
invalid mentions, resulting into proper syntax for these mentions as
`@**all**` and `@**everyone**` respectively.

Note from tabbott: This removes an old feature/syntax, which made
sense back when @Tim was also a way to mention a user with Tim as
their first name.  Given how nice typeahead is now, the user part of
the feature was removed a while ago; this should have gone at the same
time.

Fixes: #8143.
2018-02-16 11:45:08 -08:00
Andy Perez f0190333b8 bugdown: Remove whitespace strip from inline code.
Fixes #4507
2017-12-14 07:48:10 -05:00
Umair Khan 8f190e0746 user-groups: Update marked.js to do proper local echo for mentions.
We implement markdown in the JS as well to improve the
usability.  Comes complete withunit tests.

Fixes #7381.
2017-11-27 09:16:09 -08:00
Andy Perez 83f563aeff bugdown: Properly parse empty blockquotes.
The character ">" now only starts a blockquote if the resulting
blockquote would be non-empty.  Thus, by itself, ">" is now
interpreted literally by bugdown, fixing #687.  The message
with contents consisting of ">>>" is now parsed as a doubly
(not triply) nested blockquote with contents ">".  Properly
formed blockquotes have identical behavior as before, but now
bugdown can no longer produce empty blockquotes as output.

Fixes #2886, #687.
2017-11-17 11:41:44 -08:00
Andy Perez 4d51a6cfac bugdown: Fix $$ whitespace handling.
The intended use of $$ is for inline expressions, not for multiline
ones; ```math is an acceptable alternative for the latter.  Hence,
the $$-syntax for inline TeX no longer permits newlines within it.

This was also necessary for the next change to be sensible; namely
allowing for spaces around both $$ when crafting inline TeX instead of
forcing everything to be crammed together, e.g. $$x=7$$.  In order to
avoid uninentionally creating inline expressions, the opening and
closing $$'s of an inline expression must now both exactly consist of
two dollar signs, no more and no less.

Fixes: #6488.
2017-11-10 16:54:27 -08:00
Joshua Pan e6611fec99 markdown.js: Remove paragraph break from numbered lists.
Fixes #5902.
2017-07-28 12:32:20 -07:00
Joshua Pan 5e5ca48063 markdown.js: Fix inconsistent #**stream** on new line.
This commit removes all code related to headers because
(1) we don't need the code and (2) it splits #**stream**
as a paragraph, which we don't want. This commit also
fixes the inconsistency when #**stream** is on a new line.

Fixes #4678.
2017-07-18 11:07:25 -07:00
Joshua Pan ec669a0c45 markdown.js: Fix inconsistent user mention on new line.
The user mention regex was checking for multiple lines,
so it broke when the user mention was on a new line.
This changes the regex AND adds a couple tests to
test inline markdown regexes.
2017-07-18 00:07:49 -07:00
Harshit Bansal c549dea9ac bugdown: Fix the regex used for unicode emoji matching.
The regex we were using didn't cover all the unicode blocks
to which our emojis belong. This commit fixes the regex to
include all the unicode blocks and also updates the
corresponding JS regex in marked.js.

Fixes: #3460.
2017-05-16 09:05:42 -07:00
Benjamin Gilbert 4775f55ca7 Fix spelling of "occurred" in various places. 2017-04-17 20:29:07 -07:00
Yago González 34a9e1ae11 markdown: Add TeX typesetting support.
Co-authored-by: Reid Barton <rwbarton@gmail.com>

Fixes #2056.
2017-03-21 16:40:00 -07:00
Yago González ca190bd394 bug fix: Handle empty at-mentions gracefully.
An exception in the webapp was trown when an empty mention was sent.
Examples of problematic messages are "@" or "@****".

In order to fix this, the regex that identifies mentions has been
modified, so it now requires the mention to have a "content" (by
replacing the ? quantifier by +).

A test case has been added to `frontend_tests/node_tests/echo.js` to
check that this works properly in the future.
2016-12-21 15:32:12 -08:00
hackerkid 951f1d9cdb Add missing unicode range to text: replace(inline.breaks.text)
This is a followup of #2297.
2016-11-14 20:10:54 -08:00
hackerkid b2e91eae13 Add marked.js support to emojis in miscellaneous and dingbats block.
https://en.wikipedia.org/wiki/Emoji#Unicode_blocks
2016-11-14 15:58:16 -08:00
paxapy ade3bda025 markdown: Fix stream link handler in corner cases.
* Fixes handling of multiple stream links and invalid stream names.
* Fixes text regex so it handle hash sign the right way.
* Adds tests for these stream link cases.
2016-11-10 19:22:19 -08:00
Igor Tokarev 852bc6b491 bugdown: Re-enable support for italics in bugdown.
Fixes: #1103.
2016-11-10 19:14:40 -08:00
Mohsen Ibrahim 44c8e4c58c streamLinkHandler: render stream links correctly.
After adding the ability to add stream links to messages using
the following pattern '#**stream_name**' there was a problem
with rendering this using our markdown engine because '**' means
bold text so that would render just to bold text.
To solve this I had to add regular expression in marked.js to match
that pattern and when it matches I call handleStreamLinks in echo.js
which will correctly render it to HTML.

Fixes #2218.

[tweaked by tabbott to url-encode the stream name in the URL and
adding the missing "#" in the display].
2016-11-08 13:52:31 -08:00
Igor Tokarev 07d05390bf markdown: Fix client usermention regular expression.
Fixes #993.
2016-11-06 18:37:09 -08:00
hackerkid 70223ef2da [third] Update marked base to newer version 0.3.6.
This basically re-applies the Zulip changes to marked 0.3.6.

Fixes #2190.
2016-11-04 21:56:53 -07:00
hackerkid bca1953aa4 Add support for !gravatar syntax in echo.js. 2016-10-20 14:40:45 -07:00
hackerkid 95b5ac1d5d Add support for !avatar syntax in echo.js. 2016-10-20 14:40:33 -07:00
Alex Wilson b040839c76 Add unicode emoji to frontend markdown parser.
Fixes 2nd half of #1011.
2016-06-30 15:48:02 -07:00
Leo Franchi 420a7cab7a [marked] Add preprocessors in marked and allow them to insert arbitrary HTML
(imported from commit 367418266f4fbe2d723212845f49f0d4356fab52)
2014-01-28 15:35:32 -05:00
Leo Franchi b119bf74ca [third] Update marked.js to HEAD to pull in javascript escape fixes
(imported from commit f673746c7f2d424c56b3e0b5d028952f97fb2a57)
2014-01-27 12:07:48 -05:00
Leo Franchi 439633fd03 More markdown compatibility fixes
* Escape " as &quot;
* Enable GFM newlines
* Output a linebreak after <br> in the generated markup

(imported from commit c007ec422054f9fc66a810b66aac70f70a2a1952)
2014-01-23 16:28:58 -05:00
Leo Franchi b1a6fdf5e2 Add support for realm filters to marked.js
(imported from commit cea1720b6535a23469185f0ad2677edd373e7cfb)
2014-01-23 16:28:57 -05:00
Leo Franchi 0de824f73e Add a zulip module to marked, which supports emoji and user mentions
This is partially inspired by https://github.com/chjj/marked/pull/238

(imported from commit 03b84cde88fc8c2fe31c1003a5ce852cc364733e)
2014-01-23 16:28:56 -05:00
Leo Franchi 835e385b75 [third] Add marked.js, a javascript markdown parser
(imported from commit e85d9b50328ce7d31882464fcc283478f7a9f753)
2014-01-23 16:28:54 -05:00