Commit Graph

305 Commits

Author SHA1 Message Date
paxapy d965ff16fd bugdown: refactor AutoLink to use VerbosePattern.
This makes it possible to reuse the verbose pattern logic for other
regexs as well.
2016-11-01 19:50:59 -07:00
umkay 1d077d1a4f bugdown: Remove safe_mode dependencies.
Now that we have updated python-markdown, we remove the deprecated
safe_mode. We used safe_mode to escape raw html, so now instead we
pass in an EscapeHtml markdown extension to the markdown engine.

See https://pythonhosted.org/Markdown/release-2.6.html for details on
the deprecation.

Fixes: #2037 (also addresses the remaining piece of #2043).
2016-10-25 15:22:18 -07:00
K.Kanakhin 7bc0c7f0d5 bugdown: Fix patterns ordering in Markdown extension.
- Put 'modal_link' and 'link' inline patterns after avatar inline pattern
  to correct applying them to parsed text.

Fixes:  #1654
2016-10-20 11:18:41 -07:00
Brock Whittaker 77233c7e3b bugdown: Distinguish between YouTube Videos and Image Previews.
This distinguishes between YouTube Videos and Image Previews by adding
a particular “youtube-video” class to the preview along with changing
the title to the video ID rather than the link. This serves to allow
the lightbox to ID when a lightbox preview should be treated like a
YouTube video rather than an image preview.

This also modifies the tests in bug down to expect a youtube-video class
along with the title to just be the video ID on YouTube rather than the
entire URL link.
2016-10-18 14:45:47 -07:00
Mohsen Ibrahim e9899e4dc9 bugdown: Fix some Python-Markdown deprecation warnings.
Python-Markdown changed how one is supposed to declare extensions
being used.
2016-10-17 23:13:41 -07:00
Tim Abbott e37b6488b8 Bugdown: Remove checked-in CodeHilite extension.
The changes that required us to fork this extension had been merged
into upstream CodeHilite, so we can remove it and switch to using the
version that comes with python-markdown.
2016-10-16 00:40:08 -07:00
umkay c028815385 bugdown: Update python-markdown to version 2.6.7.
This updates Bugdown to reflect the changes in the updated
markdown. In particular, we now pass a default config object in the
__init__ for the Bugdown extension, update the make_md_engine function
to take kwargs as opposed to a config list, and have UListProcessor
inherit from ulist as opposed to olist (which no longer works).

We update the (forked from upstream) fenced_code extension's
makeExtension to take args and kwargs, and update
FencedBlockPreprocessor __init__ method with updated Codehilite
arguments.

We update the (forked from upstream) Codehilite extension to
mirror the logic with the latest upstream Codehilite:
	Add parse_hl_lines function
	update makeExtension to take args and kwarfs instead of config
	list
	Add regex for highlight lines
	use linenums instead of linenos
	use get_formatter_by_name instead of HtmlFormatter
	user get_lexer_by_name instead of TextLexer
	add hl_lines and use_pygments arguments to the codehlite
	constructor
2016-10-15 23:38:32 -07:00
Eklavya Sharma 41587edd2e bugdown: Change parameter name.
Change the parameter name of some functions from 'md' to 'content',
since the name 'md' seems to be the reason why this parameter was
wrongly annotated.
2016-10-11 11:31:01 -07:00
Eklavya Sharma f35d0f1c75 bugdown: Fix type annotations.
In some functions the first parameter was wrongly annotated as
`markdown.Markdown`. Change that to `text_type`.
2016-10-11 11:31:01 -07:00
Tim Abbott 22fd7ba02a avatar: Move avatar hash computations to their own file. 2016-10-02 21:19:10 -07:00
Brock Whittaker 98937ac539 bugdown: Set "title" attribute to actual image title when available.
This sets the “title” attribute on the image to the actual title of
the image specified by the user in their markdown, rather than just
the URL of the full link to it.
2016-09-27 17:34:05 -07:00
Steve Howell 40b18094ec alert words: Refactor alert word detection.
The bugdown parser no longer has a concept of which users need which
alert words, since it can't really do anything actionable with that info
from a rendering standpoint.

Instead, our calling code passes in a set of search words to the parser.
The parser returns the list of words it finds in the message.
Then the model method builds up the list of user ids that should be
flagged as having alert words in the message.

This refactoring is a little more involved than I'd like, but there are
still some circular dependency issues with rendering code, so I need to
pass in the rather complicated realm_alert_words data structure all the way
from the action through the model to the renderer.

This change shouldn't change the overall behavior of the system, except
that it does remove some duplicate regex checks that were occurring when
multiple users may have had the same alert word.
2016-09-23 15:21:17 -07:00
Steve Howell a036a72db6 bugdown: Raise BugdownRenderingException.
We now raise an exception in bugdown.do_convert() if rendering
fails, to avoid silent failures, and then calling code can convert
the exception to a JsonableError.
2016-09-16 10:01:55 -07:00
Steve Howell 152cebf33d bugdown: Extract log_bugdown_error().
(It makes it easier to mock out the logging.)
2016-09-16 10:01:54 -07:00
umkay 7eb6924841 Wrap dict view object in list for python3 compatibility.
Deleting items from a dictionary being iterated through directly
doesn't work properly in Python 3.
2016-09-13 21:21:45 -07:00
Taranjeet Singh 5e5ad9de0c bugdown/__init__.py: Fix line with length greater than 120.
This updates file zerver/lib/bugdown/__init__.py.
2016-08-19 11:56:43 -07:00
Tim Abbott e7aac717e8 Fix markdown ordering bugs with some custom syntax.
!avatar, !modal_link, !gravatar, etc. were incorrectly being processed
before the escape character for code blocks.

While we're at it, we add tests for these special syntaxes.
2016-08-18 14:47:37 -07:00
Tim Abbott 70543e059a bugdown: Remove hardcoding of mit.edu for zephyr_mirror realm config. 2016-07-26 20:30:12 -07:00
Eklavya Sharma c389d22b5f Fix twitter mock data. 2016-07-17 10:23:25 -07:00
Eklavya Sharma fb1d00fc6c Upgrade python-twitter to latest version.
Fixes #1145.
2016-07-17 10:23:25 -07:00
Eklavya Sharma 4130170da8 bugdown: Fix string handling.
Use unicode strings where needed and correctly encode/decode strings.
2016-07-17 10:23:25 -07:00
Eklavya Sharma abe7faa2f9 bugdown: Annotate some variables. 2016-07-17 10:23:25 -07:00
Taranjeet a8a4caf2c0 zerver: Fix lines with length greater than 120. 2016-07-08 11:41:43 -07:00
Eklavya Sharma 6bb266d262 bugdown's __init__.py: Add python 3 compatibility.
* Use Response.text instead of Response.content.
* Make unescaping work on python 3.
2016-07-07 09:52:24 -07:00
Eklavya Sharma 05046d9288 bugdown's codehilite.py: Fix annotations in python 3 mode.
Many stubs in xml.etree.ElementTree use Union[str, bytes] as
return type.  Mypy wants us to correctly handle each case.  This
is correct, but not useful for us since we know that we'll always
get str.  So force the return value to text_type, to supress mypy
errors.
2016-07-07 09:52:24 -07:00
Eklavya Sharma b902c1ae45 bugdown: Replace raw unicode literal with non-raw.
Raw unicode literals are disallowed in python 3.
2016-07-02 10:38:48 -07:00
Alex Wilson 8c62cff1b7 bugdown: Clean up paths to emojis. 2016-06-30 15:48:02 -07:00
Alex Wilson 5ec29101eb Add unicode emoji support to bugdown.
Fixes half of #1011.
2016-06-30 15:48:02 -07:00
Tim Abbott ce7c7d3510 Wrap youtube_re. 2016-06-25 10:52:03 -07:00
Eklavya Sharma 598fb1ff28 zerver/lib/bugdown/fenced_code.py: Add BaseHandler.
Add a class 'BaseHandler' and make it a base class of OuterHandler,
QuoteHandler and CodeHandler.  This will help annotate some functions
and improve type checking.
2016-06-16 15:21:17 -07:00
Eklavya Sharma 142bcadb68 Annotate zerver/lib/bugdown/fenced_code.py. 2016-06-16 15:21:17 -07:00
Eklavya Sharma 9b72b7b37e Annotate zerver/lib/bugdown/codehilite.py.
Also change some string literals to unicode.
2016-06-16 15:21:17 -07:00
Eklavya Sharma 744007f33f zerver/lib/bugdown/__init__.py: Fix string types.
Change important string constants from str to text_type.
Replace str by text_type in annotations where relevant.
2016-06-16 15:21:17 -07:00
Eklavya Sharma 04f44b12ad zerver/lib/bugdown/testing_mocks.py: Fix types.
Also fix clashing annotations in zerver/lib/bugdown/__init__.py.
2016-06-16 15:21:17 -07:00
Eklavya Sharma 7db0765a18 zerver/lib/bugdown/fenced_code.py: Fix types.
Change some important string literals from str to unicode.
Annotate format_code and codehilite_conf in FencedBlockProcessor.
2016-06-16 15:21:17 -07:00
Eklavya Sharma 48c5b299b6 zerver/lib/bugdown/codehilite.py: Add type annotation.
Mypy incorrectly infers a dict's type as `Dict[str, List[object]]`.
Change that to `Dict[str, List[Any]]`.
2016-06-16 15:21:17 -07:00
Eklavya Sharma a485d63975 zerver/lib/bugdown/__init__.py: Fix non-string annotations. 2016-06-16 15:21:17 -07:00
Tim Abbott eafb91719c get_tweet_id: Fix return type to by None. 2016-06-16 15:21:01 -07:00
Eklavya Sharma 72ed1f4187 Fix python 3 pyflakes error.
Change raw unicode literal to non-raw.  A raw unicode literal is a
syntax error in python 3.  This error was detected by pyflakes.
2016-06-13 19:44:56 -07:00
Tim Abbott 2219ef7bef bugdown: Rename upload_re to upload_title_re for clarity. 2016-06-13 19:44:56 -07:00
Tim Abbott 5b1cfbc977 bugdown: Fix extraction of titles for uploaded files.
The previous code was associated with a previous version of the upload
URL naming scheme, and thus never triggered in practice.
2016-06-13 19:44:51 -07:00
Umair Khan c8d139b2b1 Fix emoji urls interaction with i18n.
When accessing emojis with relative urls we should start the urls with
a slash so that language code doesn't become part of these urls.

Fixes #1014.
2016-06-13 09:10:11 -07:00
Eklavya Sharma 8acc51218e Fix realm emoji and realm filter annotations.
Change `str` to `text_type` in annotations in zerver/models.py
related to realm emoji and realm filters.
Also fix clashing annotations in zerver/lib/bugdown/__init__.py.
2016-06-12 09:25:42 -07:00
Tim Abbott 1ca7c3378b Annotate zerver.lib.testing_mocks.
Also fix some annotations in bugdown to match.
2016-06-04 22:54:49 -07:00
Tim Abbott 6f69053911 Annotate most of the rest of bugdown. 2016-06-04 11:35:29 -07:00
Tim Abbott e89730dc8f subject_links: Remove useless RealmFilter.DoesNotExist case. 2016-06-03 23:18:39 -07:00
Tim Abbott 654bd663aa bugdown: Add annotations for a few more functions.
This resolves the issue with Typeshed #244 by making `upload_re` start
with `ur`.
2016-06-03 18:11:53 -07:00
Oren Leaffer 2916fb30cb bugdown: add some type annotations.
Had to add some "type: ignore" because the pattern used in match
doesn't affect the type returned. A fix for this issue has been pushed
to typeshed - https://github.com/python/typeshed/pull/244
2016-06-03 17:03:52 -07:00
Nathan Florea 04c71fadc6 More removal of mutable default arguments.
I've left a few that clearly aren't being passed and aren't being mutated, but
I think I've gotten the rest of them.
2016-06-03 09:17:04 -07:00
Eklavya Sharma 149938d468 Change shebangs from python2.7 to python. 2016-05-29 05:03:08 -07:00
Eklavya Sharma 0dcd8b387d Make zerver/lib/bugdown/fenced_code.py pass mypy check. 2016-05-18 17:10:17 +05:30
Tim Abbott b869be9301 style: Use 'not in' consistently rather than `not foo in`. 2016-05-09 17:00:10 -07:00
Tim Abbott 762a3188ee Fix missing whitespace after # in comments. 2016-05-02 22:10:47 -07:00
Tim Abbott 5bd94c15c7 Use camo to avoid mixed content warnings when displaying emoji. 2016-05-02 17:21:31 -07:00
Tim Abbott 65207477c4 bugdown: Annotate emoji handleMatch function. 2016-05-02 17:01:09 -07:00
Tim Abbott 4644967afc dropbox preview: Remove preview_fail.png error condition.
Since we don't have a stable way to get the Dropbox preview failure
image (and it was sorta a weird setup anyway), it seems best to just
remove the condition.
2016-04-29 15:40:52 -07:00
Tim Abbott 755695d3c0 bugdown: Add type: ignore for fenced_code import.
This is a workaround to allow us to type-check files that depend on
this.  Ideally in the future we'll fix the type errors in
fenced_code.py.
2016-04-28 12:50:47 -07:00
Tim Abbott 3e5ad69ffc Extract camo encoding to a library. 2016-04-27 22:23:40 -07:00
Varshit 4e1060076d Purge 'from typing import *' from zerver/.
This is a partial implementation of #636.
2016-04-07 14:07:07 -07:00
Tim Abbott 2059f650ab Add PEP-484 type annotations to zerver/lib/. 2016-04-03 15:40:23 -07:00
Tim Abbott 96eb81e5d5 Add documentation of Zulip's markdown implementation. 2016-04-01 09:56:59 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Eklavya Sharma 186efc6a6d Handle unicode properly in camo.
In Bugdown's InlineHttpsProcessor.run, hex_encoded_url was not being
decoded to utf-8. This was causing problems in python 3.

Refer to #256.
2016-03-19 15:46:07 -07:00
Tim Abbott df0d2a726d python3: Add missing utf-8 encoding/decoding in various places. 2016-03-08 09:14:15 -08:00
Tim Abbott df1670ef59 Fix various float initialization to use 0.0 instead of 0.
This is needed to type-check these values.
2016-02-03 19:29:07 -08:00
Tim Abbott 8654b57c7b bugdown: Move definition of current_node above set_text function using it. 2016-02-03 19:25:18 -08:00
Tim Abbott aad3bff193 Harden style rule for % comprehensions and fix existing errors. 2016-02-02 23:08:19 -08:00
Tim Abbott 5bacda3662 python3: Fix usage of .keys()/.values() to handle iterators.
This fixes the places where we use the result of .keys(), .items(),
and .values() that wouldn't work with an iterator to wrap them with
list().
2016-01-26 21:11:25 -08:00
Tim Abbott 6528b18ad3 Switch all urllib/urlparse usage to six.moves.urllib.
This provides Python 2+3 compatibility for our use of urllib.

Also add a test to avoid future regressions.
2016-01-26 21:09:43 -08:00
Tim Abbott f6f8f1fe36 Use new-style classes consistently for Python 3 support.
Also add the fixer for this to our list of fixers we check.
2015-11-04 08:01:52 -08:00
Tim Abbott f7878a61e1 Apply Python 3 futurize transform libmodernize.fixes.fix_xrange_six. 2015-11-01 09:35:06 -08:00
Tim Abbott 2ea0663a4a Apply Python 3 futurize transform libmodernize.fixes.fix_imports_six. 2015-11-01 09:26:16 -08:00
Tim Abbott 651b011514 Apply Python 3 futurize transform libmodernize.fixes.fix_basestring. 2015-11-01 09:26:16 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Darren Worrall 8b002040e0 Correct twitter library in requirements.
This also requires updating the required version of oauthlib; previously an
appropriate version was being installed only because it was a dependency of
the wrong twitter library.

This only affects development environments and/or hand-built
installations relying on the contents of requirements.txt.

To fix existing environments, the incorrect api needs to be explicitly
removed with `pip uninstall twitter`.

Fixes #86.
2015-09-30 13:49:33 -07:00
Darren Worrall bec3c0943a Fix validation that twitter creds are present.
They are looked up as secrets which initialize to `None`, but the code
was checking for empty strings.

This, along with #80, fixes #81.
2015-09-30 09:27:37 -07:00
Tim Abbott 3770142635 Revert "Host "preview failed" image locally."
This reverts commit 39f2908a32c0276b1d87ecedc876c71dd35a9b2f.

We're not including the preview_fail.png image in the release.

(imported from commit 2de1451de2f9b1727fc3a7e64c380b71c0f2caa8)
2015-09-25 00:44:20 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER.
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
2015-08-21 10:33:35 -07:00
David Roe 13f3b448e5 Kill Embed.ly integration.
(imported from commit dfe42a26e57100108c3c50298041bd520e0d8b1b)
2015-08-20 19:46:48 -07:00
Waseem Daher 569abda653 Host "preview failed" image locally.
Who knows when this Cloudfront URL will stop working!

(imported from commit 39f2908a32c0276b1d87ecedc876c71dd35a9b2f)
2014-07-29 21:46:35 -07:00
Paul Ruan 12cb8ceae0 Time out when waiting for OpenGraph links that take too long.
One common place that this happens (for us) is on a local
Dropbox .dev.corp.dropbox.com instance, which can't be reached
by the Zulip servers.

This commit also:
* Fixes the test suite
* Properly previews /photos/ links

(imported from commit b4788b6236e7a9d390e1efc4673be34d9ba5e091)
2014-07-29 21:46:35 -07:00
Jason Michalski 94ec1f3741 Better previews for dropbox shmodel.
Getting previews for docs using open-graph protocol images.

(imported from commit 11a9b446eeb5786fdbf536681a100f43ad6db978)
2014-07-29 21:46:34 -07:00
Steve Howell 8f37c3f5fd Tolerate spaces after language in markdown fences.
See #2357.  We now support `~~~ .py ` with that trailing space.

Note that the test coverage is Python-side only due to
bugdown_matches_marked being set to false, since we don't yet
support language syntax on the client side.

(imported from commit ccd5fcb0eee01478d349161400103480678d7486)
2014-03-06 10:15:20 -05:00
Waseem Daher a9f01e382a Switch streams page over to use consistent buttons.
(imported from commit c4d7f658f361a759a38951349e027ac3e279fe9b)
2014-03-05 17:23:23 -05:00
Leo Franchi 05d911b913 Allow auto-linked URLs to end with a !, without including the ! in the url
(imported from commit 26fbf8d8318dc32f31f94f366750521afab94ac5)
2014-03-05 15:22:05 -05:00
Waseem Daher 243c327d0a Properly preview images from Dropbox photo albums.
Reported by jhurwitz (sorta).

(imported from commit 8ae4c5f7af86c5492842142b89be8cab1a2c3fae)
2014-02-27 20:25:35 -05:00
Jason Michalski fa285fd1a9 Use case insensitive matching when linkifying tweets
(imported from commit c8a897f05a79a2b23998ffe186c77a9f7af4ae7f)
2014-02-15 15:33:22 -05:00
Luke Faraone 89eef10d44 Match alert words inside markup.
We now will match an alert word even if it is used at the boundry of
bolding, backtick escaping, or caret quoting.

Closes trac #2186.

(imported from commit 984bc63eb621772c95a01ca5c5bfeb190767f71f)
2014-02-13 17:14:54 -05:00
Luke Faraone 4ebc4a4ef1 Expand characters allowed after an alertword.
We now support .';[ before a word and ';]! after.

Closes trac #2283.

(imported from commit b3e3cb3d8d49a8e74d0b6a108c5ec73e9d9b5f32)
2014-02-13 17:14:53 -05:00
Tim Abbott 4891da83ad Fix autolinkification in Zephyr mirrored content.
Apparently the "inline" treeprocessor is what runs the inline
patterns.  Also re-enable the rewriting-to-https support.

(imported from commit 2fde2c1f15217a784f26b16db25ee745f424f2f0)
2014-02-10 13:23:27 -05:00
Leo Franchi 8ad2ca15cd bugdown: Match up to 8 groups of nested parens
(imported from commit e3d61ec644fdb293322cab0a4449d8e1a27a513a)
2014-02-06 17:26:01 -05:00
Tim Abbott ca3430862c More correctly disable markdown processing for Zephyr mirrored messages.
(imported from commit 688eead11c329e93c649ea1633dbf56328b5587b)
2014-02-06 10:25:35 -05:00
Tim Abbott 7151af5119 Fix disabling of markdown for mit.edu.
(imported from commit 952456ddab8112bf62e6d17e29325c6d974be7a5)
2014-01-29 14:39:53 -05:00
Jason Michalski a028bda264 Allow up to three twitter previews per message
(imported from commit 566b786f9ac0582f15f8f77363f63932ee5ceec8)
2014-01-29 11:00:41 -05:00
Jessica McKellar 086eab8bb5 bugdown: support modal in-app links in messages.
(imported from commit 0e3c0ff61cf9a84bae81e37b0a6d79831dea9224)
2014-01-23 17:06:29 -05:00
Jason Michalski c30a411c10 Add custom markdown tag to render a stream subscribe button
When new streams are created we now send a message with a custom
markdown tag that renders a subscribe button.

(imported from commit 9dfba280b3b4ff4f32f6431ef9227867c8bf4b40)
2014-01-22 11:28:31 -05:00
Waseem Daher fe748f7633 Make Twitter preview look like other inline image previews.
(imported from commit ca4ff9146679a5348bf53d06e992972e9a4b628a)
2014-01-10 21:39:04 -05:00
Jason Michalski 9e822403bb Linkify twitter media links
Image and video links in the twitter API are media and need to be
handed on separately. We also include a preview image if the media link
is a to a picture.

(imported from commit 2bd00d267e51b29ad0ba681195b2bfea9b991d8c)
2014-01-10 21:39:02 -05:00
Jason Michalski 1352dccea5 Add support to linkify links and mentions on twitter
This converts links in tweets to a tags. We also convert the displayed
text to the target of the twitter short URL. Mentions are linked to the
users twitter page.

(imported from commit 192d5546a7eea82759f9ae30d82c102aed15ff71)
2014-01-10 21:39:01 -05:00
Waseem Daher 14a5c6bf66 twitter: Detect link styles more robustly.
* Deal with shorter tweet IDs
  (some old tweets don't have a full 18-character ID)
* Allow trailing slash
* Deal with old-style #! syntax
* Deal with links that link to a photo

(imported from commit 008a98c806f3b8dddd9e2f18a8f002af6932766f)
2013-12-13 18:13:23 -05:00
Kevin Mehall ebab4a59c9 Don't proxy https images through Camo on MIT
These images at least load now, but that's because Camo redirects
the browser to the origin server, so the only effect is an extra
round-trip time.

(imported from commit 0d6b9c888a5cdfaa9299272d74a085e872dfa434)
2013-12-13 17:27:01 -05:00
Leo Franchi 2c39e28ff6 Use realm filters from the database in bugdown
(imported from commit 8e12bfb1b1fe9cad694f80a189efa64b0f102e81)
2013-12-11 14:39:10 -05:00
Kevin Mehall b8a6100d2e bugdown: Open #-relative links in the same window.
Stream notifications shouldn't leave you with tons of Zulip windows.

(imported from commit e5ebfeff51f1e579d30a3a12eaf7341ef82f0449)
2013-11-25 17:34:59 -05:00
Steve Howell ace12bcab6 Support arbitrarily nested fenced quote/code blocks.
Now we can nest fenced code/quote blocks inside of quote
blocks down to arbitrary depths.  Code blocks are always leafs.
Fenced blocks start with at least three tildes or backticks,
and the clump of punctuation then becomes the terminator for
the block.  If the user ends their message without terminators,
all blocks are automatically closed.

When inside a quote block, you can start another fenced block
with any header that doesn't match the end-string of the outer
block.  (If you don't want to specify a language, then you
can change the number of backticks/tildes to avoid amiguity.)

Most of the heavy lifting happens in FencedBlockPreprocessor.run().
The parser works by pushing handlers on to a stack and popping
them off when the ends of blocks are encountered. Parents communicate
with their children by passing in a simple Python list of strings
for the child to append to.  Handlers also maintain their own
lists for their own content, and when their done() method is called,
they render their data as needed.

The handlers are objects returned by functions, and the handler
functions close on variables push, pop, and processor.  The closure
style here makes the handlers pretty tightly coupled to the outer
run() method.  If we wanted to move to a class-based style, the
tradeoff would be that the class instances would have to marshall
push/pop/processor etc., but we could test the components more
easily in isolation.

Dealing with blank lines is very fiddly inside of bugdown.

The new functionality here is captured in the test
BugdownTest.test_complexly_nested_quote().

(imported from commit 53886c8de74bdf2bbd3cef8be9de25f05bddb93c)
2013-11-21 17:13:17 -05:00
Steve Howell 0d920dae87 Extract FencedBlockPreprocessor.placeholder().
(imported from commit cf6dab74ae97a57498e1e1807c7c4f272f3f448b)
2013-11-20 16:06:53 -05:00
Steve Howell 2955cac47e Extract vars in FencedBlockPreprocessor.process_fence().
(imported from commit 092a8e5f7479f1d918219cae058b6a867743c7f9)
2013-11-20 14:30:56 -05:00
Steve Howell b3344c2f49 Move langclass var into FencedBlockPreprocessor.format_code().
(imported from commit e468a8b6a12494facb0dfa97eff90e02cd14f169)
2013-11-20 14:30:55 -05:00
Steve Howell 46838ec6d0 Simplify FencedBlockPreprocessor.format_fence().
(imported from commit 741f4f0640756d545019b6fe6850af3198b351ba)
2013-11-20 14:30:55 -05:00
Steve Howell c0f312ec29 DRY up m.group('code') in FencedBlockPreprocessor.
(imported from commit b23caae1ab63a9adb80025b628bc1a7c6a988ce0)
2013-11-20 14:30:55 -05:00
Steve Howell 6bc25d3d7b DRY up m.group('lang') in FencedBlockPreprocessor.
(imported from commit 9f9ab5604b7df711a3a369247def1277880b0b5f)
2013-11-20 14:30:55 -05:00
Steve Howell 7bb30a96ac Extract FencedBlockPreprocessor.format_fence().
(imported from commit 8a35de7df9ad956496263c82668b118910edd6b3)
2013-11-20 14:30:55 -05:00
Steve Howell 64410fae5e Extract FencedBlockPreprocessor.format_quote().
(imported from commit 59db5415d624402b37ade7fe5547b16f58ae565f)
2013-11-20 14:30:55 -05:00
Steve Howell 9d4b6d0684 Extract FencedBlockPreprocessor.format_code().
(imported from commit 3c98b9a7391a71e9c55a7b5c97abd0ee74178b44)
2013-11-20 14:30:54 -05:00
Kevin Mehall fe0dcd4313 Disable camo on enterprise.
CUSTOMER13 doesn't want it, and there's currently no nginx config
or configurable Camo URI, so it wouldn't work if image preview
were enabled.

(imported from commit 615d4a32acbc4d4d590f88cf4e7d45d8f49db1d3)
2013-11-15 14:27:16 -05:00
Kevin Mehall b469066e66 Fix emoji on enterprise.
static/ doesn't exist on enterprise, so we can't get the list from the
files there.

(imported from commit ce34a62478abf541feb013da4f970dac3c09d98a)
2013-11-14 11:18:23 -05:00
Tim Abbott 870ae09aae Add setting for disabling inline image preview.
(imported from commit 2321390eb14cfe5701347861b9ae5ad6fdac0cbb)
2013-11-14 08:47:11 -05:00
Steve Howell 761c0a0266 Support !avatar syntax in bugdown.
(imported from commit 72551c5df96f5b383d3593c9618cc82413d5d95d)
2013-11-13 13:45:09 -05:00
Steve Howell 2086ee8979 Rename Gravatar class to Avatar in bugdown.
(imported from commit 97f1fb52f2582694f5ba43c24649fc58152d3486)
2013-11-13 13:45:09 -05:00
Steve Howell 2d5a19ad1b Have !gravatar markdown serve up /avatar/<email>.
The !gravatar markdown no longer hard codes to Gravatar, but
instead it serves up our generic avatar URL.

(imported from commit 4e3e2baeb3374bcf025a18ff27a8452b975c22b7)
2013-11-13 13:45:08 -05:00
Leo Franchi db6550e99a Make built-in bots come from settings.py, and allow localserver-specific ones
(imported from commit e21933e37487314ac986147562817a19227e8960)
2013-11-01 14:13:05 -04:00
Waseem Daher db1f5065a7 Add tooltip for, e.g. !gravatar(wdaher@zulip.com) output.
(imported from commit eff24a2fc7da77edc26449074acf75832d368a17)
2013-10-30 13:29:37 -04:00
Leo Franchi e1557bef73 Cache realm user basic info dict that is used in bugdown
(imported from commit 688c47daceb73534be90bd98a031c7b4edb5546e)
2013-10-28 12:59:36 -04:00
Luke Faraone 52309b5789 Enable absolute imports for non-third-party code where it was omitted previously
(imported from commit 34856ac32c6b94b614273a6fe46c87a314058d9b)
2013-10-25 14:13:30 -04:00
Kevin Mehall 5e1caad88a Show upload filename in tooltip without other ugly parts of the URL.
This works for both public and authenticated uploads.

(imported from commit f9be340b53d0f72bf5b3c31a5d51a1fad80759be)
2013-10-24 17:01:06 -04:00
Kevin Mehall e3b2e7ece2 Allow domain-relative and fragment links
(imported from commit 0a15eeb35ad709977f3b2e6c550c3b78f1b4b402)
2013-10-24 17:01:05 -04:00
Leo Franchi a2a1867f25 Only render tweets if we have credentials
(imported from commit 46c59f430aeae87a832f33d2ccb85209f9caec74)
2013-10-24 10:19:23 -04:00
Tim Abbott b5fbf67511 bugdown: Fix support for python-markdown >= 3.2.1.
(imported from commit c3c61ad3f75ef0019517084278e34641998a7e22)
2013-10-22 14:44:46 -04:00
Tim Abbott b340add1aa bugdown: Disable linebreaks from two spaces at end-of-line.
Arguably the nl2br extension should be doing this for us.  Given that
we're using nl2br, the "two spaces at the end of a line makes a line
break" rule doesn't make any sense (since every newline leads to a
linebreak), so we disable it.

(imported from commit 5ffa2ac8a825642ad31e085c532091e076665710)
2013-10-22 14:04:11 -04:00
Kevin Mehall 71decdbe7a Fix code block auto-closing.
Trac #1162

The process_fence method replaces code blocks with placeholders, so
indexes stored before the replacement are incorrect. However, because
the closed code blocks have been replaced, we can simply search the
whole string for any remaining opening code block markers.

(imported from commit 6a9e6924840f8f3ca5175da7c52a905e27c1fabd)
2013-10-16 10:12:33 -04:00
Kevin Mehall b134c90b6b Enable backtick fenced code blocks.
Trac #1900

(imported from commit 47b3a76488a4285641fd1eb3e68bc72047a8d738)
2013-10-16 10:12:33 -04:00
Leo Franchi 1f89bf386f Parse markdown tables and show them with some styling
(imported from commit fb3c599b1dbaed2447f1e710ed7202486000ca2a)
2013-10-10 16:39:11 -04:00
Leo Franchi 6e188fb067 Tweak alert_word matches and add some more tests
(imported from commit 63dc4064c15d5d33a7ec0c992b183bf323dd4ee8)
2013-10-10 10:58:21 -04:00
Leo Franchi 08ae641dd2 Pre-fetch data from the DB and hand to markdown thread
We want to avoid opening a DB connection in the markdown thread
as its DB connection might live for a long time

(imported from commit 7700b2ca793ee5e9add7f071b92f22a4bf576b3d)
2013-10-10 10:58:21 -04:00
Tim Abbott b8b0eb4508 Some zulip => humbug updates in comments and print statements.
(imported from commit 9253569a1df7f96fda81ab162d710cdda03f30ca)
2013-10-08 08:57:29 -04:00
Kevin Mehall 72a0b67c80 Prevent other markdown operators from touching URLs.
The text of manual links are already AtomicStrings, so linkified strings
should be too.

Moves emoji detection to happen after linkification, so the emoji rule
won't look at links.

(imported from commit 9c56bce6a0e873b398255e0762dfb312a4a9a64e)
2013-10-03 17:26:47 -04:00
Kevin Mehall 8842349629 Fix links with invalid emoji in them.
InlinePatterns should return None on failure, not text that may
have placeholders in it.

(imported from commit f9d8d22b2b8cfa7a92ecf3e52a6c76b48e6f0175)
2013-10-03 17:26:47 -04:00
Tim Abbott 8a5fffa7cf Move twitter API credentials to local_settings.py.
(imported from commit 6b95db113b91816fbc5e91db4b1be90d3df8e028)
2013-09-25 15:40:21 -04:00
Kevin Mehall 8c7f1ecffa bugdown: Don't generate garbage on invalid links.
LinkPattern returned a string which contained a placeholder if the URL was
considered invalid. AtomicLinkPattern wrapped this in an AtomicString,
where the placeholder doesn't get removed properly.

m.group(0) is always incorrect because python-markdown modifies your regex
to include more than you specified (this is why part of the message got
duplicated).

(imported from commit 576bdf09c2b677cf4bc56484c363eb05f2110158)
2013-09-18 16:22:13 -04:00
Leo Franchi a67d461139 Hoist joining of message content in alert_words processor
(imported from commit 5025b7108839ddc04d6d9e575a9a288223526fe8)
2013-09-06 16:05:08 -04:00
Leo Franchi 7d76c47321 Re-enable alert word bugdown processor for staging
(imported from commit a9fb8284a47d18b97b84d3ce37482132b21d3885)
2013-09-06 14:51:40 -04:00
Leo Franchi 7b375634f9 Temporarily disable alert_words markdown processor on staging
(imported from commit ab3eed4ce36ebcd3d928d2d92fc34c9ebacc11f9)
2013-09-06 13:39:26 -04:00
Zev Benjamin cb78014eef bugdown: Allow block-level block quotes
It is triggered by specifying the "language" of a code block to
"quote" or "quoted":
    Hamlet said:
    ~~~ quote
    To be or **not** to be.

    That is the question
    ~~~

(imported from commit 847a0602e335e9f2955e32d9955adf8ac8de068c)
2013-09-05 17:34:28 -04:00
Zev Benjamin dd26260884 bugdown: Clarify variable name
(imported from commit 27968536bd6a3fcda6d26d8dde7f82d9f0990dd2)
2013-09-05 17:34:28 -04:00
Zev Benjamin 2f1551e745 Replace comment references to trac.humbughq.com with trac.zulip.net
(imported from commit 231d0884345be525d58ce51f48a76bba6960f09d)
2013-09-05 15:43:08 -04:00
Zev Benjamin 5c81148839 Switch our trac realm filter to use trac.zulip.net
(imported from commit 19163f5be01a1d9370d9ed45c55bedd670772dac)
2013-09-05 15:43:07 -04:00
Tim Abbott ce710292ca Always use external-content.zulipcdn.net for mit.edu realm.
(imported from commit 21d1101185bf52f21fcc345e791217311e9c0a13)
2013-09-05 14:35:42 -04:00
Tim Abbott b557b94d0b bugdown: Rewrite image urls to avoid mixed-content warnings.
(imported from commit fc0a41befb04f2a8aad2937a856366ac3cadb192)
2013-09-05 14:35:29 -04:00
Tim Abbott cdfc6dc10a bugdown: Fetch youtube preview images over HTTPS.
(imported from commit 9dbf3c21083cbbcf1288789f400c33ec735d4416)
2013-09-05 14:35:29 -04:00
Leo Franchi b2ddd670e4 [schema] Add backend support for per-user alert words
(imported from commit 7a9c596a010cbedbddf594c5d9c68bb9ed46d122)
2013-09-05 10:18:40 -04:00
Tim Abbott 591b42bb1c bugdown: Clarify code around inline_interesting_links processor.
(imported from commit a1d734230135ab9099fafea7814bfec25754ea4e)
2013-08-28 18:56:53 -04:00
Kevin Mehall d8146141a9 Render realm emoji.
(imported from commit 5a8dca8963bc6e2e16762844aebaefcb85a6aad2)
2013-08-23 21:39:29 -04:00
Waseem Daher abef6be12f bugdown: Unescape HTML entities in Twitter.
This closes Trac #1071.

(imported from commit 521ffa6ed13df7c08e67218564d426ca90080948)
2013-08-12 17:23:49 -04:00
Tim Abbott 1f1af26048 Change Humbug => Zulip in text/comments.
(imported from commit 2f9d73431ae40e1b9e9e11bc2f4f62f566ae758a)
2013-08-07 10:00:07 -04:00
Steve Howell 9f9c7c3e3b Loosen up emoji regex. (i.e. More emojis! 👍)
Now parsed: 🍺,🍺;🍺!

 If \w characters surround :foo:, we still say it's NOT an
 emoji, but we used to do this for \S characters, so it's loosened up.

(imported from commit 49b33d2f0ffdcfde8947ae411a4addcf4c24af9c)
2013-08-06 18:20:30 -04:00
Tim Abbott e111a2f9a5 [manual] Rename Django app from zephyr to zerver.
This needs to be deployed to both staging and prod at the same
off-peak time (and the schema migration run).

At the time it is deployed, we need to make a few changes directly in
the database:

(1) UPDATE django_content_type set app_label='zerver' where app_label='zephyr';
(2) UPDATE south_migrationhistory set app_name='zerver' where app_name='zephyr';

(imported from commit eb3fd719571740189514ef0b884738cb30df1320)
2013-08-06 07:39:36 -04:00