Commit Graph

203 Commits

Author SHA1 Message Date
Steve Howell 4372bfb4a5 tests: Fix send_message calls in test_bugdown.py. 2017-10-28 10:20:59 -07:00
Steve Howell 87b48f722d Add MentionData.get_user_ids().
This prepares us to take advantage of the user_ids
in subsequent commits related to `do_send_messages()`.
2017-10-26 22:16:47 -07:00
Steve Howell e0bc1b114e Make sure mentions refer only to active users.
An active user can share the same full name as a deactivated
user.  We now only allow mention syntax to find users who are
activated.

Fixed #6978
2017-10-12 17:11:36 -07:00
Tim Abbott 234f7960a1 tests: Add @slow decorator to various methods needing it.
Also, fix the test_message_editing test having way too many cases.

Fixes #1478.
2017-10-06 15:45:04 -07:00
Harshit Bansal 5b5bcce098 emoji: Migrate bugdown emoji to use sprite sheets.
This commit switches to use sprite sheets for rendering emojis
in all the remaining places, i.e., message bodies and composebox
typeahead. This commit also includes some changes to notifications.py
file so that the spans used for rendering emojis can be converted
to corresponding image tags so that we don't break the emoji rendering
in missed message emails since we can't use sprite sheets there.

As part of switching the bugdown system to use sprite sheets, we need
to switch the name_to_codepoint mappings to match the new sprite
sheets.  This has the side effect of fixing a bunch of emoji like
numbers and flag emoji in the emoji pickers.

Fixes: #3895.
Fixes: #3972.
2017-09-29 11:14:34 -07:00
rht 26f5d9a32c zerver/tests: Remove print_function. 2017-09-27 18:05:45 -07:00
rht daef6a7dbc zerver/tests: Remove `import six`. 2017-09-27 17:07:40 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
Steve Howell 7159f34bed Avoid getting realm emoji unless there is emoji syntax. 2017-09-15 01:09:08 -07:00
Steve Howell 8a941d03ae Avoid querying streams unless there are stream links. 2017-09-15 01:09:08 -07:00
Steve Howell c3032a7fe8 Avoid looking up emails when rendering messages.
We now fetch email -> id mappings for messages ONLY if
it potentially uses the !avatar(foo@example.com) syntax.
2017-09-15 01:09:08 -07:00
Steve Howell 4e7fce60ee Add possible_mentions() to speed up rendering.
We now triage message content for possible mentions before
going to the cache/DB to get name info.  This will create an
extra data hop for messages with mentions, but it will save
a fairly expensive cache lookup for most messages.  (This will
be especially helpful for large realms.)

[Note that we need a subsequent commit to actually make the speedup
happen here, since avatars also cause us to look up all users in
the realm.]
2017-09-15 01:09:08 -07:00
Steve Howell ba397b5109 Use user_ids, not full objects, in render path.
There is no reason for either render_incoming_message() or
render_markdown() to require full UserProfile objects just to
triage alert words.

By only asking for user_ids, we save extra queries in two
callpaths and we make it easier to start using user_ids in
do_send_messages().
2017-09-12 04:22:55 -07:00
Tim Abbott 8600efcf36 test_bugdown: Fix duplicated test name. 2017-08-16 11:29:12 -07:00
Tim Abbott d2288154f6 bugdown: Fully remove mentions matching @name.
Given typeahed and the fact that this only worked if the person had a
full name that didn't contain whitespace, this side effect of the
original @shortname mentionfeature that we removed was experienced by
users as a bug.

Fixes #6142.
2017-08-16 11:18:09 -07:00
Umair Khan 0e119c0ec2 test_bugdown: Upgrade to HttpResponse.json(). 2017-08-16 07:06:58 -07:00
Tim Abbott 4b7c23f133 markdown: Rename bugdown-data.json to a more reasonable name. 2017-07-28 17:54:01 -07:00
Harshit Bansal 4737363f0c test_bugdown: Migrate tests to use the default test realm emoji. 2017-07-27 18:59:02 +00:00
Tim Abbott c9b2ef82c9 test_bugdown: Fix test for Twitter emoji.
Our work on the Twitter emoji added a new instance of our emoji syntax
between when we wrote 2d92f9dc0b and
rebase+merged it.
2017-07-26 10:20:25 -07:00
digi0ps 2d92f9dc0b emoji: Change emoji image titles to use spaces instead of underscores.
Previously, the emoji images title display `🍼`.
This commit changes the title to display `baby bottle`.
2017-07-26 09:27:17 -07:00
Harshit Bansal a03ce01688 bugdown: Fix the rendering of emojis in twitter link previews.
Process the unicode emojis in twitter link previews and render them
properly. Before this we were not processing the unicode emojis in
twitter link previews and hence on the systems which don't have
fonts for displaying them they were rendered as blank boxes.

Fixes: #5427.
2017-07-24 17:32:05 -07:00
Harshit Bansal 127d1f995d test_bugdown.py: Add test for `url_to_a()`. 2017-07-24 17:31:08 -07:00
Aditya Bansal ee97ea1356 pep8: Add compliance with rule E261 to tests/test_bugdown.py. 2017-07-11 15:17:32 -07:00
Harshit Bansal 4cb8ac100a emoji: Fix the title and alt text for unicode emojis.
Unicode emojis when rendered should display canonical short name.
Similarly, the alt text should be of the format `:<short_name>:`.
For both of these we currently display the actual unicode symbol.
As some systems don't have the fonts necessary for displaying them
properly, they are rendered as empty square blocks. This commit also
ensures that the markup generated for emoji generated by canonical
name and by an unicode emoji is same.

Fixes: #5555.
2017-07-01 11:14:24 -07:00
Harshit Bansal dfbf8b8086 bugdown: Remove shortname mention syntax.
We used shortnames for mentioning users before we had autocomplete
feature. Since we now have autocomplete typeahead, this syntax is
no more useful and just causes problems. This commit removes the
shortname mention syntax.

Fixes: #4189.
2017-06-19 16:57:59 -04:00
Cory Lynch 552c39a036 fenced_code.py: Improve test coverage for special chars. 2017-06-19 06:53:25 -04:00
Harshit Bansal 7950f82f27 realm_emoji: Make sure that a deactivated realm emoji can't be used.
A deactivated realm emoji should neither be accepted further as a
reaction nor its further occurences in a message be rendered as an
emoji. However, all the old occurences should continue to render
normally.
2017-06-15 02:59:57 -07:00
Tommy Ip 8d6af030c0 bugdown: Fix modal_link.
The `data-toggle` property prevented the new style of overlay modals
from launching, and regardless, isn't a future-proof options for how
this should work.
2017-06-03 18:41:19 -07:00
Vishnu Ks bb98e35aa0 Replace othello@zulip.com with example_email('othello'). 2017-05-24 19:37:36 -07:00
Vishnu Ks 5230eaef1c Replace hamlet@zulip.com with example_email('hamlet'). 2017-05-24 19:37:36 -07:00
Tim Abbott e5f5e5c69f tests: Remove unnecessary maxDiff settings.
Now that ZulipTestCase does this, we don't need it in all these
individual test modules.
2017-05-24 12:43:28 -07:00
umkay ccc70445d6 mypy: Fix strict-optional errors for test files.
Fix mypy --strict-optional errors in zerver/tests
2017-05-24 12:43:28 -07:00
Vishnu Ks 075b0cae70 test_bugdown.py: Remove unused get_user_profile_by_email import. 2017-05-23 15:27:21 -07:00
Vishnu Ks c680c5f1e8 Replace sipbtest@mit.edu with mit_user('sipbtest'). 2017-05-22 19:02:42 -07:00
Rohitt Vashishtha 0414ac6df5 bugdown: Convert GitHub image-preview urls to image urls.
This makes inline image previews work for links to image files' pages
on GitHub.

Fixes #4658.
2017-05-09 11:22:37 -07:00
Steve Howell 942db9b6c5 tests: Added ZulipTestCase.example_user() function.
The example_user() function is specifically designed for
AARON, hamlet, cordelia, and friends, and it allows a concise
way of using their built-in user profiles. Eventually, the
widespread use of example_user() should help us with refactorings
such as moving the tests users out of the "zulip.com" realm
and deprecating get_user_profile_by_email.
2017-05-08 11:57:38 -07:00
K.Kanakhin f13d6a18eb realm-emoji: Add realm emoji uploading instead url providing.
- Add file_name field to `RealmEmoji` model and migration.
- Add emoji upload supporting to Upload backends.
- Add uploaded file processing to emoji views.
- Use emoji source url as based for display url.
- Change emoji form for image uploading.
- Fix back-end tests.
- Fix front-end tests.
- Add tests for emoji uploading.

Fixes #1134
2017-05-01 14:50:20 -07:00
Sampriti Panda 0e3d694df8 bugdown: Use queue for processing of links in image previews.
Earlier, a stack was being used to go through the message and search
for links.  Because of this, in some cases the images were added to
the preview in reverse.  Using a queue will keep the image previews in
the same order as they appeared in the message.

Fixes #4453.
2017-04-17 20:48:16 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Ayush Jain bddcfb1c96 Add realm-level settings to control inline image and url preview.
This gives users more control in case they don't want previews,
especially for the "previews of linked websites" feature.

Fixes: #2640.
2017-03-21 15:46:17 -07:00
Rishi Gupta 0559afb928 models: Use Realm.string_id instead of domain in RealmFilter.__unicode__. 2017-03-13 09:27:21 -07:00
Tim Abbott 39704bac6c test_bugdown: Remove unused common_bugdown_test. 2017-03-04 23:35:46 -08:00
Rishi Gupta 1453a5bfda Change string_id of test zephyr realm from mit to zephyr.
Also changes Realm.is_zephyr_mirror_realm to use string_id=zephyr instead of
domain=mit.edu.
Part of a larger migration away from Realm.domain.
2017-03-04 12:18:01 -08:00
Igor Tokarev 31dff09efa Support email changes for !avatar syntax.
Significantly modified by tabbott to avoid calling
get_user_profile_by_email in bugdown, and have 100% test coverage of
the views code.

Fixes #2041.
2017-02-28 21:56:04 -08:00
Steve Howell 5b8cc03c61 bugdown: Restore data-user-email to user mention spans.
(The commit q7ef4e40258280e202325c9295579c93fb948b replaced
data-user-email with data-user-id, but we still need to
support data-user-email for old clients like non-updated
androids and we still want to start the migration forward
to data-user-id.)
2017-02-02 12:32:18 -08:00
Tim Abbott 700089f386 bugdown: Fix need for Internet in test_inline_dropbox_bad. 2017-01-29 22:07:14 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Tim Abbott 9cc83f87fc lint: Clean up E241 PEP-8 rule. 2017-01-23 21:21:14 -08:00
Steve Howell 1137ef4e40 Mark up mention buttons with data-user-id on the backend.
We use data-user-id now in the buttons for at-mentions when
we render messages.  A previous commit already made the front
end support this new style.
2017-01-21 21:45:12 -08:00
Tim Abbott 5d52f1ec17 bugdown: Move realm_filters_key logic out of callers.
This gets rid of the confusing duplicate realm_filters_key and
message_realm arguments that previously were passed to bugdown.
2017-01-21 21:37:57 -08:00
Tim Abbott 19b89eb050 bugdown: Rename realm_id to realm_filters_key.
This should substantially improve the clarity of the code, since
inside bugdown, this is only being used as a hash key that happens to
usually be a realm ID, not used as a Realm ID.
2017-01-16 21:48:55 -08:00
Sampriti Panda 196cf4367b urls: Move /messages/render to POST endpoint 2017-01-13 16:11:51 -08:00
Rishi Gupta 494c1a2b55 Remove unnecessary uses of Realm.domain in zerver/tests. 2017-01-09 11:26:08 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rishi Gupta cf762eaf84 Change X.realm.id to X.realm_id across codebase.
This makes it more clearly the pattern in the Zulip codebase, and thus
decreases the risk of accidentally doing database queries.
2017-01-03 16:46:26 -08:00
Rishi Gupta c6e12e74be Change domain to realm_id in bugdown and realm filter dicts and caches. 2017-01-03 16:25:20 -08:00
Tim Abbott 39f0ffdedd Move static/third/gemoji to static/generated/emoji.
Fixes #1153.
2016-12-27 20:16:23 -08:00
Tim Abbott bf80873d4f tests: s/assertRaisesRegexp/assertRaisesRegex/ due to deprecation. 2016-12-15 17:11:42 -08:00
Igor Tokarev c93f1d4eda Add oembed/Open Graph/Meta tags data retrieval from inline links.
This change adds support for displaying inline open graph previews for
links posted into Zulip.

It is designed to interact correctly with message editing.

This adds the new settings.INLINE_URL_EMBED_PREVIEW setting to control
whether this feature is enabled.

By default, this setting is currently disabled, so that we can burn it
in for a bit before it impacts users more broadly.

Eventually, we may want to make this manageable via a (set of?)
per-realm settings.  E.g. I can imagine a realm wanting to be able to
enable/disable it for certain URLs.
2016-12-07 17:40:18 -08:00
bulat22101 9f68efa47b mypy: Convert zerver/tests/ to use typing.Text. 2016-12-04 14:47:21 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
Sidhant Bhavnani 8c0c12c1d9 pep8: Fix E303 violations. 2016-12-02 15:34:11 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Rafid Aslam 7a2282986a pep8: Fix E225 pep8 violations. 2016-11-28 15:21:15 -08:00
Vladislav Manchev d7e1e4a2c0 Add initial implementation of custom realm filters.
This PR was abandoned by Vladislav and then substantially modified by
Igor Tokarev and Tim Abbott to complete it and fix a number of bugs.

Fixes #544.
2016-11-17 17:11:25 -08:00
reyha 9f8630fed8 tests: Access realms by string_id.
Replaces get_realm() function by get_realm_by_string_id() function
in zerver/tests.

Fixes #2226.
2016-11-15 13:32:20 -08:00
sinwar 4582a98c09 tests: Split out ZulipTestCase and WebhookTestCase to a separate file.
Fixes #1671.
2016-11-10 19:29:43 -08:00
Tim Abbott 3caee213e1 test_render_mention_stream_api: Fix nondeterministic failures.
This previously dependended on the stream ID for Denmark, which is not
a constant across all installations.
2016-11-08 14:13:48 -08:00
Tim Abbott 5f70b15a00 test_bugdown: Fix duplicate test name. 2016-11-08 14:09:53 -08:00
Mohsen Ibrahim e0351948ba render_message_backend: Pass realm data into bugdown.
Previously, the way that render_messages was calling bugdown meant
that the preview feature didn't have access to realm data like the
list of users or streams, resulting in previews for those elements
being wrong.

Now render_message_backend uses zerver.lib.render_markdown to render
messages correctly.

[Commit message tweaked and test added by tabbott]
2016-11-08 13:43:03 -08:00
Kevin Chen 6107c877e8 bugdown: Add option to support "file:///" as hyperlink.
This contains contributions from Tim Abbott and Igor Tokarev.

Fixes #380.
2016-11-06 22:30:05 -08:00
Tim Abbott bf1a7a7f2d test_bugdown: Fix misplaced type annotation. 2016-11-01 20:40:22 -07:00
paxapy 1d1c41551b bugdown: Auto-linkify #StreamName. 2016-11-01 19:59:21 -07:00
Brock Whittaker 77233c7e3b bugdown: Distinguish between YouTube Videos and Image Previews.
This distinguishes between YouTube Videos and Image Previews by adding
a particular “youtube-video” class to the preview along with changing
the title to the video ID rather than the link. This serves to allow
the lightbox to ID when a lightbox preview should be treated like a
YouTube video rather than an image preview.

This also modifies the tests in bug down to expect a youtube-video class
along with the title to just be the video ID on YouTube rather than the
entire URL link.
2016-10-18 14:45:47 -07:00
Eklavya Sharma 1dbabfe1ab Annotate zerver/tests/test_bugdown.py. 2016-10-11 11:27:35 -07:00
Eklavya Sharma 8e2f659f3d test_bugdown: Add trivial annotations.
Add annotations of the form `type: () -> None` to test methods.
2016-10-11 11:27:35 -07:00
Steve Howell b2f84f0fa4 Move render_markdown() to lib/message.py.
This removes a bugdown circular dependency.
2016-10-04 11:34:53 -07:00
Steve Howell fc2329ccb7 tests: Add test_is_status_message(). 2016-10-04 10:35:43 -07:00
Brock Whittaker 98937ac539 bugdown: Set "title" attribute to actual image title when available.
This sets the “title” attribute on the image to the actual title of
the image specified by the user in their markdown, rather than just
the URL of the full link to it.
2016-09-27 17:34:05 -07:00
Steve Howell 0dddb9c877 tests: Add test_flush_realm_filter(). 2016-09-24 16:56:34 -07:00
Steve Howell d28c3d08a8 tests: Add test_maybe_update_realm_filters(). 2016-09-24 16:56:34 -07:00
Steve Howell e0dc6092d5 tests: Call flush_per_request_caches() in test_realm_patterns().
This is a slightly more realistic test, and it adds some test
coverage.
2016-09-24 16:56:34 -07:00
Steve Howell a04a095738 Speed up alert word detection during message sends.
We no longer use all the alert words for all the users in the
entire realm when we look for alert words in a newly sent/edited
message.  Now we limit the search to only all the alert words
for all the users who will get UserMessage records.  This will
hopefully make a big difference for big realms where most messages
are only sent to a small subset of users.
2016-09-23 15:24:55 -07:00
Steve Howell 40b18094ec alert words: Refactor alert word detection.
The bugdown parser no longer has a concept of which users need which
alert words, since it can't really do anything actionable with that info
from a rendering standpoint.

Instead, our calling code passes in a set of search words to the parser.
The parser returns the list of words it finds in the message.
Then the model method builds up the list of user ids that should be
flagged as having alert words in the message.

This refactoring is a little more involved than I'd like, but there are
still some circular dependency issues with rendering code, so I need to
pass in the rather complicated realm_alert_words data structure all the way
from the action through the model to the renderer.

This change shouldn't change the overall behavior of the system, except
that it does remove some duplicate regex checks that were occurring when
multiple users may have had the same alert word.
2016-09-23 15:21:17 -07:00
Steve Howell a67d17b9f7 tests: Add coverage for RealmFilter.__unicode__(). 2016-09-19 14:57:53 -07:00
Steve Howell 7d4995ded0 tests: Add test_render_message_api(). 2016-09-19 14:25:21 -07:00
Tim Abbott 2c19719cee test_bugdown: Fix i18n test flakiness with markdown rendering tests.
It appears that the assertRaisesRegexp approach we had before didn't
work properly on some systems, likely due to a bad interact with a
i18n (we haven't definitively determined the cause).
2016-09-16 11:49:52 -07:00
Steve Howell a036a72db6 bugdown: Raise BugdownRenderingException.
We now raise an exception in bugdown.do_convert() if rendering
fails, to avoid silent failures, and then calling code can convert
the exception to a JsonableError.
2016-09-16 10:01:55 -07:00
Steve Howell de25f07961 tests: Add BugdownErrorTests. 2016-09-16 10:01:55 -07:00
umkay 6d4f86f951 Add explanation for test_mit_rendering. 2016-09-13 21:23:57 -07:00
Tim Abbott 70543e059a bugdown: Remove hardcoding of mit.edu for zephyr_mirror realm config. 2016-07-26 20:30:12 -07:00
Eklavya Sharma 3ee15feeb8 Make twitter tests more readable. 2016-07-17 10:23:25 -07:00
Eklavya Sharma c389d22b5f Fix twitter mock data. 2016-07-17 10:23:25 -07:00
Alex Wilson 5ec29101eb Add unicode emoji support to bugdown.
Fixes half of #1011.
2016-06-30 15:48:02 -07:00
Tim Abbott 2ec0114079 test_bugdown: Add tests for mentions. 2016-06-03 23:18:39 -07:00
Tim Abbott cfff4f1d49 test_bugdown: Add a bugdown test for alert_words functionality. 2016-06-03 23:18:39 -07:00
Tim Abbott 8c757292cf test_bugdown: Add test for nonmatching realm filters. 2016-06-03 23:18:39 -07:00
Tim Abbott 68fba3579d test_bugdown: Add testing of subject_links feature. 2016-06-03 23:18:39 -07:00
Tim Abbott 5bd94c15c7 Use camo to avoid mixed content warnings when displaying emoji. 2016-05-02 17:21:31 -07:00
Tim Abbott 4644967afc dropbox preview: Remove preview_fail.png error condition.
Since we don't have a stable way to get the Dropbox preview failure
image (and it was sorta a weird setup anyway), it seems best to just
remove the condition.
2016-04-29 15:40:52 -07:00
Tim Abbott 4be3c4afd6 Use mocks so we can re-enable Dropbox integration tests. 2016-04-29 15:27:43 -07:00
Vladislav Manchev f5e6176aea Add custom realm emoji UI to administration page. 2016-04-26 13:15:54 -07:00
Tim Abbott be96cf809d Move Zulip backend tests to zerver.tests. 2016-04-11 22:16:09 -07:00