Commit Graph

60 Commits

Author SHA1 Message Date
Harshit Bansal f6fe5d6266 build_emoji: Migrate to python3 type annotations. 2018-07-13 21:19:25 +05:30
Harshit Bansal f636882e04 build_emoji: Migrate to use `emoji_names.py` file.
This migrates Zulip to use a dramatically better set of names and
aliases for our emoji set, defined in emoji_names.py (which is in turn
manually generated from our hand-curated CSV file).

This should significantly improve the experience of using Zulip's
emoji picker and emoji typeahead for finding what one is looking for.
2018-07-13 21:18:02 +05:30
Tim Abbott 7a86e83a6f build_emoji: Use with for writing to files.
This is a nice code cleanup.
2018-07-13 21:07:08 +05:30
Tim Abbott 85243c525d build_emoji: Pass around emoji_map data structure less.
It wasn't used in either of these functions.
2018-07-13 21:04:37 +05:30
Harshit Bansal 40958e0824 emoji: Switch to 64px 256 color indexed sprite sheets.
This commit switches our emoji infrastructure to use 256 color indexed
64px spritesheets. Earlier we were using non-indexed 32px spritesheets
which were blurry on high dpi displays. These indexed spritesheets not
only provide a crispier display but are also smaller in size.

This commit also removes the `emoji-datasource` package as a dependency
as all the data is now sourced from individual datasource packages.

Fixes: #7862.
2018-03-14 10:28:45 -07:00
Harshit Bansal 40fe6239c5 build_emoji: Migrate to python3. 2017-11-20 15:30:58 -08:00
David Rosa Tamsen 7072fa5b37 docs: Reorganize developer docs to improve navigation.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.

This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.

Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).

Fixes #5265.
2017-11-16 09:45:08 -08:00
rht f397e15128 tools: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
Harshit Bansal 1067cfd92a node_modules: Add npm packages for the supported emojisets.
This commit adds emoji datasource packages for the rest of the emojiset
packages which we will use for serving images in notifications.
2017-10-28 10:38:32 -07:00
Harshit Bansal ea5cb23c8e build_emoji: Extend emoji mappings to use new emojis from iamcal dataset.
Emojis which are represented by a sequence of codepoints or emojis
with ZWJ are not included until we implement a mechanism for dealing
with their unicode versions.

Fixes: #6279.
2017-10-04 23:09:14 -07:00
Harshit Bansal 521ac31d3d build_emoji: Remove the redundant `codepoints` list.
This list contained a list of codepoints of all the emojis. We now have
`codepoint_to_name` in emoji_codes.js and hence this is not required.
2017-09-29 11:17:06 -07:00
Harshit Bansal f6c621dab5 build_emoji: Remove the hack used for displaying number emojis correctly.
This hack was used to fix the broken number emojis in the emoji picker.
It was broken because of the partial migration to the iamcal dataset.
See issue #4775 for more details.
2017-09-29 11:17:06 -07:00
Harshit Bansal 0940d2cf85 build_emoji: Remove the hack used for displaying flag emojis correctly.
This hack was used to fix the broken flag emojis in emoji-picker.
It was broken due to the incomplete migration to iamcal dataset.
See issue #4775 for more details.
2017-09-29 11:17:06 -07:00
Harshit Bansal 5b5bcce098 emoji: Migrate bugdown emoji to use sprite sheets.
This commit switches to use sprite sheets for rendering emojis
in all the remaining places, i.e., message bodies and composebox
typeahead. This commit also includes some changes to notifications.py
file so that the spans used for rendering emojis can be converted
to corresponding image tags so that we don't break the emoji rendering
in missed message emails since we can't use sprite sheets there.

As part of switching the bugdown system to use sprite sheets, we need
to switch the name_to_codepoint mappings to match the new sprite
sheets.  This has the side effect of fixing a bunch of emoji like
numbers and flag emoji in the emoji pickers.

Fixes: #3895.
Fixes: #3972.
2017-09-29 11:14:34 -07:00
Harshit Bansal 995a0f3cc0 build_emoji: Remove an unnecessary check for symlink support.
We already check for symlink support in provisioning script so no
need of doing so here.
2017-09-24 04:51:33 -07:00
Harshit Bansal 0d8e68d75e build_emoji: Some code cleanup. 2017-09-24 04:51:33 -07:00
Harshit Bansal 6b1525680e build_emoji: Remove now unused `MissingGlyphError` exception.
This exception was raised if there was no glyph available for a
codepoint. We no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal 898d7d1160 build_emoji: Remove now unused `bw_font()` function.
This function was used get a black and white glyph for an emoji if there
was no corresponding image file present in the `NotoColorEmoji.ttf` but
due to the new emoji farm setup code, we no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal b95fbbe29a build_emoji: Remove now unused `color_font()` function.
This function was used to color transparent number emojis. We no
longer need to do this since now we have remapped number emojis
to the corresponding colored emojis in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 6ebacba425 build_emoji: Remove `code_point_to_file_name_map()`.
We have symlinked the old emoji farm to the old emoji farm and hence
we don't use the images from the `NotoColorEmoji.ttf` file. This
function was used to generate a map from codepoint to filename by
parsing the ttx file passed to it. We no longer need this map.
2017-09-24 04:51:33 -07:00
Harshit Bansal e295bed20f build_emoji: Extract `generate_map_files()` function.
This commit extracts out the `generate_map_files()` function from
the `dump_emojis()` function. This function generates various data
files like `emoji_codes.js`, `name_to_codepoint.json` etc which are
used by webapp, bugdown etc.
2017-09-24 04:51:33 -07:00
Harshit Bansal ef96adf7e5 build_emoji: Remove old emoji farm setup code.
This commit removes the old emoji farm generation code  in favor of
`setup_old_emoji_farm()`. Instead of having individual images in old
emoji farm we now symlink them to the images in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 273e37b2f9 build_emoji: Add `setup_old_emoji_farm()` function.
This commit adds `setup_old_emoji_farm()` function to the build_emoji
script. This will change the way of setting up the old emoji farm.
Earlier we used to extract the glyphs corresponding to each emoji from
the `NotoColorEmoji.ttf` file. But since now we already have individual
images in the new emoji farm(from iamcal's 'emoji-datasource-google' npm
package) we can just symlink old emoji files to the new image files. This
apart from helping us in cleaning up the `build_emoji` script will also
help in reducing the increased size of the release tarball due to the
addition of new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal eed7d4b8d7 build_emoji: Remove old code in favor of `generate_sprite_css_files()`. 2017-09-24 04:51:33 -07:00
Harshit Bansal beab00a5e1 build_emoji: Add `generate_sprite_css_files()` function. 2017-09-24 04:51:33 -07:00
Harshit Bansal dcf6f1e0f0 build_emoji: Build new emoji farm.
This commit adds the 'setup_emoji_farm()' which copies the individual
images from `emoji-datasource-google` packge to target emoji dump.
2017-09-24 04:51:33 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
neiljp (Neil Pilgrim) a059872856 mypy: Clarify xml.find use in build_emoji:code_point_to_file_name_map. 2017-08-25 12:04:15 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Harshit Bansal d7617085e6 build_emoji: Update `get_success_stamp()` function.
Updates `get_success_stamp()` function to use the `emoji-datasource`
package's version while calculating success stamp so that an emoji
cache rebuild gets triggered automatically if the version is changed.
2017-07-16 12:36:19 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00
Harshit Bansal a549013c50 emoji_pickers: Fix broken flag emojis.
Due to differences between the codepoints of flag emojis in
`emoji_map.json` and iamcal's dataset, we need to patch the
css classes for the flag emojis temporarily until the migration
to iamcal's dataset is complete inorder to render them properly.
There is a difference between the images of flag emojis in our
old emoji farm and iamcal's spritesheets and since we have not
yet switched to using spritesheets for displaying emojis in
messages, there is a difference between the flag emojis as
rendered in messages and in emoji pickers.
2017-05-17 08:09:50 -07:00
Harshit Bansal 0d28d5d808 build_emoji: Fix the spritesheet CSS generation code.
Modify the spritesheet generation code to account for the differences
between emoji_map.json and iamcal's dataset. Due to the differences
between the two mappings, some emojis like 1️⃣, 2️⃣ etc were not
getting rendered properly in the two emoji pickers where we are using
the iamcal's spritesheets for rendering them. This was so because there
was no CSS class corresponding to the codepoints of these emojis(as
mapped using emoji_map.json) in our spritesheet CSS(generated using
iamcal's dataset).

Fixes: #4775.
2017-05-16 13:06:12 -07:00
Harshit Bansal 0667297c6e build_emoji: Fix the coloring of the white emojis.
Fix the coloring of the white emojis to match that of the images
in the iamcal's spritesheets.
2017-05-16 13:06:12 -07:00
Brock Whittaker ed4866b9f9 message-feed: Improve scroll perf by removing scaling on emojis.
This removes scaling from the emojis by changing the background size to
a lower value and then allowing for the widths and heights of the
emojis to be proportionally smaller.

The transform: scale property would cause many more repaints in Chrome
and other browsers than should have been necessary which would render
messages above and below the feed light grey boxes that would
momentarily flash as blank before filling with content.

Modified by tabbott to use a percentage in the background-size.

Fixes #4660.
2017-05-03 14:49:47 -07:00
Tim Abbott 110f161288 emoji: Remove code for generating old sprite sheets. 2017-05-03 13:34:30 -07:00
Tim Abbott b9cb6199cf emoji: Use lower-case hex letters in iamcal data. 2017-05-03 13:01:53 -07:00
Tim Abbott cfeff245c1 emoji: Manage Zulip emoji as a custom/extra realm emoji.
This is prerequisite for converting our emoji system to work off of
iamcal sprite sheets.

Also, fixes #4308.
2017-04-30 16:40:31 -07:00
Harshit Bansal 09c521d9e8 build_emoji: Use `ujson.load()` instead of `json.load()`.
Use `ujson.load()` to load `emoji_map.json` and `emoji.json` as
it is faster than `json.load()`.
2017-04-18 13:00:46 -07:00
Harshit Bansal c70cfa2188 build_emoji: Generate CSS files for all the emoji sets.
Modify the `build_emoji` tool to copy spritesheets for all the
emojisets to emoji cache and generate CSS files for them.
2017-04-18 13:00:46 -07:00
Harshit Bansal 5674e63fc3 Download iamcal's sprite sheets during provisioning and generate CSS files.
Add code to download iamcal's sprite sheets and generate CSS files required
for displaying these sprite sheets using emoji's unicode codepoints rather
than their names.
2017-04-18 13:00:46 -07:00
Harshit Bansal 660d96038a build_emoji: Add `emoji_catalog` to emoji_code.js
Use `emoji.json` to create a emoji catalog and add it to
`emoji_code.js` file. This catalog contains the unicode
codepoints of all the emojis grouped according to their
category. Emojis are sorted according to the `sort_order`
defined in the iamcal's dataset.
2017-04-18 12:46:52 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Harshit Bansal 521e8700d7 emoji: Update NotoColorEmoji emoji set.
Update the existing NotoColorEmoji set to include emoji additions
like gendered professions, rainbow flag, single parent families etc.

Fixes: #3861.
2017-03-02 19:07:34 +05:30