Commit Graph

77 Commits

Author SHA1 Message Date
Harshit Bansal 40958e0824 emoji: Switch to 64px 256 color indexed sprite sheets.
This commit switches our emoji infrastructure to use 256 color indexed
64px spritesheets. Earlier we were using non-indexed 32px spritesheets
which were blurry on high dpi displays. These indexed spritesheets not
only provide a crispier display but are also smaller in size.

This commit also removes the `emoji-datasource` package as a dependency
as all the data is now sourced from individual datasource packages.

Fixes: #7862.
2018-03-14 10:28:45 -07:00
Harshit Bansal 40fe6239c5 build_emoji: Migrate to python3. 2017-11-20 15:30:58 -08:00
David Rosa Tamsen 7072fa5b37 docs: Reorganize developer docs to improve navigation.
This commit helps reduce clutter on the navigation sidebar.
Creates new directories and moves relevant files into them.
Modifies index.rst, symlinks, and image paths accordingly.

This commit also enables expandable/collapsible navigation items,
renames files in docs/development and docs/production,
modifies /tools/test-documentation so that it overrides a theme setting,
Also updates links to other docs, file paths in the codebase that point
to developer documents, and files that should be excluded from lint tests.

Note that this commit does not update direct links to
zulip.readthedocs.io in the codebase; those will be resolved in an
upcoming follow-up commit (it'll be easier to verify all the links
once this is merged and ReadTheDocs is updated).

Fixes #5265.
2017-11-16 09:45:08 -08:00
rht f397e15128 tools: Text-wrap long lines exceeding 110. 2017-11-10 16:24:09 -08:00
rht ec5120e807 refactor: Remove six.moves.zip import. 2017-11-07 10:46:42 -08:00
Harshit Bansal 1067cfd92a node_modules: Add npm packages for the supported emojisets.
This commit adds emoji datasource packages for the rest of the emojiset
packages which we will use for serving images in notifications.
2017-10-28 10:38:32 -07:00
rht c8090aafa4 lint: Remove several unused imports. 2017-10-18 10:56:11 -07:00
rht 691598a88b py3: Remove "from six.moves import range".
This is no longer required, since in Python 3, this is what the range
built-in does.
2017-10-17 23:28:14 -07:00
Harshit Bansal ea5cb23c8e build_emoji: Extend emoji mappings to use new emojis from iamcal dataset.
Emojis which are represented by a sequence of codepoints or emojis
with ZWJ are not included until we implement a mechanism for dealing
with their unicode versions.

Fixes: #6279.
2017-10-04 23:09:14 -07:00
Harshit Bansal 1836ac2ff5 emoji_setup_utils: Add 'get_extended_codepoint_to_name()` function. 2017-10-04 23:09:14 -07:00
Harshit Bansal 3fdf25940f emoji_setup_utils: Add `get_extended_name_to_codepoint()` function. 2017-10-04 23:09:14 -07:00
Harshit Bansal f6e113a8a3 emoji_setup_utils: Add 'get_extended_names_list()` function. 2017-10-04 23:09:14 -07:00
Harshit Bansal 95b17217b2 emoji_setup_utils: Add `get_new_emoji_dicts()` function. 2017-10-04 23:09:14 -07:00
Harshit Bansal dc4ddb1578 emoji_setup_utils.py: Add `emoji_can_be_included()` function. 2017-10-04 23:09:14 -07:00
rht bf4eda7374 tools: Remove absolute_import in most tools.
Tweaked by tabbott to not remove it from lister.py, linter_lib, and
friends, since those are intended to support both Python 2 and 3
(we're planning to extract them from the repository).
2017-09-29 12:28:43 -07:00
Harshit Bansal 521ac31d3d build_emoji: Remove the redundant `codepoints` list.
This list contained a list of codepoints of all the emojis. We now have
`codepoint_to_name` in emoji_codes.js and hence this is not required.
2017-09-29 11:17:06 -07:00
Harshit Bansal f6c621dab5 build_emoji: Remove the hack used for displaying number emojis correctly.
This hack was used to fix the broken number emojis in the emoji picker.
It was broken because of the partial migration to the iamcal dataset.
See issue #4775 for more details.
2017-09-29 11:17:06 -07:00
Harshit Bansal 0940d2cf85 build_emoji: Remove the hack used for displaying flag emojis correctly.
This hack was used to fix the broken flag emojis in emoji-picker.
It was broken due to the incomplete migration to iamcal dataset.
See issue #4775 for more details.
2017-09-29 11:17:06 -07:00
Harshit Bansal 5b5bcce098 emoji: Migrate bugdown emoji to use sprite sheets.
This commit switches to use sprite sheets for rendering emojis
in all the remaining places, i.e., message bodies and composebox
typeahead. This commit also includes some changes to notifications.py
file so that the spans used for rendering emojis can be converted
to corresponding image tags so that we don't break the emoji rendering
in missed message emails since we can't use sprite sheets there.

As part of switching the bugdown system to use sprite sheets, we need
to switch the name_to_codepoint mappings to match the new sprite
sheets.  This has the side effect of fixing a bunch of emoji like
numbers and flag emoji in the emoji pickers.

Fixes: #3895.
Fixes: #3972.
2017-09-29 11:14:34 -07:00
Harshit Bansal 995a0f3cc0 build_emoji: Remove an unnecessary check for symlink support.
We already check for symlink support in provisioning script so no
need of doing so here.
2017-09-24 04:51:33 -07:00
Harshit Bansal 6ff7da04de emoji: Remove `NotoColorEmoji.ttf`.
We no longer use glyphs from `NotoColorEmoji.ttf` so removing this.
2017-09-24 04:51:33 -07:00
Harshit Bansal c8c1c8ef43 emoji: Remove `AndroidEmoji.ttf`. 2017-09-24 04:51:33 -07:00
Harshit Bansal 0d8e68d75e build_emoji: Some code cleanup. 2017-09-24 04:51:33 -07:00
Harshit Bansal 6b1525680e build_emoji: Remove now unused `MissingGlyphError` exception.
This exception was raised if there was no glyph available for a
codepoint. We no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal 898d7d1160 build_emoji: Remove now unused `bw_font()` function.
This function was used get a black and white glyph for an emoji if there
was no corresponding image file present in the `NotoColorEmoji.ttf` but
due to the new emoji farm setup code, we no longer need this.
2017-09-24 04:51:33 -07:00
Harshit Bansal b95fbbe29a build_emoji: Remove now unused `color_font()` function.
This function was used to color transparent number emojis. We no
longer need to do this since now we have remapped number emojis
to the corresponding colored emojis in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 6ebacba425 build_emoji: Remove `code_point_to_file_name_map()`.
We have symlinked the old emoji farm to the old emoji farm and hence
we don't use the images from the `NotoColorEmoji.ttf` file. This
function was used to generate a map from codepoint to filename by
parsing the ttx file passed to it. We no longer need this map.
2017-09-24 04:51:33 -07:00
Harshit Bansal e295bed20f build_emoji: Extract `generate_map_files()` function.
This commit extracts out the `generate_map_files()` function from
the `dump_emojis()` function. This function generates various data
files like `emoji_codes.js`, `name_to_codepoint.json` etc which are
used by webapp, bugdown etc.
2017-09-24 04:51:33 -07:00
Harshit Bansal ef96adf7e5 build_emoji: Remove old emoji farm setup code.
This commit removes the old emoji farm generation code  in favor of
`setup_old_emoji_farm()`. Instead of having individual images in old
emoji farm we now symlink them to the images in the new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal 273e37b2f9 build_emoji: Add `setup_old_emoji_farm()` function.
This commit adds `setup_old_emoji_farm()` function to the build_emoji
script. This will change the way of setting up the old emoji farm.
Earlier we used to extract the glyphs corresponding to each emoji from
the `NotoColorEmoji.ttf` file. But since now we already have individual
images in the new emoji farm(from iamcal's 'emoji-datasource-google' npm
package) we can just symlink old emoji files to the new image files. This
apart from helping us in cleaning up the `build_emoji` script will also
help in reducing the increased size of the release tarball due to the
addition of new emoji farm.
2017-09-24 04:51:33 -07:00
Harshit Bansal eed7d4b8d7 build_emoji: Remove old code in favor of `generate_sprite_css_files()`. 2017-09-24 04:51:33 -07:00
Harshit Bansal beab00a5e1 build_emoji: Add `generate_sprite_css_files()` function. 2017-09-24 04:51:33 -07:00
Harshit Bansal dcf6f1e0f0 build_emoji: Build new emoji farm.
This commit adds the 'setup_emoji_farm()' which copies the individual
images from `emoji-datasource-google` packge to target emoji dump.
2017-09-24 04:51:33 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
neiljp (Neil Pilgrim) a059872856 mypy: Clarify xml.find use in build_emoji:code_point_to_file_name_map. 2017-08-25 12:04:15 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Harshit Bansal d7617085e6 build_emoji: Update `get_success_stamp()` function.
Updates `get_success_stamp()` function to use the `emoji-datasource`
package's version while calculating success stamp so that an emoji
cache rebuild gets triggered automatically if the version is changed.
2017-07-16 12:36:19 -07:00
Harshit Bansal 92f6613bdd tools: Remove `tools/setup/emoji/download-emoji-data` tool.
This tool was used for downloading sprite sheets from iamcal's
repository. Since now we have moved to using `emoji-datasource`
npm package, this tool is no longer required.
2017-07-13 21:52:01 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Harshit Bansal 821ff06629 Add `unified_reactions.json` to be used for reactions migration.
This file contains the mapping from old emoji names to new codepoints
from iamcal's dataset to be used in reaction migration.
2017-06-06 23:52:41 -07:00
Aditya Bansal 5b0b8fd8bb pep8: Add compliance with rule E261 emoji_setup_utils.py. 2017-05-31 17:07:15 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00
Harshit Bansal a549013c50 emoji_pickers: Fix broken flag emojis.
Due to differences between the codepoints of flag emojis in
`emoji_map.json` and iamcal's dataset, we need to patch the
css classes for the flag emojis temporarily until the migration
to iamcal's dataset is complete inorder to render them properly.
There is a difference between the images of flag emojis in our
old emoji farm and iamcal's spritesheets and since we have not
yet switched to using spritesheets for displaying emojis in
messages, there is a difference between the flag emojis as
rendered in messages and in emoji pickers.
2017-05-17 08:09:50 -07:00
Harshit Bansal 0d28d5d808 build_emoji: Fix the spritesheet CSS generation code.
Modify the spritesheet generation code to account for the differences
between emoji_map.json and iamcal's dataset. Due to the differences
between the two mappings, some emojis like 1️⃣, 2️⃣ etc were not
getting rendered properly in the two emoji pickers where we are using
the iamcal's spritesheets for rendering them. This was so because there
was no CSS class corresponding to the codepoints of these emojis(as
mapped using emoji_map.json) in our spritesheet CSS(generated using
iamcal's dataset).

Fixes: #4775.
2017-05-16 13:06:12 -07:00