Commit Graph

109 Commits

Author SHA1 Message Date
Umair Khan c039f73656 setup_venv: Link to the correct Python interpreter.
Travis enables different Python versions through virtual environments,
but it seems that there is a little caveat when we try to create Zulip's
virtual environment by referring Travis' virtual environment; Zulip's
virtual environment refers the system Python. We encountered this
behaviour when we tried to run our backend test suite under Python 3.5
in Travis. 'python3 --version' command before activating Zulip's
virtualenv showed 'Python 3.5.3' and after it showed 'Python 3.4.3'.
This happened when we created the virtual environment using
'virtualenv -p python3'.

The solution seems to be to explicitly give the path of the Python
interpreter in the Travis' virtual environment using 'which python3'.
2017-09-06 13:48:50 -07:00
Harshit Bansal facb5dbe85 zulip_tools.py: Extract `generate_sha1sum_emoji()`.
Given the path of a zulip installation, it returns a hash corresponding
to the emoji infrastructure of that installation.
2017-08-27 17:51:24 -07:00
neiljp (Neil Pilgrim) a059872856 mypy: Clarify xml.find use in build_emoji:code_point_to_file_name_map. 2017-08-25 12:04:15 -07:00
rht 80f4ba7a76 requirements: Replace py3_dev.txt with dev_lock.txt. 2017-08-22 10:10:57 -07:00
Greg Price a099e698e2 py3: Switch almost all shebang lines to use `python3`.
This causes `upgrade-zulip-from-git`, as well as a no-option run of
`tools/build-release-tarball`, to produce a Zulip install running
Python 3, rather than Python 2.  In particular this means that the
virtualenv we create, in which all application code runs, is Python 3.

One shebang line, on `zulip-ec2-configure-interfaces`, explicitly
keeps Python 2, and at least one external ops script, `wal-e`, also
still runs on Python 2.  See discussion on the respective previous
commits that made those explicit.  There may also be some other
third-party scripts we use, outside of this source tree and running
outside our virtualenv, that still run on Python 2.
2017-08-16 17:54:43 -07:00
Greg Price b7ec1b55dd py3: Cut Python 2 codepaths from provision. 2017-08-16 17:54:43 -07:00
Greg Price 73cb361896 install-aws-server: Cut out use of /root/zulip .
Our previous dependencies on the `/root/zulip` path should all be
long gone at this point.  Use a fresh temporary directory instead;
that's cleaner.
2017-08-15 17:41:07 -07:00
rht b0e4233d3f Dev environment setup: default to Python 3.
From the line in tools/provision it should trickle to the rest of the
scripts. This works since almost all the python scripts have been linted
to be generic.

Proof that the setup is python3 only: with this commit, within the
vagrant container env, /srv/zulip-venv is no longer present and
`./tools/run-dev.py` runs just fine.

[gnprice: Added `rm -f` and warning message, and made small edits.]
2017-08-08 15:28:11 -07:00
rht 532bd6eaa6 Ensure virtualenv is run throughout the entire Travis dev provision. 2017-08-01 15:29:13 -07:00
Harshit Bansal d7617085e6 build_emoji: Update `get_success_stamp()` function.
Updates `get_success_stamp()` function to use the `emoji-datasource`
package's version while calculating success stamp so that an emoji
cache rebuild gets triggered automatically if the version is changed.
2017-07-16 12:36:19 -07:00
Harshit Bansal 92f6613bdd tools: Remove `tools/setup/emoji/download-emoji-data` tool.
This tool was used for downloading sprite sheets from iamcal's
repository. Since now we have moved to using `emoji-datasource`
npm package, this tool is no longer required.
2017-07-13 21:52:01 -07:00
Harshit Bansal ebdd8c9e3d emoji: Switch to using `emoji-datasource` npm package.
This commit does the following things:
* Instead of using a manual tool for downloading sprite sheets, use
  `emoji-datasource` npm package.
* Modify the `build_emoji` script to use sprite sheets from the npm
  package.

Bumps PROVISION_VERSION.

Fixes: #4730.
2017-07-13 21:52:01 -07:00
Harshit Bansal a876ebb52a build_emoji: Close open file objects after reading. 2017-07-01 11:14:24 -07:00
Harshit Bansal 8c92024fae build_emoji: Add `codepoint_to_name` dict to emoji_codes.js.
This dict is a mapping from codepoint to canonical name.
2017-07-01 11:14:24 -07:00
Harshit Bansal fee4065571 build_emoji: Generate `codepoint_to_name.json` file.
Store the `codepoint_to_name` map in a JSON file which can be used
by the rest of codebase(zerver) to convert emoji codepoints to
canonical names.
2017-07-01 11:14:24 -07:00
Harshit Bansal e30e244b29 emoji: Moved `unified_reactions.json` file.
Moved `unified_reactions.json` file from `tools/setup/emoji` to
`zerver/management/data` so that it can be used in production
environment.
2017-06-09 16:38:58 -07:00
Harshit Bansal 821ff06629 Add `unified_reactions.json` to be used for reactions migration.
This file contains the mapping from old emoji names to new codepoints
from iamcal's dataset to be used in reaction migration.
2017-06-06 23:52:41 -07:00
Tim Abbott a0ef2babab text_fixtures: Rename migration status file to have test in name.
This is preparation for making things clear as we move towards not
always rebuilding the test database in provision.
2017-06-02 13:27:34 -07:00
Aditya Bansal 5b0b8fd8bb pep8: Add compliance with rule E261 emoji_setup_utils.py. 2017-05-31 17:07:15 -07:00
Tim Abbott 8aaf9b1426 Revert "Fix emoji cache setup when not using provision.py."
This reverts commit ab2cfadcaa.

This broke production installation.
2017-05-24 10:35:41 -07:00
Carey Metcalfe ab2cfadcaa Fix emoji cache setup when not using provision.py.
Moves creating the emoji folder from the provisioning script to
the build_emoji script.

Fixes the fact that the emoji cache directory wasn't being created
when not using the provision.py script.
2017-05-22 11:08:43 -07:00
Harshit Bansal a549013c50 emoji_pickers: Fix broken flag emojis.
Due to differences between the codepoints of flag emojis in
`emoji_map.json` and iamcal's dataset, we need to patch the
css classes for the flag emojis temporarily until the migration
to iamcal's dataset is complete inorder to render them properly.
There is a difference between the images of flag emojis in our
old emoji farm and iamcal's spritesheets and since we have not
yet switched to using spritesheets for displaying emojis in
messages, there is a difference between the flag emojis as
rendered in messages and in emoji pickers.
2017-05-17 08:09:50 -07:00
Harshit Bansal 0d28d5d808 build_emoji: Fix the spritesheet CSS generation code.
Modify the spritesheet generation code to account for the differences
between emoji_map.json and iamcal's dataset. Due to the differences
between the two mappings, some emojis like 1️⃣, 2️⃣ etc were not
getting rendered properly in the two emoji pickers where we are using
the iamcal's spritesheets for rendering them. This was so because there
was no CSS class corresponding to the codepoints of these emojis(as
mapped using emoji_map.json) in our spritesheet CSS(generated using
iamcal's dataset).

Fixes: #4775.
2017-05-16 13:06:12 -07:00
Harshit Bansal 0667297c6e build_emoji: Fix the coloring of the white emojis.
Fix the coloring of the white emojis to match that of the images
in the iamcal's spritesheets.
2017-05-16 13:06:12 -07:00
Brock Whittaker ed4866b9f9 message-feed: Improve scroll perf by removing scaling on emojis.
This removes scaling from the emojis by changing the background size to
a lower value and then allowing for the widths and heights of the
emojis to be proportionally smaller.

The transform: scale property would cause many more repaints in Chrome
and other browsers than should have been necessary which would render
messages above and below the feed light grey boxes that would
momentarily flash as blank before filling with content.

Modified by tabbott to use a percentage in the background-size.

Fixes #4660.
2017-05-03 14:49:47 -07:00
Tim Abbott 110f161288 emoji: Remove code for generating old sprite sheets. 2017-05-03 13:34:30 -07:00
Tim Abbott b9cb6199cf emoji: Use lower-case hex letters in iamcal data. 2017-05-03 13:01:53 -07:00
Tim Abbott cfeff245c1 emoji: Manage Zulip emoji as a custom/extra realm emoji.
This is prerequisite for converting our emoji system to work off of
iamcal sprite sheets.

Also, fixes #4308.
2017-04-30 16:40:31 -07:00
Cory Lynch 0965c43238 Add typeahead for syntax highlighting languages.
Modified composebox_typeahead.js to recognize the triple backtick
and tilde for code blocks, and added appropriate typeahead functions
in that file and in typeahead_helper.js.

Additionally, a new file pygments_data.js contains a dictionary of
the supported languages, mapping to relative popularity
rankings. These rankings determine the order of sort of the
languages in the typeahead.

This JavaScript file is actually in static/generated/pygments_data.js, as it
is generated by a Python script, tools/build_pymgents_data.py. This is
so that if Pygments adds support for new languages, the JavaScript file
will be updated appropriately. This python script uses a set of popularity
rankings defined in lang.json.

Corresponding unit tests were also added.

Fixes #4111.
2017-04-28 17:22:59 -07:00
derAnfaenger 2adf641b72 Fix provisioning failing due to partial downloads.
'$COMMIT' was originally printed in '$COMMIT_FILE_PATH' before all
respective files were downloaded, meaning that this step
wouldn't be repeated if one download failed. This commit prints
'$COMMIT' only after all downloads were successful.
2017-04-22 21:16:11 -07:00
Harshit Bansal 09c521d9e8 build_emoji: Use `ujson.load()` instead of `json.load()`.
Use `ujson.load()` to load `emoji_map.json` and `emoji.json` as
it is faster than `json.load()`.
2017-04-18 13:00:46 -07:00
Harshit Bansal c70cfa2188 build_emoji: Generate CSS files for all the emoji sets.
Modify the `build_emoji` tool to copy spritesheets for all the
emojisets to emoji cache and generate CSS files for them.
2017-04-18 13:00:46 -07:00
Harshit Bansal e52f2b5aba download-emoji-data: Download sprite sheets for the remaining emoji sets.
Modify the `download-emoji-data` tool to download the sprite sheets
for all the currently supported emoji sets(apple, emojione, google
and twitter).
2017-04-18 13:00:46 -07:00
Harshit Bansal 5674e63fc3 Download iamcal's sprite sheets during provisioning and generate CSS files.
Add code to download iamcal's sprite sheets and generate CSS files required
for displaying these sprite sheets using emoji's unicode codepoints rather
than their names.
2017-04-18 13:00:46 -07:00
Harshit Bansal d617fa8d81 emoji: Only include universal emoji in catalog.
Extracted from "Interrelated emoji infrastructure changes." by
tabbott.
2017-04-18 13:00:16 -07:00
Harshit Bansal 660d96038a build_emoji: Add `emoji_catalog` to emoji_code.js
Use `emoji.json` to create a emoji catalog and add it to
`emoji_code.js` file. This catalog contains the unicode
codepoints of all the emojis grouped according to their
category. Emojis are sorted according to the `sort_order`
defined in the iamcal's dataset.
2017-04-18 12:46:52 -07:00
Harshit Bansal 4470a3947b provision: Download emoji-data during provisioning.
Add code to download iamcal's emoji_pretty.json
(emoji-data/emoji.json) during provisioning.
2017-04-18 12:46:52 -07:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Harshit Bansal 521e8700d7 emoji: Update NotoColorEmoji emoji set.
Update the existing NotoColorEmoji set to include emoji additions
like gendered professions, rainbow flag, single parent families etc.

Fixes: #3861.
2017-03-02 19:07:34 +05:30
Harshit Bansal 3651a432d5 build_emoji: Add `name_to_codepoint` dict to emoji_codes.js file. 2017-02-26 18:28:42 -08:00
Harshit Bansal 1b0d3cbb39 tools/build_emoji: Generate `name_to_codepoint.json` file.
Generate the `name_to_codepoint.json` file in build_emoji which will be
used by bugdown/ to map emoji names to codepoints directly.
2017-02-26 18:28:42 -08:00
Harshit Bansal bc4aef0bc7 emoji: Move zulip-emoji to `images/emoji/unicode`.
Move zulip-emoji from its current location `images/emoji` to
`images/emoji/unicode` and add a symlink in `images/emoji`
to zulip.png in `images/emoji/unicode`.
2017-02-26 18:28:41 -08:00
Saumya Rawat dcd9ff642d dev: Fix postgres provision errors with non-postgres-readable homedir.
This fixes an issue where provision would fail if the user's home
directory was setup in such a way that the postgres user couldn't
access it (and thus the `sudo` command here would throw errors about
having a non-readable current working directory).
2017-02-11 23:47:07 -08:00
Saumya Rawat 151d2139ad dev: Clearer postgres error messages in postgres-init-dev-db.
This uses `pg_isready` to provide a clearer error message if postgres
isn't working.

Fixes #2419.
2017-02-11 23:46:56 -08:00
Tim Abbott 234eb7a723 emoji: Add a bunch of documentation in comments for emoji rules. 2017-01-29 12:33:44 -08:00
Rishi Gupta e19f3d5534 emoji: Make minor adjustments to emoji names list. 2017-01-29 12:19:54 -08:00
Rishi Gupta 6f307ffd08 emoji: Switch the order of thumbs_up and thumbs_down in autocomplete. 2017-01-28 23:38:03 -08:00
Rishi Gupta 8ef5197d5c emoji: Remove problematic color emoji from autocomplete and picker. 2017-01-28 23:38:03 -08:00
Rishi Gupta a2890f7d7a emoji: Remove duplicates from autocomplete and emoji picker.
Previously, if you searched for ':offi..' you would see both 🏢 and
:office_building: as possible completions, both of which are shortcodes for
the same unicode codepoint (and hence which have the same image). Also, we
sort the emoji in our emoji pickers alphabetically by shortcode, and so the
images for 🏢 and :office_building: show up next to each other, which
looks like a bug. This removes :office_building: as a shortcode, along with
several hundred other duplicates. It leaves some duplicates in that won't
give autocomplete or alphabetical ordering a problem, like (🚗,
:automobile:).
2017-01-28 23:38:03 -08:00
Rishi Gupta 6b3abce541 build_emoji: Generate emoji names and codepoints from emoji_map.
Replaces the hardcoded list of emoji_names and unicode_emoji_names in
static/js/emoji.js with a list generated from emoji_map.json, both to get
the list out of version control and so we can start modifying it for our
autocomplete. This does not change the contents of emoji_names. It sorts and
removes duplicates from unicode_emoji_names (causes no change in behavior,
since unicode_emoji_names is only used as if it were a set).
2017-01-28 17:05:32 -08:00