Commit Graph

237 Commits

Author SHA1 Message Date
Anders Kaseorg 1cdab5ae61 emoji: Resolve emoji sprite sheets and stylesheets through Webpack.
This gives them cache-compatible URLs, and also avoids some extra
copies of the sprite sheet images.

Comments on the Octopus emoji added by tabbott.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:43:46 -08:00
Anders Kaseorg 197084ab93 generate_emoji_names_table: Update SORTED_CATEGORIES.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-25 14:30:31 -08:00
Tim Abbott 8aae02de68 emoji: Add a block comment explaining spritesheet CSS. 2020-02-21 10:26:57 -08:00
Steve Howell af7923c557 refactor: Extract get_square_size() for emoji sprites.
This cleans up a few things:

    - just yield values so we don't have to do
      tedious max logic
    - use values() instead of items() for
      skin_variations loop

In the ideal world the emoji.json would reduce this
code to `get_square_size = lambda data: data['square_size']`,
but I don't think we can get the square size explicitly.
2020-02-21 10:21:12 -08:00
Steve Howell da1ce9a577 emoji sprites: Avoid hard coding CSS percentages.
This commit changes the calculation of the
background-size parameter that we use to
render emojis from sprite sheets.

In particular, it now makes the parameter
match the sizes of our latest sprite
sheets from Twitter/Google.

This should fix the geometry aspect of #13959,
but we also need to fix some issues with the
cache being sticky.

There is also some minor cleanup:

    - Remove obsolete -moz/-webkit CSS.
    - Remove needless precision in percentages.
    - Fix the transposed nrows/ncols names.
    - Add extensive commenting.

Finally, we add a minor bump to the provision
number.  This commit should be merged in the
same series as the other fix for this issue,
which will probably have a major bump, and we'll
need to rebase this appropriately.
2020-02-21 10:21:12 -08:00
Anders Kaseorg d7d8632525 pygments_data: Replace JS module with JSON module.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-12 10:09:12 -08:00
Anders Kaseorg e257253e64 emoji_codes: Replace JS module with JSON module.
webpack optimizes JSON modules using JSON.parse("{…}"), which is
faster than the normal JavaScript parser.

Update the backend to use emoji_codes.json too instead of the three
separate JSON files.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-12 10:09:12 -08:00
Anders Kaseorg d2e07ea51b setup_venv: Replace virtualenv_args list with python2 bool.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-02-05 12:38:10 -08:00
Anders Kaseorg 5e89a7d646 install-shellcheck: Upgrade ShellCheck from 0.6.0 to 0.7.0.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-11 16:26:31 -08:00
Anders Kaseorg 0af22dad18 flush-memcached: Respect MEMCACHED_LOCATION; handle errors.
Fixes #13238.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-10-01 16:05:55 -07:00
Tim Abbott 27b3c1a312 provision: Move install-shellcheck to proper directory.
Scripts in scripts/ should be exclusively code that used in
production, and this isn't.
2019-09-24 12:54:33 -07:00
Anders Kaseorg 92b42573fb requirements: Compile thumbor requirements with pip-tools on Python 2.
The reason that `pip-tools` running on Python 3 didn’t detect the
right requirements for `thumbor` on Python 2 is simply that some of
them are conditional on the Python version.

As for the requirements that had been manually added as a workaround:
`backports-abc` and `singledispatch` are now correctly detected, while
`backports.ssl-match-hostname` was vendored into `urllib3` some time
ago and `certifi` is no longer necessary.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-23 11:56:36 -07:00
Anders Kaseorg fc0554fcb8 emoji: Finish removing leftover code from banned Emoji sets.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-19 12:08:52 -07:00
Anders Kaseorg 17192ed46b Revert "provision: Give concrete NFS error message on older OSX versions."
This reverts commit 073ecaac66 (#9365).

This exception handler was overly broad in catching all `OSError`s,
and it made debugging harder by hiding the actual exception.
Furthermore, we no longer use NFS (#12963), and we’re now getting
reports of Windows users running into this message.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-09 14:47:51 -07:00
Anders Kaseorg 6701c4463c search: Remove now unnecessary tsearch_extra dependency.
Now that we're implemented tsearch_extras in pure postgres, we no
longer need a custom extension.  This should help us considerably, as
it means we no longer need to ship custom apt packages at all.

Fixes #467.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-29 12:49:26 -07:00
Anders Kaseorg 43c9a26afb setup_venvs: Use system python3.
Otherwise python3 will be perpetually copied from virtualenv to
virtualenv and will never receive updates from the system.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-23 16:33:04 -07:00
Wyatt Hoodes e331a758c3 python: Migrate open statements to use with.
This is low priority, but it's nice to be consistently using the best
practice pattern.

Fixes: #12419.
2019-07-20 15:48:52 -07:00
Anders Kaseorg 695b79c5d8 generate_zulip_bots_static_files: Write to static, not STATIC_ROOT.
Otherwise the files aren’t processed by collectstatic and don’t end up
in the staticfiles.json manifest.

Signed-off-by: Anders Kaseorg <andersk@zulipchat.com>
2019-07-19 11:33:08 -07:00
Anders Kaseorg ab89f40a66 generate-custom-icon-webfont: Replace with webpack webfonts-loader.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-07-18 12:00:00 -07:00
Anders Kaseorg 0f30c8c4ba dev-vagrant-docker: Upgrade docker-systemctl-replacement.
This fixes the 45 second boot delay bug.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-24 22:06:32 -07:00
Tim Abbott 9760c8cfc9 tools: Fix postgres-init-*-db cache handling.
Previously, it didn't properly update the stamp files that determine
our caching behavior, so if one ran test-backend afterwards, nothing
would happen.

A secondary issue that this commit does not fix is that provision will
end up rerunning the whole thing.
2019-06-17 16:24:13 -07:00
Tim Abbott 05e0e99b6e install-aws-server: Use our settings.py if available. 2019-06-13 14:39:25 -07:00
Tim Abbott 2d1c2d4802 install-aws-server: Stop using ssh -t without a real terminal.
More modern Linux versions like Bionic will block this, and what we
actually want to do is just run the code in our <<EOF block via bash,
so we should do that explicitly.
2019-06-13 14:39:25 -07:00
Anders Kaseorg 5c734f7048 dev-vagrant-docker: Clear redis-server ExecStart before overriding it.
Real systemd requires this.  docker-systemctl-replacement currently
doesn’t but maybe it will later.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-06-01 16:55:56 -07:00
Anders Kaseorg 630f0fa761 Vagrantfile: Add UBUNTU_MIRROR configuration option.
The default http://archive.ubuntu.com/ubuntu/ is pretty slow in some
locations.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-29 16:19:39 -07:00
Anders Kaseorg f21d3be5fb dev-vagrant-docker: Copy the host user’s uid for the vagrant user.
This allows /srv/zulip to be writable from the guest even if the host
user’s uid is not 1000.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 19:21:51 -07:00
Anders Kaseorg 5231c8e575 dev-vagrant-docker: Unminimize the minimized Ubuntu cloud image.
This restores man pages and other documentation that have been
stripped from the default Ubuntu cloud image and installs
ubuntu-minimal and ubuntu-standard.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-27 12:53:11 -07:00
Anders Kaseorg b594708915 Vagrantfile: Be nicer to /etc/update-motd.d.
Some of its information is helpful.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-05-24 17:33:06 -07:00
Anders Kaseorg 684ebc2a5e Vagrantfile: Support docker provider.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-05-24 15:43:23 -07:00
Anders Kaseorg 643bd18b9f lint: Fix code that evaded our lint checks for string % non-tuple.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:21:37 -07:00
Anders Kaseorg 5290519a62 scripts: Always use ON_ERROR_STOP=1 when running psql.
Also use psql -e (--echo-queries) in scripts that use ‘set -x’, so
errors can be traced to a specific query from the output.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:19 -07:00
Anders Kaseorg efd22fd2ae postgres-init-dev-db: Catch exception when recreating extant users.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-22 14:54:18 -07:00
Anders Kaseorg 53879c4673 scripts: Rename DEFAULT_USER to POSTGRES_USER.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-12 17:27:23 -07:00
Rhythm Sharma e1212c04d1 emoji: Add a few useful aliases. 2019-03-17 13:38:31 -07:00
Anders Kaseorg 8fdb62a6bc tools: Create DEPLOY_ROOT/zulip-py3-venv symlink in development too.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-07 17:12:33 -08:00
Anders Kaseorg ee8ff4df66 tools: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:10:31 -08:00
Anders Kaseorg 392175d6e8 Use #!/usr/bin/env for bash shebangs.
/bin/sh and /usr/bin/env are the only two binaries that NixOS provides
at a fixed path (outside a buildFHSUserEnv sandbox).

This discussion was split from #11004.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-12-17 17:21:08 -08:00
Sumanth V Rao cae718f184 provision: Rename generate_zulip_bots_static_files to .py.
This should make it possible to call this script as a function without
starting up a new Python interpreter.
2018-12-09 21:42:40 -08:00
Anders Kaseorg 19bdf54f33 tools/setup/emoji/build_emoji: Avoid shelling out for touch, rm, mkdir, cp.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-11-28 17:28:17 -08:00
Anders Kaseorg fc24d2e147 install-aws-server: Fix shellcheck warnings.
In tools/setup/install-aws-server line 25:
zulip_root=${ZULIP_ROOT:-$HOME/zulip}
^-- SC2034: zulip_root appears unused. Verify use (or export if used externally).

In tools/setup/install-aws-server line 40:
if [ -n "$zulip_confdir" ]; then
         ^-- SC2154: zulip_confdir is referenced but not assigned.

In tools/setup/install-aws-server line 55:
VIRTUALENV_NEEDED=$(if $(echo "$type" | grep -q app_frontend); then echo -n yes; else echo -n no; fi)
                       ^-- SC2091: Remove surrounding $() to avoid executing output.

In tools/setup/install-aws-server line 60:
SSH_OPTS=(-o HostKeyAlgorithms=ssh-rsa)
             ^-- SC2191: The = here is literal. To assign by index, use ( [index]=value ) with no spaces. To keep as literal, quote it.

In tools/setup/install-aws-server line 69:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

In tools/setup/install-aws-server line 86:
ssh "${SSH_OPTS[@]}" "$server" -t -i "$amazon_key_file" -lroot <<EOF
                                                                 ^-- SC2087: Quote 'EOF' to make here document expansions happen on the server side rather than on the client.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:51:41 -07:00
Anders Kaseorg b15063155c postgres-init-dev-db: Fix shellcheck warnings.
In tools/setup/postgres-init-dev-db line 10:
ROOT_POSTGRES="sudo -i -u "$DEFAULT_USER" psql"
                           ^-- SC2027: The surrounding quotes actually unquote this. Remove or escape them.

In tools/setup/postgres-init-dev-db line 46:
    echo 'ERROR: Try `sudo service postgresql start`?'
         ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.

In tools/setup/postgres-init-dev-db line 64:
PGPASS_ESCAPED_PREFIX="*:\*:\*:$USERNAME:"
                         ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".
                            ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-17 17:48:57 -07:00
Harshit Bansal fc7e654d7b emoji: Remove single-digit names for number emojis.
Names like `:1:`, `:2:` etc don't seem to be of much benefit rather
they seem to get rendered at places not intended.
2018-10-02 11:30:11 -07:00
Harshit Bansal 88bfb9778b build_emoji: Add infrastructure for google-blob and twitter emojisets. 2018-08-31 11:52:24 -07:00
Harshit Bansal e204fe3948 minor: Fix for failing tests. 2018-08-27 06:16:16 +00:00
Harshit Bansal d96d6a96f7 build_emoji: Add a missing check for image existence. 2018-08-26 22:16:23 -07:00
Harshit Bansal f5454dff51 build_emoji: Automatically calculate sprite dimensions.
Instead of using a hardcoded value for spritesheet dimensions,
automatically calculate it using `emoji_data`. This will free
us from updating it only emoji datasource update as well as
allow us to add google blob emojiset.
2018-08-26 22:16:23 -07:00
Harshit Bansal dbc3774c2f build_emoji: Refactor `get_emoji_code()` to not fail for old datasources.
If `non_qualified` field is not present then return the value of
`unified` field instead of failing.
2018-08-26 22:16:23 -07:00
Harshit Bansal ec2ebd1cc9 build_emoji: Change sprite CSS files to use hyphens in names.
This is more consistent with our other names in emoji infra.
2018-08-26 22:16:23 -07:00
Harshit Bansal 9bd56d0ce4 build_emoji: Change emoji sprite sheets to use hyphens in names.
Use of hyphens is more consistent with our other emoji infra names.
2018-08-26 22:16:23 -07:00
Harshit Bansal aa6f098496 build_emoji: Extract `setup_emoji_farm()`. 2018-08-26 22:16:23 -07:00