mirror of https://github.com/zulip/zulip.git
provision: Let build_emoji build its own cache.
We no longer need to maintain duplicate code related to where we set up the emoji cache directory. And we no longer need two extra steps for people doing advanced (i.e. manual) setup. There was no clear benefit to having provision build the cache directory for `build_emoji`, when it was easy to make `build_emoji` more self-sufficient. The `build_emoji` tool was already importing the library that has `run_as_root`, and it was already responsible for 99% of the create-directory kind of tasks. (We always call `build_emoji` unconditionally from `provision`, so there's no rationale in terms of avoiding startup time or something.) ASIDE: Its not completely clear to me why we need to put this directory in "/srv", instead of somewhere more local (like we already do for Travis), but maybe it's just to be like its siblings in "/srv": node_modules yarn.lock zulip-emoji-cache zulip-npm-cache zulip-py3-venv zulip-thumbor-venv zulip-venv-cache zulip-yarn I guess the caches that we keep in var are dev-only, although I think some of what's under `zulip-emoji-cache` is also dev-only in nature? ./var/webpack-cache ./var/mypy-cache In `docs/subsystems/emoji.md` we say this: ``` The `build_emoji` tool generates the set of files under `static/generated/emoji` (or really, it generates the `/srv/zulip-emoji-cache/<sha1>/emoji` tree, and `static/generated/emoji` is a symlink to that tree;we do this in order to cache old versions to make provisioning and production deployments super fast in the common case that we haven't changed the emoji tooling). [...] ``` I don't really understand that rationale for the development case, since `static/generated` is as much ignored by `git` as '/srv' is, without the complications of needing `sudo` to create it. And in production, I'm not sure how much time we're really saving, as it takes me about 1.4s to fully rebuild the cache in dev, not to mention we're taking on upgrade risk by sharing files between versions.
This commit is contained in:
parent
bf3decfd0c
commit
7eb6d32d59
|
@ -224,8 +224,6 @@ Now run these commands:
|
|||
```
|
||||
sudo ./scripts/lib/install-node
|
||||
yarn install
|
||||
sudo mkdir /srv/zulip-emoji-cache
|
||||
sudo chown -R `whoami`: /srv/zulip-emoji-cache
|
||||
./tools/setup/emoji/build_emoji
|
||||
./scripts/setup/inline_email_css.py
|
||||
./tools/setup/build_pygments_data
|
||||
|
|
|
@ -8,7 +8,7 @@ import shutil
|
|||
ZULIP_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
sys.path.append(ZULIP_PATH)
|
||||
from scripts.lib.zulip_tools import run, run_as_root, OKBLUE, ENDC, \
|
||||
from scripts.lib.zulip_tools import run, OKBLUE, ENDC, \
|
||||
get_dev_uuid_var_path, file_or_package_hash_updated
|
||||
|
||||
from version import PROVISION_VERSION
|
||||
|
@ -16,15 +16,6 @@ from version import PROVISION_VERSION
|
|||
from tools.setup.generate_zulip_bots_static_files import generate_zulip_bots_static_files
|
||||
|
||||
VENV_PATH = "/srv/zulip-py3-venv"
|
||||
|
||||
is_travis = 'TRAVIS' in os.environ
|
||||
|
||||
# TODO: De-duplicate this with emoji_dump.py
|
||||
EMOJI_CACHE_PATH = "/srv/zulip-emoji-cache"
|
||||
if is_travis:
|
||||
# In Travis CI, we don't have root access
|
||||
EMOJI_CACHE_PATH = "/home/travis/zulip-emoji-cache"
|
||||
|
||||
UUID_VAR_PATH = get_dev_uuid_var_path()
|
||||
|
||||
def create_var_directories() -> None:
|
||||
|
@ -156,9 +147,6 @@ def main(options: argparse.Namespace) -> int:
|
|||
# The `build_emoji` script requires `emoji-datasource` package
|
||||
# which we install via npm; thus this step is after installing npm
|
||||
# packages.
|
||||
if not os.access(EMOJI_CACHE_PATH, os.W_OK):
|
||||
run_as_root(["mkdir", "-p", EMOJI_CACHE_PATH])
|
||||
run_as_root(["chown", "%s:%s" % (os.getuid(), os.getgid()), EMOJI_CACHE_PATH])
|
||||
run(["tools/setup/emoji/build_emoji"])
|
||||
|
||||
# copy over static files from the zulip_bots package
|
||||
|
|
|
@ -17,7 +17,7 @@ from emoji_names import EMOJI_NAME_MAPS
|
|||
ZULIP_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../../../')
|
||||
sys.path.append(ZULIP_PATH)
|
||||
|
||||
from scripts.lib.zulip_tools import generate_sha1sum_emoji
|
||||
from scripts.lib.zulip_tools import generate_sha1sum_emoji, run_as_root
|
||||
|
||||
TARGET_EMOJI_DUMP = os.path.join(ZULIP_PATH, 'static', 'generated', 'emoji')
|
||||
TARGET_EMOJI_STYLES = os.path.join(ZULIP_PATH, 'static', 'generated', 'emoji-styles')
|
||||
|
@ -77,6 +77,14 @@ if 'TRAVIS' in os.environ:
|
|||
EMOJI_CACHE_PATH = "/home/travis/zulip-emoji-cache"
|
||||
|
||||
def main() -> None:
|
||||
if not os.access(EMOJI_CACHE_PATH, os.W_OK):
|
||||
# Note: In production, this block will fail, since we don't
|
||||
# assume sudo access; but it should never run in production
|
||||
# anyway, because EMOJI_CACHE_PATH is created by puppet before
|
||||
# build_emoji would be run.
|
||||
run_as_root(["mkdir", "-p", EMOJI_CACHE_PATH])
|
||||
run_as_root(["chown", "%s:%s" % (os.getuid(), os.getgid()), EMOJI_CACHE_PATH])
|
||||
|
||||
sha1_hexdigest = generate_sha1sum_emoji(ZULIP_PATH)
|
||||
source_emoji_dump = os.path.join(EMOJI_CACHE_PATH, sha1_hexdigest, 'emoji')
|
||||
success_stamp = os.path.join(source_emoji_dump, '.success-stamp')
|
||||
|
|
Loading…
Reference in New Issue