deps: Change npm to yarn for reliablity, security, and speed.

.travis.yml

@@ -32,8 +32,9 @@ script:
   - unset GEM_PATH
   - mispipe "./tools/travis/$TEST_SUITE" ts
 cache:
-  - apt: false
+  yarn: true
-  - directories:
+  apt: false
+  directories:
     - $HOME/zulip-venv-cache
     - $HOME/zulip-npm-cache
     - $HOME/zulip-emoji-cache

docs/dev-env-first-time-contributors.md

@@ -886,50 +886,23 @@ gigabytes of RAM, which is the minimum Zulip
 not, go to your VM settings and increase the RAM, then restart
 the VM.
 
-##### npm install errors
+##### yarn install warnings
-
-The `tools/provision` script may encounter an error related to `npm install`
-that looks something like:
 
 ```
-==> default: + npm install
+$ yarn install
-==> default: Traceback (most recent call last):
+yarn install v0.24.5
-==> default:   File "/srv/zulip/tools/provision", line 195, in <module>
+[1/4] Resolving packages...
-==> default:
+[2/4] Fetching packages...
-==> default: sys.exit(main())
+warning fsevents@1.1.1: The platform "linux" is incompatible with this module.
-==> default:   File "/srv/zulip/tools/provision", line 191, in main
+info "fsevents@1.1.1" is an optional dependency and failed compatibility check. Excluding it from installation.
-==> default:
+[3/4] Linking dependencies...
-==> default: run(["npm", "install"])
+[4/4] Building fresh packages...
-==> default:   File "/srv/zulip/scripts/lib/zulip_tools.py", line 78, in run
+$ browserify node_modules/sockjs-client/lib/entry.js --standalone SockJS > node_modules/sockjs-client/sockjs.js
-==> default:
+Done in 23.50s.
-==> default: raise subprocess.CalledProcessError(rc, args)
-==> default: subprocess
-==> default: .
-==> default: CalledProcessError
-==> default: :
-==> default: Command '['npm', 'install']' returned non-zero exit status 34
-The SSH command responded with a non-zero exit status. Vagrant
-assumes that this means the command failed. The output for this command
-should be in the log above. Please read the output to determine what
-went wrong.
 ```
 
-Usually this error is not fatal. Try connecting to the development
+These are warnings produced by spammy third party JavaScript packages.
-environment and re-trying the command from withing the virtual
+It is okay to proceed and start the Zulip server.
-machine:
-
-```
-christie@win10 ~/zulip
-$ vagrant ssh
-(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:~
-$ cd zulip
-(zulip-venv)vagrant@vagrant-ubuntu-trusty-64:~/zulip
-$ npm install
-npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
-npm WARN notsup Not compatible with your operating system or architecture: fsevents@1.0.12
-```
-
-These are just warnings so it is okay to proceed and start the Zulip server.
 
 #### vagrant-lxc errors
 

docs/dev-setup-non-vagrant.md

@@ -46,7 +46,7 @@ should work.
 Install the following non-Python dependencies:
  * libffi-dev — needed for some Python extensions
  * postgresql 9.1 or later — our database (client, server, headers)
- * nodejs 0.10 (and npm)
+ * nodejs 0.10 (and yarn)
  * memcached (and headers)
  * rabbitmq-server
  * libldap2-dev
@@ -69,7 +69,7 @@ sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \
     memcached rabbitmq-server libldap2-dev redis-server \
     postgresql-server-dev-all libmemcached-dev python-dev \
     python3-dev python-virtualenv hunspell-en-us nodejs \
-    nodejs-legacy npm git yui-compressor puppet gettext postgresql
+    nodejs-legacy git yui-compressor puppet gettext postgresql
 
 # If using Ubuntu, install PGroonga from its PPA
 sudo add-apt-repository -ys ppa:groonga/ppa
@@ -128,7 +128,7 @@ sudo apt-get update
 sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \
     memcached rabbitmq-server libldap2-dev redis-server \
     postgresql-server-dev-all libmemcached-dev python-dev \
-    hunspell-en-us nodejs nodejs-legacy npm git yui-compressor \
+    hunspell-en-us nodejs nodejs-legacy git yui-compressor \
     puppet gettext tsearch-extras
 ```
 
@@ -146,7 +146,7 @@ https://github.com/zulip/zulip.git`
 sudo dnf install libffi-devel memcached rabbitmq-server \
     openldap-devel python-devel redis postgresql-server \
     postgresql-devel postgresql libmemcached-devel freetype-devel \
-    nodejs npm yuicompressor closure-compiler gettext
+    nodejs yuicompressor closure-compiler gettext
 ```
 
 Now continue with the [Common to Fedora/CentOS](#common-to-fedora-centos-instructions) instructions below.
@@ -324,7 +324,7 @@ Now run these commands:
 
 ```
 sudo ./scripts/lib/install-node
-npm install
+yarn install
 ./tools/install-mypy
 sudo mkdir /srv/zulip-emoji-cache
 sudo chown -R `whoami`:`whoami` /srv/zulip-emoji-cache
@@ -364,10 +364,10 @@ proxy in the environment as follows:
  export http_proxy=http://proxy_host:port
  ```
 
-- And set the npm proxy and https-proxy using:
+- And set the yarn proxy and https-proxy using:
  ```
- npm config set proxy http://proxy_host:port
+ yarn config set proxy http://proxy_host:port
- npm config set https-proxy http://proxy_host:port
+ yarn config set https-proxy http://proxy_host:port
  ```
 
 ## Using Docker (experimental)

docs/directory-structure.md

@@ -61,7 +61,7 @@ templating systems.
 
 * `static/third/` Third-party JavaScript and CSS that has been vendored.
 
-* `node_modules/` Third-party JavaScript installed via `npm`.
+* `node_modules/` Third-party JavaScript installed via `yarn`.
 
 * `static/assets/` For assets not to be served to the web (e.g. the system to
                    generate our favicons).

docs/front-end-build-process.md

@@ -23,11 +23,11 @@ To add a static file to the app (JavaScript, CSS, images, etc), first
 add it to the appropriate place under `static/`.
 
 - Third-party files that we haven't patched should be installed via
-  `npm`, so that it's easy to upgrade them and third-party code
+  `yarn`, so that it's easy to upgrade them and third-party code
   doesn't bloat the Zulip repository.  You can then access them in
   `webpack.assets.json` via their paths under `node_modules`.
   You'll want to add these to the `package.json` in the root of the
-  repository, and then provision (to have `npm` download them) before
+  repository, and then provision (to have `yarn` download them) before
   continuing.  Your commit should also update `PROVISION_VERSION` in
   `version.py`.  When adding modules to `package.json`, please pin
   specific versions of them (don't using carets `^`, tildes `~`, etc).

docs/git-guide.md

@@ -1460,6 +1460,19 @@ $ tools/clean-branches --reviews
 Deleting local branch review-original-5156 (was 5a1e982)
 ```
 
+### Merge conflict on yarn.lock file
+
+If there is a merge conflict on yarn.lock, yarn.lock, yarn should be run to
+regenerate the file. *Important* don't delete the yarn.lock file. Checkout the
+latest one from origin/master so that yarn knows the previous asset versions.
+
+Run the following commands
+```
+git checkout origin/master -- yarn.lock
+yarn install
+git add yarn.lock
+git rebase --continue
+```
 
 [gitbook-rebase]: https://git-scm.com/book/en/v2/Git-Branching-Rebasing
 [gitbook-git-rebase]: https://git-scm.com/docs/git-rebase

puppet/zulip/manifests/static_asset_compiler.pp

@@ -9,7 +9,6 @@ class zulip::static_asset_compiler {
                                      $closure_compiler_package,
                                      "nodejs",
                                      "nodejs-legacy",
-                                     "npm",
                                      "yui-compressor",
                                      # Used by makemessages i18n
                                      "gettext",

scripts/lib/install-node

@@ -1,25 +1,26 @@
 #!/usr/bin/env bash
 set -e
 
-ZULIP_PATH=$(dirname "$0")
+ZULIP_PATH="$(dirname "$0")/../.."
-node_version=6.6.0
+ZULIP_SRV="/srv"
-npm_version=3.10.3
+if [ "$TRAVIS" ] ; then
-
+    ZULIP_SRV="/home/travis"
-current_npm_version="none"
-if hash npm 2>/dev/null; then
-    current_npm_version="$(npm --version)"
 fi
+YARN_BIN="$ZULIP_SRV/zulip-yarn/bin/yarn"
+node_version=6.6.0
+yarn_version=0.27.5
+
 current_node_version="none"
 if hash node 2>/dev/null; then
     current_node_version="$(node --version)"
 fi
 
-if [ "$current_npm_version" = "$npm_version" ] && [ "$current_node_version" = "v$node_version" ]; then
+if [ "$($YARN_BIN --version 2>/dev/null)" = "$yarn_version" ] && [ "$current_node_version" = "v$node_version" ]; then
-    echo "Node version $node_version and npm version $npm_version are already installed."
+    echo "Node version $node_version and yarn version $yarn_version are already installed."
     exit 0
 fi
 
-if true; then
+if [ "$current_node_version" != "v$node_version" ]; then
     export NVM_DIR=/usr/local/nvm
     if ! [ -e "$NVM_DIR/nvm.sh" ]; then
         wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.32.0/install.sh | bash
@@ -28,7 +29,6 @@ if true; then
     source "$NVM_DIR/nvm.sh"
     nvm install "$node_version" && nvm alias default "$node_version"
     export NODE_BIN="$(nvm which default)"
-    export NPM_BIN=$(echo "$NODE_BIN" | sed 's/node$/npm/')
 
     # Fix messed-up uid=500 and group write bits produced by nvm
     n=$(which node)
@@ -36,9 +36,10 @@ if true; then
     chown -R root:root "$n"
     chmod -R go-w "$n"
 
-    # Install node and npm wrappers to /usr/local/bin
+    # Install node wrapper to /usr/local/bin
-    cp "$ZULIP_PATH/../../scripts/setup/node-wrapper" /usr/local/bin/node
+    cp "$ZULIP_PATH/scripts/setup/node-wrapper" /usr/local/bin/node
     sed -i "s|NODE_PATH|$NODE_BIN|" /usr/local/bin/node
-    cp "$ZULIP_PATH/../../scripts/setup/npm-wrapper" /usr/local/bin/npm
-    sed -i "s|NPM_PATH|$NPM_BIN|" /usr/local/bin/npm
 fi
+
+# Install yarn if not installed
+bash "$ZULIP_PATH/scripts/lib/third/install-yarn.sh" "$ZULIP_SRV" --version "$yarn_version"

scripts/lib/node_cache.py

@@ -10,41 +10,49 @@ if False:
 from scripts.lib.zulip_tools import subprocess_text_output, run
 
 ZULIP_PATH = dirname(dirname(dirname(abspath(__file__))))
-NODE_MODULES_CACHE_PATH = "/srv/zulip-npm-cache"
+ZULIP_SRV_PATH = "/srv"
 
 if 'TRAVIS' in os.environ:
     # In Travis CI, we don't have root access
-    NODE_MODULES_CACHE_PATH = "/home/travis/zulip-npm-cache"
+    ZULIP_SRV_PATH = "/home/travis"
 
-def generate_sha1sum_node_modules(npm_args=None):
+
+NODE_MODULES_CACHE_PATH = os.path.join(ZULIP_SRV_PATH, 'zulip-npm-cache')
+YARN_BIN = os.path.join(ZULIP_SRV_PATH, 'zulip-yarn/bin/yarn')
+
+def generate_sha1sum_node_modules(yarn_args=None):
     # type: (Optional[List[str]]) -> str
     sha1sum = hashlib.sha1()
     sha1sum.update(subprocess_text_output(['cat', 'package.json']).encode('utf8'))
-    sha1sum.update(subprocess_text_output(['npm', '--version']).encode('utf8'))
+    sha1sum.update(subprocess_text_output(['cat', 'yarn.lock']).encode('utf8'))
+    sha1sum.update(subprocess_text_output([YARN_BIN, '--version']).encode('utf8'))
     sha1sum.update(subprocess_text_output(['node', '--version']).encode('utf8'))
-    if npm_args is not None:
+    if yarn_args is not None:
-        sha1sum.update(''.join(sorted(npm_args)).encode('utf8'))
+        sha1sum.update(''.join(sorted(yarn_args)).encode('utf8'))
 
     return sha1sum.hexdigest()
 
-def setup_node_modules(production=False, stdout=None, stderr=None, copy_modules=False):
+def setup_node_modules(production=False, stdout=None, stderr=None, copy_modules=False,
-    # type: (bool, Optional[IO], Optional[IO], bool) -> None
+                       prefer_offline=False):
+    # type: (bool, Optional[IO], Optional[IO], bool, bool) -> None
     if production:
-        npm_args = ["--production"]
+        yarn_args = ["--prod"]
     else:
-        npm_args = []
+        yarn_args = []
-    sha1sum = generate_sha1sum_node_modules(npm_args)
+    if prefer_offline:
+        yarn_args.append("--prefer-offline")
+    sha1sum = generate_sha1sum_node_modules(yarn_args)
     target_path = os.path.join(NODE_MODULES_CACHE_PATH, sha1sum)
     cached_node_modules = os.path.join(target_path, 'node_modules')
     success_stamp = os.path.join(target_path, '.success-stamp')
     # Check if a cached version already exists
     if not os.path.exists(success_stamp):
-        do_npm_install(target_path,
+        do_yarn_install(target_path,
-                       npm_args,
+                        yarn_args,
-                       success_stamp,
+                        success_stamp,
-                       stdout=stdout,
+                        stdout=stdout,
-                       stderr=stderr,
+                        stderr=stderr,
-                       copy_modules=copy_modules)
+                        copy_modules=copy_modules)
 
     print("Using cached node modules from %s" % (cached_node_modules,))
     cmds = [
@@ -54,13 +62,12 @@ def setup_node_modules(production=False, stdout=None, stderr=None, copy_modules=
     for cmd in cmds:
         run(cmd, stdout=stdout, stderr=stderr)
 
-def do_npm_install(target_path, npm_args, success_stamp, stdout=None, stderr=None,
+def do_yarn_install(target_path, yarn_args, success_stamp, stdout=None, stderr=None,
-                   copy_modules=False):
+                    copy_modules=False):
     # type: (str, List[str], str, Optional[IO], Optional[IO], bool) -> None
     cmds = [
-        ["rm", "-rf", target_path],
         ['mkdir', '-p', target_path],
-        ['cp', 'package.json', target_path],
+        ['cp', 'package.json', "yarn.lock", target_path],
     ]
     cached_node_modules = os.path.join(target_path, 'node_modules')
     if copy_modules:
@@ -68,7 +75,13 @@ def do_npm_install(target_path, npm_args, success_stamp, stdout=None, stderr=Non
         cmds.append(["cp", "-rT", "prod-static/serve/node_modules", cached_node_modules])
     else:
         print("Cached version not found! Installing node modules.")
-        cmds.append(['npm', 'install'] + npm_args + ['--prefix', target_path])
+
+        # Copy the existing node_modules to speed up install
+        if os.path.exists("node_modules"):
+            cmds.append(["cp", "-R", "node_modules/", cached_node_modules])
+        cd_exec = os.path.join(ZULIP_PATH, "scripts/lib/cd_exec")
+        cmds.append([cd_exec, target_path, YARN_BIN, "install", "--non-interactive"] +
+                    yarn_args)
     cmds.append(['touch', success_stamp])
 
     for cmd in cmds:

scripts/lib/third/install-yarn.sh

@@ -1,16 +1,27 @@
 #!/bin/sh
 set -e
 
+#####################
+# install-yarn.sh was patched to install yarn in a custom directory.
+# The following changes were made:
+# * yarn_link now just simlinks to /usr/bin
+# * yarn_detect_profile was removed
+# * Paths were changed to variables declared at the top
+# * Most of the non error coloration was removed to not distract during installs.
+# #######################
+
 reset="\033[0m"
 red="\033[31m"
-green="\033[32m"
 yellow="\033[33m"
-cyan="\033[36m"
-white="\033[37m"
 gpg_key=9D41F3C3
 
+ZULIP_ROOT="$1"
+YARN_DIR_NAME="zulip-yarn"
+YARN_DIR="$ZULIP_ROOT/$YARN_DIR_NAME"
+YARN_BIN="$YARN_DIR/bin/yarn"
+
 yarn_get_tarball() {
-  printf "$cyan> Downloading tarball...$reset\n"
+  printf "Downloading tarball...\n"
   if [ "$1" = '--nightly' ]; then
     url=https://nightly.yarnpkg.com/latest.tar.gz
   elif [ "$1" = '--rc' ]; then
@@ -32,9 +43,9 @@ yarn_get_tarball() {
   if curl --fail -L -o "$tarball_tmp#1" "$url{,.asc}"; then
     yarn_verify_integrity $tarball_tmp
 
-    printf "$cyan> Extracting to ~/.yarn...$reset\n"
+    printf "Extracting to $YARN_DIR...\n"
-    mkdir .yarn
+    mkdir "$YARN_DIR_NAME"
-    tar zxf $tarball_tmp -C .yarn --strip 1 # extract tarball
+    tar zxf $tarball_tmp -C "$YARN_DIR_NAME" --strip 1 # extract tarball
     rm $tarball_tmp*
   else
     printf "$red> Failed to download $url.$reset\n"
@@ -51,11 +62,11 @@ yarn_verify_integrity() {
   fi
 
   if [ "$YARN_GPG" == "no" ]; then
-    printf "$cyan> WARNING: Skipping GPG integrity check!$reset\n"
+    printf "WARNING: Skipping GPG integrity check!\n"
     return
   fi
 
-  printf "$cyan> Verifying integrity...$reset\n"
+  printf "Verifying integrity...\n"
   # Grab the public key if it doesn't already exist
   gpg --list-keys $gpg_key >/dev/null 2>&1 || (curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --import)
 
@@ -67,7 +78,7 @@ yarn_verify_integrity() {
 
   # Actually perform the verification
   if gpg --verify "$1.asc" $1; then
-    printf "$green> GPG signature looks good$reset\n"
+    printf "GPG signature looks good\n"
   else
     printf "$red> GPG signature for this Yarn release is invalid! This is BAD and may mean the release has been tampered with. It is strongly recommended that you report this to the Yarn developers.$reset\n"
     yarn_verify_or_quit "> Do you really want to continue?"
@@ -75,90 +86,28 @@ yarn_verify_integrity() {
 }
 
 yarn_link() {
-  printf "$cyan> Adding to \$PATH...$reset\n"
+  printf "Adding to /usr/bin\n"
-  YARN_PROFILE="$(yarn_detect_profile)"
-  SOURCE_STR="\nexport PATH=\"\$HOME/.yarn/bin:\$PATH\"\n"
 
-  if [ -z "${YARN_PROFILE-}" ] ; then
+  version=`$YARN_BIN --version` || (
-    printf "$red> Profile not found. Tried ${YARN_PROFILE} (as defined in \$PROFILE), ~/.bashrc, ~/.bash_profile, ~/.zshrc, and ~/.profile.\n"
+    printf "$red> Yarn was installed, but doesn't seem to be working :(.$reset\n"
-    echo "> Create one of them and run this script again"
+    exit 1;
-    echo "> Create it (touch ${YARN_PROFILE}) and run this script again"
+  )
-    echo "   OR"
-    printf "> Append the following lines to the correct file yourself:$reset\n"
-    command printf "${SOURCE_STR}"
-  else
-    if ! grep -q 'yarn' "$YARN_PROFILE"; then
-      if [[ $YARN_PROFILE == *"fish"* ]]; then
-        command fish -c 'set -U fish_user_paths $fish_user_paths ~/.yarn/bin'
-      else
-        command printf "$SOURCE_STR" >> "$YARN_PROFILE"
-      fi
-    fi
 
-    printf "$cyan> We've added the following to your $YARN_PROFILE\n"
+  ln -nsf "$YARN_BIN" /usr/bin/yarn
-    echo "> If this isn't the profile of your current shell then please add the following to your correct profile:"
-    printf "   $SOURCE_STR$reset\n"
 
-    version=`$HOME/.yarn/bin/yarn --version` || (
+  printf "Successfully installed Yarn $version!\n"
-      printf "$red> Yarn was installed, but doesn't seem to be working :(.$reset\n"
-      exit 1;
-    )
-
-    printf "$green> Successfully installed Yarn $version! Please open another terminal where the \`yarn\` command will now be available.$reset\n"
-  fi
 }
 
-yarn_detect_profile() {
-  if [ -n "${PROFILE}" ] && [ -f "${PROFILE}" ]; then
-    echo "${PROFILE}"
-    return
-  fi
-
-  local DETECTED_PROFILE
-  DETECTED_PROFILE=''
-  local SHELLTYPE
-  SHELLTYPE="$(basename "/$SHELL")"
-
-  if [ "$SHELLTYPE" = "bash" ]; then
-    if [ -f "$HOME/.bashrc" ]; then
-      DETECTED_PROFILE="$HOME/.bashrc"
-    elif [ -f "$HOME/.bash_profile" ]; then
-      DETECTED_PROFILE="$HOME/.bash_profile"
-    fi
-  elif [ "$SHELLTYPE" = "zsh" ]; then
-    DETECTED_PROFILE="$HOME/.zshrc"
-  elif [ "$SHELLTYPE" = "fish" ]; then
-    DETECTED_PROFILE="$HOME/.config/fish/config.fish"
-  fi
-
-  if [ -z "$DETECTED_PROFILE" ]; then
-    if [ -f "$HOME/.profile" ]; then
-      DETECTED_PROFILE="$HOME/.profile"
-    elif [ -f "$HOME/.bashrc" ]; then
-      DETECTED_PROFILE="$HOME/.bashrc"
-    elif [ -f "$HOME/.bash_profile" ]; then
-      DETECTED_PROFILE="$HOME/.bash_profile"
-    elif [ -f "$HOME/.zshrc" ]; then
-      DETECTED_PROFILE="$HOME/.zshrc"
-    elif [ -f "$HOME/.config/fish/config.fish" ]; then
-      DETECTED_PROFILE="$HOME/.config/fish/config.fish"
-    fi
-  fi
-
-  if [ ! -z "$DETECTED_PROFILE" ]; then
-    echo "$DETECTED_PROFILE"
-  fi
-}
 
 yarn_reset() {
-  unset -f yarn_install yarn_reset yarn_get_tarball yarn_link yarn_detect_profile yarn_verify_integrity yarn_verify_or_quit
+  unset -f yarn_install yarn_reset yarn_get_tarball yarn_link yarn_verify_integrity yarn_verify_or_quit
 }
 
 yarn_install() {
-  printf "${white}Installing Yarn!$reset\n"
+  printf "Installing Yarn!\n"
 
-  if [ -d "$HOME/.yarn" ]; then
+  if [ -d "$YARN_DIR" ]; then
-      if [ -e "$HOME/.yarn/bin/yarn" ] ; then
+    if [ -e "$YARN_BIN" ] ; then
       local latest_url
       local specified_version
       local version_type
@@ -178,17 +127,17 @@ yarn_install() {
         specified_version=`curl -sS $latest_url`
         version_type='latest'
       fi
-      yarn_version=`$HOME/.yarn/bin/yarn -V`
+      yarn_version=`$YARN_BIN -V`
-      yarn_alt_version=`$HOME/.yarn/bin/yarn --version`
+      yarn_alt_version=`$YARN_BIN --version`
       if [ "$specified_version" = "$yarn_version" -o "$specified_version" = "$yarn_alt_version" ]; then
         printf "Yarn is already at the $specified_version version.\n"
         exit 0
       else
-        rm -rf "$HOME/.yarn"
+        rm -rf "$YARN_DIR"
       fi
     else
-      printf "$red> $HOME/.yarn already exists, possibly from a past Yarn install.$reset\n"
+      printf "$red> $YARN_DIR already exists, possibly from a past Yarn install.$reset\n"
-      printf "$red> Remove it (rm -rf $HOME/.yarn) and run this script again.$reset\n"
+      printf "$red> Remove it (rm -rf $YARN_DIR) and run this script again.$reset\n"
       exit 0
     fi
   fi
@@ -208,5 +157,5 @@ yarn_verify_or_quit() {
   fi
 }
 
-cd ~
+cd $ZULIP_ROOT
-yarn_install $1 $2
+yarn_install $2 $3

scripts/setup/npm-wrapper

@@ -1,2 +0,0 @@
-#!/bin/bash
-NPM_PATH "$@"

tools/clean-npm-cache

@@ -12,16 +12,18 @@ NODE_MODULES_CACHE_PATH = "/srv/zulip-npm-cache"
 if "--travis" in sys.argv:
     NODE_MODULES_CACHE_PATH = os.path.join(os.environ["HOME"], "zulip-npm-cache")
     try:
-        subprocess.check_output(['npm', '--version'])
+        subprocess.check_output([os.path.join(NODE_MODULES_CACHE_PATH, "yarn/bin/yarn"), '--version'])
     except OSError:
-        print('NPM not found. Most probably we are running static-analysis and '
+        print('yarn not found. Most probably we are running static-analysis and '
-              'hence npm is not installed. Exiting without cleaning npm cache.')
+              'hence yarn is not installed. Exiting without cleaning npm cache.')
         sys.exit(0)
 
 sha1sum = generate_sha1sum_node_modules()
 current_cache_dir_base = os.path.join(NODE_MODULES_CACHE_PATH, sha1sum)
 current_success_stamp = os.path.join(current_cache_dir_base, '.success-stamp')
 
+print("Current cache stamp at %s" % (current_success_stamp,))
+
 for cache_dir_base in os.listdir(NODE_MODULES_CACHE_PATH):
     node_modules_dir = os.path.join(NODE_MODULES_CACHE_PATH, cache_dir_base)
     if node_modules_dir == current_cache_dir_base and os.path.exists(current_success_stamp):

tools/lib/provision.py

@@ -192,7 +192,7 @@ def install_apt_deps():
 def main(options):
     # type: (Any) -> int
 
-    # npm install and management commands expect to be run from the root of the
+    # yarn and management commands expect to be run from the root of the
     # project.
     os.chdir(ZULIP_PATH)
 
@@ -226,11 +226,11 @@ def main(options):
     else:
         print("No need to apt operations.")
 
-    # Here we install nvm, node, and npm.
+    # Here we install node.
     run(["sudo", "scripts/lib/install-node"])
 
-    # Install NPM packages before running other scripts so that if a script
+    # This is a wrapper around `yarn`, which we run last since
-    # requires any NPM package it can use it.
+    # it can often fail due to network issues beyond our control.
     try:
         # Hack: We remove `node_modules` as root to work around an
         # issue with the symlinks being improperly owned by root.
@@ -239,9 +239,9 @@ def main(options):
         if not os.path.isdir(NODE_MODULES_CACHE_PATH):
             run(["sudo", "mkdir", NODE_MODULES_CACHE_PATH])
         run(["sudo", "chown", "%s:%s" % (user_id, user_id), NODE_MODULES_CACHE_PATH])
-        setup_node_modules()
+        setup_node_modules(prefer_offline=True)
     except subprocess.CalledProcessError:
-        print(WARNING + "`npm install` failed; retrying..." + ENDC)
+        print(WARNING + "`yarn install` failed; retrying..." + ENDC)
         setup_node_modules()
 
     # Import tools/setup_venv.py instead of running it so that we get an

version.py

@@ -1,2 +1,2 @@
 ZULIP_VERSION = "1.6.0+git"
-PROVISION_VERSION = '6.1'
+PROVISION_VERSION = '7.0'