mirror of https://github.com/zulip/zulip.git
1065 lines
39 KiB
Markdown
1065 lines
39 KiB
Markdown
## Vagrant environment setup tutorial
|
|
|
|
This section guides first-time contributors through installing the
|
|
Zulip development environment on Windows, macOS, Ubuntu and Debian.
|
|
|
|
The recommended method for installing the Zulip development environment is to use
|
|
Vagrant with VirtualBox on Windows and macOS, and Vagrant with LXC on
|
|
Ubuntu. This method creates a virtual machine (for Windows and macOS)
|
|
or a Linux container (for Ubuntu) inside which the Zulip server and
|
|
all related services will run.
|
|
|
|
Contents:
|
|
* [Requirements](#requirements)
|
|
* [Step 0: Set up Git & GitHub](#step-0-set-up-git-github)
|
|
* [Step 1: Install Prerequisites](#step-1-install-prerequisites)
|
|
* [Step 2: Get Zulip code](#step-2-get-zulip-code)
|
|
* [Step 3: Start the development environment](#step-3-start-the-development-environment)
|
|
* [Step 4: Developing](#step-4-developing)
|
|
* [Troubleshooting and Common Errors](#troubleshooting-and-common-errors)
|
|
* [Specifying a proxy](#specifying-a-proxy)
|
|
|
|
**If you encounter errors installing the Zulip development
|
|
environment,** check
|
|
[Troubleshooting and Common Errors](#troubleshooting-and-common-errors). If
|
|
that doesn't help, please visit
|
|
[#provision help](https://chat.zulip.org/#narrow/stream/21-provision-help)
|
|
in the [Zulip development community server](../contributing/chat-zulip-org.html) for
|
|
real-time help, send a note to the
|
|
[Zulip-devel Google group](https://groups.google.com/forum/#!forum/zulip-devel)
|
|
or [file an issue](https://github.com/zulip/zulip/issues).
|
|
|
|
When reporting your issue, please include the following information:
|
|
|
|
* host operating system
|
|
* installation method (Vagrant or direct)
|
|
* whether or not you are using a proxy
|
|
* a copy of Zulip's `vagrant` provisioning logs, available in
|
|
`/var/log/provision.log` on your virtual machine
|
|
|
|
### Requirements
|
|
|
|
Installing the Zulip development environment with Vagrant requires
|
|
downloading several hundred megabytes of dependencies. You will need
|
|
an active internet connection throughout the entire installation
|
|
processes. (See [Specifying a proxy](#specifying-a-proxy) if you need
|
|
a proxy to access the internet.)
|
|
|
|
- **All**: 2GB available RAM, Active broadband internet connection, [GitHub account][set-up-git].
|
|
- **macOS**: macOS (10.11 El Capitan or newer recommended)
|
|
- **Ubuntu LTS**: 18.04, 16.04, or 14.04 64-bit
|
|
- or **Debian**: 9.0 "stretch" 64-bit
|
|
- **Windows**: Windows 64-bit (Win 10 recommended), hardware
|
|
virtualization enabled (VT-X or AMD-V), administrator access.
|
|
|
|
Other Linux distributions work great too, but we don't maintain
|
|
documentation for installing Vagrant and LXC on those systems, so
|
|
you'll need to find a separate guide and crib from the Debian/Ubuntu
|
|
docs.
|
|
|
|
### Step 0: Set up Git & GitHub
|
|
|
|
You can skip this step if you already have Git, GitHub, and SSH access
|
|
to GitHub working on your machine.
|
|
|
|
Follow our [Git Guide][set-up-git] in order to install Git, set up a
|
|
GitHub account, create an SSH key to access code on GitHub
|
|
efficiently, etc. Be sure to create an ssh key and add it to your
|
|
GitHub account using
|
|
[these instructions](https://help.github.com/articles/generating-an-ssh-key/).
|
|
|
|
### Step 1: Install Prerequisites
|
|
|
|
Jump to:
|
|
|
|
* [macOS](#macos)
|
|
* [Ubuntu](#ubuntu)
|
|
* [Debian](#debian)
|
|
* [Windows](#windows-10)
|
|
|
|
#### macOS
|
|
|
|
0. If you are running MacOS High Sierra, make sure you are not running
|
|
a version with a
|
|
[buggy NFS implementation](#importerror-no-module-named-on-macos-during-vagrant-provisioning).
|
|
Versions 10.13.2 and above have the bug fixed.
|
|
1. Install [Vagrant][vagrant-dl-macos] (2.0.2).
|
|
2. Install [VirtualBox][vbox-dl] (latest).
|
|
|
|
(For a non-free option, but better performance, you can also use [VMWare
|
|
Fusion][vmware-fusion-dl] with the [VMWare Fusion Vagrant
|
|
plugin][vagrant-vmware-fusion-dl].)
|
|
|
|
Now you are ready for [Step 2: Get Zulip Code.](#step-2-get-zulip-code).
|
|
|
|
#### Ubuntu
|
|
|
|
If you're in a hurry, you can copy and paste the following into your terminal
|
|
after which you can jump to [Step 2: Get Zulip Code](#step-2-get-zulip-code):
|
|
|
|
```
|
|
sudo apt-get -y purge vagrant && \
|
|
wget https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.deb && \
|
|
sudo dpkg -i vagrant*.deb && \
|
|
sudo apt-get -y install build-essential git ruby lxc lxc-templates cgroup-lite redir && \
|
|
vagrant plugin install vagrant-lxc && \
|
|
vagrant lxc sudoers
|
|
```
|
|
|
|
For a step-by-step explanation, read on.
|
|
|
|
##### 1. Install Vagrant
|
|
|
|
For Ubuntu 18.04 Bionic, you can just install from `apt`:
|
|
|
|
```
|
|
sudo apt install vagrant
|
|
```
|
|
|
|
For Ubuntu 16.04 Xenial and 14.04 Trusty, you'll need a more recent
|
|
version of Vagrant than what's available in the official Ubuntu
|
|
repositories.
|
|
|
|
First uninstall any vagrant package you may have installed from the Ubuntu
|
|
repository:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ sudo apt-get purge vagrant
|
|
```
|
|
|
|
Now download and install the .deb package for [Vagrant 2.0.2][vagrant-dl-deb]:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ wget https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.deb
|
|
|
|
christie@ubuntu-desktop:~
|
|
$ sudo dpkg -i vagrant*.deb
|
|
```
|
|
|
|
##### 2. Install remaining dependencies
|
|
|
|
Now install git and lxc-related packages:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ sudo apt-get install build-essential git ruby lxc lxc-templates cgroup-lite redir
|
|
```
|
|
|
|
##### 3. Install the vagrant lxc plugin:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ vagrant plugin install vagrant-lxc
|
|
Installing the 'vagrant-lxc' plugin. This can take a few minutes...
|
|
Installed the plugin 'vagrant-lxc (1.2.1)'!
|
|
```
|
|
|
|
If you encounter an error when trying to install the vagrant-lxc plugin, [see
|
|
this](#nomethoderror).
|
|
|
|
##### 4. Configure sudo to be passwordless
|
|
|
|
Finally, [configure sudo to be passwordless when using Vagrant LXC][avoiding-sudo]:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ vagrant lxc sudoers
|
|
[sudo] password for christie:
|
|
```
|
|
|
|
If you encounter an error running `vagrant lxc sudoers`, [see
|
|
this](#permissions-errors).
|
|
|
|
Now you are ready for [Step 2: Get Zulip Code.](#step-2-get-zulip-code)
|
|
|
|
#### Debian
|
|
|
|
The setup for Debian 9.0 "stretch" is very similar to that
|
|
[for Ubuntu above](#ubuntu). Follow those instructions, except with
|
|
the following differences:
|
|
|
|
**Apt package list**. In "2. Install remaining dependencies", the
|
|
command to install the dependencies is a bit shorter:
|
|
|
|
```
|
|
christie@ubuntu-desktop:~
|
|
$ sudo apt-get install build-essential git ruby lxc redir
|
|
```
|
|
|
|
**Set up LXC networking**. After completing "2. Install remaining
|
|
dependencies", you will have to set up networking for LXC containers,
|
|
because Debian's packaging for LXC does not ship any default
|
|
network setup for them. You can do this by
|
|
[following the steps][lxc-networking-quickstart] outlined in
|
|
[Debian's LXC docs](https://wiki.debian.org/LXC#network_setup).
|
|
|
|
[lxc-networking-quickstart]: https://wiki.debian.org/LXC#Minimal_changes_to_set_up_networking_for_LXC_for_Debian_.2BIBw-stretch.2BIB0_.28testing.29
|
|
|
|
Then return to the next step in the Ubuntu instructions above. After
|
|
finishing those steps, you will be ready for
|
|
[Step 2: Get Zulip Code](#step-2-get-zulip-code).
|
|
|
|
#### Windows 10
|
|
|
|
1. Install [Git for Windows][git-bash], which installs *Git BASH*.
|
|
2. Install [VirtualBox][vbox-dl] (latest).
|
|
3. Install [Vagrant][vagrant-dl-win] (version 2.0.2 is best tested).
|
|
|
|
(Note: While *Git BASH* is recommended, you may also use [Cygwin][cygwin-dl].
|
|
If you do, make sure to **install default required packages** along with
|
|
**git**, **curl**, **openssh**, and **rsync** binaries.)
|
|
|
|
Also, you must have hardware virtualization enabled (VT-X or AMD-V) in your
|
|
computer's BIOS.
|
|
|
|
#### Running Git BASH as an administrator
|
|
|
|
It is important that you **always run Git BASH with administrator
|
|
privileges** when working on Zulip code, as not doing so will cause
|
|
errors in the development environment (such as symlink creation). You
|
|
might wish to configure your Git BASH shortcut to always run with
|
|
these privileges enabled (see this [guide][bash-admin-setup] for how
|
|
to set this up).
|
|
|
|
##### Enable native symlinks
|
|
|
|
The Zulip code includes symbolic links (symlinks). By default, native Windows
|
|
symlinks are not enabled in either Git BASH or Cygwin, so you need to do a bit
|
|
of configuration. **You must do this before you clone the Zulip code.**
|
|
|
|
In **Git for BASH**:
|
|
|
|
Open **Git BASH as an administrator** and run:
|
|
|
|
```
|
|
$ git config --global core.symlinks true
|
|
```
|
|
|
|
Now confirm the setting:
|
|
|
|
```
|
|
$ git config core.symlinks
|
|
true
|
|
```
|
|
|
|
If you see `true`, you are ready for [Step 2: Get Zulip
|
|
Code.](#step-2-get-zulip-code)
|
|
|
|
Otherwise, if the above command prints `false` or nothing at all, then symlinks
|
|
have not been enabled.
|
|
|
|
In **Cygwin**:
|
|
|
|
Open a Cygwin window **as an administrator** and do this:
|
|
|
|
```
|
|
christie@win10 ~
|
|
$ echo 'export "CYGWIN=$CYGWIN winsymlinks:native"' >> ~/.bash_profile
|
|
```
|
|
|
|
Next, close that Cygwin window and open another. If you `echo` $CYGWIN you
|
|
should see:
|
|
|
|
```
|
|
christie@win10 ~
|
|
$ echo $CYGWIN
|
|
winsymlinks:native
|
|
```
|
|
|
|
Now you are ready for [Step 2: Get Zulip Code.](#step-2-get-zulip-code)
|
|
|
|
### Step 2: Get Zulip Code
|
|
|
|
1. In your browser, visit <https://github.com/zulip/zulip>
|
|
and click the `fork` button. You will need to be logged in to GitHub to
|
|
do this.
|
|
2. Open Terminal (macOS/Ubuntu) or Git BASH (Windows; must
|
|
**run as an Administrator**).
|
|
3. In Terminal/Git BASH,
|
|
[clone your fork of the Zulip repository](../git/cloning.html#step-1b-clone-to-your-machine)
|
|
and [connect the Zulip upstream repository](../git/cloning.html#step-1c-connect-your-fork-to-zulip-upstream):
|
|
|
|
```
|
|
git clone --config pull.rebase git@github.com:YOURUSERNAME/zulip.git
|
|
git remote add -f upstream https://github.com/zulip/zulip.git
|
|
```
|
|
|
|
This will create a 'zulip' directory and download the Zulip code into it.
|
|
|
|
Don't forget to replace YOURUSERNAME with your git username. You will see
|
|
something like:
|
|
|
|
```
|
|
christie@win10 ~
|
|
$ git clone --config pull.rebase git@github.com:YOURUSERNAME/zulip.git
|
|
Cloning into 'zulip'...
|
|
remote: Counting objects: 73571, done.
|
|
remote: Compressing objects: 100% (2/2), done.
|
|
remote: Total 73571 (delta 1), reused 0 (delta 0), pack-reused 73569
|
|
Receiving objects: 100% (73571/73571), 105.30 MiB | 6.46 MiB/s, done.
|
|
Resolving deltas: 100% (51448/51448), done.
|
|
Checking connectivity... done.
|
|
Checking out files: 100% (1912/1912), done.`
|
|
```
|
|
|
|
Now you are ready for [Step 3: Start the development
|
|
environment.](#step-3-start-the-development-environment)
|
|
|
|
### Step 3: Start the development environment
|
|
|
|
Change into the zulip directory and tell vagrant to start the Zulip
|
|
development environment with `vagrant up`.
|
|
|
|
```
|
|
christie@win10 ~
|
|
$ cd zulip
|
|
|
|
christie@win10 ~/zulip
|
|
$ vagrant up
|
|
```
|
|
|
|
The first time you run this command it will take some time because vagrant
|
|
does the following:
|
|
|
|
- downloads the base Ubuntu 14.04 virtual machine image (for macOS and Windows)
|
|
or container (for Ubuntu)
|
|
- configures this virtual machine/container for use with Zulip,
|
|
- creates a shared directory mapping your clone of the Zulip code inside the
|
|
virtual machine/container at `~/zulip`
|
|
- runs the `tools/provision` script inside the virtual machine/container, which
|
|
downloads all required dependencies, sets up the python environment for
|
|
the Zulip development server, and initializes a default test
|
|
database. We call this process "provisioning", and it is documented
|
|
in some detail in our [dependencies documentation](../subsystems/dependencies.html).
|
|
|
|
You will need an active internet connection during the entire
|
|
process. (See [Specifying a proxy](#specifying-a-proxy) if you need a
|
|
proxy to access the internet.) `vagrant up` can fail while
|
|
provisioning if your Internet connection is unreliable. To retry, you
|
|
can use `vagrant provision` (`vagrant up` will just boot the guest
|
|
without provisioning after the first time). Other common issues are
|
|
documented in the
|
|
[Troubleshooting and Common Errors](#troubleshooting-and-common-errors)
|
|
section. If that doesn't help, please visit
|
|
[#provision help](https://chat.zulip.org/#narrow/stream/21-provision-help)
|
|
in the [Zulip development community server](../contributing/chat-zulip-org.html) for
|
|
real-time help.
|
|
|
|
On Windows, you will see `The system cannot find the path specified.` message
|
|
several times. This is expected behavior and is not an error.
|
|
|
|
Once `vagrant up` has completed, connect to the development
|
|
environment with `vagrant ssh`:
|
|
|
|
```
|
|
christie@win10 ~/zulip
|
|
$ vagrant ssh
|
|
```
|
|
|
|
You should see something like this on Windows and macOS:
|
|
|
|
```
|
|
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64)
|
|
|
|
* Documentation: https://help.ubuntu.com/
|
|
|
|
System information as of Wed May 4 21:45:43 UTC 2016
|
|
|
|
System load: 0.61 Processes: 88
|
|
Usage of /: 3.5% of 39.34GB Users logged in: 0
|
|
Memory usage: 7% IP address for eth0: 10.0.2.15
|
|
Swap usage: 0%
|
|
|
|
Graph this data and manage this system at:
|
|
https://landscape.canonical.com/
|
|
|
|
Get cloud support with Ubuntu Advantage Cloud Guest:
|
|
http://www.ubuntu.com/business/services/cloud
|
|
|
|
0 packages can be updated.
|
|
0 updates are security updates.
|
|
```
|
|
|
|
Or something as brief as this in the case of Ubuntu:
|
|
|
|
```
|
|
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 4.4.0-21-generic x86_64)
|
|
|
|
* Documentation: https://help.ubuntu.com/
|
|
```
|
|
|
|
Congrats, you're now inside the Zulip development environment!
|
|
|
|
You can confirm this by looking at the command prompt, which starts
|
|
with `(zulip-py3-venv)vagrant@`. If it just starts with `vagrant@`, your
|
|
provisioning failed and you should look at the
|
|
[troubleshooting section](#troubleshooting-and-common-errors).
|
|
|
|
Next, start the Zulip server:
|
|
|
|
```
|
|
(zulip-py3-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip
|
|
$ ./tools/run-dev.py
|
|
```
|
|
|
|
You will see several lines of output starting with something like:
|
|
|
|
```
|
|
2016-05-04 22:20:33,895 INFO: process_fts_updates starting
|
|
Recompiling templates
|
|
2016-05-04 18:20:34,804 INFO: Not in recovery; listening for FTS updates
|
|
done
|
|
Validating Django models.py...
|
|
System check identified no issues (0 silenced).
|
|
|
|
Django version 1.8
|
|
Tornado server is running at http://localhost:9993/
|
|
Quit the server with CTRL-C.
|
|
2016-05-04 18:20:40,716 INFO Tornado loaded 0 event queues in 0.001s
|
|
2016-05-04 18:20:40,722 INFO Tornado 95.5% busy over the past 0.0 seconds
|
|
Performing system checks...
|
|
```
|
|
And ending with something similar to:
|
|
|
|
```
|
|
http://localhost:9994/webpack-dev-server/
|
|
webpack result is served from http://localhost:9991/webpack/
|
|
content is served from /srv/zulip
|
|
|
|
webpack: bundle is now VALID.
|
|
2016-05-06 21:43:29,553 INFO Tornado 31.6% busy over the past 10.6 seconds
|
|
2016-05-06 21:43:35,007 INFO Tornado 23.9% busy over the past 16.0 seconds
|
|
```
|
|
|
|
Now the Zulip server should be running and accessible. Verify this by
|
|
navigating to <http://localhost:9991/> in the browser on your main machine.
|
|
|
|
You should see something like this:
|
|
|
|
|
|
|
|
The Zulip server will continue to run and send output to the terminal window.
|
|
When you navigate to Zulip in your browser, check your terminal and you
|
|
should see something like:
|
|
|
|
```
|
|
2016-05-04 18:21:57,547 INFO 127.0.0.1 GET 302 582ms (+start: 417ms) / (unauth via ?)
|
|
[04/May/2016 18:21:57]"GET / HTTP/1.0" 302 0
|
|
2016-05-04 18:21:57,568 INFO 127.0.0.1 GET 301 4ms /login (unauth via ?)
|
|
[04/May/2016 18:21:57]"GET /login HTTP/1.0" 301 0
|
|
2016-05-04 18:21:57,819 INFO 127.0.0.1 GET 200 209ms (db: 7ms/2q) /login/ (unauth via ?)
|
|
```
|
|
|
|
Now you're ready for [Step 4: Developing.](#step-4-developing)
|
|
|
|
### Step 4: Developing
|
|
|
|
#### Where to edit files
|
|
|
|
You'll work by editing files on your host machine, in the directory where you
|
|
cloned Zulip. Use your favorite editor (Sublime, Atom, Vim, Emacs, Notepad++,
|
|
etc.).
|
|
|
|
When you save changes they will be synced automatically to the Zulip
|
|
development environment on the virtual machine/container.
|
|
|
|
Each component of the Zulip development server will automatically
|
|
restart itself or reload data appropriately when you make changes. So,
|
|
to see your changes, all you usually have to do is reload your
|
|
browser. More details on how this works are available below.
|
|
|
|
Zulip's whitespace rules are all enforced by linters, so be sure to
|
|
run `tools/lint` often to make sure you're following our coding style
|
|
(or use `tools/setup-git-repo` to run it on just the changed files
|
|
automatically whenever you commit).
|
|
|
|
#### Understanding run-dev.py debugging output
|
|
|
|
It's good to have the terminal running `run-dev.py` up as you work since error
|
|
messages including tracebacks along with every backend request will be printed
|
|
there.
|
|
|
|
See [Logging](../subsystems/logging.html) for further details on the run-dev.py console
|
|
output.
|
|
|
|
#### Committing and pushing changes with git
|
|
|
|
When you're ready to commit or push changes via git, you will do this by
|
|
running git commands in Terminal (macOS/Ubuntu) or Git BASH (Windows) in the
|
|
directory where you cloned Zulip on your main machine.
|
|
|
|
If you're new to working with Git/GitHub, check out our [Git & GitHub
|
|
Guide][rtd-git-guide].
|
|
|
|
#### Maintaining the development environment
|
|
|
|
If after rebasing onto a new version of the Zulip server, you receive
|
|
new errors while starting the Zulip server or running tests, this is
|
|
probably not because Zulip's master branch is broken. Instead, this
|
|
is likely because we've recently merged changes to the development
|
|
environment provisioning process that you need to apply to your
|
|
development environment. To update your environment, you'll need to
|
|
re-provision your vagrant machine using `vagrant provision` (this just
|
|
runs `tools/provision` from your Zulip checkout inside the Vagrant
|
|
guest); this should complete in about a minute.
|
|
|
|
After provisioning, you'll want to
|
|
[(re)start the Zulip development server](#step-3-start-the-development-environment).
|
|
|
|
If you run into any trouble, the
|
|
[#provision help](https://chat.zulip.org/#narrow/stream/21-provision-help)
|
|
in the [Zulip development community server](../contributing/chat-zulip-org.html) for
|
|
is a great place to ask for help.
|
|
|
|
#### Rebuilding the development environment
|
|
|
|
If you ever want to recreate your development environment again from
|
|
scratch (e.g. to test as change you've made to the provisioning
|
|
process, or because you think something is broken), you can do so
|
|
using `vagrant destroy` and then `vagrant up`. This will usually be
|
|
much faster than the original `vagrant up` since the base image is
|
|
already cached on your machine (it takes about 5 minutes to run with a
|
|
fast Internet connection).
|
|
|
|
Any additional programs (e.g. Zsh, emacs, etc.) or configuration that
|
|
you may have installed in the development environment will be lost
|
|
when you recreate it. To address this, you can create a script called
|
|
`tools/custom_provision` in your Zulip Git checkout; and place any
|
|
extra setup commands there. Vagrant will run `tools/custom_provision`
|
|
every time you run `vagrant provision` (or create a Vagrant guest via
|
|
`vagrant up`).
|
|
|
|
#### Shutting down the development environment for use later
|
|
|
|
To shut down but preserve the development environment so you can use
|
|
it again later use `vagrant halt` or `vagrant suspend`.
|
|
|
|
You can do this from the same Terminal/Git BASH window that is running
|
|
run-dev.py by pressing ^C to halt the server and then typing `exit`. Or you
|
|
can halt vagrant from another Terminal/Git BASH window.
|
|
|
|
From the window where run-dev.py is running:
|
|
|
|
```
|
|
2016-05-04 18:33:13,330 INFO 127.0.0.1 GET 200 92ms /register/ (unauth via ?)
|
|
^C
|
|
KeyboardInterrupt
|
|
(zulip-py3-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip$ exit
|
|
logout
|
|
Connection to 127.0.0.1 closed.
|
|
christie@win10 ~/zulip
|
|
```
|
|
Now you can suspend the development environment:
|
|
|
|
```
|
|
christie@win10 ~/zulip
|
|
$ vagrant suspend
|
|
==> default: Saving VM state and suspending execution...
|
|
```
|
|
|
|
If `vagrant suspend` doesn't work, try `vagrant halt`:
|
|
|
|
```
|
|
christie@win10 ~/zulip
|
|
$ vagrant halt
|
|
==> default: Attempting graceful shutdown of VM...
|
|
```
|
|
|
|
Check out the Vagrant documentation to learn more about
|
|
[suspend](https://www.vagrantup.com/docs/cli/suspend.html) and
|
|
[halt](https://www.vagrantup.com/docs/cli/halt.html).
|
|
|
|
#### Resuming the development environment
|
|
|
|
When you're ready to work on Zulip again, run `vagrant up`. You will also need
|
|
to connect to the virtual machine with `vagrant ssh` and re-start the Zulip
|
|
server:
|
|
|
|
```
|
|
christie@win10 ~/zulip
|
|
$ vagrant up
|
|
$ vagrant ssh
|
|
|
|
(zulip-py3-venv)vagrant@vagrant-ubuntu-trusty-64:/srv/zulip
|
|
$ ./tools/run-dev.py
|
|
```
|
|
|
|
### Next Steps
|
|
|
|
Next, read the following to learn more about developing for Zulip:
|
|
|
|
* [Git & GitHub Guide][rtd-git-guide]
|
|
* [Using the Development Environment][rtd-using-dev-env]
|
|
* [Testing][rtd-testing] (and [Configuring Travis CI][travis-ci] to
|
|
run the full test suite against any branches you push to your fork,
|
|
which can help you optimize your development workflow).
|
|
|
|
### Troubleshooting and Common Errors
|
|
|
|
Below you'll find a list of common errors and their solutions. Most
|
|
issues are resolved by just provisioning again (by running
|
|
`./tools/provision` (from `/srv/zulip`) inside the Vagrant guest or
|
|
equivalently `vagrant provision` from outside).
|
|
|
|
If these solutions aren't working for you or you encounter an issue not
|
|
documented below, there are a few ways to get further help:
|
|
|
|
* Ask in [#provision help](https://chat.zulip.org/#narrow/stream/21-provision-help)
|
|
in the [Zulip development community server](../contributing/chat-zulip-org.html),
|
|
* send a note to the [Zulip-devel Google
|
|
group](https://groups.google.com/forum/#!forum/zulip-devel), or
|
|
* [File an issue](https://github.com/zulip/zulip/issues).
|
|
|
|
When reporting your issue, please include the following information:
|
|
|
|
* host operating system
|
|
* installation method (Vagrant or direct)
|
|
* whether or not you are using a proxy
|
|
* a copy of Zulip's `vagrant` provisioning logs, available in
|
|
`/var/log/provision.log` on your virtual machine. If you choose to
|
|
post just the error output, please include the **beginning of the
|
|
error output**, not just the last few lines.
|
|
|
|
The output of `tools/diagnose` run inside the Vagrant guest is also
|
|
usually helpful.
|
|
|
|
#### Vagrant guest doesn't show (zulip-py3-venv) at start of prompt
|
|
|
|
This is caused by provisioning failing to complete successfully. You
|
|
can see the errors in `var/log/provision.log`; it should end with
|
|
something like this:
|
|
|
|
```
|
|
ESC[94mZulip development environment setup succeeded!ESC[0m
|
|
```
|
|
|
|
The `ESC` stuff are the terminal color codes that make it show as a nice
|
|
blue in the terminal, which unfortunately looks ugly in the logs.
|
|
|
|
If you encounter an incomplete `/var/log/provision.log file`, you need to
|
|
update your environment. Re-provision your vagrant machine; if the problem
|
|
persists, please come chat with us (see instructions above) for help.
|
|
|
|
After you provision successfully, you'll need to exit your `vagrant ssh`
|
|
shell and run `vagrant ssh` again to get the virtualenv setup properly.
|
|
|
|
#### Vagrant was unable to mount VirtualBox shared folders
|
|
|
|
For the following error:
|
|
```
|
|
Vagrant was unable to mount VirtualBox shared folders. This is usually
|
|
because the filesystem "vboxsf" is not available. This filesystem is
|
|
made available via the VirtualBox Guest Additions and kernel
|
|
module. Please verify that these guest additions are properly
|
|
installed in the guest. This is not a bug in Vagrant and is usually
|
|
caused by a faulty Vagrant box. For context, the command attempted
|
|
was:
|
|
|
|
mount -t vboxsf -o uid=1000,gid=1000 keys /keys
|
|
```
|
|
|
|
If this error starts happening unexpectedly, then just run:
|
|
|
|
```
|
|
vagrant reload
|
|
```
|
|
|
|
This is equivalent of running a halt followed by an up (aka rebooting
|
|
the guest). After this, you can do `vagrant provision` and `vagrant
|
|
ssh`.
|
|
|
|
#### ssl read error
|
|
|
|
If you receive the following error while running `vagrant up`:
|
|
|
|
```
|
|
SSL read: error:00000000:lib(0):func(0):reason(0), errno 104
|
|
```
|
|
|
|
It means that either your network connection is unstable and/or very
|
|
slow. To resolve it, run `vagrant up` until it works (possibly on a
|
|
better network connection).
|
|
|
|
#### Unmet dependencies error
|
|
|
|
When running `vagrant up` or `provision`, if you see the following error:
|
|
|
|
```
|
|
==> default: E:unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).
|
|
```
|
|
|
|
It means that your local apt repository has been corrupted, which can
|
|
usually be resolved by executing the command:
|
|
|
|
```
|
|
apt-get -f install
|
|
```
|
|
|
|
#### ssh connection closed by remote host
|
|
|
|
On running `vagrant ssh`, if you see the following error:
|
|
|
|
```
|
|
ssh_exchange_identification: Connection closed by remote host
|
|
```
|
|
|
|
It usually means the Vagrant guest is not running, which is usually
|
|
solved by rebooting the Vagrant guest via `vagrant reload`. See
|
|
[Vagrant was unable to communicate with the guest machine](#vagrant-was-unable-to-communicate-with-the-guest-machine)
|
|
for more details.
|
|
|
|
#### os.symlink error
|
|
|
|
If you receive the following error while running `vagrant up`:
|
|
|
|
```
|
|
==> default: Traceback (most recent call last):
|
|
==> default: File "./emoji_dump.py", line 75, in <module>
|
|
==> default:
|
|
==> default: os.symlink('unicode/{}.png'.format(code_point), 'out/{}.png'.format(name))
|
|
==> default: OSError
|
|
==> default: :
|
|
==> default: [Errno 71] Protocol error
|
|
```
|
|
|
|
Then Vagrant was not able to create a symbolic link.
|
|
|
|
First, if you are using Windows, **make sure you have run Git BASH (or Cygwin)
|
|
as an administrator**. By default, only administrators can create symbolic
|
|
links on Windows.
|
|
|
|
Second, VirtualBox does not enable symbolic links by default. Vagrant
|
|
starting with version 1.6.0 enables symbolic links for VirtualBox shared
|
|
folder.
|
|
|
|
You can check to see that this is enabled for your virtual machine with
|
|
`vboxmanage` command.
|
|
|
|
Get the name of your virtual machine by running `vboxmanage list vms` and
|
|
then print out the custom settings for this virtual machine with
|
|
`vboxmanage getextradata YOURVMNAME enumerate`:
|
|
|
|
```
|
|
christie@win10 ~/zulip
|
|
$ vboxmanage list vms
|
|
"zulip_default_1462498139595_55484" {5a65199d-8afa-4265-b2f6-6b1f162f157d}
|
|
|
|
christie@win10 ~/zulip
|
|
$ vboxmanage getextradata zulip_default_1462498139595_55484 enumerate
|
|
Key: VBoxInternal2/SharedFoldersEnableSymlinksCreate/srv_zulip, Value: 1
|
|
Key: supported, Value: false
|
|
```
|
|
|
|
If you see "command not found" when you try to run VBoxManage, you need to
|
|
add the VirtualBox directory to your path. On Windows this is mostly likely
|
|
`C:\Program Files\Oracle\VirtualBox\`.
|
|
|
|
If `vboxmanage enumerate` prints nothing, or shows a value of 0 for
|
|
VBoxInternal2/SharedFoldersEnableSymlinksCreate/srv_zulip, then enable
|
|
symbolic links by running this command in Terminal/Git BASH/Cygwin:
|
|
|
|
```
|
|
vboxmanage setextradata YOURVMNAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/srv_zulip 1
|
|
```
|
|
|
|
The virtual machine needs to be shut down when you run this command.
|
|
|
|
#### Connection timeout on `vagrant up`
|
|
|
|
If you see the following error after running `vagrant up`:
|
|
|
|
```
|
|
default: SSH address: 127.0.0.1:2222
|
|
default: SSH username: vagrant
|
|
default: SSH auth method: private key
|
|
default: Error: Connection timeout. Retrying...
|
|
default: Error: Connection timeout. Retrying...
|
|
default: Error: Connection timeout. Retrying...
|
|
|
|
```
|
|
A likely cause is that hardware virtualization is not enabled for your
|
|
computer. This must be done via your computer's BIOS settings. Look for a
|
|
setting called VT-x (Intel) or (AMD-V).
|
|
|
|
If this is already enabled in your BIOS, double-check that you are running a
|
|
64-bit operating system.
|
|
|
|
For further information about troubleshooting vagrant timeout errors [see
|
|
this post](http://stackoverflow.com/questions/22575261/vagrant-stuck-connection-timeout-retrying#22575302).
|
|
|
|
#### Vagrant was unable to communicate with the guest machine
|
|
|
|
If you see the following error when you run `vagrant up`:
|
|
|
|
```
|
|
Timed out while waiting for the machine to boot. This means that
|
|
Vagrant was unable to communicate with the guest machine within
|
|
the configured ("config.vm.boot_timeout" value) time period.
|
|
|
|
If you look above, you should be able to see the error(s) that
|
|
Vagrant had when attempting to connect to the machine. These errors
|
|
are usually good hints as to what may be wrong.
|
|
|
|
If you're using a custom box, make sure that networking is properly
|
|
working and you're able to connect to the machine. It is a common
|
|
problem that networking isn't setup properly in these boxes.
|
|
Verify that authentication configurations are also setup properly,
|
|
as well.
|
|
|
|
If the box appears to be booting properly, you may want to increase
|
|
the timeout ("config.vm.boot_timeout") value.
|
|
```
|
|
|
|
This has a range of possible causes, that usually amount to a bug in
|
|
Virtualbox or Vagrant. If you see this error, you usually can fix it
|
|
by rebooting the guest via `vagrant reload` (or equivalently, `vagrant
|
|
halt` followed by `vagrant up`):
|
|
|
|
#### Vagrant up fails with subprocess.CalledProcessError
|
|
|
|
The `vagrant up` command basically does the following:
|
|
|
|
* Downloads an Ubuntu image and starts it using a Vagrant provider.
|
|
* Uses `vagrant ssh` to connect to that Ubuntu guest, and then runs
|
|
`tools/provision`, which has a lot of subcommands that are
|
|
executed via Python's `subprocess` module. These errors mean that
|
|
one of those subcommands failed.
|
|
|
|
To debug such errors, you can log in to the Vagrant guest machine by
|
|
running `vagrant ssh`, which should present you with a standard shell
|
|
prompt. You can debug interactively by using e.g. `cd zulip &&
|
|
./tools/provision`, and then running the individual subcommands
|
|
that failed. Once you've resolved the problem, you can rerun
|
|
`tools/provision` to proceed; the provisioning system is designed
|
|
to recover well from failures.
|
|
|
|
The zulip provisioning system is generally highly reliable; the most common
|
|
cause of issues here is a poor network connection (or one where you need a
|
|
proxy to access the Internet and haven't [configured the development
|
|
environment to use it](#specifying-a-proxy).
|
|
|
|
Once you've provisioned successfully, you'll get output like this:
|
|
```
|
|
Zulip development environment setup succeeded!
|
|
(zulip-py3-venv) vagrant@vagrant-base-trusty-amd64:~/zulip$
|
|
```
|
|
|
|
If the `(zulip-py3-venv)` part is missing, this is because your
|
|
installation failed the first time before the Zulip virtualenv was
|
|
created. You can fix this by just closing the shell and running
|
|
`vagrant ssh` again, or using `source /srv/zulip-py3-venv/bin/activate`.
|
|
|
|
Finally, if you encounter any issues that weren't caused by your
|
|
Internet connection, please report them! We try hard to keep Zulip
|
|
development environment provisioning free of bugs.
|
|
|
|
##### `pip install` fails during `vagrant up` on Ubuntu
|
|
|
|
Likely causes are:
|
|
|
|
1. Networking issues
|
|
2. Insufficient RAM. Check whether you've allotted at least two
|
|
gigabytes of RAM, which is the minimum Zulip
|
|
[requires](../development/setup-vagrant.html#requirements). If
|
|
not, go to your VM settings and increase the RAM, then restart
|
|
the VM.
|
|
|
|
##### yarn install warnings
|
|
|
|
```
|
|
$ yarn install
|
|
yarn install v0.24.5
|
|
[1/4] Resolving packages...
|
|
[2/4] Fetching packages...
|
|
warning fsevents@1.1.1: The platform "linux" is incompatible with this module.
|
|
info "fsevents@1.1.1" is an optional dependency and failed compatibility check. Excluding it from installation.
|
|
[3/4] Linking dependencies...
|
|
[4/4] Building fresh packages...
|
|
$ browserify node_modules/sockjs-client/lib/entry.js --standalone SockJS > node_modules/sockjs-client/sockjs.js
|
|
Done in 23.50s.
|
|
```
|
|
|
|
These are warnings produced by spammy third party JavaScript packages.
|
|
It is okay to proceed and start the Zulip server.
|
|
|
|
#### vagrant-lxc errors
|
|
|
|
##### Permissions errors
|
|
|
|
When building the development environment using Vagrant and the LXC provider,
|
|
if you encounter permissions errors, you may need to `chown -R 1000:$(id -g)
|
|
/path/to/zulip` on the host before running `vagrant up` in order to ensure that
|
|
the synced directory has the correct owner during provision. This issue will
|
|
arise if you run `id username` on the host where `username` is the user running
|
|
Vagrant and the output is anything but 1000. This seems to be caused by
|
|
Vagrant behavior; for more information, see [the vagrant-lxc FAQ entry about
|
|
shared folder permissions][lxc-sf].
|
|
|
|
|
|
##### NoMethodError
|
|
|
|
If you see the following error when you try to install the vagrant-lxc plugin:
|
|
|
|
```
|
|
/usr/lib/ruby/2.3.0/rubygems/specification.rb:946:in `all=': undefined method `group_by' for nil:NilClass (NoMethodError)
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/bundler.rb:275:in `with_isolated_gem'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/bundler.rb:231:in `internal_install'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/bundler.rb:102:in `install'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/plugin/manager.rb:62:in `block in install_plugin'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/plugin/manager.rb:72:in `install_plugin'
|
|
from /usr/share/vagrant/plugins/commands/plugin/action/install_gem.rb:37:in `call'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/action/warden.rb:34:in `call'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/action/builder.rb:116:in `call'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:66:in `block in run'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/util/busy.rb:19:in `busy'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/action/runner.rb:66:in `run'
|
|
from /usr/share/vagrant/plugins/commands/plugin/command/base.rb:14:in `action'
|
|
from /usr/share/vagrant/plugins/commands/plugin/command/install.rb:32:in `block in execute'
|
|
from /usr/share/vagrant/plugins/commands/plugin/command/install.rb:31:in `each'
|
|
from /usr/share/vagrant/plugins/commands/plugin/command/install.rb:31:in `execute'
|
|
from /usr/share/vagrant/plugins/commands/plugin/command/root.rb:56:in `execute'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/cli.rb:42:in `execute'
|
|
from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:268:in `cli'
|
|
from /usr/bin/vagrant:173:in `<main>'
|
|
```
|
|
|
|
And you have vagrant version 1.8.1, then you need to patch vagrant manually.
|
|
See [this post](https://github.com/mitchellh/vagrant/issues/7073) for an
|
|
explanation of the issue, which should be fixed when Vagrant 1.8.2 is released.
|
|
|
|
In the meantime, read [this
|
|
post](http://stackoverflow.com/questions/36811863/cant-install-vagrant-plugins-in-ubuntu-16-04/36991648#36991648)
|
|
for how to create and apply the patch.
|
|
|
|
It will look something like this:
|
|
|
|
```
|
|
christie@xenial:~
|
|
$ sudo patch --directory /usr/lib/ruby/vendor_ruby/vagrant < vagrant-plugin.patch
|
|
patching file bundler.rb
|
|
```
|
|
#### VT-X unavailability error
|
|
|
|
Users who are unable to do "vagrant up" due to a VT-X unavailability error need to disable "Hyper-V" to get it to work.
|
|
|
|
#### Permissions errors when running the test suite in LXC
|
|
|
|
See ["Possible testing issues"](../testing/testing.html#possible-testing-issues).
|
|
|
|
#### ImportError: No module named '...' on MacOS during Vagrant provisioning
|
|
|
|
If you see following error (or similar) when you try to provision
|
|
Vagrant environment by `vagrant provision` (or during first run
|
|
`vagrant up`):
|
|
|
|
```
|
|
default: ImportError: No module named 'zerver.lib.emoji'
|
|
default: Error running a subcommand of ./lib/provision.py: tools/do-destroy-rebuild-database
|
|
default: Actual error output for the subcommand is just above this.
|
|
default: Traceback (most recent call last):
|
|
default: File "./lib/provision.py", line 413, in <module>
|
|
default: sys.exit(main(options))
|
|
default: File "./lib/provision.py", line 349, in main
|
|
default: run(["tools/do-destroy-rebuild-database"])
|
|
default: File "/srv/zulip/scripts/lib/zulip_tools.py", line 163, in run
|
|
default: subprocess.check_call(args, **kwargs)
|
|
default: File "/usr/lib/python3.4/subprocess.py", line 561, in check_call
|
|
default: raise CalledProcessError(retcode, cmd)
|
|
default: subprocess.CalledProcessError: Command '['tools/do-destroy-rebuild-database']' returned non-zero exit status 1
|
|
default:
|
|
default: Provisioning failed!
|
|
default: * Look at the traceback(s) above to find more about the errors.
|
|
default: * Resolve the errors or get help on chat.
|
|
default: * If you can fix this yourself, you can re-run tools/provision at any time.
|
|
default: * Logs are here: zulip/var/log/provision.log
|
|
default:
|
|
The SSH command responded with a non-zero exit status. Vagrant
|
|
assumes that this means the command failed. The output for this command
|
|
should be in the log above. Please read the output to determine what
|
|
went wrong.
|
|
```
|
|
|
|
This error is caused by a bug in the MacOS NFS file syncing
|
|
implementation (Zulip uses Vagrant's NFS feature for syncing files on
|
|
MacOS). In early versions of MacOS High Sierra, files present in the
|
|
directory on the host machine would appear to not be present in the
|
|
Vagrant guest (e.g. in the exception above, `zerver/lib/emoji.py` is
|
|
missing). This bug is fixed in MacOS High Sierra 10.13.2 and above,
|
|
so the fix is to upgrade to a version of MacOS with a working NFS
|
|
implementation.
|
|
|
|
You can read more about this
|
|
[here](https://github.com/hashicorp/vagrant/issues/8788).
|
|
|
|
### Specifying a proxy
|
|
|
|
If you need to use a proxy server to access the Internet, you will
|
|
need to specify the proxy settings before running `Vagrant up`.
|
|
First, install the Vagrant plugin `vagrant-proxyconf`:
|
|
|
|
```
|
|
vagrant plugin install vagrant-proxyconf
|
|
```
|
|
|
|
Then create `~/.zulip-vagrant-config` and add the following lines to
|
|
it (with the appropriate values in it for your proxy):
|
|
|
|
```
|
|
HTTP_PROXY http://proxy_host:port
|
|
HTTPS_PROXY http://proxy_host:port
|
|
NO_PROXY localhost,127.0.0.1,.example.com
|
|
```
|
|
|
|
Now run `vagrant up` in your terminal to install the development
|
|
server. If you ran `vagrant up` before and failed, you'll need to run
|
|
`vagrant destroy` first to clean up the failed installation.
|
|
|
|
**If you no longer want to use proxy with Vagrant, set values of HTTP_PROXY
|
|
and HTTPS_PROXY to `""` in `~/.zulip-vagrant-config` file and
|
|
restart Vagrant.**
|
|
|
|
You can also change the port on the host machine that Vagrant uses by
|
|
adding to your `~/.zulip-vagrant-config` file. E.g. if you set:
|
|
|
|
```
|
|
HOST_PORT 9971
|
|
```
|
|
|
|
(and halt and restart the Vagrant guest), then you would visit
|
|
http://localhost:9971/ to connect to your development server.
|
|
|
|
If you'd like to be able to connect to your development environment from other
|
|
machines than the VM host, you can manually set the host IP address in the
|
|
'~/.zulip-vagrant-config' file as well. For example, if you set:
|
|
|
|
```
|
|
HOST_IP_ADDR 0.0.0.0
|
|
```
|
|
|
|
(and restart the Vagrant guest), your host IP would be 0.0.0.0, a special value
|
|
for the IP address that means any IP address can connect to your development server.
|
|
|
|
|
|
[cygwin-dl]: http://cygwin.com/
|
|
[vagrant-dl]: https://www.vagrantup.com/downloads.html
|
|
[vagrant-dl-win]: https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.msi
|
|
[vagrant-dl-macos]: https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.dmg
|
|
[vagrant-dl-deb]: https://releases.hashicorp.com/vagrant/2.0.2/vagrant_2.0.2_x86_64.deb
|
|
[vagrant-lxc]: https://github.com/fgrehm/vagrant-lxc
|
|
[vbox-dl]: https://www.virtualbox.org/wiki/Downloads
|
|
[vmware-fusion-dl]: http://www.vmware.com/products/fusion.html
|
|
[vagrant-vmware-fusion-dl]: https://www.vagrantup.com/vmware/
|
|
[avoiding-sudo]: https://github.com/fgrehm/vagrant-lxc#avoiding-sudo-passwords
|
|
[install-advanced]: ../development/setup-advanced.html
|
|
[lxc-sf]: https://github.com/fgrehm/vagrant-lxc/wiki/FAQ#help-my-shared-folders-have-the-wrong-owner
|
|
[rtd-git-guide]: ../git/index.html
|
|
[rtd-testing]: ../testing/testing.html
|
|
[rtd-using-dev-env]: using.html
|
|
[rtd-dev-remote]: remote.html
|
|
[git-bash]: https://git-for-windows.github.io/
|
|
[bash-admin-setup]: https://superuser.com/questions/1002262/run-applications-as-administrator-by-default-in-windows-10
|
|
[set-up-git]: ../git/setup.html
|
|
[travis-ci]: ../git/cloning.html#step-3-configure-travis-ci-continuous-integration
|