diff --git a/docs/dev-env-first-time-contributors.md b/docs/dev-env-first-time-contributors.md index 19a197d387..a0eb5f785b 100644 --- a/docs/dev-env-first-time-contributors.md +++ b/docs/dev-env-first-time-contributors.md @@ -50,10 +50,8 @@ proxy](#specifying-a-proxy) if you need a proxy to access the internet.) virtualization enabled (VT-X or AMD-V), administrator access, [Cygwin][cygwin-dl], [VirtualBox][vbox-dl], [Vagrant][vagrant-dl]. -Don't see your system listed above? Check out: -* [Brief installation instructions for Vagrant development - environment](brief-install-vagrant-dev.html) -* [Installing manually on UNIX-based platforms](install-generic-unix-dev.html) +Don't see your system listed above? See [Advanced setup][install-advanced] for +details about installing for other Linux and UNIX platforms. [cygwin-dl]: http://cygwin.com/ @@ -817,3 +815,5 @@ HOST_PORT 9971 (and halt and restart the Vagrant guest), then you would visit http://localhost:9971/ to connect to your development server. + +[] diff --git a/docs/dev-overview.md b/docs/dev-overview.md index 96f2f0685f..a21a0f3c50 100644 --- a/docs/dev-overview.md +++ b/docs/dev-overview.md @@ -10,7 +10,7 @@ megabytes of dependencies, so you will need an **active, reasonably fast, internet connection throughout the entire installation processes.** You can [configure a proxy][configure-proxy] if you need one. -## Recommended method +## Recommended setup (Vagrant) **For first-time contributors on macOS, Windows, and Ubuntu, we recommend using the [Vagrant development environment][install-vagrant]**. @@ -21,7 +21,7 @@ will run. Vagrant adds a bit of overhead to using the Zulip dev server, but provides an isolated environment that is easy to install, update, and uninstall. It has been well-tested and performs well. -## Advanced methods +## Advanced setup (non-Vagrant) For more experienced contributors, or for first-time contributors who don't want to or can't use Vagrant, Zulip supports a wide range of ways to install @@ -78,9 +78,9 @@ And if you've setup the Zulip dev environment on a remote machine, take a look at our tips for [developing remotely][dev-remote]. [dev-remote]: dev-remote.html -[install-direct]: install-ubuntu-without-vagrant-dev.html -[install-docker]: install-docker-dev.html -[install-generic]: install-generic-unix-dev.html +[install-direct]: dev-setup-non-vagrant.html#installing-directly-on-ubuntu +[install-docker]: dev-setup-non-vagrant.html#using-docker-experimental +[install-generic]: dev-setup-non-vagrant.html#installing-manually-on-linux [install-vagrant]: dev-env-first-time-contributors.html [self-install-remote]: #installing-remotely [self-slow-internet]: #slow-internet-connections diff --git a/docs/dev-remote.md b/docs/dev-remote.md index 7d40b48738..3eee4efbd0 100644 --- a/docs/dev-remote.md +++ b/docs/dev-remote.md @@ -17,9 +17,8 @@ After you have connected to your remote server, you need to install the development environment. Follow the platform specific instructions for your specific remote instance: -* [Ubuntu Installation](install-ubuntu-without-vagrant-dev.html) - this installs the Zulip development environment directly on your remote server. +* [Ubuntu Installation][install-direct] - this installs the Zulip development environment directly on your remote server. * [Detailed tutorial for Vagrant development environment](dev-env-first-time-contributors.html) - this installs the development environment in a self-contained environment that is easy to remove later. -* [Brief installation instructions for Vagrant development environment](brief-install-vagrant-dev.html) - a shorter guide for the previous option. ## Running the Development Server @@ -77,3 +76,5 @@ on your host and [clone](git-guide.html) the zulip repository. At this point you should [read about developing](dev-env-first-time-contributors.html#step-4-developing) and [read about using the development environment](using-dev-environment.html). + +[install-direct]: dev-setup-non-vagrant.html#installing-directly-on-ubuntu diff --git a/docs/dev-setup-non-vagrant.md b/docs/dev-setup-non-vagrant.md new file mode 100644 index 0000000000..9b489e6c36 --- /dev/null +++ b/docs/dev-setup-non-vagrant.md @@ -0,0 +1,457 @@ +# Zulip dev environment setup without Vagrant + +Contents: + +* [Installing directly on Ubuntu](#installing-directly-on-ubuntu) +* [Installing manually on Linux](#installing-manually-on-linux) +* [Using Docker (experimental)](#using-docker-experimental) + +## Installing directly on Ubuntu + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +If you'd like to install a Zulip development environment on a computer +that's already running Ubuntu 14.04 Trusty or Ubuntu 16.04 Xenial, you +can do that by just running: + +``` +# From a clone of zulip.git +./tools/provision.py +source /srv/zulip-venv/bin/activate +./tools/run-dev.py # starts the development server +``` + +Note that there is no supported uninstallation process without Vagrant +(with Vagrant, you can just do `vagrant destroy` to clean up the +development environment). + +Once you've done the above setup, you can pick up the [documentation +on using the Zulip development +environment](dev-env-first-time-contributors.html#step-4-developing), +ignoring the parts about `vagrant` (since you're not using it). + +## Installing manually on Linux + +* [Debian or Ubuntu systems](#on-debian-or-ubuntu-systems) +* [Fedora 22 (experimental)](#on-fedora-22-experimental) +* [CentOS 7 Core (experimental)](#on-centos-7-core-experimental) +* [OpenBSD 5.8 (experimental)](#on-openbsd-5-8-experimental) +* [Fedora/CentOS common steps](#common-to-fedora-centos-instructions) +* [Steps for all systems](#all-systems) + +If you really want to install everything manually, the below instructions +should work. + +Install the following non-Python dependencies: + * libffi-dev — needed for some Python extensions + * postgresql 9.1 or later — our database (client, server, headers) + * nodejs 0.10 (and npm) + * memcached (and headers) + * rabbitmq-server + * libldap2-dev + * python-dev + * redis-server — rate limiting + * tsearch-extras — better text search + * libfreetype6-dev — needed before you pip install Pillow to properly generate emoji PNGs + +### On Debian or Ubuntu systems: + +#### Using the official Ubuntu repositories, PGroonga PPA and `tsearch-extras` deb package: + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \ + memcached rabbitmq-server libldap2-dev redis-server \ + postgresql-server-dev-all libmemcached-dev python-dev \ + hunspell-en-us nodejs nodejs-legacy npm git yui-compressor \ + puppet gettext postgresql + +# Next, install PGroonga from its PPA +sudo add-apt-repository -ys ppa:groonga/ppa +sudo apt-get update +# On 14.04 +sudo apt-get install postgresql-9.3-pgroonga +# On 16.04 +sudo apt-get install postgresql-9.5-pgroonga + +# Next, install Zulip's tsearch-extras postgresql extension +# If on 14.04 or 16.04, you can use the Zulip PPA for tsearch-extras: +cd zulip +sudo apt-add-repository -ys ppa:tabbott/zulip +# On 14.04 +sudo apt-get install postgresql-9.3-tsearch-extras +# On 16.04 +sudo apt-get install postgresql-9.5-tsearch-extras + + +# Otherwise, you can download a .deb directly +# If on 12.04 or wheezy: +wget https://dl.dropboxusercontent.com/u/283158365/zuliposs/postgresql-9.1-tsearch-extras_0.1.2_amd64.deb +sudo dpkg -i postgresql-9.1-tsearch-extras_0.1.2_amd64.deb + +# If on 14.04: +https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/postgresql-9.3-tsearch-extras_0.1.3_amd64.deb +sudo dpkg -i postgresql-9.3-tsearch-extras_0.1.3_amd64.deb + +# If on 15.04 or jessie: +wget https://dl.dropboxusercontent.com/u/283158365/zuliposs/postgresql-9.4-tsearch-extras_0.1_amd64.deb +sudo dpkg -i postgresql-9.4-tsearch-extras_0.1_amd64.deb + +# If on 16.04 or stretch +wget https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/postgresql-9.5-tsearch-extras_0.2_amd64.deb +sudo dpkg -i postgresql-9.5-tsearch-extras_0.2_amd64.deb +``` + +Alternatively, you can always build the package from [tsearch-extras +git](https://github.com/zulip/tsearch_extras). + +Now continue with the [All Systems](#all-systems) instructions below. + +#### Using the [official Zulip PPA](https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+packages) (for 14.04 Trusty or 16.04 Xenial): + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +sudo add-apt-repository ppa:tabbott/zulip +sudo apt-get update +sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \ + memcached rabbitmq-server libldap2-dev redis-server \ + postgresql-server-dev-all libmemcached-dev python-dev \ + hunspell-en-us nodejs nodejs-legacy npm git yui-compressor \ + puppet gettext tsearch-extras +``` + +Now continue with the [All Systems](#all-systems) instructions below. + +### On Fedora 22 (experimental): + +These instructions are experimental and may have bugs; patches +welcome! + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +sudo dnf install libffi-devel memcached rabbitmq-server \ + openldap-devel python-devel redis postgresql-server \ + postgresql-devel postgresql libmemcached-devel freetype-devel \ + nodejs npm yuicompressor closure-compiler gettext +``` + +Now continue with the [Common to Fedora/CentOS](#common-to-fedora-centos-instructions) instructions below. + +### On CentOS 7 Core (experimental): + +These instructions are experimental and may have bugs; patches +welcome! + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +# Add user zulip to the system (not necessary if you configured zulip +# as the administrator user during the install process of CentOS 7). +useradd zulip + +# Create a password for zulip user +passwd zulip + +# Allow zulip to sudo +visudo +# Add this line after line `root ALL=(ALL) ALL` +zulip ALL=(ALL) ALL + +# Switch to zulip user +su zulip + +# Enable EPEL 7 repo so we can install rabbitmq-server, redis and +# other dependencies +sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm + +# Install dependencies +sudo yum install libffi-devel memcached rabbitmq-server openldap-devel \ + python-devel redis postgresql-server postgresql-devel postgresql \ + libmemcached-devel wget python-pip openssl-devel freetype-devel \ + libjpeg-turbo-devel zlib-devel nodejs yuicompressor \ + closure-compiler gettext + +# We need these packages to compile tsearch-extras +sudo yum groupinstall "Development Tools" + +# clone Zulip's git repo and cd into it +cd && git clone https://github.com/zulip/zulip && cd zulip/ + +## NEEDS TESTING: The next few DB setup items may not be required at all. +# Initialize the postgres db +sudo postgresql-setup initdb + +# Edit the postgres settings: +sudo vi /var/lib/pgsql/data/pg_hba.conf + +# Change these lines: +host all all 127.0.0.1/32 ident +host all all ::1/128 ident +# to this: +host all all 127.0.0.1/32 md5 +host all all ::1/128 md5 +``` + +Now continue with the [Common to Fedora/CentOS](#common-to-fedora-centos-instructions) instructions below. + +### On OpenBSD 5.8 (experimental): + +These instructions are experimental and may have bugs; patches +welcome! + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +doas pkg_add sudo bash gcc postgresql-server redis rabbitmq \ + memcached node libmemcached py-Pillow py-cryptography py-cffi + +# Get tsearch_extras and build it (using a modified version which +# aliases int4 on OpenBSD): +git clone https://github.com/blablacio/tsearch_extras +cd tsearch_extras +gmake && sudo gmake install + +# Point environment to custom include locations and use newer GCC +# (needed for Node modules): +export CFLAGS="-I/usr/local/include -I/usr/local/include/sasl" +export CXX=eg++ + +# Create tsearch_data directory: +sudo mkdir /usr/local/share/postgresql/tsearch_data + + +# Hack around missing dictionary files -- need to fix this to get the +# proper dictionaries from what in debian is the hunspell-en-us +# package. +sudo touch /usr/local/share/postgresql/tsearch_data/english.stop +sudo touch /usr/local/share/postgresql/tsearch_data/en_us.dict +sudo touch /usr/local/share/postgresql/tsearch_data/en_us.affix +``` + +Finally continue with the [All Systems](#all-systems) instructions below. + +### Common to Fedora/CentOS instructions + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +``` +# Build and install postgres tsearch-extras module +wget https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/tsearch-extras_0.1.3.tar.gz +tar xvzf tsearch-extras_0.1.3.tar.gz +cd ts2 +make +sudo make install + +# Hack around missing dictionary files -- need to fix this to get the +# proper dictionaries from what in debian is the hunspell-en-us +# package. +sudo touch /usr/share/pgsql/tsearch_data/english.stop +sudo touch /usr/share/pgsql/tsearch_data/en_us.dict +sudo touch /usr/share/pgsql/tsearch_data/en_us.affix + +# Edit the postgres settings: +sudo vi /var/lib/pgsql/data/pg_hba.conf + +# Add this line before the first uncommented line to enable password +# auth: +host all all 127.0.0.1/32 md5 + +# Start the services +sudo systemctl start redis memcached rabbitmq-server postgresql + +# Enable automatic service startup after the system startup +sudo systemctl enable redis rabbitmq-server memcached postgresql +``` + +Finally continue with the [All Systems](#all-systems) instructions below. + +### All Systems: + +Make sure you have followed the steps specific for your platform: + +* [Debian or Ubuntu systems](#on-debian-or-ubuntu-systems) +* [Fedora 22 (experimental)](#on-fedora-22-experimental) +* [CentOS 7 Core (experimental)](#on-centos-7-core-experimental) +* [OpenBSD 5.8 (experimental)](#on-openbsd-5-8-experimental) +* [Fedora/CentOS](#common-to-fedora-centos-instructions) + +For managing Zulip's python dependencies, we recommend using +[virtualenvs](https://virtualenv.pypa.io/en/stable/). + +You must create two virtualenvs. One for Python 2 and one for Python 3. +You must also install appropriate python packages in them. + +You should either install the virtualenvs in `/srv`, or put symlinks to +them in `/srv`. If you don't do that, some scripts might not work correctly. + +You can run `tools/setup/setup_venvs.py` to do this. This script will create two +virtualenvs - /srv/zulip-venv and /srv/zulip-py3-venv. + +If you want to do it manually, here are the steps: + +``` +sudo virtualenv /srv/zulip-venv -p python2 # Create a python2 virtualenv +sudo chown -R `whoami`:`whoami` /srv/zulip-venv +source /srv/zulip-venv/bin/activate # Activate python2 virtualenv +pip install --upgrade pip # upgrade pip itself because older versions have known issues +pip install --no-deps -r requirements/py2_dev.txt # install python packages required for development + +sudo virtualenv /srv/zulip-py3-venv -p python3 # Create a python3 virtualenv +sudo chown -R `whoami`:`whoami` /srv/zulip-py3-venv +source /srv/zulip-py3-venv/bin/activate # Activate python3 virtualenv +pip install --upgrade pip # upgrade pip itself because older versions have known issues +pip install --no-deps -r requirements/py3_dev.txt # install python packages required for development +``` + +Now run these commands: + +``` +./tools/install-mypy +./tools/setup/download-zxcvbn +./tools/setup/emoji_dump/build_emoji +./scripts/setup/generate_secrets.py --development +if [ $(uname) = "OpenBSD" ]; then sudo cp ./puppet/zulip/files/postgresql/zulip_english.stop /var/postgresql/tsearch_data/; else sudo cp ./puppet/zulip/files/postgresql/zulip_english.stop /usr/share/postgresql/9.*/tsearch_data/; fi +./scripts/setup/configure-rabbitmq +./tools/setup/postgres-init-dev-db +./tools/do-destroy-rebuild-database +./tools/setup/postgres-init-test-db +./tools/do-destroy-rebuild-test-database +./manage.py compilemessages +sudo ./tools/setup/install-node +npm install +``` + +To start the development server: + +``` +./tools/run-dev.py +``` + +… and visit [http://localhost:9991/](http://localhost:9991/). + +#### Proxy setup for by-hand installation + +If you are building the development environment on a network where a +proxy is required to access the Internet, you will need to set the +proxy in the environment as follows: + +- On Ubuntu, set the proxy environment variables using: + ``` + export https_proxy=http://proxy_host:port + export http_proxy=http://proxy_host:port + ``` + +- And set the npm proxy and https-proxy using: + ``` + npm config set proxy http://proxy_host:port + npm config set https-proxy http://proxy_host:port + ``` + +## Using Docker (experimental) + +Start by cloning this repository: `git clone +https://github.com/zulip/zulip.git` + +The docker instructions for development are experimental, so they may +have bugs. If you try them and run into any issues, please report +them! + +You can also use Docker to run a Zulip development environment. +First, you need to install Docker in your development machine +following the [instructions][docker-install]. Some other interesting +links for somebody new in Docker are: + +* [Get Started](https://docs.docker.com/engine/installation/linux/) +* [Understand the architecture](https://docs.docker.com/engine/understanding-docker/) +* [Docker run reference](https://docs.docker.com/engine/reference/run/) +* [Dockerfile reference](https://docs.docker.com/engine/reference/builder/) + +[docker-install]: https://docs.docker.com/engine/installation/ + +Then you should create the Docker image based on Ubuntu Linux, first +go to the directory with the Zulip source code: + +``` +docker build -t user/zulipdev . +``` + + +Commit and tag the provisioned images. The below will install Zulip's dependencies: +``` +docker run -itv $(pwd):/srv/zulip -p 9991:9991 user/zulipdev /bin/bash +$ /usr/bin/python /srv/zulip/tools/provision.py --docker +docker ps -af ancestor=user/zulipdev +docker commit -m "Zulip installed" user/zulipdev:v2 +``` + +Now you can run the docker server with: + +``` +docker run -itv $(pwd):/srv/zulip -p 9991:9991 user/zulipdev:v2 \ + /srv/zulip/tools/start-dockers +``` + +You'll want to +[read the guide for Zulip development](dev-env-first-time-contributors.html#step-4-developing) +to understand how to use the Zulip development. Note that +`start-dockers` automatically runs `tools/run-dev.py` inside the +container; you can then visit http://localhost:9991 to connect to your +new Zulip Docker container. + + +To view the container's `run-dev.py` console logs to get important +debugging information (and e.g. outgoing emails) printed by the Zulip +development environment, you can use: +``` +docker logs --follow +``` + +To restart the server use: +``` +docker ps +docker restart +``` + +To stop the server use: +``` +docker ps +docker kill +``` + +If you want to connect to the Docker instance to run commands +(e.g. build a release tarball), you can use: + +``` +docker ps +docker exec -it /bin/bash +$ source /home/zulip/.bash_profile +$ +$ exit +``` + +If you want to run all the tests you need to start the servers first, +you can do it with: + +``` +docker run -itv $(pwd):/srv/zulip user/zulipdev:v2 /bin/bash +$ tools/test-all-docker +``` + +You can modify the source code in your development machine and review +the results in your browser. + + +Currently, the Docker workflow is substantially less convenient than +the Vagrant workflow and less documented; please contribute to this +guide and the Docker tooling if you are using Docker to develop Zulip! + diff --git a/docs/index.rst b/docs/index.rst index 629b95cd8f..b33d87ab89 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -62,14 +62,11 @@ Contents: :maxdepth: 2 :caption: Installation for developers - dev-overview - dev-env-first-time-contributors - brief-install-vagrant-dev - install-ubuntu-without-vagrant-dev - install-generic-unix-dev - install-docker-dev - dev-remote - using-dev-environment + Overview + Recommended setup (Vagrant) + Advanced setup (non-Vagrant) + Using the dev environment + Developing remotely .. _tutorial-docs: diff --git a/docs/install-docker-dev.md b/docs/install-docker-dev.md index 728a77cf9c..98d357a4d0 100644 --- a/docs/install-docker-dev.md +++ b/docs/install-docker-dev.md @@ -1,97 +1,3 @@ -Using Docker (experimental) ---------------------------- -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` +# Using Docker (experimental) -The docker instructions for development are experimental, so they may -have bugs. If you try them and run into any issues, please report -them! - -You can also use Docker to run a Zulip development environment. -First, you need to install Docker in your development machine -following the [instructions][docker-install]. Some other interesting -links for somebody new in Docker are: - -* [Get Started](https://docs.docker.com/engine/installation/linux/) -* [Understand the architecture](https://docs.docker.com/engine/understanding-docker/) -* [Docker run reference](https://docs.docker.com/engine/reference/run/) -* [Dockerfile reference](https://docs.docker.com/engine/reference/builder/) - -[docker-install]: https://docs.docker.com/engine/installation/ - -Then you should create the Docker image based on Ubuntu Linux, first -go to the directory with the Zulip source code: - -``` -docker build -t user/zulipdev . -``` - - -Commit and tag the provisioned images. The below will install Zulip's dependencies: -``` -docker run -itv $(pwd):/srv/zulip -p 9991:9991 user/zulipdev /bin/bash -$ /usr/bin/python /srv/zulip/tools/provision.py --docker -docker ps -af ancestor=user/zulipdev -docker commit -m "Zulip installed" user/zulipdev:v2 -``` - -Now you can run the docker server with: - -``` -docker run -itv $(pwd):/srv/zulip -p 9991:9991 user/zulipdev:v2 \ - /srv/zulip/tools/start-dockers -``` - -You'll want to -[read the guide for Zulip development](dev-env-first-time-contributors.html#step-4-developing) -to understand how to use the Zulip development. Note that -`start-dockers` automatically runs `tools/run-dev.py` inside the -container; you can then visit http://localhost:9991 to connect to your -new Zulip Docker container. - - -To view the container's `run-dev.py` console logs to get important -debugging information (and e.g. outgoing emails) printed by the Zulip -development environment, you can use: -``` -docker logs --follow -``` - -To restart the server use: -``` -docker ps -docker restart -``` - -To stop the server use: -``` -docker ps -docker kill -``` - -If you want to connect to the Docker instance to run commands -(e.g. build a release tarball), you can use: - -``` -docker ps -docker exec -it /bin/bash -$ source /home/zulip/.bash_profile -$ -$ exit -``` - -If you want to run all the tests you need to start the servers first, -you can do it with: - -``` -docker run -itv $(pwd):/srv/zulip user/zulipdev:v2 /bin/bash -$ tools/test-all-docker -``` - -You can modify the source code in your development machine and review -the results in your browser. - - -Currently, the Docker workflow is substantially less convenient than -the Vagrant workflow and less documented; please contribute to this -guide and the Docker tooling if you are using Docker to develop Zulip! +Moved to [Advanced setup (Docker)](dev-setup-non-vagrant.html#using-docker-experimental). diff --git a/docs/install-generic-unix-dev.md b/docs/install-generic-unix-dev.md index eb6599a8b9..6104174a97 100644 --- a/docs/install-generic-unix-dev.md +++ b/docs/install-generic-unix-dev.md @@ -1,325 +1,3 @@ # Installing manually on UNIX -* [Debian or Ubuntu systems](#on-debian-or-ubuntu-systems) -* [Fedora 22 (experimental)](#on-fedora-22-experimental) -* [CentOS 7 Core (experimental)](#on-centos-7-core-experimental) -* [OpenBSD 5.8 (experimental)](#on-openbsd-5-8-experimental) -* [Fedora/CentOS common steps](#common-to-fedora-centos-instructions) -* [Steps for all systems](#all-systems) - -If you really want to install everything manually, the below instructions -should work. - -Install the following non-Python dependencies: - * libffi-dev — needed for some Python extensions - * postgresql 9.1 or later — our database (client, server, headers) - * nodejs 0.10 (and npm) - * memcached (and headers) - * rabbitmq-server - * libldap2-dev - * python-dev - * redis-server — rate limiting - * tsearch-extras — better text search - * libfreetype6-dev — needed before you pip install Pillow to properly generate emoji PNGs - -### On Debian or Ubuntu systems: - -#### Using the official Ubuntu repositories, PGroonga PPA and `tsearch-extras` deb package: - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \ - memcached rabbitmq-server libldap2-dev redis-server \ - postgresql-server-dev-all libmemcached-dev python-dev \ - hunspell-en-us nodejs nodejs-legacy npm git yui-compressor \ - puppet gettext postgresql - -# Next, install PGroonga from its PPA -sudo add-apt-repository -ys ppa:groonga/ppa -sudo apt-get update -# On 14.04 -sudo apt-get install postgresql-9.3-pgroonga -# On 16.04 -sudo apt-get install postgresql-9.5-pgroonga - -# Next, install Zulip's tsearch-extras postgresql extension -# If on 14.04 or 16.04, you can use the Zulip PPA for tsearch-extras: -cd zulip -sudo apt-add-repository -ys ppa:tabbott/zulip -# On 14.04 -sudo apt-get install postgresql-9.3-tsearch-extras -# On 16.04 -sudo apt-get install postgresql-9.5-tsearch-extras - - -# Otherwise, you can download a .deb directly -# If on 12.04 or wheezy: -wget https://dl.dropboxusercontent.com/u/283158365/zuliposs/postgresql-9.1-tsearch-extras_0.1.2_amd64.deb -sudo dpkg -i postgresql-9.1-tsearch-extras_0.1.2_amd64.deb - -# If on 14.04: -https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/postgresql-9.3-tsearch-extras_0.1.3_amd64.deb -sudo dpkg -i postgresql-9.3-tsearch-extras_0.1.3_amd64.deb - -# If on 15.04 or jessie: -wget https://dl.dropboxusercontent.com/u/283158365/zuliposs/postgresql-9.4-tsearch-extras_0.1_amd64.deb -sudo dpkg -i postgresql-9.4-tsearch-extras_0.1_amd64.deb - -# If on 16.04 or stretch -wget https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/postgresql-9.5-tsearch-extras_0.2_amd64.deb -sudo dpkg -i postgresql-9.5-tsearch-extras_0.2_amd64.deb -``` - -Alternatively, you can always build the package from [tsearch-extras -git](https://github.com/zulip/tsearch_extras). - -Now continue with the [All Systems](#all-systems) instructions below. - -#### Using the [official Zulip PPA](https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+packages) (for 14.04 Trusty or 16.04 Xenial): - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -sudo add-apt-repository ppa:tabbott/zulip -sudo apt-get update -sudo apt-get install closure-compiler libfreetype6-dev libffi-dev \ - memcached rabbitmq-server libldap2-dev redis-server \ - postgresql-server-dev-all libmemcached-dev python-dev \ - hunspell-en-us nodejs nodejs-legacy npm git yui-compressor \ - puppet gettext tsearch-extras -``` - -Now continue with the [All Systems](#all-systems) instructions below. - -### On Fedora 22 (experimental): - -These instructions are experimental and may have bugs; patches -welcome! - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -sudo dnf install libffi-devel memcached rabbitmq-server \ - openldap-devel python-devel redis postgresql-server \ - postgresql-devel postgresql libmemcached-devel freetype-devel \ - nodejs npm yuicompressor closure-compiler gettext -``` - -Now continue with the [Common to Fedora/CentOS](#common-to-fedora-centos-instructions) instructions below. - -### On CentOS 7 Core (experimental): - -These instructions are experimental and may have bugs; patches -welcome! - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -# Add user zulip to the system (not necessary if you configured zulip -# as the administrator user during the install process of CentOS 7). -useradd zulip - -# Create a password for zulip user -passwd zulip - -# Allow zulip to sudo -visudo -# Add this line after line `root ALL=(ALL) ALL` -zulip ALL=(ALL) ALL - -# Switch to zulip user -su zulip - -# Enable EPEL 7 repo so we can install rabbitmq-server, redis and -# other dependencies -sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm - -# Install dependencies -sudo yum install libffi-devel memcached rabbitmq-server openldap-devel \ - python-devel redis postgresql-server postgresql-devel postgresql \ - libmemcached-devel wget python-pip openssl-devel freetype-devel \ - libjpeg-turbo-devel zlib-devel nodejs yuicompressor \ - closure-compiler gettext - -# We need these packages to compile tsearch-extras -sudo yum groupinstall "Development Tools" - -# clone Zulip's git repo and cd into it -cd && git clone https://github.com/zulip/zulip && cd zulip/ - -## NEEDS TESTING: The next few DB setup items may not be required at all. -# Initialize the postgres db -sudo postgresql-setup initdb - -# Edit the postgres settings: -sudo vi /var/lib/pgsql/data/pg_hba.conf - -# Change these lines: -host all all 127.0.0.1/32 ident -host all all ::1/128 ident -# to this: -host all all 127.0.0.1/32 md5 -host all all ::1/128 md5 -``` - -Now continue with the [Common to Fedora/CentOS](#common-to-fedora-centos-instructions) instructions below. - -### On OpenBSD 5.8 (experimental): - -These instructions are experimental and may have bugs; patches -welcome! - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -doas pkg_add sudo bash gcc postgresql-server redis rabbitmq \ - memcached node libmemcached py-Pillow py-cryptography py-cffi - -# Get tsearch_extras and build it (using a modified version which -# aliases int4 on OpenBSD): -git clone https://github.com/blablacio/tsearch_extras -cd tsearch_extras -gmake && sudo gmake install - -# Point environment to custom include locations and use newer GCC -# (needed for Node modules): -export CFLAGS="-I/usr/local/include -I/usr/local/include/sasl" -export CXX=eg++ - -# Create tsearch_data directory: -sudo mkdir /usr/local/share/postgresql/tsearch_data - - -# Hack around missing dictionary files -- need to fix this to get the -# proper dictionaries from what in debian is the hunspell-en-us -# package. -sudo touch /usr/local/share/postgresql/tsearch_data/english.stop -sudo touch /usr/local/share/postgresql/tsearch_data/en_us.dict -sudo touch /usr/local/share/postgresql/tsearch_data/en_us.affix -``` - -Finally continue with the [All Systems](#all-systems) instructions below. - -### Common to Fedora/CentOS instructions - -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -``` -# Build and install postgres tsearch-extras module -wget https://launchpad.net/~tabbott/+archive/ubuntu/zulip/+files/tsearch-extras_0.1.3.tar.gz -tar xvzf tsearch-extras_0.1.3.tar.gz -cd ts2 -make -sudo make install - -# Hack around missing dictionary files -- need to fix this to get the -# proper dictionaries from what in debian is the hunspell-en-us -# package. -sudo touch /usr/share/pgsql/tsearch_data/english.stop -sudo touch /usr/share/pgsql/tsearch_data/en_us.dict -sudo touch /usr/share/pgsql/tsearch_data/en_us.affix - -# Edit the postgres settings: -sudo vi /var/lib/pgsql/data/pg_hba.conf - -# Add this line before the first uncommented line to enable password -# auth: -host all all 127.0.0.1/32 md5 - -# Start the services -sudo systemctl start redis memcached rabbitmq-server postgresql - -# Enable automatic service startup after the system startup -sudo systemctl enable redis rabbitmq-server memcached postgresql -``` - -Finally continue with the [All Systems](#all-systems) instructions below. - -### All Systems: - -Make sure you have followed the steps specific for your platform: - -* [Debian or Ubuntu systems](#on-debian-or-ubuntu-systems) -* [Fedora 22 (experimental)](#on-fedora-22-experimental) -* [CentOS 7 Core (experimental)](#on-centos-7-core-experimental) -* [OpenBSD 5.8 (experimental)](#on-openbsd-5-8-experimental) -* [Fedora/CentOS](#common-to-fedora-centos-instructions) - -For managing Zulip's python dependencies, we recommend using -[virtualenvs](https://virtualenv.pypa.io/en/stable/). - -You must create two virtualenvs. One for Python 2 and one for Python 3. -You must also install appropriate python packages in them. - -You should either install the virtualenvs in `/srv`, or put symlinks to -them in `/srv`. If you don't do that, some scripts might not work correctly. - -You can run `tools/setup/setup_venvs.py` to do this. This script will create two -virtualenvs - /srv/zulip-venv and /srv/zulip-py3-venv. - -If you want to do it manually, here are the steps: - -``` -sudo virtualenv /srv/zulip-venv -p python2 # Create a python2 virtualenv -sudo chown -R `whoami`:`whoami` /srv/zulip-venv -source /srv/zulip-venv/bin/activate # Activate python2 virtualenv -pip install --upgrade pip # upgrade pip itself because older versions have known issues -pip install --no-deps -r requirements/py2_dev.txt # install python packages required for development - -sudo virtualenv /srv/zulip-py3-venv -p python3 # Create a python3 virtualenv -sudo chown -R `whoami`:`whoami` /srv/zulip-py3-venv -source /srv/zulip-py3-venv/bin/activate # Activate python3 virtualenv -pip install --upgrade pip # upgrade pip itself because older versions have known issues -pip install --no-deps -r requirements/py3_dev.txt # install python packages required for development -``` - -Now run these commands: - -``` -./tools/install-mypy -./tools/setup/download-zxcvbn -./tools/setup/emoji_dump/build_emoji -./scripts/setup/generate_secrets.py --development -if [ $(uname) = "OpenBSD" ]; then sudo cp ./puppet/zulip/files/postgresql/zulip_english.stop /var/postgresql/tsearch_data/; else sudo cp ./puppet/zulip/files/postgresql/zulip_english.stop /usr/share/postgresql/9.*/tsearch_data/; fi -./scripts/setup/configure-rabbitmq -./tools/setup/postgres-init-dev-db -./tools/do-destroy-rebuild-database -./tools/setup/postgres-init-test-db -./tools/do-destroy-rebuild-test-database -./manage.py compilemessages -sudo ./tools/setup/install-node -npm install -``` - -To start the development server: - -``` -./tools/run-dev.py -``` - -… and visit [http://localhost:9991/](http://localhost:9991/). - -#### Proxy setup for by-hand installation - -If you are building the development environment on a network where a -proxy is required to access the Internet, you will need to set the -proxy in the environment as follows: - -- On Ubuntu, set the proxy environment variables using: - ``` - export https_proxy=http://proxy_host:port - export http_proxy=http://proxy_host:port - ``` - -- And set the npm proxy and https-proxy using: - ``` - npm config set proxy http://proxy_host:port - npm config set https-proxy http://proxy_host:port - ``` +Moved to [Advanced setup (Installing manually on Linux)](dev-setup-non-vagrant.html#installing-manually-on-linux). diff --git a/docs/install-ubuntu-without-vagrant-dev.md b/docs/install-ubuntu-without-vagrant-dev.md index f5ade841ad..6c4dc4f15f 100644 --- a/docs/install-ubuntu-without-vagrant-dev.md +++ b/docs/install-ubuntu-without-vagrant-dev.md @@ -1,24 +1,3 @@ # Installing directly on Ubuntu -Start by cloning this repository: `git clone -https://github.com/zulip/zulip.git` - -If you'd like to install a Zulip development environment on a computer -that's already running Ubuntu 14.04 Trusty or Ubuntu 16.04 Xenial, you -can do that by just running: - -``` -# From a clone of zulip.git -./tools/provision.py -source /srv/zulip-venv/bin/activate -./tools/run-dev.py # starts the development server -``` - -Note that there is no supported uninstallation process without Vagrant -(with Vagrant, you can just do `vagrant destroy` to clean up the -development environment). - -Once you've done the above setup, you can pick up the [documentation -on using the Zulip development -environment](dev-env-first-time-contributors.html#step-4-developing), -ignoring the parts about `vagrant` (since you're not using it). +Moved to [Advanced setup (Install directly on Ubuntu)](dev-setup-non-vagrant.html#installing-directly-on-ubuntu).