2019-10-14 21:40:48 +02:00
|
|
|
# Upgrade or modify Zulip
|
|
|
|
|
|
|
|
This page explains how to upgrade, patch, or modify Zulip, including:
|
|
|
|
|
|
|
|
- [Upgrading to a release](#upgrading-to-a-release)
|
2020-10-23 02:43:28 +02:00
|
|
|
- [Upgrading from a Git repository](#upgrading-from-a-git-repository)
|
2021-04-22 00:04:09 +02:00
|
|
|
- [Updating `settings.py` inline documentation](#updating-settings-py-inline-documentation)
|
2019-10-14 21:40:48 +02:00
|
|
|
- [Troubleshooting and rollback](#troubleshooting-and-rollback)
|
2021-04-22 00:04:09 +02:00
|
|
|
- [Preserving local changes to service configuration files](#preserving-local-changes-to-service-configuration-files)
|
2019-10-14 21:40:48 +02:00
|
|
|
- [Upgrading the operating system](#upgrading-the-operating-system)
|
2020-06-26 23:40:36 +02:00
|
|
|
- [Upgrading PostgreSQL](#upgrading-postgresql)
|
2019-10-14 21:40:48 +02:00
|
|
|
- [Modifying Zulip](#modifying-zulip)
|
|
|
|
- [Applying changes from master](#applying-changes-from-master)
|
|
|
|
|
|
|
|
## Upgrading to a release
|
|
|
|
|
|
|
|
Note that there are additional instructions if you're [using
|
|
|
|
docker-zulip][docker-upgrade], have [patched Zulip](#modifying-zulip),
|
|
|
|
or have [modified Zulip-managed configuration
|
2021-04-22 00:04:09 +02:00
|
|
|
files](#preserving-local-changes-to-service-configuration-files). To upgrade
|
2019-10-14 21:40:48 +02:00
|
|
|
to a new Zulip release:
|
|
|
|
|
2020-06-23 00:55:55 +02:00
|
|
|
1. Read the [upgrade notes](../overview/changelog.html#upgrade-notes)
|
|
|
|
for all releases newer than what is currently installed.
|
|
|
|
|
2019-10-14 21:40:48 +02:00
|
|
|
1. Download the appropriate release tarball from
|
|
|
|
<https://www.zulip.org/dist/releases/> You can download the latest
|
|
|
|
release with:
|
|
|
|
|
|
|
|
```
|
2021-06-25 01:28:27 +02:00
|
|
|
curl -LO https://www.zulip.org/dist/releases/zulip-server-latest.tar.gz
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
You also have the option of upgrading Zulip [to a version in a Git
|
|
|
|
repository directly](#upgrading-from-a-git-repository) or creating
|
|
|
|
your own release tarballs from a copy of the [zulip.git
|
|
|
|
repository](https://github.com/zulip/zulip) using
|
|
|
|
`tools/build-release-tarball`.
|
|
|
|
|
2020-08-11 02:20:10 +02:00
|
|
|
1. Log in to your Zulip and run as root:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
|
|
|
/home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-VERSION.tar.gz
|
|
|
|
```
|
|
|
|
|
|
|
|
The upgrade process will:
|
|
|
|
* Run `apt-get upgrade`
|
2019-12-13 01:58:10 +01:00
|
|
|
* Install new versions of Zulip's dependencies (mainly Python packages).
|
|
|
|
* (`upgrade-zulip-from-git` only) Build Zulip's frontend assets using `webpack`.
|
|
|
|
* Shut down the Zulip service
|
2019-10-14 21:40:48 +02:00
|
|
|
* Run a `puppet apply`
|
|
|
|
* Run any database migrations
|
|
|
|
* Bring the Zulip service back up on the new version.
|
|
|
|
|
|
|
|
Upgrading will result in brief downtime for the service, which should
|
|
|
|
be under 30 seconds unless there is an expensive database migration
|
|
|
|
involved (these will be documented in the [release
|
|
|
|
notes](../overview/changelog.md), and usually can be avoided with
|
|
|
|
some care). If downtime is problematic for your organization,
|
|
|
|
consider testing the upgrade on a
|
|
|
|
[backup](../production/export-and-import.html#backups) in advance,
|
|
|
|
doing the final upgrade at off hours, or buying a support contract.
|
|
|
|
|
|
|
|
See the [troubleshooting guide](#troubleshooting-and-rollback) if you
|
|
|
|
run into any issues or need to roll back the upgrade.
|
|
|
|
|
2020-08-11 01:47:54 +02:00
|
|
|
## Upgrading from a Git repository
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
Zulip supports upgrading a production installation to any commit in a
|
|
|
|
Git repository, which is great for [running pre-release changes from
|
|
|
|
master](#applying-changes-from-master) or [maintaining a
|
|
|
|
fork](#making-changes). The process is simple:
|
|
|
|
|
|
|
|
```
|
|
|
|
# Upgrade to an official release
|
|
|
|
/home/zulip/deployments/current/scripts/upgrade-zulip-from-git 1.8.1
|
2020-02-29 03:32:23 +01:00
|
|
|
# Upgrade to a branch (or other Git ref)
|
|
|
|
/home/zulip/deployments/current/scripts/upgrade-zulip-from-git 2.1.x
|
2019-10-14 21:40:48 +02:00
|
|
|
/home/zulip/deployments/current/scripts/upgrade-zulip-from-git master
|
|
|
|
```
|
|
|
|
|
|
|
|
Zulip will automatically fetch the relevant Git commit and upgrade to
|
2020-02-29 03:32:23 +01:00
|
|
|
that version of Zulip.
|
|
|
|
|
|
|
|
Branches with names like `2.1.x` are stable release branches,
|
|
|
|
containing the changes planned for the next minor release
|
|
|
|
(E.g. 2.1.5); we support these stable release branches as though they
|
|
|
|
were a published release.
|
|
|
|
|
|
|
|
The `master` branch contains changes planned for the next major
|
2020-06-25 01:25:14 +02:00
|
|
|
release (E.g. 3.0); see our documentation on [running
|
2020-02-29 03:32:23 +01:00
|
|
|
master](#upgrading-to-master) before upgrading to it.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
By default, this uses the main upstream Zulip server repository, but
|
|
|
|
you can configure any other Git repository by adding a section like
|
|
|
|
this to `/etc/zulip/zulip.conf`:
|
|
|
|
|
|
|
|
```
|
|
|
|
[deployment]
|
|
|
|
git_repo_url = https://github.com/zulip/zulip.git
|
|
|
|
```
|
|
|
|
|
|
|
|
See also our documentation on [upgrading
|
|
|
|
docker-zulip](https://github.com/zulip/docker-zulip#upgrading-from-a-git-repository).
|
|
|
|
|
2021-04-22 00:04:09 +02:00
|
|
|
## Updating `settings.py` inline documentation
|
|
|
|
|
|
|
|
Zulip installations often upgrade many times over their lifetime, and
|
|
|
|
we strive to keep all configuration files backwards-compatible.
|
|
|
|
However, our practice of leaving the `/etc/zulip/settings.py`
|
|
|
|
unchanged during upgrades means that there may be new features which
|
|
|
|
are not documented in that file, since it was based on a template
|
|
|
|
provided by an earlier version of Zulip, during the initial install.
|
|
|
|
|
|
|
|
After upgrading across major versions of Zulip Server, we recommend
|
|
|
|
comparing your `/etc/zulip/settings.py` file to the current settings
|
|
|
|
template, which can be found in
|
|
|
|
`/home/zulip/deployments/current/zproject/prod_settings_template.py`. We
|
|
|
|
suggest using that updated template to update
|
|
|
|
`/etc/zulip/settings.py`:
|
|
|
|
|
|
|
|
1. Copy the current `settings.py` to make a backup (especially if you
|
|
|
|
do not have a recent [complete backup][backups]), and make a copy
|
|
|
|
of the current template:
|
|
|
|
|
|
|
|
```
|
|
|
|
cp -a /etc/zulip/settings.py ~/zulip-settings-backup.py
|
|
|
|
cp -a /home/zulip/deployments/current/zproject/prod_settings_template.py /etc/zulip/settings-new.py
|
|
|
|
```
|
|
|
|
|
|
|
|
1. Open both `/etc/zulip/settings.py` and `/etc/zulip/settings-new.py`
|
|
|
|
files in an editor; for each setting set in `settings.py`, find its
|
|
|
|
section in `/etc/zulip/settings-new.py` and copy the setting from
|
|
|
|
`settings.py` into there.
|
|
|
|
|
2021-04-22 00:21:46 +02:00
|
|
|
The following tool may help, by finding the most likely version of
|
|
|
|
the template that your `/etc/zulip/settings.py` was installed
|
|
|
|
using, and the differences that your file has from that:
|
|
|
|
|
|
|
|
```
|
|
|
|
/home/zulip/deployments/current/scripts/setup/compare-settings-to-template
|
|
|
|
```
|
|
|
|
|
2021-04-22 00:04:09 +02:00
|
|
|
If there are settings which you cannot find documented in
|
|
|
|
`/etc/zulip/settings-new.py`, check the [changelog][changelog] to see
|
|
|
|
if they have been removed.
|
|
|
|
|
|
|
|
1. Overwriting the configuration with the updated file, and restart
|
|
|
|
the server to pick up the new file; this should be a no-op, but it
|
|
|
|
is much better to discover immediately if it is not:
|
|
|
|
|
|
|
|
```
|
|
|
|
cp -a /etc/zulip/settings-new.py /etc/zulip/settings.py
|
|
|
|
su zulip -c '/home/zulip/deployments/current/scripts/restart-server'
|
|
|
|
```
|
|
|
|
|
|
|
|
[backups]: ../production/export-and-import.html#backups
|
|
|
|
[changelog]: ../overview/changelog.md
|
|
|
|
|
2019-10-14 21:40:48 +02:00
|
|
|
## Troubleshooting and rollback
|
|
|
|
|
|
|
|
See also the general Zulip server [troubleshooting
|
|
|
|
guide](../production/troubleshooting.md).
|
|
|
|
|
|
|
|
The upgrade scripts are idempotent, so there's no harm in trying again
|
|
|
|
after resolving an issue. The most common causes of errors are:
|
|
|
|
|
|
|
|
* Networking issues (e.g. your Zulip server doesn't have reliable
|
2020-08-11 01:47:54 +02:00
|
|
|
Internet access or needs a proxy set up). Fix the networking issue
|
2019-10-14 21:40:48 +02:00
|
|
|
and try again.
|
|
|
|
* Especially when using `upgrade-zulip-from-git`, systems with the
|
|
|
|
minimal RAM for running Zulip can run into out-of-memory issues
|
|
|
|
during the upgrade process (generally `tools/webpack` is the step
|
|
|
|
that fails). You can get past this by shutting down the Zulip
|
|
|
|
server with `supervisorctl stop all` to free up RAM before running
|
|
|
|
the upgrade process.
|
|
|
|
|
|
|
|
Useful logs are available in a few places:
|
|
|
|
* The Zulip upgrade scripts log all output to
|
|
|
|
`/var/log/zulip/upgrade.log`.
|
|
|
|
* The Zulip server logs all Internal Server Errors to
|
|
|
|
`/var/log/zulip/errors.log`.
|
|
|
|
|
|
|
|
If you need help and don't have a support contract, you can visit
|
|
|
|
[#production
|
|
|
|
help](https://chat.zulip.org/#narrow/stream/31-production-help) in the
|
|
|
|
[Zulip development community
|
|
|
|
server](../contributing/chat-zulip-org.md) for best-effort help.
|
|
|
|
Please include the relevant error output from the above logs in a
|
2020-08-11 01:47:49 +02:00
|
|
|
[Markdown code
|
2021-05-05 02:05:47 +02:00
|
|
|
block](https://zulip.com/help/code-blocks)
|
2019-10-14 21:40:48 +02:00
|
|
|
in any reports.
|
|
|
|
|
|
|
|
### Rolling back to a prior version
|
|
|
|
|
2019-12-13 01:58:10 +01:00
|
|
|
This rollback process is intended for minor releases (e.g. `2.0.3` to
|
|
|
|
`2.0.6`); a more complicated process is required to roll back database
|
|
|
|
migrations before downgrading to an older major release.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
The Zulip upgrade process works by creating a new deployment under
|
|
|
|
`/home/zulip/deployments/` containing a complete copy of the Zulip server code,
|
|
|
|
and then moving the symlinks at `/home/zulip/deployments/{current,last,next}`
|
|
|
|
as part of the upgrade process.
|
|
|
|
|
|
|
|
This means that if the new version isn't working,
|
|
|
|
you can quickly downgrade to the old version by running
|
|
|
|
`/home/zulip/deployments/last/scripts/restart-server`, or to an
|
|
|
|
earlier previous version by running
|
|
|
|
`/home/zulip/deployments/DATE/scripts/restart-server`. The
|
|
|
|
`restart-server` script stops any running Zulip server, and starts
|
|
|
|
the version corresponding to the `restart-server` path you call.
|
|
|
|
|
2021-04-22 00:04:09 +02:00
|
|
|
## Preserving local changes to service configuration files
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```eval_rst
|
|
|
|
.. warning::
|
2021-04-22 00:04:09 +02:00
|
|
|
If you have modified service configuration files installed by
|
2019-10-14 21:40:48 +02:00
|
|
|
Zulip (e.g. the nginx configuration), the Zulip upgrade process will
|
2019-11-26 00:25:49 +01:00
|
|
|
overwrite your configuration when it does the ``puppet apply``.
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
You can test whether this will happen assuming no upstream changes to
|
|
|
|
the configuration using `scripts/zulip-puppet-apply` (without the
|
2020-10-23 02:43:28 +02:00
|
|
|
`-f` option), which will do a test Puppet run and output and changes
|
2019-10-14 21:40:48 +02:00
|
|
|
it would make. Using this list, you can save a copy of any files
|
|
|
|
that you've modified, do the upgrade, and then restore your
|
|
|
|
configuration.
|
|
|
|
|
|
|
|
That said, Zulip's configuration files are designed to be flexible
|
|
|
|
enough for a wide range of installations, from a small self-hosted
|
|
|
|
system to Zulip Cloud. Before making local changes to a configuration
|
|
|
|
file, first check whether there's an option supported by
|
|
|
|
`/etc/zulip/zulip.conf` for the customization you need. And if you
|
|
|
|
need to make local modifications, please report the issue so that we
|
2020-10-23 02:43:28 +02:00
|
|
|
can make the Zulip Puppet configuration flexible enough to handle your
|
2019-10-14 21:40:48 +02:00
|
|
|
setup.
|
|
|
|
|
|
|
|
### nginx configuration changes
|
|
|
|
|
|
|
|
If you need to modify Zulip's `nginx` configuration, we recommend
|
|
|
|
first attempting to add configuration to `/etc/nginx/conf.d` or
|
|
|
|
`/etc/nginx/zulip-include/app.d`; those directories are designed for
|
|
|
|
custom configuration.
|
|
|
|
|
|
|
|
## Upgrading the operating system
|
|
|
|
|
|
|
|
When you upgrade the operating system on which Zulip is installed
|
2020-06-09 07:11:25 +02:00
|
|
|
(E.g. Ubuntu 18.04 Bionic to Ubuntu 20.04 Focal), you need to take
|
2019-10-14 21:40:48 +02:00
|
|
|
some additional steps to update your Zulip installation, documented
|
|
|
|
below.
|
|
|
|
|
|
|
|
The steps are largely the same for the various OS upgrades aside from
|
2020-10-26 22:27:53 +01:00
|
|
|
the versions of PostgreSQL, so you should be able to adapt these
|
2019-10-14 21:40:48 +02:00
|
|
|
instructions for other supported platforms.
|
|
|
|
|
2020-06-09 07:11:25 +02:00
|
|
|
### Upgrading from Ubuntu 18.04 Bionic to 20.04 Focal
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-06-25 01:25:14 +02:00
|
|
|
1. Upgrade your server to the latest Zulip release (at least 3.0,
|
2020-06-09 07:11:25 +02:00
|
|
|
which adds support for Ubuntu Focal).
|
2020-04-16 23:38:44 +02:00
|
|
|
|
|
|
|
2. As the Zulip user, stop the Zulip server and run the following
|
|
|
|
to back up the system:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
|
|
|
supervisorctl stop all
|
|
|
|
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/release-upgrade.backup.tar.gz
|
|
|
|
```
|
|
|
|
|
2020-04-16 23:38:44 +02:00
|
|
|
3. Switch to the root user and upgrade the operating system using the
|
|
|
|
OS's standard tooling. E.g. for Ubuntu, this means running
|
|
|
|
`do-release-upgrade` and following the prompts until it completes
|
|
|
|
successfully:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
|
|
|
sudo -i # Or otherwise get a root shell
|
2020-07-03 19:51:38 +02:00
|
|
|
do-release-upgrade -d
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
2020-07-03 19:51:38 +02:00
|
|
|
The `-d` option to `do-release-upgrade` is required because Ubuntu
|
|
|
|
20.04 is new; it will stop being necessary once the first point
|
|
|
|
release update of Ubuntu 20.04 LTS is released.
|
|
|
|
|
2019-10-14 21:40:48 +02:00
|
|
|
When `do-release-upgrade` asks you how to upgrade configuration
|
2020-10-26 22:27:53 +01:00
|
|
|
files for services that Zulip manages like Redis, PostgreSQL,
|
2020-10-23 02:43:28 +02:00
|
|
|
Nginx, and memcached, the best choice is `N` to keep the
|
2019-10-14 21:40:48 +02:00
|
|
|
currently installed version. But it's not important; the next
|
|
|
|
step will re-install Zulip's configuration in any case.
|
|
|
|
|
2020-06-26 23:33:38 +02:00
|
|
|
4. As root, upgrade the database to the latest version of PostgreSQL:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
2020-10-26 22:50:18 +01:00
|
|
|
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
2020-06-26 23:33:38 +02:00
|
|
|
5. Finally, we need to reinstall the current version of Zulip, which
|
|
|
|
among other things will recompile Zulip's Python module
|
2020-07-05 14:57:52 +02:00
|
|
|
dependencies for your new version of Python and rewrite Zulip's
|
|
|
|
full-text search indexes to work with the upgraded dictionary
|
|
|
|
packages:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
|
|
|
rm -rf /srv/zulip-venv-cache/*
|
|
|
|
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
|
2020-07-05 14:57:52 +02:00
|
|
|
/home/zulip/deployments/current/ --ignore-static-assets --audit-fts-indexes
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
2020-08-11 00:20:10 +02:00
|
|
|
This will finish by restarting your Zulip server; you should now be
|
|
|
|
able to navigate to its URL and confirm everything is working
|
|
|
|
correctly.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-06-09 07:11:25 +02:00
|
|
|
### Upgrading from Ubuntu 16.04 Xenial to 18.04 Bionic
|
|
|
|
|
2020-06-26 23:37:20 +02:00
|
|
|
1. Upgrade your server to the latest Zulip `2.1.x` release. You can
|
|
|
|
only upgrade to Zulip 3.0 and newer after completing this process,
|
|
|
|
since newer releases don't support Ubuntu 16.04 Xenial.
|
2020-06-09 07:11:25 +02:00
|
|
|
|
|
|
|
2. Same as for Bionic to Focal.
|
|
|
|
|
|
|
|
3. Same as for Bionic to Focal.
|
|
|
|
|
|
|
|
4. As root, upgrade the database installation and OS configuration to
|
|
|
|
match the new OS version:
|
|
|
|
|
|
|
|
```
|
|
|
|
touch /usr/share/postgresql/10/pgroonga_setup.sql.applied
|
|
|
|
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
|
|
|
|
pg_dropcluster 10 main --stop
|
|
|
|
systemctl stop postgresql
|
|
|
|
pg_upgradecluster 9.5 main
|
|
|
|
pg_dropcluster 9.5 main
|
|
|
|
apt remove postgresql-9.5
|
|
|
|
systemctl start postgresql
|
|
|
|
systemctl restart memcached
|
|
|
|
```
|
|
|
|
|
2020-08-03 21:41:14 +02:00
|
|
|
5. Finally, we need to reinstall the current version of Zulip, which
|
|
|
|
among other things will recompile Zulip's Python module
|
|
|
|
dependencies for your new version of Python:
|
|
|
|
|
|
|
|
```
|
|
|
|
rm -rf /srv/zulip-venv-cache/*
|
|
|
|
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
|
|
|
|
/home/zulip/deployments/current/ --ignore-static-assets
|
|
|
|
```
|
2020-06-09 07:11:25 +02:00
|
|
|
|
2020-08-11 00:20:10 +02:00
|
|
|
This will finish by restarting your Zulip server; you should now
|
|
|
|
be able to navigate to its URL and confirm everything is working
|
|
|
|
correctly.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-08-03 22:24:38 +02:00
|
|
|
6. [Upgrade to the latest Zulip release](#upgrading-to-a-release), now
|
|
|
|
that your server is running a supported operating system.
|
|
|
|
|
2020-08-11 00:30:07 +02:00
|
|
|
7. As root, finish by verifying the contents of the full-text indexes:
|
|
|
|
|
|
|
|
```
|
2020-08-15 01:55:06 +02:00
|
|
|
/home/zulip/deployments/current/manage.py audit_fts_indexes
|
2020-08-11 00:30:07 +02:00
|
|
|
```
|
|
|
|
|
2019-12-13 01:58:10 +01:00
|
|
|
### Upgrading from Ubuntu 14.04 Trusty to 16.04 Xenial
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-06-26 23:37:20 +02:00
|
|
|
1. Upgrade your server to the latest Zulip `2.0.x` release. You can
|
|
|
|
only upgrade to Zulip `2.1.x` and newer after completing this
|
|
|
|
process, since newer releases don't support Ubuntu 14.04 Trusty.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-06-09 07:11:25 +02:00
|
|
|
2. Same as for Bionic to Focal.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-06-09 07:11:25 +02:00
|
|
|
3. Same as for Bionic to Focal.
|
2020-04-16 23:38:44 +02:00
|
|
|
|
|
|
|
4. As root, upgrade the database installation and OS configuration to
|
2020-08-11 00:20:10 +02:00
|
|
|
match the new OS version:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
2019-12-13 01:58:10 +01:00
|
|
|
apt remove upstart -y
|
2019-10-14 21:40:48 +02:00
|
|
|
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
|
2019-12-13 01:58:10 +01:00
|
|
|
pg_dropcluster 9.5 main --stop
|
2019-10-14 21:40:48 +02:00
|
|
|
systemctl stop postgresql
|
2019-12-13 01:58:10 +01:00
|
|
|
pg_upgradecluster -m upgrade 9.3 main
|
|
|
|
pg_dropcluster 9.3 main
|
|
|
|
apt remove postgresql-9.3
|
2019-10-14 21:40:48 +02:00
|
|
|
systemctl start postgresql
|
2019-12-13 01:58:10 +01:00
|
|
|
service memcached restart
|
2019-10-14 21:40:48 +02:00
|
|
|
```
|
|
|
|
|
2020-08-03 21:41:14 +02:00
|
|
|
5. Finally, we need to reinstall the current version of Zulip, which
|
|
|
|
among other things will recompile Zulip's Python module
|
|
|
|
dependencies for your new version of Python:
|
|
|
|
|
|
|
|
```
|
|
|
|
rm -rf /srv/zulip-venv-cache/*
|
|
|
|
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
|
|
|
|
/home/zulip/deployments/current/ --ignore-static-assets
|
|
|
|
```
|
2020-04-16 23:38:44 +02:00
|
|
|
|
2020-08-11 00:20:10 +02:00
|
|
|
This will finish by restarting your Zulip server; you should now be
|
|
|
|
able to navigate to its URL and confirm everything is working
|
|
|
|
correctly.
|
2020-06-26 23:39:07 +02:00
|
|
|
|
2020-08-03 22:24:38 +02:00
|
|
|
6. [Upgrade from Xenial to
|
|
|
|
Bionic](#upgrading-from-ubuntu-16-04-xenial-to-18-04-bionic), so
|
|
|
|
that you are running a supported operating system.
|
|
|
|
|
2020-04-16 23:38:44 +02:00
|
|
|
### Upgrading from Debian Stretch to Debian Buster
|
|
|
|
|
2020-06-26 23:37:20 +02:00
|
|
|
1. Upgrade your server to the latest Zulip `2.1.x` release. You can
|
|
|
|
only upgrade to Zulip 3.0 and newer after completing this process,
|
|
|
|
since newer releases don't support Ubuntu Debian Stretch.
|
2020-04-16 23:38:44 +02:00
|
|
|
|
2020-06-26 23:40:04 +02:00
|
|
|
2. Same as for Bionic to Focal.
|
2020-04-16 23:38:44 +02:00
|
|
|
|
|
|
|
3. Follow [Debian's instructions to upgrade the OS][debian-upgrade-os].
|
|
|
|
|
|
|
|
[debian-upgrade-os]: https://www.debian.org/releases/buster/amd64/release-notes/ch-upgrading.html
|
|
|
|
|
|
|
|
When prompted for you how to upgrade configuration
|
2020-10-26 22:27:53 +01:00
|
|
|
files for services that Zulip manages like Redis, PostgreSQL,
|
2020-10-23 02:43:28 +02:00
|
|
|
Nginx, and memcached, the best choice is `N` to keep the
|
2020-04-16 23:38:44 +02:00
|
|
|
currently installed version. But it's not important; the next
|
|
|
|
step will re-install Zulip's configuration in any case.
|
|
|
|
|
|
|
|
4. As root, upgrade the database installation and OS configuration to
|
|
|
|
match the new OS version:
|
|
|
|
|
|
|
|
```
|
|
|
|
apt remove upstart -y
|
|
|
|
/home/zulip/deployments/current/scripts/zulip-puppet-apply -f
|
2020-07-23 01:34:46 +02:00
|
|
|
pg_dropcluster 11 main --stop
|
2020-04-16 23:38:44 +02:00
|
|
|
systemctl stop postgresql
|
2020-07-23 01:34:46 +02:00
|
|
|
pg_upgradecluster -m upgrade 9.6 main
|
|
|
|
pg_dropcluster 9.6 main
|
|
|
|
apt remove postgresql-9.6
|
2020-04-16 23:38:44 +02:00
|
|
|
systemctl start postgresql
|
|
|
|
service memcached restart
|
|
|
|
```
|
|
|
|
|
2020-08-03 21:41:14 +02:00
|
|
|
5. Finally, we need to reinstall the current version of Zulip, which
|
|
|
|
among other things will recompile Zulip's Python module
|
|
|
|
dependencies for your new version of Python:
|
|
|
|
|
|
|
|
```
|
|
|
|
rm -rf /srv/zulip-venv-cache/*
|
|
|
|
/home/zulip/deployments/current/scripts/lib/upgrade-zulip-stage-2 \
|
|
|
|
/home/zulip/deployments/current/ --ignore-static-assets
|
|
|
|
```
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-08-11 00:20:10 +02:00
|
|
|
This will finish by restarting your Zulip server; you should now
|
|
|
|
be able to navigate to its URL and confirm everything is working
|
|
|
|
correctly.
|
2020-06-26 23:39:07 +02:00
|
|
|
|
2020-08-03 22:24:38 +02:00
|
|
|
6. [Upgrade to the latest Zulip release](#upgrading-to-a-release), now
|
|
|
|
that your server is running a supported operating system.
|
|
|
|
|
2020-08-11 00:30:07 +02:00
|
|
|
7. As root, finish by verifying the contents of the full-text indexes:
|
|
|
|
|
|
|
|
```
|
2020-08-15 01:55:06 +02:00
|
|
|
/home/zulip/deployments/current/manage.py audit_fts_indexes
|
2020-08-11 00:30:07 +02:00
|
|
|
```
|
|
|
|
|
2020-06-26 23:40:36 +02:00
|
|
|
## Upgrading PostgreSQL
|
|
|
|
|
|
|
|
Starting with Zulip 3.0, we use the latest available version of
|
2021-04-27 01:11:02 +02:00
|
|
|
PostgreSQL at installation time (currently version 13). Upgrades to
|
2020-06-26 23:40:36 +02:00
|
|
|
the version of PostgreSQL are no longer linked to upgrades of the
|
2021-04-27 01:11:02 +02:00
|
|
|
distribution; that is, you may opt to upgrade to PostgreSQL 13 while
|
2020-06-26 23:40:36 +02:00
|
|
|
running Ubuntu 18.04 Bionic.
|
|
|
|
|
|
|
|
To upgrade the version of PostgreSQL on the Zulip server:
|
|
|
|
|
|
|
|
1. Upgrade your server to the latest Zulip release (at least 3.0).
|
|
|
|
|
|
|
|
2. Stop the server and take a backup:
|
|
|
|
|
|
|
|
```
|
|
|
|
supervisorctl stop all
|
|
|
|
/home/zulip/deployments/current/manage.py backup --output=/home/zulip/postgresql-upgrade.backup.tar.gz
|
|
|
|
```
|
|
|
|
|
|
|
|
3. As root, run the database upgrade tool:
|
|
|
|
|
|
|
|
```
|
2020-10-26 22:50:18 +01:00
|
|
|
/home/zulip/deployments/current/scripts/setup/upgrade-postgresql
|
2020-06-26 23:40:36 +02:00
|
|
|
```
|
|
|
|
|
2020-10-26 22:50:18 +01:00
|
|
|
`upgrade-postgresql` will have finished by restarting your Zulip server;
|
2020-07-05 14:57:52 +02:00
|
|
|
you should now be able to navigate to its URL and confirm everything
|
|
|
|
is working correctly.
|
2020-06-26 23:40:36 +02:00
|
|
|
|
|
|
|
|
2019-10-14 21:40:48 +02:00
|
|
|
## Modifying Zulip
|
|
|
|
|
|
|
|
Zulip is 100% free and open source software, and you're welcome to
|
|
|
|
modify it! This section explains how to make and maintain
|
|
|
|
modifications in a safe and convenient fashion.
|
|
|
|
|
|
|
|
If you do modify Zulip and then report an issue you see in your
|
|
|
|
modified version of Zulip, please be responsible about communicating
|
|
|
|
that fact:
|
|
|
|
|
|
|
|
* Ideally, you'd reproduce the issue in an unmodified version (e.g. on
|
|
|
|
[chat.zulip.org](../contributing/chat-zulip-org.md) or
|
2020-06-08 23:04:39 +02:00
|
|
|
[zulip.com](https://zulip.com)).
|
2019-10-14 21:40:48 +02:00
|
|
|
* Where that is difficult or you think it's very unlikely your changes
|
|
|
|
are related to the issue, just mention your changes in the issue report.
|
|
|
|
|
|
|
|
If you're looking to modify Zulip by applying changes developed by the
|
|
|
|
Zulip core team and merged into master, skip to [this
|
|
|
|
section](#applying-changes-from-master).
|
|
|
|
|
|
|
|
## Making changes
|
|
|
|
|
|
|
|
One way to modify Zulip is to just edit files under
|
|
|
|
`/home/zulip/deployments/current` and then restart the server. This
|
|
|
|
can work OK for testing small changes to Python code or shell scripts.
|
|
|
|
But we don't recommend this approach for maintaining changes because:
|
|
|
|
|
|
|
|
* You cannot modify JavaScript, CSS, or other frontend files this way,
|
|
|
|
because we don't include them in editable form in our production
|
|
|
|
release tarballs (doing so would make our release tarballs much
|
|
|
|
larger without any runtime benefit).
|
|
|
|
* You will need to redo your changes after you next upgrade your Zulip
|
|
|
|
server (or they will be lost).
|
|
|
|
* You need to remember to restart the server or your changes won't
|
|
|
|
have effect.
|
|
|
|
* Your changes aren't tracked, so mistakes can be hard to debug.
|
|
|
|
|
|
|
|
Instead, we recommend the following GitHub-based workflow (see [our
|
|
|
|
Git guide][git-guide] if you need a primer):
|
|
|
|
|
|
|
|
* Decide where you're going to edit Zulip's code. We recommend [using
|
|
|
|
the Zulip development environment](../development/overview.md) on
|
|
|
|
a desktop or laptop as it will make it extremely convenient for you
|
|
|
|
to test your changes without deploying them in production. But if
|
|
|
|
your changes are small or you're OK with risking downtime, you don't
|
|
|
|
strictly need it; you just need an environment with Git installed.
|
|
|
|
* **Important**. Determine what Zulip version you're running on your
|
|
|
|
server. You can check by inspecting `ZULIP_VERSION` in
|
|
|
|
`/home/zulip/deployments/current/version.py` (we'll use `2.0.4`
|
|
|
|
below). If you apply your changes to the wrong version of Zulip,
|
|
|
|
it's likely to fail and potentially cause downtime.
|
|
|
|
* [Fork and clone][fork-clone] the [zulip/zulip][] repository on
|
|
|
|
[GitHub](https://github.com).
|
|
|
|
* Create a branch (named `acme-branch` below) containing your changes:
|
|
|
|
|
|
|
|
```
|
|
|
|
cd zulip
|
|
|
|
git checkout -b acme-branch 2.0.4
|
|
|
|
```
|
|
|
|
|
|
|
|
* Use your favorite code editor to modify Zulip.
|
|
|
|
* Commit your changes and push them to GitHub:
|
|
|
|
|
|
|
|
```
|
|
|
|
git commit -a
|
|
|
|
|
|
|
|
# Use `git diff` to verify your changes are what you expect
|
|
|
|
git diff 2.0.4 acme-branch
|
|
|
|
|
|
|
|
# Push the changes to your GitHub fork
|
|
|
|
git push origin +acme-branch
|
|
|
|
```
|
|
|
|
|
2020-08-11 02:20:10 +02:00
|
|
|
* Log in to your Zulip server and configure and use
|
2019-10-14 21:40:48 +02:00
|
|
|
[upgrade-zulip-from-git][] to install the changes; remember to
|
|
|
|
configure `git_repo_url` to point to your fork on GitHub and run it as
|
|
|
|
`upgrade-zulip-from-git acme-branch`.
|
|
|
|
|
|
|
|
This workflow solves all of the problems described above: your change
|
|
|
|
will be compiled and installed correctly (restarting the server), and
|
|
|
|
your changes will be tracked so that it's convenient to maintain them
|
|
|
|
across future Zulip releases.
|
|
|
|
|
|
|
|
### Upgrading to future releases
|
|
|
|
|
2019-12-13 01:58:10 +01:00
|
|
|
Eventually, you'll want to upgrade to a new Zulip release. If your
|
|
|
|
changes were integrated into that Zulip release or are otherwise no
|
|
|
|
longer needed, you can just [upgrade as
|
2020-07-08 21:14:51 +02:00
|
|
|
usual](#upgrading-to-a-release). If you [upgraded to
|
|
|
|
master](#upgrading-to-master); review that section again; new
|
|
|
|
maintenance releases are likely "older" than your current installation
|
|
|
|
and you might need to upgrade to the master again rather than to the
|
|
|
|
new maintenance release.
|
|
|
|
|
|
|
|
Otherwise, you'll need to update your branch by rebasing your changes
|
|
|
|
(starting from a [clone][fork-clone] of the [zulip/zulip][]
|
|
|
|
repository). The example below assumes you have a branch off of 2.0.4
|
|
|
|
and want to upgrade to 2.1.0.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
```
|
|
|
|
cd zulip
|
|
|
|
git fetch --tags upstream
|
|
|
|
git checkout acme-branch
|
2019-12-13 01:58:10 +01:00
|
|
|
git rebase --onto 2.1.0 2.0.4
|
2020-08-11 01:47:54 +02:00
|
|
|
# Fix any errors or merge conflicts; see Zulip's Git guide for advice
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
# Use `git diff` to verify your changes are what you expect
|
|
|
|
git diff 2.1.0 acme-branch
|
|
|
|
|
|
|
|
git push origin +acme-branch
|
|
|
|
```
|
|
|
|
|
|
|
|
And then use [upgrade-zulip-from-git][] to install your updated
|
|
|
|
branch, as before.
|
|
|
|
|
|
|
|
### Making changes with docker-zulip
|
|
|
|
|
|
|
|
If you are using [docker-zulip][], there are two things that are
|
|
|
|
different from the above:
|
|
|
|
|
|
|
|
* Because of how container images work, editing files directly is even
|
|
|
|
more precarious, because Docker is designed for working with
|
|
|
|
container images and may lose your changes.
|
|
|
|
* Instead of running `upgrade-zulip-from-git`, you will need to use
|
|
|
|
the [docker upgrade workflow][docker-zulip-upgrade] to build a
|
|
|
|
container image based on your modified version of Zulip.
|
|
|
|
|
|
|
|
[docker-zulip]: https://github.com/zulip/docker-zulip
|
|
|
|
[docker-zulip-upgrade]: https://github.com/zulip/docker-zulip#upgrading-from-a-git-repository
|
|
|
|
|
|
|
|
## Applying changes from master
|
|
|
|
|
|
|
|
If you are experiencing an issue that has already been fixed by the
|
|
|
|
Zulip development community, and you'd like to get the fix now, you
|
|
|
|
have a few options. There are two possible ways you might get those
|
|
|
|
fixes on your local Zulip server without waiting for an official release.
|
|
|
|
|
|
|
|
### Applying a small change
|
|
|
|
|
|
|
|
Many bugs have small/simple fixes. In this case, you can use the Git
|
|
|
|
workflow [described above](#making-changes), using:
|
|
|
|
|
|
|
|
```
|
|
|
|
git fetch upstream
|
|
|
|
git cherry-pick abcd1234
|
|
|
|
```
|
|
|
|
|
|
|
|
instead of "making changes locally" (where `abcd1234` is the commit ID
|
|
|
|
of the change you'd like).
|
|
|
|
|
2019-12-13 01:58:10 +01:00
|
|
|
In general, we can't provide unpaid support for issues caused by
|
2019-10-14 21:40:48 +02:00
|
|
|
cherry-picking arbitrary commits if the issues don't also affect
|
2019-12-13 01:58:10 +01:00
|
|
|
master or an official release.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2019-12-13 01:58:10 +01:00
|
|
|
The exception to this rule is when we ask or encourage a user to apply
|
|
|
|
a change to their production system to help verify the fix resolves
|
|
|
|
the issue for them. You can expect the Zulip community to be
|
2019-12-13 02:11:45 +01:00
|
|
|
responsive in debugging any problems caused by a patch we asked
|
2019-12-13 01:58:10 +01:00
|
|
|
you to apply.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-02-29 03:32:23 +01:00
|
|
|
Also, consider asking whether a small fix that is important to you can
|
|
|
|
be added to the current stable release branch (E.g. `2.1.x`). In
|
|
|
|
addition to scheduling that change for Zulip's next bug fix release,
|
|
|
|
we support changes in stable release branches as though they were
|
|
|
|
released.
|
|
|
|
|
2019-10-14 21:40:48 +02:00
|
|
|
### Upgrading to master
|
|
|
|
|
2020-07-08 21:14:51 +02:00
|
|
|
Many Zulip servers (including chat.zulip.org and zulip.com) upgrade to
|
|
|
|
master on a regular basis to get the latest features. Before doing
|
|
|
|
so, it's important to understand how to happily run a server based on
|
|
|
|
master.
|
|
|
|
|
|
|
|
For background, it's backporting arbitrary patches from master to an
|
|
|
|
older version requires some care. Common issues include:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
* Changes containing database migrations (new files under
|
2019-12-13 01:58:10 +01:00
|
|
|
`*/migrations/`), which includes most new features. We
|
2019-10-14 21:40:48 +02:00
|
|
|
don't support applying database migrations out of order.
|
|
|
|
* Changes that are stacked on top of other changes to the same system.
|
2020-07-08 21:14:51 +02:00
|
|
|
* Essentially any patch with hundreds of lines of changes will have
|
|
|
|
merge conflicts and require extra work to apply.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
While it's possible to backport these sorts of changes, you're
|
|
|
|
unlikely to succeed without help from the core team via a support
|
|
|
|
contract.
|
|
|
|
|
2019-12-13 02:11:45 +01:00
|
|
|
If you need an unreleased feature, the best path is usually to
|
2019-12-13 01:58:10 +01:00
|
|
|
upgrade to Zulip master using [upgrade-zulip-from-git][]. Before
|
|
|
|
upgrading to master, make sure you understand:
|
2019-10-14 21:40:48 +02:00
|
|
|
|
2020-07-08 21:14:51 +02:00
|
|
|
* In Zulip's version numbering scheme, `master` will always be "newer"
|
|
|
|
than the latest maintenance release (E.g. `3.1` or `2.1.6`) and
|
|
|
|
"older" than the next major release (E.g. `3.0` or `4.0`).
|
2019-10-14 21:40:48 +02:00
|
|
|
* The `master` branch is under very active development; dozens of new
|
2020-07-08 21:14:51 +02:00
|
|
|
changes are integrated into it on most days. The `master` branch
|
|
|
|
can have thousands of changes not present in the latest release (all
|
|
|
|
of which will be included in our next major release). On average
|
|
|
|
`master` usually has fewer total bugs than the latest release
|
|
|
|
(because we fix hundreds of bugs in every major release) but it
|
|
|
|
might have some bugs that are more severe than we would consider
|
|
|
|
acceptable for a release.
|
|
|
|
* We deploy `master` to chat.zulip.org and zulip.com on a regular
|
2019-12-13 01:58:10 +01:00
|
|
|
basis (often daily), so it's very important to the project that it
|
|
|
|
be stable. Most regressions will be minor UX issues or be fixed
|
2020-07-08 21:14:51 +02:00
|
|
|
quickly, because we need them to be fixed for Zulip Cloud.
|
2019-10-14 21:40:48 +02:00
|
|
|
* The development community is very interested in helping debug issues
|
|
|
|
that arise when upgrading from the latest release to master, since
|
|
|
|
they provide us an opportunity to fix that category of issue before
|
|
|
|
our next major release. (Much more so than we are in helping folks
|
|
|
|
debug other custom changes). That said, we cannot make any
|
|
|
|
guarantees about how quickly we'll resolve an issue to folks without
|
|
|
|
a formal support contract.
|
2020-07-08 21:14:51 +02:00
|
|
|
* We do not support downgrading from `master` to earlier versions, so
|
|
|
|
if downtime for your Zulip server is unacceptable, make sure you
|
|
|
|
have a current
|
|
|
|
[backup](../production/export-and-import.html#backups) in case the
|
|
|
|
upgrade fails.
|
2019-10-14 21:40:48 +02:00
|
|
|
* Our changelog contains [draft release
|
|
|
|
notes](../overview/changelog.md) available listing major changes
|
|
|
|
since the last release. The **Upgrade notes** section will always
|
|
|
|
be current, even if some new features aren't documented.
|
2020-07-08 21:14:51 +02:00
|
|
|
* Whenever we push a security or maintenance release, the changes in
|
|
|
|
that release will always be merged to master; so you can get the
|
|
|
|
security fixes by upgrading to master.
|
|
|
|
* You can always upgrade from master to the next major release when it
|
|
|
|
comes out, using either [upgrade-zulip-from-git][] or the release
|
|
|
|
tarball. So there's no risk of upgrading to `master` resulting in
|
|
|
|
a system that's not upgradeable back to a normal release.
|
2019-10-14 21:40:48 +02:00
|
|
|
|
|
|
|
## Contributing patches
|
|
|
|
|
|
|
|
Zulip contains thousands of changes submitted by volunteer
|
|
|
|
contributors like you. If your changes are likely to be of useful to
|
|
|
|
other organizations, consider [contributing
|
|
|
|
them](../overview/contributing.md).
|
|
|
|
|
|
|
|
[fork-clone]: ../git/cloning.html#get-zulip-code
|
|
|
|
[upgrade-zulip-from-git]: #upgrading-from-a-git-repository
|
|
|
|
[upgrade-zulip]: #upgrading
|
|
|
|
[git-guide]: ../git/index.md
|
|
|
|
[zulip/zulip]: https://github.com/zulip/zulip/
|
|
|
|
[docker-upgrade]: https://github.com/zulip/docker-zulip#upgrading-the-zulip-container
|