zulip/docs/production/ssl-certificates.md

# Installing SSL certificates

To keep your communications secure, Zulip runs over HTTPS only.
You'll need an SSL/TLS certificate.

Fortunately, since about 2017, new options can make getting and
maintaining a genuine, trusted-by-browsers certificate no longer the
chore (nor expense) that it used to be.

## Manual install

If you already have an SSL certificate, just install (or symlink) its
files into place at the following paths:

- `/etc/ssl/private/zulip.key` for the private key
- `/etc/ssl/certs/zulip.combined-chain.crt` for the certificate.

Your certificate file should contain not only your own certificate but
its **full chain, including any intermediate certificates** used by
your certificate authority (CA). See the [nginx
documentation][nginx-chains] for details on what this means. If
you're missing part of the chain, your server may work with some
browsers, but not others and not the Zulip mobile and desktop apps.
The desktop apps support [configuring a custom CA][desktop-certs] to
allow validation of certificates generated by an internal CA.

[nginx-chains]: https://nginx.org/en/docs/http/configuring_https_servers.html#chains

### Testing

Just trying in a browser is not an adequate test, because some
browsers ignore errors that others don't.

Two good tests include:

- If your server is accessible from the public Internet, use the [SSL
  Labs tester][ssllabs-tester]. Be sure to check for "Chain issues";
  if any, your certificate file is missing intermediate certificates.

- Alternatively, run a command like `curl -SsI https://zulip.example.com`
  (using your server's URL) from a machine that can reach your server.
  Make sure that on the same machine,
  `curl -SsI https://incomplete-chain.badssl.com` gives an error;
  `curl` on some machines, including Macs, will accept incomplete
  chains.

[ssllabs-tester]: https://www.ssllabs.com/ssltest/analyze.html

## Certbot (recommended)

[Let's Encrypt](https://letsencrypt.org/) is a free, completely
automated CA launched in 2016 to help make HTTPS routine for the
entire Web. Zulip offers a simple automation for
[Certbot](https://certbot.eff.org/), a Let's Encrypt client, to get
SSL certificates from Let's Encrypt and renew them automatically.

We recommend most Zulip servers use Certbot. You'll want something
else if:

- you have an existing workflow for managing SSL certificates
  that you prefer;
- you need wildcard certificates (support from Let's Encrypt released
  in [March 2018][letsencrypt-wildcard]); or
- your Zulip server is not on the public Internet. (In this case you
  can [still use Certbot][certbot-manual-mode], but it's less
  convenient; and you'll want to ignore Zulip's automation.)

[letsencrypt-wildcard]: https://certbot.eff.org/faq#does-let-s-encrypt-issue-wildcard-certificates
[certbot-manual-mode]: https://certbot.eff.org/docs/using.html#manual

### At initial Zulip install

To enable the Certbot automation when first installing Zulip, just
pass the `--certbot` flag when [running the install script][doc-install-script].

The `--hostname` and `--email` options are required when using
`--certbot`. You'll need the hostname to be a real DNS name, and the
Zulip server machine to be reachable by that name from the public
Internet.

If you need to configure a multiple domain certificate, you can generate
one as described in the section below after installing Zulip.

[doc-install-script]: ../production/install.html#step-2-install-zulip

### After Zulip is already installed

To enable the Certbot automation on an already-installed Zulip
server, run the following commands:

```bash
sudo -s  # If not already root
/home/zulip/deployments/current/scripts/setup/setup-certbot --email=EMAIL HOSTNAME [HOSTNAME2...]
```

where HOSTNAME is the domain name users see in their browser when
using the server (e.g., `zulip.example.com`), and EMAIL is a contact
address for the server admins. Additional hostnames can also be
specified to issue a certificate for multiple domains.

### How it works

When the Certbot automation in Zulip is first enabled, by either
method, it creates an account for the server at the Let's Encrypt CA;
requests a certificate for the given hostname; proves to the CA that
the server controls the website at that hostname; and is then given a
certificate. (For details, refer to
[Let's Encrypt](https://letsencrypt.org/how-it-works/).)

### Renewal

Let's Encrypt certificates expire after 90 days. Short expiration
periods are good for security, but they also mean that it's important
to automatically renew them to avoid regular maintenance work.

Zulip configures automatic renewal for you. As a result, a Zulip
server configured with Certbot does not require any ongoing work to
maintain a current valid SSL certificate.

Specifically, the `setup-certbot` tool (and by extension, the
installer option) enables the Certbot `auto_renew` property in
`/etc/zulip/zulip.conf`. This, in turn, configures a cron job
(`/etc/cron.d/certbot`) that will renew any Certbot certificates that
are due for renewal. The renewal process repeats the Certbot
proof-of-control process, receives the new certificate from Certbot,
installs the new certificate, and then reloads `nginx`.

#### Troubleshooting

If your Certbot certificate expires, it is usually because of firewall
rules preventing the Certbot renewal process (which is essentially
identical to the initial certificate request process) from
working. You can debug interactively by running the command from the
cron job,
`/home/zulip/deployments/current/scripts/lib/certbot-maybe-renew`, as
`root`.

## Self-signed certificate

If you aren't able to use Certbot, you can generate a self-signed SSL
certificate. This can be convenient for testing, but isn't
recommended for production, as it is insecure. The Zulip desktop and
mobile apps will not connect to a server if they cannot validate its
SSL certificate. The desktop apps support [configuring a custom
certificate authority][desktop-certs] to allow validation of an
internal certificate.

To generate a self-signed certificate when first installing Zulip,
just pass the `--self-signed-cert` flag when
[running the install script][doc-install-script].

To generate a self-signed certificate for an already-installed Zulip
server, run the following commands:

```bash
sudo -s  # If not already root
/home/zulip/deployments/current/scripts/setup/generate-self-signed-cert HOSTNAME
```

where HOSTNAME is the domain name (or IP address) to use on the
generated certificate.

After replacing the certificates, you need to reload `nginx` by
running the following as `root`:

```bash
service nginx reload
```

[desktop-certs]: https://zulip.com/help/custom-certificates

## Troubleshooting

### The Android app can't connect to the server

This is most often caused by an incomplete certificate chain. See
discussion in the [Manual install](#manual-install) section above.

### The iOS app can't connect to the server

This can be caused by a server set up to support only TLS 1.1 or
older (including TLS 1.0, SSL 3, or SSL 2.)

TLS 1.2 has been a standard for over 10 years, and all modern web
server software supports it. Starting in early 2020, all major
browsers [will _require_ TLS 1.2 or later][tls12-required-news], and
will refuse to connect over TLS 1.1 or older. And on iOS, Apple [has
since iOS 9][apple-ats] required TLS 1.2 for all connections made by
apps, unless the app specifically opts into lower security.

[tls12-required-news]: https://www.thesslstore.com/blog/apple-microsoft-google-disable-tls-1-0-tls-1-1/
[apple-ats]: https://developer.apple.com/library/archive/releasenotes/General/WhatsNewIniOS/Articles/iOS9.html

If your server is reachable from the public Internet, a convenient way
to check what TLS versions it supports is the [SSL Labs
tester][ssllabs-tester].

To resolve this issue, update your server to support TLS 1.2,
and preferably also TLS 1.3. For nginx, see [the `ssl_protocols`
directive][nginx-doc-protocols] in your configuration.

[nginx-doc-protocols]: https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_protocols

### The Android app connects to the server on some devices but not others

An issue on Android 7.0 ([report][android7.0-tls-issue],
[description][android7.0-tls-issue-so]) in the system TLS/SSL stack,
which the Zulip app relies on, makes it finicky about the server's TLS
configuration.

[android7.0-tls-issue]: https://issuetracker.google.com/issues/37122132#comment13
[android7.0-tls-issue-so]: https://stackoverflow.com/a/42047877

The issue is that Android 7.0 supports only the curve `secp256r1` when
doing elliptic-curve cryptography for TLS, and not other curves like
`secp384r1` or `secp512r1`. If your server's TLS/SSL configuration
offers only other curves, then Android 7.0 clients will be unable to
connect.

By default `nginx` (and therefore a Zulip server) offers the
`secp256r1` curve among others, and so everything works. You can
control the offered curves with `ssl_ecdh_curve` in the `nginx`
configuration on your server. See [nginx docs][nginx-doc-curve] for
details.

[nginx-doc-curve]: https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_ecdh_curve

Two signs for diagnosing this issue in contrast to some other root
cause:

- This issue affects only Android 7.0; it's fixed in Android 7.1.1 and
  later.

- If your server is reachable from the public Internet, use the [SSL
  Labs tester][ssllabs-tester]. Under "Cipher Suites" you may see
  lines beginning with `TLS_ECDHE`, for cipher suites which use
  elliptic-curve cryptography. These lines will have further text
  like `ECDH secp256r1` or `ECDH secp384r1`, which identifies specific
  elliptic curves your server offers to use. This issue applies if
  your server does not offer `secp256r1`.