zulip/docs/production/ssl-certificates.md

Installing SSL certificates

To keep your communications secure, Zulip runs over HTTPS only. You'll need an SSL/TLS certificate.

Fortunately, since about 2017, new options can make getting and maintaining a genuine, trusted-by-browsers certificate no longer the chore (nor expense) that it used to be.

Manual install

If you already have an SSL certificate, just install (or symlink) its files into place at the following paths:

• /etc/ssl/private/zulip.key for the private key
• /etc/ssl/certs/zulip.combined-chain.crt for the certificate.

Your certificate file should contain not only your own certificate but its full chain, including any intermediate certificates used by your certificate authority (CA). See the nginx documentation for details on what this means. If you're missing part of the chain, your server may work with some browsers, but not others and not the Zulip mobile and desktop apps. The desktop apps support configuring a custom CA to allow validation of certificates generated by an internal CA.

Testing

Just trying in a browser is not an adequate test, because some browsers ignore errors that others don't.

Two good tests include:

• If your server is accessible from the public Internet, use the SSL Labs tester. Be sure to check for "Chain issues"; if any, your certificate file is missing intermediate certificates.

• Alternatively, run a command like curl -SsI https://zulip.example.com (using your server's URL) from a machine that can reach your server. Make sure that on the same machine, curl -SsI https://incomplete-chain.badssl.com gives an error; curl on some machines, including Macs, will accept incomplete chains.

Certbot (recommended)

Let's Encrypt is a free, completely automated CA launched in 2016 to help make HTTPS routine for the entire Web. Zulip offers a simple automation for Certbot, a Let's Encrypt client, to get SSL certificates from Let's Encrypt and renew them automatically.

We recommend most Zulip servers use Certbot. You'll want something else if:

• you have an existing workflow for managing SSL certificates that you prefer;
• you need wildcard certificates (support from Let's Encrypt released in March 2018); or
• your Zulip server is not on the public Internet. (In this case you can still use Certbot, but it's less convenient; and you'll want to ignore Zulip's automation.)

At initial Zulip install

To enable the Certbot automation when first installing Zulip, just pass the --certbot flag when running the install script.

The --hostname and --email options are required when using --certbot. You'll need the hostname to be a real DNS name, and the Zulip server machine to be reachable by that name from the public Internet.

If you need to configure a multiple domain certificate, you can generate one as described in the section below after installing Zulip.

After Zulip is already installed

To enable the Certbot automation on an already-installed Zulip server, run the following commands:

sudo -s  # If not already root
/home/zulip/deployments/current/scripts/setup/setup-certbot --email=EMAIL HOSTNAME [HOSTNAME2...]

where HOSTNAME is the domain name users see in their browser when using the server (e.g., zulip.example.com), and EMAIL is a contact address for the server admins. Additional hostnames can also be specified to issue a certificate for multiple domains.

How it works

When the Certbot automation in Zulip is first enabled, by either method, it creates an account for the server at the Let's Encrypt CA; requests a certificate for the given hostname; proves to the CA that the server controls the website at that hostname; and is then given a certificate. (For details, refer to Let's Encrypt.)

Renewal

Let's Encrypt certificates expire after 90 days. Short expiration periods are good for security, but they also mean that it's important to automatically renew them to avoid regular maintenance work.

Zulip configures automatic renewal for you. As a result, a Zulip server configured with Certbot does not require any ongoing work to maintain a current valid SSL certificate.

The certbot package configures a systemd timer (similar to a cron job) that will renew any Certbot certificates that are due for renewal. The renewal process repeats the Certbot proof-of-control process, receives the new certificate from Certbot, installs the new certificate, and then reloads nginx.

Troubleshooting

If your Certbot certificate expires, it is usually because of firewall rules preventing the Certbot renewal process (which is essentially identical to the initial certificate request process) from working. You can debug interactively by running the command from the cron job, /usr/bin/certbot renew, as root.

Self-signed certificate

If you aren't able to use Certbot, you can generate a self-signed SSL certificate. This can be convenient for testing, but isn't recommended for production, as it is insecure. The Zulip desktop and mobile apps will not connect to a server if they cannot validate its SSL certificate. The desktop apps support configuring a custom certificate authority to allow validation of an internal certificate.

To generate a self-signed certificate when first installing Zulip, just pass the --self-signed-cert flag when running the install script.

To generate a self-signed certificate for an already-installed Zulip server, run the following commands:

sudo -s  # If not already root
/home/zulip/deployments/current/scripts/setup/generate-self-signed-cert HOSTNAME

where HOSTNAME is the domain name (or IP address) to use on the generated certificate.

After replacing the certificates, you need to reload nginx by running the following as root:

service nginx reload

Troubleshooting

The Android app can't connect to the server

This is most often caused by an incomplete certificate chain. See discussion in the Manual install section above.

The iOS app can't connect to the server

This can be caused by a server set up to support only TLS 1.1 or older (including TLS 1.0, SSL 3, or SSL 2.)

TLS 1.2 has been a standard for over 10 years, and all modern web server software supports it. Starting in early 2020, all major browsers will require TLS 1.2 or later, and will refuse to connect over TLS 1.1 or older. And on iOS, Apple has since iOS 9 required TLS 1.2 for all connections made by apps, unless the app specifically opts into lower security.

If your server is reachable from the public Internet, a convenient way to check what TLS versions it supports is the SSL Labs tester.

To resolve this issue, update your server to support TLS 1.2, and preferably also TLS 1.3. For nginx, see the ssl_protocols directive in your configuration.

The Android app connects to the server on some devices but not others

An issue on Android 7.0 (report, description) in the system TLS/SSL stack, which the Zulip app relies on, makes it finicky about the server's TLS configuration.

The issue is that Android 7.0 supports only the curve secp256r1 when doing elliptic-curve cryptography for TLS, and not other curves like secp384r1 or secp512r1. If your server's TLS/SSL configuration offers only other curves, then Android 7.0 clients will be unable to connect.

By default nginx (and therefore a Zulip server) offers the secp256r1 curve among others, and so everything works. You can control the offered curves with ssl_ecdh_curve in the nginx configuration on your server. See nginx docs for details.

Two signs for diagnosing this issue in contrast to some other root cause:

• This issue affects only Android 7.0; it's fixed in Android 7.1.1 and later.

• If your server is reachable from the public Internet, use the SSL Labs tester. Under "Cipher Suites" you may see lines beginning with TLS_ECDHE, for cipher suites which use elliptic-curve cryptography. These lines will have further text like ECDH secp256r1 or ECDH secp384r1, which identifies specific elliptic curves your server offers to use. This issue applies if your server does not offer secp256r1.