docs: Update proxy docs.

Notable changes:
 - Describe `X-Forwarded-For` by name.
 - Switch each specific proxy to numbered steps.
 - Link back to the `X-Forwarded-For` section in each proxy
 - Default to using HTTPS, not HTTP, for the backend.
 - Include the HTTP-to-HTTPS redirect code for all proxies; it is
   important that it happen at the proxy, as the backend is unaware of
   it.
 - Call out Apache2 modules which are necessary.
 - Specify where the dhparam.pem file can be found.
 - Call out the `Host:` header forwarding necessary, and document
   `USE_X_FORWARDED_HOST` if that is not possible.
 - Standardize on 20 minutes of connection timeout.

docs/production/deployment.md

@@ -327,10 +327,15 @@ HTTP as follows:
 
 #### Configuring Zulip to trust proxies
 
-Before placing Zulip behind a reverse proxy, it needs to be configured to trust
+Before placing Zulip behind a reverse proxy, it needs to be configured
-the client IP addresses that the proxy reports. This is important to have
+to trust the client IP addresses that the proxy reports via the
-accurate IP addresses in server logs, as well as in notification emails which
+`X-Forwarded-For` header. This is important to have accurate IP
-are sent to end users.
+addresses in server logs, as well as in notification emails which are
+sent to end users. Zulip doesn't default to trusting all
+`X-Forwarded-For` headers, because doing so would allow clients to
+spoof any IP address; we specify which IP addresses are the Zulip
+server's incoming proxies, so we know how much of the
+`X-Forwarded-For` header to trust.
 
 1. Determine the IP addresses of all reverse proxies you are setting up, as seen
    from the Zulip host. Depending on your network setup, these may not be the
@@ -354,24 +359,38 @@ are sent to end users.
 
 ### nginx configuration
 
-For `nginx` configuration, there's two things you need to set up:
+Below is a working example of a full nginx configuration. It assumes
+that your Zulip server sits at `https://10.10.10.10:443`; see
+[above](#configuring-zulip-to-allow-http) to switch to HTTP.
 
-- The root `nginx.conf` file. We recommend using
+1. Follow the instructions to [configure Zulip to trust
+   proxies](#configuring-zulip-to-trust-proxies).
+
+1. Configure the root `nginx.conf` file. We recommend using
    `/etc/nginx/nginx.conf` from your Zulip server for our recommended
    settings. E.g. if you don't set `client_max_body_size`, it won't be
    possible to upload large files to your Zulip server.
-- The `nginx` site-specific configuration (in
+
+1. Configure the `nginx` site-specific configuration (in
    `/etc/nginx/sites-available`) for the Zulip app. The following
    example is a good starting point:
 
-```nginx
+   ```nginx
-server {
+   server {
+           listen 80;
+           listen [::]:80;
+           location / {
+                   return 301 https://$host$request_uri;
+           }
+   }
+
+   server {
            listen                  443 ssl http2;
            listen                  [::]:443 ssl http2;
-        server_name             zulip.example.net;
+           server_name             zulip.example.com;
 
-        ssl_certificate         /path/to/fullchain-cert.pem;
+           ssl_certificate         /etc/letsencrypt/live/zulip.example.com/fullchain.pem;
-        ssl_certificate_key     /path/to/private-key.pem;
+           ssl_certificate_key     /etc/letsencrypt/live/zulip.example.com/privkey.pem;
 
            location / {
                    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -379,18 +398,14 @@ server {
                    proxy_http_version      1.1;
                    proxy_buffering         off;
                    proxy_read_timeout      20m;
-                proxy_pass              https://zulip-upstream-host;
+                   proxy_pass              https://10.10.10.10:443;
            }
-}
+   }
-```
+   ```
 
-Don't forget to update `server_name`, `ssl_certificate`,
+   Don't forget to update `server_name`, `ssl_certificate`,
-`ssl_certificate_key` and `proxy_pass` with the appropriate values for
+   `ssl_certificate_key` and `proxy_pass` with the appropriate values
-your installation.
+   for your deployment.
-
-On the Zulip side, you will need to add the `nginx` server IP as a trusted
-reverse proxy. Follow the instructions to [configure Zulip to trust
-proxies](#configuring-zulip-to-trust-proxies).
 
 [nginx-proxy-longpolling-config]: https://github.com/zulip/zulip/blob/main/puppet/zulip/files/nginx/zulip-include-common/proxy_longpolling
 [standalone.pp]: https://github.com/zulip/zulip/blob/main/puppet/zulip/manifests/profile/standalone.pp
@@ -399,26 +414,24 @@ proxies](#configuring-zulip-to-trust-proxies).
 ### Apache2 configuration
 
 Below is a working example of a full Apache2 configuration. It assumes
-that your Zulip sits at `http://localhost:5080`. You first need to
+that your Zulip server sits at `https://internal.zulip.hostname:443`.
-make the following changes in two configuration files.
+Note that if you wish to use SSL to connect to the Zulip server,
+Apache requires you use the hostname, not the IP address; see
+[above](#configuring-zulip-to-allow-http) to switch to HTTP.
 
-1. Follow the instructions for [Configure Zulip to allow HTTP](#configuring-zulip-to-allow-http).
+1. Follow the instructions to [configure Zulip to trust
+   proxies](#configuring-zulip-to-trust-proxies).
 
-2. Add the following to `/etc/zulip/settings.py`:
+1. Set `USE_X_FORWARDED_HOST = True` in `/etc/zulip/settings.py` and
+   restart Zulip.
 
-   ```python
+1. Enable some required Apache modules:
-   EXTERNAL_HOST = 'zulip.example.com'
+
-   ALLOWED_HOSTS = ['zulip.example.com', '127.0.0.1']
+   ```
-   USE_X_FORWARDED_HOST = True
+   a2enmod ssl proxy proxy_http headers rewrite
    ```
 
-3. Restart your Zulip server with `/home/zulip/deployments/current/scripts/restart-server`.
+1. Create an Apache2 virtual host configuration file, similar to the
-
-4. Follow the instructions to [configure Zulip to trust
-   proxies](#configuring-zulip-to-trust-proxies). For this example, the reverse
-   proxy IP would be `127.0.0.1`.
-
-5. Create an Apache2 virtual host configuration file, similar to the
    following. Place it the appropriate path for your Apache2
    installation and enable it (E.g. if you use Debian or Ubuntu, then
    place it in `/etc/apache2/sites-available/zulip.example.com.conf`
@@ -436,22 +449,20 @@ make the following changes in two configuration files.
      ServerName zulip.example.com
 
      RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
-     RequestHeader set "X-Forwarded-SSL" expr=%{HTTPS}
 
      RewriteEngine On
-     RewriteRule /(.*)           http://localhost:5080/$1 [P,L]
+     RewriteRule /(.*)           https://internal.zulip.hostname:443/$1 [P,L]
 
      <Location />
        Require all granted
-       ProxyPass  http://localhost:5080/  timeout=300
+       ProxyPass https://internal.zulip.hostname:443/ timeout=1200
-       ProxyPassReverse  http://localhost:5080/
-       ProxyPassReverseCookieDomain  127.0.0.1  zulip.example.com
      </Location>
 
      SSLEngine on
      SSLProxyEngine on
      SSLCertificateFile /etc/letsencrypt/live/zulip.example.com/fullchain.pem
      SSLCertificateKeyFile /etc/letsencrypt/live/zulip.example.com/privkey.pem
+     # This file can be found in ~zulip/deployments/current/puppet/zulip/files/nginx/dhparam.pem
      SSLOpenSSLConfCmd DHParameters "/etc/nginx/dhparam.pem"
      SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
      SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
@@ -461,24 +472,38 @@ make the following changes in two configuration files.
    </VirtualHost>
    ```
 
+   Don't forget to update `ServerName`, `RewriteRule`, `ProxyPass`,
+   `SSLCertificateFile`, and `SSLCertificateKeyFile` as are
+   appropriate for your deployment.
+
 ### HAProxy configuration
 
-If you want to use HAProxy with Zulip, this `backend` config is a good
+Below is a working example of a HAProxy configuration. It assumes that
-place to start.
+your Zulip server sits at `https://10.10.10.10:443`see
+[above](#configuring-zulip-to-allow-http) to switch to HTTP.
 
-```text
+1. Follow the instructions to [configure Zulip to trust
-backend zulip
+   proxies](#configuring-zulip-to-trust-proxies).
+
+1. Configure HAProxy. The below is a minimal `frontend` and `backend`
+   configuration:
+
+   ```text
+   frontend zulip
        mode http
-    balance leastconn
+       bind *:80
-    reqadd X-Forwarded-Proto:\ https
+       bind *:443 ssl crt /etc/ssl/private/zulip-combined.crt
-    server zulip 10.10.10.10:80 check
+       http-request redirect scheme https code 301 unless { ssl_fc }
-```
+       default_backend zulip
 
-Since this configuration uses the `http` mode, you will also need to
+   backend zulip
-[configure Zulip to allow HTTP](#configuring-zulip-to-allow-http) as
+       mode http
-described above. Additionally, you will need to [add the the HAProxy server IP
+       timeout server 20m
-address as a trusted load balancer](#configuring-zulip-to-trust-proxies)
+       server zulip 10.10.10.10:443 check ssl ca-file /etc/ssl/certs/ca-certificates.crt
-to have Zulip respect the addresses in `X-Forwarded-For` headers.
+   ```
+
+   Don't forget to update `bind *:443 ssl crt` and `server` as is
+   appropriate for your deployment.
 
 ### Other proxies
 
@@ -494,7 +519,13 @@ things you need to be careful about when configuring it:
    has the actual IP addresses of clients, not the IP address of the
    proxy server.
 
-2. Ensure your proxy doesn't interfere with Zulip's use of
+1. Configure your proxy to pass along the `Host:` header as was sent
+   from the client, not the internal hostname as seen by the proxy.
+   If this is not possible, you can set `USE_X_FORWARDED_HOST = True`
+   in `/etc/zulip/settings.py`, and pass the client's `Host` header to
+   Zulip in an `X-Forwarded-Host` header.
+
+1. Ensure your proxy doesn't interfere with Zulip's use of
    long-polling for real-time push from the server to your users'
    browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
    does this.
@@ -507,7 +538,7 @@ things you need to be careful about when configuring it:
    - `proxy_buffering off`. If you don't do this, your `nginx` proxy may
      return occasional 502 errors to clients using Zulip's events API.
 
-3. The other tricky failure mode we've seen with `nginx` reverse
+1. The other tricky failure mode we've seen with `nginx` reverse
    proxies is that they can load-balance between the IPv4 and IPv6
    addresses for a given hostname. This can result in mysterious errors
    that can be quite difficult to debug. Be sure to declare your