nginx: Make uwsgi timeout shorter than nginx-to-uwsgi timeout.

The nginx-to-uwsig-timeout defaults to 60s, which is exactly the same
as the current "harakiri" timeout configured in uwsgi (which limits
the length a request can run before the worker is terminated).  This
causes a race, where if nginx hits its 60s before uwsgi, then we
return a 504; otherwise, we get a 502.

Make the nginx-to-uwsgi timeout explicit, and shorten the "harakiri"
timeout to be explicitly less than that.  Document the 60s timeout,
which all outer reverse proxies must be set to _longer than_ in order
to have proper "onion" timeouts.

docs/production/reverse-proxies.md

@@ -255,13 +255,13 @@ things you need to be careful about when configuring it:
    browsers. This [nginx code snippet][nginx-proxy-longpolling-config]
    does this.
 
-   The key configuration options are, for the `/json/events` and
+   The key configuration options are:
-   `/api/1/events` endpoints:
 
-   - `proxy_read_timeout 1200;`. It's critical that this be
+   - `proxy_read_timeout 1200;`. It's critical that this be significantly above
-     significantly above 60s, but the precise value isn't important.
+     60s, but the precise value isn't important. This is most important for the
-   - `proxy_buffering off`. If you don't do this, your `nginx` proxy may
+     events API, but must be applied to all endpoints.
-     return occasional 502 errors to clients using Zulip's events API.
+   - `proxy_buffering off`. If you don't do this, your `nginx` proxy may return
+     occasional 502 errors to clients using Zulip's events API.
 
 1. The other tricky failure mode we've seen with `nginx` reverse
    proxies is that they can load-balance between the IPv4 and IPv6

puppet/zulip/files/nginx/uwsgi_params

@@ -18,4 +18,8 @@ uwsgi_param HTTP_X_FORWARDED_PROTO $trusted_x_forwarded_proto;
 uwsgi_param HTTP_X_FORWARDED_SSL "";
 uwsgi_param HTTP_X_PROXY_MISCONFIGURATION $x_proxy_misconfiguration;
 
+# This value is the default, and is provided for explicitness; it must
+# be longer than the configured 55s "harakiri" timeout in uwsgi
+uwsgi_read_timeout 60s;
+
 uwsgi_pass django;

puppet/zulip/templates/uwsgi.ini.template.erb

@@ -47,7 +47,7 @@ auto-procname=true
 procname-prefix-spaced=zulip-django
 
 # Longest response allowed, in seconds, before killing the worker
-harakiri=60
+harakiri=55