puppet: Set proxy environment variables.

These are respected by `urllib`, and thus also `requests`.  We set
`HTTP_proxy`, not `HTTP_PROXY`, because the latter is ignored in
situations which might be running under CGI -- in such cases it may be
coming from the `Proxy:` header in the request.

docs/production/deployment.md

@@ -183,6 +183,32 @@ behind reverse proxies.
 
 [using-http]: ../production/deployment.html#configuring-zulip-to-allow-http
 
+## Using an outgoing HTTP proxy
+
+Zulip supports routing all of its outgoing HTTP and HTTPS traffic
+through an HTTP `CONNECT` proxy, such as [smokescreen][smokescreen];
+this includes outgoing webhooks, image and website previews, and
+mobile push notifications.  You may wish to enable this feature to
+provide a consistent egress point, or enforce access control on URLs.
+
+To enable an outgoing HTTP proxy:
+
+1. Add the following block to `/etc/zulip/zulip.conf`, substituting in
+   your proxy's hostname/IP and port:
+
+    ```
+    [http_proxy]
+    host = 192.168.0.1
+    port = 4750
+    ```
+
+1. As root, run
+   `/home/zulip/deployments/current/scripts/zulip-puppet-apply`.  This
+   will reconfigure services to use the outgoing proxy, and restart
+   Zulip.
+
+[smokescreen]: https://github.com/stripe/smokescreen
+
 ## Putting the Zulip application behind a reverse proxy
 
 Zulip is designed to support being run behind a reverse proxy server.
@@ -560,3 +586,16 @@ configured to consume; defaults to 1/8th of the total server memory.
 
 Comma-separated list of IP addresses or netmasks of external
 load balancers whose `X-Forwarded-For` should be respected.
+
+
+
+### `[http_proxy]`
+
+#### `host`
+
+The hostname or IP address of an [outgoing HTTP `CONNECT`
+proxy](#using-an-outgoing-http-proxy).
+
+#### `port`
+
+The TCP port of the HTTP `CONNECT` proxy on the host specified above.

docs/production/mobile-push-notifications.md

@@ -11,12 +11,19 @@ will need to register your Zulip server with the Zulip mobile push
 notification service.  This service will forward push notifications
 generated by your server to the Zulip mobile app automatically.
 
+
+
 ## How to sign up
 
 Starting with Zulip 1.6 for both Android and iOS, Zulip servers
 support forwarding push notifications to a central push notification
-forwarding service.  You can enable this for your Zulip server as
-follows:
+forwarding service.  Accessing this service requires outgoing HTTPS
+access to the public Internet; if that is restricted by a proxy, you
+will need to [configure Zulip to use your outgoing HTTP
+proxy](../production/deployment.html#using-an-outgoing-http-proxy)
+first.
+
+You can enable this for your Zulip server as follows:
 
 1. Uncomment the `PUSH_NOTIFICATION_BOUNCER_URL =
    'https://push.zulipchat.com'` line in your `/etc/zulip/settings.py`

docs/production/requirements.md

@@ -77,9 +77,12 @@ on hardware requirements for larger organizations.
 * Incoming port 25 if you plan to enable Zulip's [incoming email
   integration](../production/email-gateway.md).
 * Outgoing HTTP(S) access (ports 80 and 443) to the public Internet so
-  that Zulip can properly manage image previews and embeds.  Outgoing
-  Internet access is not required if you [disable those
-  features](https://zulip.com/help/allow-image-link-previews).
+  that Zulip can properly manage image and website previews and mobile
+  push notifications.  Outgoing Internet access is not required if you
+  [disable those
+  features](https://zulip.com/help/allow-image-link-previews), or
+  configure an [existing outgoing HTTP
+  proxy](../production/deployment.html#using-an-outgoing-http-proxy).
 * Outgoing SMTP access (usually port 587) to your [SMTP
   server](../production/email.md) so that Zulip can send emails.
 * A domain name (e.g. `zulip.example.com`) that your users will use to

puppet/zulip/manifests/app_frontend_base.pp

@@ -88,6 +88,13 @@ class zulip::app_frontend_base {
     $uwsgi_default_processes = 4
   }
   $tornado_ports = $zulip::tornado_sharding::tornado_ports
+  $proxy_host = zulipconf('http_proxy', 'host', '')
+  $proxy_port = zulipconf('http_proxy', 'port', '')
+  if $proxy_host != '' and $proxy_port != '' {
+    $proxy = "http://${proxy_host}:${proxy_port}"
+  } else {
+    $proxy = ''
+  }
   file { "${zulip::common::supervisor_conf_dir}/zulip.conf":
     ensure  => file,
     require => [Package[supervisor], Exec['stage_updated_sharding']],

puppet/zulip/templates/supervisor/zulip.conf.template.erb

@@ -9,6 +9,7 @@
 
 [program:zulip-django]
 command=nice -n5 /home/zulip/deployments/current/zulip-current-venv/bin/uwsgi --ini /etc/zulip/uwsgi.ini
+environment=HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=100                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -27,7 +28,7 @@ directory=/home/zulip/deployments/current/
 [program:zulip-tornado]
 command=/home/zulip/deployments/current/manage.py runtornado 127.0.0.1:98%(process_num)02d
 process_name=zulip-tornado-port-98%(process_num)02d
-environment=PYTHONUNBUFFERED=1
+environment=PYTHONUNBUFFERED=1,HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=200                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -43,7 +44,7 @@ numprocs=<%= @tornado_ports.length %>
 <% else -%>
 [program:zulip-tornado]
 command=/home/zulip/deployments/current/manage.py runtornado 127.0.0.1:9800
-environment=PYTHONUNBUFFERED=1
+environment=PYTHONUNBUFFERED=1,HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=200                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -61,6 +62,7 @@ directory=/home/zulip/deployments/current/
 <% @queues.each do |queue| -%>
 [program:zulip_events_<%= queue %>]
 command=nice -n10 /home/zulip/deployments/current/manage.py process_queue --queue_name=<%= queue %>
+environment=HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=300                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -76,6 +78,7 @@ directory=/home/zulip/deployments/current/
 <% else %>
 [program:zulip_events]
 command=nice -n10 /home/zulip/deployments/current/manage.py process_queue --multi_threaded <%= @queues.join(' ') %>
+environment=HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=300                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -93,6 +96,7 @@ killasgroup=true              ; Without this, we leak processes every restart
 
 [program:zulip_deliver_enqueued_emails]
 command=nice -n15 /home/zulip/deployments/current/manage.py deliver_email
+environment=HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=350                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)
@@ -107,6 +111,7 @@ directory=/home/zulip/deployments/current/
 
 [program:zulip_deliver_scheduled_messages]
 command=nice -n15 /home/zulip/deployments/current/manage.py deliver_scheduled_messages
+environment=HTTP_proxy="<%= @proxy %>",HTTPS_proxy="<%= @proxy %>"
 priority=350                   ; the relative start priority (default 999)
 autostart=true                 ; start at supervisord start (default: true)
 autorestart=true               ; whether/when to restart (default: unexpected)

scripts/lib/email-mirror-postfix

@@ -114,6 +114,10 @@ def send_email_mirror(
         # work, and connections to 127.0.0.1 are secure without SSL.
         verify_ssl = False
 
+    # Because this script is run from postfix, it does not have any
+    # http proxy environment variables set which might interfere with
+    # access to localhost.
+
     context = None
     if not verify_ssl:
         context = ssl.create_default_context()