17 KiB

Raw Blame History

Mobile push notification service

Zulip's iOS and Android mobile apps support receiving push notifications from Zulip servers to let users know when new messages have arrived. This is an important feature for having a great mobile app experience.

To set up mobile push notifications, you will need to register your Zulip server with the Zulip mobile push notification service. This service will forward push notifications generated by your server to users' mobile apps.

You can enable the mobile push notification service for your Zulip server as follows:

Make sure your server has outgoing HTTPS access to the public Internet. If that is restricted by a proxy, you will need to configure Zulip to use your outgoing HTTP proxy first.
Decide whether to share usage statistics with the Zulip team.

By default, Zulip installations using the Mobile Push Notifications Service submit additional usage statistics that help Zulip's maintainers allocate resources towards supporting self-hosted installations (details). You can disable submitting usage statistics now or at any time by setting SUBMIT_USAGE_STATISTICS=False in /etc/zulip/settings.py.

Note that all systems using the service upload basic metadata about the organizations hosted by the installation.
Uncomment the PUSH_NOTIFICATION_BOUNCER_URL = 'https://push.zulipchat.com' line in your /etc/zulip/settings.py file (i.e., remove the # at the start of the line), and restart your Zulip server.
Run the registration command. If you installed Zulip directly on the server (without Docker), run as root:
```
su zulip -c '/home/zulip/deployments/current/manage.py register_server'
```
Or if you're using Docker, run:
```
docker exec -it -u zulip <container_name> /home/zulip/deployments/current/manage.py register_server
```
This command will print the registration data it would send to the mobile push notifications service, ask you to accept the terms of service, and if you accept, register your server. If you have trouble, contact Zulip support with the output of this command.
If you or your users have already set up the Zulip mobile app, you'll each need to log out of the mobile app, and log back in again in order to start getting push notifications.

Congratulations! You've successfully set up the service. You can now test mobile push notifications by following these instructions.

Plan management for a Zulip organization

On a self-hosted Zulip server running Zulip 8.0+, organization owners and billing administrators can conveniently access plan management from the Zulip app. See help center documentation for detailed instructions.

You can add billing administrators using the change_user_role management command, passing the organization's string_id, and the email address of the Zulip user who should be added as a billing administrator.

/home/zulip/deployments/current/manage.py change_user_role -r '' username@example.com is_billing_admin

You can remove a user's billing administrator permissions with the --revoke option:

/home/zulip/deployments/current/manage.py change_user_role --revoke -r '' username@example.com is_billing_admin

Plan management for an entire Zulip server

Servers running Zulip releases older than Zulip 8.0 can start the plan management log in process at https://selfhosting.zulip.com/serverlogin/. This option is also available for Zulip 8.0+ servers, and makes it possible to use a single plan for multiple organizations on one installation. See help center documentation for detailed log in instructions.

You will use your server's zulip_org_id and zulip_org_key as the username and password to access plan management. You can obtain these from /etc/zulip/zulip-secrets.conf on your Zulip server, or via the following commands:

/home/zulip/deployments/current/scripts/get-django-setting ZULIP_ORG_ID
/home/zulip/deployments/current/scripts/get-django-setting ZULIP_ORG_KEY

Why a push notification service is necessary

Both Google's and Apple's push notification services have a security model that does not support mutually untrusted self-hosted servers sending push notifications to the same app. In particular, when an app is published to their respective app stores, one must compile into the app a secret corresponding to the server that will be able to publish push notifications for the app. This means that it is impossible for a single app in their stores to receive push notifications from multiple, mutually untrusted, servers.

Zulip's solution to this problem is to provide a central push notification forwarding service, which allows registered Zulip servers to send push notifications to the Zulip app indirectly (through the forwarding service).

Security and privacy

Use of the push notification bouncer is subject to the Zulip Cloud Terms of Service, Privacy Policy and Rules of Use. By using push notifications, you agree to these terms.

We've designed this push notification bouncer service with security and privacy in mind:

A central design goal of the Push Notification Service is to avoid any message content being stored or logged by the service, even in error cases.
The Push Notification Service only stores the necessary metadata for delivering the notifications to the appropriate devices and otherwise operating the service:
- The APNS/FCM tokens needed to securely send mobile push notifications to iOS and Android devices, one per device registered to be notified by your Zulip server.
- User ID numbers generated by your Zulip server, needed to route a given notification to the appropriate set of mobile devices. These user ID numbers are opaque to the Push Notification Service and Kandra Labs.
- Basic organization metadata, optional usage statistics, and aggregate statistics about how many push notifications are sent by each customer.
The Push Notification Service receives (but does not store) the contents of individual mobile push notifications:
- The numeric message ID generated by your Zulip server.
- Metadata on the message's sender (name and avatar URL).
- Metadata on the message's recipient (stream name + ID, topic, direct message recipients, etc.).
- A timestamp.
- The message's content.
There's a PUSH_NOTIFICATION_REDACT_CONTENT setting available to disable any message content being sent via the push notification bouncer (i.e. message content will be replaced with ***REDACTED***). Note that this setting makes push notifications significantly less usable.

We plan to replace that setting with end-to-end encryption which would eliminate that usability tradeoff and additionally allow us to not have any access to the other details mentioned in this section.
All of the network requests (both from Zulip servers to the Push Notification Service and from the Push Notification Service to the relevant Google and Apple services) are encrypted over the wire with SSL/TLS.
The code for the push notification forwarding service is 100% open source and available as part of the Zulip server project on GitHub (specifically, here).
The push notification forwarding servers are professionally managed by a small team of security-sensitive engineers.

If you have any questions about the security model, contact Zulip support.

Uploading basic metadata

All Zulip installations running Zulip 8.0 or greater that are registered for the Mobile Push Notifications Service regularly upload to the service basic metadata about the organizations hosted by the installation. (Older Zulip servers upload these metadata only if uploading usage statistics is enabled).

Uploaded metadata consists of, for each organization hosted by the installation:

A subset of the basic metadata returned by the unauthenticated GET /server_settings API endpoint.

The purpose of that API endpoint is to serve the minimal data needed by the Zulip mobile apps in order to:
- Verify that a given URL is indeed a valid Zulip server URL
- Present a correct login form, offering only the supported features and authentication methods for that organization and Zulip server version.
Most of the metadata it returns is necessarily displayed to anyone with network access to the Zulip server on the login and signup pages for your Zulip organization as well.

(Some fields returned by this endpoint, like the organization icon and description, are not included in uploaded metadata.)
The organization type and creation date.
The number of user accounts with each role.

Our use of uploaded metadata is governed by the same Terms of Service and Privacy Policy that covers the Mobile Push Notifications Service itself.

Uploading usage statistics

By default, Zulip installations that register for the Mobile Push Notifications Service upload the following usage statistics. You can disable these uploads any time by setting SUBMIT_USAGE_STATISTICS=False in /etc/zulip/settings.py.

Totals for messages sent and read with subtotals for various combinations of clients and integrations.
Totals for active users under a few definitions (1day, 7day, 15day) and related statistics.

Some of the graphs on your server's usage statistics page can be generated from these statistics.

When enabled, usage statistics are submitted via an hourly cron job. If you'd like to access plan management immediately after enabling SUBMIT_USAGE_STATISTICS=True on a pre-8.0 Zulip server, you can run the analytics job manually via:

/home/zulip/deployments/current/manage.py update_analytics_counts

Our use of uploaded usage statistics is governed by the same Terms of Service and Privacy Policy that covers the Mobile Push Notifications Service itself.

Rate limits

The Mobile Push Notifications Service API has a very high default rate limit of 1000 requests per minute. A Zulip server makes requests to this API every time it sends a push notification, which is fairly frequent, but we believe it to be unlikely that a self-hosted installation will hit this limit.

This limit is primarily intended to protect the service against DoS attacks (intentional or otherwise). If you hit this limit or you anticipate that your server will require sending more push notifications than the limit permits, please contact support.

Updating your server's registration

Your server's registration includes the server's hostname and contact email address (from EXTERNAL_HOST and ZULIP_ADMINISTRATOR in /etc/zulip/settings.py, aka the --hostname and --email options in the installer). You can update your server's registration data by running manage.py register_server again.

If you'd like to rotate your server's API key for this service (zulip_org_key), you need to use manage.py register_server --rotate-key option; it will automatically generate a new zulip_org_key and store that new key in /etc/zulip/zulip-secrets.conf.

Deactivating your server's registration

If you are deleting your Zulip server or otherwise no longer want to use the Mobile Push Notification Service, you can deactivate your server's registration.

Cancel any paid plans associated with your server.

Run the deregistration command. If you installed Zulip directly on the server (without Docker), run as root:

su zulip -c '/home/zulip/deployments/current/manage.py register_server --deactivate'

Or if you're using Docker, run:

docker exec -it -u zulip <container_name> /home/zulip/deployments/current/manage.py register_server --deactivate

Comment out the PUSH_NOTIFICATION_BOUNCER_URL = 'https://push.zulipchat.com' line in your /etc/zulip/settings.py file (i.e., add # at the start of the line), and restart your Zulip server.

If you ever need to reactivate your server's registration, contact Zulip support.

Pausing use of the Mobile Push Notification Service

You can temporarily stop using the Mobile Push Notification Service. Comment out the PUSH_NOTIFICATION_BOUNCER_URL = 'https://push.zulipchat.com' line in your /etc/zulip/settings.py file (i.e., add # at the start of the line), and restart your Zulip server. This approach makes it easy to start using the service again by uncommenting the same line.

Sending push notifications directly from your server

This section documents an alternative way to send push notifications that does not involve using the Mobile Push Notification Service at the cost of needing to compile and distribute modified versions of the Zulip mobile apps.

We don't recommend this path -- patching and shipping a production mobile app can take dozens of hours to set up even for an experienced developer, and even more time to maintain. And it doesn't provide material privacy benefits -- your organization's push notification data would still go through Apple/Google's servers, just not Kandra Labs'. But in the interest of transparency, we document in this section roughly what's involved in doing so.

As discussed above, it is impossible for a single app in the Google or Apple store to receive push notifications from multiple, mutually untrusted, servers. The Mobile Push Notification Service is one of the possible solutions to this problem.

The other possible solution is for an individual Zulip server's administrators to build and distribute their own copy of the Zulip mobile apps, hardcoding a key that they possess. This solution is possible with Zulip, but it requires the server administrators to publish their own copies of the Zulip mobile apps. There's nothing the Zulip team can do to eliminate this onerous requirement.

The main work is thus distributing your own copies of the Zulip mobile apps configured to use APNS/FCM keys that you generate. This is not for the faint of heart! If you haven't done this before, be warned that one can easily spend hundreds of dollars (on things like a DUNS number registration) and a week struggling through the hoops Apple requires to build and distribute an app through the Apple app store, even if you're making no code modifications to an app already present in the store (as would be the case here). The Zulip mobile app also gets frequent updates that you will have to either forgo or republish to the app stores yourself.

If you've done that work, the Zulip server configuration for sending push notifications through the new app is quite straightforward:

Create an FCM push notifications key in the Google Developer console and set android_gcm_api_key in /etc/zulip/zulip-secrets.conf to that key.
In Apple's developer console, register a token or certificate for sending push notifications. Then in /etc/zulip/settings.py, set APNS_SANDBOX=False, and:
- If using APNs certificate-based authentication, set APNS_CERT_FILE to the path of your APNs certificate file.
- If using APNs token-based authentication, set APNS_TOKEN_KEY_FILE to the path of your APNs token key file, APNS_TOKEN_KEY_ID to the corresponding 10-character key ID, and APNS_TEAM_ID to your 10-character Apple team ID.
Restart the Zulip server.

17 KiB Raw Blame History