Commit Graph

183 Commits

Author SHA1 Message Date
Tim Abbott da8f4bc0e9 push notifications: Add support for removing GCM push notifications.
This uses the recently introduced active_mobile_push_notification
flag; messages that have had a mobile push notification sent will have
a removal push notification sent as soon as they are marked as read.

Note that this feature is behind a setting,
SEND_REMOVE_PUSH_NOTIFICATIONS, since the notification format is not
supported by the mobile apps yet, and we want to give a grace period
before we start sending notifications that appear as (null) to
clients.  But the tracking logic to maintain the set of message IDs
with an active push notification runs unconditionally.

This is designed with at-least-once semantics; so mobile clients need
to handle the possibility that they receive duplicat requests to
remove a push notification.

We reuse the existing missedmessage_mobile_notifications queue
processor for the work, to avoid materially impacting the latency of
marking messages as read.

Fixes #7459, though we'll need to open a follow-up issue for
using these data on iOS.
2018-08-10 13:58:39 -07:00
Tim Abbott e9f4d9db2b push_notifications: Fix interface for handle_remove_push_notification.
This really should just accept a message ID.
2018-08-01 16:36:42 -07:00
Kunal Gupta bc43eefbfb notifications: Add function for cancelling GCM notifications.
This adds a new function called handle_remove_push_notification in
zerver/lib/push_notifications.py which requires user_profile id and
the message id which has to be removed in the function.

For now, the function only supports GCM (and is mostly there for
prototyping).

The payload which is being delivered needs to contain the narrow
information and the content of the message.
2018-08-01 15:59:04 -07:00
Tim Abbott 5f8d193bb7 notifications: Include realm_uri in push notifications.
This should make it much simpler for the mobile apps to line up the
data from server_settings against the data in the notifications.

Addresses part of #10094.
2018-08-01 15:46:15 -07:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Tim Abbott 80a331385e push_notifications: Fix missing mypy annotation. 2018-05-21 11:39:58 -07:00
Tim Abbott cec7686f3d push notifications: Clean up unregistered/bad APNS tokens.
We've had this sort of logic for GCM for a long time; it's worth
adding for APNS as well.

Writing this is a bit of a reminder that I'm not a fan of how our unit
tests for push notifications work.
2018-05-21 11:30:56 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Rishi Gupta 42a99e8c1d zilencer: Rename remote_server_unregister_push. 2018-05-03 12:50:29 -07:00
Rishi Gupta 76650f5930 zilencer: Rename remote_server_register_push to register_remote_push_device. 2018-05-03 12:50:29 -07:00
Rohitt Vashishtha 2f6da2661f push_notifications: Format blockquotes properly in text_output.
New output is of the format:

Hamlet said:
> Polonius said:
> > This is the.
> > Second layer of nesting.
> First layer of nesting.

Fixes #9251.
2018-05-02 08:57:17 -07:00
Tim Abbott 62c0d27d1e push_notifications: Handle errors with token reuse.
If an Android token has been used to connect a given device with
multiple Zulip servers, and then is expired, we would 500 in trying to
remove the Zulip-side registration for it, because the code assumed
there was only one such registration.  If a token is no longer valid,
it's invalid for all servers, so the correct fix is to just remove them all.
2018-03-23 11:02:55 -07:00
Umair Khan 5086ba8568 push_notification: Update comment for unregister. 2018-03-01 22:14:32 -08:00
Umair Khan 8a8b0dd87d code-search: Add comments to help grepping. 2018-03-01 22:14:32 -08:00
Greg Price 693a9a5e70 push notifs: For group PMs, identify the users in the group. 2018-02-16 16:06:03 -08:00
Greg Price ecbc72b857 push notifs: Add a diagnostic in API of whether push notifs enabled.
When the answer is False, this will allow the mobile app to show a
warning that push notifications will not work and the server admin
should set them up.

Based partly on Kunal's PR #7810.  Provides the necessary backend API
for zulip/zulip-mobile#1507.
2018-02-12 14:34:59 -08:00
Greg Price 8be2dfa81c APNs: Fix a case I broke while working out tests.
I got distracted, came back later to a successful test run in my
terminal, and thought I remembered finishing the change and just
kicking off a final test run to check.

In fact, there was an `assert False` right in the normal case for
production, and I just hadn't finished a test for that path. (m.-)
Definitely the most grateful I've been for our coverage checks,
which highlighted this for me.

Remove the `assert False`, and also finish writing the test it was
there to help me write.  Those lines are covered now.
2018-02-09 18:28:16 -08:00
Greg Price 7f6a1714aa APNs: Don't try to send anything when not configured. 2018-02-09 17:16:21 -08:00
Greg Price 776af2e248 push notifs: Simplify content-truncation code slightly.
This should be a pure refactor.
2017-12-18 15:32:56 -08:00
Greg Price 014900c2e5 push notifs: Include same stream and sender info on GCM as on APNs.
This takes the information added to APNs payloads by #7080, and adds
to our GCM payloads the parts that it didn't already have.
2017-12-18 15:32:56 -08:00
Greg Price e077d6fc74 push notifs: Factor out common payload fields for APNs and GCM.
This should be a pure refactor, producing exactly the same payloads.
2017-12-18 15:32:56 -08:00
Greg Price 8a432c5134 APNs: Update comment on versions of our APNs payload format. 2017-12-11 20:55:20 -08:00
kunall17 98943a8333 Include data of stream/private sender details in apns push notifications 2017-12-11 20:51:09 -08:00
Tim Abbott 6bc1a682de notifications: Separate push and email notifications checks.
This is an early step in a larger refactor to properly decouple the
email and push notification code paths.
2017-11-28 17:51:18 -08:00
rht ee546a33a3 zerver/lib: Use python 3 syntax for typing.
Edited by tabbott to improve various line-wrapping decisions.
2017-11-28 17:15:14 -08:00
rht 229a8b38c0 zerver/lib: Use Python 3 syntax for typing for several files.
This adds a number of annotations that had been missed in previous
passes.
2017-11-28 17:02:24 -08:00
Tim Abbott 982bd9c38d Fix push notifications for soft-deactivated users.
Previously, these push notification events were being generated, but
then ignored in handle_push_notification because there was no
user_message object.
2017-11-09 15:52:29 -08:00
Tim Abbott c99d20db63 push_notifications: Shrink scope of try/except block.
There's no reason this needed to indent the entire function.
2017-11-09 15:30:21 -08:00
Tim Abbott 64b4d83038 push_notifications: Improve debugging of lookup problems.
We're having this error being thrown often, and don't know why.
2017-11-09 15:20:24 -08:00
rht 19bd335cbb Change urllib import to be Python 3-specific. 2017-11-07 10:46:42 -08:00
Steve Howell 8302689789 Extract Message.is_stream_message().
This sets us up to denormalize on to Message some notion of
whether a message is for a stream (without having to hit the
Recipient table).
2017-10-28 17:57:39 -07:00
Tim Abbott fa55d7ed33 retry_event: Remove requirement of failed_tries being already there.
This fixes a bug where retries in the signups queue threw an exception.
2017-10-27 18:15:53 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Maarten Rijke 841f02934d push_notifications: Add setting to redact content.
This commit adds a setting that toggles redacting content of push
notifications and replacing it with "***REDACTED***".
2017-10-19 18:16:06 -07:00
Tim Abbott e98ca0714b notifications: Simplify how triggers are passed around.
This removes the utterly unnecessary `triggers` dict (which always was
a dict with exactly one value True) in favor of a single field,
'trigger'.

Inspired by Kunal Gupta's work in #6659.
2017-10-18 21:42:05 -07:00
Tim Abbott b5c107ed27 push_notifications: Remove unnecessary check for no devices.
This should have been checked by the caller anyway.
2017-10-13 17:30:20 -07:00
Tim Abbott 27a450b58d push_notifications: Improve error message for GCM sending issues.
This addresses one of the sources of confusion in #6993.
2017-10-13 17:30:11 -07:00
Tim Abbott 298c59f7fd push_notifications: Fix error message for unregistered bouncer.
Previously, we were just returning a JSON error to the client, when it
was a server problem.

Fixes #6639.
2017-10-11 19:09:24 -07:00
Harshit Bansal 5a6584890d push_notifications: Start using `get_mobile_push_content()` function. 2017-10-06 16:47:25 -07:00
Harshit Bansal 28628eeaeb push_notifications: Add `truncate_content()` function.
This function truncates the textual content at correct length.
(It will be updated later to handle corner cases of unicode
combining characters and tags when we start supporting them.)
2017-10-06 16:44:19 -07:00
Harshit Bansal b5a1aacfb3 push_notifications: Add `get_mobile_push_content()` function.
Given the rendered content of a message, this function strips
all the markup replacing emojis with their corresponding unicode
representation.
2017-10-06 16:44:18 -07:00
kunall17 f091823fc8 push_notifications: Set APNS badge as 0 for now.
We don't yet have support for tracking the number of open APNS
notifications server-side, so setting this to 0 will at least avoid
confusing users.
2017-10-03 17:07:49 -07:00
derAnfaenger d1afab7199 Replace deprecated Logging.warn calls with Logging.warning. 2017-10-02 11:11:42 +02:00
Greg Price 7b8f725707 APNs: Accept 1.6-format payloads in bouncer.
This is just enough of a quick fix to work with a stock Zulip 1.6
server.  We should really also make this robust to arbitrary input
from the remote Zulip server, even though it'll be a little tedious.
2017-09-28 10:01:16 -07:00
rht f43e54d352 zerver/lib: Remove absolute_import. 2017-09-27 10:00:39 -07:00
kunall17 1436d558a3 APNs: Add message body to notification payload.
(Edited by greg to leave the badge logic as is for now.)
2017-09-18 16:37:02 -07:00
Tim Abbott a33c98f23e push_notifications: Require the message.triggers fields to exist.
The code was already assuming it implicitly, and making it explicit
improves the readability.
2017-09-16 03:14:56 -07:00
Tim Abbott 5722237f59 push: Rename received_pm to private_message.
This is a clearer name for this now more broadly used interface.
2017-09-14 05:41:37 -07:00
Sarah 97571a203d push: Add new formatting for stream message push and add tests.
This should make the push notifications for messages to streams with
the new stream push notifications setting enabled make sense.
2017-09-14 05:41:37 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Greg Price a4bcf1a64b APNs: Handle HTTP connection errors, and retry.
Should help with #6321 as at least a band-aid.
2017-08-29 15:27:41 -07:00
Tim Abbott 00036ac8db push_notifications: Fix mypy error. 2017-08-26 14:33:43 -07:00
Greg Price 613d093d7d push notifs: Implement APNs with new API.
And it works!

A couple of things still to do:

 * When a device token is no longer active, we'll get HTTP status 410.
   We should then remove the token from the database so we don't keep
   trying to push to it.  This is fairly urgent.

 * The library we're using has a nice asynchronous API, but this
   version doesn't use it.  This is OK now, but async will be
   essential at scale.
2017-08-26 14:16:05 -07:00
Greg Price 35db1b2f11 push notifs: Organize imports. 2017-08-26 14:16:05 -07:00
Greg Price ba673526ab push notifs: Organize the code a bit.
This commit only reorders the code in the file, without touching the
insides of any function, and adds some heading comments.
2017-08-26 14:16:05 -07:00
Greg Price d02101a401 APNs: Rip out the existing, broken implementation.
This code empirically doesn't work.  It's not entirely clear why, even
having done quite a bit of debugging; partly because the code is quite
convoluted, and because it shows the symptoms of people making changes
over time without really understanding how it was supposed to work.

Moreover, this code targets an old version of the APNs provider API.
Apple deprecated that in 2015, in favor of a shiny new one which uses
HTTP/2 to meet the same needs for concurrency and scale that the old
one had to do a bunch of ad-hoc protocol design for.

So, rip this code out.  We'll build a pathway to the new API from
scratch; it's not that complicated.
2017-08-26 14:16:05 -07:00
Greg Price 3bceeec89f push notifs: Add logging on creating device tokens.
We'd been getting errors from APNs that appeared to say that the
device tokens we were trying to send to were invalid.  It turned out
that the device tokens didn't match the "topic" (i.e. app ID) we were
sending, which was because the topic was wrong, which was because we
were using the wrong SSL cert.  But for a while we thought it might be
that we were somehow messing up the device tokens we put into the
database.  This logging helped us work out that wasn't the issue, and
would have helped our debugging sooner.
2017-08-26 14:16:05 -07:00
Umair Khan a5c05f9812 push_notification: Retry event in case of error.
Fixes #5301
2017-08-22 11:16:48 -07:00
Tim Abbott d6e0960ca2 push_notifications: Fix mypy annotation logic around push tokens.
I'm not 100% confident this is long-term correct, but at least it's
consistent.
2017-07-07 10:54:37 -07:00
kunall17 d8310724b6 Use absolute URLs in GCM push notifications.
This will eventually allow us to simplify the logic required for the
mobile apps processing GCM notifications.
2017-06-15 03:56:59 -07:00
Steve Howell e3c6c22269 Optimize push_notifications.response_listener().
We were needlessly getting a UserProfile object out of cache,
when we only needed the user_id.
2017-06-10 10:43:57 -07:00
Aditya Bansal 8680c87f56 pep8: Add compliance with rule E261 zerver/lib/push_notifications.py. 2017-05-31 17:07:15 -07:00
Umair Khan bc15085098 gcm: Increase retries to 10 while pushing. 2017-05-17 12:14:32 -07:00
Umair Khan 38ecc35cd9 push_notifications: Catch IOError while pushing to GCM. 2017-05-17 12:14:32 -07:00
Umair Khan 7e8f4ca4e8 push_notifications: Include GCM in end-to-end test. 2017-05-17 08:09:19 -07:00
Tim Abbott 0b2388bda9 push_notifications: Remove DeviceTokenType logic.
The syntax wasn't valid on Python 3.5, and the new code is somewhat
more readable anyway.
2017-05-16 12:26:55 -07:00
Tim Abbott 9d63a5ab3a push_notification: Delete obsolete DBX_IOS_APP_ID code.
I'm not sure that this was ever actually used, but it's definitely
just clutter for Zulip today.
2017-05-16 12:26:55 -07:00
Umair Khan 286f9a40e7 push_notifications: Bring file to 100% coverage. 2017-05-16 12:26:55 -07:00
Umair Khan fc0b9358e7 handle_push_notification: Remove the if condition.
'if apple_devices or android_devices' doesn't servce any purpose.
2017-05-16 12:26:55 -07:00
Umair Khan ab411ab7b3 push_notifications: Delete remote server tokens in APNs response. 2017-05-16 12:26:55 -07:00
Umair Khan 33332539df send_android_push_notification: Delete correct tokens.
Now this function will delete tokens from RemotePushDeviceToken if it
is running on notification bouncer or PushDeviceToken if it is running
on a server which doesn't use notification bouncer.
2017-05-16 12:26:55 -07:00
Umair Khan 4a864c7515 push_notification: Send data to notification bouncer. 2017-05-16 12:26:55 -07:00
Umair Khan 329b377e6a push_notification: Add uses_notification_bouncer().
This function abstracts the logic to ascertain if we are using
notification bouncer service or not. Makes our code more
maintainable.
2017-05-11 12:04:16 -07:00
Umair Khan 112a67097b push_notifications: Don't get alert message.
Now we set the alert message in the payload functions for both GCM and APNs so
no need to get it here.
2017-05-11 12:04:16 -07:00
Umair Khan 3c32bc6d8a push_notification: Create get_gcm_payload. 2017-05-11 12:04:16 -07:00
Umair Khan d90774b9db push_notifications: Create get_apns_payload.
This function creates the payload that we will send to APNs.
2017-05-11 12:04:16 -07:00
Umair Khan 5ff7fdf83a push_notification: Create get_alert_from_message.
This function returns the alert shown by iOS or Android device.
2017-05-11 12:04:16 -07:00
Aditya Bansal 69cf11d786 pep8: Add compliance with rule E261 to push_notifications.py. 2017-05-07 23:21:50 -07:00
Tim Abbott c0d7e83333 logging: Change missing push notification keys from error to warning.
This is a configuration problem, but it just means a feature is not
enabled, not that attention is required, so it should be a warning,
not an error.
2017-04-25 13:52:36 -07:00
Tim Abbott cddee49e75 Add support infrastructure for push notification bouncer service.
This is an incomplete cleaned-up continuation of Lisa Neigut's push
notification bouncer work.  It supports registration and
deregistration of individual push tokens with a central push
notification bouncer server.

It still is missing a few things before we can complete this effort:
* A registration form for server admins to configure their server for
  this service, with tests.
* Code (and tests) for actually bouncing the notifications.
2017-04-18 23:03:06 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Tim Abbott 89eb7636ba remove_push_device_token: Remove unused request argument. 2017-03-05 22:01:28 -08:00
Tim Abbott 657dfcbddc push: Move remove_push_device_token to library. 2017-03-05 19:15:17 -08:00
Tim Abbott c0ad9c02fd push: Extract validate_token helper function. 2017-03-05 19:15:17 -08:00
Tim Abbott 271bd5a282 push: Move add_push_device_token to library. 2017-03-05 19:15:16 -08:00
Tim Abbott 410c0626a6 push: Pass apple alert via the zulip dict.
This completes the process of simplifying the interface of the
send_*_push_notification functions, so that they can effectively
support a push notification forwarding workflow.
2017-03-05 18:37:00 -08:00
Tim Abbott fc192d6b3b push: Don't pass a UserProfile into send_apple_push_notification.. 2017-03-05 18:37:00 -08:00
Tim Abbott 393c4d2eaa push: Don't pass a UserProfile into _do_push_to_apns_service. 2017-03-05 18:37:00 -08:00
Tim Abbott 53c9e3b4ca push: Don't pass a full UserProfile into APNsMessage. 2017-03-05 18:37:00 -08:00
Tim Abbott c8bfd568bc push: Pass devices list into send_apple_push_notification. 2017-03-05 18:36:58 -08:00
Tim Abbott 0ddaf5c610 push: Pass reg_ids list into send_android_push_notification.
This refactoring is preparation for being able to forward push
notifications to users on behalf of another Zulip server.

The goal is to remove access to the current server's database from the
send_*_push_notification code paths.
2017-03-05 18:36:07 -08:00
Tim Abbott b1b2d8d2c7 push: Move handle_push_notifications to push_notifications.py. 2017-03-05 18:32:09 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott de99f48ce7 lint: Clean up E401 PEP-8 rule. 2017-01-23 21:36:39 -08:00
Robert Hönig 0917493588 mypy: Convert zerver/lib to use typing.Text. 2016-12-25 10:33:45 -08:00
Umair Khan c24fbd28b2 Shift to python-gcm for Android push notifications.
Fixes #2650
2016-12-15 12:17:07 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
Umair Khan acd76eb604 Annotate push_notifications.py. 2016-08-08 15:17:02 -07:00
Umair Khan 0dd7d0dda1 push_notifications: Change name to 'identifier'. 2016-08-08 15:15:29 -07:00