Commit Graph

176 Commits

Author SHA1 Message Date
Tim Abbott 88fae0b6a9 remote_server: Extract remote_server.py library.
This moves the network request code for connecting to the push
notification bouncer service into its own module.
2019-01-31 15:08:46 -08:00
kunal-mohta c07f85250d messages: Extend do_delete_message to do_delete_messages.
do_delete_message has been renamed to do_delete_messages and all
occurrences of the function replaced with the appropriate new version.
2019-01-23 16:49:12 -08:00
ishanrai05 4105fb683b notifications: Optimize push notifications code path in tests.
This checks if push_notification_enabled() is set to false in
handle_push_notification and adds an early return statement.

This is a significant performance optimization for our unit tests
because the push notifications code path does a number of database
queries, and this migration means we don't end up doing those queries
the hundreds of times we send PMs or mentions in our tests where we're
not trying to test the push notifications functionality.

This should also have a small message sending scalability improvement
for any Zulip servers without push notifications enabled.

Tweaked by tabbott to fix a few small issues.

Fixes #10895.
2018-12-15 11:12:43 -08:00
Tim Abbott a63eae48cc test_push_notifications: Fix leak that can leak to test flakes.
While reviewing #11012, I discovered a nondeterministic result for
test_signup, which I tracked down to specifically this triple of tests
failing when run in this order:

test-backend GCMSuccessTest \
  zerver.tests.test_push_notifications.TestAPNs.test_get_apns_client \
  zerver.tests.test_signup.LoginTest.test_register

with a query count mismatch like this:

expected length: 73
actual length: 79

Comparing the list of queries, it's clear that test_register was
seeing `push_notifications_enabled()` returning True in this test order.

It's not clear why GCMSuccessTest was required here (it was!), but
further debugging determined the problem was that
`test_get_apns_client` left the _apns_client initialization system in
a state where get_apns_client would return a non-None value, resulting
in push_notifications_enabled() returning True for future tests.

The immediate fix is to just reset the `_apns_client` and
`_apns_client_initializedstate` state properly after the test runs;
but arguably we should do a larger refactor to make this less
fragile.
2018-12-15 11:12:43 -08:00
Tim Abbott 380231af9d push_notifications: Add tests for BrokenPipeError case.
This was missing in d723dbfef7.
2018-12-05 10:44:25 -08:00
Tim Abbott b47535d8bb push notifications: Fix exception when handling deleted messages.
If a user deletes message between when it triggered a potential push
notification for another user, and when that notification was actually
sent, we'd end up with a situation where the `Message` table didn't
have an entry for the requested message ID.

The clean fix for this is to still throw an exception in the event
that the message doesn't exist at all, but if it exists in
ArchivedMessage, don't throw a user-facing exception.
2018-12-05 10:38:37 -08:00
Tim Abbott 7b930124d9 push notifications: Add a logger (default-off in tests).
This should suppress some spammy logging output about push
notifications that we were seeing in a large number of unit tests.
2018-11-27 09:45:45 -08:00
Tim Abbott 38a6003472 push notifications: Improve logging for missing configuration.
While it could make sense to print these logging statements at WARN
level on server startup, it doesn't make sense to do so on every
message (though it perhaps did make sense to do so before more recent
changes added good ways to discover you forgot to configure push
notifications).

Instead, we now just do a WARN log on queue processor startup, and
then at DEBUG level for individual messages.

Fixes #10894.
2018-11-27 09:37:57 -08:00
Steve Howell 710095920f subject -> topic: Fix push notification tests.
We also make the topic name more clear.
2018-11-12 15:47:11 -08:00
Steve Howell 4b38260a27 tests: Fix message.subject references. 2018-11-12 15:47:11 -08:00
Jack Zhang 5902a573be push_notifications: Play 'default' sound for iOS notifications.
Fixes zulip/zulip-mobile#2651.

This was tested on an iPhone 7 running iOS 12.
2018-11-02 17:01:50 -07:00
Jack Zhang f116aba490 push notifications: Reword APNs payload alert titles.
Also, rename get_alert_from_message to get_gcm_alert.

With the implementation of the and get_apns_alert_title and
get_apns_alert_subtitle, the logic within get_alert_from_message
is only relevant to the GCM payload, so we adjust the name
accordingly.

Progresses #9949.
Resolves https://github.com/zulip/zulip-mobile/issues/1316.
2018-10-26 16:02:04 -07:00
Jack Zhang 92a100798c push notifications: Remove tests for get_alert_from_message.
The string that is returned from get_alert_from_message is
dependent upon the same message that is passed into get_apns_payload
and get_gcm_payload. The contents of those payloads that are tested via
TestGetAPNsPayload and TestGetGCMPayload, which makes the tests for
get_alert_from_message redundant.

Also, simplify the logic by removing the last elif conditional.
2018-10-26 15:55:26 -07:00
Tim Abbott 695d8d0bd1 get_apns_payload: Require a UserProfile object for the recipient.
This is preparatory work for being able to display an unread count
badge on iOS, in which case we need to know who the current user is.
2018-10-18 15:09:18 -07:00
Tim Abbott 2eebacf2dc push_notifications: Lazily import APNS libraries.
The APNS client libraries (especially the hyper.http20 one) were
determined via profiling to take significant time during the import
process, so we move them to be lazily imported in order to optimize
the overall Zulip import process.  This save up to about 100ms in
import time.

These libraries are only used in certain Django processes inside
zulipchat.com, and so are unnecessary both in development as well as
for self-hosted Zulip servers.
2018-10-17 11:59:33 -07:00
Tim Abbott 68ab71eb8b push: Fix exceptions when removing push notifications.
Now that we allow multiple users to have registered the same token, we
need to configure calls to unregister tokens to only query the
targeted user_id.

We conveniently were already passing the `user_id` into the push
notification bouncer for the remove API, so no migration for older
Zulip servers is required.
2018-10-12 11:19:23 -07:00
Tim Abbott 54e90deda8 notifications: Handle APNS "Unregistered" errors properly.
Apparently, the APNS library we're using is inconsistent about the
format of its result entries; some are strings while others are
tuples.
2018-08-28 12:12:46 -07:00
Tim Abbott da8f4bc0e9 push notifications: Add support for removing GCM push notifications.
This uses the recently introduced active_mobile_push_notification
flag; messages that have had a mobile push notification sent will have
a removal push notification sent as soon as they are marked as read.

Note that this feature is behind a setting,
SEND_REMOVE_PUSH_NOTIFICATIONS, since the notification format is not
supported by the mobile apps yet, and we want to give a grace period
before we start sending notifications that appear as (null) to
clients.  But the tracking logic to maintain the set of message IDs
with an active push notification runs unconditionally.

This is designed with at-least-once semantics; so mobile clients need
to handle the possibility that they receive duplicat requests to
remove a push notification.

We reuse the existing missedmessage_mobile_notifications queue
processor for the work, to avoid materially impacting the latency of
marking messages as read.

Fixes #7459, though we'll need to open a follow-up issue for
using these data on iOS.
2018-08-10 13:58:39 -07:00
Tim Abbott e9f4d9db2b push_notifications: Fix interface for handle_remove_push_notification.
This really should just accept a message ID.
2018-08-01 16:36:42 -07:00
Kunal Gupta bc43eefbfb notifications: Add function for cancelling GCM notifications.
This adds a new function called handle_remove_push_notification in
zerver/lib/push_notifications.py which requires user_profile id and
the message id which has to be removed in the function.

For now, the function only supports GCM (and is mostly there for
prototyping).

The payload which is being delivered needs to contain the narrow
information and the content of the message.
2018-08-01 15:59:04 -07:00
Tim Abbott 5f8d193bb7 notifications: Include realm_uri in push notifications.
This should make it much simpler for the mobile apps to line up the
data from server_settings against the data in the notifications.

Addresses part of #10094.
2018-08-01 15:46:15 -07:00
Tim Abbott cec7686f3d push notifications: Clean up unregistered/bad APNS tokens.
We've had this sort of logic for GCM for a long time; it's worth
adding for APNS as well.

Writing this is a bit of a reminder that I'm not a fan of how our unit
tests for push notifications work.
2018-05-21 11:30:56 -07:00
Aditya Bansal 5416d137d3 zerver/tests: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott 7cbff8b521 push registration: Use standard error message for auth problems.
This avoids adding an unnecessary new translated string.
2018-05-04 09:04:39 -07:00
Tim Abbott 43098a6f7c zilencer: Add automated signup system for push notifications.
Based on an initial version by Rishi Gupta.

Fixes #7325.
2018-05-03 21:27:49 -07:00
Tim Abbott 6f87091120 test_push_notifications: Cover the last lines of validate_api_key.
This push notification bouncer error case wasn't previously tested.
2018-04-25 22:36:48 -07:00
Tim Abbott 2217285ac0 test_push_notifications: Add a better test for auth code path.
This is mostly to prevent an issue similar to the one fixed in the
last commit.
2018-04-25 21:51:24 -07:00
Tim Abbott 2fa58fe9ad decorator: Fix exception format for invalid API key.
This exception class was clearly missing the part where `role` gets
stored, which was intended to be inherited from
InvalidZulipServerError.

This fixes an unnecessary 500 error in the push notifications bouncer.
2018-04-25 21:44:31 -07:00
Preston Hansen e168f9938c tests: Refactor use of test and webhook data fixtures. 2018-04-19 21:50:29 -07:00
Preston Hansen 76d6c71595 tests: Move zerver/fixtures to zerver/tests/fixtures for clarity.
Fixes #9153.
2018-04-19 21:50:17 -07:00
Aditya Bansal 9eeb1c59f6 bugdown: Remove email from rendered content of messages with mentions.
This field has been unused by clients for some time, and isn't great
for our public archive feature plans (where we'll not want to be
including email addresses in messages).
2018-03-04 20:04:27 -08:00
Greg Price 693a9a5e70 push notifs: For group PMs, identify the users in the group. 2018-02-16 16:06:03 -08:00
Greg Price ecbc72b857 push notifs: Add a diagnostic in API of whether push notifs enabled.
When the answer is False, this will allow the mobile app to show a
warning that push notifications will not work and the server admin
should set them up.

Based partly on Kunal's PR #7810.  Provides the necessary backend API
for zulip/zulip-mobile#1507.
2018-02-12 14:34:59 -08:00
Greg Price 8be2dfa81c APNs: Fix a case I broke while working out tests.
I got distracted, came back later to a successful test run in my
terminal, and thought I remembered finishing the change and just
kicking off a final test run to check.

In fact, there was an `assert False` right in the normal case for
production, and I just hadn't finished a test for that path. (m.-)
Definitely the most grateful I've been for our coverage checks,
which highlighted this for me.

Remove the `assert False`, and also finish writing the test it was
there to help me write.  Those lines are covered now.
2018-02-09 18:28:16 -08:00
Greg Price 7f6a1714aa APNs: Don't try to send anything when not configured. 2018-02-09 17:16:21 -08:00
Greg Price 014900c2e5 push notifs: Include same stream and sender info on GCM as on APNs.
This takes the information added to APNs payloads by #7080, and adds
to our GCM payloads the parts that it didn't already have.
2017-12-18 15:32:56 -08:00
Alena Volkova 39915d9835 tests: Replace all get_auth usages with auth methods and delete it. 2017-12-18 09:24:09 -05:00
Alena Volkova 26eb23afc7 tests: Replace api_auth usages with auth methods in zerver/tests. 2017-12-18 09:24:09 -05:00
kunall17 98943a8333 Include data of stream/private sender details in apns push notifications 2017-12-11 20:51:09 -08:00
Tim Abbott 6bc1a682de notifications: Separate push and email notifications checks.
This is an early step in a larger refactor to properly decouple the
email and push notification code paths.
2017-11-28 17:51:18 -08:00
Vishnu Ks b9bc1c2b33 Eliminate get_user_profile_by_email from test_classes. 2017-11-26 15:47:56 -08:00
rht 3ec90f8b33 zerver/tests: Use python 3 syntax for typing (final). 2017-11-21 22:01:19 -08:00
rht e3daa09b05 zerver/tests: Use python 3 syntax for typing. 2017-11-21 22:01:19 -08:00
Tim Abbott 982bd9c38d Fix push notifications for soft-deactivated users.
Previously, these push notification events were being generated, but
then ignored in handle_push_notification because there was no
user_message object.
2017-11-09 15:52:29 -08:00
Tim Abbott 64b4d83038 push_notifications: Improve debugging of lookup problems.
We're having this error being thrown often, and don't know why.
2017-11-09 15:20:24 -08:00
rht c7fa1d4146 zerver/tests: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Tim Abbott 9b791acf50 test_push_notifications: Fix tests failing due to rebase.
message.triggers got recently renamed.
2017-10-19 18:34:28 -07:00
Maarten Rijke 841f02934d push_notifications: Add setting to redact content.
This commit adds a setting that toggles redacting content of push
notifications and replacing it with "***REDACTED***".
2017-10-19 18:16:06 -07:00
Tim Abbott e98ca0714b notifications: Simplify how triggers are passed around.
This removes the utterly unnecessary `triggers` dict (which always was
a dict with exactly one value True) in favor of a single field,
'trigger'.

Inspired by Kunal Gupta's work in #6659.
2017-10-18 21:42:05 -07:00
Tim Abbott b5c107ed27 push_notifications: Remove unnecessary check for no devices.
This should have been checked by the caller anyway.
2017-10-13 17:30:20 -07:00
Tim Abbott 27a450b58d push_notifications: Improve error message for GCM sending issues.
This addresses one of the sources of confusion in #6993.
2017-10-13 17:30:11 -07:00
Tim Abbott 676a6af8cb push_notifications: Get back to 100% test coverage.
We didn't have tests for a few new code paths.
2017-10-11 20:17:27 -07:00
Tim Abbott 298c59f7fd push_notifications: Fix error message for unregistered bouncer.
Previously, we were just returning a JSON error to the client, when it
was a server problem.

Fixes #6639.
2017-10-11 19:09:24 -07:00
Harshit Bansal 7b88e3eea7 tests: Add a test suite for verifying the mobile push notifs content.
This test suite works by using the expected_output and new text_output
fields in the bugdown test cases to verify that each syntax is
correctly translated by this new function.

Some of these translations, like strikethrough, are kinda poor; but
this framework should make it easy to iterate on the formatting.

Fixes: #6720.
2017-10-06 16:47:27 -07:00
Harshit Bansal 5a6584890d push_notifications: Start using `get_mobile_push_content()` function. 2017-10-06 16:47:25 -07:00
kunall17 f091823fc8 push_notifications: Set APNS badge as 0 for now.
We don't yet have support for tracking the number of open APNS
notifications server-side, so setting this to 0 will at least avoid
confusing users.
2017-10-03 17:07:49 -07:00
Tim Abbott 7581e0e1c5 settings: Remove remaining references to REALMS_HAVE_SUBDOMAINS. 2017-10-02 16:43:54 -07:00
derAnfaenger d1afab7199 Replace deprecated Logging.warn calls with Logging.warning. 2017-10-02 11:11:42 +02:00
Greg Price 7b8f725707 APNs: Accept 1.6-format payloads in bouncer.
This is just enough of a quick fix to work with a stock Zulip 1.6
server.  We should really also make this robust to arbitrary input
from the remote Zulip server, even though it'll be a little tedious.
2017-09-28 10:01:16 -07:00
rht 26f5d9a32c zerver/tests: Remove print_function. 2017-09-27 18:05:45 -07:00
rht 1e87a4b68c zerver/tests: Remove absolute_import. 2017-09-27 10:00:39 -07:00
kunall17 1436d558a3 APNs: Add message body to notification payload.
(Edited by greg to leave the badge logic as is for now.)
2017-09-18 16:37:02 -07:00
Tim Abbott 5722237f59 push: Rename received_pm to private_message.
This is a clearer name for this now more broadly used interface.
2017-09-14 05:41:37 -07:00
Sarah 97571a203d push: Add new formatting for stream message push and add tests.
This should make the push notifications for messages to streams with
the new stream push notifications setting enabled make sense.
2017-09-14 05:41:37 -07:00
Sarah c3a8138f74 user_settings: Add push notifications for all stream messages.
Add setting to enable push notifications for all stream messages.
2017-09-14 05:41:37 -07:00
Greg Price a4bcf1a64b APNs: Handle HTTP connection errors, and retry.
Should help with #6321 as at least a band-aid.
2017-08-29 15:27:41 -07:00
Greg Price 780e1ac5b2 push notifs: Add a simple test for the new APNs provider. 2017-08-29 15:27:41 -07:00
Steve Howell 0959c978c3 Fix lint error from recent subdomains commit.
We did a code sweep recently for subdomains (see
60be89d0).
2017-08-29 08:35:37 -07:00
Tim Abbott 60be89d00e test_push_notifications: Declare subdomains explicitly. 2017-08-28 23:19:07 -07:00
Tim Abbott 92efe94a27 tests: Remove unnecessary apns mock. 2017-08-26 15:00:08 -07:00
Tim Abbott f0637cb01a push_notifications: Fix one last lint error. 2017-08-26 14:34:17 -07:00
Greg Price 613d093d7d push notifs: Implement APNs with new API.
And it works!

A couple of things still to do:

 * When a device token is no longer active, we'll get HTTP status 410.
   We should then remove the token from the database so we don't keep
   trying to push to it.  This is fairly urgent.

 * The library we're using has a nice asynchronous API, but this
   version doesn't use it.  This is OK now, but async will be
   essential at scale.
2017-08-26 14:16:05 -07:00
Greg Price d02101a401 APNs: Rip out the existing, broken implementation.
This code empirically doesn't work.  It's not entirely clear why, even
having done quite a bit of debugging; partly because the code is quite
convoluted, and because it shows the symptoms of people making changes
over time without really understanding how it was supposed to work.

Moreover, this code targets an old version of the APNs provider API.
Apple deprecated that in 2015, in favor of a shiny new one which uses
HTTP/2 to meet the same needs for concurrency and scale that the old
one had to do a bunch of ad-hoc protocol design for.

So, rip this code out.  We'll build a pathway to the new API from
scratch; it's not that complicated.
2017-08-26 14:16:05 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Umair Khan a5c05f9812 push_notification: Retry event in case of error.
Fixes #5301
2017-08-22 11:16:48 -07:00
Tim Abbott 47eb7ee003 push_notifications: Fix incorrect URL being used in bouncer test.
This was actually testing the validation in the direct push
notifications code path, not the push notification boucner code path.
2017-07-31 13:08:06 -07:00
Greg Price e18baff32c JsonableError: Rename message from `error` to `msg`.
The whole thing is an error, so "message" is a more apt word for the
error message specifically.  We abbreviate that as `msg` in the actual
HTTP responses and in the signatures of `json_error` and friends, so
do the same here.
2017-07-24 16:41:22 -07:00
Tim Abbott 0606ba88df push_notifications: Verify that token types are valid.
We only have two types of push notification tokens, so we should
validate that in the bouncer code path.
2017-07-07 10:28:36 -07:00
Tim Abbott eaa33cce73 push_notifications: Validate format of APNS tokens.
This fixes a bug where we would previously not validate the format of
APNS tokens before writing them to the database, which could lead to
exceptions in the push notifications system if a buggy mobile app
submitted invalid format tokens.
2017-07-07 10:28:36 -07:00
kunall17 d8310724b6 Use absolute URLs in GCM push notifications.
This will eventually allow us to simplify the logic required for the
mobile apps processing GCM notifications.
2017-06-15 03:56:59 -07:00
Vishnu Ks 5230eaef1c Replace hamlet@zulip.com with example_email('hamlet'). 2017-05-24 19:37:36 -07:00
Rick Chern e53d1c3885 tests: Remove get_user_profile_by_email from most tests. 2017-05-24 13:05:19 -07:00
Umair Khan 38ecc35cd9 push_notifications: Catch IOError while pushing to GCM. 2017-05-17 12:14:32 -07:00
Umair Khan 7e8f4ca4e8 push_notifications: Include GCM in end-to-end test. 2017-05-17 08:09:19 -07:00
Tim Abbott 9d63a5ab3a push_notification: Delete obsolete DBX_IOS_APP_ID code.
I'm not sure that this was ever actually used, but it's definitely
just clutter for Zulip today.
2017-05-16 12:26:55 -07:00
Umair Khan fa5c66c439 push_notifications: Add num_push_devices_for_user tests. 2017-05-16 12:26:55 -07:00
Umair Khan 88f5d29e19 push_notifications: Add send_to_push_bouncer tests. 2017-05-16 12:26:55 -07:00
Umair Khan af27ad607a push_notifications: Add handle_push_notification tests. 2017-05-16 12:26:55 -07:00
Umair Khan 8a6498f55d push_notifications: Create BouncerTestCase.
Adds bounce_request method to simulate a bounce.
2017-05-16 12:26:55 -07:00
Umair Khan 5907877038 push_notifications: Add test for send_notifications_to_bouncer. 2017-05-16 12:26:55 -07:00
Umair Khan fab4249893 push_notifications: Add get_gcm_payload tests. 2017-05-16 12:26:55 -07:00
Umair Khan 136a950041 push_notifications: Add test for APNs payload. 2017-05-16 12:26:55 -07:00
Umair Khan 47d0f7d996 push_notifications: Add tests for get_alert_from_message. 2017-05-16 12:26:55 -07:00
Umair Khan df0d29aaff push_notifications: Add response_listener tests. 2017-05-16 12:26:55 -07:00
Steve Howell b609911656 Remove unused get_user_profile_by_email() imports. 2017-05-08 11:57:38 -07:00
Steve Howell 7f9057ba99 tests: Use example_user() in more places. 2017-05-08 11:57:38 -07:00
Steve Howell 6bc8424c71 Use self.example_user() in more places.
This fixes most cases where we were assigning a user to
the var email and then calling get_user_profile_by_email with
that var.

(This was fixed mostly with a script.)
2017-05-08 11:57:38 -07:00
Aditya Bansal fd4ac33073 pep8: Add compliance with rule E261 to test_push_notifications.py. 2017-05-07 23:21:50 -07:00
Tim Abbott c0d7e83333 logging: Change missing push notification keys from error to warning.
This is a configuration problem, but it just means a feature is not
enabled, not that attention is required, so it should be a warning,
not an error.
2017-04-25 13:52:36 -07:00